Ancient DNA has provided new insights into many aspects of human history. However, we lack comprehensive studies of the Y chromosomes of Denisovans and Neanderthals because the majority of specimens that have been sequenced to sufficient coverage are female. Sequencing Y chromosomes from two Denisovans and three Neanderthals shows that the Y chromosomes of Denisovans split around 700 thousand years ago from a lineage shared by Neanderthals and modern human Y chromosomes, which diverged from each other around 370 thousand years ago. The phylogenetic relationships of archaic and modern human Y chromosomes differ from the population relationships inferred from the autosomal genomes and mirror mitochondrial DNA phylogenies, indicating replacement of both the mitochondrial and Y chromosomal gene pools in late Neanderthals. This replacement is plausible if the low effective population size of Neanderthals resulted in an increased genetic load in Neanderthals relative to modern humans.