Humans settled the Caribbean about 6,000 years ago, and ceramic use and intensified agriculture mark a shift from the Archaic to the Ceramic Age at around 2,500 years ago(1-3). Here we report genome-wide data from 174 ancient individuals from The Bahamas, Haiti and the Dominican Republic (collectively, Hispaniola), Puerto Rico, Curacao and Venezuela, which we co-analysed with 89 previously published ancient individuals. Stone-tool-using Caribbean people, who first entered the Caribbean during the Archaic Age, derive from a deeply divergent population that is closest to Central and northern South American individuals; contrary to previous work(4), we find no support for ancestry contributed by a population related to North American individuals. Archaic-related lineages were >98% replaced by a genetically homogeneous ceramic-using population related to speakers of languages in the Arawak family from northeast South America; these people moved through the Lesser Antilles and into the Greater Antilles at least 1,700 years ago, introducing ancestry that is still present. Ancient Caribbean people avoided close kin unions despite limited mate pools that reflect small effective population sizes, which we estimate to be a minimum of 500-1,500 and a maximum of 1,530-8,150 individuals on the combined islands of Puerto Rico and Hispaniola in the dozens of generations before the individuals who we analysed lived. Census sizes are unlikely to be more than tenfold larger than effective population sizes, so previous pan-Caribbean estimates of hundreds of thousands of people are too large(5,6). Confirming a small and interconnected Ceramic Age population(7), we detect 19 pairs of cross-island cousins, close relatives buried around 75 km apart in Hispaniola and low genetic differentiation across islands. Genetic continuity across transitions in pottery styles reveals that cultural changes during the Ceramic Age were not driven by migration of genetically differentiated groups from the mainland, but instead reflected interactions within an interconnected Caribbean world(1,8). Ancient DNA reveals genetic differences between stone-tool users and people associated with ceramic technology in the Caribbean and provides substantially lower estimates of population sizes in the region before European contact.