Context: The numbers of vegans and vegetarians have increased in the last decades. However, the impact of these diets on bone health is still under debate. Objective: This systematic review and meta-analysis sought to study the impact of vegetarian and vegan diets on bone mineral density (BMD) and fracture risk. Data Sources: A systematic search was conducted of PubMed, Scopus, and Science Direct, covering the period from the respective start date of each database to November 2017. Data Extraction: Two investigators evaluated 275 studies against the inclusion criteria (original studies in humans, written in English or Spanish and including vegetarian or vegan diets and omnivorous diets as factors with BMD values for the whole body, lumbar spine, or femoral neck and/or the number of fractures as the outcome) and exclusion criteria (articles that did not include imaging or studies that included participants who had suffered a fracture before starting the vegetarian or vegan diet). The quality assessment tool for observational cohort and cross-sectional studies was used to assess the quality of the studies. Results: Twenty studies including 37 134 participants met the inclusion criteria. Compared with omnivores, vegetarians and vegans had lower BMD at the femoral neck and lumbar spine and vegans also had higher fracture rates. Conclusions: Vegetarian and vegan diets should be planned to avoid negative consequences on bone health.