While internal migration is widely occurring in countries across the world and older people are more likely to be left behind by family members who out-migrated to other locations, little attention has been paid to the cognitive health of those people who have been left behind (PLB). Understanding how these demographic patterns relate to older persons’ cognitive health may inform efforts to reduce the disease burden due to cognitive decline. Data came from the China Health and Nutrition Survey in 1997, 2000 and 2004. Participants aged 55 to 93 who participated in a cognitive function screening test (score range: 0–31) in two or more waves and provided information on family members’ migration (n = 1,267) were included in the analysis. A mixed linear model was used to investigate the association between being left behind by any members who had not resided in the household for at least 6 months at baseline and cognitive function. Approximately 10% of the participants had been left behind by family members who migrated out of their communities. A significant interaction was observed in relation to cognitive function between being left behind and the number of years from the first test. Specifically, there was a less steep decline in cognitive function of PLB compared to people not left behind. This longitudinal study showed that PLB tended to have a higher cognitive function compared to those not left behind due to their relatively stable transition in cognitive function during the study period.