Global evidence indicates minimal connection between immigration and crime. Nordic research, however, has been generally carried out on individuals and shows that immigrants are over-represented in crime. This has led to claims that high crime rates are due to immigration. We directed our study towards these claims by analyzing immigrant population prevalence, defined as the percent of foreign-born individuals, and violent crime in Swedish municipalities between 2000 and 2020. Nearly all municipalities had higher violent crime rates in 2020 relative to 2000. To discern whether drastic increases in municipality-level crime rates could be connected to municipality-level immigrant population prevalence, a retrospective case–control design was used to select 20 municipalities with the highest increase in reported violent crime rates and 20 municipalities with the lowest increase in reported violent crime rates. Immigrant population prevalence had little association with high rates of reported violent crime. The average association between immigrant population prevalence and violent crime rates calculated from all municipalities was also weak and non-significant (p > 0.05). Municipalities with a high increase in crime tended to have more crime correlates than municipalities with a low increase in crime. However, more research is needed on the impact of migration in small towns, especially those that have experienced economic and social stagnation.