We bought a place in a tiny town of 400, about half an hour from a couple low-mid 5 digit cities. Instead of a terrible rumor mill, we found the most welcoming and helpful neighbors we'd ever imagined. Anytime something went wrong, they were there to help out without being asked. We ended up not staying because of the distance from our families, but I really regret not making that move permanent.