You are correct. They turn down people for all sorts of reasons, including foot conditions, shaving bumps, allergies, partial blindness, etc. Why? Because it impacts performance and ability, which impact combat readiness. Too many special accommodations would have to be made, etc.

Ultimately though it doesn't matter. There is no right to serve in the military constitutionally or by statute, generally, and people are precluded regularly from serving in the U.S. military for all sorts of reasons that simply don't apply in the civilian world.