Likely most Catholics who grew up pre-Vatican II were taught that St. Peter was crucified upside down and hence the symbol. I know my parents and their peers know way more about such things.

These days the state of catechesis is awful in most places so you have a large percentage of Catholics who are ignorant of large amounts of their faith even if they go through Catholic School.