This, yes. I was raised in a religious household, and it took me many years to admit to my family that I was an atheist. I even went through a full agnostic phase while dealing with the thoughts of their reactions and the fear of being ostracized. Fortunately, my family is no longer affiliated with a church and practices their faith privately, so there isn't that pollution of peer pressure to distance themselves from me.