I see what you mean, Christy. People might notice, and fleetingly wonder why a person doesn't want to join in a group activity (not just hot tub soaking, but anything, really -- softball game, board game, whatever), but nobody's going to stay awake at night thinking about it. It is true that we tend to focus only on what pertains to ourselves.
*I* would notice if only one person -- particularly a woman -- did not get into a bathing suit. But then, I have special radar for anything that hints at an eating disorder. Once you've been there, no matter how long ago, you never forget, and you can never not notice.