Traci and Em speak well to this issue.
I would add that there will always be someone who has a comment about our appearance, no matter what we look like or how we dress. At some point, it may come after years of experience, you have that inner "Ef it" moment and walk freely.
If you are ALWAYS (by which I could define everyone you meet every day) getting crap, then another alternative is to change your environment - easier said than done of course.
Many of us also have several looks, modulating our style for work, weekend, family. It does not mean giving up one's identity, though there are some so steadfast to their vision and self-knowledge that they really "own it" no matter what and no matter what is said. I admire them!
I like to look at Leandra Medine (Man Repeller) and Sophistique Noir for inspiration. Leandra is a crazy high fashion magpie, Sophistique is goth/steampunk - and she modulates her look subtly for office to club.
You look happy in your pictures. Sometimes I think the best advice is "What would I do if no one was looking" and do that. Maybe I have reached the "Ef 'em" point?