Oh, I agree with Bijou as well, and have proven it as that's how we became friends! It's up to you, but it sounds like you need to think about it a bit more before you decide to change anything.
Sometimes I feel tired of sticking out. I want to wear what I like but then when I wear it out I feel stared at or like a misfit and that's tiresome. But does that mean I should dress differently? I have no answers....I'm grappling with this same quesiton.
I will say, in Brene Brown's talk on Netflix she emphasizes the difference between belonging and fitting in. Something to consider. Do you want to fit in, or do you want to belong? If you want to belong you need to bring your true self so others can see and know you.