Judging someone based on their fashion choices is what's shallow and vain, imho, rather than having a love and appreciation for fashion making someone shallow and vain. Who are they to judge and why should I (or you) care?
I've always been a "live out loud and appreciate every day" kinda gal, and this is reflected in my style choices. If someone has an issue with it, it is their problem, not mine. I'm not in this world to please everyone (most especially not strangers who may want to criticize my style or attribute a generic personality trait to someone based on their clothing ).
Never judge a book by its cover, I always say.
If your friends are your friends (or even associates) they will appreciate you for who you are...lover of fashion and all! As to every one else, who cares really?
As a side note, I have also been known to purposely dress funky at the corporate events I need to attend. Where everyone is in their "uniform" of basic corporate attire. I find it telling who actually comes up and wants to engage in conversation with me versus those who actually seem surprised when I can carry on a stimulating and thought provoking conversation about something as deep as global issues or data analysis as applied to human behavior or as "shallow" as how much I appreciate a really cool hat!
My style is very much an expression of creativity. No different than when I used to paint back when I was in art school. Now my canvas is me and my style choices.
Just like not every masterwork painting appeals to everyone neither does everyone's wardrobe choices. That's what makes us such unique and interesting creatures. And that is a good thing, imho.