Just a quick question, but is this guy a victim of a stroke (or similar?) That's a brain injury and people who have had one often lose their "filters". They don't mean to hurt anyone, they simply don't understand. My dad was that way during the last few years of his life, after strokes and brain surgeries. It WAS embarrassing sometimes, and we'd apologize if we were there, but in reality I'm sure he hurt peoples feelings without ever meaning to.
Although my Dad as a tough guy I had never in all my years seen him act that way until he had a stroke.
If that's the case please move on. He may think it makes you look "huge", but does his opinion really need to be what you think about yourself? You can't live your life (or buy your clothes) to suit anyone but yourself. If YOU love then we love it with you! Wear with pride and enjoy the color in the dull winter!
Besides, when did you start taking fashion advice from a man (that's not Tim Gunn or similar! LOL)