For my part, it has more to do with not wanting to be a vehicle for advertising (thus picking the dealership label off my car...). It feels like a very different thing to remove tags/logos that (in my eyes) aren't an integral part of the item's design or don't add to the design (again, in my eyes). I'm a minimal dresser, and honestly, I prefer minimal design, down to preferring clothes without embroidered logos regardless of maker (I've been looking for a cream cable sweater with no embroidery for while and good luck with that). I think there's room for simply not caring for the aesthetic.
That said, yes, part of it is definitely a reaction to the overly logo-ed place that fashion seems to be, although it feels less so than a few years back. I'd rather admire the quality of an item than an announcement of its maker. Adding to Mo's thoughts...what are the labels standing in for, if anything? If that's reverse snobbery, then I guess you can count me in.
I do respect that others feel differently, and that's what makes the world go 'round.