These issues are COMPLICATED. As I've mentioned before, my brother works in international trade policy for a large retailer and talking to him has really made me see nuances that I was completely blind to before (see Christina's comment above)...and I'm someone who regularly seeks out reading material about supply chains, labor practices and environmental challenges in the clothing industry.
Just thinking about these things at all is a big step, though, and that's the point of, if not ethical, mindful consumerism. Every choice that we make differently because we're thinking through some of these issues is a choice that will have an effect on the big picture.