Brianna, thanks for the shoutout and it's good to see you here I have pondered this quite a bit so I can make my best recommendation to you.
The truth is that what you said about AA is true about all mainstream apparel companies. We live in a capitalist, corporate-based society and unfortunately that leads to a lot of exploitation--of people, animals, and the environment alike. If you wanted to avoid all of that, you would have to avoid everything aside from some obscure indie designers, used clothing, and Etsy sellers, and even that might not completely avoid these issues. Somewhere along the production line, it is highly likely that there were questionable practices. An Etsy seller could be using a fabric created by children in India, for example. Vegan clothing companies use a lot of PVC, which is not environmentally friendly (however, I feel obligated to point out that leather tanning is no better).
So that said, I think one must choose his or her battles. My ethical issue is mainly with animals. Recently, however, I have started considering wearing USED leather because it is kinder to the environment by repurposing something rather than buying something new, and doing so doesn't fund any exploitative practices. But I do feel like a bit of a hypocrite because I know I can't even touch real fur, let alone wear it, regardless of whether or not it is used...hence "considering." I also wear leather shoes, and that is a battle I have just had to force myself to ignore. I continue to research vegan brands and I'm sure one day the technology an engineering will exist to create a manmade material that breathes and stretches, but meanwhile despite my efforts, I haven't found anything that truly goes the distance. Hopefully one day my fussy feet can be pacified without compromising my ideals. I guess what I am saying is that I do my very best, but I still try to be reasonable and rational. I'm not a bleeding heart and never will be because even if I do everything I can, it still won't be enough. It must be a collective effort. I think if everyone in the world chose *a* battle--any battle--we would see a vast improvement in everything from animal welfare to quality of life to the environment. If everyone made some small effort, even if they aren't perfect, it would truly add up.
Sooo...now that I have preached all that...about shopping: As I said, all major retailers will engage in some seedy practices. Sweatshop labor is almost guaranteed these days, unfortunately, because most of these garments are cheaply produced in countries without proper labor laws. Rather than worrying about where to shop, or boycotting certain stores or brands, I would be more inclined to support organizations that work to create better conditions for these workers. With the environment, there are plenty of brands out there that claim to be sustainable or eco-friendly, or that have a sustainable/eco-friendly range. This is not my area of expertise as I usually find these brands and ranges too expensive for me, but I'm sure others will chime in.