A top made of technical fabric- instead of a t-shirt - is worth every penny. It's not only that the top will wick away moisture and be more comfortable, it will be healthier too. Many of these technical fabrics impede the growth of bacteria.
Also, when you wear something more form-fitting it helps you see your posture in the mirror if you are weight training, which can help prevent you from injuring yourself by lifting wrong. AND, interestingly!!! ... there is research showing that if you watch yourself in the mirror while you lift, you will see faster improvements than if you don't. There is something about the mind imagining growing muscles, which actually MAKES THEM GROW. I'm not making this up!!! If you are lifting weights, do throw out the baggy tee shirt.
So it's not really about looking good and dressing to impress. You can find good workout gear at every price point. I have had stuff from Target and K-mart ... but I do agree that the cheaper stuff doesn't last as long. If you don't mind replacing often, it's a good place to start.
I have two pieces from LuLulemon, both given to me as gifts, and they really are well made and cut to flatter, and come in pretty colors too. Plus, they will hem your pants, for free, which is a plus for those of us who are short. However, they do not recommend their yoga gear be worn for running (unless you wear a running bra underneath). Just FYI! If you are into yoga or lifting weights or some other low-sweat, low-impact activity, and you want to look uber-stylish, and don't mind spending top $, this is your brand.
But most of my favorite and longest-lasting work-out gear is from New Balance. That's because I live near an outlet, and we frequently get 50% off coupons!!! My family is always buying me gear from that outlet so I have quite a variety now. If you are petite, I can vouch for New Balance: I find their tops and pants fit me fine, without needing to be hemmed. I can't say that for other brands like Nike and Champion. Those are more hit-or-miss - I have to try on first.