Makrame, you already sound more disciplined than I was when I gave myself the challenge. And Angie hit the nail squarely on the head with her advice to WEAR what is in your closet.
My problem was a wardrobe surplus. For example, if T's were on sale 3/$$, I'd pick up three, but then I'd tend to wear my favorite colour over and over, leaving the other 2 languishing in the closet. And the next time I was out shopping, I'd find another style I'd like... So the issue was not about tossing the worn-out, faded T at the end of the summer, but dealing with a stack of almost unworn garments that kept growing in my closet. Tossing almost new items into the recycle box made me feel like an idiot.
QM also made an excellent point when she mentions that it is often possible to "get by" with what you currently have instead of always thinking of wardrobe "holes". Since I take good care of my clothes, I found restricting my buying made me rethink self-imposed categories, as in why couldn't I wear that six-year-old, slightly dated silk blouse with my jeans instead of buying a new casual top. And, because I was now wearing it casually, it seemed to look fresh instead of dated.
I think that the "rules" you'd set for yourself ought to reflect your needs and goals. If cotton cardigans fit your lifestyle, then it makes sense to continue wearing them. If stuff gets faded because you've washed and worn it several time, then it would make total sense to my mind if YOUR "rules" have an exemption for replacing worn-out workhorses. Challenges are OK, but, to quote Angie, horses for courses (I get a kick out of that phrase!).