Very interesting! I've been thinking about this, and for the most part I think my closet falls into either aspirational or resentful. Since I finished my PhD but have not found a job, or really even any idea of what I'd actually like to do, I have been flailing around in a sort of ongoing personality crisis. This is coupled with long-standing and intense issues with body image, failure to come to terms with my size, and self-esteem. So I have a relatively large section of items that are business-y, things that are cute but uncomfortable or fussy in some way, stuff that I can fit into but is just a bit too small to be comfortable, and the stuff that I am actually wearing right now.
The latter group is also transitional, but since it is a transition that I don't want to make (to a bigger size) I think I am resisting making it nice. There is also a lot of guilt and shame bound up with my clothes -- guilt at letting go of things that actually fit me, since they are rather difficult to come by, guilt at having spent money that I don't really have, and shame that these are the only clothes I can get into because of my size. Small wonder I have trouble purging my closet -- it's full of negativity. I can't even think of any garment in there that actually makes me feel fab at all, so there is very little reason to choose one item over another. The main criteria for keeping something is "does it fit?". If it fits, it stays.