Are you saying that your self-critical attitude is such that you don't feel good about most of your clothes, but that you think some of those clothes you don't feel good about might in fact be worth keeping?
When I did my drastic wardrobe edit I had the opposite problem -- of feeling good about most of my clothes and thus being at a loss to know how to start paring them down. (I had already got rid of everything polyester and/or that contains elastane, that feel icky to me.)
When I really thought about it, there were some clothes and pairs of shoes that when I held them in my hands and asked myself how I felt about them, they did not give me joy. Some, I associated with bad memories; some made me feel guilty for having wasted so much money on them, some, I kept not feeling like wearing, some didn't feel good when I actually wore them, and so on. If I put something on and don't feel good in it, it goes straight into the donation box. If I wear something an find it fiddly/uncomfortable/frumpy/silly/whatever, then again, I donate it. So for me, noticing negative feelings in association with particular items really helped me get rid of a lot of stuff.
Another thing that helped concentrate my mind on paring down was my experience of having to go through a dead relative's belongings. Even though the person was by no means a hoarder, the whole thing was so psychologically difficult that it gave me an aversion to acquiring more stuff, and made me want to get rid of just about everything I own. Stopping buying more stuff is a good start. Having a pause and shopping your existing wardrobe can help bring perspective and calm, which makes it easier to make good decisions about your stuff.
Another thing that helped me was to identify the different spheres of my life/activities and what I prefer to wear in each sphere, how often each thing happens, how many times per season I want to wear each outfit, and thus I calculated what I actually need for each different activity/sphere per season. So, for example, what do I prefer to wear when I go shopping? What do I prefer to wear to conferences? What do I prefer to wear when at home? What do I prefer to wear for girls' nights out? What do I wear for dates with my husband? Work events? Travelling? And so on. When I thought about it like that, I discovered that my actual preferences for what to wear could be narrowed down to a few distinct looks/styles (I gave each style a name/descriptor), some of which apply to more than one sphere/activity of life.
To come up with these named 'looks'/styles wasn't totally straightforward. It did take time, and as part of that process I initially came up with many different 'looks' I enjoy and thought about each one and made a point of wearing each one and seeing how I felt about it. I discovered, for example, that although I have collected silk dresses and silk tops for years and love the feel of silk, when it comes to actually wearing silk dresses, I have an aversion to wearing floaty silk dresses in most spheres of my life. I feel silly/frumpy in them. This was a great surprise to me, because I have always liked the idea of silk dresses! I have thus donated a whole lot of silk dresses.
By going through this process of identifying spheres/activities of my life and what kind of 'looks' I actually prefer to wear in each, etc., one thing I discovered immediately was that I had vastly too many super-fancy red carpet evening dresses that I only wear perhaps once a year -- as in, I only have ONE red carpet event per year (if that!). This made it easy to get rid of almost all my red carpet dresses.
It also showed me that I had too many pairs of high heel shoes, and it made it easy to decide which to keep and which to donate, because, for example, when I am in London, I walk for miles and miles and miles (like 12 miles, not just 1 or 2 -- I LOVE walking in big cities, especially London and New York), so that means I want to be wearing playful Dr Martens, not either high heel or flat pointy toed shoes that will squish my toes and hurt both in that respect and by virtue of their lack of cushioning on the soles. When I go shopping I also prefer to wear very comfortable, playful shoes. When I go on a date with my husband, I often wear high heels, but given his preferences the range I wear is narrower than I had. I worked out that I need MORE playful super-comfortable walking shoes, and many fewer high heels, and which kinds of high heels I need and which I am really unlikely to wear much.
Another thought that helped me pare down my wardrobe was imagining how I would feel were I to meet particular individuals (significant to me) while wearing a given outfit or item. This considerably narrowed the number of different 'looks' I feel like wearing. It made me less high fashiony except when I'm in London, where I feel completely free in a fashion sense. I found that I simply didn't feel like wearing my most high fashion 'look' at all except in London. It also made me completely go off the idea of a hard/edgy/masculine 'look'. Perhaps a decade ago I was enjoying that, but in recent years it has been leaving me cold. Going through this process helped me notice that that was the case.
That is another thing that is worth considering: does a given item fit your CURRENT style preferences, or is it now out of date in terms of your own individual specific preferences? This is not necessarily about what is out of date or in fashion (though that can be a further consideration); it is about how it feels to YOU. Some of the footwear I donated felt too fashiony; some felt no longer current for my specific individual preferences about what to wear; and some simply felt out of date. (I tend to be a fashion early adopter when it comes to footwear styles, and tend to find things 'out' when they are becoming popular!)
Then, when I considered climate/temperature issues, that enabled me to get rid of vast amounts of stuff that realistically, given where I have been spending my time in recent years, is simply not right for the temperatures I will be in.
Another exercise that you might find helpful is to imagine that you have to pick only the n items that you love the most, where n = the number you choose, and that you have to do this RIGHT NOW, without hesitation. Set a timer and start writing a list of (or bodily gathering) all the items you most love -- perhaps imagine you can only pack one suitcase, or that you have very limited time to save most loved things from rising flood water or something. The point is to QUICKLY establish what you would most hate to lose. Then ask yourself why you are hanging on to the other stuff.
Do keep an outfit diary, listing what you wear (or taking photo), and HOW YOU FELT IN IT. This is a huge help for anyone finding it difficult to pare down.
But if most of your clothes make you feel bad, and thus, it is going to be difficult to decide which to keep, then unless you can afford to make an expensive mistake of donating too high a proportion of your wardrobe, you might want to hold off on the paring down until you have come up with some preferred styles/looks/uniforms that you can then start trying out in real life, and thereby perhaps narrow down to one or more 'looks' that you actually enjoy wearing -- that actually make you feel GOOD -- and THEN do the paring down. Of course if you feel overwhelmed by too much stuff, you won't want to wait for this style clarity. But in that case, separate all the items that don't make you feel bad, then, when considering the remainder, identify the items that make you feel PARTICULARLY bad and donate them immediately, and put the items that merely don't feel good into a holding zone -- potential donations that you are not going to actually donate just yet.
Having massively over-bought for a while I now buy about 10 new items per year (and last year only 7!) apart from gear for exercise/skiing etc.
I hope this helps! Best of luck!
Sarah