Hmm have you considered a holding zone? So if all these clothing items are scattered throughout the house, instead group them so that the items which *do* currently fit are in the most easily accessible & visible spot. Anything which doesn't fit, keep it in more hidden or inconvenient storage spaces.
Try getting dressed from the 'easy' spot & make a note of what you do & don't wear - how often do you reach for something that's in the 'hidden' zone, only to find it's not there? Similarly, I also like to track wears by hanging all the items in one direction then switching the position of an item once it goes back on its hanger (after being worn).
It's up to you how long you use the hidden 'holding zone' - some say 1 month, others 6. Either way, after you decide it's done, you can always 'rescue' any MVPs that you thought about (even whilst you couldn't see them in your primary closet). Hopefully that will help you to decide how much clothing you truly need
If you need help beyond that then, as the others have mentioned, Angie has many useful articles on style. I found this one in particular helpful for starting from scratch:
https://youlookfab.com/2015/11.....e-renewal/