DH's "loungewear" consists entirely of a pair of grey flannel pants I gave him a couple years ago, plus graphic tees that he used to wear to work but which are now to casual for his current office. Sometimes he wears his robe, or puts an Icebreaker merino hoodie on top (he seems to have a sartorial allergy to plain old cotton hoodies, and will only wear technical ones...). He actually has been on my case to get him the grey flannel shirt to match his pants, which I think is very cute of him to want, so maybe for Fathers Day.
My loungewear selection has gotten more extensive, thanks to being home with a young child a lot of the time. My core is a set of Patagonia sweatpants and hoodies from a few years back, and last year I added a sweatpant/hoodie set from Everlane. The Patagonia stuff is dark grey and olive, the Everlane stuff is blush and black, so it all mixes and matches nicely. I wear my band t-shirts and old graphic t-shirts underneath, since they've stopped feeling relevant to my style otherwise. I'm thinking of trying out a pair of American Giant sweatpants this year. My family teases me about my "designer" sweats but I've always been very picky about sweatpants and find poorly made ones to be so uncomfortable I won't wear them. If the waist, leg bindings, etc are wrong, you won't want to wear them. I spend so much time in my "house clothes" (a couple hours each morning, an hour at night, and more on the weekend) that they earn their keep quite quickly. All of the pieces I mention above are close to 30 wears this year alone.
I also just picked up two pairs of these Uniqlo boyfriend jeans. They are a unicorn for me, because they are jeans that are loose all over without falling down, and I did not need to alter them. Basically, they are as comfortable as sweatpants, but (for me, where I live) look a little more presentable. I'll go out and walk the dog do groceries, etc in these, which I don't like to do in sweatpants. I know you complained about lyocell in denim in an earlier post; I'd always been a skeptic too, but in this case it makes the jeans very soft and allows them to relax into just the right shape (for my body at least).