Sterling, I'm with you on jeans. I find a FEW pairs (and I'm quite picky) comfortable enough to wear out and about, to work, etc. but not for lounging.
Confession: I used to wear only pajamas when lounging around the house. However, that changed when we became dog owners. I need something to wear that is comfortable enough to lounge in, but is presentable enough to run out to the yard, or go for a walk.
One of my goals this Spring was to increase my "doggy capsule" - kind of a loungewear/activewear hybrid. This involves yoga style pants or cropped activewear pants, knit tops, and Zella zip-up jackets. Inside the house I'm generally barefoot or wear my wool Halfinger "doggy" slippers, and instead of the jacket I'll wear a cardigan or else a modal robe. I'll either slip on sneakers or a pair of "demoted" flats that are too beat up for work wear but are fine for running outside quickly.