It's not a crisis--it's a journey!
I'm not a SAHM but I did have some issues with feeling like I needed a better 'loungewear" and at-home wear look, but I still wanted to be very comfy.
Here are some thoughts to ponder as you sort through your styles. They're a little random, so bear with me:
Read some of the posts on identifying your "activities" and on wardrobe capsules.
Write down (or identify) some of your basic activities & don't think they are all the same--playing on floor with baby, housework, errands and many different kinds--peds visits, play dates, shopping, out to park, out with friends, out with DH, etc.
I think one thing you might be aiming for is a) to look and feel more fab and b) not to look the same all the time (although always looking fab is good!) if the default is to your most casual look.
You don't have to have a different outfit or look for EVERY activity, but thinking about it, you'll find that's where you can introduce some variety and fun with fashion.
First, if you wear PJ's, get nice ones you like and don't wear them for something else. Don't wear your day mom outfits to bed. Boom, you've already established PJ's are for bedtime and not daywear.
Look at your current "leisure wear" and think how you can upgrade it while still being comfy. So, okay, jeans are great for some but if tight, or too crisp, or whatever, not as comfy as knits, maybe. But, baggy faded sweats may look the worst. So, think about finding soft and comfortable jeans or whether you're a yoga pant girl, and if so aim for really good-fitting yoga pants. Footwear similarly--big clunky track shoes are looking "out" but slimmer-profile athletic shoes, Converse, athletic Mary Janes may work. I have fussy feet so I won't wear something just because it's "in," but I'm learning to recognize more updated looks in shoes that are comfortable for me.
I'm advocating for not aiming too hoity-toity with baby-care wear because i don't think people can keep it up--instead, you're trying to look at folks on the street, browse some magazines or YLF, and see, what iterations of comfortable at-home wear look more modern and fresh, and what FITS you really well?
I used to wear a lot of drawstring or elastic-waist comfy sweat pant type outfits but they look pretty sloppy on me and don't do anything for my shape. Also baggy T shirts and overshirts too big. As just an example, found some nice Eddie Bauer knit tops with some lace detail, and sweaters with some stitching and ruffle detail, that were just as comfy but were more fitted and looked more together, and I kept just neat-fitting pants and jeans that were soft but not dumpy. I'm still on the lookout for other items to create a few more outfits that have perfect curl up and read comfort that is NOT my pajamas and robe, as this is a goal for me.
Take slippers--no matter how great shoes are, there's just something about slippers.... But I have 81/2-9 feet and get XL in slippers, and usually the poufy or shapeless kind is what is comfortable. I finally found a brand --blocking on name--that is a kind of pile-lined slipper-moc that passed the total comfort test plus looks more streamlined, and has rubber sole so I can run out to car or recycle bin or whatever. I bought in 2 colors!
Then, maybe you can amp it up one notch for errands. I used to (and still do occasionally) run errands & to grocery store in pretty sloppy wear, but I'm having fun wearing nicer jeans and shoes (still comfortable, like leather slip-ons) and maybe a necklace (ooh la la!) and one of my nicer jackets instead of a hoodie. The point is, I felt I looked better and was having fun.
I kind of moved all my activity outfits up the food chain a notch or 2--an outfit I might wear to the movies became a nice loungewear outfit, my usual movie or casual date night outfit became more of an errand outfit, I aimed a little higher for date night, etc.
Still, aim for comfort and great fit or you won't wear it--it needs to still feel like "you", but you can stretch a little. You'll have ups and downs, but that's fashion experimentation!
So you can be thinking of outfits or where you'd like to go with your outfits, and the capsules concept helps when you a) need several outfits of a similar nature or b) have clothing items that can work for more than one activity.
Hope these ideas give you some encouragement and direction.