I haven't seen any frump in your outfits! But I do understand your dichotomy: you have two very different audiences to dress for. I don't think there's anything wrong with it unless it bothers you.
I don't work outside the house, and aside from my daily walks, I don't actually leave the house every day, so if I didn't wear my clothes around the house I wouldn't wear them at all! I know this because I spent a year trying to figure out a loungewear capsule while neglecting most of my 'normal' clothes before realising it wasn't going to work. I don't have to worry about adhering to a work dress code, so I don't come home at the end of the day longing to change. Instead, if I'm not feeling well, I just put on comfortable clothes from the start! And I got over the mental worry of 'saving' my favourite clothes and not wanting to ruin them just in case of an eating/dog jumping/etc incident, by deciding it was better to wear something and love it and have it die an honourable death instead of moldering away in my closet. Although actually so far I've been able to launder out any stains and mend ripped seams, so that proved to be a false fear. The only difference between what I wear around the house and what I wear outside of it that isn't reflected in my WIW pics is that if I don't wear bras at home. If I'm wearing a clingier top that makes my bralessness obvious, I wear one outside & for both the pics to spare YLFers and preserve my modesty. Oh and when I'm cooking I wear an apron, which hasn't yet made a WIW appearance.
I'm really glad you asked this, because lately I've been trying to figure out why I neglect my indoor jackets/blazers. I've been incorporating more of them into my dog walking outfits and really enjoying it, so I was a bit mystified as to how they all became orphans. But I just realised that in my mind jackets/blazers are too constrictive for in-house all day wear, so I don't consider them an integral part of my outfit. I think I'm going to try wearing one all day soon and see if it's really uncomfortable or just a mental block of mine.