Like others, I don't see this outfit as frumpy. I wonder if ponte pants in a trouser style just have that feel to them, because they're a traditional pant style that feel like sweatpants--two items than can feel quite frumpy indeed.
I have a pair of ponte trousers I just bought this fall, by Vince Camuto, with narrow legs and a slim but not tight fit. I thought about taking them on my trip to Ireland, but I passed them up for my black slouchy skinnies with zippered pockets and blingy pocket flaps that can be tucked in and hidden away or left out. My reasoning: While I was on vacation, I didn't want to feel like I was dressing for work! And, although the Vince Camuto ponte pants are comfy and I think they look good, I don't really feel fab in them.