Do you have to wear a sweater dress too? I mean, a skirt/dress of heavy fabrication?

If not, how do you balance the visual weight? Big scarf?