I thought you looked really good in the "long shorts" awhile back, so you might want to try the culotte thing. Maybe yours would be just below knee and not the long ones, and maybe a straighter style, so not too voluminous. I think you could rock a sort of cropped top with it--just at high hip. That style is often available in more structured fabrics. Or, use a drapey T with semi-tuck or full tuck. And you could use a graphic T with dressier footwear, or more dressy top with sneakers.
I like the look of interesting jewelry with dressy T's. The "statement" necklaces overwhelm me and I don't like huge collar ones, or bling, so I tend to look for artisanal or ethnic items, interesting pendant, stones, shells, that sort of thing. Would you do a T and a funky wristwatch or cuff? Bracelets slide around and annoy me but I would wear a colored or chunky watch still, and that is a way to add something to short-sleeved or sleeveless looks.
Also, some kinds of linen-blend pants instead of jeans, that you can roll. Or rayon-blend joggers, if they are not "out" already. I still like the gray linen pants from BR I copied from Sveta last year and they will be my main summer not-BF jean casual pants.
Lastly, summer is actually more vest-friendly than winter, because they aren't warm enough for winter. Is there a kind of vest-over-T look that works for you? Would it be denim, or utility vest, or more avant-garde?
I'm hearing a kind of sporty-luxe look in the combinations of casual, but juxtaposed, hot-weather items.