I like it. It's got some interesting detail. I would try just one button - the one at the waist. Not two. And take this out shopping with you and try on a bunch of ruffly blouses underneath. Some ruffles at the neckline could punch it up. (Okay so I'm on a ruffle jag lately....). Or a blousy silky tie-neck style, if you're not into ruffles.
If you think the sleeves are too short (I have no idea - being short, I always have the exact opposite problem!!!) maybe a long sleeve shirt that has cuffs that peak out?
Have you tried putting a classic collared shirt underneath this? I'm wondering... it might work, if you only buttoned at the waist.
The black buttons make it seem like black pants are the obvious choice.... but what about (just thinking out loud here) a microcheck in a brown with some black, and then some black shoes to tie in with the buttons?