I feel like this may be the stupidest fit question of all time, and one I've struggled with forever.

When I sit down, my button up shirts gape between the buttons. Because of this I pretty much gave up on ever wearing button ups for 2 decades. Recently I've started wearing button ups again and notice I'm constantly tugging at the front to fix the gape. I'm assuming the answer is to size up, but in that case I lose the fit everywhere else. I can try to catch some pictures if it would help. I wouldn't say the shirts are tight fitting by any means.

Is there a fix?