Very often, side seam pockets on pants gape out on me. Does anyone know why this happens?

I am trying this pair of COS pants that I love & fit well except for side-pocket-gape. Would not size up or down.

Have you ever sewn pockets down to get the look you want?

I am going to pass on these even though I generally love because of the pocket issue plus cost. If cost goes down I might re-buy and sew down. In the past I might have kept them despite because the comfort and look are otherwise there. (My time here has changed my eye, I think.)