Anne, this is a really common problem and has nothing to do with body fat -- you are plenty slim and fit. This particular sports bra is cut high and wide, as many of them are. It may be better suited to someone who has more breast tissue. When I was shopping for a bra for yoga (my really structured running bras always feel a bit too stiff for yoga), I found this a LOT. Seriously, I had to try on a dozen sports bras to find one that worked best for me, and even then, this still happens a bit because sports bras are meant to provide the ultimate level of support. (My challenge is fitting a small band with a large cup size without cutting into my armpits too, because the assumption among bra makers seems to be that if you are big busted, you also are a large person with a longer torso. I actually had to size up to a *gulp* 32G to get it right.)
I agree with Krishnidoux that the more structured, separated style of sports bra may provide better support without this extra fabric on the sides. If you really want the right fit, be prepared to try on multiple styles, and multiple sizes in those styles. There is no shortcut to good bra fit.