I would be inclined to keep looking honestly. I haven't seem the bags so I might be wrong, but I like to have contrast between my bags and clothes. Even if it isn't a different color, texture is always nice. If the black were patent, crinkled, quilted, moc croc, or something to give it more depth, I would go with that (of course that looks odd on a slouch bag). If the brown were a lighter, warmer shade of brown, it could also work as a nice contrast. The fact that they are both so similar and basically flat colors makes me think it might be worth continuing to look around, unless you don't mind the fact that your bag isn't really gong to be a prominent accessory, which is completely legitimate. I think of bags and shoes as instant ways to liven up an outfit, and since my feet are fussy, I tend to rely on bags more. Maybe you don't have to.
I have seen a lot of women tying short scarves onto their bags here (some come that way as well). I don't know if that is happening where you are or if you even like the idea, but it does give the option to add color.
For what it's worth, I have a black trench coat and wear a lot of black in general, and I love how my cream cross-body looks against it.