I do think, if you are wearing black from head to toe, you'll need more than one pop-of-color item or it will look random. For example, let's say you are wearing a black dress, or a black top with black slacks. Adding red shoes only will not be enough. Add a pink cami and red shoes and now you are getting somewhere. Extra credit: add a third color in the bag. If that seems too much, at least try a cream bag or animal print.
If you're new to it all, one suggestion is to not add color but instead add more interesting neutrals that have texture and/or sheen. For example, instead of red shoes, try silver shoes. Instead of a turquoise cardigan, add a cream jacket. Instead of a yellow belt, try a chestnut brown one on a black dress.
You'll see I do this a lot myself. Grey, white, black will be my neutrals and I will wear them together with metallic shoes. The pop of yellow from the banana bag - that did take some getting used to for me!!! When I first started carrying that bag, it felt off and random to me, and I felt it was screaming "look at me" in an otherwise quiet, neutral outfit. But I've been using that bag for a few weeks now and I am used to it enough that I've taken risks wearing it with other colors, like purple. So yes, sometimes it is simply a matter of readjusting our own eye!
Personally I think the easiest way to adapt to color is to pick a pattern and take your color inspiration from it. For example, find a pretty scarf - maybe it has black in it to tie in with your black dress. Then pull out the other colors in the scarf for shoes, belt, purse, cami. It won't seem so random to you that way - even if you simply tie the scarf to your purse instead of around your neck.