I tend to feel that everyone has a yellow that they can wear... it's a matter of figuring it out. If you want. Cuz there are many many shades of it!
I only came to yellow fairly recently. The shades I CAN wear are bold in your face big bird yellow, and citron. I don't like the bold yellow, I find it just a bit too much. But I guess I do have it as a colour mixed with others... namely two items that Angie also has, the Boden sweater and J Crew scarf (which I love love love together). But citron... I love. It has to be clear, and not veering in to chartreuse.
One of my first purchases was a purse, and I recommend that as a way to start... it goes with so much. I get compliments every single time I wear my citron coat and sneakers (not together, just every time I wear them). I've also got a couple of tops. The linen tee was probably my most worn tee last summer, much to my surprise.