I voted nay, since I have a very long history of buying tops and dresses that are PERFECT in every way except one. And that one thing is always an issue to be negotiated every time I wear it, whether it's mitigating the color, dealing with a less-than-ideal length (top, skirt, sleeve), having to wear a belt even though nothing I have is perfect, etc. They don't turn into pieces that I can just reach for easily. And color, on something by my face, is one of my biggest hesitations. Layering items like scarves and jackets for me usually ends up covering the other things I like the item for.
But all that said, I'm glad you're going to try it. You're very disciplined about returning what doesn't work, and you will give it a fair trial.