I battle with this. I am a staunch proponent that if something REALLY works, it only has to work one way. I think Beth Ann made an excellent point with her dress, and others have said similar things. If you have a piece that you can only wear one way, but will wear it often enough to get your money's worth, why worry about it.
Still, I find it prudent to try to make something work two ways. If you have a piece that works in two outfits, if something happens to it's mate, it doesn't become an orphan. This is also good for more expensive items like jackets and blazers.
However, because my wardrobe is small and I am starting from scratch, sometimes I understand the pull of trying to squeeze maximum versatility out of items. Sometimes I really feel myself reaching for a navy or tweedy blazer or jacket, but have held off on buying them because black is so much more versatile and "could go with more" on my current budget. At the same time--- it is a mistake for me to expect that one black jacket is going to complete every look I imagine. There is no one black jacket that can do everything, and searching endlessly would just be a disappointment!