The few times I've experienced a hypersensitivity to a fragrance it can be in something like a cleaning product, air freshener, even a scented candle -- not just a perfume.
Like what happened to me when I tried Clinique Happy, I've smelled a perfume sample in a magazine and liked it a lot there on the page, then applied the little sample to my skin and then go on to develop an aversion to the smell on me and feel seriously ill from it over a few hours time.
To those that have never experienced this reaction, it isn't a simple aesthetic dislike to a smell -- it's a complete physiological sensory overload negative reaction.
To compare it to something that maybe more people can understand or relate to, it's like the difference between hearing a house alarm or car alarm in the distance (mildly annoying and distracting) and to that of standing directly under a piercing, shrill fire alarm or smoke detector that won't turn off. You get truly physically ill: headache, dizzy, nauseous. It's way more than just smelling an offensive and unpleasant odor like B.O., garlic breath, or stale cigarette and coffee breath.
It's very odd, like others that posted who developed a passing fragrance sensitivity during pregnancy, I developed a passing aversion sensitivity to the smell of Italian food made with tomato sauce, but only during my first pregnancy -- and this is one of my favorite foods!
Totally unrelated, but this conversation made me think of this: have any of you read or heard about the taste reaction in a certain percentage of the population to the herb cilantro? There are certain people that can't bear the taste, it tastes like eating a bar of soap to them. Scientists aren't sure why, but it seems to be genetic and may have to do with an enzyme in their saliva. To those that love the flavor of cilantro in their food, it's strange that others would be repelled and disgusted by it because it's so yummy to them in their salsa. The simple solution here is to stay away from food with cilantro in it -- but much tougher in a situation, say when breathing is required, like at work, in an airplane, or a long car ride with others when the windows are closed.
While I have never told someone not to wear a fragrance, having experienced it myself, I am aware that some people may be allergic and hypersensitive, so like others have stated, I pick and choose the situation where and when I wear perfume and am conscience of applying a subtle amount. That's all.