I echo Suz on this point in particular:
"In my current more temperate climate, I still wear cashmere but it can itch if I don't layer underneath and/or of the weather is damp or not cold enough. The minute the weather gets dryer and colder -- no itch."
I have found this with wool and merino as well. When I'm really cold, there's no itch. But if I start getting too warm, itch. (Plus, Eucalan. It's specifically formulated to help soften wool and other animal fibers.)
On the topic of animal fibers, there can be a real difference in function. Cotton is a cold fiber. When it gets wet, it doesn't dry super fast and feels cold. Acrylic and polyester as well. But wool does not feel cold to the touch when it gets wet. Furthermore, the fibers swell and become more dense, making the garment actually warmer if anything.
I found the truth of this myself some winters ago. At work I wore tall leather boots, with tights and socks both for warmth and fit. Even with an under-desk heater though my toes would be cold. One day I got home and felt my sock, and it was wet with sweat and cold. I switched to Smartwool socks and there was no more problem.
As far as cashmere in particular goes, YMMV. It can be very soft. I got a Neiman Marcus long-sleeve cashmere tee at an estate sale that is thin and so, so soft. I had a cashmere sweater from Kohl's that I gave away eventually since it was a little small and not a good color for me. Cashmere is not a necessity for me, but it's nice to have.
Most sweaters, cotton and acrylic and poly blends though they may be, work fine for what I need. I had an acrylic chenille sweater once that I was either hot or cold in, so I gave it away; but that's my worst experience with acrylic. If it's going to be extra cold and I want real warmth, then I'll reach for a wool cardigan or merino sweater. Wool is a lovely fiber TBH. But I would never call any of these a necessity.