I AM the textile police.
And I live in a very similar climate to yours. (You get more snow and possibly hotter summers. I get more freezing rain. FEH!! But also milder breezier summers, I think.)
Here are my thoughts for what they are worth:
TOPS:
I wear cotton (in some weights); silk (in some weights); chambray; and some poly jersey knits (in air conditioning or in evening) all year round.
I wear linen pretty much in summer only for tops. I might wear it into warm fall. Or start wearing it in late spring. In a white, heavy weight top like the one you pictured the other day, I can imagine wearing it all year round with the proper pieces.
KNITS:
The only knits I wear in summer (no air conditioning) are cotton or linen. If I worked or lived in air conditioning I would probably include silk blend knits. As it is, in a "normal" or typical summer I would probably not wear a lot of knits, period. This year has been cooler so I have worn a lot of them.
In spring/ early fall, I wear lightweight merino and silk/cotton blends as well as some man-made blends. Layered over a shirt or on their own. Plus poly jersey knits. Or modal.
I reserve my thicker merino wools, my cashmere, and what I consider all "true sweaters" (i.e. knits of a heavier weight or with turtlenecks) for cold autumn, winter, and cold spring (i.e. later March/ April in our climate.)
BOTTOMS:
Denim I wear year round, but rarely as full length jeans in summer. Just too hot. This year is an exception. 100% cotton jeans work better, even so.
I also separate my jeans into "warmer weather" preferred jeans, and those that I wear exclusively in winter. I have a couple that just don't seem right in warmer temps. and others that don't feel right in cooler temps.
I have the same lightweight wool shorts Angie described from Club Monaco. Those for me are a late spring/ summer/ early autumn item. Not winter. No way. Not warm enough.
My VC jersey knit skirts are a summer/ warm fall- weight item for me.
My Halogen seamed ponte skirts (lined) are a spring/ fall/ winter but not summer item.
I wear lightweight wool and heavier weight wool trousers all year -- but will not wear the lightest weight ones on super cold days. No way. I think I might do this in Vancouver, but not in my current climate.
Linen pants: Summer only. Maybe early fall for a true trouser style. Anything drawstring or beachy looking stays in summer.
JACKETS:
I don't like cotton jackets in winter. Even my denim jackets mostly get laid aside then, except for the one with leather sleeves. And I don't wear my leather jackets in the summer. Just too warm for them.