I can't think of a time of year when I don't layer, tbh. Summer mornings can be chilly where I live, so I like to have a variety of cotton and linen toppers to wear on summer mornings -- blazer, field jacket, denim jacket, boxy cardigan, hoodies, and my flannel shirts (which I keep out as jackets in the summer, and which then become mid-layers in the winter). Usually in the summer it's just one layer underneath -- sometimes a tee or tank for casual, sometimes a button-front shirt if want more structure.
Cold-weather layering, I usually start with a camisole, either wool or silk or a blend of the two (unlike Angie I cannot wear camisoles in summer heat -- but maybe that's due to the humidity here). Then usually a lightweight merino top or sweater, or sometimes a substantial cotton shirt -- think flannel, chamois, or a thick St John style Breton top. Like Firecracker, I have a couple long-sleeve Uniqlo Heattech baselayers, with deep scoop necks, that can be layered under lighter weight tops to make them winter-worthy -- they provide a little more warmth than a camisole does. I have a very thin, lightweight down vest that I can slip over some outfits, and which fits underneath almost all of my coats, so that comes out on very cold days -- or acts as an outer layer on fall days.
I like scarves but have a hard time wearing them indoors, even summer scarves -- I almost always end up removing them for a good chunk of the day, so they rarely feel like an integral part of my outfits. I still like them, but I tend to think of them the same way I do hats and mittens -- you take them off when you need to cook or clean or eat or light a fire, or what have you.
I don't get too tripped up by color/pattern, only because I'm so consistent (boring?) in my choices. Black, olive, navy, charcoal, a bit of red and burgundy... there's very little "clash" left in my wardrobe, mostly I have to work to add a "pop" and not just become a blob of dark neutrals, especially in the winter. With maximalism trending, I'm even entertaining the idea of wearing my striped tops underneath my plaid tops, something that usually makes my eyes hurt just thinking about it!