Like you said, the temperatures in NYC can really vary like crazy in the spring. I don't know if this will help, but here is my NYC travel capsule for when I go back:
- Trench that dresses up and down
- Cross-body bag that closes securely, with pockets I can access when open and lots of storage.
- VERY comfortable shoes (cannot emphasize this one enough - the last 2 trips to NYC, I forgot how much walking I would be doing and had to buy a pair of sneakers. Yes, one pair of sneakers for each trip. Fun but pathetic. My feet were much tougher when I lived there!) Unless you're going to only take taxis, or have robust feet, go easy on the heels. And one pair does need to be boots, because in spring it rains.
- long-sleeved shirts, tunics, or dresses for layering, one for every day there
- two black cardigans for layering under the trench and over the tops; possibly one vest as well.
- 3 pairs pants plus black opaque hose if I have skirts with me.
- 1 or two scarves to keep my neck warm - must be breathable fabrics! NYC can be humid, and if your scarf doesn't breathe your neck will get soggy, and it is horrible.
Also - don't worry about looking like a NY-er. You won't. It's an ineffable ability you need to live there to grasp, kindof like French fashion, so just ignore it.