Nice to meet you, Pame!
This rule seems to be relaxing. White pants look great with fall browns and olives, and I am looking forward to wearing white with ice blue in winter. It's such a classy color, and snow is white after all, so what's the big deal, right?
The only exception I would make is if restricting white to summertime is fun for you, like a summer tradition that makes it feel special.