I think it's time to wear boots when you want to and you're comfortable. I plan on breaking mine out ASAP. I'm in Maryland where daytime temps are still hitting the 70s, but Iceland made me crave more boot-wearing weather!
I'll start with short booties and lighter colors and styles -- the heavier ones will come out later when the temps drop more.