Hah I love that you would pull over and hike! Sounds like how we roll on the weekends as well. Last year I determined that my summer non-work wear needed to be what I call "active casual" -- basically what you just described, something that would be fine for if we pulled over to hike on the way to a social event. This can be a tall order! I'm still working all the kinks out but I am doing a lot better this summer with it. A mix of some gear is sort of essential to this, and trying to figure out which gear pieces I feel fab enough in has been the main challenge.
My primary gear item is refined sneakers; I have the chambray Vans below, and am planning to pick up another pair in another shade. From there I look for pieces with natural fibers, or with at least 40% of a natural fiber. I have one pair of jeans for "weekend" that I can romp around in, along with a pair of denim shorts. Several patterned or colorful cotton or linen based tanks/tees/sweaters/shirts that I constantly remind myself "it's ok to mess this up!". And a lightweight dress or two. I'm also picking up a pair of slouchy capris (gear) for this little capsule. So far this has been working out very nicely!
An example in action: On Saturday hubby said let's go get lunch. I wore the dress below with gold smoking slipper flats; it was close to 90F, so the loose cotton dress was appropriately cooling. Somehow what we ended up actually doing was driving out to one of the nearby mountain passes, eating sandwiches at the base of the ski lodge there, taking a ski lift halfway up a mountain, and taking a short hike. This sort of thing happens often
