In general I tend to reach for the more unstructured bags for weekend (social/casual), and the structured bags for during the week/business. There is room in my closet for both. It never ceases to amaze me how a simple swap of bag (and shoes too) can transform the look of an outfit from business-appropriate to smart casual.
Sometimes I'm in the mood for juxtaposition, so I will play with adding a super-structured bag with an otherwise super-casual outfit like baggy boyfriend jeans and tee shirt, or the other direction, adding an unstructured bag to a more polished outfit.
But overall, on the weekends, I reach for the unstructured bag that I can sling over my shoulder and which conforms to my body, and that I do not have to carry in my hands or the crook of my elbow, as that can get tiring.
Interestingly, I prefer my clutches to be on the soft/squishy/unstructured side. But that's probably because I can pack more things into them.
I always need ONE slouchy unstructured bag, and last season it drove me nuts because I had such a hard time finding one that met my requirements, because structured bags were the fashion. I am seeing more slouchy options for fall - yeah!