I, too, have an aversion to ironing, so to me you don't look TOO disheveled. In fact, I really like the outfit/colors. However, button down shirts are the pits. There are two things I do to keep them from running wrinkled:
1. pull them out of the dryer ASAP, hang them on a hanger, and pop the collar. For some reason letting it cool with a popped collar allows me to lay it more crisply lately.
2. Iron just the collar and the placket (where the buttons are... I think that's the word). Maybe the hem, too. The rest will get wrinkled anyways as you sit and move, but having clean outer lines makes the inner lines less noticeable.