You know, it's interesting about relaxing, being yourself, and acceptance.
On the one hand, I believe that totally. If you're comfortable, others will be comfortable with you. You won't get a hard time. People will like you and what you're wearing.
On the other, I'm not sure if you'll "get picked for dodgeball," if you don't seem to be on the team. One of my first bosses told me, after we'd gotten to be friends, I was his pick because I came dressed "button down."