It depends on what you are worried about. Etiquette rules do not require a woman to remove her hat indoors except for a few isolated situations. But what was proper 40 years ago when everyone wore hats is considerably different from what people are willing to do today, now that hats are more casual and much less worn.
But, if you're not worried about what the older generation think of your hat etiquette, and you're only concerned with how you look to the younger generation, then wearing a hat indoors will not evoke any gasps. I am always dismayed at seeing men and boys wearing hats indoors. My dad has always been a very chivalrous man, so it was instilled in me that men remove their hats indoors, during National Anthems, and formal processions, Like funerals.
Here's a great article on hat etiquette both historically and modern day.
http://www.villagehatshop.com/hat_etiquette.html
