Just for starters, the restaurant at the gallery (Frank) is really nice and really good. Not cheap, but good. The gallery is situated in the midst of the old Chinatown. These days most of the Chinese have moved to the 'burbs but there are still quite a lot of businesses and a lively street scene in the area. I used to live just a couple of blocks to the north back in Bowie's heyday!
If the weather's nice it would be fun to walk (if you are up for it) from the gallery up through the University of Toronto campus and from there to Bloor and Yorkville -- all the higher end stores are up there -- check out Holt Renfrew and the boutiques and smaller galleries in Yorkville. This is about a half hour to 45 minute walk (depending on your speed.) You could have drinks at the Four Seasons. You'd be moving through several distinct neighbourhoods and getting a little taste of them all.
You will be staying in the entertainment district. There is tons of shopping along Queen West. Used to be a very funky arty area but has become more box stores; there's still some interesting stuff if you continue west towards Bathurst. Also on King West. All the boutique hotels around there are pretty cool and most have bars. The Thompson is one. The Ritz Carleton on Wellington is pretty spiffy. There's also the Drake.
Toronto's truly a city of neighbourhoods. You can pick a neighbourhood that you want to explore, get there by transit, and have a grand time just checking out whatever there is to offer. Little Italy over on College West is a great place to spend some time.
Avoid Yonge Street as much as possible. It is like a pathetic anemic imitation of Times Square but nowhere near as fun.
I will pop back in with more ideas once I collect my thoughts.
