Well, I think it's a great idea, having just bought a diaper bag with an express eye to future use as unisex travel tote! I wish I'd thought of it sooner!
I *love* the many compartments these usually have for organizing, as you say, a small stash of clothes for a quick change as well as whatever you might want on the flight and for work if you're getting straight down to business on arrival. (I always pack at least a day's change of clothes, so two sets, just in case check-in luggage goes astray.)
I actually like the idea of a wipe-clean interior, because I often bung in edibles and a small water bottle for bringing on those low-budget airlines, and find at least one of my clutch of pens also typically springs a leak on the flight --- every time without fail!
The one I bought isn't so much like a woman's handbag, since my partner selected it so that he could use it too (without turning too many heads, that is!): http://www.mothercare.com/Moth.....lt,pd.html (we got the denim version) I like that it looks like a regular large messenger bag/satchel on the outside.
Another plus: access from the top without having to undo the flap. Gotta love that for getting through security a tad faster (stuff in scarf, belt, phone, sunglasses etc instead of queueing for trays).
If I was buying just for myself, I may have got something more fun and eye-catching to match my carry-on case. I'd been eyeing this one: http://www.mothercare.com/Rose.....lt,pd.html