I second the couch-to-5K. It's essentially what you are talking about doing, but with more structure to it, i.e., run for 5 minutes, walk for 2, etc... until you are running the whole 30 minutes straight.
I have recently rebooted my own running program after a hiatus, and I'm doing more of a "freestyle" version of C25K... I run for a song, then walk a minute until I catch my breath, then run for another song... If you are in an urban area, you can also try running one block, walking the next, etc.
Basically, at my age, I've learned to just listen to my body, taking breaks whenever I need it, which sometimes just isn't on any schedule. Most important tip I can give you is to GO SLOW - run slowwwwwww, take lots of breaks, do less than you think you can do, and always listen to your body. As for the yoga, I would recommend doing it after your run, rather than before. YMMV, but this is what works best for me. Before running, I walk for at least 5 minutes to warm up, then my first few minutes of running are very slow.
As for what to wear, do invest in some "technical" running gear. Needn't spend a fortune - you can find good gear at Target, Wal-Mart, TJ Maxx, Marshall's. I will also recommend New Balance Outlet - which you can find online. Good deals and quality stuff that fits well.
It is worth investing in technical gear specifically for running - it really does make a difference. Look for fabrics that wick sweat away from your skin. *So* much better than cotton! But spend your most $ on a decent jog bra (note: some yoga bras won't provide enough support for jogging). You'll probably need two of those.
Right now, in this heat, I am wearing a new balance jog bra that extends so it is like a tank but with a bra built in. Covers my tummy (which I don't like to flash in public ever), and eliminates the need for both a tee and a bra. With a pair of loose technical shorts. And double-layer cool-max socks. (The extra layer eliminates chafing). That's all you really need for now - if you keep up with it into the fall, then post again and we'll tell you what to treat yourself to next.
Of course, decent sneaks are most important, but I'm assuming you already have those?