Another member of Team Blazer checking in! I wear blazers with my skirts in casual outfits, and there are a number of things that contribute to how "corporate" the final outfits end up. Many good points mentioned above and I agree especially about shrunken/cropped lengths which is a major key for myself (esp. with flared skirts), as well as looking for printed/colorful items which would make things less traditionally classic and work-ish.
One of the other key things is having enough casual fabrication in the outfit. So if you have a thicker, heavier skirt you'll want to maybe wear a jersey top rather than a silky blouse, or maybe a knit blazer over one with some sheen, or a cable knit sweater versus a fine gauge. Of course having the skirt itself be in a casual fabrication would be the easiest (woven and floaty, knit/jersey, denim, etc.) because then you can wear a blouse for a dressier casual or a tee for less dressy casual.
And of course shoes play a big part; plain pumps may look more corporate than say cork flats or an espadrilles sandal.
EDIT: Casual accessories can also help! Like a watch with a woven cloth strap versus a metal band etc.