I can barely stand wearing, or buying gear - but I do so for yoga practice because it's BY FAR the best way to practice! I need a sports bra, salutation pants, and a tight top to move the way I want to. Wearing regular clothes interferes with my practice.
As for outdoorsy trails, woodsy walks, and hikes with our doggies - I don't wear gear. I wear jeans, utility pants, and sneakers or casual waterproof boots. Short puffer, sweater or shirt. Works well and I'm very comfortable
FWIW, if I was a serious runner, skier, swimmer, or cyclist - I would absolutely deck out in gear!