I don't think you look equestrian at all, actually. Take the sweater off, put on a peplum jacket and a ruffled blouse, and THEN we might have to have a conversation. ;D
You look great! I'm with Antje--maybe you could add an interesting colorful accessory to the outfit. Also, if you wanted to go more equestrian (as you stated above--go for a "look", which I'm all for), I'd consider replacing the button down with a ruffled top, or wear a blazer instead of the sweater. That has a more of a horse vibe to it.