It wouldn't worry me. Earrings generally aren't that big, and they're really far away from your feet. But I'm not a particularly big fan of matchy-matchy, so it might bother me less than it would bother others. If your style is very polished and put-together, then I could see why you might want to match your earrings to your shoes.
How big are the silver earrings in question? In my experience, people don't even notice your earrings unless they're huge and shiny, or so long they nearly graze your shoulders. Small tasteful silver hoops, or even relatively big but slim silver hoops, just aren't going to be that noticeable. Unless you also have a pixie.