I remember my first encounter with a LV Speedy bag. I lived in a small town growing up, too. When I was 18, I worked in a music (musical instruments) shop. One day, a gal came in to buy sheet music or something, and she was carrying a white one with colorful LVs on it. As I checked out her bag I thought to myself, "Those look like the bags they are selling on those stands at the mall (knockoffs, but I had no clue who Louis Vuitton even was at the time). I like the color combo--why would you want Roman Numerals all over your purse, though?" I told the lady I liked her purse and said I saw they were starting to sell them at the mall. Did she get it at the mall? She got really huffy and said, "Well, I got mine at a boutique in Hawaii." When she left I thought, "Why would go all the way to Hawaii to get a purse like that when you can get one at the mall?" Anyway, later on when I figured out the LV was a logo splashed all over the bag, I was rather less impressed with it.
BUT--here's the thing. You have loved the bag since you were in college, and because you love it (I hope) you would get a lot of use out of it. You have the money to spend on it, and the Speedy is pretty classic, too. Unless something is seriously unflattering (and I would not put it in that category), does it really matter what we think--if you love it that much?
I think you should get it. Just expect some 18 year old to ask if you got it at the mall.