Aproprio, Well, DH isn't here right now, but if you got it years ago then I personally feel like it's more wardrobe heavy hitter or Heritage/heirloom piece than appropriation. (Also, Peruvian connection sweaters catalogs are mailed out to Americans across the continent). I have a vintage Colombian ruana given to me by a Colombian friend, she got it in Bogota in 1982 and didn't wear it anymore. I wear it because it was a gift from somebody of that culture, it has a story (a cool one, at that), and I wear it not to be trendy but because I genuinely love it.
I think, for me, I only feel like my cardigan pushes into appropriation because I am so OBVIOUSLY not of indigenous descent (I'm pale with pale eyes and pale hair), AND it isn't a pattern associated with my husband's particular tribe. I'm not letting it eat my lunch, because I'm at least not wearing it due to it's being the cool thing to wear to coachella (or something like that).
If the Peruvian sweater has been wth you a long time and you wear it because you love it, I think you're ok. You could also always look into the pattern and know the meaning behind it if you want to feel more respectful toward the culture. To me, appropriation is gratuitous, not respectful (if that makes sense). Maybe somebody else will weigh in with deeper wisdom than me.