If you have an underlayer that ends at the same point at the sweater, or intentionally peeks out (like a layered shirt) than to my eye you can go with the same color as the sweater or with a contrasting look. I've found that sometimes the under-layers aren't quite the right length though, especially if the sweater is long/oversized or cropped. Then what works best for me is a cami that exactly matches my pants, so that even if I move around, the topper over a column of color look stays consistent.
Also I have fun with trying lace back tops on backwards to see if they could work that way too, because with a jacket over the sweater the design detail is hidden.
