Hamm, I think olive is one of those colors that has multiple definitions. Could be brownish, greenish, or grayish. You could try looking for army green or similar colors instead.
Eta: I think Aida is on to something as the color of my olive culottes is indeed "moss"!