They do stretch, and they shrunk a bit washed and hung dry, but stretched back out. I definitely think you could do the XS for a less sloppy/slouchy fit., as Rae wanted. With regard to height, you can compare me and Jackiec in them as she is much taller than me... They are surprisingly versatile.
Denise, Natalie, Rae, Cathy (cciele), Jackiec, and I are some of the members here that have them if you want to look up other pics.