Well I am so glad you asked! Focusing on second hand, it really depends on the level of curation you are looking for - going from least to most:
Housing Works - various locations, I visited Crosby near Houston (Soho area), the E23rd Street and the W17th street locations. I find these to be the cream of the charity shop crop, and a good cause. I didn't buy anything but saw a number of interesting items (Marc Jacobs, Everlane, Pucci (!))
Brooklyn Exchange - W 26th St near 6th Ave and E 11st St near 1st.
I found for myself Rag & Bone jeans, and Ulla Johnson blouse, M.Gemi loafers, and M Missoni long shorts all for around $25 each.
Beacon's Closet - W 13th St. I didn't buy anything but saw a lot of Rachel Comey in particular, which is unusual.
Tokio7 - E 7th near 1st Ave. They had a sign that they are closed for a week in July so check in advance. A mind boggling array of top tier designers including the whole spectrum from Issey Miyake, Yohji Yamamato, Commes des Garcon, Isabel Marant, Chloe, Prada, Marni, etc. Prices start around $60 and go way up, but there are bargains to be found.
I was mostly looking this year and focusing my shopping on makeup. If you are interested the Glossier Showroom on the border of Soho and Chinatown and a lot of fun. Broadway in Soho has become an open air mall of major brands, but if you don't have them locally it is fun. Lots of street style people watching too. I did not look for boutiques but they are still there somewhere.
Let me know if you have any other questions on one of my favorite topics!
