Here's my understanding.
Technically, a "button down" shirt (in men's clothing) is exactly what you describe -- a shirt whose collar can be buttoned down.
But on YLF people seem to call all shirts with buttons "button downs."
I call them "button up" or button front shirts because Mr. Suz (who hails from New England) would get very fussed about my terminology if I called them (in his view, inaccurately) "button downs."
An additional wrinkle: A shirt is not the same as a blouse.
A shirt is (like) what men wear. Straighter lines. Definitely buttons up the front. Fabrication can vary but is usually crisp. May be tailored or loose, may have different collar styles.
A blouse, on the other hand, is softer, more fluid, often a softer fabrication like silk, may not have buttons or may button in the back. It....blouses!
Men haven't really worn blouse type tops for a long time, although maybe some wore them in the 60s, and before that in the Romantic era. In the contemporary world it's more a women's thing.
