I think it's a matter of practicality (sometimes) -- as in, your pants might fall down without one.
And a matter of taste at other times.
I like the look of a belt with jeans, in particular, but feel less need for one and less aesthetic imperative with higher waisted jeans (or pants or skirts). Partly because I am short waisted. Yet I most often do wear a belt anyway...and then partly hide it by semi-tucking, blousing the top, etc.
I think if you wear necklaces, have long hair, or wear tops with a lot going on, a belt can be too much. But if you don't, and have short hair, like me, it can add pizzazz.
There are no rules though, except DYOT.