Sandals have levels of dressiness or refinement, just like any other shoe or boot. You already understand this, as your post implies. Match the shoe to the clothing and you are fine.
I'd put sandals on a continuum. Dressiest would be high heeled, thin straps, patent leather or sparkles, etc. Those would be for evenings.
Then there are refined low heels or wedges -- no big platform, just a delicate wedge or self coloured heel. Leather in neutrals or burnished metallics. This kind of sandals could go to a business casual office where sandals are allowed.
Then there are more casual platforms or espadrilles -- best left for weekends.
Flat sandals can also go either way, but are typically more casual in feel.