This must be one of those personal preference things, because I always feel pulled-together and chic in combos like that. To me, it's a polished way to wear an almost all-black ensemble and I love neutrals.
If you can combine that dark knitwear with the standard aids of colorful scarves, belts and jewelry, that may provide the pop you need. Also consider colorful camis and cardis. Another option is to go for a slightly lighter wash in jeans, though at some point that may affect your 'smart casual' look.
If you have chocolate colored boots they can look quite chic with a black top -mixing black and brown is not a no-no if you do it well.