Jules, I got that vibe from Gus and Max as well. Giancarlo Esposito played Gus so well. What an interesting character! (Does anyone else remember that actor from Homicide? He played Gee's son.)
Lydia was a calculating control freak, filled with anxiety and the need to protect her own at the expense of anyone else. I enjoyed her character but could see how Walt would want her gone too. He really wanted to erase that entire operation as thoroughly as he could. She was the nearest to the top of that pyramid (above Todd and the Nazis) as he could strike.
Plus, Lydia could pose a very real threat to Skyler, as Claudia pointed out. Just like everything Walt did, there was a motive for his own personal power, but also a sub-motive, to protect his family.