Be honest to him.
But really you need to get out of this. Especially after the fact he told you he'd be depressed if you broke up with him. This is often true for people who are strongly attached, but saying it, especially if he really worded it like this, is very nefarious.