If it's bugging you that much and you're just going to get angry, then break up.
But rather than that, I advise you try and find out what the matter is? I was in a similar predicament once, he was manic depressive. Now we just curl up together.
Simple activities like making cakes is nice. Find something nice.