How is simply saying "It really isn't any of your business," lying? Lying would be saying "I never slept with him."