he wants you to give him some attention and is visible irritated about you, not giving any.

only 1 remedy. be very clear. say you don't like him, and will never, ever, fall in love with him.