OP here with a minor update. He turned up to the social gathering yesterday. When we were clearing the tables away, he was having trouble folding one, so I went over to help him. I teased him, and he laughed about it, and that was the only exchange we had all day. The rest of it (the gathering itself, the after tea session) we sat apart and talked in different conversations. Going home, we went separate ways, so that was a no-go as well.

I felt kind of empty by the end of the day. I feel like we're both walking different paths that only cross when one of us has to express more interest than is normal, which makes us vulnerable at those points so we struggle to act like we're less interested. It's very frustrating, and what's more, because no one else at the social gathering knows we have these 1:1 lunches sometimes, neither of us are willing to talk about it when anyone else is around. Because it sounds shady? I'm not sure...

I guess I'll email him tomorrow morning to ask if he's good for lunch this week... but I'm a little drained. Should it have to be this hard for me to get to know him even just a little better? Nothing's easy, I guess...

I'll update again when there's further development. Thanks for your insights and replies so far, I really appreciate it.