I feel the same. If you want to engage in conversation with someone you're doing business with, then make it the kind of business where there are huge amounts of time otherwise doing nothing. Like the hairdresser, or a masseuse.
If I had maids it would be much easier.