Yeah, it would be weird to say 'listeners' when talking to a group. But social conventions and language shifts. "chat" has established itself for pretty obvious reasons, so I'm not surprised to see it catch on in the physical world. It's a bit like people saying 'lol' in person was super weird at first, but isn't that weird any more.

Also, I don't think it is anywhere near as weird as how politicians address what they are saying to "Mr. Speaker" when they are clearly actually not talking to that person at all.