I would rather a VR "LAN Party" than a VR game itself.

[Lumialle]

VRChat already has something like this. You can pick whatever avatar you want to interact with others, and you can play on desktop so your friends without VR won't feel left out.
There are worlds with games in them (including board games and a DND world) so you can really pick your poison. You can also just hang out and watch videos. Give it a try!