What game proponents seem to miss is that most people are mostly interested in other people (we are predominantly social creatures after all). Books, movies, tv shows can create extremely lifelike immersive experiences that center around people. Games, on the other hand, suck at it - humans are way to hard for a computer to simulate. The current fallback is to have scripted bits of bad movies/books embedded in between the mechanistic gameplay, which makes most people just want to grab for the real thing (movie/book). Hence games are mostly popular among young people, who haven't developed much taste yet and who have a lot of time to kill, and fill it with the mechanistic gameplay.
Weird, I would say the opposite. Video games, and now VR, are the most immersive experience you can get. I wonder if you would change your mind after playing a game like the last of us.