Are there any solutions to game streaming that build an RPC on top of the DirectX/Vulkan API and data structures? I feel like streaming a serialized form of the command queue over the network would be more efficient than streaming video frames.
What's the point in streaming a video game from one computer to another if the client machine still needs the expensive and power hungry dedicated graphics hardware to display it?
You could use that to have a chungus dGPU with a massive amount of VRAM keep all the assets, position data and god knows what else to do the heavy lifting - determine what needs to be drawn and where, deal with physics, 3D audio simulation etc. - and then offload only a (comparatively) small-ish amount of work to the client GPU.
But those are the cheap parts compared to the rendering itself. A small-ish amount of work is what we're already sending, the final image, because anything else takes much, much more GPU work.
The client GPU still needs all those assets in its own VRAM in order to render any scene that uses them. You would need to stream all of those assets in real time to the client, and last I checked consumer network interfaces are checks notes slower than 16 PCI Express lanes.
I'm still unsure what computational resources are being saved for the client here, the actual rasterization is where the bulk of the work is done.
Only once, then subsequent references to the texture(s) would be done via descriptor. Most game engines will preload large assets like textures before rendering a scene.
That really depends on the game. Loading every texture in advance isn't practical for all games - many newer "open world" games will stream textures to the GPU as needed based on the player's location and what they're doing.
True, on-demand loading/unloading of large textures still needs to be handled. Video streaming handles congestion by sacrificing fidelity to reduce bitrate. A similar approach could be taken with textures by downsampling them (or, better yet, streaming them with a compression codec that supports progressive decoding).
