AI is a tool, not a thing. With better integration into the traditional programming environment it will become yet another programming tool.

This is basically like when programs went from being text based to UI based. Future software will have AI, LLMs, as a component/interface.

The "LLM is the program" is not adequate, reliable, scalable.

I am also working on something similar. I have written a grammar system for controlling the llm that is not context free.

One of the main challenges is to have a database of what is reality, id -> object, and then have the llm return ids for objects referenced by the user. I am doing a system where something that has no id has not been created yet.

I use story templates with labeled segments to create characters, items, location as well as events. Example: "Once upon a time there was a <role>cowardly knight</role> named <name>Billy Bonkers</name>. The named qualities become the data structure associated with the id of the new character. It can be hilarious to prompt the llm to do nerdy space humor!

I seems to be very possible to create a "narrator driven" game happening in a "real" world.

I plan on using a multimodal llm and all the stuff that is created will have an image. That combined with id-less objects existing but not having a detailed description will make qualities and objects visible in the created images also part of the world.