All the code generation tools, StarCoder included, still have hallucinations. In this context code that looks good, but doesn't work or has a subtle bug. How do we address that?
> All the code generation tools, StarCoder included, still have hallucinations.
This also includes humans. We "hallucinate" in very similar ways. For example mistaking localhost:8080 for localhost:8008 in a large config file. Attempting to use methods that were deprecated and no longer exist, etc.
IMO there's two ways to prevent this is - one is to make better performing models (architecture/training data/training amount/etc)
The other is the exact same as humans. Compile time tools that let it know immediately if it hallucinated, types, linting, tests, etc.
You just do it as a loop the exact same as a human. You write code, the compiler tells you that method doesn't exist, you adjust your code/consult the documents (also doable with agents).
Sounds same as when I started codding, hallucinated some code, compiler told me it is non-sense, and eventually I understood most of the rules... I still mess it up...
