What frustrates me the most about OpenAI is that as recently as this summer they were talking non-stop about how gigantic $100 billion models are all you need for AGI and that it’s just a matter of time until we reach this scale. And if you didn’t see this you’re a simpleton who doesn’t understand how exponential curves work.
And then all of the sudden o1 comes out and the narrative from them has shifted entirely. “Obviously massive models aren’t enough to get to AGI, but all we have to do is scale up inference time compute and we’ll get there!” And if you don’t see this you’re just a simpleton.
I wish that OpenAI was called out for this shift more often. Because I haven’t heard even one of their employees acknowledge this. At some point you have to just ignore them until they start actually publishing science that supports their beliefs, but that won’t happen because it doesn’t generate revenue.
> that won’t happen because it doesn’t generate revenue.
OpenAI made real progress towards a computational understanding of human language and cognition. I'm sorry they have become a for-profit entity (the paperwork lags behind reality, of course). A fiduciary duty does not serve humanity. The quality and credibility of their communications have fallen dramatically.
> how gigantic $100 billion models are all you need for AGI and that it’s just a matter of time until we reach this scale. And if you didn’t see this you’re a simpleton who doesn’t understand how exponential curves work.
OpenAI made no such claim. LLM stans on the internet definitely made such claim, that the Stargate project would be AGI and whatnot. But, like crypto bros, GPT hyperfans are just constantly deluded/lying so you shouldn't project their claims onto the corporations they simp for.
That being said, Anthropic's CEO made a claim closer to what you're saying, that a 10-100 billion dollars model would be better than a human in almost every way.
America gave a trillion dollars out so a lot of people can have 1500 dollars. We have enough money, and we are all not going to live forever. I don’t know what the hold up is.
It’s not an exact figure, but plenty of employees claimed that after we train a model with 2-3 orders of magnitude of compute over GPT-4 we’d reach AGI, which puts us at $10-$100 billion.
See Situational Awareness for one example of this.
This summer he made a comment saying "we can make GPTs 5,6,7 more powerful" and someone editorialized it as being "via parameter count/scale" which he didn't say.
And then all of the sudden o1 comes out and the narrative from them has shifted entirely. “Obviously massive models aren’t enough to get to AGI, but all we have to do is scale up inference time compute and we’ll get there!” And if you don’t see this you’re just a simpleton.
I wish that OpenAI was called out for this shift more often. Because I haven’t heard even one of their employees acknowledge this. At some point you have to just ignore them until they start actually publishing science that supports their beliefs, but that won’t happen because it doesn’t generate revenue.