Initially I thought it's a bloody stupid idea, however at this stage I reckon we need it or a lot of boomers are going to be ones hotted into singing away all their wealth away.
Documentation is a bit sparse but TL;DR - deploy it in a cloudflare worker and now you can access about 15 providers (the one that matter - OpenAI, Cohere, Azure, Bedrock, Gemini, etc) all with the same API without any issues.
Coming back to write something more full-throated: Klu.ai is a rare thing in the LLM space, well-thought out, has the ancillary tools you need, is beautiful, and isn't a giveaway from a BigCo that is a privacy nightmare: ex. Cloudflare has some sort of halfway similar nonsense that, in all seriousness, logs all inputs/outputs.
I haven't tried it out in code, it's too late for me and I'm doing native apps, but I can tell you this is a significant step up in the space.
Even if you don't use multiple LLMs yet, and your integration is working swell right now, you will someday. These will be commodities, valuable commodities, but commodities. It's better to get ahead of it now.
Ex. If you were using GPT-4 2 months ago, you'd be disappointed by GPT-4o, and it'd be an obvious financial and quality decision to at least _try_ Claude 3.5 Sonnet.
It's a weird one. Benchmarks great. Not bad. Pretty damn good. But ex. It's now the only provider I have to worry about for RAG. Prompt says "don't add footnotes, pause at the end silently, and I will provide citations", and GPT-4o does nonsense like saying "I am now pausing silently for citations: markdown formatted divider"
Hey folks, any of you tried with this stream=True?
It works in the playground, but it appears that you cannot actually stream completions from this model via the python openai package.
Wondering if it's a weird PEBKAC or someone else has had the same experience?
Absolutely agree here. We should be intolerant of intolerance in democratic societies and reciprocate with the more questionable counter parties from countries with .. shall we say different moral norms.
Initially I thought it's a bloody stupid idea, however at this stage I reckon we need it or a lot of boomers are going to be ones hotted into singing away all their wealth away.