The CL-SPICE library I used, that wraps the SPICE C library through CFFI, doesn't cover the type of SPICE kernel that I wanted to use for the Comms module. I could try and add it, but it could be more involved than what I expected and put the thing on hold.
So I used the FORTRAN SDK for SPICE, since I had used it before, and it's reasonably small and easy. The alternative coud be using the C SDK, but I went with FORTRAN since I already had most of the code from a previous project.
I have heard this argument before, but never actually seen concrete evals.
The argument goes that because we are intentionally constraining the model - I believe OAI’s method is a soft max (I think, rusty on my ML math) to get tokens sorted by probability then taking the first that aligns with the current state machine - we get less creativity.
Maybe, but a one-off vibes example is hardly proof. I still use structured output regularly.
Oh, and tool calling is almost certainly implemented atop structured output. After all, it’s forcing the model to respond with a JSON schema representing the tool arguments. I struggle to believe that this is adequate for tool calling but inadequate for general purpose use.
The team behind the Outlines library has produced several sets of evals and repeatedly shown the opposite: that constrained decoding improves model performance (including examples of "CoT" which the post claims isn't possible). [0,1]
There was a paper that claimed constrained decoding hurt performance, but it had some fundamental errors which they also wrote about [2].
People get weirdly superstitious when it comes to constrained decoding as though t somehow "limiting the model" when it's just a simple as applying a conditional probably distribution to the logits. I also suspect this post is largely to justify the fact that BAML parses the results (since the post is written by them).
To be fair, there is "real harm" from constraining LLM outputs related to, for example, forcing lipograms or the letter "E" and a model responding with misspellings of words (deleted E) rather than words that don't actually have the letter "E" at all. This is why some authors propose special decoders to fix that diversity problem. See this paper and most of what it cites around it for examples of this: https://arxiv.org/abs/2410.01103
This is independent from a "quality" or "reasoning" problem which simply does not exist/happen when using structured generation.
Edit (to respond):
I am claiming that there is no harm to reasoning, not claiming that CoT reasoning before structured generation isn't happening.
> "reasoning" problem which simply does not exist/happen when using structured generation
The first article demonstrates exactly how to implement structured generation with CoT. Do you mean “reasoning” other than traditional CoT (like DeepSeek)? I’ll have to look for an reference but I recall the Outlines team also handling this latter case.
Others have pointed it out, but it’s the juxtaposition of the fact that she’s definitely not an expert in this subject with a lesson in the subject.
There’s some subtle bits to the humor depending on how charitable you’re feeling. It might just be absurdist, as in “Blackbeard’s guide to astrobiology”, or it may be more mean spirited and playing on a belief that she is not intelligent.
TL;DR - the joke formula is just:
subject=…
person_not_familiar_with_subject=…
joke=“${person_not_familiar_with_subject}’s guide to {subject}“
And the amount of implied cruelty in the comparison is variable.
Those of us subscribing to AI newsletters are constantly slapped in the face with “___ is all you need” style jokes, such that I didn’t even register someone might not make that connection.
I wonder if any sense this is criticism (or actual criticism) is based on implementers of SaaS who have it so deeply ingrained that “haha what if the users of this software did this really extreme thing” is more like “oh shit what if the users of this software did this really extreme thing”.
When I worked on Google cloud storage, I once shipped a feature that briefly broke single-shot uploads of more than 2gb. I didn’t consider this use case because it was so absurd - anything larger than 2mb is recommended to go through a resumable/retryable flow, not a one-shot that either sends it all correctly the first time or fails. Client libraries enforced this, but not the APIs! It was an easy fix with that knowledge, but the lesson remained to me that whatever extreme behaviors you allow in your API will be found, so you have to be very paranoid about what you allow if you don’t want to support it indefinitely (which we tried to do, it was hard).
Anyway in this case that level of paranoia would make no sense. The programmers of this age made amazing, highly coreographed programs that ran exactly as intended on the right hardware and timing.
The idea of giving every character this sort of agency and seeing what opinion builds up about the world is incredibly fascinating.
Depending on how well we assume an LLM would do at this task, it’s an interesting way to see what “real people” would think about a very hypothetical situation.
An important point that’s missed in this is that these small farms are a vital part of the US’ food security. So regardless of what an analogous business in another sector may choose to do, we really want small farms to be sustainable all over the country.
> these small farms are a vital part of the US’ food security
Hmm, that's only true if the kinds of crops those small farms are regularly growing are the kinds we'd want to have already in the ground as an unexpected "food security" crisis occurs. In other words, durable staples with long shelf-lives, as opposed to cash-crops for export, quick-spoiling luxuries, etc.
Are there any stats that might confirm/disprove that? Because if most those small farms are geared to pistachios or asparagus or hemp, then they aren't really serving as a national safety net.
What is the basis for the idea that small farms are vital for food security? Farms, sure. Small farms? We need low-productivity small farms for food security?
I'm no expert in the field, but if asked, I would prefer thousands of small farms protecting our food security rather than a handful of companies too big to fail.
When you have more diversity, I imagine you would get more resilience, more competition, and market forces work properly.
I don't get the small businesses fetishism. Large companies are efficient, resilient, more likely to follow the letter of the law, and more scalably prosecutable if they don't.
The US is currently experiencing a fairly extreme security threat across multiple industries due dependency on a single provider. Boeing and Intel jump to mind, for example.
Small farms are inefficient though. Farming needs a diverse range of skilled labor but infrequently: the industry as a whole trends towards larger land holders being serviced by specialist labor forces which work seasonally or regionally.
And then within that bucket, it's fairly obvious that larger holders are going to be much more resilient to regional problems if they have diversified land holdings where one area can support a downturn in another.
Inefficient from an economic perspective may not be what we need as a society.
Perhaps we need more farmers living the the county, spending money in small county towns, with more work for people with those special labor skills. Get people out of the city and more people into primary industries.
No straight up inefficient. They produce less. More land is occupied for non productive buildings. Heavy equipment cannot be used at scale. Farms which go bankrupt and fall into disrepair don't produce, and the fields degrade.
The only way what you wrote could make sense would be if it's referring to crops grown for animal feed, and comparing that to if the crops were directly consumed by people.
So, the claim is thinly veiled propaganda for vegetarianism.
If that's your answer then you're vastly underestimating how much farmland there is. We have 1.6 million small farms, averaging a couple hundred acres. Large farms usually have a couple thousand acres.
If every farm in the US was around 3000 acres, we would still have a quarter million farms. Even 100k acre mega farms would leave us with 8000+ farms.
"“Seed, chemicals or fertilizer, it’s all in the hands of a few companies that are the only game in town."... "“They all tell me they’re aware of a monopoly problem, and they don’t deny it exists. But they do nothing."
sorry, when I said "thousands of small farms" I meant thousands of small companies producing Seed, Chemicals and Fertilizer.. as well as actually growing plants and animals.
So if that's the actual problem, what's the motivation for preserving small farms? Those small farms are still being supplied by those large suppliers.
Not the OP, but I assume they mean that it's encoded in the type system.
For example Rust gives you a `Result<Thing, ErrorType>`, which might be a `Thing` or might be one of the possible error types given by `ErrorType`. So when you get a function's return value, you have to deal with the fact that it might have failed with this specific error.
reply