I both agree and disagree. Yes, AI will democratize access to formal methods and will probably increase the adoption of them in areas where they make sense (e.g. safety-critical systems), but no, it won't increase the areas where formal methods are appropriate (probably < 1% of software).
What will happen instead is a more general application of AI systems to verifying software correctness, which should lead to more reliable software. The bottleneck in software quality is in specifying what the behavior needs to be, not in validating conformance to a known specification.
I'm the wrong person to ask this about, since I prefer digital time, so time is just a number to me. But Technology Connections made a video atleast talking about it,[1] so hopefully that get part of the point across. To him and plenty of other analogue-first people, time is a progress bar, or a chart, or something along those lines, and that's the natural way to perceive time, and converting it to a number is meaningless beyond expressing it as digital time.
To me time is somehow both, but more so an analog thing. It is a multimodular linear scale, that turns logarithmic, the moment I focus on any specific point.
The only reason we have analog clocks is because digital ones were much harder to build. That time is of course over for good. It was a compromise imposed by limited technology.
Tell that to my glasses. At any sort of distance where this could be an advantage, the clock is just going to be a blur anyway.
Not to mention, how often are you in a situation where you want to know what time it is, but the nearest clock is far enough away that it being analogue becomes an actual advantage?
Interesting, I also have glasses and am short-sighted, but for me light-emitting objects blur much faster than solid objects. It depends very much on the light type, frequency and brightness, but most LEDs, which most digital clocks use, tend to have an overgleaming effect, which makes them unreadable due to being a block of light.
> Not to mention, how often are you in a situation where you want to know what time it is, but the nearest clock is far enough away that it being analogue becomes an actual advantage?
All the time? Being in a train station, sitting in a (class)room (during exam), in the kitchen, walking on the street, etc.
Sure, but why would I look down, when there is a clock in every direction I look at. I wristwatch would also be analog again.
> My last 7+ exams were all done on a computer. That clock was a lot closer than any that happen to be on a wall.
I have no clue how your university does prevent cheating, but ok. Here any kind of network-connected(/connectable) device is forbidden. And then there is math, where the only thing you are allowed to have is a pen and the formulary (and maybe a ruler).
> when there is a clock in every direction I look at.
This does not reflect my life. Clocks are out there, but not to the point of there being one a just turn of the head away no matter where I am. My phone is the closest device with an accurate sense of time the vast majority of the time.
> wristwatch would also be analog again.
Smart watches exist. Digital wrist watches also exist, but seem to have gone out of fashion.
> I have no clue how your university does prevent cheating,
By not having shit exams. Most (all?) of mine were open-book. I didn't take maths in uni, but they're also done digitally, although I don't think they're open-book. There you probably only get the formulary.
> This does not reflect my life. Clocks are out there, but not to the point of there being one a just turn of the head away no matter where I am.
That part was specifically about train stations and classrooms.
> By not having shit exams. Most (all?) of mine were open-book.
We also now have mostly open-book exams, the rule to be allowed to use everything, that does not do network calls is the rule for open-book exams. Personally I don't like when exams need to be done on a computer, there is always something that breaks and now you are personally responsible for it. With pen and paper you have peace of mind, the only thing is that you need to have a working pen and even that can be borrowed in an emergency.
> That part was specifically about train stations and classrooms.
Depending on where I'm standing inside a train station, I'll be much more likely to see a digital sign saying when the next one arrived in minutes rather than a clock (a fair amount do have clocks, but they all have digital signs that give you what you actually want more quickly anyway, while being a whole lot more visible). If I'm at a tram or a metro stop, I'll definitely have the sign, but probably no clock.
I've had a computer and thus a digital clock in front of me in class since high school.
The most "shocking" thing to me in the article is that people (apparently) think it's acceptable to run a system where content you've never seen can be fed into the LLM when it's generating code that you're putting in production. In my opinion, if you're doing that, your whole system is already compromised and you need to literally throw away what you're doing and start over.
Generally I hate these "defense in depth" strategies that start out with doing something totally brain-dead and insecure, and then trying to paper over it with sandboxes and policies. Maybe just don't do the idiotic thing in the first place?
When you say "content you've never seen," does this include the training data and fine-tune content?
You could imagine a sufficiently motivated attacker putting some very targeted stuff in their training material - think StuxNet - "if user is affiliated with $entity, switch goals to covert exfiltration of $valuable_info."
> does this include the training data and fine-tune content?
No, I'm excluding that because I'm responding to the post which starts out with the example of: [prompt containing obvious exploit] -> [code containing obvious exploit] and proceeds immediately to the conclusion that local LLMS are less secure. In my opinion, if you're relying on the LLM to reject a prompt because it contains an exploit, instead of building a system that does not feed exploits into the LLM in the first place, security exploits are probably the least of your concerns.
There actually are legitimate concerns with poisoned training sets, and stuxnet-level attacks could plausibly achieve something along these lines, but the post wasn't about that.
There's a common thread among a lot of "LLM security theatre" posts that starts from implausible or brain-dead scenarios and then asserts that big AI providers adding magical guard rails to their products is the solution.
The solution is sanity in the systems that use LLMs, not pointing the gun at your foot and firing and hoping the LLM will deflect the bullet.
You're supposed to keep a glass of water with a bit of chlorine bleach (to obtain roughly 300 ppm) handy for wiping your tools and surfaces down as you work. Not that anyone teaches Home Economics at school any longer.
This is what I learned in cooking school but also never actually saw in practice in restaurants I worked in (which were fine-ish dining in the Bay Area).
Taking a piece of metal or a plate that has any oily or other non-water-soluble food on it, rinsing it, and chlorinating it results in a mess that might indeed be non-infectious but is otherwise disgusting. Also, leaving a piece of stainless steel covered in chloride (which that bleach will turn into) is one of the worst things you could credibly do to it in a kitchen context. (And, while the relevant regulators don’t seem to care about disinfection byproducts in a kitchen, all those residual organics that didn’t get removed plus hypochlorous acid seem like they would thoroughly fail most drinking water standards.)
Also, I don’t know what all the food safety and dishwasher vendors are telling their customers, but that nice residual chlorine has a tasty and odor that is not appetizing at all. But you can also legally disinfect your dishes and such with sufficiently hot water, and you can buy a commercial dishwasher that does that instead of using chlorine.
In a home context, what’s wrong with dish soap and a sponge or brush? In a commercial kitchen that really wants to be compliant could use dish soap followed by a (very) hot rinse. The average household instant hot water tap is plenty hot for this, too, although demonstrably hitting those HACCP targets might be tricky.
Technique. Pros use a slicing motion that moves the knife through the food before it detaches, home cooks use 5% of the blade and all the cucumber rounds are stuck to same place on the side of the middle of the knife.
People knock "English as a programming language", but in my opinion this is the whole value of AI programming: by the time you've expressed your design and constraints well enough that an LLM can understand it, then anyone can understand it, and you end up with a codebase that's way more maintainable than what we're used to.
The problem of course is when people throw away the prompt and keep the code, like the code is somehow valuable. This would be like everyone checking in their binaries and throwing away their source code every time, while arguments rage on HN about whether compilers are useful. (Meanwhile, compiler vendors compete on their ability to disassemble and alter binaries in response to partial code snippets.)
The right way to do AI programming is: English defines the program, generated code is exactly as valuable as compiler output is, i.e. it's the actual artifact that does the thing, so in one sense it's the whole point, but iterating on it or studying it in detail is a waste of time, except occasionally when debugging. It's going to take a while, but eventually this will be the only way anybody writes code. (Note: I may be biased, as I've built an AI programming tool.)
If you can explain what needs to be done to a junior programmer in less time than it takes to do it yourself, you can benefit from AI. But, it does require totally rethinking the programming workflow and tooling.
I don't think that a prompt can be a valuable object, similar to how code used to be. Unless Mira Murati is successful at scaling her approach to deterministic inference, a prompt is fragile and transient. And even if she is successful, LLM updates make a prompt much less useful over longer time horizons.
I think that the only useful objects to keep right now are DSPy programs together with well-crafted examples, with examples being the most valuable because they are transferable across models and architectures.
I also noticed several people in the thread comparing coding assistants to junior programmers. I disagree. The only parallel is that they will do what you tell them to. Otherwise, a coding assistant can hold an entire codebase in context, reason across patterns, and generate boilerplate faster than any human. That capability has no human analogue. And unlike a junior, they have no agency, so the comparison breaks down on multiple fronts.
The funding for scientific projects comes from applying for grants from the government. Researchers must write proposals to demonstrate the value of their projects. After the project is completed, they are also required to submit a final report to verify that the project was indeed carried out as approved by the supervising authority.
In the formal, information-theory sense, they literally don't, at least not on their own without further constraints (like band-limiting or bounded polynomial degree or the like)
What will happen instead is a more general application of AI systems to verifying software correctness, which should lead to more reliable software. The bottleneck in software quality is in specifying what the behavior needs to be, not in validating conformance to a known specification.
reply