Hacker Newsnew | past | comments | ask | show | jobs | submit | jareds's commentslogin

Is there a good place for easy comparisons of different models? I know gpt-oss-20b and gpt-oss-120b have different numbers of parameters, but don't know what this means in practice. All my experience with AI has been with larger models like Gemini and GPT. I'm interested in running models on my own hardware but don't know how small I can go and still get useful output both for simple things like fixing spelling and grammar, as well as complex things like programming.

One easy way to test different models is purchase $20 worth of tokens from one of the Open Router-like sites. This will let you asks tons of questions and try out lots of models.

Realistically, the biggest models you can run at a reasonable price right now are quantized versions of things like the Qwen3 30B A3B family. A 4-bit quantized version fits in roughly 15GB of RAM. This will run very nicely on something like an Nvidia 3090. But you can also use your regular RAM (though it will be slower).

These models aren't competitive with GPT 5 or Opus 4.5! But they're mostly all noticeably better than GPT-4o, some by quite a bit. Some of the 30B models will run as basic agentic coders.

There are also some great 4B to 8B models from various organizations that will fit on smaller systems. A 8B model, for example, can be a great translator.

(If you have a bunch of money and patience, you can also run something like GPT OSS 120B or GLM 4.5 Air locally.)


I wrote https://tools.nicklothian.com/llm_comparator.html so you can compare different models.

OpenRouter gives you $10 credit when you sign up - stick your API key in and compare as many models as you want. It's all browser local storage.


> (If you have a bunch of money and patience, you can also run something like GPT OSS 120B or GLM 4.5 Air locally.)

Don't need patience for these, just money. A single RTX 6000 Pro runs those great and super fast.


> GPT OSS 120B

This one runs at perfectly servicable pace locally on a laptop 5090 with 64gb system ram with zero effort required. Just download ollama and select this model from the drop-down.


Oh... 8 thousand of eurobucks for the thing.

Or 4 thousand for the NVIDIA RTX A6000 which also runs the 120b just fine (quantized).

Or a single AMD Strix Halo with lots of RAM, which could be had before the RAM crisis for ~1.5k eur.

Or why not just buy a blackwell rack?

Runs everything today with bleeding edge performance.

Overall whats the difference between 8k or 30k?

/s


You jest, but there's a ton of people on /r/localLLaMA which have an RTX 6000 Pro. No one has a Blackwell rack.

As long as you have the money this hardware is easily accessible to normal people, unlike fancy server hardware.


This is the answer. There's a half dozen sites that let you run these models by the token, and actually $20 is excessive. $5 will get you a long long way.


I don't know how it works now, but in my case the Doctor had nothing to do with it. It's obvious I'm blind since I use a cane. I showed the person in charge of accommodations how bulky a braille printer along with all its paper is, and the noise it makes that's loud enough to wake anyone who may be trying to sleep in the same room. They granted me the accommodation since I had to use braille for math, physics, chemistry, and computer science. I think in some ways it's easier having an obvious disability. You can't hide it, and the only time people don't believe your blind when using a cane is at the bar on Halloween.


Yes, my comment doesn't apply to you. Sorry if I implied that it did.


Why are frequent attendance exemptions granted? I'm totally blind and when I went to college my lack of attendance had nothing to do with the fact that I was blind and everything to do with the fact that I made poor choices like other college students. If I didn't have the mobility skills to get to class then I shouldn't have been granted an exception, I should have been told to get better mobility skills before going to college. I think the only time I asked for an attendance exemption was during finals week. There was a blizzard at the same time as one of my finals and the sidewalks and streets were not plowed. This made it incredibly dangerous for me to go to take the test. I just emailed explaining the situation and took the test the next day.


My understanding is that attendance exemptions are mostly to allow a student to regularly see healthcare professionals (ie weekly respiratory therapist visits) without suffering the wrath of a prof who feels that anyone missing more than 2 lectures deserves to auto-fail a course.


I didn't realize that using disability accommodations to get a single was so common. I used the fact that I was blind to get a single in the early 2000's. It may not have been strictly necessary, but I justified it by the fact I had an incredibly loud braille printer that took up a bunch of space. I didn't try to stack accommodations though, since I could walk as well as anyone else I didn't get preferential treatment when it came to location.


What I don't understand (but also wouldn't be surprised about if it is misrepresented by the article) is:

- why would you get a single, for ADHD, non-social-related anxiety, non-sever autism or depression (especially in the later case you probably shouldn't be in a single)

- I mean sure social anxiety, sever autism can be good reasons for a single.

through in general the whole US dorms thing is strange to me (in the EU there are dorms, but optional (in general). And 50%+ of studentsfind housing outside of it (but depends on location). This allows for a lot more individualized living choices.)


It's also useful if your blind, I know this from personal experience. The ability to recognize objects, read package labels, read bios and boot menus, etc has been very useful to me. Claiming that the only things it's good for is white collar work or battlefield targeting isn't accurate. In spite of how useful I've found it I'm not claiming it's going to be net positive, I have no idea how this will all turn out.


Where are these providers and do they offer batch processing? If they don't how does there cost compare to Gemini and OpenAI batch processing? For the hobby project I'm working on batch processing is a great fit. The only cost comparison tool I've been able to find is openrouter and it doesn't support batch processing for cost savings.


Aws?


I'm rooting against Kotlin since it appears to be only usable with the JetBrains ide. I'm totally blind and Jetbrains tools are not nearly as accessible or easy to use as VS Code with all the Java extensions in my experience. At all the jobs I've had no one cared if I didn't use Idea, but considering it looks like there's no good VS Code tooling for Kotlin if I have to use Kotlin professionally it's going to be painful.


Oh I didn't know about that. Is JetBrains improving on accessibility, at least, or not really?


What's unexpected about that observation is that they have actually completely separated the presentation layer from the business logic because such a thing was required to have "Code with Me" and their "projector" project wherein one could use IJ from a browser https://jetbrains.github.io/projector-client/mkdocs/latest/a...

But, I am fully talking out of school because I don't know what the actual, no kidding, accessibility hurdles one faces when trying to do work in such a setup, nor what concessions VS Code has made to fix those problems

But I do know that YouTrack has a dedicated categorization for accessibility reports and I am sure they would welcome hearing about how they could win back those audiences https://youtrack.jetbrains.com/issues?q=%23%7BUI.%20Accessib...


My major issue is that the IDE doesn't use the same hotkeys or similar interface to VS Code. This isn't necessarily a problem, but the fact that it's so different means the learning curve is absolutely brutal so it's not worth the effort for me to look at it. If there was a basic getting started tutorial that walked you through building a simple program, finding and fixing compilation errors, performing basic debugging etc, that was written from the perspective of a screen reader user that would be incredibly helpful. Something like this may exist, but if it does I haven't found it.


As of somewhat recently, Kotlin has a language server: https://github.com/Kotlin/kotlin-lsp

Using it in vscode, while not perfect yet, is starting to be alright.


Thanks, I've starred the project so I can keep up with it. I don't currently have any plans to do anything with Kotlin but will look at this if I do in the future.


Extraordinary claims require extraordinary proof. AMD's processors got much better, more people started buying them, the stock went up.


I figured it out, it's the CIA. Edward Snowden got no punishment for leaking despite "NSA reads minds" and yet the CIA leakers got 40 years and 8 months for not even leaking the code. The CIA has an AI that wrote several hundreds of millions of lines of software exploit code according to Vault 7. CIA and their Artifical Intelligence(s) are the deep state.


Re: another of my comments it is that bit about the stock market which got me to think that this is about psychiatry instead of technology.

I can't entirely discount even the most egregious possibilities of backdoors in software and hardware constrained by: (1) the difficulty of maintaining a conspiracy with a large number of conspirators (the number of people who know about it must be small) and (2) almost a physical law that any device which falls into the hands of the enemy will give up any secrets it has, especially if many instances of that device are available. Granted, in many cases you can make a back door look like an accident, if it is a C program for instance you are going to make a "careless" mistake that introduces a stack or heap overflow.

The thing is that if the US chip industry is caught doing something like than then you will see Europe wake out of its slumber and create an Airbus of chipmaking, that kind of thing has consequences.

That bit about the stock market is a "tell"

Paranoia though is a thought process and it is not going to stop with one idea which may or may not be true but it just runs continously and I suspect if you interviewed this person for an hour you'd see this thought process go through multiple times. My experience with psychosis is that delusions run on rails and you rarely see new or creative delusions but rather a fascination with perpetual motion machines, cures for cancer, unified field theories and the stock market (not like... shitcoins, swaps, options, other derivatives) It's always the same thing and there's barely a pause where one of them ends and something else from the standard playbook begins. Had he not said that about the stock market I might have engaged with it at face value.

The OP really should seek medical help —- a person in this condition who doesn’t have good social support could easily lose their housing.


> (1) the difficulty of maintaining a conspiracy with a large number of conspirators (the number of people who know about it must be small) and (2) almost a physical law that any device which falls into the hands of the enemy will give up any secrets it has, especially if many instances of that device are available.

They've literally done this. They released their historical document for it in 2018. It's a good read.

https://en.wikipedia.org/wiki/Crypto_AG

And you're right: in the Crypto AG story, an engineer noticed their intentional cryptography weaknesses and tried to get them fixed, so they noted that they needed to think of sneakier backdoors. They really have to be careful about who they let in on it.


I'd rather someone rich who wants political change focus on getting ranked choice voting in battleground states. As someone who would like to vote for a third party but generally leans toward one of the two major parties I can't justify most likely throwing my vote away. I'd be happy to put a third party candidate as my first choice in ranked choice voting since I know it wouldn't be a wasted vote.


Why would self-centered rich people push for a system that would reduce the impact of their money on policy?


TO bad they didn't use any of that increased team size to add in any basic accessibility features that would allow me to play with my children even though I'm totally blind. Unfortunately games like Forza that did add accessibility are not available on the Switch 2 and don't offer local multi player.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: