It is expected that many of the more complex Python libraries will break, as they assume no race condition on things that could have now a race condition when GIL is removed.
It will also mean that we can use threads instead of processes and a lot of pain related tot his should disappear in long term.
> "No matter what happens there is going to be a lock occurring. No two PyThreadStates can execute Python bytecode at the same time. However, they can execute multiple C calls at the same time which is why for long running pure C operations extension and embedding developers are encouraged to release the GIL temporarily."
Very exciting! I wonder what the first set of motivating applications are and what kind of performance gains are they expecting.
In the TV show "The Orville" a character has their matter replicator create "500 cigarettes", but it only created 463 after someone counted them all. This is a meme that's been making the rounds for a few weeks in some circles now.
Afaik, you could have multiple CPython interpreters per process before, but the GIL was global to the address space, not a per interpreter instance. I believe the GIL work also put interpreter state behind a single struct instead of a handful of global variables.
This could be totes wrong, all from memory, too tired to fact check.
Putting interpreter state behind a struct allows for multiple CPython instances per address space and each can no run on their own thread independently, they all get their own GI(L) lock. These interpreter instances would still have their own Global to This interpreter Lock, but not a Global lock for all CPython interpreters running in a process (like it was before with interpreter state in globals).
Unrelated but complementary. We could remove the GIL, making Python multithreaded, but if interpreter state was still in a handful globals, you could have only one interpreter per process.
Does this mean that even the Python 3.13 version is around the corner we have to expect some issues when running threads on a program?
Maybe I’m slow today, but are the author complaining (rightfully so) about the buggy no-gil implementation?
Should we wait until the feature is more stable before implementing new libraries? I was thinking on making use of it in new libraries I plan to develop.
What are the most blatant use case to test this feature?
The article says that embedding (calling Python from C/C++) is currently buggy. It is not clear whether the Python interpreter was compiled with --disable-gil.
I did not check the claim, since, as the article rightly points out, there are general issues in the Python space and other languages like Lua are far better for embedding.
It might not have been obvious, but I used the currently available Python 3.12.5 release. However upon filing a ticket, and being asked for more testing, we found this issue persists into 3.13 and 3.14 alpha with and without the GIL disabled :(
These threading issues may also be present when not using embedding. Embedding the interpreter frequently shows general threading bugs much quicker.
In other words, when "torturing" the interpreter with embedding a threading bug might show up after 30 min. The same threading bug might show up once every 6 months in a web server (which is a nasty security risk, since it is hard to detect and to reproduce).
I did open an issue on GitHub with the CPython implementation and it's being looked into. The crash happens for sure in the debug version of Python. However we're seeing valgrind and address sanitizer fire off for the upcoming 3.13 release in release mode when using the C API as I showed in my example. However, this behavior only appears to occur when embedding python directly, not via extending it, into an application. Because of how the (hidden because it is experimental) `_xxsubinterpreters` module works internally, it's extra internal bookkeeping seems to give enough time for the race condition to disappear.
Sometimes articles just don't get traction and then they're invited to resubmit (by dang, the moderator). Since the submitter submitted it twice, that was probably what happened here.
As to why it's "more popular", I guess you mean is getting more upvotes, that's part of the trick with submissions. Post it on a slow Friday work day before a holiday and you'll get more clicks than some other days or times.
It was entered into the second chance pool[1]. Interim posts were probably not as popular because I accidentally broke my site's CSS briefly that wasn't obvious in the preview deployments, but was present on the actual site. I only noticed about an hour ago
It should be a brighter orange closer to what you see on HN's top bar. I've been waffling on the magenta. It works "fine" in low usage. This is the first post where I used it a great deal. I'm gonna be spending some time working with the colors over the weekend, as you're the 5th person to say something to me about it.
It could be worse, however. My last design would flashbang you, even if you used darkreader
Or is the functionality as of 3.13 still limited to low level python embedding applications?