> On the other hand using things like debuggers and reading stack traces in python/js "just work" for me. Maybe because the tooling and the language have evolved together over a longer period of time.
Well, Python and JS don't have threads, so async/await are their only concurrency construct, and it's supported by tools. But Java has had tooling that works with threads for a very long time. Adding async/await would have required teaching all of them about this new construct, not to mention the need for duplicate APIs.
> I also feel like the reimplementation of all functions to support async is not a big deal because the actual pattern is generally very simple. You can start by awaiting every async function at the call site.
First, you'd still need to duplicate existing APIs. Second, the async/await (cooperative) model is inherently inferior to the thread model (non-cooperative) because scheduling points must be statically known. This means that adding a blocking (i.e. async) operation to an existing subroutine requires changing all of its callers, who might be implicitly assuming there can't be a scheduling point. The non-cooperative model is much more composable, because any subroutine can enforce its own assumptions on scheduling: If it requires mutual exclusion, it can use some kind of mutex without affecting any of the subroutines it calls or any that call it.
Of course, locks have their own composability issues, but they're not as bad as async/await (which correspond to a single global lock everywhere except around blocking, i.e. async, calls)
So when is async/await more useful than threads? When you add it to an existing language that didn't have threads before, and so already had an implicit assumption of no scheduling points anywhere. That is the case of JavaScript.
> New libraries can be async only.
But why if you already have threads? New libraries get to enjoy high-scale concurrency and old libraries too!
I agree with your point that for CPU bound tasks, the threading model is going to result in better performing code with less work.
As for the point about locks, I think this one is also a question of IO-bound vs CPU bound work. For work that is CPU bottlenecked, there is a performance advantage to using threads vs async/await.
As for the tooling stuff, I'm still not really convinced. Python has almost always had threads and I've worked on multimillion line codebases that were in the process of migrating from thread based concurrency to async/await. Now JS also has threads (workers). I also use coroutines in C++ where threads have existed for a long time. I've never had a problem debugging async/await code in these languages, even with multiple threads. I guess I just have had good experiences with tooling but It doesn't seem that hard to retrofit a threaded language like C++/Python.
> I guess I just have had good experiences with tooling but It doesn't seem that hard to retrofit a threaded language like C++/Python.
But why would you want to if you can make threads lightweight (which, BTW, is not the case for C++)? By adding async/await on top of threads you're getting another incompatible and disjoint world that provides -- at best -- the same abstraction as the one you already have.
I think the async/await debugging experience is easier to understand. For example in the structured concurrency example, it seems like it would require a lot of tooling support to get a readable stack trace for something like this (in python)
Traceback (most recent call last):
File "/Users/mgraczyk/tmp/test.py", line 19, in <module>
asyncio.run(call_tree(directions))
File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/Users/mgraczyk/tmp/test.py", line 16, in call_tree
await right(directions[1:])
File "/Users/mgraczyk/tmp/test.py", line 4, in right
await call_tree(directions)
File "/Users/mgraczyk/tmp/test.py", line 14, in call_tree
await left(directions[1:])
File "/Users/mgraczyk/tmp/test.py", line 7, in left
await call_tree(directions)
File "/Users/mgraczyk/tmp/test.py", line 16, in call_tree
await right(directions[1:])
File "/Users/mgraczyk/tmp/test.py", line 4, in right
await call_tree(directions)
File "/Users/mgraczyk/tmp/test.py", line 16, in call_tree
await right(directions[1:])
File "/Users/mgraczyk/tmp/test.py", line 4, in right
await call_tree(directions)
File "/Users/mgraczyk/tmp/test.py", line 14, in call_tree
await left(directions[1:])
File "/Users/mgraczyk/tmp/test.py", line 7, in left
await call_tree(directions)
File "/Users/mgraczyk/tmp/test.py", line 11, in call_tree
raise Exception("call stack");
Exception: call stack
No, the existing tooling will give you such a stack trace already (and you don't need any `async` or `await` boilerplate, and you can even run code written and compiled 25 years ago in a virtual thread). But you do realise that async/await and threads are virtually the same abstraction. What makes you think implementing tooling for one would be harder than for the other?
JDK methods can be annotated as "internal" and optionally hidden in stack-traces, but in this case it's unnecessary. The fork call takes place on the parent thread and isn't part of any stack trace when an exception in a child occurs. The regular structuring of exception stack traces takes care of the rest.
Remember that Java has been multithreaded since its inception, and virtual threads don't change any of the threading model. They just make Java threads cheap. It's as if they've been there all along.
Wouldn't the exception actually come from throwIfFailed? Or does it come from resultNow? How does the tool show it as corresponding to the call to findUser?
throwIfFailed throws an exception that wraps one thrown by the child as a "caused by" and lists their stack traces. Java users have had this for many years, but threads were simply costly, so they were shared among tasks. The new thing structured concurrency brings -- in addition to making some best practices easier to follow -- is that the runtime now records parent-child relationships among threads (that now make sense when threads are no longer shared). You can see these relationships and the tree hierarchy for the entire application with a new JSON thread-dump.
Well, Python and JS don't have threads, so async/await are their only concurrency construct, and it's supported by tools. But Java has had tooling that works with threads for a very long time. Adding async/await would have required teaching all of them about this new construct, not to mention the need for duplicate APIs.
> I also feel like the reimplementation of all functions to support async is not a big deal because the actual pattern is generally very simple. You can start by awaiting every async function at the call site.
First, you'd still need to duplicate existing APIs. Second, the async/await (cooperative) model is inherently inferior to the thread model (non-cooperative) because scheduling points must be statically known. This means that adding a blocking (i.e. async) operation to an existing subroutine requires changing all of its callers, who might be implicitly assuming there can't be a scheduling point. The non-cooperative model is much more composable, because any subroutine can enforce its own assumptions on scheduling: If it requires mutual exclusion, it can use some kind of mutex without affecting any of the subroutines it calls or any that call it.
Of course, locks have their own composability issues, but they're not as bad as async/await (which correspond to a single global lock everywhere except around blocking, i.e. async, calls)
So when is async/await more useful than threads? When you add it to an existing language that didn't have threads before, and so already had an implicit assumption of no scheduling points anywhere. That is the case of JavaScript.
> New libraries can be async only.
But why if you already have threads? New libraries get to enjoy high-scale concurrency and old libraries too!