>80%-90% or so of real life vectorization can be achieved in C or C++ just by writing code in a way that it can be autovectorized.
Yep. I was pleasantly surprised by the autovectorization quality with recent clang at work a few days ago. If you write code that the compiler can infer to be multiples of 4, 8, etc the compiler goes off and emits pretty decent NEON/AVX code. The rest as you say is handled quite well by intrinsics these days.
Autovectorization was definitely poorer 5-10 years ago on older compiler toolchains.
1 - Very few people conduct "proper scholarship", and fail to trace ideas back to their original inception and cite them correctly. This happens time and again in deep learning, where 30+ year old ideas are claimed as "novel" over and over. Many times out of malice by the authors, sometimes out of ignorance.
2 - Peer review in many parts of the industry+research is a joke. Mostly shouldered by early graduate students who don't really know the field well and an incredibly noisy process.
3 - It is common practice now to dump out one's "kitchen sink" of ideas rather than properly refined stuff. Hence the increase in LinkedIn spam, blog spam, arXiv spam style of papers.
> I don't think there are many (or any) upsides to the well documented downsides.
C++ template metaprogramming still remains extremely powerful. Projects like CUTLASS, etc could not be written to give best performance in as ergonomic a way in Rust.
There is a reason why the ML infra community mostly goes with Python-like DSL's, or template metaprogramming frameworks.
Last I checked there are no alternatives at scale for this.
As others have pointed out, these phenomena are well known to many folks across companies in the AI infra space. It doesn't really break new ground. This article is a good exposition of the basic strategies though.
What I would have loved is a discussion around collectives/multi-node setups. And showing how to get determinism at low performance penalty for multi-node reduction collectives.
+1 - there are just so many Asian recipes that can not be done anywhere near as easily on induction stovetops (high heat from direct flame for flatbreads, etc).
Plus a whole bunch of cookware doesn't work with induction (clay pots, non ferromagnetic bases, etc). I do wonder if any of these "environmental" estimates factor in the environmental cost of replacing a bunch of cookware just to satisfy induction requirements.
Simply not true.
There are induction woks available for East Asian recipies.
South Asian flatbreads like naans, rotis, dosas and parathas can definitely be made well with induction. Plus the precision control of heating opens up new possibilities with all cuisine types.
As for embodied replacement costs - that talking point has been used or rather misused to dismiss everything from solar panels to EVs to wind turbines. Just because there is a payback period doesn't mean that it's insurmountable. What's the payback period on fast fashion and other consumerist nonsense? Infinity right?
I guess you have never worked with a slow induction cooktop. Literally we had to spend 15 minutes more for cooking things on induction compared with our previous apartment's gas connection.
Maybe they are better now but it is certainly not the case that all induction cooktops have these magical properties; many are cheap and skimp on something. While in the 5+ apartments I have been in gas has always delivered the same heating experience that I can rely on.
And to your point about rotis, no - it can not be done unless you get a different, heavier bottomed pan suitable for induction. Exactly what I was saying regarding the replacement costs.
Yep - a gas ban basically bans major parts of various cultures. But also even for typical recipes, you can’t do things like tilt a pan to use the flame to heat different parts differently.
As for environmental costs - the thing that surprises me is that induction easily warps even higher end pans. But yes you’re right, you can’t use many different materials.
There is a vast number of sysctl in xnu that have not really been re-examined in over 15 years. Many tunings date back to the spinning rust drive era (for example). There are plenty of examples like this.
Disclaimer: I worked at Apple and poked xnu a bit.
The big problem is a bunch of folks actually take these things seriously and use it as an excuse to freeze the junior hiring pipeline.
At the senior levels this is not actually believed by the powers that be, since a bunch of hiring is still happening to compensate for overdone layoffs in spots, etc.
> Large corporations believe anyone is replaceable.
This is definitely true. By design, large corporations are structured so that there is no single point of failure.
> Again I am an IC & don’t see/hear any extra work done for retention.
Even in large corporations, extra work definitely happens for retention (I have experienced it myself as an IC). Even though everyone is by design replaceable, the organization has some incentive to work on retention:
a) Bad retention hurts the organization's reputation and future hiring (horror stories spread very fast)
b) Within the team, losing a great teammate hurts morale and output and managers know it will result in a hit on their metrics at least for the next half.
c) Managers may not always be able to backfill, and losing an employee can reduce the size of their "empire" that they are often trying so hard to establish at whatever cost.
There's also a constant tension between using fear as a motivator to squeeze more work out of employees, but not squeeze so hard that they quit. Different companies find their own spot on this continuum. For example, Amazon is famously in the sweatshop part of the scale and they could care less about their reputation. They seem to be doing OK though.
Same. The compensation is substantially better at FAANG, but in terms of actual on the ground work being rewarded, almost never the case.
Meta-work (lots of "cross functional" documents, alignment meetings, sync ups with senior tech leads to brown nose, deliberately creating low quality output to justify hiring more people/growing one's "scope") is 90% of it.
Any actual output is largely accidental, coming from the 20% still naive, or idealistic enough to actually care about what they produce.
Fundamentally it is more efficient to process a batch of tokens from multiple users/requests than processing them from a single user's request on device.
Yep. I was pleasantly surprised by the autovectorization quality with recent clang at work a few days ago. If you write code that the compiler can infer to be multiples of 4, 8, etc the compiler goes off and emits pretty decent NEON/AVX code. The rest as you say is handled quite well by intrinsics these days.
Autovectorization was definitely poorer 5-10 years ago on older compiler toolchains.