I see the benefits from both for now, but I’m not capturing how they could agree on the long-term negotiation.

There is so much upside in a few months for only one side of that deal… That was always the issue with Uber promising self-driving cars: human drivers have an unknown, uncontrollable deadline.

If I were Uber I would structure this as "we'll integrate waymo as a trial and if it's successful then we'll acquire waymo after x period of time"

Google will absolutely not sell Waymo after such an incredibly steep investment.

Are you sure Uber sees no viable replacement for Waymo?

They could buy a lot of Teslas or partner with a Waymo competitor, but I doubt anyone with a working automated fleet will need Uber’s human drivers. Acquiring riders is cheap—good PR, a handful of promo codes: branded cars all over the city will do the rest.

That partnership makes sense in the transition period when Waymo can’t buy enough cars to scale for football matches, handle edge cases, and build a larger fleet operation team. I think the individual Uber talent (operations, software, etc.) are great. Their experimentation team, for instance, is 100% something I’d recommend acquiring wholesale. Even with a thousand better options at Google, I’m sure their back-end solutions are work acqui-hiring. But Uber itself? The confusing everything app? It’s a bad brand whose toe-stepping values are misaligned with what anyone is trying to do (except maybe Tesla) and with a ton of tech debt. I know there was a lot of effort to escape it, but I don’t see great synergies. With Booking, maybe?

Also Tesla has backed off on claims their cars will be self driving without a human safety driver. They’ll get there eventually but Waymo is far ahead of them on actual deployment of this tech.

With the boss mentioning multiple times that he wants end to end AI - ie. pixels in, steering angle out, I think this may change quickly.

Nobody else has managed to get end to end AI working, but this sort of stuff scales with training data, and tesla has the opportunity to collect far more than any other manufacturer, with possibly 1000x the number of cars collecting video data of the next closest competitor.

And the thing about end to end AI is I suspect that if anyone can get it working at all, it may not have the issues with the long tail of unusual situations that other approaches have. Things like "Someone has drawn a picture of a bear on the road in chalk, should I drive over it or around it?" might be answered better by the end to end system than a rules based system.

If all of those things work out, then Tesla will win this race. Otherwise, they'll lose.

As you say, no one has figured end to end out. The way AI works some tasks get figured out quickly and some take a very long time. Elon has been talking about an end to end system for a very long time, and I’ve watched every lecture from Andrej Karpathy on their architecture. I like their plan, but they have been working towards end to end for years and I see no reason to believe anything will “change quickly”. AI is very much a “don’t count your chickens before they’re hatched” kind of technology. People can promise anything about AI, because when it works it feels like magic. But it’s not magic, and we can’t expect every goal to be met just because someone is using AI. Nobody knows what challenges lie ahead for an end to end self driving system precisely because no one has done it. And it should be noted that with their “hydra net” architecture it is relatively end to end but it is not a single large network, but an engineered assortment of many small networks. Quite a lot of human choice goes in to decisions around how to design that, and this also leads to edge cases. But even more general systems like GPT-4 suffer from issues with edge cases.

We still don’t even know how plausible a camera based system is. Human eyes and the human brain are so much more advanced than a computer and cameras, it may not follow that “images are all you need”.

> We still don’t even know how plausible a camera based system is.

I hope they have tested this by remote-controlling cars using only the camera inputs.

ie. verify a human can safely drive the car using only what the computer sees.

If a human can't safely drive in those conditions, a computer probably can't either.

Very interesting question. Certainly I’ve never heard them discuss this in any of the AI system lectures. It’s always stated as “we know humans can drive using only our eyes so we have an existence proof”.

Plus, a human could drive in a variety of conditions but there’s just so many possible scenarios the car could encounter, and humans crash too, so the real question is what is the relative performance between a human and the machine across thousands of potential or actual crashes. Kinda hard to measure.

> They’ll get there eventually

Debatable.

So many of their issues are to do with the lack of LiDAR sensors.

True, true. We may see a change of heart for them if solid state LiDAR ever actually hits the market, but last I checked Velodyne/Ouster keep announcing things and then letting them disappear. I did see them saying they have samples available for 2025 model years cars so perhaps Tesla could be testing with that now. I don’t see them deploying mechanical LiDAR but solid state is similar enough to a camera they may do it.

Edit: then again for such a critical component Tesla might want to make their own solid state LiDAR, or at least license someone’s technology so they can produce them on their own terms.

LiDAR also has FoV issues and is sensitive to lens debris among other things. Also, the addition of another layer of high resolution data may make it impractical for the compute power requirement to synthesize it all.

Well they currently devote a lot of compute power to building a 3D occupancy map of the world from cameras so it’s possible that fusion with LiDAR could be done with less compute. Also their computers keep getting more powerful. But generally you’re correct, we don’t know how it would go.

Using LiDAR means you are actually saving on compute resources since you're now getting direct distance measurements instead of inferring using ML.

Lidar only works to a certain subset of distances, so you need cameras anyway to infer long distances beyond the scope. Also, since distance alone isn’t sufficient, you need cameras anyway for inference of other things.

Lidar works up to 300 meters. That’s plenty for highway driving. Everyone using lidar also use multiple cameras so they are getting best of both worlds (using sensor fusion).

Everyone except Tesla uses LiDAR.

So any compute issues are not preventing adoption.

Compute for full self driving hasn’t been solved, I hope it is, and I don’t know which strategy will work. It’s entirely possible LiDaR makes it work better but requires more GPUs on vehicle :)