It gives you a type/interface level guarantee.

e.g. you know

    public Optional<File> loadFile(String filename);

can fail to return a file, while

    public Int sum(Int a, Int b);

can't fail to return an Int.

Obviously, because nulls exist in java, this guarantee isn't provable on compilation the way it is in e.g. Scala, but if your entire team writes code this way, at least you can rely on that for internal libraries. Like generics, this is one of those ideas that's ok in java 1.X, but doesn't shine as much until a backwards incompatible java 2.X

Nullable types provide Optional already. I find the use of a custom wrapper type to be a massive wart, and Optional is a common enough thing that it's worth adding a bit of syntax for it.

Which is why I like Kotlin's approach. Same guarantees, one letter instead of ten. If you're writing Java, though, I can see why Optional has some use.

Nulls exist in Scala. You can return null for an Option[T].

Can't in Kotlin, though.

When interoperating with other libraries on the JVM you very easily can, which is a very common use case.

You're right. Kotlin provides the null-or-throw operator (!!) to convert a Derp? into a Derp, though.

Often when you're dealing with a value that may be null, you might want to do a series of transformations to it that may change its value or type, and may even return null themselves. Optional lets you chain these actions together without concerning yourself at every single step whether or not the value is there until the very end.

Let's take this back to map/filter on lists. Remember the bad old days when you didn't have map and filter on lists? If you had some code that iterated heavily and then you needed to add another step you often ended up refactoring a lot of manual for code that explicitly handled iteration logic. Now with streams you can just chain another call onto the same stream and as long as the types match up you're ready to go test it. Optional gives you the same thing, but instead of working with lists it handles values that might be missing.

Because null is overloaded with meaning - it could be an IO error leading to an empty result, or IO could have produced an actually empty result, or ...

Yes, you could argue that you will only use null for an actually empty result. But two things: 1) You won't keep to it because the compiler won't force you to do so, and 2) potential contributors will not comply. When using optionals/either/enums, compiler will not allow a wildcard result (such as null).

.empty is just more semantically specific, and if you read a method's API for the first time, semantics helps you grasp it a lot faster. That's the reason I'm using enums in Swift all over the place now. Compared to the nilling out of Objective-C, I can be way more confident that method results really are what they claim they are.

The biggest reason- I very rarely need to check if my optional is empty or not. I can write optional-chaining code like in the article above and it implicitly handles both cases.

The code I've written using optionals instead is much shorter, much clearer in its intent, and generally has fewer bugs.

That's anecdotal experience, not proofs. All I can really recommend is that you try it with an open mind.