My thinking is since it's postgres wire compatible the existing tests should work?

Jepsen does much more than basic end-to-end tests, including intentionally partitioning the cluster. Tests written for a non-distributed system are downright friendly compared to what Jepsen does to distributed systems.

Does Jepsen just run automated tests?

I mean they're automated but also hand crafted.

https://github.com/jepsen-io/jepsen

Those are very expensive and booked months in advance. Aphyr sometimes comments on HN and is of course a better source on this but that's what I recall reading.

It’s the price you pay to reassure customers that your newfangled distributed database won’t experience data loss or inconsistency.