Hacker Newsnew | past | comments | ask | show | jobs | submit | mdcallag's commentslogin

Author here. The interesting point was that RSS is much too large with glibc malloc + RocksDB/MyRocks. The performance differences aren't a big deal, but OOM because RSS is 2X or 3X larger than it should be is a big deal. Of course, that problem is workload specific, but RocksDB/MyRocks are a workload on which I focus.

I lurk on email lists for RocksDB and OOM from glibc malloc is a problem for users -- the solution is easy, use jemalloc or tcmalloc.


Blog author here.

I use the latest LTS release of Ubuntu, which is Ubuntu 22.04, and the library versions are what Ubuntu provides. So these are the libraries that most of us use when using Ubuntu LTS. Regardless, I am happy that tcmalloc and jemalloc continue to improve.

Occasionally I get "your version of X is too old" when I publish results for X, and I have had mixed results in the past when doing even more work to test the newer thing. In this case I am not going to put in that extra time. But in a few years I might revisit the topic and get newer versions of X, of course, there will be feedback that the version of X I test in 2028 is from 2026 and the story repeats.


Ubuntu LTS releases come out every two years, so the latest one is 24.04.


Yeah, I missed that. I have ~10 servers in my test pool, all are still on 22.04. Perhaps I upgrade them later this year.


But the thing that doesn't get much discussion in public is the drama that happens from upgrades when behavior changes WRT file systems, kernels, device drivers, CPU frequency governors, etc.

It took a few weeks to figure that out for several of my servers. So I am slow to upgrade things once that gets stable -- and it has been stable for my servers for about 6 months.


Thanks for doing the sensitivity analysis. I struggle with that issue a lot -- how can I shorten my benchmark duration (use less data, run for less time) so I can get more work done.


README.md was interesting to read. I enjoyed learning about the history of the effort and I appreciate how much work got it to this point.

I enjoy both benchmarking and benchmarketing. Although benchmarketing is more fun when it isn't about the DBMS I am working on.


You did an awesome job on the 5.0 to 5.1 migration, especially in chasing down at least one obscure problem in InnoDB.


How many nines will managed RonDB provide when running in public clouds?


For the moment managed RonDB will provide the availability of the cloud, but remember that the managed version is still in development. The steps to go 6 9s requires 1) Integrate cloud APIs such that we know when the cloud provider will freeze the instances 2) Provide global replication between cloud regions and failover handling of this. As mentioned reaching 6 9s requires both RonDB SW that is capable of reaching 6 9s as well as operational competence to actually deliver it. This is what we're aiming at, to make this availability reachable for normal users without this operational competence to deliver 6 9s.


This post inpsired a lot of great discussion about the Spanner family of distributed databases. Someone asked about TiDB there, but there was no follow up. Can someone answer that question?

http://dbmsmusings.blogspot.com/2018/09/newsql-database-syst...


I'll pass this onto the engineering team. Thanks!

For a quick answer, I believe Daniel Abadi's followup comment addresses it:

> This post did not discuss a central oracle. It discusses the difference between partitioned vs global consensus.

In TiDB the pd leader is a central oracle.


There are many options but most don't have to be set. We need to improve the tuning experience. See http://smalldatum.blogspot.com/2018/09/5-things-to-set-when-...


MyRocks isn't 5.6 only. It is in Percona 5.7 and MariaDB 10.3

I agree that tuning is too complex and we should do much better there. This explains where to ask for advice - http://smalldatum.blogspot.com/2018/05/where-to-ask-question...


jemalloc and tcmalloc have been much better than glibc for me, especially when it comes to avoiding fragmentation with some server workloads http://smalldatum.blogspot.com/2017/11/concurrent-large-allo... http://smalldatum.blogspot.com/2018/04/myrocks-malloc-and-fr... http://smalldatum.blogspot.com/2015/10/myrocks-versus-alloca...


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: