(Hard to draw many meaningful conclusions from a single, extremely simple query without much explanation?)
Graph shows PostgreSQL as taking a long time, but doesn't say anything about configuration or parallelization. PostgreSQL should be able to parallelize that type of query since 9.6+, but I think they didn't use parallelization in these experiments with PostgreSQL, even though they used a bunch of parallel threads with QuestDB?
So would be good to know:
- What version of Postgres
- How many parallel workers for this query
- If employing JIT'ing the query
- If pre-warming the cache in PostgreSQL and configuring it to store fully in memory (as benchmarks with QuestDB appeared to do a two-pass to first mmap into memory, and only accounting for the second pass over in-memory data).
etc
Database benchmarking is pretty complex (and easy to bias), and most queries do not look like this toy one.
I agree that our blog post lacks of details, here are some:
- PostgreSQL 12
- 12
- No
- We ran the test using the pg_prewarm [0] module, the difference was negligible
Regarding the "toy" query, the reason we are showcasing this instead of other more complex queries is because this is a simple, easily reproducible benchmark. It provides a point of reference for performance figures.
> Database benchmarking is pretty complex (and easy to bias), and most queries do not look like this toy one.
I would say that benchmarking is very hard. We tried not to perform a biased benchmark by running something that is not time-series specific and which does not put us in advantage compared to what Postgres should do.
The takeaway from this is that configuration is important and we should expose it. The next benchmark we do will have an associated repository so people can review our config and point non optimal items if any.
Graph shows PostgreSQL as taking a long time, but doesn't say anything about configuration or parallelization. PostgreSQL should be able to parallelize that type of query since 9.6+, but I think they didn't use parallelization in these experiments with PostgreSQL, even though they used a bunch of parallel threads with QuestDB?
So would be good to know:
- What version of Postgres
- How many parallel workers for this query
- If employing JIT'ing the query
- If pre-warming the cache in PostgreSQL and configuring it to store fully in memory (as benchmarks with QuestDB appeared to do a two-pass to first mmap into memory, and only accounting for the second pass over in-memory data).
etc
Database benchmarking is pretty complex (and easy to bias), and most queries do not look like this toy one.