One possible drawback of this kind of system is performance (or broker CPU) getting dragged down by crazy/bad filtering queries.
Normally, those issues are solved the usual way (monitor, identify, fix). It’s rarer to see systems that proactively detect/reject costly arbitrary queries when they’re issued, though.
Proactively detecting potentially bad SQL queries in RDBMSes relies on table statistics (can’t be known for streams) or query text/plan analysis heuristics (hairy, subjective/error prone).
But it just occurred to me: could RabbitMQ’s choice of Erlang enable the easy rejection of query plans above a certain cost?
Could the BEAM be easily made to reject a query plan (assuming the plan—or a worst-case version of it at least—can be compiled into a loopless/unrolled chunk of BEAM bytecode ahead of time) with a reduction count more than a user specified threshold?
That might be interesting, if possible. Most runtimes don’t have user-surfaced equivalents of reduction counts, so there might be some mechanical sympathy in RabbitMQ’s case.
Normally, those issues are solved the usual way (monitor, identify, fix). It’s rarer to see systems that proactively detect/reject costly arbitrary queries when they’re issued, though.
Proactively detecting potentially bad SQL queries in RDBMSes relies on table statistics (can’t be known for streams) or query text/plan analysis heuristics (hairy, subjective/error prone).
But it just occurred to me: could RabbitMQ’s choice of Erlang enable the easy rejection of query plans above a certain cost?
Could the BEAM be easily made to reject a query plan (assuming the plan—or a worst-case version of it at least—can be compiled into a loopless/unrolled chunk of BEAM bytecode ahead of time) with a reduction count more than a user specified threshold?
That might be interesting, if possible. Most runtimes don’t have user-surfaced equivalents of reduction counts, so there might be some mechanical sympathy in RabbitMQ’s case.