The haters will hate, but tap guides are great (e.g. https://biggatortools.com/v-tapguide-faqs, but even a block of hard wood with a clearance hole drilled in it works fine).
Unless you're tapping something super tough (306?), Amazon taps are fine for hand tapping. Go in straight, use a good lubricant.
I've got two of those tap guides, one for US and one for metric. They're great. Also for drilling since I don't have a drill press at home.
I've examined cheap taps under a microscope. Maybe they are of varying quality, but the ones I got had burrs all along the cutting edges. A tap that I borrowed from a machine shop was flawless in comparison. So maybe the middle ground is caveat emptor.
Another trick for tapping is to use something pointy in the drill chuck to center the tap after drilling, assuming you've clamped down your workpiece in a drill press or mill. This works for really big taps when you don't have a guide for them. Likewise the tailstock of a lathe can be used for this purpose.
I don't think either is a bad choice, but Aurora has some advantages if you're not a DB expert. Starting with Aurora Serverless:
- Aurora storage scales with your needs, meaning that you don't need to worry about running out of space as your data grows.
- Aurora will auto-scale CPU and memory based on the needs of your application, within the bounds you set. It does this without any downtime, or even dropping connections. You don't have to worry about choosing the right CPU and memory up-front, and for most applications you can simply adjust your limits as you go. This is great for applications that are growing over time, or for applications with daily or weekly cycles of usage.
The other Aurora option is Aurora DSQL. The advantages of picking DSQL are:
- A generous free tier to get you going with development.
- Scale-to-zero and scale-up, on storage, CPU, and memory. If you aren't sending any traffic to your database it costs you nothing (except storage), and you can scale up to millions of transactions per second with no changes.
- No infrastructure to configure or manage, no updates, no thinking about replicas, etc. You don't have to understand CPU or memory ratios, think about software versions, think about primaries and secondaries, or any of that stuff. High availability, scaling of reads and writes, patching, etc is all built-in.
I spoke about this exact thing at a conference (HPTS’19) a while back. This can work, but introduces modal behaviors into systems that make reasoning about availability very difficult and tends to cause meta stable behaviors and long outages.
The feedback loop is replicas slow -> traffic increases to primary -> primary slows -> replicas slow, etc. The only way out of this loop is to shed traffic.
Practically, the difference in availability for typical internet connected application is very small. Partitions do happen, but in most cases its possible to route user traffic around them, given the paths that traffic tends to take into large-scale data center clusters (redundant, typically not the same paths as the cross-DC traffic). The remaining cases do exist, but are exceedingly rare in practice.
Note that I’m not saying that partitions don’t happen. They do! But in typical internet connected applications the cases where a significant proportion of clients is partitioned into the same partition as a minority of the database (i.e. the cases where AP actually improves availability) are very rare in practice.
For client devices and IoT, partitions off from the main internet are rare, and there local copies of data are a necessity.
Yes, you can do stuff like that. You might enjoy the CRAQ paper by Terrace et al, which does something similar to what you are saying (in a very different setting, chain replication rather than DBs).
(Op here) No deadlocks needed! There’s nothing about providing strong consistency (or even strong isolation) that requires deadlocks to be a thing. DSQL, for example, doesn’t have them*.
Event sourcing architectures can be great, but they also tend to be fairly complex (a lot of moving parts). The bigger practical problem is that they make it quite hard to offer clients ‘outside the architecture’ meaningful read-time guarantees stronger than a consistent prefix. That makes clients’ lives hard for the reasons I argue in the blog post.
I really like event-based architectures for things like observability, metering, reporting, and so on where clients can be very tolerant to seeing bounded stale data. For control planes, website backends, etc, I think strongly consistent DB architectures tend to be both simpler and offer a better customer experience.
* Ok, there’s one edge case in the cross-shard commit protocol where two committers can deadlock, which needs to be resolved by aborting one of them (the moral equivalent of WAIT-DIE). This never happens with single-shard transactions, and can’t be triggered by any SQL patterns.
> There’s nothing about providing strong consistency (or even strong isolation) that requires deadlocks to be a thing. DSQL, for example, doesn’t have them*.
If you want to have the kind of consistency people expect (transactional) in this kind of environment, they're unavoidable, right? I see you have optimistic concurrency control, which, sure, but that then means read-modify-write won't work the way people expect (the initial read may be a phantom read if their transaction gets retried), and fundamentally there's no good option here, only different kinds of bad option.
> Event sourcing architectures can be great, but they also tend to be fairly complex (a lot of moving parts).
Disagree. I would say event sourcing architectures are a lot simpler than consistent architectures; indeed most consistent systems are built on top of something that looks rather like an event based architecture underneath (e.g. that's presumably how your optimistic concurrency control works).
> The bigger practical problem is that they make it quite hard to offer clients ‘outside the architecture’ meaningful read-time guarantees stronger than a consistent prefix. That makes clients’ lives hard for the reasons I argue in the blog post.
You can give them a consistent snapshot quite easily. What you can't give them is the ability to do in-place modification while maintaining consistency.
It makes clients' lives hard if they want to do the wrong thing. But the solution to that is to not let them do that thing! Yes, read-modify-write won't work well in an event-based architecture, but read-modify-write never works well in any architecture. You can paper over the cracks at an ever-increasing cost in performance and complexity and suppress most of the edge cases (but you'll never get rid of them entirely), or you can commit to not doing it at the design stage.
You can have a system that puts out heartbeat timestamps and treat that as the single root upstream for all your inputs, or you can allow each source to timestamp its inputs and make a rule about how far your clocks are allowed to drift (which can be quite generous), or you can give every source an id and treat your timestamps as vectors and have a rule that any "combine events from source x and source y" must happen in a single canonical place.
The point of that section, which maybe isn’t obvious enough, is to reflect on how eventually-consistent read replicas limit the options of the database system builder (rather than the application builder). If I’m building the transaction layer of a database, I want to have a bunch of options for where to send me reads, so I don’t have the send the whole read part of every RMW workloads to the single leader.
(OP here). I don’t love leaking this kind of thing through the API. I think that, for most client/server shaped systems at least, we can offer guarantees like linearizability to all clients with few hard real-world trade-offs. That does require a very careful approach to designing the database, and especially to read scale-out (as you say) but it’s real and doable.
By pushing things like read-scale-out into the core database, and away from replicas and caches, we get to have stronger client and application guarantees with less architectural complexity. A great combination.
That’s a fair point. To be fair to the academic definitions, “eventually consistent” is a quiescent state in most definitions, and there are more specific ones (like “bounded staleness”, or “monotonic prefix”) that are meaningful to clients of the system.
But I agree with you in general - the dynamic nature of systems means, in my mind, that you need to use client-side guarantees, rather than state guarantees, to reason about this stuff in general. State guarantees are nicer to prove and work with formally (see Adya, for example) while client side guarantees are trickier and feel less fulfilling formally (see Crooks et al “Seeing is Believing”, or Herlihy and Wing).
I have no beef with the academic, careful definitions, although I dislike the practice where academics redefine colloquial terms more formally. That actually causes more, not less confusion. I was talking about the colloquial use of the term.
If I search for "eventual consistency", the AI tells me that one of the cons for using eventual consistency is: "Temporary inconsistencies: Clients may read stale or out-of-date data until synchronization is complete."
I see time and time again in actual companies that have "modern" business systems based on microservices that developers can state the same idea but have never actually paused to think that you something is needed to do the "synchronization". Then they build web UIs that just ignore the fact, causing application to become flaky.
Inspired me to write this blog post: https://brooker.co.za/blog/2023/04/20/hobbies.html
reply