One approach is that the app server doesn’t possess the encryption key at all (or at least not the master key). Instead, it calls a remote service to decrypt each item as needed (or the item-level data key aka envelope encryption key).

In this way, an attacker can compromise the entire data set and the app server, but they still can't decrypt the data. They have to maintain ongoing access to a compromised app server in order to decrypt data items one by one, which (i) could take a long time for large data sets (ii) can be slowed down by rate-limiting (iii) runs the risk of being noticed (iv) if detected, attacker's access can be shut off immediately.

Additionally, you might manage and monitor your key management servers differently than your app servers; for example, you have the expectation that no one will ever log into key servers routinely, so any interactive access or unexpected running processes can generate alarms. The set of people who have access to key servers is different and much more limited than the set of people who have access to the app server. The key server can run in a different virtual network from the app server while providing extremely limited access to the app server (just e.g. TCP on the one port needed to provide this service).

This approach contrasts to schemes where the app server or data store has an encryption key. If the attacker compromises that, they can lift the entire data set and encryption key out of your systems and process it later -- and it's irretrievably gone. With the key server approach, stolen encrypted data provides no value on its own, and the attacker needs ongoing access to the key server to make sense of the data.