Yep. Not sure the details of AWS, but in GCP access to KMS APIs and specific keys is controlled by IAM, and you can set "conditions" on IAM policies to restrict access by things like IP of the request: https://cloud.google.com/iam/docs/conditions-overview

KMS is managed by IAM so you can attach roles or use STS (create an access key / password) that can assume a role that can encrypt.