AWS multi-region KMS keys and Data Lifecycle Manager: better together

Sometimes, a cloud vendor enables a giant leap forward in infrastructure management. I’m going to overview just such an advance in this post. AWS has introduced a new way to backup, copy and/or migrate encrypted AWS EC2 instances AMIs between regions.

There have been a series of long-standing challenges when using Amazon Machine Images (AMIs) for disaster recovery, application recovery and failover. It’s been hard to manage AMI creation on a schedule (you had to roll your own), make sure that old AMIs are deleted and, above all, it took an amazing amount of work to copy encrypted AMI snapshots across multiple regions.

Suddenly, these problems have melted away due to two (relatively) new functionality announcements from AWS. If you combine the newly announced Key Management Service (KMS) multi-region key support with the enhanced EC2 Data Lifecycle Manager (DLM) support for EBS-backed AMIs, your life will be better. Trust me.

Let’s take a quick look at both of these, starting with multi-region KMS keys.

KMS customer-managed keys (CMK) typically live in a specific AWS region. Now, when defining a new CMK you can specify that it can be replicated to one or more additional regions. All the usual attributes of a KMS key are available, for example IAM policies for administration and usage. But what’s really impressive is that when creating the multi-region key, you can create a key alias that will be the same in all the regions the key is replicated to. The first screenshot below shows where to create a multi-region key in the AWS console; the second shows the new mrk- resource ID for multi-region keys as well as its alias(es).

AWS create Key Management Service (KMS) multi-region customer managed key (CMK)
Create a KMS multi-region customer-managed key (click to enlarge)
A multi-region KMS key will have the same alias in all regions (click to enlarge)

A KMS multi-region customer managed key is almost magic. Consider all the things AWS KMS does for you with a multi-region CMK. You can:

  • Use the same alias in multiple regions to refer to the key. This means your CloudFormation templates don’t have to use different ARNs to refer to the key.
  • Rotate the key from the primary replica on a schedule and AWS will ensure that all the replicas have the new key before switching to the new key. AWS also keeps all previous versions of the key, in case you need them for playback decryption.
  • Add as many regions as you like.

Before multi-region KMS keys, if you needed to move anything encrypted with a CMK to another region you had to decrypt it in the sending region, transmit it unencrypted and re-encrypt the data in the new region with a new key. It’s obvious how a multi-region key makes this process far better than before.

Check out the “regionality” settings of the KMS multi-region CMK shown below. The first screenshot shows the primary key, in this case in us-east-1. The second screenshot shows the replicated key in us-west-1. Note that only the region name in the ARN differs.

The replica key has the same alias in all regions (click to enlarge)
AWS KMS CMK regionality settings
Same key, different region (click to enlarge)

You can combine multi-region keys with EC2 Data Lifecycle Manager’s support for scheduled AMI creation, management and deletion. This allows for easy, secure cross-region EC2 AMI replication with encrypted volume snapshots.

DLM’s functions include AMI scheduling which select EC2 instances based on tags, AMI management including deletion based on schedule parameters and — best of all — automatic replication to a secondary region.

The screenshots below tell the DLM story. They show a DLM schedule with options to replicate to a secondary region using a KMS multi-region key (the same one shown above) on a schedule that creates an AMI hourly and keeps only two AMIs in both the original region and the secondary region. The first screenshot shows the scheduling options; the other confirms that encrypted AMIs originally created in us-east-1 have been replicated to us-west-1.

An EC2 Data Lifecycle Manager AMI creation schedule showing the option to replicate to a secondary region using a KMS multi-region key (click to enlarge)
DLM replicated AMIs in secondary region (click to enlarge)

One more DLM tip: be sure to specify that DLM should reboot the EC2 instance when creating the AMI. The default is to not do this and that means your AMIs themselves may not be bootable. This is especially crucial for Windows Server EC2 instances since DLM does not use VSS to create volume snapshots.

In summary, DLM and multi-region KMS keys make creating and managing secure, multi-region full image backups of IaaS resources fast, reliable, easy and even a bit of fun.



, ,



2 responses to “AWS multi-region KMS keys and Data Lifecycle Manager: better together”

  1. Zak Avatar

    Hello Alex, very interesting, thank you for sharing!
    I am currently trying to use DLM to create snapshots for my EBS volumes and copy these snapshots in another account in another region.
    Do you know if doing that is possible with DLM, even if the EBS volumes in the source account are encrypted with the default KMS key ?
    In my source account, in the EBS snapshot policy I have enabled cross-region copy and enabled encryption with a customer managed key that has been shared with the target account. I have also enabled cross-account copy but unfortunately I cannot find my snapshots in my target account.

    1. Alex Neihaus Avatar
      Alex Neihaus

      Thanks for the feedback.

      I think you’ve answered your question: the default EBS KMS key is region-specific.

      I think you are going to have to re-create the volumes with the cross-region KMS key. I’d try shutting down the instance, detaching the volumes, create new equivalent volumes using the cross-region key, spinning up an AWS Linux instance, attaching the volumes and dd‘ing the volumes with the region-specific key to the empty cross-region keyed volumes.

      Then, you could try attaching those volumes back to the instance and see if you can create an AMI from it. Or, you could try packaging the new volumes into an AMI.

      Good luck.

Leave a Reply

Your email address will not be published. Required fields are marked *