Over two and a half years ago, I wrote about SFTP on this blog saying, “Despite years of attempts to retire inter-organization file transfers based on SFTP, the protocol is alive and well and continues to be deeply, deeply embedded in enterprises’ workflows.”
Nothing has changed. And I doubt it soon will. SFTP is still a major protocol used by many enterprises to exchange crucial data among themselves.
Unfortunately, most of the them do it insecurely.
There’s more — a lot more — to securing SFTP beyond relying exclusively on ssh encryption.
And that’s the purpose of this blog post: to describe a deployment of SFTP using the AWS Transfer service that is arguably more secure than what one might achieve by creating an AWS SFTP server manually and then attempting to secure it after the fact. The example here — a reference architecture — integrates many different AWS resources together to make an AWS SFTP server more secure. The only effective way to do this is in code. So this post includes a complete, tested CloudFormation template that deploys the SFTP reference architecture in one deployment, including all required resources.
The template also makes the SFTP reference architecture repeatable. You need only change three parameters to make it work for you. Or, you can customize it to fit your needs and security requirements and still get a repeatable, secure AWS SFTP server every time. In fact, this template can be the basis for a secure, low-cost, cloud-scalable Managed File Transfer (MFT) system. You can create just the aspects of MFT you need without having to resort to MFT vendors, with their legacy, IaaS offerings or their transfer clouds which make you give them too much control over this core capability.
How to make SFTP servers secure
This CloudFormation template deploys some basic concepts from the reference architecture:
- SFTP should always run in a dedicated VPC. Sorry, I just don’t buy the counterargument that it should be part of your regular network. It shouldn’t attach to a transit gateway; it shouldn’t have routing to any other VPC. It should be a VPC all unto itself. The CloudFormation template creates a VPC with a single public subnet, an internet gateway, a security group permitting TCP 22 from anywhere and associates the proper routing and an EIP. Because of this latter exposure you would not want an SFTP server accessible from your normal VPCs, especially one with a publicly exposed IP address. So, a dedicated VPC is perfect.
- You need an externally-resolvable DNS name for the SFTP server. The template creates a Route 53 hosted zone and a CNAME record for the subdomain of that zone that points to the SFTP server. This makes it easy for the other end to connect to you by name instead of by IP. There’s no security advantage in using IP addresses alone, as so many SFTP servers do. It’s tempting to use IP addresses only so you can allow them in your firewall. If that makes you feel better, fine. But good security isn’t just about blocking traffic and endlessly administering changing IP addresss; it’s about having tight control of authentication. So…
- The SFTP server accepts only public key/private key authentication. This is the most secure way to manage an SFTP server, especially one you use to connect to partners, vendors, customers and suppliers. In this template, a single user named
sftpuseris created and a public key that has been pre-loaded into AWS Secrets Manager is assigned to it.sftpuserhas an IAM role assigned to it that scopes its access to a specific S3 key (“subdirectory”). You can add additional users and private keys to the template if you wish but my suggestion is that you create one entire deployment for each entity you connect with. The cost is low enough to support this and by keeping a one-server-per-connection environment, you can respond non-disruptively to changes and remove a connection by simply deleting the stack. - An encrypted S3 bucket is used for source and destination transfers. You could spend a lot of time getting the S3 encryption and access policies correct. But here, the CloudFormation template does it for you.
- SFTP activity is logged in CloudWatch. This may seem obvious but it’s not a default so the template sets it up.
The image below summarizes the resources created by the template and the relationships among the elements. You will see how this is a more secure SFTP environment, one which allows you to isolate SFTP into a VPC of its own, locks it down to a single connection partner and user but still allows properly authorized applications to access incoming and outgoing files via S3. You can also automate workflows based on the data using AWS Lambda to add true MFT capabilities, again both securely and at lower cost than MFT products.
How to run the example SFTP CloudFormation template
You can change any or all parameters and resources in the SFTP CloudFormation template. But only three are really needed to create a useable deployment. I’d recommend you change just these three parameters to suit your environment, run the template and see how it all fits together before customizing the template further.
Two of the three parameters describe the DNS settings. In the image below, you can see the Route 53 hosted zone (sftp.air11.com) from a deployment I created. In this hosted zone (which is a subdomain of air11.com), I created a host named refarch1. That host is CNAME’d to the default AWS Transfer for SFTP hostname. Note that for this to work, I had to add NS records to the air11.com root domain which is registered elsewhere. If your domain is registered in Route 53, you should still create a subdomain and create the proper NS records to point to the nameservers that Route 53 assigns to the hosted zone.
This screenshot shows the relationship between the parameters in the CloudFormation template and the SFTP public key that will be assigned to sftpuser from AWS Secrets Manager. Only the public key is required so AWS Transfer for SFTP can associate it with sftpuser. Note that AWS Secrets Manager is also a good place to store the private key as I did here. The template doesn’t use this but it’s a way to make sure the private key isn’t lost and helps support the one-server-per-partner-connection idea at the heart this this reference architecture.
Using Filezilla to transfer files to/from AWS SFTP
Here is an animated .gif showing how you would use a utility like Filezilla to connect to an AWS SFTP server created by the template. Note that when you enter the server information into Filezilla, be sure to use the sftp:// method indicator in the hostname field.
The example SFTP template
Below is the complete SFTP CloudFormation template. It’s in YAML. But if you prefer JSON, you can easily convert this to JSON using the AWS CloudFormation Designer. Note that the three parameters discussed above have generic names you will need to change.
One parting piece of advice: do not give in to the temptation to use password authentication. I could (and maybe one day will) write a blog post about why that’s a disaster for inter-enterprise communication. But I hope my many years of hard knocks in the field with SFTP will help convince you that password authentication is both dangerous and administratively impossible to control.
I am very interested in your feedback and hope this helps you deploy a secure SFTP environment in AWS. I appreciate any comments left below or feel free to tweet me at @yobyot.
---
AWSTemplateFormatVersion: "2010-09-09"
Description: >-
This sample AWS CloudFormation template creates an R53 hosted domain which points to a CNAME,
a VPC with a single public subnet with an associated IGW, a S3 bucket for the SFTP server to
use and and AWS SFTP transfer endpoint.
Portions of this template are derived from this GitHub Gist: https://gist.github.com/glnds/dac9fb18c3cad10ba42a203526b8caf2
Copyright 2021 Air11 Technology LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alex Neihaus 2021-08-23
Parameters:
VpcCidr:
Default: 10.10.0.0/16
Description: CIDR block for entire VPC.
Type: String
AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
SftpPublicSubnetCidr:
Default: 10.10.10.0/24
Description: CIDR block for the public subnet
Type: String
AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
SftpAllowedNetworks:
AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$
ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/0-28
Default: 0.0.0.0/0
Description: Networks allowed to ssh to instances in public subnet.
Type: String
SftpHostedZoneName:
Type: String
Default: sftp.yourdomain.com.
Description: The DNS name of the Route 53 hosted zone which will contain the SFTP Server. Override when launching the template to use a different DNS domain name
SftpServerDnsName:
Type: String
Default: refarch1
Description: The DNS host name of the SFTP server in SftpHostedZoneName. Override when launching the template to use a different DNS host name
SshKeyUserSecretName:
Type: String
Default: sftp-server-keypair
Description: This is the name of the Secrets Manager secret containing the public key for the SFTP Server. Override when launching the template to use a different key
SshUserPublicKeyJsonKey:
Type: String
Default: public-key
Description: This is the name of the Secrets Manager secret containing the public key for the SFTP Server. Override when launching the template to use a different key
S3BucketName:
Type: String
Default: ref-architecture-sftp-bucket
Description: The name of the S3 bucket to be created for use by the SFTP server. Override when launching the template to use a different bucket name
OwnerTagValue:
Type: String
Default: 'Alex Neihaus'
Description: String used in the Owner tag applied to many resources in this template
SomeOtherTag:
Type: String
Default: '123456'
Description: An example of adding another tag to many of the resources in this template
Resources:
# The next few resources create a VPC with a single public subnet, an IGW, an SG that permits only TCP 22 from 0.0.0.0/0, an associated route table and connects them all together.
Vpc:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
CidrBlock: !Ref 'VpcCidr'
Tags:
- Key: Name
Value: !Sub 'Vpc ${VpcCidr}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: !Sub 'IGW ${VpcCidr}'
- Key: CloudFormationStack
Value: !Ref 'AWS::StackId'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref 'Vpc'
CidrBlock: !Ref 'SftpPublicSubnetCidr'
Tags:
- Key: Name
Value: !Sub 'PublicSubnet ${SftpPublicSubnetCidr}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref 'Vpc'
InternetGatewayId: !Ref 'InternetGateway'
Eip:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
Tags:
- Key: Name
Value: !Sub 'Eip ${Vpc}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
PublicSubnetRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref 'Vpc'
Tags:
- Key: Name
Value: !Sub 'RouteTable ${Vpc}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
PublicRoute:
Type: AWS::EC2::Route
DependsOn: AttachGateway
Properties:
RouteTableId: !Ref 'PublicSubnetRouteTable'
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref 'InternetGateway'
PublicSubnetRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref 'PublicSubnet'
RouteTableId: !Ref 'PublicSubnetRouteTable'
PublicSubnetSg:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !Ref 'Vpc'
GroupDescription: Enable SSH access via port 22
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '22'
ToPort: '22'
CidrIp: '0.0.0.0/0'
Tags:
- Key: Name
Value: !Sub 'VpcSecurityGroup ${SftpPublicSubnetCidr}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
# The next two resources create a hosted R53 zone whose domain and host name are specified by parameters. The host record is a CNAME to the SFTP server name. If a subdomain is used (for example, sftp1.subdomain.example.com) make sure that example.com's records have NS records pointing to the nameservers R53 sets up for subdomain.example.com.
R53HostedZone:
Type: AWS::Route53::HostedZone
Properties:
Name: !Sub ${SftpHostedZoneName}
HostedZoneTags:
- Key: Name
Value: !Sub 'HostedZoneName ${SftpHostedZoneName}'
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
SftpServerDNSRecord:
Type: AWS::Route53::RecordSet
Properties:
Name: !Sub ${SftpServerDnsName}.${SftpHostedZoneName}
HostedZoneId: !Ref R53HostedZone
Type: CNAME
TTL: 300
ResourceRecords:
- !Sub ${SftpServer.ServerId}.server.transfer.${AWS::Region}.amazonaws.com
# The next resource creates an S3 bucket with the correct encryption (S3-SSE) to be used by the SFTP server.
SftpServerS3Bucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
Properties:
BucketName: !Ref S3BucketName
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: 'AES256' # This specifies SSE-S3 encryption
Tags:
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
# This resource creates an IAM role and associated policy to permit logging of SFTP server sends/receives/errors in CloudWatch. The logs are log group log-group:/aws/transfer
SftpServerLoggingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- transfer.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: ReferenceArchitectureSftpServerCloudWatchLogging
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:DescribeLogStreams
- logs:CreateLogGroup
- logs:PutLogEvents
Resource: 'arn:aws:logs:*:*:log-group:/aws/transfer/*'
# The following resource creates the SFTP server with a VPC endpoint, placing it in the VPC just created.
SftpServer:
Type: AWS::Transfer::Server
Properties:
LoggingRole: !GetAtt SftpServerLoggingRole.Arn
Domain: S3
EndpointType: VPC
EndpointDetails:
VpcId: !Ref Vpc
SubnetIds:
- !Ref PublicSubnet
SecurityGroupIds:
- !Ref PublicSubnetSg
AddressAllocationIds:
- !GetAtt Eip.AllocationId
Protocols:
- SFTP
Tags:
- Key: Name
Value: !Ref SftpServerDnsName
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
# This resource creates an IAM role and associated policy to "step down" (limit) access to specific key (subdirectories) in the S3 bucket. We only create one userid (sftpuser) but this policy can be applied to additional users who may be manually created.
SftpUserScopeDownRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- transfer.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: ReferenceArchitectureAllowS3ListingAndLocation
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- s3:ListAllMyBuckets
- s3:GetBucketLocation
Resource: "*"
- PolicyName: ReferenceArchitectureAllowListingOfUserFolder
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- s3:ListBucket
Resource: !GetAtt SftpServerS3Bucket.Arn
- PolicyName: ReferenceArchitectureAllowAccessToUserBuckets
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:DeleteObject
- s3:DeleteObjectVersion
Resource: !Sub "${SftpServerS3Bucket.Arn}/*"
# This resource creates a user (sftpuser) and associates it with the server and sets up authentication to use the public key retrieved from a Secrets Manger secret. DO NOT USE OR ALLOW THE PRIVATE KEY TO ESCAPE YOUR CONTROL.
SftpUser:
Type: AWS::Transfer::User
Properties:
ServerId: !GetAtt SftpServer.ServerId
UserName: sftpuser
HomeDirectory: !Sub "/${SftpServerS3Bucket}/home/sftpuser"
Policy: >
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReferenceArchitectureAllowS3ListHomeFolderSftpAdminUser",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::${transfer:HomeBucket}",
"Condition": {
"StringLike": {
"s3:prefix": [
"home/${transfer:UserName}/*",
"home/${transfer:UserName}"
]
}
}
},
{
"Sid": "ReferenceArchitectureAllowS3ObjectAccessInHomeFolderSftpAdminUser",
"Effect": "Allow",
"Resource": "arn:aws:s3:::${transfer:HomeDirectory}*",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectVersion",
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:GetObjectACL",
"s3:PutObjectACL"
]
}
]
}
Role: !GetAtt SftpUserScopeDownRole.Arn
# The user's public key is stored in an AWS Secrets Manager key, along with the private key (which must never be given out externally)
# A CloudFormation dynamic parameter retrieves the public key using the secret name and JSON key for the secret stored in the Secrets Manager resource
SshPublicKeys:
- !Join
- ''
- - '{{resolve:secretsmanager:'
- !Ref SshKeyUserSecretName
- ':SecretString:'
- !Ref SshUserPublicKeyJsonKey
- '}}'
Tags:
- Key: Owner
Value: !Ref OwnerTagValue
- Key: AppID
Value: !Ref SomeOtherTag
Outputs:
StackId:
Description: Stack ID of the CloudFormation Stack
Value: !Ref 'AWS::StackId'
VPCId:
Description: VPCId of the newly created VPC
Value: !Ref 'Vpc'
EipAddress:
Description: EIP allocated to NAT gateway
Value: !Ref 'Eip'
PublicSubnetId:
Description: SubnetId of the public subnet
Value: !Ref PublicSubnet
PublicSubnetRouteTableId:
Description: Public route table
Value: !Ref PublicSubnetRouteTable
R53HostedZoneId:
Description: Route 53 hosted zone ID
Value: !Ref R53HostedZone
SftpServerDNSRecord:
Description: SFTP DNS record
Value: !Ref SftpServerDNSRecord
SftpServerS3Bucket:
Description: S3 bucket used by the SFTP server
Value: !Ref SftpServerS3Bucket
SftpServerId:
Description: ID of SFTP server
Value: !Ref SftpServer
...
Leave a Reply to suresh Cancel reply