One of my clients has a site-to-site VPN in AWS that uses dynamic BGP. While BGP routes can be automatically propagated to an AWS VPC route table, if you are using VPC security groups you must also add the propagated routes to the appropriate SG.
Since the other end of the VPN connection can update advertised BGP routes at any time, a Lambda function launched on a schedule is an ideal way to automate keeping the security group rules for those routes in sync. I developed the PowerShell function below to do this. How it works and what it expects are detailed in the comments in the function code itself.
As you’ll see, most of the work in the function is accomplished with PowerShell’s Compare-Object cmdlet. Compare-Object is extremely elegant (this is my first brush with it). Like me, I think once you use it you’ll find many additional uses for it.
Compare-Object eliminates heavy lifting when comparing object collections and makes it incredibly easy to determine the presence or absence of one object in a different collection.
Once you get two object collections set up for the comparison — this is what most of the code in the Lambda function is doing — you can use a simple switch statement inside a foreach loop to process each comparison. The output of Compare-Object is itself a collection with two values: InputObject and SideIndicator, where the first is the value from the collection you want to test the contents of and the second is a text string indicating the result. The SideIndicator text string points “left”, “right” or “equal” to signal what Compare-Object found when comparing the input object with the comparison object.
Compare-Object at its simplest requires two inputs. The reference object is the collection you wish to compare with the difference object. In this example, the reference object contains all the CIDRs of the existing AWS security group plus additional CIDRs specified in the JSON passed to the function. Those additional CIDRs will never appear in the reference object but should be kept anyway.
The difference object contains CIDRs (propagated BGP routes) from a VPC route table associated with the BGP connection. The IncludeEqual parameter allows the comparison to return results that are in both the reference and difference collections.
As you can see in the function code, simple switch conditions test for routes that are in the difference object but not in the reference object. On that condition, the route is added to the reference object. That has the effect of updating the VPC security group with a newly advertised BGP route.
That’s a pretty simple test to code even without Compare-Object. But it’s much harder to determine in code when an object in the reference object (advertised routes) has been removed but still exists in the comparison object (the VPC security group). In this example, if the other side of the VPN connection stopped advertising a route it previously was advertising, it would appear in the reference collection but not in the difference collection. We need a way to clean up routes that once existed but are no longer advertised. And we don’t want to store all routes we’ve ever added somewhere, read them into an array and look up the CIDR to see if it was once advertised but now isn’t.
Here’s where Compare-Object‘s elegance really shines. If the function’s comparison returns that a CIDR exists in the reference collection but not in the comparison collection, it could be that a BGP route was, in fact, deleted and therefore removed from the route table by AWS VPC BGP propagation. Or, it could mean that this CIDR could not have come from the BGP route. A simple test determines which case this is and takes appropriate action. For the case in which a CIDR isn’t propagated but must be kept in the security group, the Lambda function’s input JSON allows for an array of CIDRs that should never be deleted — IOW, they will always appear in the reference object but never in the difference object.
In this way, Compare-Object enables us to never have to save any history of what routes may have existed in the past. We simply need to know if the route that is in the security group but not in the current list of advertised routes is one we want to keep. Using EventBridge to pass a “target” (a bad name for this, IMHO) to the Lambda function on launch makes it easy to list non-BGP-advertised routes that should remain in the security group.
This function runs nightly in the client’s AWS environment and, as you can see, sends output to an SNS topic. The Lambda function also has another SNS topic specified as a destination for both success and failure that, as you can also see in the code, writes the $AWSHistory.LastServiceResponse variable to the topic so subscribers can, if necessary debug what happened.
I’m sure there might be better ways of achieving a daily update of BGP routes from a propagated VPC route table into an AWS network security group. But this has worked for me…and I hope it helps you. And even if you don’t use this script for that purpose, it might be useful for you to explore the power of Compare-Object.
<#
This Lambda function:
- Expects a JSON input structure like the one below to be provided via $LambdaInput at invocation containing:
- the CasE SEnSITIve Name tag of the security group containing subnets that may or may not be BGP routes
- the cASe SENSItive Name tag of the route table containing propagated routes from a dynamic BGP site-to-site AWS VPN
- a list of CIDRs that should ALWAYS be in the security group. These are likely to be the local VPC.
- an SNS topic that is used to send a completion email. This SNS topic is separate from the SNS topic used as the destination for the Lambda function and which is specified in the AWS console settings for the function. This allows for the detailed output of $AWSHistory objects to be sent to a separate set of subscribers from the function's Write-Host output (which is collected into an array in the snsWriteLine function below).
The function uses this input to:
- Create an array of all existing rules' CIDRs in the target security group
- Combine those rules with the always-retain rules CIDRs
- Create a collection of route CIDRs from the specified route table that were placed there by route propagation (Origin -eq 'EnableVgwRoutePropagation')
- Uses Compare-Object with the left side containing existing and must-keep CIDRs and the right side containing the route table CIDRs
- Based on the matching, adds or deletes the CIDR from the security group.
THE ONLY CIDRs THAT ARE ADDED OR DELETED ARE FOR TCP 443. IOW, if a rule has a different protocol/port combo, this function will NOT disturb it.
Each call to an AWS cmdlet results in an $AWSHistory object being added to an array. This array is then passed on the pipeline at the end of the function. This is designed to be used with an SNS destination specified in the Lambda function's properties to email the (longish) output to someone who might need it for debugging.
A separate SNS topic can be specfied in $LambdaInput, as shown in the sample below, in order to send a completion message or other info to a separate set of subscribers.
$LambdaInput = @"
{
"sgNameTag": "ApplicationsSecurityGroup",
"routeTableNameTag": "ApplicationsRouteTable",
"alwaysRetainCidrs": [
"10.41.76.96/27",
"10.41.76.64/27",
"10.41.76.0/22"
],
"snsTopicArn": "arn:aws-us-gov:sns:us-east-1:1234576890AB:SnsLambdaNotify"
}
"@
(c) 2022 Air11 Technology LLC -- licensed under the Apache OpenSource 2.0 license, https://opensource.org/licenses/Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Author's blog: https://yobyot.com
an 2022-07-27
#>
#region Required modules
#Requires -Modules @{ModuleName='AWS.Tools.Common';ModuleVersion='4.1.90'}
#Requires -Modules @{ModuleName='AWS.Tools.EC2';ModuleVersion='4.1.90'}
#Requires -Modules @{ModuleName='AWS.Tools.SimpleNotificationService';ModuleVersion='4.1.90'}
#endregion Required modules
#region Define Script scope variables
$responses = @() # Define array to hold $AWSHistory responses
$snsOutput = @() # Define array to hold line output for SNS message
#endregion Define Script scope variables
#region Log invocation objects
Write-Host (ConvertTo-Json -InputObject $LambdaInput -Compress -Depth 99)
Write-Host (ConvertTo-Json -InputObject $LambdaContext -Compress -Depth 99)
#endregion Log invocation objects
#region Function to collect $AWSHistory responses in an array that should be sent to the pipeline at Lambda function end
function awsResponses
{
param (
$r
)
$obj = New-Object -TypeName psobject
$obj = $r
return $obj
}
#endregion
#region Function to create an array of output lines to be written to SNS
function snsWriteLine
{
param (
$l
)
$obj = New-Object -TypeName psobject
$obj = $l
return $obj
}
#endregion
#region Function to update security group based on presence/absence in the comparison of propagated routes to a security group
function Update-SecurityGroup
{
param (
$action,
$cidr
)
switch ($action)
{
"Add"
{
$IpRange = New-Object -TypeName Amazon.EC2.Model.IpRange
$IpRange.CidrIp = $cidr
$IpRange.Description = "Rule added $(Get-Date -f yyyy-MM-dd@HH:mm:ss) by Lambda function"
$rule = New-Object -TypeName Amazon.EC2.Model.IpPermission
$rule.IpProtocol = 'tcp'
$rule.FromPort = '443'
$rule.ToPort = '443'
$rule.Ipv4Ranges = $IpRange
Grant-EC2SecurityGroupIngress -GroupId $sg.GroupId -IpPermission @($rule)
}
"Remove"
{
$rule = New-Object -TypeName Amazon.EC2.Model.IpPermission
$rule.IpProtocol = 'tcp'
$rule.FromPort = '443'
$rule.ToPort = '443'
$rule.IpRanges.Add($cidr)
Revoke-EC2SecurityGroupIngress -GroupId $sg.GroupId -IpPermission @($rule)
}
}
}
#endregion
$sg = Get-EC2SecurityGroup -Filter @{name = 'tag:Name'; values = $LambdaInput.sgNameTag }
$responses += awsResponses $AWSHistory.LastCommand # Write $AWSHistory.LastCommand object to array
[array]$sgExistingRules = (
Get-EC2SecurityGroupRule -Filter @{name = 'group-id'; values = $($sg.GroupId) } | `
Where-Object {
!($_.IsEgress) -and ($_.CidrIpv4) -and ($_.FromPort -eq '443') -and ($_.ToPort -eq '443') # Eliminate egress rules AND rules that do not have a valid IPV4 CIDR in them (for example, rules that specify other security groups) AND rules that are NOT TLS (TCP 443). Then, just keep the CIDRs. These CIDRs will be the part of the "left" side of the upcoming comparison
}
).CidrIpv4 # Just keep the CIDRs from the existing rules
$responses += awsResponses $AWSHistory.LastCommand # Write $AWSHistory.LastCommand object to array
$allSgRules = ($sgExistingRules + $LambdaInput.alwaysRetainCidrs) | Select-Object -Unique # Combine the CIDRs we want to keep at all times with existing CIDRs in the SG and de-dup the resulting array
$routes = Get-EC2RouteTable -Filter @{name = 'tag:Name'; values = $LambdaInput.routeTableNameTag } | `
Select-Object -ExpandProperty Routes | `
Where-Object Origin -eq 'EnableVgwRoutePropagation' # Find the routes in the Route Table that were propagated by the VGW into the specified route table eliminating routes that may have been added manually. IOW, do NOT include routes for the VPC and/or things like NAT gateways that were added by something other than route propagation
$responses += awsResponses $AWSHistory.LastCommand # Write $AWSHistory.LastCommand object to array
[array]$rtbCidrs = $routes | Select-Object -ExpandProperty DestinationCidrBlock # Create the "right" side of the upcoming comparison
$comparedObjects = Compare-Object -ReferenceObject $allSgRules -DifferenceObject $rtbCidrs -IncludeEqual # Compare the merged SG CIDRs with the CIDRs from the route table
ForEach ($comparison in $comparedObjects)
{
switch ($comparison.SideIndicator) # Left side is array of existing and must-keep CIDRs; right side is VGW-propagated route table CIDRs
{
'==' # The CIDR is in both lists and therefore should be kept
{
Write-Host "$($comparison.InputObject) found in both routes and rules and will be kept" # This message ends up in CloudWatch
$snsOutput += snsWriteLine "$($comparison.InputObject) found in both routes and rules and will be kept`n" # This ends up in an array to be sent to the specified SNS topic
}
'=>' # The CIDR in the VGW list is NOT in the security group rules and must be added.
{
Write-Host "$($comparison.InputObject) will be added to security group"
$snsOutput += snsWriteLine "$($comparison.InputObject) will be added to security group`n"
Update-SecurityGroup 'Add' $($comparison.InputObject)
$responses += awsResponses $AWSHistory.LastCommand # Write $AWSHistory.LastCommand object to array
}
'<=' # The CIDR was not in the list of VGW routes but WAS in the SG rules. This could mean it's either a CIDR that must be retained OR it could be that the route has been deleted in the VGW list of routes.
{
If ($comparison.InputObject -in $LambdaInput.alwaysRetainCidrs) # Is this CIDR one we want to keep because it may be a local CIDR and/or was manually added?
{
Write-Host "The rule allowing TLS inbound from $($comparison.InputObject) will be kept because it is in the always retain list"
$snsOutput += snsWriteLine "The rule allowing TLS inbound from $($comparison.InputObject) will be kept because it is in the always retain list`n"
}
else # No, it's not an always retain CIDR and it's not in the VGW propagated routes -- so, it should be deleted because it's likely to be a deleted BGP route
{
Write-Host "The rule allowing TLS inbound from $($comparison.InputObject) will be deleted because it is not a propagated route"
$snsOutput += snsWriteLine "The rule allowing TLS inbound from $($comparison.InputObject) will be deleted because it is not a propagated route`n"
Update-SecurityGroup 'Remove' $($comparison.InputObject)
$responses += awsResponses $AWSHistory.LastCommand # Write $AWSHistory.LastCommand object to array
}
}
}
}
If ($($LambdaInput.snsTopicArn)) # If the EventBridge rule specified snsTopicArn, use it to write a message.
{
Publish-SNSMessage -TopicArn $($LambdaInput.snsTopicArn) -Subject "AtosUpdateBgpRoutesInSecurityGroup Lambda function" -Message "Merging propagated routes from route table $($LambdaInput.routeTableNameTag) into security group $($LambdaInput.sgNameTag)`n`n$($snsOutput)`nLambda function AtosUpdateBgpRoutesInSecurityGroup completed on $(Get-Date -Format "yyyy-MM-dd_HH-mm-ss")"
$responses += awsResponses $AWSHistory.LastCommand
}
Write-Host (ConvertTo-Json -InputObject $responses -Compress -Depth 99) # Write the accumulated $AWSHistory.LastCommand objects to the CloudWatch log
(ConvertTo-Json -InputObject $responses -Compress -Depth 99) # The last item in the pipeline when a Lambda function ends is sent to the destination specified in the Lambda configuration. In this case, it's the outputs from $AWSHistory.LastCommand. This _should_ cause a statusCode 200 in responseContext and the contents should be visible in requestPayload, if the destination is set correctly for the function and it was invoked asynchronously via an EventBridge rule or via Invoke-LMFunction ... -InvocationType Event, this will be written to the SNS topic specified in the destination
Leave a Reply