In November 2017 AWS released AWS Direct Connect Gateway, which is probably one of the biggest innovations within the AWS Direct Connect product in recent years. At the same time Direct Connect Gateway is often overlooked or not well understood. Therefore this blog post will dive deeper at the ins and outs of this extremely helpful AWS network capability.
AWS Direct Connect Gateway allows you establish connectivity that spans Virtual Private Clouds (VPCs) spread across multiple AWS Regions. Instead of establishing multiple BGP sessions for each VPC, you only need to establish a single BGP session with the Direct Connect Gateway per DX location. As the AWS Direct Connect Gateway is a global object, VPCs and DX locations in any location (except China) can be bridged.
The rules of IP routing still apply and you will only be able to reach VPCs from on-premises locations if the IP CIDRs don’t overlap.
This functionality of AWS Direct Connect Gateway allows you to accomplish the following very common use cases:
Before Direct Connect Gateway you were only able to reach the AWS Region associated with a DX location via a Private Virtual Interface (VIF). With this the AWS DX location at e.g. CoreSite NY1, New York, NY could only establish a Private VIF for usage in the US-East (N. Virginia) region, but not for the US-West (Oregon) region (See Figure 1).
If you required to access multiple AWS regions via Direct Connect over Private VIFs you had to work with AWS Direct Connect Partners to establish a presence in a DX location associated with the AWS Region of interest. Depending on your location and the location of the DX location this could become very expensive.
With AWS Direct Connect Gateway you can now access Virtual Private Clouds (VPC) via Virtual Private Gateways (VGW) in any AWS region (except China) from any Direct Connect location (except China), simplifying this use case dramatically (See Figure 2).
Direct Connect Gateway. " width="800" height="401" />
This dramatically improves flexibility and reduces cost, when connecting from on-premises to AWS regions. Looking at the AWS Direct Connect data transfer cost for AWS Regions in North America, you can see that the data transfer OUT price to any of the Direct Connect locations in North America is the same at currently $0.02. Therefore in this case picking a Direct Connect location within North America that is the closest to your on-premises location does not increase the AWS cost.
If you want to connect from on-premises to multiple Virtual Private Clouds (VPCs) within AWS, without Direct Connect Gateway you had to create a separate Private Virtual Interface on your DX connection for each of these VPCs. In addition you had to create a separate BGP peering over the different VLANs for each of these Private Virtual Interfaces between your physical router and the Virtual Private Gateway (VGW) in AWS.
As depicted in Figure 2, with Direct Connect Gateway there is only one Private Virtual Interface necessary to connect to multiple VPCs. As a result you now only need to establish a single BGP session between your on-premises router and the Direct Connect gateway. When attaching additional VPCs their CIDRs are propagated towards on-premises - with the Direct Connect Gateway’s ASN as the origin - towards on-premises.
This also simplifies the physical setup of your Direct Connect dramatically.
If you want to connect an AWS Transit Gateway to on-premises via AWS Direct Connect, you have to leverage AWS Direct Connect Gateway (See Figure 3). It is not possible to connect directly to a Direct Connect connection from a Transit Gateway.
Direct Connect Gateway with Transit Gateway. " width="851" height="291" />
AWS Direct Connect Gateway is a global AWS object which supports “Virtual Interface Attachments” on the on-premises facing side and “Gateway Associations” on the AWS facing side. These attachments and associations cannot be freely mixed and matched, but instead only allow two fixed combinations. Below is an overview of these two combinations and resulting attachments and associations.
AWS Direct Connect gateway supports various forms of automation via it’s API.
Similar to AWS Direct Connect itself, DX Gateway also support multi-account setups. This is especially important for larger customer that want to split ownership of the components across teams or units. Also customers of AWS GovCloud (US) benefit from this capability, as various components can be managed from a standard commercial account instead of the GovCloud account.
Looking at a standard deployment as depicted in Figure 4, it is possible - although not necessary - to split ownership of the three component types across different accounts.
To share ownership between DX Gateway and the DX Connection, the owner of the DX connection creates a new Virtual Interface and assigns it to the account that owns the DX Gateway. The account owning the DX Gateway can then accept this Virtual Interface and attach it to a DX Gateway.
Similarly if the account owning the AWS Virtual Private Gateway (VGW) or AWS Transit Gateway (TGW), as well as the account owning the Direct Connect gateway belong to the same AWS payer account ID, sharing is here possible as well. In that case the account owner of the AWS Virtual Private Gateway (VGW) or AWS Transit Gateway (TGW) initiates the association proposal, which has to be approved by the account owning the Direct Connect Gateway.
The following limits and restrictions apply to Direct Connect Gateway.
The number of AWS Direct Connect gateways and associated objects is limited within a single AWS account to the following values:
Direct Connect Gateway is geared towards data flows between AWS-facing Gateway associations and on-premises facing VIF attachments. This is depicted in Figure 5 as green paths. Data flow between associated Gateways connected to the same Direct Connect Gateway, depicted in Figure 5 as red path, is only possible if an aggregate route (e.g. 0.0.0.0/0) for the other gateway is announced over Direct Connect. In the depicted example announcing 172.16. 0.0/12 and 192.168. 0.0/16 from one of the on-premises locations would enable the traffic path shown in red. Data flow between multiple VIFs is only possible via the optional SiteLink capability. This is depicted in Figure 5 as orange path.
Direct Connect Gateway. " width="801" height="361" />
You are also limited by the number of BGP prefixes that you can announce from on-premises networks towards AWS, as well as from AWS towards on-premises (Figure 6).
Direct Connect Gateway. " />
Looking at the example in Figure, you also want to keep aggregates in mind. While you can e.g. announce 100 prefixes from each of the two corporate data center depicted, in the case of these prefixes being non-overlapping, not all 200 prefixes will make it into an associated VPC route table. That’s due to the VPC route table only being able to hold 100 BGP advertised (propagated) routes.
If you advertise more than 100 routes over the BGP session, the BGP session will go into an idle state.
Behind the scenes Direct Connect Gateway is implemented as a distributed BGP route reflector. Individual nodes of this BGP route reflector are distributed across multiple AWS Regions, which means that even in case of an entire AWS Region, the functionality of Direct Connect Gateway is not impaired. It’s also important to point out that a Route Reflector - and thereby Direct Connect Gateway - is not in the traffic path, but rather part of the control path. Therefore traffic does not actually flow through a Direct Connect Gateway. With this, it’s sometimes better suited to depict a Direct Connect Gateway as such a distributed route reflector outside the traffic path (Figure 7).
Direct Connect Gateway depicted as a distributed route reflector outside the traffic path. " width="559" height="317" />
This article provided deeper insights into AWS Direct Connect Gateway, covering use cases, the Direct Connect Gateway components, multi-account setup support, as well as limits and restrictions.
Updated: September 6, 2019
