0

AWS Direct Connect (DX) Direct Connect Gateway (DXGW) and AWS-TGW Design and Limitations

Disclaimer

AWS keeps changing the limits and design option from time to time. For most accurate and up to date information please consult with the AWS documentation and links provided in this article

Direct Connect (DX)

  • DX is region specific offering
    • It allows On-Prem physical locations to connect to a specific AWS region/location
  • DX supports max of 50 VIFs (including Private and Public) per physical connection
  • DX does not support Transit VIF for AWS-TGW connectivity

DXGW

  • What is DXGW?
  • Only supports Private and Transit VIFs
    • DXGW mainly used to access private resources in VPCs
  • Does not support public VIF
    • DXGW does not provide any benefit of Public Internet Connectivity
  • VGW associated with a DXGW must be “attached” to a VPC
  • Does not support transitive routing or transit connectivity
    • VPC in Region-1 cannot directly communicate with VPC in Region-2
    • DX Location-1 cannot directly communicate with DX Location-2
  • Up to 30 DX physical connections can connect to one single DXGW for physical link redundancy purposes
    • In another words 30 DX locations/regions
  • DX supports max of 50 VIFs (for DXGW only Private and Transit VIFs are applicable)
    • It means one can have Max of 50 DXGW per physical DX link
    • But one DXGW can connect to max of 10 VPCs
    • It means Max of 500 VPCs (50 x 10 VPC) per physical DX link across accounts and regions

 

 

DXGW is Must for AWS-TGW

  1. Transit VIF is a must when terminating DirecConnect (DX) circuit on AWS-TGW
  2. But Transit VIF can only be attached to a DXGW
  3. That means AWS-TGW mandates deploying DXGW

Max of 3 AWS-TGW Behind a Direct Connect Circuit

  1. Max of 3 AWS-TGW can be attached to one DXGW behind one Transit VIF
  2. And only one Transit VIF is possible per DirectConnect circuit
  3. Aviatrix Transit does not have this limitation because it uses Private VIF

Transit VIF and Private VIF are not allowed on same DXGW

A single DXGW cannot attach to both Private and Transit VIF

One cannot attach a DX-GW to an AWS-TGW when the DX-GW is already associated with an AWS-VGW or is attached to a Private VIF.

I did a simple test in my lab, and I get an error when I try to connect a Private VIF to DX-GW. This DX-GW had a Transit VIF attached to it.

 

Also confirmed this from following AWS doc

https://docs.aws.amazon.com/directconnect/latest/UserGuide/direct-connect-transit-gateways.html

 

DXGW with and without AWS-TGW Comparison

DXGW without AWS-TGW DXGW with AWS-TGW
10 VPCs per DXGW 3 TGWs per DXGW
50 DXGW max (b/c of 50 Private VIF) With Transit  VIF only one DXGW is possible
500 VPCs total 5,000 VPCs per TGW
15,000 VPC per DX physical link
Private VIF supported on all Direct Connect connection types Transit VIF supported only on dedicated or hosted connections of speed 1Gbps and above
No additional charges Additional charge for TGW data processing

DXGW with AWS-TGW Routing Limitations

* Only 20 routes from AWS to On-Prem per AWS-TGW

* Only 100 routes from on-prem to AWS

 

Reference:

Transit Gateway Reference Architectures for Many VPCs NET406-R1 PDF

Transit Gateway Reference Architectures for Many VPCs NET406-R1 VoD

Intra-Region AWS-TGW Peering is not Allowed

When multiple AWS Transit Gateways are required in the same region (separation between prod/dev air gap, separate NGFWs or other reasons), inter-region peering cannot be used to route traffic between VPCs attached to the AWS Transit Gateways.

Two AWS Transit Gateways can only be peered when they are in different regions.

 

 

Aviatrix Transit Solution

 

Summary

  • Transit VIF can only be attached to a DXGW
  • Only one Transit VIF for any AWS Direct Connect 1/2/5/10 Gbps connection
  • Less than 1G connections does not support Transit VIF
  • Max of 3 AWS-TGW can connect to one DXGW behind one Transit VIF
  • A single DXGW cannot attach with both Private and Transit VIF
    • This could be a serious limitation for some customers
    • I think the underline assumption is that if a customer is already using AWS-TGW then why would he want to use a private VIF attached to the same DXGW?
  • Aviatrix Transit Solution is not bound to these limits

AWS References

https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-limits.html
https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html

Reply Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
Like Follow
  • 1 yr agoLast active
  • 354Views
  • 1 Following