How do I implement a cost-effective centralized L4 Firewall for inspecting network traffic across VPCs in AWS, Azure and GCP?

  • 4 April 2020
  • 0 replies

Userlevel 3
Badge +5

Problem Statement

All the major cloud service providers (AWS, Azure, GCP) natively provide layer-4 firewalls functioanlity. These firewalls are applied to the virtual nic of each instance/virtual machine. The rules are usually also limited to a single VPC/VNet. This means when you need to apply this firewall rule you have to create it for each instance and manage it individually. The distributed nature has its challenges especially at scale when you need to troubleshoot and apply compliance and governance.

Lets take few minutes to understand these in each cloud provider:

AWS Native L4 Firewall Implementation

AWS offers three native ways to implement security policies across VPCs:

  1. Network ACLs,

  2. Security groups

  3. AWS Network Firewall

Both of these have limitations that can get in the way of cloud adoption.

AWS Native L4 Firewall Limitations

We will not cover these limitations in detail here. But, on a high level:

  1. AWS NACLs (Network ACLs) are not stateful which makes it extremely challenging in mid-to-large-scale environments. This means you need 2 rules (inbound and outbound) for each traffic flow. There is also a limit of 20 rules per NACL.

  2. AWS Security Groups are tied to individual instances and have a limit of 50 rules.

  3. AWS Network Firewall offers Distributed and Centralized models

    1. Distributed Model:

      1. No load balancing across Availability Zones (GWLB Endpoint only load balances within AZ)

      2. Very expensive (~$0.40 per hour per AZ plus data plus Firewall Manager)

      3. For internet egress traffic, You must NAT traffic BEFORE it hits the FW hence there is no Source IP visibility in logs nor you can apply rules based on Source IP

      4. There is no visibility. At the time of this update (Nov 25, 2020), the Logging feature is not supported hence there is zero visibility

      5. Each Firewall is account based so if you have several accounts, each account must manage its own firewall rules

      6. 5-tuple FW rules are not tied to Domain rules. Meaning, if you configure an HTTP at L4 (5-tuple rules) and then want to do FQDN filtering, all traffic will be allowed and not be checked by FQDN

      7. There is no discovery of URLs (or an observation mode), you need to exactly know all the FQDNs to allow

      8. Customer owns planning subnets, managing and configuring route tables. At the minimum you need to manage workload subnet route table, NAT subnet Route table, Firewall Subnet Route table, IGW Edge route table. This is uniquely needed per VPC.

      9. No intra-vpc inspection thru the AWS Network Firewall.

Following picture shows some of the per VPC customer owned route table management


2. Centralized Model

AWS Network Firewall also offers a centralized model. Without going in the details of the firewall functionality itself, the orchestration and plumbing required is extremely complicated. As with TGW, there is no visibility at all on where the traffic is going, which route table it is hitting, which attachment is it going thru. 

At the minimum, customer needs to manage following pieces manually and with no visibility it gets very complicated, very quickly.

  1. Spoke VPC Route table

  2. TGW attachment

  3. TGW route propagation

  4. TGW route table and association with right attachments

  5. Inspection VPC where FW is deployed

  6. Require 2 subnets per NFW (one for TGW Ingress and another for FW)

  7. TGW Ingress subnets will require unique configuration

  8. Firewall subnet route table

  9. Inspection VPC attachment

  10. Inspection VPC TGW route table association

  11. CLI configuration on TGW to avoid asymmetric traffic to Network Firewall

  12. Egress VPC

  13. Multiple Subnets in Egress VPC (one for TGW Ingress and another for NAT)

  14. NAT Gateway and its subnet

  15. Unique Route tables for ingress subnet and NAT GW Subnet

  16. Manual route entries in TGW route table to support Egress routes

  17. Ingress VPC

  18. NLB

  19. NLB Subnet and its route table

  20. Ingress subnet TGW attachment, association, propagation and route table

The following diagram does a good job at summarizing all the things customers have to manage:


Because of these limitations, AWS customers commonly use security groups to firewall within a VPC. As NFW is very new, I will update this space as we learn more.

Azure NSG and GCP Firewall rules are same in nature as AWS Security Group

Additional Details

Cross VPC/VNet firewalling has become an important consideration in recent years as the number of VPCs have grown to the 100s for a typical cloud customer. Some customers have implemented commercially available firewalls in each VPC. This will serve the purpose, but at a much higher cost. If you are trying to implement a L4 stateful firewall for cross VPC traffic and log the traffic (stats/allows/denies) there is an easier and more cost effective solution to consider – use Aviatrix gateways.

Aviatrix Brings Holistic L4 Firewall to Cloud and Multi-Cloud

Aviatrix gateways have a Layer-4 stateful firewall built into the software. This firewall feature is automatically used with features such as transit peering or FQDN filtering, but can also be used by itself.

Aviatrix L4 Firewall Solution in AWS

The following diagram shows how you can implement this solution in AWS:

Aviatrix L4 Firewall Solution in Azure

The following diagram shows how you can implement this solution in Azure:


Aviatrix L4 Firewall Solution in GCP

The following diagram shows how you can implement this solution in GCP:


Solution Implementation Details

To enable this feature in Aviatrix first follow the Transit Workflow. Once deployed, the Stateful Firewall feature can be enabled by using the following steps:

Log into the Aviatrix Controller to create firewall policies:

  1. Go to the security tab: Security -> Stateful Firewall -> Policy tab

  2. Select each gateway from the list -> Click Edit on the top of the list

  3. Use the Security Policy UI to Allow or Deny traffic based on Source, Destination, Protocol and Port Range.


Additional Features:

  1. You can also log traffic to Log Analytics tools like Splunk, DataDog, SumoLogic etc. for audit and compliance purposes.

    1. Enable Logging (under Settings) to your log tool of choice.

    2. Select “Enable Packet Logging” in the Security Policy UI (screenshot above).

  2. You can also group IP addresses and tag them with human-readable names. This makes it easier to manage and apply firewall policies. Tagging can be accomplished under the Tag Management tab in the Security view.


0 replies

Be the first to reply!