Networking and security for VPC Endpoint (PrivateLink) in Single or Multiple Clouds
Applications running in a VPC including those running in an EC2 instance or those running as pods in EKS need access to other services that live outside of the VPC. These services can be broadly categorized as:
a. Services provided by the cloud provider themselves, such as AWS Endpoint Services (Macie, API Gateway, CodeBuild, Code Commit etc).
b. Services provided by 3rd party software vendors such as Snowflake, Mulesoft etc
c. Services that you may have created for use within your organization. Perhaps you need to expose a service to another line of business etc.
d. Services that may live in a different cloud Provider
VPC Endpoints provide an easy way for application in VPCs to connect to services outside of VPC. Mapping to the above-mentioned categories, you can create a VPC Endpoint for services provided by the cloud provider, service exposed to you by a vendor or something you or someone created within the organization.
Network Infrastructure and Security Challenges of VPC Endpoints:
VPC Endpoint is a great service (for cases where source and destination are in same cloud) that makes consuming services easy. It however poses a few challenges for enterprises, such as
- Infrastructure security teams see this is a backdoor for applications to access services bypassing all enterprise control. These enterprise controls may mean limiting such access to designated instances only or enforcing inspection by NGFW appliances.
- How to limit and control sprawl of multiple VPC Endpoints across 100s of VPCs
- It becomes very hard to enforce governance and compliance when there is no centralized way to manage, visualize and control.
- For support teams, it becomes a challenge as every troubleshooting session may be different and hence may require a lot more cycles.
- There is lack of visualization of traffic flows which makes troubleshooting even harder
- This requires IT support teams to have very deep knowledge of native cloud constructs, IAM Roles, Policies etc.
In-Cloud Endpoint Access:
One approach that is very easily enabled by Aviatrix Transit based architecture is the Centralized Services VPC which is the only VPC that is allowed to host VPC Endpoints. When a new VPC Endpoint is needed, IT or the app teams can create them but access to them has to go thru IT to ensure security, compliance, governance and support issues don’t become challenging down the road. The following design pattern shows that a ‘Designated VPC for Centralized handoff to PrivateLInk Services’ has been enabled connected to Aviatrix Transit. With this design, all traffic from the spoke VPCs need to be redirected to the PrivateLinks created in this VPC (which can be single or multiple). Now all the traffic goes thru the enterprise IT controlled Transit which also provides all the required visibility, troubleshooting and control.
Cross-Cloud Access of Private Link:
However, if your source is in a different public cloud then the destination, you must go over internet. For ex, if you have an application in Azure that needs access to S3, you must access is over internet. This poses challenges such as
- higher cost
- securing connectivity over internet
- compliance & data governance
- inserting NGFW inspection
- support challenges
- lack of visibility etc.
The solution that works for single cloud can easily be extended to multi-cloud as well. Imagine you have applications that live in Azure (IaaS, AKS etc.) or GCP (Compute Engine, GKE, etc.) that need to access AWS S3 buckets. Usually you would do this by accessing the public S3 bucket URL however as that is public, it rides over internet and is very expensive. Using Aviatrix architecture, you can route this traffic privately and access S3 buckets via VPC (Gateway) Endpoint created in AWS while your application lives in Azure or GCP.
Benefits of this approach
- There is no data processing or hourly charges for using Gateway Type VPC endpoints (for S3 and DynamoDB) so you are only paying for data that traverses the cloud and not for Endpoint service.
- Even for paid services, your VPC Endpoint limits cost compared to if retrieved over internet
- All the traffic is private which means you can enforce security controls
- CoPilot FlowIQ allows visualizing trends, flow information, monitoring, serviceability
- Troubleshooting is now in your control as traffic is going via predictable paths
- Better and more predictable performance as paths are more deterministic
Following diagram shows an example where a customer needed its AKS and GKE clusters to access data on S3 and connect with AWS Lambda.
By leveraging the Aviatrix MultiCloud transit alongwith Aviatrix S3 GWs, you can expose specific S3 buckets to be routed via private path.
More details on the Aviatrix S3 GWs can be found here: https://docs.aviatrix.com/HowTos/sfc_faq.html
The same architecture supports all other types of VPC Endpoints such as Lambda, CodeBuild, Kinesis etc. In these scenarios, an ENI is exposed in the subnet that has a private IP address. Aviatrix makes that private IP reachable by any VPC or VNet in same or different cloud. With this reachability, your applications can be in any cloud and still leverage service provided by AWS as an example.