Search the Community

Amazon EventBridge Pipes now supports event delivery through AWS PrivateLink, allowing you to send events from an event source located in an Amazon Virtual Private Cloud (VPC) to a Pipes target without traversing the public internet. With today’s launch, you can use Pipes to poll from Amazon Managed Streaming for Apache Kafka (MSK), self-managed Kafka, and Amazon MQ sources residing in a private subnet without the need to deploy a NAT gateway, configure firewall rules, or set up proxy servers. View the full article

AWS PrivateLink now supports Amazon QuickSight, providing private connectivity between the QuickSight website, virtual private clouds (VPCs) or on-premises networks without exposing traffic to the public internet. With this integration, administrators can also use VPC endpoint policies to restrict access to QuickSight accounts that are not authorized on their network. View the full article

Amazon DynamoDB now supports AWS PrivateLink. With AWS PrivateLink, you can simplify private network connectivity between virtual private clouds (VPCs), DynamoDB, and your on-premises data centers using interface VPC endpoints and private IP addresses. AWS PrivateLink is compatible with AWS Direct Connect and AWS Virtual Private Network (VPN) to facilitate private network connectivity, and helps you eliminate the need to use public IP addresses, configure firewall rules, or configure an internet gateway to access DynamoDB from your on-premises data centers. As a result, AWS PrivateLink helps you maintain compliance for your DynamoDB workloads over the private network. View the full article

Introduction The adoption and large-scale growth of Kubernetes in recent years has resulted in businesses deploying multiple Amazon Elastic Kubernetes Service (Amazon EKS) clusters to support their growing number of microservice based applications. The Amazon EKS clusters are usually deployed in separate Amazon Virtual Private Clouds (Amazon VPCs) and often in separate AWS accounts. A common challenge we hear from our customers is that their clients or applications have requirements to communicate with their Amazon EKS Kubernetes API across individual VPC and account-level boundaries and don’t want to route traffic over the Internet. In this post, we’ll explore a blueprint for using AWS PrivateLink as a solution to enable cross-VPC and cross-account private access for the Amazon EKS Kubernetes API. Amazon VPC connectivity options There are a number of AWS native networking services that offer connectivity across VPCs and across accounts without exposing your traffic to the public Internet. These include: VPC Peering – which creates a connection between two or more VPCs that allow endpoints to communicate as if they were on the same network. It can connect VPCs that are in different accounts and in different regions. AWS Transit Gateway – which acts as a highly scalable cloud router connecting multiple VPCs in a region with a hub-and spoke architecture. Transit Gateway works across accounts using AWS Resource Access Manager (AWS RAM). AWS PrivateLink – which provides private connectivity to a service in one VPC that can be consumed from a VPC in the same account or from another account. PrivateLink supports private access to customer application services, AWS Marketplace partner services, and to certain AWS services. VPC Peering and Transit Gateway are great options for cross-VPC and cross-account connectivity when you want to integrate the VPCs into a larger virtual network. However, this may not be an option for some customers due to security concerns, compliance requirements, or overlapping CIDR blocks. AWS PrivateLink AWS PrivateLink is a purpose-built technology designed to enable private connectivity between VPCs and supported AWS services in a highly available, reliable, and secure manner. A service powered by PrivateLink is created directly inside your VPC as an interface VPC endpoint. For each subnet (from your VPC) you choose for your endpoint, an Elastic Network Interface (ENI) is provisioned with a private IP addresses from the subnet address range. This keeps your traffic on the AWS backbone network and removes the need for Internet and NAT gateways to access supported AWS services. AWS PrivateLink offers several advantages when it comes to securely connecting services and resources across different VPCs and accounts: PrivateLink provides a higher level of security and isolation by enabling private connectivity directly between VPCs without exposing traffic to the public Internet. This ensures that sensitive data and services remain within the AWS network, reducing potential attack vectors. Setting up and managing PrivateLink is often simpler and more straightforward compared to VPC Peering and Transit Gateway, which can be complex to configure, especially in larger and more complex network architectures. PrivateLink can be a preferred choice for organizations with strict compliance requirements, as it helps maintain data within the AWS network, which reduces the risk of data exposure. PrivateLink can handle overlapping IP address ranges between VPCs. It simplifies network design in situations where address conflicts might occur. AWS PrivateLink and Amazon EKS Private access to the Amazon EKS management API is available with AWS PrivateLink. The EKS management API allows you to provision and manage the lifecycle of EKS clusters, EKS managed node groups, and EKS add-ons. Each EKS cluster you provision has a separate API (i.e., the Kubernetes API) that’s accessed using tools like kubectl and Helm to provision and manage the lifecycle of Kubernetes resources in that cluster. An EKS cluster’s Kubernetes API is publicly available on the Internet by default. You can enable private access to the Kubernetes API by configuring your EKS cluster’s endpoint access. While the EKS Management API is available as a PrivateLink endpoint, the Kubernetes API isn’t and has historically required configuring VPC Peering or Transit Gateway to enable cross-VPC and cross-account access. AWS PrivateLink for the Amazon EKS Kubernetes API Native AWS PrivateLink support for the Amazon EKS Kubernetes API isn’t currently available, though there is an open roadmap request. However, customers can create a custom AWS PrivateLink powered service, known as a VPC endpoint service, for any application or service available behind a Network Load Balancer (NLB). To create a PrivateLink powered endpoint service for the EKS Kubernetes API, you need to create a NLB that targets the Kubernetes control plane ENIs that EKS creates in your subnets. In addition, there are two challenges that need to addressed: The EKS ENIs for the Kubernetes control plane are deleted and recreated in response to a number of events including, but not limited to: Kubernetes version upgrades, EKS platform upgrades, control plane autoscaling, enabling secret encryption, and associating/disassociating an OIDC identity provider. The NLB target group must be updated each time the ENIs are created and deleted. A new DNS hostname is created for the PrivateLink interface VPC endpoint that isn’t included in the EKS Kubernetes API TLS certificate. Clients connections to the Kubernetes API using the PrivateLink endpoint DNS fail with a certificate error. Solution overview To addresses the challenge of the deleted and recreated Amazon EKS ENI’s behind the NLB target group, a combination of Amazon EventBridge and AWS Lambda are used to respond to these changes and automatically update the NLB target group. Two EventBridge rules are created to process these events with Lambda functions: The first rule responds to the CreateNetworkInterface API call, which adds the newly created EKS ENIs to the NLB target group. The second rule uses the EventBridge Scheduler to run every 15 minutes and removes targets for ENIs that have been deleted. To address the challenge of the Kubernetes API certificate error, an Amazon Route 53 private hosted zone is created in the client VPC to override the DNS of the EKS API server endpoint. An alias record, matching the DNS name of EKS Kubernetes API server, is added to the private hosted zone and routes traffic to the AWS PrivateLink interface VPC endpoint in the client VPC. This eliminates certificate errors and enables secure and reliable communication with the EKS cluster. This post introduces the Amazon EKS Blueprints PrivateLink Access pattern and the architecture is shown in the following diagram. PrivateLink is used to create a secure, private connection between the Client VPC and a private EKS cluster in the Service Provider VPC. A few key points to highlight: The Amazon Elastic Compute Cloud (Amazon EC2) instance in the client VPC can access the private EKS cluster as if it were in its own VPC, without the use of an Internet gateway, NAT device, VPN connection, or AWS Direct Connect The client VPC has overlapping IPs with the Service Provider VPC. AWS PrivateLink makes it easy to publish an API or application endpoint between VPCs, including those that have overlapping IP address ranges. To connect through AWS PrivateLink, an interface VPC endpoint is created in the Client VPC, which establishes connections between the subnets in Client VPC and Amazon EKS in Service Provider VPC using network interfaces. This Service Provider creates a PrivateLink endpoint service pointing to internal NLB that targets the Kubernetes control plane ENIs. Walkthrough This post walks you through the following steps: Download and configure the PrivateLink Access Blueprint Deploy the PrivateLink Access Blueprint Test access to EKS Kubernetes API server endpoint End-to-end Testing with kubectl Prerequisites We need a few tools to deploy our blueprint. Ensure you have each of the following tools in your working environment: AWS Command Line Interface (AWS CLI) v2 Session Manager plugin for AWS CLI Terraform kubectl jq 1. Download the PrivateLink Access Blueprint In a terminal session, run the following set of commands to download the repo for the PrivateLink Access Blueprint: git clone https://github.com/aws-ia/terraform-aws-eks-blueprints.git cd terraform-aws-eks-blueprints/patterns/privatelink-access 2. Deploy the PrivateLink Access Blueprint Run the following steps (in the order shown) to initialize the Terraform repository, create, and configure the Private NLB in the service provider VPC and Amazon EventBridge rules before the rest of the resources are created. This ordered execution of steps ensures that the NLB, Lambda functions and EventBridge rules are in place and ready to capture creation of (and/or changes to the) EKS Kubernetes API endpoints (i.e., ENIs) when the cluster is in the process of being created (or upgraded). Note: The EKS Kubernetes API endpoint is temporarily made public so that Terraform has access to update the aws-auth ConfigMap. The Client Amazon EC2 instance’s AWS Identify and Access Management (AWS IAM) role is configured with administrative access for testing. terraform init terraform apply -target=module.eventbridge -target=module.nlb --auto-approve terraform apply --auto-approve Once the pattern has successfully deployed, you’ll be provided with multiple output values. Copy and save them to use later in the post. Review the Terraform output value for cluster_endpoint_private, it should look similar to snippet below: aws eks update-cluster-config \ --region us-west-2 \ --name privatelink-access \ --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true Copy the command and run it in a terminal session to take the cluster API endpoint private as originally intended. The above command causes the cluster to update, which takes a few minutes to complete. You can check on the status of the cluster by running the following command. aws eks describe-cluster --name privatelink-access | jq -r '.cluster.status' When the cluster is updating the output will be UPDATING , else the output will be ACTIVE. 3. Test access to EKS Kubernetes API server endpoint Of the other Terraform output values, the value ssm_test is provided to aid in quickly testing the connectivity from the client EC2 instance to the private EKS cluster via PrivateLink. Copy the output value, which looks like the snippet shown below (as an example) and paste it into your terminal to execute and check the connectivity. If configured correctly, the value returned should be ok. A successful test validates that: Private hosted zone for EKS API server endpoint worked The PrivateLink access worked thru the NLB all the way to the private API endpoints of the EKS cluster both of which are private in service provider VPC The EKS Kubernetes API server endpoint is ready and functional Note: The following snippet is shown only as an example. COMMAND="curl -ks https://XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.gr7.us-west-2.eks.amazonaws.com/readyz" COMMAND_ID=$(aws ssm send-command --region region-name \ --document-name "AWS-RunShellScript" \ --parameters "commands=[$COMMAND]" \ --targets "Key=instanceids,Values=i-XXXXXXXXXXXXXXXXX" \ --query 'Command.CommandId' \ --output text) aws ssm get-command-invocation --region region-name \ --command-id $COMMAND_ID \ --instance-id i-XXXXXXXXXXXXXXXXX \ --query 'StandardOutputContent' \ --output text 4. End-to-end testing with kubectl On the Client EC2 instance, creating the kubeconfig file and interacting with the EKS cluster’s API server with kubectl should work seamlessly without any additional configuration as the IAM role associated with the Client EC2 instance is: added to the aws-auth ConfigMap and mapped to Kubernetes role of system:masters (cluster admin role) and based on the attached IAM policies, has the permissions to eks:DescribeClusterand eks:ListClusters Follow the steps below to access the cluster’s API server with kubectl from the Client EC2 instance. Log into the Client EC2 instance Start a new AWS Systems Manager session (SSM session) on the Client EC2 instance using the provided ssm_start_session output value. It should look similar to the following code snippet shown (as an example). Copy the output value and paste it into your terminal to execute. Your terminal will now be connected to the Client EC2 instance. Note: The following snippet is shown only as an example. aws ssm start-session --region us-west-2 --target i-XXXXXXXXXXXXXXXXX Update Kubeconfig On the Client EC2 machine, run the following command to update the local ~/.kube/config file to enable access to the cluster: aws eks update-kubeconfig --region us-west-2 --name privatelink-access Test complete access with kubectl Test access by listing the pods running on the cluster: kubectl get pods -A The output should look similar to the sample shown below: NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-4f8g8 1/1 Running 0 1m kube-system coredns-6ff9c46cd8-59sqp 1/1 Running 0 1m kube-system coredns-6ff9c46cd8-svnpb 1/1 Running 0 2m kube-system kube-proxy-mm2zc 1/1 Running 0 1m Cleaning up Before we destroy or teardown all the resources created, we need to ensure that the cluster state is restored for Terraform to do a complete cleanup. This means making the cluster API endpoint public again. Review the output value for cluster_endpoint_public, it should look similar to snippet below: aws eks update-cluster-config \ --region us-west-2 \ --name privatelink-access \ --resources-vpc-config endpointPublicAccess=true,endpointPrivateAccess=true Copy the command and run it in a terminal session to take cluster API endpoint public and wait till the operation completes. Once verified that the cluster API endpoint is public, then proceed to clean up all the AWS resources created by Terraform by running the following command: terraform destroy --auto-approve Conclusion In this post, we described an innovative blueprint for harnessing AWS PrivateLink as a solution to enable private, cross-VPC, and cross-account access to the Amazon EKS Kubernetes API. The solution involves PrivateLink, EventBridge, Lambda functions, and a Route53 Private Hosted Zone built as part of the EKS Blueprint to present an effective way to ensure private access to EKS Kubernetes APIs, which enhanced security and compliance for businesses leveraging microservices across multiple VPCs and accounts. Reference De-mystifying cluster networking for Amazon EKS worker nodes View the full article

AWS Cloud Map now supports AWS PrivateLink. You can now use AWS PrivateLink to privately access AWS Cloud Map APIs from your Amazon Virtual Private Cloud (VPC) without exposing your data through public internet. Creating VPC Endpoints incurs charges. See the AWS PrivateLink pricing page for more information. View the full article

You can now choose IP addresses for your AWS PrivateLink VPC endpoints. AWS PrivateLink is a fully-managed, private connectivity service that enables customers to share and access services including AWS services, third-party services, and internal enterprise services hosted on AWS in a secure and scalable manner while keeping network traffic private. View the full article

You can now enable CloudWatch Contributor Insights on your AWS PrivateLink-powered VPC Endpoint Services in 6 new regions- Asia Pacific (Hyderabad, Jakarta, Melbourne), Europe (Spain, Zurich) and Middle East (UAE). AWS PrivateLink is a fully-managed private connectivity service that enables customers to access AWS services, third-party services or internal enterprise services hosted on AWS in a secure and scalable manner while keeping network traffic private. CloudWatch Contributor Insights analyzes time-series data to report the top contributors and number of unique contributors in a dataset. View the full article

You can now use AWS PrivateLink to privately access AWS App Mesh APIs from your Amazon Virtual Private Cloud (VPC) without exposing your data through public internet. Creating VPC Endpoints incurs charges. See the AWS PrivateLink pricing page for more information. View the full article

You can now use AWS PrivateLink to privately access Amazon Pinpoint from your Amazon Virtual Private Cloud (Amazon VPC) without using public IPs, and without requiring the public internet. View the full article

Amazon SWF now supports AWS PrivateLink allowing you to run workflows from your Virtual Private Cloud (VPC) without traversing the public internet. View the full article

You can now use AWS PrivateLink to privately access Amazon Connect Cases from your Amazon Virtual Private Cloud (Amazon VPC) without using public IPs, and without requiring the public internet. View the full article

Amazon Omics now supports AWS PrivateLink and AWS CloudFormation. You can now use AWS PrivateLink to privately access Amazon Omics APIs from your Amazon Virtual Private Cloud (VPC). Creating VPC Endpoints incurs charges. See the AWS PrivateLink pricing page for more information. With CloudFormation support, you can now use AWS CloudFormation templates to create, update, and delete your Amazon Omics resources. This helps you automate and standardize DevOps processes across your AWS accounts and AWS Regions for Amazon Omics. View the full article

You can now use AWS PrivateLink to privately access the Amazon Elastic Kubernetes Service (Amazon EKS) management APIs from your Amazon Virtual Private Cloud (VPC). AWS PrivateLink provides private connectivity between VPCs, AWS services, and your on-premises networks. You can now manage your Amazon EKS clusters in your VPC using AWS PrivateLink to help your organization’s security and compliance requirements. You can also access the VPC endpoint from on-premises environments or from other VPCs using AWS VPN, AWS Direct Connect, or VPC Peering. Creating VPC Endpoints incurs charges, see the AWS PrivateLink pricing page for more information. View the full article

Today, we would like to announce enhancements to AWS Mainframe Modernization service including an expansion to 5 new regions and support for AWS CloudFormation, AWS PrivateLink, and AWS Key Management Service (KMS) resulting in repeatable deployment, and greater security and compliance. View the full article

Amazon OpenSearch Service now supports managed VPC endpoints (powered by AWS PrivateLink) to connect to your Amazon OpenSearch Service VPC-enabled domain in a Virtual Private Cloud (VPC). With an Amazon OpenSearch Service managed endpoint, you can now privately access your OpenSearch Service domain within your VPC from your client applications in other VPCs, within the same or across AWS accounts, without using public IPs or requiring traffic to traverse the Internet. View the full article

AWS Cloud Control API now supports AWS PrivateLink, providing access for customers to leverage AWS Cloud Control API through private Virtual Private Cloud (VPC) endpoints within their virtual private network. Customers can now manage their cloud infrastructure in a consistent manner and use the latest AWS capabilities faster using Cloud Control API’s common application programming interfaces (APIs) through private IP addresses in their Amazon VPC. These customers can use AWS Cloud Control API without having to use public IPs, firewall rules, or an internet gateway. View the full article

You can now use tags to better track and manage your AWS PrivateLink-powered VPC Endpoint Services. AWS PrivateLink is a fully-managed private connectivity solution that enables customers to connect to other services hosted on AWS using a secure and scalable method while keeping network traffic private. View the full article

Amazon AppFlow, a fully managed integration service that helps customers to securely transfer data between AWS services and software-as-a-service (SaaS) applications in just a few clicks, now supports Salesforce API version 55.0 which is the latest API in the Salesforce Summer ’22 release. View the full article

You can now enable CloudWatch Contributor Insights on your AWS PrivateLink-powered VPC Endpoint Services. AWS PrivateLink is a fully-managed private connectivity service that enables customers to access AWS services, third-party services or internal enterprise services hosted on AWS in a secure and scalable manner while keeping network traffic private. CloudWatch Contributor Insights analyzes time-series data to report the top contributors and number of unique contributors in a dataset. View the full article

You can now use AWS PrivateLink to privately access the AWS Migration Hub Refactor Spaces APIs from your virtual private cloud (Amazon VPC). AWS PrivateLink provides private connectivity between VPCs, AWS services, and your on-premises networks. Starting today, you can manage your Refactor Spaces resources using AWS PrivateLink and meet your organization’s security and compliance requirements. To use AWS PrivateLink, create an interface VPC endpoint for Refactor Spaces in your VPC using the Amazon VPC console, SDK, or CLI. You can also access the VPC endpoint from on-premises environments or from other VPCs using AWS VPN, AWS Direct Connect, or VPC Peering. View the full article

AWS Backup now supports AWS PrivateLink for VMware workloads, providing direct access to AWS Backup from your VMware environment via a private endpoint within your virtual private network in a scalable manner. With this launch, you can now secure your network architecture by connecting to AWS Backup using private IP addresses in your Amazon Virtual Private Cloud (VPC), eliminating the need to use public IPs, firewall rules, or an Internet Gateway. AWS PrivateLink is available at a low per-GB charge for data processed and a low hourly charge for interface VPC endpoints. See AWS PrivateLink pricing for more information. View the full article

AWS Panorama customers can now use AWS PrivateLink to access AWS Panorama from their Amazon Virtual Private Cloud (Amazon VPC) without using public endpoints, and without requiring the traffic to traverse the Internet. Using AWS PrivateLink, you can access AWS Panorama endpoints easily and securely by keeping your traffic within the AWS network, while simplifying your internal network architecture. You no longer need to use an internet gateway, Network Address Translation (NAT) devices, or firewall proxies to connect to AWS Panorama. View the full article

Amazon S3 on Outposts now supports AWS PrivateLink, providing direct access to manage your S3 on Outposts storage capacity via a private endpoint within your virtual private network. This allows you to simplify your internal network architecture and perform management operations on your S3 storage by using private IP addresses in your Virtual Private Cloud (VPC), eliminating the need to use public IPs or proxy servers. View the full article

You can now connect over IPv6 to your services hosted in AWS using AWS PrivateLink. AWS PrivateLink is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC Endpoint Services), third-party SaaS services and supported AWS Marketplace partner services. View the full article

Amazon Transcribe is an automatic speech recognition (ASR) service that you can use to add speech-to-text capabilities to your applications. Starting today, AWS customers can use AWS PrivateLink to access the Amazon Transcribe batch API from their Amazon Virtual Private Cloud (Amazon VPC) without using public IPs or requiring the traffic to traverse the Internet. AWS PrivateLink provides private connectivity between VPCs and AWS services, without ever leaving the Amazon network. With this launch, AWS PrivateLink is now supported for both Batch and Streaming APIs. View the full article