Search the Community

Introduction Kubecost provides real-time cost visibility and insights for teams using Kubernetes. It has an intuitive dashboard to help you understand and analyze the costs of running your workloads in a Kubernetes cluster. Kubecost is built on OpenCost, which was recently accepted as a Cloud Native Computing Foundation (CNCF) Sandbox project, and is actively supported by AWS. Amazon EKS optimized bundle of Kubecost Earlier last year, Amazon Elastic Kubernetes Service (Amazon EKS) announced the availability of an Amazon EKS-optimized bundle of Kubecost for cluster cost visibility. The bundle is available to customers free of charge and includes Kubecost troubleshooting support. Kubernetes platform administrators and finance leaders can use Kubecost to visualize a breakdown of their Amazon EKS charges, allocate costs, and chargeback organizational units (e.g., application teams). Kubecost gives internal teams and business units transparent and accurate cost data based on AWS bill. Customers can also get personalized suggestions for cost optimization tailored to their infrastructure environment and usage patterns. Using Kubecost’s intuitive dashboard, customers can monitor, analyze, and allocate cluster costs. When customers deploy Kubecost in a cluster, the dashboard is secured by NGINX basic authentication, which isn’t recommended in production environments. This post shows how to make the dashboard accessible to external audiences, such as finance leaders, and secure access using Amazon Cognito. Solution overview We make the Kubecost dashboard accessible outside the cluster by exposing it using an ingress, which uses Application Load Balancer (ALB). Integrating Amazon Cognito with the ALB, the solution adds support for authenticating and authorizing users to the Kubecost dashboard. To learn more about how ALB and Cognito integrate, please see How to use Application Load Balancer and Amazon Cognito to authenticate users for your Kubernetes web apps. In this post, we use the secure ingress-auth-cognito EKS Blueprints pattern to set up: Application Load Balancer, Amazon Cognito, and a Transport Layer Security (TLS) Certificate on AWS Certificate Manager (ACM) with Amazon Route 53 hosted zone to authenticate users to Kubecost Deployment of Kubecost application using Kubecost add-on for EKS CDK Blueprints Kubernetes Ingress with annotations for Amazon Cognito and a TLS Certificate (using Amazon Certificate Manager) for securely authenticating user to Kubecost Customers can use this pattern to manage multiple clusters across environments with GitOps. Please see Continuous Deployment and GitOps delivery with Amazon EKS Blueprints and ArgoCD to learn about GitOps-driven delivery using EKS Blueprints Patterns. The secure ingress-auth-cognito Cloud Development Kit (CDK) pattern includes an Amazon EKS cluster configuration, compute capacity configuration, and add-ons required by Kubecost. Prerequisites You need the following to complete the steps in this post: AWS Command Line Interface (AWS CLI) version 2 AWS CDK version 2.80.0 or later Node version 20.0.0 or later NPM version 8.19.2 or later kubectl version 1.24 or later Git An Amazon Route 53 public hosted zone Let’s start by setting a few environment variables: ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text) export AWS_REGION=${AWS_REGION:=us-west-2} Clone the cdk-eks-blueprints-patterns repository and install dependency packages. This repository contains CDK v2 code written in TypeScript. git clone https://github.com/aws-samples/cdk-eks-blueprints-patterns.git cd cdk-eks-blueprints-patterns npm install make build The secure ingress-auth-cognito EKS Blueprints pattern is at lib/secure-ingress-auth-cognito/index.ts. In this file, you can find the blueprint definition with all the configurations above using the blueprints.EksBlueprint.builder() method. Bootstrap CDK The first step to any CDK deployment is bootstrapping the environment. Bootstrapping is the process of provisioning resources for the AWS CDK before you can deploy AWS CDK applications into an AWS environment (an AWS environment is a combination of an AWS account and Region). If you already use CDK in a region, then you don’t need to repeat the bootstrapping process. Execute the commands below to bootstrap the AWS environment in your Region: cdk bootstrap aws://$ACCOUNT_ID/$AWS_REGION Deploy Kubecost with secured access In this solution, we’ll allow access to the Kubecost dashboard based on user email addresses. You can control access to the dashboard by allow-listing an entire domain or individual email addresses. Users are required to sign-up before they can access the Kubecost dashboard. The pre sign-up Lambda trigger only allows sign-ups when the user’s email domain matches the allow-listed domains. When users sign-up, Amazon Cognito sends a verification code to their email address. Users have to verify access (using the one time valid code) to their email before they get access to the dashboard. First, we’ll create an AWS Systems Manager (SSM) parameter to store the value of the email domain that users use to sign up. Next, we’ll create an environment variable to store the domain name that hosts the Kubecost dashboard. The email domain and the domain used to host the Kubecost dashboard can be same or different. For example, you may choose to host the dashboard at kubecost.myorg.mycompany.com and use email@mycompany.org to login to the dashboard. Create below parameters with allowed email addresses and domains in the AWS Systems Manager Parameter Store: export SSM_PARAMETER_KEY="/secure-ingress-auth-cognito/ALLOWED_DOMAINS" export SSM_PARAMETER_VALUE="emaildomain1.com,emaildomain2.com" aws ssm put-parameter \ --name "$SSM_PARAMETER_KEY" \ --value "$SSM_PARAMETER_VALUE" \ --type "String" \ --region $AWS_REGION If you’d like to limit access to the dashboard by email addresses, then you can also create a parameter to store allowed email addresses and add a logic to the pre authentication Lambda trigger as shown here. Next, create a secret in AWS Secrets Manager that you’ll use to access ArgoCD. The argo-admin-password secret must be defined as plain text (not key/value): aws secretsmanager create-secret --name argo-admin-secret \ --description "Admin Password for ArgoCD" \ --secret-string "password123$" \ --region $AWS_REGION The CDK code expects the allowed domain and subdomain names in the CDK context file (cdk.json). Create two environment variables. The PARENT_HOSTED_ZONE variable contains the name of your Route 53 public hosted zone. The DEV_SUBZONE_NAME will be the address for your Kubecost dashboard. Generate the cdk.json file: PARENT_HOSTED_ZONE=mycompany.a2z.com DEV_SUBZONE_NAME=kubecost.mycompany.a2z.com cat << EOF > cdk.json { "app": "npx ts-node dist/lib/common/default-main.js", "context": { "parent.hostedzone.name": "${PARENT_HOSTED_ZONE}", "dev.subzone.name": "${DEV_SUBZONE_NAME}" } } EOF Run the below command from the root of this repository to deploy the solution: make pattern secure-ingress-cognito deploy secure-ingress-blueprint This blueprint will deploy the following: Amazon Virtual Private Cloud (Amazon VPC) with public and private subnets, Network Address Translation (NAT) gateways in each availability zone (AZ), and an Internet Gateway An Amazon EKS cluster with the following Kubernetes add-ons Metrics Server Cluster Autoscaler Amazon Elastic Block Store (Amazon EBS) Container Storage Interface (CSI) Driver Amazon EKS AWS Load Balancer Controller Amazon VPC CNI ExternalDNS Kubecost Argo CD Amazon Cognito user pool, user pool client, domain and also pre-sign-up and pre-authentication lambda triggers to run custom logic to validate users before allowing them to either sign-up or authentication. Once the deployment is complete, you will see the output similar to shown below in your terminal: Outputs: secure-ingress-blueprint.secureingressblueprintClusterNameD6A1BE5C = secure-ingress-blueprint secure-ingress-blueprint.secureingressblueprintConfigCommandD0275968 = aws eks update-kubeconfig —name secure-ingress-blueprint —region us-west-2 —role-arn arn:aws:iam::<ACCOUNT ID>:role/secure-ingress-blueprint-secureingressblueprintMas-XXXXXXXXXX secure-ingress-blueprint.secureingressblueprintGetTokenCommand21BE2184 = aws eks get-token —cluster-name secure-ingress-blueprint —region us-west-2 —role-arn arn:aws:iam::<ACCOUNT ID>:role/secure-ingress-blueprint-secureingressblueprintMas-XXXXXXXXXX Stack ARN: arn:aws:cloudformation:us-west-2:<ACCOUNT ID>:stack/secure-ingress-blueprint/XXXXXXXXXX To update your Kubernetes configuration for your new cluster, copy and run the aws eks update-kubeconfig command (the second command in the output) in your terminal. export EKS_KUBECONFIG=$(aws cloudformation describe-stacks \ --stack-name secure-ingress-blueprint \ --query "Stacks[0].Outputs[?starts_with(OutputKey, 'secureingressblueprintConfigCommand')].OutputValue" \ --region $AWS_REGION \ --output text) eval $EKS_KUBECONFIG Validate the access to your Amazon EKS cluster using below kubectl listing all namespaces: kubectl get namespace You should see the following namespaces in the cluster: NAME STATUS AGE argocd Active 30m default Active 39m external-dns Active 30m kube-node-lease Active 39m kube-public Active 39m kube-system Active 39m kubecost Active 30m The stack deploys Kubecost resources in the kubecost namespace. kubectl -n kubecost get all NAME READY STATUS RESTARTS AGE pod/kubecost-cost-analyzer-84d5775f7b-zg8mq 2/2 Running 0 88m pod/kubecost-cost-analyzer-grafana-69d77ccd6d-9r8rc 2/2 Running 0 88m pod/kubecost-cost-analyzer-kube-state-metrics-789fc978c8-ch8lb 1/1 Running 0 88m pod/kubecost-cost-analyzer-prometheus-node-exporter-w9w75 1/1 Running 0 88m pod/kubecost-cost-analyzer-prometheus-server-6dc99564bf-mz9nw 2/2 Running 0 88m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubecost-cost-analyzer-cost-analyzer ClusterIP 172.20.193.130 <none> 9003/TCP,9090/TCP 88m service/kubecost-cost-analyzer-grafana ClusterIP 172.20.143.32 <none> 80/TCP 88m service/kubecost-cost-analyzer-kube-state-metrics ClusterIP 172.20.165.147 <none> 8080/TCP 88m service/kubecost-cost-analyzer-prometheus-node-exporter ClusterIP None <none> 9100/TCP 88m service/kubecost-cost-analyzer-prometheus-server ClusterIP 172.20.54.102 <none> 80/TCP 88m NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/kubecost-cost-analyzer-prometheus-node-exporter 1 1 1 1 1 <none> 88m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/kubecost-cost-analyzer 1/1 1 1 88m deployment.apps/kubecost-cost-analyzer-grafana 1/1 1 1 88m deployment.apps/kubecost-cost-analyzer-kube-state-metrics 1/1 1 1 88m deployment.apps/kubecost-cost-analyzer-prometheus-server 1/1 1 1 88m Testing the authentication Point your browsers to the URL you associated with the DEV_SUBZONE_NAME key from the CDK context to access the Kubecost dashboard. The value is also stored as an environment variable: echo $DEV_SUBZONE_NAME Your browser will be redirected to an Amazon Cognito hosted User Interface (UI) sign-in page. Since this is your first time accessing the application, select sign up. The Pre sign-up AWS Lambda trigger for Amazon Cognito User pool is configured to allow users to register only from certain allow-listed email domains. The allow-listed email domains are configured as an environmental variable in the AWS Lambda function. Let us try sign up a new user using an email id, whose domain is not part of the allow list. You’ll get an error since the domain is not allow-listed. Let’s sign up as a new user with one of the allow-listed email domains. This time, you’ll get a prompt to confirm your account. Get the verification code sent to your email and confirm your account. After verifying email address, sign in to access the Kubecost dashboard Once you sign in, the ALB will redirect you to the Kubecost dashboard: Cleaning up You continue to incur cost until deleting the infrastructure that you created for this post. Use the commands below to delete resources created during this post: make pattern secure-ingress-cognito destroy secure-ingress-blueprint Conclusion In this post we showed you how to secure the Kubecost dashboard while making it accessible to users without needing access to the Kubernetes cluster. We used an ALB to expose the dashboard and secured access using Cognito. We also created a record in Route 53 so users can easily access the dashboard. We used Cognito user pool to store user information. If you already have an identity provider that provides OpenID Connect (OIDC) or SAML 2.0 support, then you can integrate it with Cognito to skip the sign-up and sign-in to the Kubecost dashboard. View the full article

Running across clouds or on-prem, Kubecost provides the spend visibility and allocation necessary to significantly and continually optimize Kubernetes costs San Francisco, CA and KubeCon + CloudNativeCon EU 2022, May 16, 2022 – Kubecost, a solution for monitoring, managing, and optimizing Kubernetes spend at scale, today announced it has more than doubled adoption recently as […] The post Kubecost Expands Open Source Project, Community Involvement, and Enterprise Customers in 1H of 2022 appeared first on DevOps.com. View the full article