Monitoring & Observability

Amazon CloudWatch Synthetics, an outside-in monitoring capability to continually verify your customer experience even when you don’t have any customer traffic on your applications, introduced a new capability to create custom groups of canaries. By creating a group of canaries, you can track success/failure status at a group or application level yet with an easy drill down to the failing canary, making it easier to pinpoint the canary failures in the context of the group or application. When groups consist of canaries across multiple AWS regions, this new capability allows you to more easily isolate region-specific issues. View the full article

Amazon DocumentDB (with MongoDB compatibility) is a database service that is purpose-built for JSON data management at scale, fully managed and integrated with AWS, and enterprise-ready with high durability. View the full article

Amazon OpenSearch Service, with the availability of OpenSearch 1.3., now gives customers the ability to organize their logs, traces and visualizations in an application-centric view. Customers can also benefit from enhanced log monitoring support with live tailing of logs, the ability to see surrounding log data, and the ability to do powerful ad-hoc analysis of unformatted log data at query time. View the full article

Observability has become one of the most overused buzzwords in IT and cybersecurity. Today, the term is used by vendors to refer to everything from application performance to network monitoring, cybersecurity and data and analytics. While the term’s ubiquity has created confusion for everyone from end users to journalists, startups in the space have also […] The post Deciphering the Observability Market appeared first on DevOps.com. View the full article

Today, we are announcing the general availability of a new feature, Log Anomaly Detection and Recommendations for Amazon DevOps Guru. As part of this feature, DevOps Guru will ingest Amazon CloudWatch Logs for AWS resources that make up your application, with Lambda being first. Logs will provide new enrichment data in an insight to enable more accurate understanding of the root cause behind an application issue, and provide more precise remediation steps. View the full article

EC2 Auto Scaling now publishes predictive scaling policy’s forecasts as a CloudWatch metric, enabling you to analyze, monitor, and set alarms on the accuracy of predictive scaling. Predictive Scaling is a scaling policy that proactively increases the capacity of your Auto Scaling group ahead of predicted demand, improving the availability of your application while reducing the need to stay overprovisioned that otherwise would have increased your EC2 bill. As predictive scaling only increases the capacity for your Auto Scaling groups, applying it to your current scaling configurations strictly enhances your application availability. However, an inaccurate prediction can po…

Amazon OpenSearch Service now allows users to view default quota and applied quota information through Service Quotas. Quotas, also referred to as limits in AWS services, are the maximum values for the resources, actions, and items in your AWS account. Each AWS service defines its quotas and establishes default values for those quotas. Depending on your business needs, you might need to increase your service quota values. Service Quotas enables you to look up your service quotas and to request quota increase. AWS Support might approve, deny, or partially approve your requests. View the full article

Developers can now access Amazon CloudWatch Logs within Visual Studio using the AWS Toolkit for Visual Studio. Directly from the IDE, it is now possible to search and filter log groups, log streams, and events. Additionally, log groups can be accessed from their associated resources, and log events can be downloaded to a file. View the full article

Amazon QuickSight now supports monitoring of QuickSight assets by sending metrics to Amazon CloudWatch. QuickSight developers and administrators can use these metrics to observe and respond to the availability and performance of their QuickSight ecosystem in near real time. They can monitor dataset ingestions, dashboards, and visuals to provide their readers with a consistent, performant, and uninterrupted experience on QuickSight. For more information, visit here. View the full article

It’s every on-call’s nightmare—awakened by a text at 3 a.m. from your alert system that says there’s a problem with the cluster. You need to quickly determine if the issue is with the Amazon EKS managed control plane or the new custom application you just rolled out last week. Even though you installed the default dashboards the blogs recommended, you’re still having difficulty understanding the meaning of the metrics you are looking at. If only you had a dashboard that was focused on the most common problems seen in the field—one where you understood what everything means right away, letting you quickly scan for even obscure issues efficiently… View the full ar…

Amazon OpenSearch Service now supports tag-based authorization for HTTP methods, making it easier for you to manage access control for data read and write operations. You can use Identity policies in AWS Identity and Access Management (IAM) to define permissions for read and write HTTP methods, allowing coarse-grained access control of data on your Amazon OpenSearch Service domains. View the full article

Amazon Keyspaces (for Apache Cassandra), a scalable, highly available, and fully managed Apache Cassandra-compatible database service, now helps you monitor your table-level storage costs through Amazon CloudWatch. View the full article

Amazon OpenSearch Service now supports cross-cluster search across regions, enabling you to perform searches, aggregations, and visualizations across multiple domains in different regions with a single query. View the full article

Amazon Lookout for Metrics announces the launch of backtesting when using Amazon CloudWatch as a data source connector. Backesting is a new anomaly detection mode you can now select when setting up your detector. You can seamlessly connect to your data in CloudWatch to set up a highly accurate anomaly detector across metrics, dimensions, and namespaces of your choice. Amazon Lookout for Metrics uses machine learning (ML) to automatically detect and diagnose anomalies (outliers from the norm) without requiring any prior ML experience. Amazon CloudWatch provides you with actionable insights to monitor your applications, respond to system-wide performance changes, optimize r…

Amazon CloudWatch now supports AWS Elemental MediaTailor logs as part of Vended Logs. Vended logs are specific AWS service logs natively published by AWS services on behalf of the customer and available at volume discount pricing. View the full article

Amazon Managed Grafana now supports a new API for creating Grafana API tokens, as well as support for new plugins, Grafana version 8.4, and workspace tags. With CreateWorkspaceApiKey, customers can create Grafana API tokens without having to log into the Grafana workspace console, enabling users to programmatically create, delete, and manage Grafana resources such as dashboards, alerts, and data sources. Amazon Managed Grafana adds support for Github, Moogsoft, Pixie, and Windrose plugins, enabling customers to connect, query, and visualize data from additional data sources. Existing and new Amazon Managed Grafana workspaces now support Grafana version 8.4, with no action…

Amazon CloudWatch is introducing enhancements to the console experience, which improve dashboard data visualizations and console navigation. The enhancements include new dashboard widgets as well as more options to access frequently used dashboards, log groups and alarms. View the full article

Amazon CloudWatch Synthetics now supports deletion of underlying canary resources along with the canary deletion. When you delete a canary you can choose whether to also delete related resources created by the canary, thus making canary resources management easier and efficient. Synthetics canaries that run on a defined frequency to monitor the health and performance of your endpoints and APIs creates these resources as part of canary creation step. View the full article

AWS Secrets Manager now publishes a metric to Amazon CloudWatch for the number of secrets in your account. With this feature, you can easily review how many secrets you are using in Secrets Manager. You can also set alarms for an unexpected increase or decrease in number of secrets. View the full article

AWS Step Functions now provides a new console experience for viewing and debugging your workflow executions that makes it easier to search, filter, and root cause issues in your executions. View the full article

Amazon Managed Service for Prometheus usage metrics are now available in Amazon CloudWatch at no additional charge. Amazon Managed Service for Prometheus is a fully managed Prometheus-compatible monitoring service that makes it easy to monitor and alarm on operational metrics at scale. Prometheus is a popular Cloud Native Computing Foundation open-source project for monitoring and alerting that is optimized for container environments. With Amazon CloudWatch usage metrics, you can check your Amazon Managed Service for Prometheus workspace usage, and can start to proactively manage your quotas. View the full article

Prometheus and Grafana can serve the needs of both on-premises or cloud-based companies, and Hosted Prometheus and Grafana by MetricFire can also be set up on-premises or on cloud. View the full article

CloudWatch Synthetics now supports the use of environment variables with canaries. This allows you to save time by using a single canary script to create different canaries that have a similar task. View the full article

Amazon CloudWatch Lambda Insights, now available in preview, enables you to monitor, troubleshoot, and optimize the performance of AWS Lambda functions. With this preview, you have access to automated dashboards summarizing the performance and health of your Lambda functions that provide visibility into issues such as memory leaks or performance changes caused by new function versions. View the full article

You can now send logs from AWS Lambda functions directly to a destination of your choice by using AWS Lambda Extensions. AWS Lambda Extensions are a new way for monitoring, observability, security, and governance tools to integrate with Lambda, and today, you can use extensions that send logs to the following providers: Datadog, New Relic, Sumo Logic, Honeycomb, Lumigo, and Coralogix. View the full article

Grafana is an open-source platform for monitoring and observability. It allows you to query, visualize, alert on and understand your metrics no matter where they are stored. View the full article

When using CloudWatch Synthetics, you can now get a snapshot of the canary health with a prebuilt monitoring dashboard. The new monitoring dashboard provides canary data trends over time for latency, availability, and error counts. The dashboard also provides expected latency based on historical data of the previous canary runs. This helps you spot anomalies sooner which in turn allows you to respond faster to ensure a better end user experience. View the full article

CloudWatch Synthetics now supports customizing the default launch settings on the Chrome browser with a new minor runtime version, syn-nodejs-2.1. This allows for more flexibility in the canary launched browser settings such as viewport, setting chromium flags, and handling errors. With syn-nodejs-2.1, you can also configure canary scripts to not take screenshots on a canary step, thereby reducing costs, and avoiding screenshots for sensitive data. View the full article

During the online GrafanaCONline conference today, Grafana Labs announced that version 8.0 of its open source visualization software widely employed by DevOps teams is now generally available. In addition, version 1.0 of Grafana Tempo, an open source back end platform for collecting distributed traces that are needed to drive observability across an application environment, is […] The post Grafana Labs Advances Open Source Visualization and Observability appeared first on DevOps.com. View the full article

CloudWatch Embedded Metric Format enables you to ingest complex high-cardinality application data in the form of logs and easily generate actionable metrics from them. It has traditionally been hard to generate actionable custom metrics from your ephemeral resources such as Lambda functions, and containers. By sending your logs in the Embedded Metric Format, you can now easily create custom metrics without having to instrument or maintain separate code, while gaining powerful analytical capabilities on your log data. View the full article

Amazon CloudWatch now gives operators and developers visibility and actionable insights into their utilization of control plane APIs across AWS services. You can find API call count metrics organized by AWS service in the CloudWatch console, and search and discover usage metrics from thousands available in the AWS/Usage namespace. View the full article

Amazon CloudWatch Resource Health is a new feature that enables you to automatically discover, manage, and visualize the health and performance of Amazon Elastic Compute Cloud (Amazon EC2) hosts across your applications in a single view. With Resource Health, you can visualize the health of your Amazon EC2 hosts in a map (or list) view by performance dimension such as CPU or Memory, and slice and dice hundreds of hosts using tags and available filters such as instance type, instance state, and status check. This helps in reducing your Mean time to resolution (MTTR) by easily isolating EC2 hosts that are performing sub-optimally. View the full article

Amazon CloudWatch Logs announces Dimension support for Metric Filters. CloudWatch Logs Metric Filters allow you to create filter patterns to search for and match terms, phrases, or values in your CloudWatch Logs log events, and turn these into metrics that you can graph in CloudWatch Metrics or use to create a CloudWatch Alarm. Now with Dimension support for Metric Filters you can create metrics from JSON or space-delimited logs with up to 3 dimensions, where a dimension is a key-value pair that is part of the identity of a metric. View the full article

CloudWatch Synthetics, a feature that supports monitoring your REST APIs, URLs, and website content every minute, 24/7, is now available in the AWS Asia Pacific (Osaka) Region. With CloudWatch Synthetics, you can continually verify your customer experience even when there is no customer traffic on your applications. This helps you discover issues before your customers do and react quickly to fix them. View the full article

You can now view and manage all Amazon CloudWatch Logs transactional API service quotas with Service Quotas. Service Quotas consolidates the default values and your account specific quotas for CloudWatch Logs in one single view with the Service Quotas console. With the CloudWatch Logs and Service Quotas integration you can now easily view and adjust your quotas. View the full article

Now you can easily setup monitoring, alarms and dashboards for your applications deployed in Amazon Elastic Container Service (ECS), Amazon Elastic Kubernetes Service (EKS) and Kubernetes on EC2 containers running on AWS with CloudWatch Application Insights. CloudWatch Application Insights is a capability that helps customers monitor and troubleshoot their enterprise applications running on AWS resources. The new feature adds monitoring tier options for capturing the metrics, telemetry and logs for monitoring the health and wellness of applications running in containers on AWS. View the full article

Amazon Elasticsearch Service now offers instances from the AWS Graviton2 instance family. Instance types include general purpose (M6g), compute optimized (C6g), and memory optimized (R6g, R6gd). Customers can enjoy up to 38% improvement in indexing throughput, 50% reduction in indexing latency, and 30% improvement in query performance when compared to the corresponding x86-based instances from the current generation (M5, C5, R5). View the full article

We’re excited to announce the launch of Amazon CloudWatch Monitoring Framework, a reference architecture that makes it easier for customers to set up Amazon CloudWatch dashboards to monitor Apache workloads running on AWS. View the full article

You can now publish the Redis slow log from your Amazon ElastiCache for Redis clusters to Amazon CloudWatch Logs and Amazon Kinesis Data Firehose. The Redis slow log provides visibility into the execution time of commands in your Redis cluster, enabling you to continuously monitor the performance of these operations. You can choose to send these logs in either JSON or text format to Amazon CloudWatch Logs and Amazon Kinesis Data Firehose. View the full article

Amazon Elasticsearch Service now supports open source Elasticsearch 7.10 and its corresponding version of Kibana. This minor release includes bug fixes and enhancements. View the full article

Amazon Elasticsearch Service now supports Asynchronous Search. Asynchronous Search lets you submit a query that gets executed asynchronously, monitor the progress of the request, and retrieve results at a later stage. You can also retrieve partial results as they become available even before the search has fully completed. Once the search completes, it can be stored for consumption at a later time up to an expiry duration. View the full article

Amazon Elasticsearch Service now supports integrating with Microsoft Power BI, a business analytics service that delivers insights to enable fast, informed decisions. Powered by the Open Distro for Elasticsearch ODBC Driver you can now integrate your Microsoft Power BI environment with you Amazon Elasticsearch Service domains using the Open Distro for Elasticsearch SQL Engine. View the full article

You can now use Amazon CloudWatch Lambda Insights to monitor, troubleshoot, and optimize the performance of AWS Lambda functions which are packaged and deployed as container images. With CloudWatch Lambda Insights you have access to automated dashboards summarizing the performance and health of your Lambda functions. View the full article

Amazon CloudWatch Logs now supports two subscription filters per log group, enabling you to deliver a real-time feed of log events from CloudWatch Logs to an Amazon Kinesis Data Stream, Amazon Kinesis Data Firehose, or AWS Lambda for custom processing, analysis, or delivery to other systems. View the full article

Amazon Elasticsearch Service now supports automated memory management of Elasticsearch clusters with the new Auto-Tune feature. Auto-Tune is an adaptive resource management system that automatically adjusts Elasticsearch internal settings to handle dynamic workloads, optimizing cluster resources to improve efficiency and performance. With Auto-Tune, you can achieve performance boost in ingestion throughput for log analytics workloads, and reduced tail latencies for search queries. View the full article

Amazon Elasticsearch Service now publishes events to Amazon CloudWatch and Amazon EventBridge to provide better visibility into the service. Events to indicate the availability of a service software update for a domain, the start of an update, and the completion of an update will be included in the initial release. You will also be able to view these events under the new ‘Notifications’ view in the Amazon Elasticsearch Service console. View the full article

Amazon Elasticsearch Service now supports tag-based authorization for easy management of access to configuration APIs that are used for operations such as creating, modifying, or updating Amazon Elasticsearch Service domains. View the full article

CloudWatch Synthetics now supports storing your canary run artifacts, including log files, screenshots, and HAR files, in an Amazon Simple Storage Service (S3) bucket in another Region with a new major runtime version, syn-nodejs-puppeteer-3.0. CloudWatch Synthetics now also supports upgraded major versions of the Puppeteer, Chromium, and Node.js dependencies. View the full article

DevOps introduced more automation to the software development process and lifecycle, allowing new applications to be on the market at a quicker pace. But with progress comes changes and new requirements for developing, testing, and deploying applications, thus requiring transformation for modern monitoring systems. Monitoring provides feedback from production and delivers information about an application’s performance and usage patterns. When performance or other issues arise, relevant data about the issues are sent back to development teams through automated monitoring... The post The Importance of Monitoring in DevOps appeared first on DevOps Online. View…

Amazon CloudWatch Contributor Insights for Amazon DynamoDB now supports AWS CloudFormation, enabling you to manage Contributor Insights settings for DynamoDB with CloudFormation templates. View the full article