Monitoring & Observability

Amazon OpenSearch Service now provides new Auto-Tune metrics and improved Auto-Tune events that give you better visibility into the cluster performance optimizations made by Auto-Tune. View the full article

Customers can now monitor and troubleshoot application environments that span multiple accounts (within a region) using Amazon CloudWatch Application Insights. CloudWatch Application Insights helps customers gain actionable insights for their application environments by making it easier for them to set up and monitor, recognize problems, and use data to make decisions. With this release, customers can analyze and correlate cross-account telemetry data and problems through a centralized view of the monitoring results across their accounts. View the full article

What are the Grafana offerings for modernizing applications on AWS? View the full article

Amazon CloudWatch announces support of a new Metric Math function called DB_PERF_INSIGHTS() to create CloudWatch alarms and dashboards on Amazon RDS Performance Insights metrics. View the full article

Logz.io is thrilled to have earned over 20 Fall 2023 G2 Badges for our Logz.io Open 360™ essential observability platform! G2 Research is a tech marketplace where people can discover, review, and manage the software they need to reach their potential. We’ve earned the following Fall 2023 G2 Badges for Application Performance Monitoring (APM) and […]View the full article

Amazon CloudWatch Synthetics announces a new update to Synthetics Python runtime version syn-python-selenium-2.0 and recommends that customers migrate Synthetics canaries to the latest runtime version. Runtime version syn-python-selenium-2.0 includes updates to third-party dependency packages (Selenium v4.10.0 and Chromium v111.0.5563.146). View the full article

We are excited to announce that Amazon OpenSearch Serverless has expanded its auto-scaling capabilities to efficiently handle tens of thousands of query transactions per minute. With this new feature, you can rely on OpenSearch Serverless to help handle unpredictable surges in your search and query traffic. View the full article

We are excited to announce regular expression support for Amazon CloudWatch Logs filter pattern syntax, making it easier to search and match relevant logs. Customers use filter pattern syntax today to search logs, extract metrics using metric filters, and send specific logs to other destinations with subscription filters. With today’s launch, customers will be able to further customize these operations to meet their needs with flexible and powerful regular expressions within filter patterns. Now customers can define one filter to match multiple IP subnets or HTTP status codes using a regular expression such as ‘{ $.statusCode=%4[0-9]{2}% }’ rather than having to define mu…

We are excited to announce the integration of Amazon OpenSearch Serverless with AWS User Notifications. OpenSearch Serverless is the serverless option for Amazon OpenSearch Service that makes it simple for you to run search and analytics workloads without having to think about infrastructure management. View the full article

We are excited to announce that Amazon OpenSearch Serverless can now scan and search up to 6TB of time series data which includes one or more indexes within a collection. OpenSearch Serverless is a serverless deployment option for Amazon OpenSearch Service that makes it simple for you to run search and analytics workloads without having to think about infrastructure management. With the support for larger datasets, you can unlock valuable operational insights and make data driven decisions to troubleshoot application downtime, improve system performance, or identify fraudulent activities. View the full article

Amazon Personalize launches a new integration with self-managed OpenSearch that enables customers to personalize search results for each user and assists in predicting their search needs. The Amazon Personalize Search Ranking plugin within OpenSearch helps customers to leverage the deep learning capabilities offered by Amazon Personalize and add personalization to OpenSearch search results, without any ML expertise. View the full article

You can now run OpenSearch version 2.7 in Amazon OpenSearch Service. With OpenSearch 2.7, we have made several improvements to observability, security analytics, index management, and geospatial capabilities in OpenSearch Service. View the full article

Amazon OpenSearch Service now lets you provision up to 16,000 IOPS and 1000 MiB/s throughput for every 3 TiB gp3 volume size provisioned per data node, enabling you to achieve better search and indexing performance. View the full article

Amazon OpenSearch Service now lets you update cluster manager (master node) instance type or instance count without requiring a blue/green deployment, helping you complete the updates faster with the least potential disruption to your cluster operations and without involving any data movement. View the full article

Amazon OpenSearch Ingestion now allows you to ingest events from Amazon Security Lake in real-time, reducing the time taken to index your security data in Amazon OpenSearch Service and uncover valuable insights into potential security issues. Amazon Security Lake automatically centralizes security data from AWS environments, SaaS providers and on- premises into a purpose-built data lake. With this integration, customers can now use the extensive security analytics capabilities and rich dashboard visualizations of Amazon OpenSearch Service to quickly make sense of all their security data. View the full article

Amazon Managed Grafana now supports Trace Analytics with the OpenSearch Grafana data source plugin, in addition to the existing support for Log Analytics. Amazon Managed Grafana is a fully managed service for Grafana, a popular open-source analytics platform that enables you to query, visualize, and alert on your metrics, logs, and traces. OpenSearch is an open-source search and analytics engine for use cases such as log analytics, observability, real-time application monitoring, and clickstream analysis that is also offered as a fully managed Amazon OpenSearch Service. View the full article

Amazon CloudWatch Logs is excited to announce a new Logs Insights command, dedup, which enables customers to eliminate duplicate results when analyzing logs. Customers frequently want to query their logs and view only unique results based on one or more fields. You can now use the new dedup command in your Amazon CloudWatch Logs Insights queries to view unique results based on one or more fields. For example, you can view the most recent error message for each hostname by executing the dedup command on the hostname field. View the full article

Amazon OpenSearch Service now supports a new ‘skip unavailable’ setting for cross cluster search connections. If skip unavailable is enabled on connections, cross-cluster search ignores any remote cluster that might not available during the search. View the full article

Amazon CloudWatch Logs is excited to announce support for account level data protection policy configuration, you can now create a data protection policy that will be applied to all existing and future log groups within your AWS account. View the full article

We are excited to announce Amazon CloudWatch Logs Live Tail, a new interactive log analytics experience feature that helps you detect and debug anomalies in applications. You can now view your logs interactively in real-time as they’re ingested, which helps you to analyze and resolve issues across your systems and applications. View the full article

Amazon Kinesis Data Firehose customers can now send data to Amazon OpenSearch Service using OpenSearch Service auto-generated document ID option. This configuration option enables write-heavy operations, such as log analytics and observability, to consume fewer CPU resources at the OpenSearch domain, resulting in improved performance. View the full article

Today, Amazon OpenSearch Service announces Multi-AZ with Standby, a new deployment option that enables 99.99% availability and consistent performance for business-critical workloads. With Multi-AZ with Standby, OpenSearch Service domains are resilient to potential infrastructure failures, such as a node or an Availability Zone (AZ) failure. Multi-AZ with Standby also ensures OpenSearch Service domains follow recommended best practices, simplifying configuration and management. View the full article

We are excited to announce Amazon CloudWatch Logs data protection is now available in Middle East (UAE), Asia Pacific (Hyderabad), Europe (Spain), Europe (Zurich), and Asia Pacific (Melbourne). Data protection is a feature that leverages pattern matching and machine learning capabilities to detect and protect sensitive log data-in-transit. Amazon CloudWatch Logs enables you to centralize the logs from all of your systems, applications, and AWS services, in a single, highly scalable service. With log data protection in Amazon CloudWatch Logs, you can now detect and protect sensitive log data in-transit such as, Credit Card Numbers or Government ID’s logged by your systems,…

Today Amazon OpenSearch Service announces Amazon OpenSearch Ingestion, a new fully managed data ingestion tier that allows you to ingest and process petabyte-scale data before indexing it in OpenSearch-managed clusters or serverless collections. OpenSearch Ingestion provides a no-code capability to filter, transform, redact, and route data prior to indexing it in OpenSearch. OpenSearch Ingestion automatically provisions and scales the underlying resources for even the most demanding and unpredictable workloads. View the full article

Today we launched new observability features including log patterns, metrics analytics and support for Jaeger traces with OpenSearch 2.5 in Amazon OpenSearch Service. View the full article

Amazon CloudWatch Logs now supports ingesting enriched metadata introduced in Amazon Virtual Private Cloud (Amazon VPC) flow logs as part of versions 3 to 5 additional to the default fields. This launch includes metadata fields that provide more insights about the network interface, traffic type, and the path of egress traffic to the destination. View the full article

Amazon OpenSearch Service announces security analytics that provides new threat monitoring, detection, and alerting features. These capabilities help you to detect and investigate potential security threats that may disrupt your business operations or pose a threat to sensitive organizational data. View the full article

You can now run OpenSearch and OpenSearch Dashboards version 2.5 in Amazon OpenSearch Service. With OpenSearch 2.5, OpenSearch Service adds several new features and enhancements such as support for Security Analytics, support for Point in Time Search, improvements to observability and geospatial functionality. View the full article

We are excited to announce support for the Amazon Graviton2 instance family in four additional regions. Supported instance types include compute optimized (C6g), general purpose (M6g), and memory optimized (R6g) instances. Support for C6g, M6g, and R6g is available in the Asia Pacific (Hyderabad), Asia Pacific (Jakarta), Asia Pacific (Melbourne), and Europe (Milan) regions. View the full article

Amazon OpenSearch Service now lets you schedule service software and auto-tune updates during off-peak hours, helping you plan deployments to your domain better. In addition, with improved notifications through EventBridge events, and notifications on the OpenSearch Service console, you have better visibility of scheduled updates, when the updates start, and complete. View the full article

Amazon OpenSearch Service now supports enabling Security Assertion Markup Language (SAML) authentication for OpenSearch Dashboards during domain creation. SAML authentication for OpenSearch Dashboards enables users to integrate directly with identity providers (IDPs) such as Okta, Ping Identity, OneLogin, Auth0, Active Directory Federation Services (ADFS) and Azure Active Directory. View the full article

AWS IoT Core announces General Availability of the capability to send device logs from Internet of Things (IoT) devices to Amazon CloudWatch Logs in batches, enabling you to optimize the cost of using CloudWatch Log Action in IoT Rules. View the full article

Amazon OpenSearch Service adds a new connection mode for cross-cluster connection, simplifying the setup required to remote reindex between a local domain and remote VPC domains. Remote reindex enables you to migrate data from a source domain to a target domain. Remote reindex is also useful when you have to upgrade your clusters across multiple major versions. View the full article

Now generally available, Amazon OpenSearch Serverless is a new serverless option for Amazon OpenSearch Service. OpenSearch Serverless streamlines the process of running petabyte-scale search and analytics workloads without having to configure, manage, or scale OpenSearch clusters. OpenSearch Serverless automatically provisions and scales the underlying resources to deliver fast data ingestion and query responses for even the most demanding and unpredictable workloads. With OpenSearch Serverless, you pay only for the resources consumed. View the full article

Amazon OpenSearch Service now lets you validate configuration changes before applying them to your clusters. With the enhanced dry run option, Amazon OpenSearch Service checks for validation errors that might occur when deploying your configuration changes and provides a summary of these errors, if any. The dry run feature will also indicate whether a blue/green deployment will be required to apply a change, so that you can plan for these changes accordingly. View the full article

Starting today, Amazon CloudWatch Logs is removing the 5 requests per second log stream quota when calling Amazon CloudWatch Logs PutLogEvents API. There will be no new per log stream quota. With this change we have removed the need for splitting your log ingestion across multiple log streams to prevent log stream throttling. View the full article

We are excited to announce support for the Amazon Graviton2 instance family in four additional regions. Supported instance types include general purpose (M6g), compute optimized (C6g), and memory optimized (R6g and R6GD) instances. Support for C6g, M6g, R6g and R6gd is available in the Europe (Paris) region. In addition, the Asia Pacific (Mumbai), South America (Sao Paulo), and Canada (Central) regions already supported C6g, M6g and R6g instance families, and we have now added support for R6gd in these regions. View the full article

Amazon Kinesis Data Firehose can now deliver streaming data to an Amazon OpenSearch Serverless. With few clicks, you can easily ingest, transform, and reliably deliver streaming data into an Amazon OpenSearch Serverless without building and managing your own data ingestion and delivery infrastructure. Kinesis Data Firehose is a fully managed service that automatically scales to match the throughput of your data and without ongoing administration. View the full article

Amazon OpenSearch Service now offers a new serverless option, Amazon OpenSearch Serverless. This option simplifies the process of running petabyte-scale search and analytics workloads without having to configure, manage, or scale OpenSearch clusters. OpenSearch Serverless automatically provisions and scales the underlying resources to deliver fast data ingestion and query responses for even the most demanding and unpredictable workloads. With OpenSearch Serverless, you pay only for the resources consumed. View the full article

Most AWS analytics services have compelling serverless offerings that make it even easier for customers to analyze vast amounts of data without having to configure, scale, or manage the underlying infrastructure. Along with other serverless analytics, such as Amazon QuickSight for business intelligence and AWS Glue for data integration, we have introduced Amazon EMR Serverless, Amazon MSK Serverless, and Amazon Redshift Serverless this year. Today, we announce the preview release of a new serverless option for Amazon OpenSearch Service that makes it easy for customers to run large-scale search and analytics workloads without managing clusters. It automatically provisi…

Today we are announcing Amazon CloudWatch Logs data protection, a new set of capabilities for Amazon CloudWatch Logs that leverage pattern matching and machine learning (ML) to detect and protect sensitive log data in transit. While developers try to prevent logging sensitive information such as Social Security numbers, credit card details, email addresses, and passwords, sometimes it gets logged. Until today, customers relied on manual investigation or third-party solutions to detect and mitigate sensitive information from being logged. If sensitive data is not redacted during ingestion, it will be visible in plain text in the logs and in any downstream system that consu…

We are excited to announce data protection in Amazon CloudWatch Logs, a new set of capabilities that leverage pattern matching and machine learning capabilities to detect and protect sensitive log data-in-transit. Amazon CloudWatch Logs enables you to centralize the logs from all of your systems, applications, and AWS services, in a single, highly scalable service. With log data protection in Amazon CloudWatch Logs, you can now detect and protect sensitive log data-in-transit logged by your systems, and applications. View the full article

Amazon OpenSearch Service now supports OpenSearch and OpenSearch Dashboards version 2.3. With this version, Amazon OpenSearch Service adds several features such as new algorithms to the machine learning (ML) commons library, improvements to aggregations, improvements to map visualizations, alerting, anomaly detection, and more. View the full article

Amazon CloudWatch Logs now supports exporting logs to Amazon Simple Storage Service (S3) buckets encrypted using Server side encryption with KMS (SSE-KMS) keys. View the full article

Amazon OpenSearch Service now supports managed VPC endpoints (powered by AWS PrivateLink) to connect to your Amazon OpenSearch Service VPC-enabled domain in a Virtual Private Cloud (VPC). With an Amazon OpenSearch Service managed endpoint, you can now privately access your OpenSearch Service domain within your VPC from your client applications in other VPCs, within the same or across AWS accounts, without using public IPs or requiring traffic to traverse the Internet. View the full article

OpenSearch is an open-source search and analytics suite. Developers build solutions for search, data observability, data ingestion and more using OpenSearch. Another popular use case is log analytics. You take the logs from applications, servers and network elements, feed them into OpenSearch, and use the rich search and visualisation functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to spot unless you have a real-time graph of all the HTTP status codes the server has thrown in the past twenty-four hours. You can use OpenSearch Dashboards to build these kinds of visualisations from data i…

Amazon Web Services (AWS) is pleased to announce the new Amazon OpenSearch Service Delivery specialization for AWS Partners that help customers perform interactive log analytics, real-time application monitoring, website search, and more. Amazon OpenSearch Service manages software installation, upgrades, patching, scaling (up to 3 PB), and cross-region replication with no downtime. Amazon OpenSearch Service is also bundled with a dashboard visualization tool, OpenSearch Dashboards. This tool helps visualize not only log and trace data, but also machine-learning powered results for anomaly detection and search relevance ranking. View the full article

Amazon Managed Service for Prometheus now provides Alert Manager & Ruler logs to help customers troubleshoot their alerting pipeline and configuration in Amazon CloudWatch Logs. Amazon Managed Service for Prometheus is a fully managed Prometheus-compatible monitoring service that makes it easy to monitor and alarm on operational metrics at scale. Prometheus is a popular Cloud Native Computing Foundation open source project for monitoring and alerting that is optimized for container environments. The Alert Manager allows customers to group, route, deduplicate, and silence alarms before routing them to end users via Amazon Simple Notification Service (Amazon SNS). The R…

Amazon OpenSearch Service now provides improved visibility into validation failures during domain updates. You can monitor the progress of a domain update, which could involve a blue/green deployment, from the OpenSearch Service console, or through the configuration APIs. OpenSearch Service will publish any validation failure events to Amazon EventBridge. You can also view these validation events in the Notifications tab of the OpenSearch Service console. View the full article

Starting today, customers of AWS Cost Anomaly Detection will see a new interface in the console, where they view and analyze anomalies and their root causes. AWS Cost Anomaly Detection monitors customers’ spending patterns to detect and alert on anomalous (increased) spend, and to provide root cause analyses. View the full article