Monitoring & Observability

We are pleased to announce Amazon OpenSearch Serverless now offers improved security options for workloads with the support of Transport Layer Security (TLS) version 1.3. OpenSearch Serverless is the serverless option for Amazon OpenSearch Service that makes it simpler for you to run search and analytics workloads without having to think about infrastructure management. View the full article

Amazon OpenSearch Service now lets you update cluster volume size, volume type, IOPS and throughput without requiring a blue/green deployment. This makes it easier for you to make changes to your EBS settings without having to plan upfront for a blue/green deployment. View the full article

AWS AppSync is a fully managed service that allows customers to connect applications to data and events with GraphQL APIs. AppSync allows customers to create APIs that connect to multiple data sources like microservice APIs, relational databases, and NoSQL databases. With AppSync APIs, applications can efficiently fetch data from different sources in one request. View the full article

Amazon Managed Blockchain (AMB) Query now supports Amazon CloudWatch usage metrics, enabling customers to monitor their AMB Query API usage. View the full article

Amazon CloudWatch Synthetics announces release of Synthetics NodeJS Runtime versions - syn-nodejs-puppeteer-6.2, syn-nodejs-puppeteer-5.2 - and Python Runtime version - syn-python-selenium-2.1. This release brings updated Chromium dependency libs for forward compatibility with Lambda OS and adds new Lambda Ephemeral Storage usage metric in customer account. To learn more, see release notes. View the full article

Amazon OpenSearch Service now provides improved visibility into the progress of domain updates. You can see granular status values representing different stages of an update, simplifying monitoring and automation of configuration changes. View the full article

Starting today, AWS Elemental MediaConnect offers additional output metrics for SRT and MediaLive outputs to improve video stream monitoring. The new metrics are total packets sent, Forward Error Correction (FEC) packets, Automatic Repeat Requests (ARQ), number of resent packets, number of packets not recovered, round trip time, and bitrate for each output from a MediaConnect flow. View the full article

Platform Engineering is a hot topic these days. We’ve seen the hype around it in 2023, and I expect we shall see it becoming production-grade as we move into 2024. I wanted to look into this topic, and learn from those who’ve already implemented it at scale: the e-commerce hyperscaler Shopify. In the latest episode […]View the full article

At Logz.io, we recently announced the release of App 360, a new solution that aims to shift the paradigm around application performance monitoring (APM) systems. To better give our customers a look at the new solution within the Logz.io Open 360™ platform for essential observability, we recently hosted a webinar explaining App 360 in greater […]View the full article

Amazon CloudWatch Logs is excited to announce support for creating account-level subscription filters using the put-account-policy API. This new capability enables you to deliver real-time log events that are ingested into Amazon CloudWatch Logs to an Amazon Kinesis Data Stream, Amazon Kinesis Data Firehose, or AWS Lambda for custom processing, analysis, or delivery to other destinations using a single account level subscription filter. View the full article

OpenSearch Service 2.11 now supports hybrid query score normalization. It is now easier than ever for search practitioners to leverage a combination of lexical and semantic search to improve their search relevance with OpenSearch. View the full article

In the ever-evolving landscape of software development and IT operations, monitoring tools play a pivotal role in ensuring the performance, reliability, and availability of your applications. Two key disciplines in this domain are observability and Application Performance Management (APM). This post will help you understand the nuances between observability and APM, exploring their unique characteristics, […]View the full article

Amazon OpenSearch Service now offers support for the Amazon Graviton2 instance family in six additional regions- Africa (Cape Town), Asia Pacific (Osaka), Europe (Zurich), Middle East (Bahrain). Israel (Tel Aviv), and AWS GovCloud (US-West). Graviton-based instances (C6g/M6g/R6g) in OpenSearch Service provide up to 30% better price-performance than comparable x86-based (C5/M5/R5) Amazon Elastic Compute Cloud instances. View the full article

Amazon OpenSearch Service adds support for Transport Layer Security (TLS) version 1.3 amongst its transport security options for domain endpoint security. TLS 1.3 offers customers enhanced security and performance as compared to older TLS versions. In addition, we now support perfect forward secrecy, which provides additional safeguards against eavesdropping of encrypted data, through the use of a unique random session key. View the full article

Introduction We have observed a growing adoption of container services among both startups and established companies. This trend is driven by the ease of deploying applications and migrating from on-premises environments to the cloud. One platform of choice for many of our customers is Amazon Elastic Container Service (Amazon ECS). The powerful simplicity of Amazon ECS allows customers to scale from managing a single task to overseeing their entire enterprise application portfolio and to reach thousands of tasks. Amazon ECS eliminates the management overhead associated with running your own container orchestration service. When working with customers, we have observed …

With 2023 drawing to a close, the final OpenObservability Talks of the year focused on what happened this year in open source, DevOps, observability and more, with an eye towards the future. I was delighted to be joined by a special guest, Kelsey Hightower, a renowned figure in the tech community, especially known for his […]View the full article

For businesses reliant on customers’ positive digital experiences to achieve their goals, the seamless operation of cloud applications and infrastructure is paramount for financial success. Observability holds a pivotal role in modern enterprises, offering critical insights into your IT system’s health and performance. However, persistent issues of complexity and high costs have plagued the observability […]View the full article

Amazon OpenSearch Service adds multimodal support on Neural Search for OpenSearch 2.11 deployments. This empowers builders to create and operationalize multimodal search applications with significantly reduced undifferentiated heavy-lifting. For years, customers have been building vector search applications on OpenSearch k-NN, but they’ve been burdened with building middleware to integrate text embedding models into search and ingest pipelines. OpenSearch builders can now power multimodal search through out-of-the-box integrations with Amazon Bedrock text and image multimodal APIs to power search pipelines that run on-cluster. View the full article

Post co-written by Shahar Azulay, CEO and Co-Founder at GroundCover Introduction The abstraction introduced by Kubernetes allows teams to easily run applications at varying scale without worrying about resource allocation, autoscaling, or self-healing. However, abstraction isn’t without cost and adds complexity and difficulty tracking down the root cause of problems that Kubernetes users experience. To mitigate these issues, detailed observability into each application’s state is key but can be challenging. Users have to ensure they’re exposing the right metrics, emitting actionable logs, and instrumenting their application’s code with specific client-side libraries …

As we continue to navigate the ongoing evolution of the observability landscape, Logz.io is constantly striving to provide our customers with the advanced platform capabilities needed to make sense of their increasingly complex environments. Sometimes that means taking a new approach to long-standing practices. For years, organizations have relied on application performance monitoring (APM) tools […]View the full article

The Migration Assistant for Amazon OpenSearch Service is an open-source AWS solution designed for the effortless migration of self-managed OpenSearch and Elasticsearch clusters to Amazon OpenSearch Service (Managed clusters and Serverless collections). View the full article

Years before founding Logz.io, I was a software engineer, working with various tools to ensure my products and services performed correctly. There were few tools I dreaded using more than application performance management (APM), and I know that I’m not alone. I hated traditional APM. It’s heavy. It’s hard to implement. It’s expensive. It takes […]View the full article

As applications in the cloud become more distributed and complex, the Mean Time To Resolution (MTTR) for production issues is getting longer. Modern systems are built with hundreds of distinct, ephemeral, and interconnected cloud components, which can make it exceptionally hard for engineers to understand the current state of their applications, what problems are impacting […]View the full article

We are excited to announce regular expression support for Amazon CloudWatch Logs filter pattern syntax, making it easier to search and match relevant logs in the AWS GovCloud (US) Regions. Customers use filter pattern syntax today to search logs, extract metrics using metric filters, and send specific logs to other destinations with subscription filters. With today’s launch, customers will be able to further customize these operations to meet their needs with flexible and powerful regular expressions within filter patterns. Now customers can define one filter to match multiple IP subnets or HTTP status codes using a regular expression such as ‘{ $.statusCode=%4[0-9]{2}% }…

Amazon OpenSearch Service introduces OR1, the OpenSearch Optimized Instance family, that delivers up to 30% price-performance improvement over existing instances in internal benchmarks and uses Amazon S3 to provide 11 9s of durability. The new OR1 instances are best suited for indexing-heavy workloads, and offers better indexing performance compared to the existing memory optimized instances available on OpenSearch Service. View the full article

You can use the new OR1 instances to create Amazon OpenSearch Service clusters that use Amazon Simple Storage Service (Amazon S3) for primary storage. You can ingest, store, index, and access just about any imaginable amount of data, while also enjoying a 30% price/performance improvement over existing instance types, eleven nines of data durability, and a zero-time Recovery Point Objective (RPO). You can use this to perform interactive log analytics, monitor application in real time, and more. New OR1 Instances These benefits are all made possible by the new OR1 instances, which are available in eight sizes and used for the data nodes of the cluster: Ins…

Amazon OpenSearch Service zero-ETL integration with Amazon S3, a new way for customers to query operational logs in Amazon S3 and S3-based data lakes without needing to switch between tools to analyze operational data, is available for customer preview. Customers can boost the performance of their queries and build fast-loading dashboards using the built-in query acceleration capabilities of Amazon OpenSearch Service zero-ETL integration with Amazon S3. View the full article

Today, AWS announces the general availability of vector engine for Amazon OpenSearch Serverless. Vector engine for OpenSearch Serverless is a simple, scalable, and high-performing vector database which makes it easier for developers to build machine learning (ML)–augmented search experiences and generative artificial intelligence (AI) applications without having to manage the underlying vector database infrastructure. Developers can rely on the vector engine's cost-efficient, secure, and mature serverless platform to seamlessly transition from application prototyping to production. View the full article

Today we are announcing a preview of Amazon OpenSearch Service zero-ETL integration with Amazon S3, a new way to query operational logs in Amazon S3 and S3-based data lakes without needing to switch between services. You can now analyze infrequently queried data in cloud object stores and simultaneously use the operational analytics and visualization capabilities of OpenSearch Service. Amazon OpenSearch Service direct queries with Amazon S3 provides a zero-ETL integration to reduce the operational complexity of duplicating data or managing multiple analytics tools by enabling customers to directly query their operational data, reducing costs and time to action. This zer…

Today we are announcing the general availability of the vector engine for Amazon OpenSearch Serverless with new features. In July 2023, we introduced the preview release of the vector engine for Amazon OpenSearch Serverless, a simple, scalable, and high-performing similarity search capability. The vector engine makes it easy for you to build modern machine learning (ML) augmented search experiences and generative artificial intelligence (generative AI) applications without needing to manage the underlying vector database infrastructure. You can now store, update, and search billions of vector embeddings with thousands of dimensions in milliseconds. The highly performant…

Amazon DynamoDB zero-ETL integration with Amazon OpenSearch Service provides customers advanced search capabilities, such as full-text and vector search, on their Amazon DynamoDB data. With a few button clicks in the AWS console, customers can now seamlessly synchronize their data from Amazon DynamoDB to Amazon OpenSearch Service, eliminating the need to write any custom code to extract, transform, and load the data. Amazon DynamoDB zero-ETL integration with Amazon OpenSearch Service is now available for both Amazon OpenSearch Service managed clusters and serverless collections. View the full article

Today, we are announcing the general availability of Amazon DynamoDB zero-ETL integration with Amazon OpenSearch Service, which lets you perform a search on your DynamoDB data by automatically replicating and transforming it without custom code or infrastructure. This zero-ETL integration reduces the operational burden and cost involved in writing code for a data pipeline architecture, keeping the data in sync, and updating code with frequent application changes, enabling you to focus on your application. With this zero-ETL integration, Amazon DynamoDB customers can now use the powerful search features of Amazon OpenSearch Service, such as full-text search, fuzzy search…

Logz.io is excited to announce Service Map, a new way to visualize the data flow, dependencies, and critical performance metrics throughout your microservices architecture, which makes it easy to gather critical troubleshooting context as you investigate production issues. After sending your trace data to Logz.io Open 360™, Service Map automatically discovers and maps your services […]View the full article

To make it easy to interact with your operational data, Amazon CloudWatch is introducing today natural language query generation for Logs and Metrics Insights. With this capability, powered by generative artificial intelligence (AI), you can describe in English the insights you are looking for, and a Logs or Metrics Insights query will be automatically generated. This feature provides three main capabilities for CloudWatch Logs and Metrics Insights: Generate new queries from a description or a question to help you get started easily. Query explanation to help you learn the language including more advanced features. Refine existing queries using guided iteration…

Searching through log data to find operational or business insights often feels like looking for a needle in a haystack. It usually requires you to manually filter and review individual log records. To help you with that, Amazon CloudWatch has added new capabilities to automatically recognize and cluster patterns among log records, extract noteworthy content and trends, and notify you of anomalies using advanced machine learning (ML) algorithms trained using decades of Amazon and AWS operational data. Specifically, CloudWatch now offers the following: The Patterns tab on the Logs Insights page finds recurring patterns in your query results and lets you analyze them…

Here at Logz.io, we realize Kubernetes is the most common infrastructure component that organizations are running on to keep their applications going. In return, we’ve made a big investment to support Kubernetes properly and give customers the tools they need to investigate and troubleshoot any issues that arise. Check out the replay of our webinar […]View the full article

OpenSearch Service 2.11 now comes with OpenSearch Neural Sparse Retrieval. Search practitioners now have an additional search method to use for their search applications with improved semantic understanding, while keeping computational cost and computational latency low, more in line with lexical search. View the full article

Amazon OpenSearch Service now supports Neural Search on OpenSearch 2.9, enabling builders to create and operationalize semantic search applications with reduced undifferentiated heavy-lifting. For years, customers have been building semantic search applications on OpenSearch k-NN, but they’ve been burdened with building middleware to integrate text embedding models into search and ingest pipelines. Amazon OpenSearch Service customers can power Neural Search through integrations with Amazon SageMaker and Amazon Bedrock enabling semantic search pipelines that run on-cluster. View the full article

When things go wrong, we’d all love the ability to go back in time, return things to the way they were, and fix whatever issues pop up at the start so they never happen in the first place. This is no different when maintaining complex microservices-based architectures. With any complex system, things are bound to […]View the full article

Do you find yourself lying awake late at night, worried that your greatest observability fears will materialize as one of the most horrific specters of Kubernetes-driven chaos reaches up through your mattress to consume your very soul? Even as your mind races and you wonder just who that creepy character sneaking around the metaphysical boiler […]View the full article

In the first part of our 2023 PromCom recap, we spent OpenObservability Talks exploring the Perses open source project. We found heavy users of open source Grafana who found themselves grappling with issues arising from managing a vast number of dashboards, and the need to manage dashboards as code in a GitOps fashion. In this […]View the full article

Amazon OpenSearch Service adds support for four new language analyzer plugins- Nori (Korean), Sudachi (Japanese), Pinyin (Chinese), and STConvert Analysis (Chinese) plugins. These are available as optional plugins that you can associate with your OpenSearch Service clusters. View the full article

Amazon CloudWatch announces out-of-the box, best practice alarm recommendations for AWS service-vended metrics. It provides alarm recommendations and alarm configurations for key vended metrics, along with the ability to download pre-filled infrastructure-as-code templates for these alarms. Furthermore, you can now see in-line descriptions for AWS service metrics across the AWS console, which enables you to easily see metric details to help you troubleshoot or assess system health. View the full article

Search Pipelines, a new feature in OpenSearch 2.9, make it easy to build query and result processing pipelines. This lets you build search query and result processing as a composition of modular processing steps without complicating your application software. View the full article

Starting today, we are introducing a new Amazon CloudWatch metric called Attached EBS Status Check to monitor if one or more Amazon EBS volumes attached to your EC2 instances are reachable and able to complete I/O operations. With this new metric, you can now quickly detect and respond to any EBS impairments that may potentially be impacting the performance of your applications running on Amazon EC2 instances. View the full article

You can now launch Amazon CloudWatch Internet Monitor directly from the Amazon Virtual Private Cloud (VPC) console. Internet Monitor provides visibility into how internet issues impact the performance and availability for your application hosted on AWS. To use Internet Monitor, you create a monitor and associate it with one or more resources: VPCs, Network Load Balancers, Amazon CloudFront distributions, or Amazon WorkSpaces directories. View the full article

OpenSearch Service 2.9 now supports the ability for customers to manage and overlay alerts and anomalies onto dashboard visualization line charts. Customers can create new, or associate existing, alerting monitors and anomaly detectors from dashboard line charts. For example, if a customer opts to create a new monitor or detector, the new monitor will inherit the setting of the line chart and prepopulate the creation form. If customers have existing monitors or detectors present that they want to use, they can associate them with the line chart visualization on the dashboard. View the full article

OpenSearch Service 2.9 now comes with OpenSearch Service Integrations, where customers can take advantage of new schema standards such as Open Telemetry and build dashboards based on an agreed up on schema between your ingestion pipeline and OpenSearch Service. View the full article

You can now run OpenSearch version 2.9 in Amazon OpenSearch Service. With OpenSearch 2.9, we have made several improvements to Search, Observability, Security analytics, and Machine Learning (ML) capabilities in OpenSearch Service. View the full article

We are excited to announce that the Amazon OpenSearch Service has expanded its geospatial capacities. With new aggregation support in version 2.9, you can do more statistical analysis on your data, making it simpler to draw conclusions and interpret them. View the full article