Search the Community

Observe Inc. added a Trace Explorer tool to its observability platform that simplifies search, analysis and visualization of billions of traces.View the full article

We are excited to announce the general availability of Snowflake Event Tables for logging and tracing, an essential feature to boost application observability and supportability for Snowflake developers. In our conversations with developers over the last year, we’ve heard that monitoring and observability are paramount to effectively develop and monitor applications. But previously, developers didn’t have a centralized, straightforward way to capture application logs and traces. Enter the new Event Tables feature, which helps developers and data engineers easily instrument their code to capture and analyze logs and traces for all languages: Java, Scala, JavaScript, Python and Snowflake Scripting. With Event Tables, developers can instrument logs and traces from their UDFs, UDTFs, stored procedures, Snowflake Native Apps and Snowpark Container Services, then seamlessly route them to a secure, customer-owned Event Table. Developers can then query Event Tables to troubleshoot their applications or gain insights into performance and code behavior. Logs and traces are collected and propagated via Snowflake’s telemetry APIs, then automatically ingested into your Snowflake Event Table. Logs and traces are captured in the active Event Table for the account. Simplify troubleshooting in Native Apps Event Tables are also supported for Snowflake Native Apps. When a Snowflake Native App runs, it is running in the consumer’s account, generating telemetry data that’s ingested into their active Event Table. Once the consumer enables event sharing, new telemetry data will be ingested into both the consumer and provider Event Tables. Now the provider has the ability to debug the application that’s running in the consumer’s account. The provider only sees the telemetry data that is being shared from this data application—nothing else. For native applications, events and logs are shared with the provider only if the consumer enables Event sharing. Improve reliability across a variety of use cases You can use Event Tables to capture and analyze logs for various use cases: As a data engineer building UDFs and stored procedures within queries and tasks, you can instrument your code to analyze its behavior based on input data. As a Snowpark developer, you can instrument logs and traces for your Snowflake applications to troubleshoot and improve their performance and reliability. As a Snowflake Native App provider, you can analyze logs and traces from various consumers of your applications to troubleshoot and improve performance. Snowflake customers ranging from Capital One to phData are already using Event Tables to unlock value in their organization. “The Event Tables feature simplifies capturing logs in the observability solution we built to monitor the quality and performance of Snowflake data pipelines in Capital One Slingshot,” says Yudhish Batra, Distinguished Engineer, Capital One Software. “Event Tables has abstracted the complexity associated with logging from our data pipelines—specifically, the central Event Table gives us the ability to monitor and alert from a single location.” As phData migrates its Spark and Hadoop applications to Snowpark, the Event Tables feature has helped architects save time and hassle. “When working with Snowpark UDFs, some of the logic can become quite complex. In some instances, we had thousands of lines of Java code that needed to be monitored and debugged,” says Nick Pileggi, Principal Solutions Architect at phData. “Before Event Tables, we had almost no way to see what was happening inside the UDF and correct issues. Once we rolled out Event Tables, the amount of time we spent testing dropped significantly and allowed us to have debug and info-level access to the logs we were generating in Java.” One large communications service provider also uses logs in Event Tables to capture and analyze failed records during data ingestion from various external services to Snowflake. And a Snowflake Native App provider offering geolocation data services uses Event Tables to capture logs and traces from their UDFs to improve application reliability and performance. With Event Tables, you now have a built-in place to easily and consistently manage logging and tracing for your Snowflake applications. And in conjunction with other features such as Snowflake Alerts and Email Notifications, you can be notified of new events and errors in your applications. Try Event Tables today To learn more about Event Tables, join us at BUILD, Snowflake’s developer conference. Or get started with Event Tables today with a tutorial and quickstarts for logging and tracing. For further information about how Event Tables work, visit Snowflake product documentation. The post Collect Logs and Traces From Your Snowflake Applications With Event Tables appeared first on Snowflake. View the full article

OpenTelemetry traces hold a treasure trove of information to understand and troubleshoot distributed systems—but your services must be first instrumented to emit OpenTelemetry traces to realize that value. Then, those traces need to be sent to an observability backend that allows you to get answers to arbitrary questions on that data. Observability is an analytics problem ... https://www.timescale.com/blog/generate-and-store-opentelemetry-traces-automatically/?utm_id=FAUN_Kaptain321_Link_title

OpenTelemetry is a Cloud Native Computing Foundation (CNCF) initiative that provides open, vendor-neutral standards and tools for instrumenting services and applications. Many organizations use OpenTelemetry’s collection of APIs, SDKs, and tools to collect and export observability data from their environment to their preferred backend. As part of our ongoing commitment to OpenTelemetry, we are proud […] The post Ingest OpenTelemetry Traces and Metrics with the Datadog Agent appeared first on DevOps.com. View the full article