Data Engineering & Data Science

Retailers have long shared sales and inventory data with their suppliers. Combined access to this information enables the two parties to assess consumer... View the full article

Databricks has obtained the International Standards Organization (ISO) 27701 certification as a data processor https://www.databricks.com/blog/databricks-obtains-iso-27701-certification

We’re excited to announce that Databricks has obtained the International Standards Organization (ISO) 27701 certification as a data processor. This certification reflects our c... View the full article

Written in partnership with Shell. The energy industry is all about physical assets – from terminals, ships and pipelines to refineries and wind f... View the full article

A common challenge data scientists encounter when developing machine learning solutions is training a model on a dataset that is too large to... View the full article

This blog post was written in collaboration with Eric Schwartz, Director of Partnerships at Ribbon Health, and David Kulwin, Director, Databricks Marketplace. Ensuring... View the full article

Today, we’re excited to announce Brickbuilder Accelerators, an expansion to the Brickbuilder Program that pairs the expertise of system integrator and consulting partners w... View the full article

This blog was written in collaboration with Sukh Sekhon, Software Engineer, Cloud Infrastructure and Helen Li, Sr. Director of Engineering at Exai Bio... View the full article

Biomechanical data has emerged as a game-changing factor for Major League Baseball (MLB) teams, offering a competitive edge in enhancing player performance and... View the full article

This article represents a collaborative effort between Plotly, Ballard Power Systems, and Databricks. Fleets of buses worldwide run on hydrogen fuel cells made... View the full article

We are excited to announce the public preview of the next generation of Databricks SQL dashboards, dubbed Lakeview dashboards. Available today, this new... View the full article

In August, Snowflake released new features around Snowpark for Python, DevOps, pipeline replication, and more. Read on to learn more about the full set of features that were just announced. Snowpark Python Updates Snowpark support for Python 3.9 and 3.10 – general availability Snowpark External Access – public preview Tabular Return Values from Python Stored Procedures – general availability Vectorized User-Defined Table Functions – public preview Deploy and Manage Snowflake objects and code with ease – public preview Notifications for better observability – general availability Data pipelines replication – public p…

Now in preview, AWS Glue Elastic Views is a new capability of AWS Glue that makes it easy to build materialized views that combine and replicate data across multiple data stores without you having to write custom code. With AWS Glue Elastic Views, you can use familiar Structured Query Language (SQL) to quickly create a virtual table—a materialized view—from multiple different source data stores. AWS Glue Elastic Views copies data from each source data store and creates a replica in a target data store. AWS Glue Elastic Views continuously monitors for changes to data in your source data stores, and provides updates to the materialized views in your target data stores autom…