Data Engineering & Data Science

We are excited to partner with Meta to release the Llama 3.1 series of models on Databricks, further advancing the standard of powerful... View the full article

Evaluating long-form LLM outputs quickly and accurately is critical for rapid AI development. As a result, many developers wish to deploy LLM-as-judge methods... View the full article

Today, we're thrilled to announce that Mosaic AI Model Training's support for fine-tuning GenAI models is now available in Public Preview. At Databricks... View the full article

Written in collaboration with Navin Sharma and Joe Pindell, Stardog Across industries, the impact of post-delivery failure costs (recalls, warranty claims, lost goodwill... View the full article

How long should you train your language model? How large should your model be? In today's generative AI landscape, these are multi-million dollar... View the full article

Forecasting models are critical for many businesses to predict future trends, but their accuracy depends heavily on the quality of the input data... View the full article

Unlocking True Water Risk Assessment Across Insurance, Finance, Public Safety, and Beyond Check out the solution accelerator to download the notebooks referred to... View the full article

We are excited to announce the General Availability of serverless compute for notebooks, jobs and Delta Live Tables (DLT) on AWS and Azure... View the full article

Hallucinations in large language models (LLMs) occur when models produce responses that do not align with factual reality or the provided context. This... View the full article

Discover how we built a real-time personalization engine View the full article

We are thrilled to welcome the Prodvana team to Databricks. At Databricks, we are building one of the world’s largest multi-cloud platforms to... View the full article

We are proud to announce two new analyst reports recognizing Databricks in the data engineering and data streaming space: IDC MarketScape: Worldwide Analytic... View the full article

Databricks announced the public preview of Mosaic AI Agent Framework & Agent Evaluation alongside our Generative AI Cookbook at the Data + AI... View the full article

Mixture-of-Experts (MoE) has emerged as a promising LLM architecture for efficient training and inference. MoE models like DBRX , which use multiple expert... View the full article

Generative AI (GenAI) can unlock immense value. Organizations are cognizant of the potential but wary of the need to make smart choices about... View the full article

Introduction Financial institutions face a demanding environment with complex regulatory examinations and a pressing need for flexible and comprehensive risk management solutions. The... View the full article

Today, we are thrilled to announce the general availability of Databricks Assistant and AI-Generated Comments on all cloud platforms . Our mission at... View the full article

The recent Data + AI Summit 2024 was our biggest ever. Over 16,000 of our top customers, prospects, and partners attended in person... View the full article

We’re excited to introduce a revamped Catalog Explorer to streamline your day to day interactions, now live across your Unity Catalog-enabled workspaces. The... View the full article

Thousands of data architects, engineers, and scientists met at Data + AI Summit in San Francisco to hear from industry luminaries like Fei... View the full article

Radiology is an important component of diagnosing and treating disease through medical imaging procedures such as X-rays, computed tomography (CT), magnetic resonance imaging... View the full article

Enhancing DLT development experience is a core focus because it directly impacts the efficiency and satisfaction of developers building data pipelines with DLT... View the full article

In the insurance sector, customers demand personalized, fast, and efficient service that addresses their needs. Meanwhile, insurance agents must access a large amount... View the full article

We are excited to announce that Gartner has recognized Databricks as a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and... View the full article

Today, we are excited to announce Databricks LakeFlow, a new solution that contains everything you need to build and operate production data pipelines... View the full article

Discover how RudderStack reduced Customer Success response times by 50% using LLMs integrated with kapa.ai and Thena View the full article

At Databricks, our mission is to democratize data + AI. An open approach to sharing and collaboration is critical to maximize reach and... View the full article

We are excited to announce that we are open sourcing Unity Catalog, the industry’s first open source catalog for data and AI governance... View the full article

In an era marked by rapid advancements in artificial intelligence and an explosion of data and Gen AI tools, enterprises face fragmented data... View the full article

The annual Data Team Awards showcase how different data teams from across the globe are delivering solutions to some of the world’s most... View the full article

“FactSet’s mission is to empower clients to make data-driven decisions and supercharge their workflows and productivity. To deliver AI-driven solutions across our entire... View the full article

Today, we are excited to announce Databricks AI/BI , a new type of business intelligence product built from the ground up to deeply... View the full article

RudderStack Profiles enables data teams to power their businesses with complete customer profiles and now supports the Databricks Data Intelligence Platform.View the full article

Generative AI fever shows no signs of cooling off. As pressure and excitement build to execute strong GenAI strategies, data leaders and practitioners... View the full article

We're excited to announce the General Availability of Databricks Predictive Optimization. This capability intelligently optimizes your table data layouts for faster queries and... View the full article

The Databricks Partner Ecosystem, comprising over 3,800 partners worldwide, plays a pivotal role in building and delivering premier data and AI solutions globally... View the full article

In the dynamic, innovative landscape of the San Francisco Bay Area, Databricks stands out not just for our groundbreaking data and AI solutions... View the full article

This is a collaborative post from Databricks and Google Cloud. We thank Nicole Huynh , Partner Marketing Manager - Data Cloud, for her... View the full article

In June 2023, we launched Databricks Marketplace as an open marketplace for all your data, analytics, and AI needs, powered by the open... View the full article

This is a collaborative post from Databricks and Microsoft. We thank Mohini Verma , Senior Product Marketing Manager, for her contributions. Data +... View the full article

This blog is authored by Bhaskar Palit , Senior Director, Data & Analytics, PepsiCo, and Sudipta Das , Data Architect Senior Manager, PepsiCo... View the full article

BigQuery, now with first-party support for Delta Lake, grows Delta Lake’s vibrant connector ecosystem and simplifies its integration with Databricks. View the full article

Introduction The Internet of Things (IoT) is generating an unprecedented amount of data. IBM estimates that annual IoT data volume will reach approximately... View the full article

We are excited to announce that Forrester has recognized Databricks as a Leader in The Forrester Wave™: AI Foundation Models for Language, Q2... View the full article

Data + AI Summit 2024 will be held in person and virtually on June 10-13, 2024, with a highly anticipated lineup of keynotes... View the full article

Introduction Today, manufacturers’ field maintenance is often more reactive than proactive, which can lead to costly downtime and repairs. Historically, data warehouses have... View the full article

The annual Data Team Awards spotlight data teams and the pivotal role they play in business operations across industries and markets. By continually... View the full article

Over the last year, we’ve been listening to feedback and iterating on new ideas with a single goal: to build the best data-focused... View the full article

We are excited to announce that we have agreed to acquire Tabular, Inc, a data management company founded by Ryan Blue, Daniel Weeks... View the full article

Businesses are making remarkable progress on their data and AI journeys. They’re advancing from a few pilot projects confined to use cases likely... View the full article