Data Engineering & Data Science

We’re excited to announce that the Databricks Assistant , now fully hosted and managed within Databricks, is available in public preview! This version... View the full article

We are excited to announce that Azure Private Link is now Generally Available (GA) for Databricks serverless and Mosaic AI Model Serving workloads... View the full article

We’re excited to announce a new integration between Databricks Notebooks and AI/BI Dashboards, enabling you to effortlessly transform insights from your notebooks into... View the full article

Effective data governance is crucial for organizations to harness their data assets. Learn how bp uses Databricks Unity Catalog to enhance their data governance framework, highlighting challenges, strategies, and benefits. View the full article

We are thrilled to unveil the finalists for the Databricks Generative AI Startup Challenge , a competition designed to spotlight innovative early-stage startups... View the full article

We are excited to introduce the gated Public Preview of Predictive Optimization for statistics. Announced at the Data + AI Summit, Predictive Optimization... View the full article

While GenAI is the focus today, most enterprises have been working for a decade or longer to make data intelligence a reality within... View the full article

As organizations increasingly leverage the Databricks Data Intelligence Platform for data and AI needs, upgrading to Unity Catalog is a key step in... View the full article

The Data + AI Skills Gap The “skills gap” has been a concern for CEOs and leaders for many years, and the gap... View the full article

Databricks is turning up the heat at AWS re:Invent 2024 , and we’re bringing more than just data and AI solutions to the... View the full article

In an era where data is the lifeblood of medical advancement, the clinical trial industry finds itself at a critical crossroads. The current... View the full article

Providence Health's extensive network spans 50+ hospitals and numerous other facilities across multiple states, presenting many challenges in predicting patient volume and daily... View the full article

When the Generative AI boom first ignited, every enterprise rushed to deploy the technology. For many, that excitement remains. But companies are also... View the full article

Many AI use cases now depend on transforming unstructured inputs into structured data. Developers are increasingly relying on LLMs to extract structured data... View the full article

The Future: From Rules Engines to Instruction-Following AI Agent Systems In sectors such as banking and insurance, rules engines have long played a... View the full article

Whether you’re coming from healthcare, aerospace, manufacturing, government or any other industries the term big data is no foreign concept; however how that... View the full article

Monolithic to Modular The proof of concept (POC) of any new technology often starts with large, monolithic units that are difficult to characterize... View the full article

The most recent wave of artificial intelligence (AI), spearheaded by the advent and mass adoption of large language models (LLM), showed the potential... View the full article

Over the last few years, we've seen tremendous growth and adoption of Databricks SQL , our intelligent data warehouse purpose-built on the Data... View the full article

Today, we are excited to announce the general availability of Databricks Assistant Autocomplete on all cloud platforms. Assistant Autocomplete provides personalized AI-powered code... View the full article

With the pace of modern business and the competitive need for more and more data, organizations now correctly ask whether their data management... View the full article

In industries like finance and retail, vast data is leveraged to generate billions in profits. Yet, in healthcare, the struggle to access critical... View the full article

Retrieval Augmented Generation (RAG) is the top use case for Databricks customers who want to customize AI workflows on their own data. The... View the full article

We consistently hear from our customers that one of the headwinds to transitioning Generative AI applications from pilot to production is the accuracy... View the full article

Summary Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS... View the full article

We are announcing the General Availability of Provider Usage Analytics for Databricks Marketplace providers. This feature lets you analyze lead generation and product... View the full article

We are thrilled to announce that embedding for AI/BI Dashboards is now available. Embedding enables you to seamlessly integrate Databricks AI/BI Dashboards into... View the full article

Introduction Applying Large Language Models (LLMs) for code generation is becoming increasingly prevalent, as it helps you code faster and smarter. A primary... View the full article

Many of our customers are shifting from monolithic prompts with general-purpose models to specialized compound AI systems to achieve the quality needed for... View the full article

The buzz around compound AI systems is real, and for good reason. Compound AI systems combine the best parts of multiple AI models... View the full article

Introduction Retrieval-augmented generation (RAG) has revolutionized how enterprises harness their unstructured knowledge base using Large Language Models (LLMs), and its potential has far-reaching... View the full article

What is enterprise AI? Enterprise AI combines artificial intelligence, machine learning and natural language processing (NLP) capabilities with business intelligence. Organizations use enterprise... View the full article

The upcoming AVEVA World Conference in Paris (Oct 14-17) promises to be a landmark event for the future of industrial AI, with Databricks... View the full article

In the two decades since the completion of the first draft of the human genome, the landscape of biological research has undergone a... View the full article

AI has quickly moved from an emerging technology to a business imperative as organizations recognize its potential to transform operations and keep them... View the full article

We are excited to partner with Meta to launch the latest models in the Llama 3 series on the Databricks Data Intelligence Platform... View the full article

“So often I’m asked to produce a dashboard but the request isn’t always clear, even after having a conversation with the person. This... View the full article

We are excited to announce that Databricks now supports Amazon EC2 G6 instances powered by NVIDIA L4 Tensor Core GPUs. This addition marks... View the full article

Special thanks to Daniel Benito (CTO, Bitext), Antonio Valderrabanos(CEO, Bitext), Chen Wang (Lead Solution Architect, AI21 Labs), Robbin Jang (Alliance Manager, AI21 Labs)... View the full article

Introduction The bin packing problem is a classic optimization challenge that has far-reaching implications for enterprise organizations across industries. At its core, the... View the full article

We are excited to announce that Mosaic AI Model Training now supports the full context length of 131K tokens when fine-tuning the Meta... View the full article

Today, we're excited to introduce Databricks Assistant Quick Fix , a powerful new feature designed to automatically correct common, single-line errors such as... View the full article

Generative AI technology has been in the headlines for many months now and there are varying opinions on the state of the technology... View the full article

At Databricks, we know that data is one of your most valuable assets. Our product and security teams work together to deliver an... View the full article

Transformer models, the backbone of modern language AI, rely on the attention mechanism to process context when generating output. During inference, the attention... View the full article

Are you an entrepreneur or startup with a groundbreaking Generative AI use case built on Databricks? Then we have a Challenge for you... View the full article

Today, we are excited to announce the support for named parameter markers in the SQL editor. This feature allows you to write parameterized... View the full article

We are updating this blog to show developers how to leverage the latest features of Databricks and the advancements in Spark. Most data... View the full article

Personal Access Tokens (PATs) are a convenient way to access services like Azure Databricks or Azure DevOps without logging in with your password... View the full article

We are excited to announce that Databricks was named one of the 2024 Fortune Best Workplaces in Technology™ . This award reflects our... View the full article