data trends Data Trends 2024: Strategies for an AI-Ready Data Foundation

[Snowflake]

A company’s data strategy is always in motion. Since the explosion of interest in generative AI and large language models (LLMs), that is more true than ever, with business leaders discussing how quickly they should adopt these technologies to stay competitive.

Some emerging approaches may be seen in our newly released Snowflake Data Trends 2024, looking at how users in the Data Cloud are working with their data. Understanding which features, languages and approaches our users embrace provides real indications of how organizations are preparing their data for this fast-moving new age of advanced AI.

For the report, we focused on two broad aspects of data strategy. We looked at activity around LLMs and the applications that work with them. But before a company can be successful with generative AI, LLMs and other innovative technologies, it has to build a strong data foundation. So we first looked at foundational activities that could suggest whether organizations are shifting or accelerating the work of preparing for advanced new technologies.

We saw that collectively, organizations are definitely preparing their data to be used more effectively with powerful, new AI technologies. The most marked finding was around governance. Strong data governance is essential to meet security and compliance obligations, but it is often regarded as a hindrance. If IT “locks down” the data, it can’t be used to derive insight and refine strategy — or so the complaint often goes. Looking at activity in the Data Cloud, we found quite the opposite.

• Application of individual governance features (tags applied to data objects, masking policies assigned to data sets, etc.) rose between 72% and 98% from January 2023 to January 2024.
• The cumulative number of queries run against this policy-protected data rose as well, by 142%.

This suggests that even as organizations increase the granularity of their data governance practices, they’re able to do more, not less, with the data.

The report also covers increased usage of AI-friendly programming language Python, which grew by 571%. We also saw a lot more work with unstructured data, which has great AI potential, since estimates consistently put the share of all data that’s unstructured at 80% to 90%. 

Looking at AI work and applications, our most exciting activity was from our Streamlit developer community. 

• From April 2023 to January 2024, more than 20,000 developers worked on 33,000+ LLM applications, including apps in development. 
• Across that period, the percentage of such apps that were chatbots increased notably, from 18% in April to 46% by the end of January.

We would expect much of this to have been experimentation and pilot projects, but the fact that so many devs are eager to work with these complex AI models only confirms the expectation that a transformative wave of innovation is beginning.

Further insights into the potential shape of the future: The fast-growing adoption of the Snowflake Native App Framework (generally available on AWS and Azure, private preview on GCP), since it entered public preview last summer, tells us that our mantra of bringing computation to a single, secure set of data — rather than farming out copies of your data to various environments — resonates with our users.  

When we look at how the features of the Data Cloud are actually being used, we not only refine our own product plans, we also see the trends and the future of enterprise data. We continue to innovate at Snowflake to shape that future. I hope that this report gives business and IT leaders ideas and indicators that will help them shape their data strategies within the Data Cloud and beyond.

For more, read Snowflake Data Trends 2024.

The post Data Trends 2024: Strategies for an AI-Ready Data Foundation appeared first on Snowflake.

View the full article