Search the Community

In their haste to deploy LLM tools, organizations may overlook crucial security practices. The rise in threats like Remote Code Execution indicates an urgent need to improve security measures in AI development. The post Vulnerabilities for AI and ML Applications are Skyrocketing appeared first on Security Boulevard. View the full article

Join us on a journey of becoming a professional MLOps engineer by mastering essential tools, frameworks, key concepts, and processes in the field.View the full article

MLflow is an open source platform, used for managing machine learning workflows. It was launched back in 2018 and has grown in popularity ever since, reaching 10 million users in November 2022. AI enthusiasts and professionals have struggled with experiment tracking, model management and code reproducibility, so when MLflow was launched, it addressed pressing problems in the market. MLflow is lightweight and able to run on an average-priced machine. But it also integrates with more complex tools, so it’s ideal to run AI at scale. A short history Since MLflow was first released in June 2018, the community behind it has run a recurring survey to better understand user needs and ensure the roadmap s address real-life challenges. About a year after the launch, MLflow 1.0 was released, introducing features such as improved metric visualisations, metric X coordinates, improved search functionality and HDFS support. Additionally, it offered Python, Java, R, and REST API stability. MLflow 2.0 landed in November 2022, when the product also celebrated 10 million users. This version incorporates extensive community feedback to simplify data science workflows and deliver innovative, first-class tools for MLOps. Features and improvements include extensions to MLflow Recipes (formerly MLflow Pipelines) such as AutoML, hyperparameter tuning, and classification support, as well as improved integrations with the ML ecosystem, a revamped MLflow Tracking UI, a refresh of core APIs across MLflow’s platform components, and much more. In September 2023, Canonical released Charmed MLflow, a distribution of the upstream project. Why use MLflow? MLflow is often considered the most popular ML platform. It enables users to perform different activities, including: Reproducing results: ML projects usually start with simplistic plans and tend to go overboard, resulting in an overwhelming quantity of experiments. Manual or non-automated tracking implies a high chance of missing out on finer details. ML pipelines are fragile, and even a single missing element can throw off the results. The inability to reproduce results and codes is one of the top challenges for ML teams. Easy to get started: MLflow can be easily deployed and does not require heavy hardware to run. It is suitable for beginners who are looking for a solution to better see and manage their models. For example, this video shows how Charmed MLflow can be installed in less than 5 minutes. Environment agnostic: The flexibility of MLflow across libraries and languages is possible because it can be accessed through a REST API and Command Line Interface (CLI). Python, R, and Java APIs are also available for convenience. Integrations: While MLflow is popular in itself, it does not work in a silo. It integrates seamlessly with leading open source tools and frameworks such as Spark, Kubeflow, PyTorch or TensorFlow. Works anywhere: MLflow runs on any environment, including hybrid or multi-cloud scenarios, and on any Kubernetes. MLflow components MLFlow is an end-to-end platform for managing the machine learning lifecycle. It has four primary components: MLflow Tracking MLflow Tracking enables you to track experiments, with the primary goal of comparing results and the parameters used. It is crucial when it comes to measuring performance, as well as reproducing results. Tracked parameters include metrics, hyperparameters, features and other artefacts that can be stored on local systems or remote servers. MLflow Models MLflow Models provide professionals with different formats for packaging their models. This gives flexibility in where models can be used, as well as the format in which they will be consumed. It encourages portability across platforms and simplifies the management of the machine learning models. MLflow projects Machine learning projects are packaged using MLflow Projects. It ensures reusability, reproducibility and portability. A project is a directory that is used to give structure to the ML initiative. It contains the descriptor file used to define the project structure and all its dependencies. The more complex a project is, the more dependencies it has. They come with risks when it comes to version compatibility as well as upgrades. MLflow project is useful especially when running ML at scale, where there are larger teams and multiple models being built at the same time. It enables collaboration between team members who are looking to jointly work on a project or transfer knowledge between them or to production environments. MLflow model registry Model Registry enables you to have a centralised place where ML models are stored. It helps with simplifying model management throughout the full lifecycle and how it transitions between different stages. It includes capabilities such as versioning and annotating, and provides APIs and a UI. Key concepts of MLflow MLflow is built around two key concepts: runs and experiments. In MLflow, each execution of your ML model code is referred to as a run. All runs are associated with an experiment. An MLflow experiment is the primary unit for MLflow runs. It influences how runs are organised, accessed and maintained. An experiment has multiple runs, and it enables you to efficiently go through those runs and perform activities such as visualisation, search and comparisons. In addition, experiments let you run artefacts and metadata for analysis in other tools. Kubeflow vs MLflow Both Kubeflow and MLFlow are open source solutions designed for the machine learning landscape. They received massive support from industry leaders, and are driven by a thriving community whose contributions are making a difference in the development of the projects. The main purpose of both Kubeflow and MLFlow is to create a collaborative environment for data scientists and machine learning engineers, and enable teams to develop and deploy machine learning models in a scalable, portable and reproducible manner. However, comparing Kubeflow and MLflow is like comparing apples to oranges. From the very beginning, they were designed for different purposes. The projects evolved over time and now have overlapping features. But most importantly, they have different strengths. On the one hand, Kubeflow is proficient when it comes to machine learning workflow automation, using pipelines, as well as model development. On the other hand, MLFlow is great for experiment tracking and model registry. From a user perspective, MLFlow requires fewer resources and is easier to deploy and use by beginners, whereas Kubeflow is a heavier solution, ideal for scaling up machine learning projects. Read more about Kubefllow vs. MLflow Go to the blog Charmed MLflow vs the upstream project Charmed MLflow is Canonical’s distribution of the upstream project. It is part of Canonical’s growing MLOps portfolio. It has all the features of the upstream project, to which we add enterprise-grade capabilities such as: Simplified deployment: the time to deployment is less than 5 minutes, enabling users to also upgrade their tools seamlessly. Simplified upgrades using our guides. Automated security scanning: The bundle is scanned at a regular cadence.. Security patching: Charmed MLflow follows Canonical’s process and procedure for security patching. Vulnerabilities are prioritised based on severity, the presence of patches in the upstream project, and the risk of exploitation. Maintained images: All Charmed MLflow images are actively maintained. Comprehensive testing: Charmed MLflow is thoroughly tested on multiple platforms, including public cloud, local workstations, on-premises deployments, and various CNCF-compliant Kubernetes distributions. Get started easily with Charmed MLflow Further reading [Whitepaper] Tookit to machine learning [Blog] What is MLOps? [Webinar] Kubeflow vs. MLflow [Blog] LLMs explained Book a meeting View the full article

Artificial General Intelligence, when it exists, will be able to do many tasks better than humans. For now, the machine learning systems and generative AI solutions available on the market are a stopgap to ease the cognitive load on engineers, until machines which think like people exist. Generative AI is currently dominating headlines, but its backbone, neural networks, have been in use for decades. These Machine Learning (ML) systems historically acted as cruise control for large systems that would be difficult to constantly maintain by hand. The latest algorithms also proactively respond to errors and threats, alerting teams and recording logs of unusual activity. These systems have developed further and can even predict certain outcomes based on previously observed patterns. This ability to learn and respond is being adapted to all kinds of technology. One that persists is the use of AI tools in envirotech. Whether it's enabling new technologies with vast data processing capabilities, or improving the efficiency of existing systems by intelligently adjusting inputs to maximize efficiency, AI at this stage of development is so open ended it could theoretically be applied to any task. AI’s undeniable strengths GenAI isn’t inherently energy intensive. A model or neural network is no more energy inefficient than any other piece of software when it is operating, but the development of these AI tools is what generates the majority of the energy costs. The justification for this energy consumption is that the future benefits of the technology are worth the cost in energy and resources. Some reports suggest many AI applications are ‘solutions in search of a problem’, and many developers are using vast amounts of energy to develop tools that could produce dubious energy savings at best. One of the biggest benefits of machine learning is its ability to read through large amounts of data, and summarize insights for humans to act on. Reporting is a laborious and frequently manual process, time saved reporting can be shifted to actioning machine learning insights and actively addressing business-related emissions. Businesses are under increasing pressure to start reporting on Scope 3 emissions, which are the hardest to measure, and the biggest contributor of emissions for most modern companies. Capturing and analyzing these disparate data sources would be a smart use of AI, but would still ultimately require regular human guidance. Monitoring solutions already exist on the market to reduce the demand on engineers, so taking this a step further with AI is an unnecessary and potentially damaging innovation. Replacing the engineer with an AI agent reduces human labor, but removes a complex interface, just to add equally complex programming in front of it. That isn’t to say innovation should be discouraged. It’s a noble aim, but do not be sold a fairy tale that this will happen without any hiccups. Some engineers will be replaced eventually by this technology, but the industry should approach it carefully. Consider self-driving cars. They're here, they're doing better than an average human-driver. But in some edge cases they can be dangerous. The difference is that it is very easy to see this danger, compared to the potential risks of AI. Today’s ‘clever’ machines are like naive humans AI agents at the present stage of development are comparable to human employees - they need training and supervision, and will gradually become out of date unless re-trained from time to time. Similarly, as has been observed with ChatGPT, models can degrade over time. The mechanics that drive this degradation are not clear, but these systems are delicately calibrated, and this calibration is not a permanent state. The more flexible the model, the more likely it can misfire and function suboptimally. This can manifest as data or concept drift, an issue where a model invalidates itself over time. This is one of many inherent issues with attaching probabilistic models to deterministic tools. A concerning area of development is the use of AI in natural language inputs, trying to make it easier for less technical employees or decision makers to save on hiring engineers. Natural language outputs are ideal for translating the expert, subject specific outputs from monitoring systems, in a way that makes the data accessible for those who are less data literate. Despite this strength even summarizations can be subject to hallucinations where data is fabricated, this is an issue that persists in LLMs and could create costly errors where AI is used to summarize mission critical reports. The risk is we create AI overlays for systems that require deterministic inputs. Trying to make the barrier to entry for complex systems lower is admirable, but these systems require precision. AI agents cannot explain their reasoning, or truly understand a natural language input and work out the real request in the way a human can. Moreover, it adds another layer of energy consuming software to a tech stack for minimal gain. We can’t leave it all to AI The rush to ‘AI everything’ is producing a tremendous amount of wasted energy, with 14,000 AI startups currently in existence, how many will actually produce tools that will benefit humanity? While AI can improve the efficiency of a data center by managing resources, ultimately that doesn't manifest into a meaningful energy saving as in most cases that free capacity is then channeled into another application, using any saved resource headroom, plus the cost of yet more AI powered tools. Can AI help achieve sustainability goals? Probably, but most of the advocates fall down at the ‘how’ part of that question, in some cases suggesting that AI itself will come up with new technologies. Climate change is now an existential threat with so many variables to account for it stretches the comprehension of the human mind. Rather than tackling this problem directly, technophiles defer responsibility to AI in the hope it will provide a solution at some point in future. The future is unknown, and climate change is happening now. Banking on AI to save us is simply crossing our fingers and hoping for the best dressed up as neo-futurism. We've listed the best collaboration platform for teams. This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro View the full article

At Databricks, we’re committed to building the most efficient and performant training tools for large-scale AI models. With the recent release of DBRX... View the full article

Data science’s essence lies in machine learning algorithms. Here are ten algorithms that are a great introduction to machine learning for any beginner!View the full article

Begin your MLOps journey with these comprehensive free resources available on GitHub.View the full article

I had the pleasure to be invited by Canonical’s AI/ML Product Manager, Andreea Munteanu, to one of the recent episodes of the Canonical AI/ML podcast. As an enthusiast of automotive and technology with a background in software, I was very eager to share my insights into the influence of artificial intelligence (AI) in the automotive industry. I have a strong belief that the intersection of AI and cars represents a pivotal point where innovation meets practical implementation, and leads to safer, more efficient and more user-friendly cars. In the episode, several key issues in the use of AI in cars and automotive in general came up. It’s not just the use of AI that we should be thinking about, but a whole range of safety, ethics, and privacy concerns that can eclipse simple technical challenges. This underscores the importance of considering the broader societal impacts and ethical implications of integrating AI into automotive technologies. This blog explores the key takeaways from the engaging conversation we’ve had, diving into the present and future implications of AI in the world of automobiles. We talked about a lot in the half-hour discussion, but a stand-out moment for me was when we spoke about the impact AI implementation has on costs. I’ll get more into why I thought this was the most important part of our discussion in a bit, but for now you can listen to the entire conversation yourself in the podcast episode. AI is everywhere in automotive AI is already embedded in every aspect of the automotive sector. This key role is not just limited to autonomous vehicles: AI is integral to manufacturing processes, predictive maintenance, and supply chain management. In almost every part of the automobiles – whether it’s conceptualising and building cars, driving them, or monitoring their performance throughout their lifecycle – AI is critical. Safety considerations Cars driving themselves around makes people very nervous, especially when algorithms are tasked with making intricate split-second decisions that boil down to “don’t swerve into oncoming traffic”. It’s no surprise that safety is the paramount factor in vehicle AI conversations. Therefore, it is imperative to address the safety concerns associated with the integration of AI in automotive technology. “Would you protect the driver and the vehicle occupants versus all the surrounding pedestrians? In some cases, the vehicle will have to choose”* Bertrand Boisseau It’s a troubling ethical concern: do machines have a right to make decisions about human life, and what are the limits to that decision-making process? AI and autonomous vehicle engineers have their work cut out for them, as these decisions are incredibly complex and happen at the speed of life. When a glitch happens on your desktop, it’s not so bad because you’re not travelling at 100 km/hr through 2-lane traffic with oncoming trucks and pedestrians on every side. While these challenges are significant and lead to a lot of uncertainty about whether it is safe to let Autonomous Driving (AD) vehicles drive around at the maximum speed limit, we should pause for a second to reflect on the extreme and ongoing testing and retesting that they undergo. Driverless cars often make headlines when accidents happen. But it’s important to remember that accidents are part of driving, whether it’s with a human or autonomous tech. In reality, driving carries risks, and you’re likely to get in a car accident in your lifetime. So, while one accident might spark concerns, it’s crucial to see it in the bigger picture of transportation safety. Also, a study comparing human ride-hail drivers and self-driving cars in San Francisco revealed that human drivers are more likely to crash, cause crashes, and injure others than autonomous vehicles. Human drivers had a crash rate of 50.5 crashes per million miles, while self-driving cars had a lower rate of 23 crashes per million miles. Additionally, the development of robust fail-safe mechanisms and redundant systems can serve as safeguards against potential algorithmic errors or malfunctions. Furthermore, ongoing collaboration between industry stakeholders, regulatory bodies, and research institutions fosters the establishment of comprehensive safety standards and guidelines for the integration of AI in automotive technology. By prioritising safety considerations and adopting a multi-faceted approach encompassing technological innovation, rigorous testing, and regulatory oversight, the automotive industry can effectively address the safety challenges associated with AI integration, paving the way for safer and reliable autonomous driving systems. Diverse applications beyond driving While self-driving cars often take centre stage, AI solves a broader spectrum of problems for the automotive industry: optimising manufacturing processes; predictive maintenance for parts replacement; and enhancing supply chain management efficiency, to name a few. It will also transform the in-car experience with advanced voice recognition and personalised assistance. “I do believe that having advanced personal assistant will be noticeable for the user. Once you start putting voice recognition in there, it can become, I think, very useful.”* Bertrand Boisseau Challenges and concerns On the podcast, we mention that safety is the most obvious concern when it comes to the use of AI in cars, but there are even greater challenges and concerns that developer automotive industry figures should be thinking about. These include privacy issues, the role of regulation in the use of AI, public trust in AI systems, job displacement fears, and the substantial costs associated with running AI/ML models, both in terms of processing power and energy consumption. “You want to make sure that whatever is sent to the training models still complies with data privacy concerns: how do you collect data, how do you share vehicle data -which is usually private data-, how do you train these models?”* Bertrand Boisseau When it comes to training machine learning models for autonomous vehicles, maintaining data privacy is crucial. We need to be mindful of how we collect and share vehicle data, ensuring it aligns with privacy concerns. It’s vital to gather data ethically and responsibly, while also validating its quality to prevent biases and inaccuracies. After all, if we feed the models with flawed data (from bad drivers, for example), we risk compromising their performance and safety. So, robust data validation processes are essential to ensure the effectiveness and reliability of autonomous vehicle technology. The evolution of jobs As AI evolves, so too do the nature of jobs in the automotive industry. Take developers as an example: as AI gains a stronger foothold in automotive development, our roles will transform from manually coding algorithms to focusing on simulating and validating AI models. “I don’t agree with the idea of having job displacement in any way, but I do think that there is going to be a shift [in] the market, and there is a clear skill gap or understanding gap.”* Andreea Munteanu The industry faces a growing need for individuals with expertise in both AI and automotive engineering, bridging the gap between technology and traditional automotive skills. However, it’s also crucial to acknowledge the widespread concerns about the potential impact of autonomous vehicles on various job sectors within transportation, including taxi drivers, delivery drivers, truck drivers, valets, and e-hailing service contractors. While autonomous technology is advancing rapidly, broad legislation still typically mandates the presence of a human driver to take over the wheel if necessary, meaning fully human-free cars aren’t imminent. The use of open source Open source software will play a key role in the automotive sector. Open source software presents indispensable advantages such as unparalleled transparency, enabling thorough inspection and auditability of the codebase. “Open source software in general and even [especially] in AI/ML would be the wiser choice in most cases.”* Bertrand Boisseau This transparency not only fosters trust and reliability but also empowers developers to identify and rectify potential issues swiftly, ensuring the highest standards of quality and security. Additionally, going with closed source might mean that Original Equipment Manufacturers (OEMs), or even the customers, have to pay extra fees per year just for licences. Imagine having a “smarter” car that becomes useless if a licence lapses or expires. Open source cuts down on these costs since you’re not constrained by licences, making software cheaper to create, keep up, and expand. Fewer closed source licences mean less complexity in the user experience. The adoption of open-source models, tools, and frameworks is likely to grow, especially as companies aim to balance innovation and security. Data privacy As AI becomes increasingly integrated into the automotive industry, ensuring robust data privacy measures is paramount. The vast amounts of data generated by connected vehicles, ranging from driver behaviour to location information, raise significant privacy concerns. It’s essential to implement strict and clear data protection protocols to safeguard sensitive information from unauthorised access or misuse. Additionally, transparent data collection practices and clear consent mechanisms must be established to ensure that users have control over their data. Failure to address data privacy issues adequately not only risks violating privacy regulations but also erodes consumer trust, hindering widespread adoption of AI-driven automotive technologies. With the implementation of EU policies such as GDPR, fines can be as high as 10 million euros or up to 2% of the company’s entire global turnover of the preceding fiscal year (whichever is higher), further emphasising the importance of robust data privacy measures. AI can reduce costs in automotive Cost considerations are another crucial aspect of integrating AI into the automotive industry. While AI technologies hold immense potential to optimise operations, enhance safety, and improve the driving experience, they often come with significant upfront and ongoing costs. The automotive industry is also fiercely focused on cost optimisation: cars that are more expensive are a severe risk for sales, especially in saturated markets. What good is AI and all the hardware and infrastructure it will need if it just leads to cars that their usual buyers can no longer afford? Additionally, ensuring compatibility with existing systems and regulatory compliance may incur other expenses. Moreover, there are ongoing costs associated with maintaining and updating AI systems, as well as training personnel to effectively use and manage these technologies. However, despite the initial investment, the potential long-term benefits, such as increased efficiency, reduced accidents, and improved customer satisfaction, can outweigh the costs over time. Therefore, while cost is a critical factor to consider, automotive companies must carefully weigh the upfront investment against the potential long-term returns and strategic advantages offered by AI integration. Regulations: the wild west won’t stay wild forever Navigating regulatory frameworks generally presents significant challenges. This is already true for the integration of AI into the automotive industry. Regulators are often slow to react to the rapid pace of technological advancements, resulting in a lag between the emergence of new AI-driven automotive technologies and the establishment of appropriate regulations. This delay can create uncertainty and hinder innovation within the industry as companies navigate ambiguous regulatory landscapes. However, once regulatory wheels are set in motion, they can hit like a truck, with stringent requirements and compliance measures impacting the entire automotive ecosystem. The sudden imposition of regulations can disrupt ongoing projects, necessitate costly adjustments, and delay the deployment of AI technologies. Therefore, automotive companies must remain vigilant and proactive in engaging with regulators, advocating for clear and forward-thinking regulatory frameworks that balance innovation with safety and compliance. By fostering collaboration and dialogue between industry stakeholders and regulators, the automotive industry can navigate regulatory challenges more effectively and ensure the responsible and sustainable integration of AI technologies. Reconciling AI and sustainability Sustainability and energy consumption are crucial topics of debate in the automotive industry, especially concerning the integration of AI technologies. Data centres, which are essential for processing the vast amounts of data generated by AI-driven systems, consume substantial amounts of energy. The energy usage of a single data centre can be equivalent to that of a small town, highlighting the significant environmental impact associated with AI infrastructure. “If you need processing power, you need energy. The big [AI/ML] players have also been saying that we will need to build nuclear power plants to run all the requests.”* Bertrand Boisseau Similarly, badly optimised, individual autonomous cars, with their sophisticated sensor systems and computational requirements, might also consume considerable energy during operation. As the automotive industry embraces AI, it must address the sustainability implications of increased energy consumption and explore strategies to minimise environmental impact, such as optimising algorithms for efficiency, utilising renewable energy sources, and implementing energy-saving technologies. Addressing criticisms of automotive automation Automation in the automotive industry presents significant potential, yet it’s essential to address ongoing discussions surrounding the broader concept of automation, particularly in social media and consumer circles. Questions arise, challenging the value of autonomous driving and whether every aspect of a car’s operation needs to be automated. While these debates hold merit, they often overlook the broader implications and benefits that automation can bring. Arguments against automation often highlight concerns regarding the potential loss of manual driving skills and the ability to react to unforeseen situations beyond the scope of automated systems. However, it’s crucial to consider that historical transitions in automotive technology, such as the shift from manual to automatic transmission or the adoption of adaptive cruise control, have not resulted in increased accidents — quite the opposite, in fact. On top of that, the advancement of automation extends beyond driverless vehicles alone, encompassing a multitude of frameworks, optimisations, and breakthroughs with far-reaching impacts. Drawing parallels to other technological achievements, such as the space program, sheds light on the extensive benefits that arise from ambitious projects despite initial scepticism. Much like criticisms were raised against space exploration, which questioned its necessity or deemed it a misallocation of resources, the collective efforts in the automotive industry toward automation yield a number of innovations and enhancements. These advancements not only streamline operation and maintenance but also significantly enhance safety for drivers and road users alike. Therefore, while discussions surrounding automation provoke diverse perspectives, embracing its potential fosters progress and innovation within the automotive landscape, and beyond. The future of AI in automotive In the future, AI in the automotive industry will certainly be widespread; but the application of AI will dominate more specific use cases, such as autonomous driving systems, personal assistants or predictive maintenance. The reasons for this are quite simple: the data processing and warehousing for each automated vehicle become difficult to design and expensive to run, especially when the financial returns on AI products and their long-term financial sustainability are still unproven. There are still strong challenges when it comes to generating revenue from AI investments, particularly in the automotive realm, where return on investment and sustainable business models are still evolving. I found our podcast conversation on AI in the automotive industry incredibly engaging, especially when we delved into the potential impact on safety and driving experiences. It’s fascinating to envision how AI will revolutionise not just the way we drive, but also how vehicles are manufactured and maintained. As AI paves the roads of tomorrow, the integration of AI into the automotive industry promises a transformative journey. As a passionate car enthusiast, I think we’re headed towards a new era of innovation. AI will be in our cars, homes, jobs, buses, and perhaps even our law-making offices. As it grows and evolves, it’ll be even more important to keep track of its progression and adoption – which is why I’m glad that podcasts like ours exist. If you want to stay ahead of AI/ML and GenAI in the automotive industry – or indeed, any industry – and watch its interplay with open source applications, follow the Ubuntu AI Podcasts by Canonical. *quotations edited for clarity and brevity Listen to the podcast episode Contact Us Further reading Want to learn more about Software Defined Vehicles? Download our guide! Learn about the next-generation automotive operating system: EB corbos Linux – built on Ubuntu How to choose an OS for software development in automotive View the full article

Today, enterprises are focused on enhancing decision-making with the power of AI and machine learning (ML). But the complexity of ML models and data science techniques often leaves behind organizations without data scientists or with limited data science resources. And for those organizations with strong data analyst resources, complex ML models and frameworks may seem overwhelming, potentially preventing them from driving faster, higher-quality insights. That’s why Snowflake Cortex ML Functions were developed: to abstract away the complexity of ML frameworks and algorithms, automate much of the data science process, and democratize ML for everyone. These functions make activities such as data quality monitoring through anomaly detection, or retail sales forecasting through time series forecasting, faster, easier and more robust — especially for data analysts, data engineers, and citizen data scientists. As a continuation of this suite of functions, Snowflake Cortex ML Classification is now in public preview. It enables data analysts to categorize data into predefined classes or labels, and both binary classification (two classes) and multi-class classification (more than two classes) are supported. All of this can be done with a simple SQL command, for use cases such as lead scoring or churn prediction. How ML Classification works Imagine you are a data analyst on a marketing team and want to ensure your team takes quick action on the highest-priority sales leads, optimizing the value from investments in sales and marketing. With ML Classification, you can easily classify certain leads as having a higher likelihood to convert, and thus give them a higher priority for follow-up. And for those with a low likelihood to convert, your marketing team can choose to nurture those or contact them less frequently. ML Classification can be accomplished in two simple steps: First, train a machine learning model using your CRM data for all leads you’ve pursued in the past and labeled as either “Converted” or “Not converted.” Then, use that model to classify your new set of leads as likely to convert or not. When you generate your Snowflake ML Classification predictions, you’ll get not only the predicted “class” (likely to convert vs. not likely), but also the probability of that prediction. That way, you can prioritize outreach and marketing to leads that have the highest probability of converting — even within all leads that are likely to convert. Here’s how to use Classification with just a few lines of SQL: -- Train a model on all historical leads. CREATE OR REPLACE SNOWFLAKE.ML.CLASSIFICATION my_lead_model( INPUT_DATA => SYSTEM$REFERENCE('TABLE', 'historical_leads'), TARGET_COLNAME => 'CONVERT' ); -- Generate predictions. CREATE TABLE my_predictions AS SELECT my_lead_model!PREDICT(object_construct(*)) as prediction FROM new_leads; The above SQL generates an ML model you can use repeatedly to assess whether new leads are likely to convert. It also generates a table of predictions that includes not only the expected class (likely to convert vs. not likely) but also the probability of each class. If you’re interested in pulling out just the predicted class and probability of that class, you can use the following SQL to parse the results: CREATE TABLE my_predictions AS SELECT prediction:class as convert_or_not, prediction['probability']['"1"'] as convert_probability FROM (SELECT my_lead_model!PREDICT(object_construct(*)) as prediction FROM new_leads); To support your assessment of the model (“Is this good enough for my team to use?”) and understanding of the model (“What parts of the data I’ve trained the model on are most useful to the model?”), this classification function produces evaluation metrics and feature importance data. -- Get evaluation metrics CALL my_lead_model!SHOW_EVALUATION_METRICS(); CALL my_lead_model!SHOW_GLOBAL_EVALUATION_METRICS(); CALL my_lead_model!SHOW_CONFUSION_MATRIX(); -- Get feature importances CALL my_lead_model!SHOW_FEATURE_IMPORTANCE(); ML Classification can be used for other use cases as well, such as churn prediction. For example, customers classified as having a high likelihood to churn can be targeted with special offers, personalized communication or other retention efforts. The two problems we describe above — churn prediction and lead scoring — are binary classification problems, where the value we’re predicting takes on just two values. This classification function can also solve multi-class problems, where the value we’re predicting takes on three or more values. For example, say your marketing team segments customers into threethree groups (Bronze, Silver, and Gold) (Bronze, Silver, and Gold) based on their purchasing habits, demographic and psychographic characteristics. This classification function could help you bucket new customers and prospects into those three value-based segments with ease. -- Train a model on all existing customers. CREATE OR REPLACE SNOWFLAKE.ML.CLASSIFICATION my_marketing_model( INPUT_DATA => SYSTEM$REFERENCE('TABLE', 'customers'), TARGET_COLNAME => 'value_grouping' ); -- Generate predictions for prospects. CREATE TABLE my_value_predictions AS SELECT my_marketing_model!PREDICT(object_construct(*)) as prediction FROM prospects; -- Parse results. CREATE TABLE my_predictions_parsed AS SELECT prediction:class as value_grouping, prediction['probability'][class] as probability FROM my_value_predictions; How Faraday uses Snowflake Cortex ML Classification Faraday, a customer behavior prediction platform, has been using ML Classification during private preview. For Faraday, having classification models right next to their customers’ Snowflake data accelerates their use of next-generation AI/ML and drives value for their customers. “Snowflake Cortex ML Functions allow our data engineering team to run complex ML models where our customers’ data lives. This provides us out-of-the-box data science resources and means we don’t have to move our customers’ data to run this analysis,” said Seamus Abshere, Co-Founder and CTO at Faraday. “The public release of Cortex ML Classification is a big unlock; it disrupts a long tradition of separating data engineering and data science.” What’s next? To continue improving the ML Classification experience, we plan to release support for text and timestamps in training and prediction data. We are also continuously improving the amount of data that can be used in training and prediction and the speed of training and prediction – as well as model accuracy. Not only do we want to put AI and ML in the hands of all data analysts and data engineers, but we want to empower business users, too. That’s why the Snowflake Cortex UI is now in private preview. This clickable user interface helps our Snowflake customers discover Snowflake Cortex functions from Snowsight and guides users through the process of selecting data, setting parameters and scheduling recurring training and prediction for AI and ML models — all through an easy-to-use interface. To learn more about Snowflake Cortex ML functions, visit Snowflake documentation or try out this Quickstart. The post Predict Known Categorical Outcomes with Snowflake Cortex ML Classification, Now in Public Preview appeared first on Snowflake. View the full article

Bandwidth estimation (BWE) and congestion control play an important role in delivering high-quality real-time communication (RTC) across Meta’s family of apps. We’ve adopted a machine learning (ML)-based approach that allows us to solve networking problems holistically across cross-layers such as BWE, network resiliency, and transport. We’re sharing our experiment results from this approach, some of the challenges we encountered during execution, and learnings for new adopters. Our existing bandwidth estimation (BWE) module at Meta is based on WebRTC’s Google Congestion Controller (GCC). We have made several improvements through parameter tuning, but this has resulted in a more complex system, as shown in Figure 1. Figure 1: BWE module’s system diagram for congestion control in RTC. One challenge with the tuned congestion control (CC)/BWE algorithm was that it had multiple parameters and actions that were dependent on network conditions. For example, there was a trade-off between quality and reliability; improving quality for high-bandwidth users often led to reliability regressions for low-bandwidth users, and vice versa, making it challenging to optimize the user experience for different network conditions. Additionally, we noticed some inefficiencies in regards to improving and maintaining the module with the complex BWE module: Due to the absence of realistic network conditions during our experimentation process, fine-tuning the parameters for user clients necessitated several attempts. Even after the rollout, it wasn’t clear if the optimized parameters were still applicable for the targeted network types. This resulted in complex code logics and branches for engineers to maintain. To solve these inefficiencies, we developed a machine learning (ML)-based, network-targeting approach that offers a cleaner alternative to hand-tuned rules. This approach also allows us to solve networking problems holistically across cross-layers such as BWE, network resiliency, and transport. Network characterization An ML model-based approach leverages time series data to improve the bandwidth estimation by using offline parameter tuning for characterized network types. For an RTC call to be completed, the endpoints must be connected to each other through network devices. The optimal configs that have been tuned offline are stored on the server and can be updated in real-time. During the call connection setup, these optimal configs are delivered to the client. During the call, media is transferred directly between the endpoints or through a relay server. Depending on the network signals collected during the call, an ML-based approach characterizes the network into different types and applies the optimal configs for the detected type. Figure 2 illustrates an example of an RTC call that’s optimized using the ML-based approach. Figure 2: An example RTC call configuration with optimized parameters delivered from the server and based on the current network type. Model learning and offline parameter tuning On a high level, network characterization consists of two main components, as shown in Figure 3. The first component is offline ML model learning using ML to categorize the network type (random packet loss versus bursty loss). The second component uses offline simulations to tune parameters optimally for the categorized network type. Figure 3: Offline ML-model learning and parameter tuning. For model learning, we leverage the time series data (network signals and non-personally identifiable information, see Figure 6, below) from production calls and simulations. Compared to the aggregate metrics logged after the call, time series captures the time-varying nature of the network and dynamics. We use FBLearner, our internal AI stack, for the training pipeline and deliver the PyTorch model files on demand to the clients at the start of the call. For offline tuning, we use simulations to run network profiles for the detected types and choose the optimal parameters for the modules based on improvements in technical metrics (such as quality, freeze, and so on.). Model architecture From our experience, we’ve found that it’s necessary to combine time series features with non-time series (i.e., derived metrics from the time window) for a highly accurate modeling. To handle both time series and non-time series data, we’ve designed a model architecture that can process input from both sources. The time series data will pass through a long short-term memory (LSTM) layer that will convert time series input into a one-dimensional vector representation, such as 16×1. The non-time series data or dense data will pass through a dense layer (i.e., a fully connected layer). Then the two vectors will be concatenated, to fully represent the network condition in the past, and passed through a fully connected layer again. The final output from the neural network model will be the predicted output of the target/task, as shown in Figure 4. Figure 4: Combined-model architecture with LSTM and Dense Layers Use case: Random packet loss classification Let’s consider the use case of categorizing packet loss as either random or congestion. The former loss is due to the network components, and the latter is due to the limits in queue length (which are delay dependent). Here is the ML task definition: Given the network conditions in the past N seconds (10), and that the network is currently incurring packet loss, the goal is to characterize the packet loss at the current timestamp as RANDOM or not. Figure 5 illustrates how we leverage the architecture to achieve that goal: Figure 5: Model architecture for a random packet loss classification task. Time series features We leverage the following time series features gathered from logs: Figure 6: Time series features used for model training. BWE optimization When the ML model detects random packet loss, we perform local optimization on the BWE module by: Increasing the tolerance to random packet loss in the loss-based BWE (holding the bitrate). Increasing the ramp-up speed, depending on the link capacity on high bandwidths. Increasing the network resiliency by sending additional forward-error correction packets to recover from packet loss. Network prediction The network characterization problem discussed in the previous sections focuses on classifying network types based on past information using time series data. For those simple classification tasks, we achieve this using the hand-tuned rules with some limitations. The real power of leveraging ML for networking, however, comes from using it for predicting future network conditions. We have applied ML for solving congestion-prediction problems for optimizing low-bandwidth users’ experience. Congestion prediction From our analysis of production data, we found that low-bandwidth users often incur congestion due to the behavior of the GCC module. By predicting this congestion, we can improve the reliability of such users’ behavior. Towards this, we addressed the following problem statement using round-trip time (RTT) and packet loss: Given the historical time-series data from production/simulation (“N” seconds), the goal is to predict packet loss due to congestion or the congestion itself in the next “N” seconds; that is, a spike in RTT followed by a packet loss or a further growth in RTT. Figure 7 shows an example from a simulation where the bandwidth alternates between 500 Kbps and 100 Kbps every 30 seconds. As we lower the bandwidth, the network incurs congestion and the ML model predictions fire the green spikes even before the delay spikes and packet loss occur. This early prediction of congestion is helpful in faster reactions and thus improves the user experience by preventing video freezes and connection drops. Figure 7: Simulated network scenario with alternating bandwidth for congestion prediction Generating training samples The main challenge in modeling is generating training samples for a variety of congestion situations. With simulations, it’s harder to capture different types of congestion that real user clients would encounter in production networks. As a result, we used actual production logs for labeling congestion samples, following the RTT-spikes criteria in the past and future windows according to the following assumptions: Absent past RTT spikes, packet losses in the past and future are independent. Absent past RTT spikes, we cannot predict future RTT spikes or fractional losses (i.e., flosses). We split the time window into past (4 seconds) and future (4 seconds) for labeling. Figure 8: Labeling criteria for congestion prediction Model performance Unlike network characterization, where ground truth is unavailable, we can obtain ground truth by examining the future time window after it has passed and then comparing it with the prediction made four seconds earlier. With this logging information gathered from real production clients, we compared the performance in offline training to online data from user clients: Figure 9: Offline versus online model performance comparison. Experiment results Here are some highlights from our deployment of various ML models to improve bandwidth estimation: Reliability wins for congestion prediction connection_drop_rate -0.326371 +/- 0.216084 last_minute_quality_regression_v1 -0.421602 +/- 0.206063 last_minute_quality_regression_v2 -0.371398 +/- 0.196064 bad_experience_percentage -0.230152 +/- 0.148308 transport_not_ready_pct -0.437294 +/- 0.400812 peer_video_freeze_percentage -0.749419 +/- 0.180661 peer_video_freeze_percentage_above_500ms -0.438967 +/- 0.212394 Quality and user engagement wins for random packet loss characterization in high bandwidth peer_video_freeze_percentage -0.379246 +/- 0.124718 peer_video_freeze_percentage_above_500ms -0.541780 +/- 0.141212 peer_neteq_plc_cng_perc -0.242295 +/- 0.137200 total_talk_time 0.154204 +/- 0.148788 Reliability and quality wins for cellular low bandwidth classification connection_drop_rate -0.195908 +/- 0.127956 last_minute_quality_regression_v1 -0.198618 +/- 0.124958 last_minute_quality_regression_v2 -0.188115 +/- 0.138033 peer_neteq_plc_cng_perc -0.359957 +/- 0.191557 peer_video_freeze_percentage -0.653212 +/- 0.142822 Reliability and quality wins for cellular high bandwidth classification avg_sender_video_encode_fps 0.152003 +/- 0.046807 avg_sender_video_qp -0.228167 +/- 0.041793 avg_video_quality_score 0.296694 +/- 0.043079 avg_video_sent_bitrate 0.430266 +/- 0.092045 Future plans for applying ML to RTC From our project execution and experimentation on production clients, we noticed that a ML-based approach is more efficient in targeting, end-to-end monitoring, and updating than traditional hand-tuned rules for networking. However, the efficiency of ML solutions largely depends on data quality and labeling (using simulations or production logs). By applying ML-based solutions to solving network prediction problems – congestion in particular – we fully leveraged the power of ML. In the future, we will be consolidating all the network characterization models into a single model using the multi-task approach to fix the inefficiency due to redundancy in model download, inference, and so on. We will be building a shared representation model for the time series to solve different tasks (e.g., bandwidth classification, packet loss classification, etc.) in network characterization. We will focus on building realistic production network scenarios for model training and validation. This will enable us to use ML to identify optimal network actions given the network conditions. We will persist in refining our learning-based methods to enhance network performance by considering existing network signals. The post Optimizing RTC bandwidth estimation with machine learning appeared first on Engineering at Meta. View the full article

The domain of GenAI and LLMs has been democratized and tasks that were once purely in the domain of AI/ML developers must now be reasoned with by regular application developers into everyday products and business logic. This is leading to new products and services across banking, security, healthcare, and more with generative text, images, and videos. Moreover, GenAI’s potential economic impact is substantial, with estimates it could add trillions of dollars annually to the global economy. Docker offers an ideal way for developers to build, test, run, and deploy the NVIDIA AI Enterprise software platform — an end-to-end, cloud-native software platform that brings generative AI within reach for every business. The platform is available to use in Docker containers, deployable as microservices. This enables teams to focus on cutting-edge AI applications where performance isn’t just a goal — it’s a necessity. This week, at the NVIDIA GTC global AI conference, the latest release of NVIDIA AI Enterprise was announced, providing businesses with the tools and frameworks necessary to build and deploy custom generative AI models with NVIDIA AI foundation models, the NVIDIA NeMo framework, and the just-announced NVIDIA NIM inference microservices, which deliver enhanced performance and efficient runtime. This blog post summarizes some of the Docker resources available to customers today. Docker Hub Docker Hub is the world’s largest repository for container images with an extensive collection of AI/ML development-focused container images, including leading frameworks and tools such as PyTorch, TensorFlow, Langchain, Hugging Face, and Ollama. With more than 100 million pull requests for AI/ML-related images, Docker Hub’s significance to the developer community is self-evident. It not only simplifies the development of AI/ML applications but also democratizes innovation, making AI technologies accessible to developers across the globe. NVIDIA’s Docker Hub library offers a suite of container images that harness the power of accelerated computing, supplementing NVIDIA’s API catalog. Docker Hub’s vast audience — which includes approximately 27 million monthly active IPs, showcasing an impressive 47% year-over-year growth — can use these container images to enhance AI performance. Docker Hub’s extensive reach, underscored by an astounding 26 billion monthly image pulls, suggests immense potential for continued growth and innovation. Docker Desktop with NVIDIA AI Workbench Docker Desktop on Windows and Mac helps deliver NVIDIA AI Workbench developers a smooth experience on local and remote machines. NVIDIA AI Workbench is an easy-to-use toolkit that allows developers to create, test, and customize AI and machine learning models on their PC or workstation and scale them to the data center or public cloud. It simplifies interactive development workflows while automating technical tasks that halt beginners and derail experts. AI Workbench makes workstation setup and configuration fast and easy. Example projects are also included to help developers get started even faster with their own data and use cases. Docker engineering teams are collaborating with NVIDIA to improve the user experience with NVIDIA GPU-accelerated platforms through recent improvements to the AI Workbench installation on WSL2. Check out how NVIDIA AI Workbench can be used locally to tune a generative image model to produce more accurate prompted results: In a near-term update, AI Workbench will use the Container Device Interface (CDI) to govern local and remote GPU-enabled environments. CDI is a CNCF-sponsored project led by NVIDIA and Intel, which exposes NVIDIA GPUs inside of containers to support complex device configurations and CUDA compatibility checks. This simplifies how research, simulation, GenAI, and ML applications utilize local and cloud-native GPU resources. With Docker Desktop 4.29 (which includes Moby 25), developers can configure CDI support in the daemon and then easily make all NVIDIA GPUs available in a running container by using the –device option via support for CDI devices. docker run --device nvidia.com/gpu=all <image> <command> LLM-powered apps with Docker GenAI Stack The Docker GenAI Stack lets teams easily integrate NVIDIA accelerated computing into their AI workflows. This stack, designed for seamless component integration, can be set up on a developer’s laptop using Docker Desktop for Windows. It helps deliver the power of NVIDIA GPUs and NVIDIA NIM to accelerate LLM inference, providing tangible improvements in application performance. Developers can experiment and modify five pre-packaged applications to leverage the stack’s capabilities. Accelerate AI/ML development with Docker Desktop Docker Desktop facilitates an accelerated machine learning development environment on a developer’s laptop. By tapping NVIDIA GPU support for containers, developers can leverage tools distributed via Docker Hub, such as PyTorch and TensorFlow, to see significant speed improvements in their projects, underscoring the efficiency gains possible with NVIDIA technology on Docker. Securing the software supply chain Docker Hub’s registry and tools, including capabilities for build, digital signing, Software Bill of Materials (SBOM), and vulnerability assessment via Docker Scout, allow customers to ensure the quality and integrity of container images from end to end. This comprehensive approach not only accelerates the development of machine learning applications but also secures the GenAI and LLM software supply chain, providing developers with the confidence that their applications are built on a secure and efficient foundation. “With exploding interest in AI from a huge range of developers, we are excited to work with NVIDIA to build tooling that helps accelerate building AI applications. The ecosystem around Docker and NVIDIA has been building strong foundations for many years and this is enabling a new community of enterprise AI/ML developers to explore and build GPU accelerated applications.” Justin Cormack, Chief Technology Officer, Docker “Enterprise applications like NVIDIA AI Workbench can benefit enormously from the streamlining that Docker Desktop provides on local systems. Our work with the Docker team will help improve the AI Workbench user experience for managing GPUs on Windows.” Tyler Whitehouse, Principal Product Manager, NVIDIA Learn more By leveraging Docker Desktop and Docker Hub with NVIDIA technologies, developers are equipped to harness the revolutionary power of AI, grow their skills, and seize opportunities to deliver innovative applications that push the boundaries of what’s possible. Check out NVIDIA’s Docker Hub library and NVIDIA AI Enterprise to get started with your own AI solutions. View the full article

Learn how to automate machine learning training and evaluation using scikit-learn pipelines, GitHub Actions, and CML.View the full article

Ready to become a SAS Certified Specialist in Statistics for Machine Learning? Here’s everything you need to know about the recently released certification from SAS. View the full article

Artificial intelligence (AI) and machine learning (ML) can play a transformative role across the software development lifecycle, with a special focus on enhancing continuous testing (CT). CT is especially critical in the context of continuous integration/continuous deployment (CI/CD) pipelines, where the need for speed and efficiency must be balanced with the demands for quality and […] View the full article

Learn how to enhance the quality of your machine learning code using Scikit-learn Pipeline and ColumnTransformer.View the full article

AISecOps, the application of DevSecOps principles to AI/ML and generative AI, means integrating security into models' life cycles. View the full article

The podcasts that would improve your ML knowledge.View the full article

At SAP Sapphire last year, Alphabet, Google's parent company, revealed that it leverages SAP S/4HANA systems on Google Cloud to manage its critical business operations, including Finance, Supply Chain, Logistics, Planning, and Allocations. This has allowed Alphabet to consolidate over 50 billion transactions and process multiple terabytes of data every month, and better support its global operations. To scale its business operations, Alphabet leverages many Google Cloud products and services using the ABAP SDK. Created by Google Cloud’s SAP Engineering team, the ABAP SDK provides an accurate representation of Google Cloud APIs in ABAP, providing an integration point between SAP and Alphabet’s internal systems, powering AI/ML solutions that drive decisions, and enabling business process automation. The ABAP SDK provides in-built functionality for authentication, security, data serialization, etc., making it easier for Alphabet developers to integrate with Google Cloud products and services, and drastically decreasing time-to-value! ABAP SDK: a story of engineering excellenceSince launching internally, ABAP SDK has been used for many Alphabet production use cases: to run mission-critical applications in finance, corporate asset management (IT, audio/video, real estate), supply chain (plan and procure to pay including forward/reverse logistics), business event notifications, exception alerting, business process monitoring, and data warehousing. Security and scalability were key requirements for Alphabet business users, which they were able to achieve quickly with the ABAP SDK. Alphabet also uses the ABAP SDK to run ML inference in the context of a SAP business transaction analytics, surfacing ML predictions. Here are some great productionized use cases from Google Cloud for inspiration: Stream information from SAP to multiple subscribers with Pub/Sub - SAP is the source of truth for many master and transaction data objects. SAP sends information to other applications to ensure data consistency across applications. Sub-second speed and highly reliable messaging is key. Alphabet uses SAP Streaming Pub/Sub messages for various use cases, including network equipment status for deriving trust management, order status tracking, stock availability, transaction posting status, order comments and changes exchanged from the commerce portal to the order processing team.Use Pub/Sub for orchestration - To scale bulk data processing submitted via APIs at Alphabet, non-SAP enterprise applications use SAP OData to submit bulk updates to the SAP Application Interface Framework (AIF). SAP AIF asynchronous processing completion raises a Google Cloud CPS message to confirm processing results to the API client. ABAP SDK plays a crucial role in orchestrating many use cases like these including jobs, business events, and logs.Automate translation of finance documents to multiple languages - SAP S/4 is the source of truth for statutory reporting in multiple local languages. Alphabet boundary systems send journal entry data in English, but the headers and line text of various finance documents must be maintained in the local language, as per the company’s code definition. Alphabet uses the ABAP SDK to seamlessly interact with Google Translate, helping finance documents to comply with system and legal requirements.Integrate AI/ML models on Google Cloud for better decision making - Before Alphabet began using SAP to manage IT hardware orders worldwide, order routing across fulfillment channels was a time-consuming process. While a rules-based decision table can predict cost-effective order routes in some cases, the majority of orders required manual review, which was highly inefficient. To address this issue, the Alphabet engineering team developed a BigQuery Machine Learning model to predict routes based on historic manual decisions, and used the ABAP SDK to read the BigQuery Machine Learning predictions in real time. This continual learning model has helped to predict routes more accurately, reducing the need for manual review and automating the shipping process.Offload attachments from expensive HANA Storage - Alphabet utilizes the ABAP SDK to streamline and simplify the management of attachments (documents) for all applications in the SAP landscape. Due to the scale of the required storage capacity, it was imperative to offload the storage of attachment documents from Hana DB in-memory to cheaper and highly secured Cloud Storage buckets. By using the ABAP SDK, we were able to seamlessly read and write attachments from SAP applications to Cloud Storage without any impact to SAP critical user journeys (CUJs). End-user access to documents is controlled strictly based on authorization given in SAP, leveraging the standard Google Cloud Security Model.From internal to generally availableOver the years, many of our customers have asked for ways to easily integrate their SAP systems with Google Cloud services. With the launch of the ABAP SDK last year, the same ABAP SDK that has been powering Alphabet’s business operations since 2019 is now generally available for Google Cloud customers. Download the SDK for free and onboard in a few simple steps. While our public documentation contains the latest product updates, we also invite you to collaborate and engage with us on the Google Cloud developer community forum. We also have a public repository on GitHub so that developers can clone sample programs and quickstarts into their development environment. We are excited to see and assist you in creating enterprise and business solutions using ABAP SDK. View the full article

For the past few years, Fidelity Investments has been moving a significant percentage of its applications to a cloud-based infrastructure. As part of that transition, Fidelity has consolidated its analytics data into its Enterprise Analytics Platform, which is engineered using the Snowflake Data Cloud, making it easier for teams and departments across the company to access the data they need. Fidelity’s data scientists use the AI/ML-enabled Enterprise Analytics Platform to process a large volume of structured and unstructured data for deeper insights and better decision-making. Historically, the platform was housed in physical servers. In 2020, Fidelity kicked off its digital transformation and established an Enterprise Data Lake (EDL) along with Data Labs. Recently, the company wanted to conduct parallel analytics in the cloud and decided to use Snowpark and Snowpark ML. Fidelity has two main enterprise data architecture guiding principles for its data scientists and data engineers: For data storage, Snowflake is the platform for storing all of the company’s structured and semi-structured analytical data in its Enterprise Data Lake and Data Labs. All of Fidelity’s storage abides by its data security framework and data governance policies, and provides a holistic approach to metadata management. For compute, Fidelity’s principles are to minimize the transfer of data across networks, avoid duplication of data, and process the data in the database — bringing the compute to the data wherever possible. Feature engineering in focus Fidelity creates and transforms features to improve the performance of its ML models. Some common feature engineering techniques include encoding, data scaling and correlation analysis. The company’s data science architecture team was running into computation pain points, especially around feature engineering. Feature engineering is a stage in the data science process — before refinement and after expansion and encoding — where data can be at its peak volume. Pandas DataFrames offer a flexible data structure to manipulate various types of data and apply a wealth of computations. However, the trade-off with Pandas DataFrames is the restriction of memory, including the size of the DataFrame in memory and the expansion of memory due to the space complexity of the computation being applied to the data. This was only exacerbated by the speed of single-node processing, where memory contention and distribution of work had limited resources. The team also considered Spark ML purely for the flexibility of distributed processing, but Spark involves complex configuration and tasks and required maintenance overhead for both the hardware and software. Fidelity wanted to leverage capabilities like parallel processing without the complexity of Spark, so the company turned to Snowpark ML. Benefits of Snowpark ML Snowpark ML includes the Python library and underlying infrastructure for end-to-end ML workflows in Snowflake. Fidelity decided to work with the Snowpark ML Modeling API for feature engineering because of the improved performance and scalability with distributed execution for common sklearn-style preprocessing functions. In addition to being simple to use, it offered a number of additional benefits: All the computation is done within Snowflake, enabling in-database processing. It handles large data volumes and scales both vertically and horizontally. The correlation and preprocessing computation linearly scales the size of data to Snowflake standard warehouse size. Data is not duplicated nor transferred across the network. It leverages extensive RBAC controls, enabling tightly managed security. Lazy evaluation avoids unnecessary computation and data transfer, and improves memory management. Comparing three scenarios The Fidelity team compared Snowpark ML for three different scenarios: MinMax scaling, one-hot encoding and Pearson correlation. MinMax scaling is a critical preprocessing step to get Fidelity’s data ready for modeling. For numerical values, Fidelity wanted to scale its data into a fixed range between zero and one. With Pandas, the performance is fine for small data sets but does not scale to large data sets with thousands or millions of rows. Snowpark ML eliminates all data movement and scales out execution for much better performance. Figure 1. Performance improvement of 77x with Snowpark ML, compared to in-memory processing for MinMax scaling. One-hot encoding is a feature transformation technique for categorical values. With Snowpark ML, the execution is much faster by leveraging the distributed parallel processing for the data transformation and eliminating the data read and write times. Figure 2. Performance improvement of 50x with Snowpark ML, compared to in-memory processing for one-hot encoding. By using Snowpark ML to derive Pearson product moment or Pearson correlation matrix, Fidelity achieved a magnitude of performance improvement by scaling the computation both vertically and horizontally. This is especially useful for use cases with large and wide data sets in which there are, for example, 29 million rows and over 4,000 columns. Figure 3. Performance improvement of 17x with Snowpark ML, compared to in-memory processing for Pearson correlation. Fidelity achieved significant time, performance and cost benefits by bringing the compute closer to the data and increasing the capacity to handle more load. By speeding up computations, the company’s data scientists now iterate on features faster. Those time savings have allowed the team to become more innovative with feature engineering, explore new and different algorithms, and improve model performance. For more details, check out Fidelity’s full presentation on Snowpark ML for feature engineering. Ready to start building models of your own with Snowpark ML? Refer to Snowflake’s developer documentation for technical details, or try it for yourself with our step-by-step quickstart. The post Fidelity Optimizes Feature Engineering With Snowpark ML appeared first on Snowflake. View the full article

Here are six distinct technology trends that are poised to be particularly influential for DevOps in the new year.View the full article

This week on KDnuggets: Discover GitHub repositories from machine learning courses, bootcamps, books, tools, interview questions, cheat sheets, MLOps platforms, and more to master ML and secure your dream job • Data engineers must prepare and manage the infrastructure and tools necessary for the whole data workflow in a data-driven company • And much, much more!View the full article

The blog covers machine learning courses, bootcamps, books, tools, interview questions, cheat sheets, MLOps platforms, and more to master ML and secure your dream job.View the full article

Today, AWS announces the general availability of Amazon Elastic Compute Cloud (Amazon EC2) Capacity Blocks for ML. You can use EC2 Capacity Blocks to reserve GPU instances in an Amazon EC2 UltraCluster for a future date for the amount of time that you require to run your machine learning (ML) workloads. This is an innovative way to reserve capacity where you can schedule GPU instances to be available on a future date for just the amount of time that you require those instances. View the full article

Snowflake has invested heavily in extending the Data Cloud to AI/ML workloads, starting in 2021 with the introduction of Snowpark, the set of libraries and runtimes in Snowflake that securely deploy and process Python and other popular programming languages. Since then, we’ve significantly opened up the ways Snowflake’s platform, including its elastic compute engine can be used to accelerate the path from AI/ML development to production. Since Snowpark takes advantage of that scale and performance of Snowflake’s logically integrated but physically separated storage and compute, our customers are seeing a median of 3.5 times faster performance and 34% lower costs for their AI/ML and data engineering use cases. As of September 2023, we’ve already seen many organizations benefit from bringing processing directly to the data, with over 35% of Snowflake customers using Snowpark on a weekly basis. To further accelerate the entire ML workflow from development to production, the Snowflake platform continues to evolve with a new development interface and more functionality to securely productionize both features and models. Let’s unpack these announcements! ... View the full article

This list of machine learning algorithms is a good place to start your journey as a data scientist. You should be able to identify the most common models and use them in the right applications.View the full article