Search the Community

A fundamental requirement for any data-driven organization is to have a streamlined data delivery mechanism. With organizations collecting data at a rate like never before, devising data pipelines for adequate flow of information for analytics and Machine Learning tasks becomes crucial for businesses. As organizations gather information from multiple sources and data can come in […]View the full article

Docker, in collaboration with Snowflake, introduces an enhanced level of developer productivity when you leverage the power of Docker Desktop with Snowpark Container Services (private preview). At Snowflake BUILD, Docker presented a session showcasing the streamlined process of building, iterating, and efficiently managing data through containerization within Snowflake using Snowpark Container Services. Watch the session to learn more about how this collaboration helps streamline development and application innovation with Docker, and read on for more details. Docker Desktop with Snowpark Container Services helps empower developers, data engineers, and data scientists with the tools and insights needed to seamlessly navigate the intricacies of incorporating data, including AI/ML, into their workflows. Furthermore, the advancements in Docker AI within the development ecosystem promise to elevate GenAI development efforts now and in the future. Through the collaborative efforts showcased between Docker and Snowflake, we aim to continue supporting and guiding developers, data engineers, and data scientists in leveraging these technologies effectively. Accelerating deployment of data workloads with Docker and Snowpark Why is Docker, a containerization platform, collaborating with Snowflake, a data-as-a-service company? Many organizations lack formal coordination between data and engineering teams, meaning every change might have to go through DevOps, slowing project delivery. Docker Desktop and Snowpark Container Services (private preview) improve collaboration between developers and data teams. This collaboration allows data and engineering teams to work together, removing barriers to enable: Ownership by streamlining development and deployment Independence by removing traditional dependence on engineering stacks Efficiency by reducing resources and improving cross-team coordination With the growing number of applications that rely on data, Docker is invested in ensuring that containerization supports the changing development landscape to provide consistent value within your organization. Streamlining Snowpark deployments with Docker Desktop Docker Desktop provides many benefits to data teams, including improving data ingestion or enrichment and improving general workarounds when working with a data stack. Watch the video from Snowflake BUILD for a demo showing the power of Docker Desktop and Snowpark Container Services working together. We walk through: How to create a Docker Image using Docker Desktop to help you drive consistency by encapsulating your code, libraries, dependencies, and configurations in an image. How to push that image to a registry to make it portable and available to others with the correct permissions. How to run the container as a job in Snowpark Container Services to help you scale your work with versioning and distributed deployments. Using Docker Desktop with Snowpark Container Services provides an enhanced development experience for data engineers who can develop in one environment and deploy in another. For example, with Docker Desktop you can create on an Arm64 platform, yet deploy to Snowpark, an AMD64 platform. This functionality shows multi-platform images, so you can have a great local development environment and still deploy to Snowpark without any difficulty. Boosting developer productivity with Docker AI In alignment with Docker’s mission to increase the time developers spend on innovation and decrease the time they spend on everything else, Docker AI assists in streamlining the development lifecycle for both development and data teams. Docker AI, available in early access now, aims to simplify current tasks, boosting developer productivity by offering context-specific, automated guidance. When using Snowpark Container Services, deploying the project to Snowpark is the next step once you’ve built your image. Leveraging its trained model on Snowpark documentation, Docker AI offers relevant recommendations within your project’s context. For example, it autocompletes Docker files with best practice suggestions and continually updates recommendations as projects evolve and security measures change. This marks Docker’s initial phase of aiding the community’s journey in simplifying using big data and implementing context-specific AI guidance across the software development lifecycle. Despite the rising complexity of projects involving vast data sets, Docker AI provides support, streamlining processes and enhancing your experience throughout the development lifecycle. Docker AI aims to deliver tailored, automated advice during Dockerfile or Docker Compose editing, local docker build debugging, and local testing. Docker AI leverages the wealth of knowledge from the millions of long-time Docker users to autogenerate best practices and recommend secure, updated images. With Docker AI, developers can concentrate more on innovating their applications and less time on tools and infrastructure. Sign up for the Docker AI Early Access Program now. Improving the collaboration across development and data teams Our continued investment in Docker Desktop and Docker AI, along with our key collaborators like Snowflake, help you streamline the process of building, iterating, and efficiently managing data through containerization. Download Docker Desktop to get started today. Check with your admins — you may be surprised to find out your organization is already using Docker! Learn more Review Snowpark Container Services GitHub documentation. Follow the Snowflake tutorial to leverage your Snowflake data and build a Docker Image. Learn more about LLM and Hugging Face. Sign up for the Docker AI Early Access Program. View the full article

DevOps has shown to be successful in streamlining the product delivery process over time. A well-structured framework for maximizing the value of corporate data had to be established when firms all over the world adopted a data-driven strategy. These data-driven insights enabled consumers to make wise decisions based on verifiable evidence rather than relying on incorrect assumptions and forecasts. To better understand the distinction between DataOps and DevOps, it is meaningful to first define a clear definition. View the full article

Here are 5 trends that startups should keep an eye on ... https://www.snowflake.com/blog/five-trends-changing-startup-ecosystem/

A third era of software development management is emerging: It’s not waterfall or agile, it’s data-driven. View the full article

As analytics tools and machine learning capabilities mature, healthcare innovators are speeding up the development of enhanced treatments supported by Azure’s GPU-accelerated AI infrastructure powered by NVIDIA. Improving diagnosis and elevating patient care Man’s search for cures and treatments for common ailments has driven millennia of healthcare innovation. From the use of traditional medicine in early history to the rapid medical advances of the past few centuries, healthcare providers are locked in a constant search for effective solutions to old and emerging diseases and conditions. The pace of healthcare innovation has increased exponentially over the past few decades, with the industry absorbing radical changes as it transitions from a health care to a health cure society. From telemedicine, personalized wellbeing, and precision medicine to genomics and proteomics, all powered by AI and advanced analytics, modern medical researchers can access more supercomputing capabilities than ever before. This quantum leap in computational capability, powered by AI, enables healthcare services dissemination and consumption in ways, and at a pace, that were previously unimaginable. Today, health and life sciences leaders leverage Microsoft Azure high-performance computing (HPC) and purpose-built AI infrastructure to accelerate insights into genomics, precision medicine, medical imaging, and clinical trials, with virtually no limits to the computing power they have at their disposal. These advanced computing capabilities are allowing healthcare providers to gain deeper insights into medical data by deploying analytics and machine learning tools on top of clinical simulation data, increasing the accuracy of mathematical formulas used for molecular dynamics and enhancing clinical trial simulation. By utilizing the infrastructure as a service (IaaS) capabilities of Azure HPC and AI, healthcare innovators can overcome the challenges of scale, collaboration, and compliance without adding complexity. And with access to the latest GPU-enabled virtual machines, researchers can fuel innovation through high-end remote visualization, deep learning, and predictive analytics. Data scalability powers rapid testing capabilities Take the example of the National Health Service, where the use of Azure HPC and AI led to the development of an app that could analyze COVID-19 tests at scale, with a level of accuracy and speed that is simply unattainable for human readers. This drastically improved the efficiency and scalability of analysis as well as capacity management. Another advance worth noting, is that with Dragon Ambient Experience (DAX), an AI-based clinical solution offered by Nuance, doctor-patient experiences are optimized through the digitization of patient conversations into highly accurate medical notes, helping ensure high-quality care. By freeing up time for doctors to engage with their patients in a more direct and personalized manner, DAX improves the patient experience, reducing patient stress and saving time for doctors. “With support from Azure and PyTorch, our solution can fundamentally change how doctors and patients engage and how doctors deliver healthcare.”—Guido Gallopyn, Vice President of Healthcare Research at Nuance. Another exciting partnership between Nuance and NVIDIA brings directly into clinical settings medical imaging AI models developed with MONAI, a domain-specific framework for building and deploying imaging AI. By providing healthcare professionals with much needed AI-based diagnostic tools, across modalities and at scale, medical centers can optimize patient care at fractions of the cost compared to traditional health care solutions. “Adoption of medical imaging AI at scale has traditionally been constrained by the complexity of clinical workflows and the lack of standards, applications, and deployment platforms. Our partnership with Nuance clears those barriers, enabling the extraordinary capabilities of AI to be delivered at the point of care, faster than ever.”—David Niewolny, Director of Healthcare Business Development at NVIDIA. GPU-accelerated virtual machines are a healthcare game changer In the field of medical imaging, progress relies heavily on the use of the latest tools and technologies to enable rapid iterations. For example, when Microsoft scientists sought to improve on a state-of-the-art algorithm used to screen blinding retinal diseases, they leveraged the power of the latest NVIDIA GPUs running on Azure virtual machines. Using Microsoft Azure Machine Learning for computer vision, scientists reduced misclassification by more than 90 percent from 3.9 percent to a mere 0.3 percent. Deep learning model training was completed in 10 minutes over 83,484 images, achieving better performance than a state-of-the-art AI system. These are the types of improvements that can assist doctors in making more robust and objective decisions, leading to improved patient outcomes for patients. For radiotherapy innovator Elekta, the use of AI could help expand access to life-saving treatments for people around the world. Elekta believes AI technology can help physicians by freeing them up to focus on higher-value activities such as adapting and personalizing treatments. The company accelerates the overall treatment planning process for patients undergoing radiotherapy by automating time-consuming tasks such as advanced analysis services, contouring targets, and optimizing the dose given to patients. In addition, they rely heavily on the agility and power of on-demand infrastructure and services from Microsoft Azure to develop solutions that help empower their clinicians, facilitating the provision of the next generation of personalized cancer treatments. Elekta uses Azure HPC powered by NVIDIA GPUs to train its machine learning models with the agility to scale storage and compute resources as its research requires. Through Azure’s scalability, Elekta can easily launch experiments in parallel and initiate its entire AI project without any investment in on-premises hardware. “We rely heavily on Azure cloud infrastructure. With Azure, we can create virtual machines on the fly with specific GPUs, and then scale up as the project demands.”—Silvain Beriault, Lead Research Scientist at Elekta. With Azure high-performance AI infrastructure, Elekta can dramatically increase the efficiency and effectiveness of its services, helping to reduce the disparity between the many who need radiotherapy treatment and the few who can access it. Learn more Leverage Azure HPC and AI infrastructure today or request an Azure HPC demo. Read more about Azure Machine Learning: Multimodal 3D Brain Tumor Segmentation with Azure ML and MONAI. Practical Federated Learning with Azure Machine Learning. View the full article