Search the Community

Online and remote labor growth increases businesses' workforce potential but dilutes data across platforms.View the full article

Amazon Macie has introduced new managed data identifiers to expand its capabilities for discovering and identifying Stripe API keys, Google Cloud API keys, Driver’s license numbers (India) and national identification numbers (India) in Amazon Simple Storage Service (Amazon S3). Understanding the presence and location of such data in your S3 storage helps you to better plan data security, governance, and privacy of your organization. With over 100+ managed data identifiers, Macie helps protect your sensitive data at scale. View the full article

There is an ongoing trend in enterprise software: Using GraphQL as an aggregation layer to unify access to underlying data sources and REST APIs. Some call this the supergraph, and others refer to it as a meta layer. The idea is that by providing a more consumable layer to a wide range of internal sources, […] The post The Quest To Unify With GraphQL, The Modern Data API appeared first on DevOps.com. View the full article

Software systems continue to produce more and more data. And making use of it has proven benefits — so much so that many analysts have, over the years, referred to data as the new oil. As a result, the majority of organizations are expending effort into refining their data — in fact, a recent study […] View the full article

We are excited to announce that Amazon SageMaker Ground Truth now provides support so you can generate labeled synthetic data without collecting large amounts of real-world, manually labeled data. Amazon SageMaker provides two data labeling offerings, Amazon SageMaker Ground Truth Plus and Amazon SageMaker Ground Truth. You can use both options to identify raw data (such as images, text files, and videos) and add informative labels to create high-quality training datasets for your machine learning (ML) models. View the full article

This blog is part of a series in collaboration with our partners and customers leveraging the newly announced Azure Health Data Services. Azure Health Data Services, a platform as a service (PaaS) offering designed exclusively to support Protected Health Information (PHI) in the cloud, is a new way of working with unified data—providing care teams with a platform to support both transactional and analytical workloads from the same data store and enabling cloud computing to transform how we develop and deliver AI across the healthcare ecosystem. Microsoft Cloud for Healthcare and the Azure Health Data Services product engineering team are committed to global patient health information interoperability. We believe interoperability is table stakes to unlock and derive a more comprehensive assessment of the available clinical evidence... View the full article

Starting today, you can invoke SageMaker Autopilot from SageMaker Data Wrangler to automatically train, tune and build machine learning models. SageMaker Data Wrangler reduces the time to aggregate and prepare data for machine learning (ML) from weeks to minutes. SageMaker Autopilot automatically builds, trains, and tunes the best machine learning models based on your data, while allowing you to maintain full control and visibility. Previously, customers used Data Wrangler to prepare their data for machine learning and Autopilot for training machine learning models independently. With this unified experience, you can now prepare your data in SageMaker Data Wrangler and easily export to SageMaker Autopilot for model training. With just a few clicks, you can automatically build, train, and tune machine learning models, making it easier to automatically employ state-of-the-art feature engineering techinques, train high quality machine learning models, and gain insights from your data faster. View the full article

Today we are announcing the general availability of splitting data into train and test splits with Amazon SageMaker Data Wrangler. Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes. With SageMaker Data Wrangler, you can simplify the process of data preparation and feature engineering, and complete each step of the data preparation workflow, including data selection, cleansing, exploration, and visualization from a single visual interface. With SageMaker Data Wrangler’s data selection tool, you can quickly select data from multiple data sources, such as Amazon S3, Amazon Athena, Amazon Redshift, AWS Lake Formation, Snowflake, and Databricks Delta Lake. View the full article

The cloud is becoming a large cost center for many organizations. Though often touted as a cost-saver—and while many cloud migrations are driven by the desire to conserve IT costs—the cloud does create operating expenses at the same time it cuts down on capital expenses. Moreover, unlike traditional IT investments, cloud costs can be unpredictable […] The post Saving on AWS Costs with Data Classification appeared first on DevOps.com. View the full article

The global events of the last couple of years have introduced significant changes to how companies operate and the way we work, accelerating digital transformation for many as they seek the additional flexibility, scale, and cost savings of the cloud. More companies are choosing Azure SQL for their SQL Server workloads and it’s easy to see why. Azure SQL provides a full range of deployment options ranging from edge to cloud and a consistent unified experience that makes the most of your on-premises skills and experience. It’s very cost-effective, too, when you use the Azure Hybrid Benefit to maximize your on-premises licensing investments. Customers moving SQL Server workloads to the cloud have choices. Whether simply migrating to virtual machines (VMs) to offload infrastructure costs or modernizing on fully-managed database services that do more on your behalf, every choice on Azure is a great one… View the full article

Amazon Kinesis Data Analytics is now authorized as FedRAMP Moderate in US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon) and as FedRAMP High in AWS GovCloud (US-West) and AWS GovCloud (US-East). View the full article

Company continues observability data management innovation to help IT reduce costs and gain real-time analytics Seattle, WA, May 17, 2022 – Era Software, the observability data management company, today announced the private beta version of EraStreams, a no-code data pipeline that lets users integrate, transform, and route observability data to EraSearch, the company’s petabyte-scale log […] The post Era Software Introduces EraStreams for Scalable and Cost-Effective Observability Data Management appeared first on DevOps.com. View the full article

Change introduces risk. It’s one of those foundational principles of software development that most of us learned very early in our careers. Nevertheless, it always seems to keep cropping up in spite of those repeated life lessons. Our inability to foresee the impact of changes, even small ones, often leads to negative outcomes. As the […] The post Why Data Lineage Matters and Why it’s so Challenging appeared first on DevOps.com. View the full article

Description We're at an inflection point in data, where our data management solutions no longer match the complexity of organizations, the proliferation of data sources, and the scope of our aspirations to get value from data with AI and analytics. In this practical book, author Zhamak Dehghani introduces data mesh, a decentralized sociotechnical paradigm drawn from modern distributed architecture that provides a new approach to sourcing, sharing, accessing, and managing analytical data at scale. Dehghani guides practitioners, architects, technical leaders, and decision makers on their journey from traditional big data architecture to a distributed and multidimensional approach to analytical data management. Data mesh treats data as a product, considers domains as a primary concern, applies platform thinking to create self-serve data infrastructure, and introduces a federated computational model of data governance. Get a complete introduction to data mesh principles and its constituents Design a data mesh architecture Guide a data mesh strategy and execution Navigate organizational design to a decentralized data ownership model Move beyond traditional data warehouses and lakes to a distributed data mesh URL: https://www.oreilly.com/library/view/data-mesh/9781492092384/