Databases

Amazon Relational Database Service (Amazon RDS) for MySQL now supports MySQL minor versions 8.0.41 and 8.4.4. We recommend that you upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of MySQL, and to benefit from the bug fixes, performance improvements, and new functionality added by the MySQL community. Learn more about the enhancements in RDS for MySQL 8.0.41 and 8.4.4 in the Amazon RDS user guide. You can leverage automatic minor version upgrades to automatically upgrade your databases to more recent minor versions during scheduled maintenance windows. You can also leverage Amazon RDS Managed Blue/Green deployments for safer…

In the ever-evolving landscape of cloud computing and data management, AWS has consistently been at the forefront of innovation. One of the groundbreaking developments in recent years is zero-ETL integration, a set of fully managed integrations by AWS that minimizes the need to build extract, transform, and load (ETL) data pipelines. This post will explore brief history of zero-ETL, its importance for customers, and introduce an exciting new feature: history mode for Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon Relational Database Service (Amazon RDS) for MySQL, and Amazon DynamoDB zero-ETL integration with Amazon Redshift. …

It is the 21st century and you are leading a fast-growing fintech startup that is about to hit a breaking point. The data team has doubled in size over six months, but chaos is reigning. Analysts are wasting hours reconciling conflicting reports, engineers are scrambling to fix broken pipelines, and leaders can’t agree on priorities. […]View the full article

You work with data to gain insights, improve decisions, and develop new ideas. With more and more data coming from all sorts of places, it’s super important to have a good data plan. That’s where big data integration comes in! It’s all about combining data from different sources to get a complete picture. For today’s […]View the full article

Big data is now crucial for driving business decisions. Companies are tapping into it to gain valuable insights and make smarter moves. To unlock this power, they’re using tools like data warehouses, BI tools, and cloud storage. One key innovation? The semantic layer: Its role is simple—standardize data definitions, making them more accessible and easier […]View the full article

Whether in healthcare or the retail industry, everyone needs data to succeed in their business. Data helps make clear decisions and helps businesses understand people and their needs. That is why data integration in business intelligence is very important. In this blog, you will explore data integration in business intelligence, its frameworks and components, its […]View the full article

The current data-centric environment changes how organizations handle business information by implementing cloud data integration methods. By seamlessly connecting different data sources, companies gain real-time insights that drive smarter decisions and improve daily operations. Like for example Netflix, its advanced cloud strategies help process a staggering 550 billion events every day, generating 1.3 petabytes of […]View the full article

Amazon Relational Database Service (RDS) for MySQL announces Amazon RDS Extended Support minor version 5.7.44-RDS.20250103. We recommend that you upgrade to this version to fix known security vulnerabilities and bugs in prior versions of MySQL. Learn more about the bug fixes and patches in this version in the Amazon RDS User Guide. Amazon RDS Extended Support provides you more time, up to three years, to upgrade to a new major version to help you meet your business requirements. During Extended Support, Amazon RDS will provide critical security and bug fixes for your RDS for MySQL databases after the community ends support for a major version. You can run your MySQL da…

Amazon Relational Database Service (RDS) for PostgreSQL now supports the latest minor versions 17.3, 16.7, 15.11, 14.16, and 13.19. We recommend that you upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of PostgreSQL, and to benefit from the bug fixes added by the PostgreSQL community. This release also includes updates for PostgreSQL extensions such as pg_active 2.1.4, pg_cron 1.6.5, pg_partman 5.2.4, and others. You can use automatic minor version upgrades to automatically upgrade your databases to more recent minor versions during scheduled maintenance windows. You can also use Amazon RDS Blue/Green deployments for RDS for…

A new minor version of Microsoft SQL Server is now available on Amazon RDS for SQL Server, providing performance enhancements and security fixes. Amazon RDS for SQL Server now supports this latest minor version of SQL Server 2022 across the Express, Web, Standard, and Enterprise editions. We encourage you to upgrade your Amazon RDS for SQL Server database instances at your convenience. You can upgrade with just a few clicks in the Amazon RDS Management Console or by using the AWS CLI. Learn more about upgrading your database instances from the Amazon RDS User Guide. The new minor version is SQL Server 2022 CU17 - 16.0.4175.1. This minor version is available in all A…

In the rapidly evolving world of data and analytics, organizations are constantly seeking new ways to optimize their data infrastructure and unlock valuable insights. Amazon Redshift is changing the game for thousands of businesses every day by making analytics straightforward and more impactful. Fully managed, AI powered, and using parallel processing, Amazon Redshift helps companies uncover insights faster than ever. Whether you’re a small startup or a big player, Amazon Redshift helps you make smart decisions quickly and with the best price-performance at scale. Amazon Redshift Serverless is a pay-per-use serverless data warehousing service that eliminates the need for…

Amazon Redshift Serverless announces reduction in IP Address Requirements to 3 per Subnet. When using Amazon Redshift Serverless without Enhanced VPC Routing (EVR) enabled, you only need 3 free IP addresses in each subnet in your Amazon VPC. The new enhancement makes starting with Amazon Redshift Serverless easier, and you do not have to worry about free IP addresses in your Amazon VPC subnet network. Before this announcement, you must have at least 9 free IP addresses in your subnet when creating an Amazon Redshift Serverless workgroup (workgroup) or when updating your workgroup for the Redshift Processing Units (RPUs), you must have at least 10 free IP addresses in y…

Amazon Relational Database Service (Amazon RDS) for Oracle now supports the January 2025 Release Update (RU) for Oracle Database versions 19c and 21c. To learn more about Oracle RUs supported on Amazon RDS for each engine version, see the Amazon RDS for Oracle Release notes. If the auto minor version upgrade (AmVU) option is enabled, your DB instance is upgraded to the latest quarterly RU six to eight weeks after it is made available by Amazon RDS for Oracle in your AWS Region. These upgrades will happen during the maintenance window. To learn more, see the Amazon RDS maintenance window documentation. For more information about the AWS Regions where Amazon RDS for O…

Amazon Redshift announces the general availability of Query Editor V2 with Amazon Redshift in the Asia Pacific (Malaysia) region. Amazon Redshift Query Editor V2 makes data in your Amazon Redshift data warehouse and data lake more accessible with a web-based tool for SQL users such as data analysts, data scientists, and database developers. With Amazon Redshift Query Editor V2, users can explore, analyze, and collaborate on data. It reduces the operational costs of managing query tools by providing a web-based application that allows you to focus on exploring your data without managing your infrastructure. The Amazon Redshift Query Editor V2 is a separate web-based SQL…

A new minor version of Microsoft SQL Server is now available on Amazon RDS for SQL Server, providing performance enhancements and security fixes. Amazon RDS for SQL Server now supports this latest minor version of SQL Server 2019 across the Express, Web, Standard, and Enterprise editions. We encourage you to upgrade your Amazon RDS for SQL Server database instances at your convenience. You can upgrade with just a few clicks in the Amazon RDS Management Console or by using the AWS CLI. Learn more about upgrading your database instances from the Amazon RDS User Guide. The new minor version is SQL Server 2019 CU30 - 15.0.4415.2. This minor version is available in all A…

Amazon Redshift Concurrency Scaling is now available in the Asia Pacific (Malaysia) region. Amazon Redshift Concurrency Scaling elastically scales query processing power to provide consistently fast performance for hundreds of concurrent queries. Concurrency Scaling resources are added to your Redshift cluster transparently in seconds, as concurrency increases, to process queries without wait time. Amazon Redshift customers with an active Redshift cluster earn up to one hour of free Concurrency Scaling credits, which is sufficient for the concurrency needs of most customers. Concurrency scaling allows you to specify usage control providing customers with predictability …

Amazon RDS Custom for SQL Server now offers enhanced storage and performance capabilities, supporting up to 64TiB of storage and 256,000 I/O operations per second (IOPS) with io2 Block Express volumes. This represents an improvement from the previous limit of 16 TiB and 64,000 IOPS with io2 Block Express. These enhancements enable transactional databases and data warehouses to handle larger workloads on a single Amazon RDS Custom for SQL Server database instance. The support for 64TiB and 256,000 IOPS with io2 Block Express for Amazon RDS Custom for SQL Server is now generally available in all AWS regions where both Amazon RDS io2 Block Express volumes and Amazon RDS C…

Choosing the right data transformation tool can make all the difference for efficient data workflows. Coalesce and dbt are two of the most popular choices that bring unique features to the table for data teams. While dbt is known for its SQL-based, modular approach to transformations, Coalesce provides a low-code, column-aware interface with automation capabilities. […]View the full article

Given the era of big data, organizations are producing and analyzing enormous amounts of data daily. They use tools that enable streamlining data ingestion, transformation, and analysis to try to understand it all. Two of the most popular tools on the modern data stack, dbt (Data Build Tool) and Hevo, occupy different but complementary spaces. […]View the full article

With growing businesses, marketing teams are flooded with a wealth of data from various platforms such as social media, email campaigns, customer feedback, websites, and offline in-store. The real challenge lies in “how to integrate this data into a unified structure in a meaningful way ?”. This is where “Marketing Data Integration” comes into play. […]View the full article

Amazon Redshift now offers enhanced query monitoring capabilities, enabling you to efficiently identify and isolate performance bottlenecks. This feature provides comprehensive insights to track, evaluate, and diagnose query performance within data warehouses, eliminating the need to manually analyze system tables and logs. Accessible through the AWS console, enhanced query monitoring allows you to view performance history for trend analysis, detect workload changes and understand how query performance has changed over time and diagnose performance issues with query profiler. You can analyze a specific timeframe and find problematic queries, review performance trends, …

Amazon Redshift announces enhanced security defaults to help you adhere to best practices in data security and reduce the risk of potential misconfigurations. These changes include disabling public accessibility, enabling database encryption, and enforcing secure connections by default when creating a new data warehouse. The enhanced security defaults bring three key changes: First, public accessibility is disabled by default for all newly created provisioned clusters and clusters restored from snapshots. In this configuration, connections to clusters will only be permitted from client applications within the same Virtual Private Cloud (VPC). Second, database encryptio…

Is your business incapacitated due to slow and unreliable data pipelines in today’s hyper-competitive environment? Data pipelines are the backbone that guarantees real-time access to critical information for informed and quicker decisions. The data pipeline market is set to grow from USD 6.81 billion in 2022 to USD 33.87 billion by 2030 at a CAGR […]View the full article

Amazon Redshift is announcing the general availability of Multi-AZ deployments for RA3 clusters in the Asia Pacific (Thailand) and Mexico (Central) AWS regions. Redshift Multi-AZ deployments support running your data warehouse in multiple AWS Availability Zones (AZ) simultaneously and continue operating in unforeseen failure scenarios. A Multi-AZ deployment raises the Amazon Redshift Service Level Agreement (SLA) to 99.99% and delivers a highly available data warehouse for the most demanding mission-critical workloads. Enterprise customers with mission critical workloads require a data warehouse with fast failover times and simplified operations that minimizes impact t…

Today, Amazon Redshift announces the launch of history mode for zero-ETL integrations. This new feature enables you to build Type 2 Slowly Changing Dimension (SCD 2) tables on your historical data from databases, out-of-the-box in Amazon Redshift, without writing any code. History mode simplifies the process of tracking and analyzing historical data changes, allowing you to gain valuable insights from your data's evolution over time. With history mode, you can easily run advanced analytics on historical data, build lookback reports, and perform trend analysis across multiple zero-ETL data sources, including Amazon DynamoDB, Amazon RDS for MySQL, Amazon Aurora MySQL, an…

Today, Amazon Redshift announced the launch of three new SQL features for zero-ETL integrations: QUERY_ALL_STATES, TRUNCATECOLUMNS, and ACCEPTINVCHARS. Zero-ETL integrations enable you to break down data silos in your organization and run timely analytics and machine learning (ML) on the data from your databases. With the launch of these new features, Amazon Redshift further enhances the functionality and reliability of zero-ETL integrations, allowing customers to work more efficiently with their data while maintaining data integrity. The new SQL features provide significant benefits and further enhance the experience of using zero-ETL integrations. QUERY_ALL_STATES al…

Amazon Relational Database Service (Amazon RDS) for Oracle now offers Oracle Database Standard Edition 2 (SE2) with the License-Included (LI) purchase option in additional AWS Regions for R6i instance class. RDS for Oracle R6i LI instances are now available in Asia Pacific (Malaysia) and Canada West (Calgary). In the LI service model, you don’t need to separately purchase Oracle licenses. Amazon RDS for Oracle LI pricing includes the software license, the underlying hardware resources, and all database management capabilities. Simply launch an Oracle SE2 instance in the AWS Management Console or using the AWS CLI and specify the License-Included option. Configuration d…

Amazon Redshift extends support for Hexagonal Hierarchical Geospatial Indexing System, H3 for short, by adding two new H3 functions to Amazon Redshift’s previously announced H3 Indexing support in February 2024. H3 Indexing increases the performance of spatial queries at scale since the location information is pre-indexed. See this Amazon Big Data Blog on Amazon Redshift H3 Indexing for more information on the benefits and use-cases of H3 Indexing. H3_Center returns the centroid of an H3 cell ID from an input index, which can be used to compute the geometric center of an arbitrary area that can be represented by H3 indexed cells, for example by finding the H3 cell with…

Digital tools and technologies help organizations generate large amounts of data daily, requiring efficient governance and management. This is where the AWS data lake comes in. With the AWS data lake, organizations and businesses can store, analyze, and process structured and unstructured data of any size. This article will focus on how businesses and organizations […]View the full article

The right data integration platform is crucial for the effective management and analysis of data. Rivery offers robust capabilities in data integration and transformation, but it may not fit every business’s unique needs. Fortunately, there are several Rivery alternatives available, each with distinct features, pricing, and use cases. Explore these options and find the perfect […]View the full article

Every year, Gartner rolls out its Magic Quadrant for Data Integration Tools, a trusted guide for data leaders on the hunt for the perfect integration tool. Think of it as a cheat sheet that cuts through the noise—evaluating tools based on how well they perform, how innovative they are, and how clear their vision is […]View the full article

Amazon RDS for MariaDB now supports MariaDB Innovation Release 11.7 in the Amazon RDS Database Preview Environment, allowing you to evaluate the latest Innovation Release on Amazon RDS for MariaDB. You can deploy MariaDB 11.7 in the Amazon RDS Database Preview Environment that has the benefits of a fully managed database, making it simpler to set up, operate, and monitor databases. MariaDB 11.7 is the latest Innovation Release from the MariaDB community, and includes support for vector datatype, indexing, and search capabilities. MariaDB Innovation releases are supported by the community until the next Innovation release, whereas MariaDB Long Term Maintenance Releases,…

Let’s face it: Data engineering is like playing Tetris, always moving objects around to fit them into the right places. The data is never static; pipelines, schemas, transformations, workflows, and flow are always puzzles that must be solved. Yes, it is an unmistakable kind of job; however, let me assure you, it is not always […]View the full article

One of the most common things in data analytics is running the same analytics queries over and over again by different end users over various times and snapshots of the data. This action, in particular, makes running warehousing solutions expensive and time-consuming at the same time. How about, we store the results of such expensive […]View the full article

Have you ever felt like data engineering is evolving at the speed of light? With new tech emerging almost daily, it’s no surprise that staying ahead of the curve is harder than ever. As we step into the fantastic year 2025 ahead, the rate at which data engineering changes is at an all-time high. New […] View the full article

In today’s data-driven world, data lakes have emerged as the data architecture of choice when storing and analyzing large volumes of data. However, implementing a successful data lake requires diligent planning and design, as it can quickly become a data swamp with no additional value. This blog post will delve into data lake best practices, […]View the full article

The Oracle Database is used by many companies around the world as the basis for the storage and processing of information. It is well adopted across all markets, including the financial sector and healthcare, where data security and management is critical. A lot of companies rely on Oracle database to store and manage their critical […]View the full article

Amazon Aurora PostgreSQL is now available as a quick create vector store in Amazon Bedrock Knowledge Bases. With the new Aurora quick create option, developers and data scientists building generative AI applications can select Aurora PostgreSQL as their vector store with one click to deploy an Aurora Serverless cluster preconfigured with pgvector in minutes. Aurora Serverless is an on-demand, autoscaling configuration where capacity is adjusted automatically based on application demand, making it ideal as a developer vector store. Knowledge Bases securely connects foundation models (FMs) running in Bedrock to your company data sources for Retrieval Augmented Generation…

We know just how hard it is to run your marketing data. The variety of campaigns running through platforms, such as Google Ads, Facebook, and HubSpot, among others, gives the kind of information that would just flood you. This is why we talk about a marketing data warehouse today that is powerful enough to make […]View the full article

Matillion is a cloud-based ETL tool known for its user-friendly, low-code interface. It’s great for teams that want to get pipelines up and running quickly without heavy coding. It also integrates seamlessly with cloud platforms like Snowflake, BigQuery, and Redshift, making it a solid choice for companies already working in the cloud.Airflow, on the other […]View the full article

When it comes to data integration, Fivetran has established a solid reputation as one of the industry leaders. With its robust feature set, Fivetran has become a go-to option for many enterprises. But there’s a catch- Fivetran’s pricing model. It’s unique and depends on usage. However, Fivetran’s pricing model can be confusing and complex for […]View the full article

With so many data integration tools available these days, it can become very overwhelming to choose one that best suits your needs. Here in this blog post, I have broken down an all-comprehensive comparison of two leading platforms: Airbyte VS Stitch. So, let’s get into the main discussion. Whether you seek flexibility, scalability, or ease […]View the full article

When searching for a reliable data integration platform, many options might cross your mind. However, Hevo Data stands out as a no-code, fully managed solution. Recognized in G2’s Fall 2021 report, Hevo delivers unmatched ease of use, setup simplicity, and comprehensive support. Trusted by over 2000+ companies, including brands like Postman and Thoughtspot, Hevo enables […]View the full article

At Snowflake BUILD, we are introducing powerful new features designed to accelerate building and deploying generative AI applications on enterprise data, while helping you ensure trust and safety. These new tools streamline workflows, deliver insights at scale, and get AI apps into production quickly. Customers such as Skai have used these capabilities to bring their generative AI solution into production in just two days instead of months. Here’s how Snowflake Cortex AI and Snowflake ML are accelerating the delivery of trusted AI solutions for the most critical generative AI applications... View the full article

Amazon Redshift is an online, petabyte-scale Data Warehouse service. It is dedicated to enterprise use, collecting large amounts of data and extracting analysis and insights from it. Redshift helps organizations query large DBs in real-time. Nonetheless, Redshift provides flexibility in performance as long as the cost aspect is well-handled to minimize cloud expenses. In this […]View the full article

Every business based on data-driven insights in the modern data ecosystem needs effective ETL tools. Your choice of ETL will go a long way in affecting the efficiency, speed, and cost of your data operations. Among well-recognized ETL tools, Hevo and Matillion come with different capabilities that make it very important to understand their features, […]View the full article

As the dependency on high-quality, real-time data availability increases, the need for event/data streaming tools becomes increasingly crucial. Apache Kafka has become one of the most trending event streaming platforms, and its popularity has led to wide organizational acceptance in various functions related to large-scale real-time data streams. Today, we will explore the Top 5 […]View the full article

If you have decided to start your journey with cloud databases, you probably have encountered AWS RDS – Amazon Web Services Relational Database Service, and CDC – Change Data Capture. In this blog, you will learn about AWS RDS, what CDC is, and how to integrate AWS RDS CDC into your data operations. If you […]View the full article

In today’s fast-paced data environment, Change Data Capture (CDC) transforms how organizations handle and synchronize their expanding data volumes. According to the Market Analysis Report, the global data management market size was valued at USD 89.34 billion in 2022 and is expected to grow at a compound annual growth rate (CAGR) of 12.1% from 2023 […]View the full article

In today’s world of big data, it’s important for companies to quickly and effectively change and analyze large data sets to get useful information. Businesses need tools that help them gather, change, and use data easily so they can make smart decisions based on that data. Among the many ETL/ELT tools available, Matillion and dbt […]View the full article