This walkthrough uses credit card transaction streaming data. You can alert customers or update the application to mitigate risk.You transform the streaming data and generate ML predictions.The Redshift ML model is trained using historical transactional data. You build, train, and deploy an ML model using Redshift ML.An Amazon Redshift Streaming Ingestion materialized view is created on top of the data stream, which automatically ingests streaming data into Amazon Redshift.The data stream stores the incoming credit card transaction data.The EC2 instance simulates a credit card transaction application, which inserts credit card transactions into the Kinesis data stream.The following diagram illustrates the architecture and process flow. You train and build a Redshift ML model to generate real-time inferences against the streaming data. You set up an Amazon Redshift Streaming Ingestion materialized view on Amazon Redshift, where streaming data is received. Solution overviewīy following the steps outlined in this post, you’ll be able to set up a producer streamer application on an Amazon Elastic Compute Cloud (Amazon EC2) instance that simulates credit card transactions and pushes data to Kinesis Data Streams in real time. This post demonstrates how Amazon Redshift allows you to build near-real-time ML predictions by using Amazon Redshift streaming ingestion and Redshift ML features with familiar SQL language. Amazon Redshift streaming ingestion allows you to achieve low latency in the order of seconds while ingesting hundreds of megabytes of data into your data warehouse. We’re excited to launch Amazon Redshift Streaming Ingestion for Amazon Kinesis Data Streams and Amazon Managed Streaming for Apache Kafka (Amazon MSK), which enables you to ingest data directly from a Kinesis data stream or Kafka topic without having to stage the data in Amazon Simple Storage Service (Amazon S3). In this post, we show how Amazon Redshift can deliver streaming ingestion and machine learning (ML) predictions all in one platform.Īmazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL.Īmazon Redshift ML makes it easy for data analysts and database developers to create, train, and apply ML models using familiar SQL commands in Amazon Redshift data warehouses. For example, a financial institute can predict if a credit card transaction is fraudulent by running an anomaly detection program in near-real-time mode rather than in batch mode. Many organizations are realizing that near-real-time data ingestion along with advanced analytics opens up new opportunities. Traditionally, data warehouses are refreshed in batch cycles, for example, monthly, weekly, or daily, so that businesses can derive various insights from them. The importance of data warehouses and analytics performed on data warehouse platforms has been increasing steadily over the years, with many businesses coming to rely on these systems as mission-critical for both short-term operational decision-making and long-term strategic planning.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |