CertificationService Delivery Program
Amazon Managed Streaming for Apache Kafka (MSK)
We make it easy for our customers to build real-time data streaming solutions and implement an event driven architecture that can scale automatically without any infrastructure to manage.
Our services make it easy to ingest, process and analyse streaming data, providing customers a modern scalable, secure and cost-effective streaming solution.
Overview
Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that enables you to build and run applications that use Apache Kafka to process streaming data. Amazon MSK provides the control-plane operations, such as those for creating, updating, and deleting clusters. It lets you use Apache Kafka data-plane operations, such as those for producing and consuming data. It also runs open-source versions of Apache Kafka. This means existing applications, tooling, and plugins from partners and the Apache Kafka community are supported without requiring changes to application code.
Amazon Managed Streaming for Apache Kafka or Amazon MSK allows you to run applications that utilize Apache Kafka within AWS. Kafka provides a platform for stream processing and operates as a publisher/subscriber based durable messaging system. Its key features are the ability to intake data with extreme fault tolerance, allowing for continuous streams of these records that preserve the integrity of the data, including the order in which it was received.
Apache Kafka then acts as a buffer between these data producing entities and the customers that are subscribed to it. Subscribers receive information from Kafka topics on a first in, first out basis or FIFO, allowing the subscriber to have a correct timeline of the data that was produced.
The diagram below highlights how it works in a typical scenario.
Ingest and process log and event streams
Capture events with MSK, and then express your stream processing logic within Apache Zeppelin notebooks to derive insights from data streams in milliseconds.
Run centralised state or data buses
Use Amazon MSK and the Apache Kafka log structure to form real-time, centralised, and privately accessible data buses.
Power your event-driven systems
Ingest and respond to digital changes occurring throughout your applications and business infrastructure in real time.
With Amazon MSK, you pay only for what you use. There are no minimum fees or upfront commitments. You pay for the time your broker instances run, the storage you use monthly and standard data transfer fees for data in and out of your cluster. You do not pay for Apache ZooKeeper nodes that Amazon MSK provisions for you, or data transfer that occurs between brokers and nodes within clusters. For an accurate calculator, visit this link.
Amazon Managed Streaming for Apache Kafka or Amazon MSK allows you to run applications that utilize Apache Kafka within AWS. Kafka provides a platform for stream processing and operates as a publisher/subscriber based durable messaging system. Its key features are the ability to intake data with extreme fault tolerance, allowing for continuous streams of these records that preserve the integrity of the data, including the order in which it was received.
Apache Kafka then acts as a buffer between these data producing entities and the customers that are subscribed to it. Subscribers receive information from Kafka topics on a first in, first out basis or FIFO, allowing the subscriber to have a correct timeline of the data that was produced.
Ingest and process log and event streams
Capture events with MSK, and then express your stream processing logic within Apache Zeppelin notebooks to derive insights from data streams in milliseconds.
Run centralised state or data buses
Use Amazon MSK and the Apache Kafka log structure to form real-time, centralised, and privately accessible data buses.
Power your event-driven systems
Ingest and respond to digital changes occurring throughout your applications and business infrastructure in real time.
With Amazon MSK, you pay only for what you use. There are no minimum fees or upfront commitments. You pay for the time your broker instances run, the storage you use monthly and standard data transfer fees for data in and out of your cluster. You do not pay for Apache ZooKeeper nodes that Amazon MSK provisions for you, or data transfer that occurs between brokers and nodes within clusters. For an accurate calculator, visit this link.
Want to find out more?
Got questions? We’ve got answers! Find out how our team can get you up and running with Amazon MSK today.