Streaming Ecosystem besides Kinesis


The Extended Streaming Ecosystem

Amazon MSK (Managed Streaming for Apache Kafka)

  • What it is: Managed Kafka.

  • The Context: Kinesis is AWS's proprietary version of Kafka. MSK is actual open-source Kafka managed by AWS.

  • Data Engineer Note: If you are building a new system on AWS, you usually choose Kinesis because it's easier. If you are migrating an existing Kafka cluster from on-premise, you choose MSK.

  • Equivalents:

    • GCP: Pub/Sub is the native rival, but GCP often partners with Confluent Cloud for Kafka.

    • Azure: Azure Event Hubs (which has a "Kafka Compatibility" mode).


Last updated