[Tech Simplified] Kafka

Introduction to Kafka

Photo by Colton Duke on Unsplash

Read for free: https://liverungrow.medium.com/kafka-6567627dbdcf?sk=4c0704bb04ad44e7f8998573bc42d66d

Kafka is a distributed system consisting of servers and clients that communicate via TCP network protocol.

• To publish (write) and subscribe to (read) streams of events from sources like databases, sensors, mobile devices, cloud services, and software applications
• Storing these event streams durably for a defined period
• Processing event streams in real-time as well as retrospectively
• Routing the event streams to different destination technologies as needed.

Terminology

Event/Message:

• A single record that is being transmitted in Kafka.

Producer:

• Sources that produces Events or Messages to Kafka

Consumers:

• Those that subscribe and React to the Messages.

Consumer groups:

• Several consumers are grouped to consume a given topic. Consumers in the same consumer group are assigned the same group-id value. This concept ensures that a message is only read by a single consumer in the group.