Consumers

• aka Subscribers

• They read data from a topic (identified by name)
• They know which broker to read from
• Data is read in order from each partition
• Consumers actually poll data to get the latest data

Consumer Group

• Consumers read data in consumer groups
• Each consumer within a group reads from exclusive partitions
• If you have more consumers than partitions, some consumers will be inactive
• If one consumer reads then offset increases and others cannot read it

Consumer Offsets

• Kafka stores the offset at which a consumer group has been reading
• The offsets are committed live in Kafka topic named __consumer_offsets
• If consumer dies it can start back from where it left off

Consumer Configs

• https://kafka.apache.org/20/documentation.html#consumerconfigs
• https://kafka.apache.org/documentation/#consumerconfigs
• bootstrap.servers
• key.deserializer
• value.deserializer
• group.id — Consumer Group ID
• auto.offset.reset
  • earliest
  • latest
  • none
• enable.auto.commit
• isolation.level
  • read_committed
  • read_uncommitted
• partition.assignment.strategy
• max.poll.interval.ms
  • The max delay between invocations of poll()
• max.poll.records
  • The max number of records returned in a single call to poll()

Delivery Semantics for Consumers

• Consumers choose when to commit offsets
• 3 delivery semantics:
  • At least once
  • At most once
  • Exactly once

• Consumer Group
• Inactive Consumer
• Delivery semantics