Kafka is one idea repeated everywhere: an append-only file that readers walk at their own pace. Every command, guarantee, and failure mode on this board falls out of that single fact. Commands are tagged by which side of the system they touch: produce, consume, or administer.
A topic is a named stream of records, split into partitions, and each partition is an append-only file on disk. Nothing below is a separate feature bolted on. Each card is the same topic, orders, viewed from a different angle: how it is stored, how a record picks a partition, how readers divide the work, how it survives a dead machine, and how it is eventually reclaimed.
orders topic created with six partitions and replication factor three is simultaneously all five pictures below. It is six append-only logs (card one), each of which receives records whose key hashes to it (card two), which are divided among the members of every consumer group independently (card three), and each of which exists as three copies on three brokers with one of them elected leader (card four), and each of which discards old segments on whatever policy you set (card five). One create command, quoted in full on card one, produces all of that.An offset is a position in one partition, assigned by the broker on append and never reused. It is not a global sequence number and it means nothing in another partition. Records are immutable once written: there is no update and no delete by key, only appending and eventually reclaiming from the tail.
# create the topic: 6 partitions, 3 copies of each kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic orders --partitions 6 --replication-factor 3 # append records kafka-console-producer.sh --bootstrap-server localhost:9092 \ --topic orders # read the whole log from offset 0, then keep following it kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --from-beginning # read one partition from one exact offset kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --partition 0 --offset 3
A record carries an optional key. If the key is present, the producer hashes it and takes the remainder against the partition count, so the same key always lands on the same partition. If the key is null, the producer spreads records across partitions in batches instead.
Those five results are the real output of Kafka's default partitioner, which is the murmur2 hash with the sign bit cleared. Note that user-42 and user-108 both land on partition 4, and that cart-42 does not land with user-42: partitions are shared by unrelated keys, and only exact key equality guarantees co-location.
# produce keyed records, tab separated kafka-console-producer.sh --bootstrap-server localhost:9092 \ --topic orders --property parse.key=true --property key.separator=$'\t' # show the partition list and where each leader lives kafka-topics.sh --bootstrap-server localhost:9092 \ --describe --topic orders # read back with the key and partition printed, to verify placement kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --from-beginning \ --property print.key=true --property print.partition=true
order-9001 and every event for that order is ordered, while events for different orders are not ordered against each other, which is almost always what you actually want.A consumer group is a set of processes sharing one subscription. The group coordinator assigns each partition to exactly one member, so adding members increases parallelism only up to the partition count. A sixth partition and a seventh consumer means the seventh consumer sits idle.
The committed offset is stored per group, per partition, in an internal topic named __consumer_offsets. That is why a second group reading the same topic is completely independent: it has its own row in that topic and its own position. It is also why lag is defined per group, as the difference between the end of the partition and the group's committed offset.
# two shells running this line form a two-member group kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --group billing # the single most useful operational command: per-partition lag kafka-consumer-groups.sh --bootstrap-server localhost:9092 \ --describe --group billing # rewind the whole group to the start, group must be stopped kafka-consumer-groups.sh --bootstrap-server localhost:9092 \ --group billing --topic orders \ --reset-offsets --to-earliest --execute
Replication factor three means each partition is stored on three brokers. One replica is elected leader and serves all reads and writes for that partition; the other two are followers that continuously fetch from the leader. A follower that is caught up is in the in-sync replica set, written ISR.
With that state, the ISR holds two replicas. If the topic is configured to require two in-sync replicas and the producer asks for full acknowledgement, writes still succeed. If broker 2 also falls out, the ISR drops to one, and those writes now fail rather than silently accepting data that only exists on a single machine.
# require at least 2 of the 3 replicas to hold the write kafka-configs.sh --bootstrap-server localhost:9092 \ --alter --entity-type topics --entity-name orders \ --add-config min.insync.replicas=2 # producer side, the other half of the same guarantee acks=all # shows Leader, Replicas, and Isr per partition kafka-topics.sh --bootstrap-server localhost:9092 \ --describe --topic orders # list partitions whose leader is not the preferred one kafka-topics.sh --bootstrap-server localhost:9092 \ --describe --topic orders --under-replicated-partitions
acks=all alone means "all replicas currently in sync", which is one replica if the other two have fallen out, so it is not durable by itself. min.insync.replicas=2 alone does nothing if the producer does not wait. Quoting both together is the answer the interviewer is listening for.A log cannot grow forever, and the policy you choose changes what the topic fundamentally is. Time or size retention makes the topic a window of recent history. Compaction makes it a table: it keeps the most recent record for every key and discards earlier records for that key.
Offsets 0, 1, and 2 are gone and their offset numbers are never reassigned, so a compacted log has gaps. A consumer reading from the beginning now receives one record per key, which is exactly the state of the accounts table. Writing a record with a key and a null value, called a tombstone, deletes that key from the compacted result.
# window of history: keep 7 days kafka-configs.sh --bootstrap-server localhost:9092 \ --alter --entity-type topics --entity-name orders \ --add-config retention.ms=604800000 # table of latest state per key kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic accounts --partitions 6 --replication-factor 3 \ --config cleanup.policy=compact # both: compact, but also drop anything older than 30 days --config cleanup.policy=compact,delete --config retention.ms=2592000000
Every performance claim about Kafka comes from four decisions: writes are sequential, reads are served from the operating system page cache, the broker keeps no per-consumer state, and records move in batches. Understanding these is what lets you defend a number in an interview instead of guessing one.
A partition is appended to and never modified in place, so the disk head never seeks and both a spinning disk and a solid state drive run near their sequential ceiling. Written data lands in the page cache, which is memory managed by the operating system, and is flushed to disk in the background.
Consumers reading recent records therefore rarely touch the disk, because the bytes are still in the page cache from when they were written. A consumer that has fallen far behind does hit disk, which is one reason lag hurts the whole cluster and not only the slow consumer.
Because the record on disk is byte for byte what goes on the wire, the broker can hand a file region straight to the network socket instead of copying it into the application and back out.
acks=all with min.insync.replicas so the record exists on more than one machine before it is acknowledged. Say this out loud: it is the difference between a correct answer about data loss and a confused one.A traditional broker tracks which messages each consumer has taken and acknowledged, so its bookkeeping grows with the number of consumers and messages in flight. Kafka does not. The broker knows only where each partition's log ends. The consumer knows where it is, and stores that position by committing it to an ordinary internal topic.
Three consequences follow. Adding a tenth consumer group costs the broker almost nothing beyond the extra reads. Replaying is trivial, because rewinding means writing a smaller number into that offset topic. And an acknowledgement is not per record: committing offset 5107 asserts that everything before it is done, which is why a single bad record cannot be individually parked without extra machinery.
The producer accumulates records per partition and sends them as one compressed batch. Compressing a whole batch is far more effective than compressing each record, and the batch stays compressed on the broker's disk and is handed to consumers still compressed. The broker does not decompress it to serve reads.
The tradeoff is deliberate latency. Waiting a few milliseconds to fill a batch is what turns a per-record round trip into a bulk transfer, and it is usually the single most effective throughput change available.
| Setting | Typical | Effect |
|---|---|---|
| linger.ms | 0 to 20 | How long the producer waits to fill a batch before sending. Zero still batches under load, because records accumulate while a send is in flight. |
| batch.size | 16 to 256 KB | Maximum bytes per partition batch. If it fills first, the batch ships immediately regardless of linger. |
| compression.type | lz4 or zstd | lz4 is cheap and fast, zstd gives the best ratio at more processor cost. A measurable win on both network and storage. |
A partition is a directory of segments, each capped by size or age, not one growing file. The newest segment is open and being appended to and the rest are closed and immutable. Beside each segment sit an index from offset to byte position and an index from timestamp to offset, which is how a consumer seeks to a position or a point in time without scanning.
Retention then deletes whole closed segments rather than editing anything, which is why expiring data costs almost nothing.
# disk usage per broker, per partition kafka-log-dirs.sh --bootstrap-server localhost:9092 \ --describe --topic-list orders # decode a segment: offsets, keys, timestamps, batch boundaries kafka-dump-log.sh --print-data-log \ --files /var/lib/kafka/data/orders-0/00000000000000000000.log # when a segment rolls closed, which is when retention can act on it kafka-configs.sh --bootstrap-server localhost:9092 \ --alter --entity-type topics --entity-name orders \ --add-config segment.bytes=1073741824,segment.ms=604800000
These are the figures worth carrying into an interview. Say them as orders of magnitude with the assumption attached, never as precise guarantees, because every one of them moves with record size, batching, and hardware.
| Quantity | Order of magnitude | What moves it |
|---|---|---|
| Throughput per broker | hundreds of MB per second | Network card and sequential disk bandwidth are the ceiling. Small unbatched records collapse this by an order of magnitude. |
| Records per second per partition | tens of thousands | Rarely the real limit. One consumer's processing rate on that partition usually binds first. |
| End to end latency | single digit to tens of milliseconds | Producer linger, replication acknowledgement, and consumer poll interval. Kafka is low latency, not microsecond real time. |
| Partitions per cluster | low hundreds of thousands | Metadata and open file handles. Much higher since the metadata quorum moved inside Kafka, but still a real budget. |
| Partitions per topic | tens to low thousands | Consumer parallelism needed, weighed against rebalance time and file handle count. More is not free. |
| Retention | hours to indefinite | Broker disk, or object storage if tiered storage is enabled, which decouples retention from broker disk entirely. |
Kafka has a small vocabulary, and these eight objects cover essentially every design answer. Each card carries the commands or settings that create and inspect the thing it describes.
| Command | Scope | What it does |
|---|---|---|
| kafka-topics.sh --create | cluster | Creates the topic with its partition count and replication factor. |
| kafka-topics.sh --describe | topic | Per partition: leader broker, replica list, in-sync replica list. The first command to run when anything looks wrong. |
| kafka-topics.sh --alter --partitions 12 | topic | Increases the partition count. One way only: it can never be decreased. |
| kafka-configs.sh --alter | topic | Changes retention, cleanup policy, segment size, and the in-sync replica floor without recreating the topic. |
| kafka-reassign-partitions.sh | cluster | Moves partition replicas between brokers. This is how you add a broker and actually get data onto it. |
| Setting | Default | What it does |
|---|---|---|
| acks | all | 0 never waits, 1 waits for the leader only, all waits for every in-sync replica. Use all unless you can name why not. |
| enable.idempotence | true | The broker deduplicates producer retries using a producer id and a per-partition sequence number, so a retry cannot append the same record twice. |
| delivery.timeout.ms | 120000 | Total time the producer keeps trying before failing the send back to your code. This, not the retry count, is the real control. |
| max.in.flight.requests.per.connection | 5 | Requests in flight per connection. With idempotence on, ordering holds up to five. With it off, anything above one can reorder on retry. |
| partitioner.class | default | Override to control placement directly, for example to pin a large tenant to dedicated partitions. |
| Setting | Default | What it does |
|---|---|---|
| group.id | none | Names the group. Two processes sharing it split the partitions; two processes with different ones each receive everything. |
| auto.offset.reset | latest | Where to start when the group has no committed offset. earliest reads all retained history, latest skips it. A frequent cause of "my consumer sees nothing". |
| enable.auto.commit | true | Commits position on a timer, which can commit records you have not finished processing. Turn it off for anything that matters. |
| max.poll.records | 500 | Records handed back per poll call. Lower it when per-record work is slow. |
| max.poll.interval.ms | 300000 | If your code does not call poll again inside this window the group assumes the member died and reassigns its partitions. The classic cause of a permanent rebalance loop. |
| kafka-consumer-groups.sh --describe | group | Current offset, log end offset, and lag per partition, plus which member owns each partition. |
Committing is itself a produce, to the compacted internal topic __consumer_offsets, keyed by group, topic, and partition. Because that topic is compacted, it holds the latest position per key, which is exactly the group's state.
| Action | Where | What it means |
|---|---|---|
| commitSync | consumer | Blocks until the broker confirms the position. Slower, and the one to call after processing has actually succeeded. |
| commitAsync | consumer | Fire and forget with a callback. Faster, and requires a final synchronous commit during shutdown. |
| seek and seekToBeginning | consumer | Moves this consumer's position in memory, ignoring the committed value. The programmatic form of a replay. |
| --reset-offsets --to-datetime | group | Repositions a whole stopped group by wall clock time using the timestamp index. How a backfill is actually triggered. |
A transactional producer writes to several partitions and topics and then commits once. Consumers configured to read only committed data never see the records of an aborted transaction. This is what makes the consume, process, produce loop atomic, because the offset commit can be placed inside the same transaction as the output records.
| Setting or call | Side | What it does |
|---|---|---|
| transactional.id | producer | Stable name that lets the broker fence out a zombie instance of the same logical producer after a restart. |
| beginTransaction and commitTransaction | producer | Delimit the atomic unit. Aborting discards every buffered record across every partition involved. |
| sendOffsetsToTransaction | producer | Puts the consumer's offset commit inside the transaction. This one call is what exactly-once stream processing is built on. |
| isolation.level=read_committed | consumer | Hides aborted records and does not read past an open transaction. Without it a consumer sees uncommitted writes. |
A share group is a second kind of group whose members cooperatively consume the same partitions instead of owning them exclusively. Consumers can outnumber partitions, and each record is acknowledged individually, with a broker-side delivery count that eventually parks a record that keeps failing.
| Command or call | Scope | What it does |
|---|---|---|
| --group-type share | admin | Creates or addresses a share group rather than a classic consumer group. |
| acknowledge(ACCEPT) | consumer | Marks this one record done, rather than everything before it. |
| acknowledge(RELEASE) | consumer | Returns the record for redelivery to any member and increments its delivery count. |
| acknowledge(REJECT) | consumer | Declares the record unprocessable so it is never redelivered. |
Reach for this when work items are independent and you want consumers to scale past the partition count. Stay with classic consumer groups when per-key ordering matters, because share groups deliberately give ordering up.
Connect is a separate worker cluster that runs source connectors, which pull from an external system into Kafka, and sink connectors, which push from Kafka into an external system. It handles offset tracking, restarts, and scaling, so the answer to "how does data get from our database into Kafka" is usually a connector rather than application code.
The source connector family that matters most in interviews is change data capture, abbreviated CDC, which tails a database's write-ahead log and emits one record per row change, keyed by primary key. The result is a compacted topic that mirrors the table.
| Object | Direction | Typical use |
|---|---|---|
| source connector | into Kafka | Database change capture, file tailing, bridging an external queue. |
| sink connector | out of Kafka | Search index, data warehouse, object storage, cache warming. |
| single message transform | either | Per-record reshaping such as masking a field or routing by content, without a stream processing job. |
Streams is a client library, not a cluster: your application links it and scales by running more instances in a consumer group. It provides joins, aggregations, and windowing, keeping local state in an embedded key-value store that is mirrored to a compacted Kafka topic so an instance can rebuild after a crash.
| Concept | Backed by | What it gives you |
|---|---|---|
| KStream | topic | A record by record event stream, where every record is an independent fact. |
| KTable | compacted topic | Latest value per key. The table view of the same log. |
| state store | changelog topic | Local storage for aggregations and joins, restored from Kafka on restart. |
| windowed aggregation | state store | Counts and sums over tumbling, hopping, or session windows, with a grace period for late records. |
If a question asks for counting, joining, or enriching a stream, naming Streams or an equivalent processor such as Flink is enough. Reaching for a database to do it is the answer that gets probed.
Choosing the record key is the one Kafka decision that is expensive to change later. It fixes what is ordered, what is parallel, and where the load concentrates. Everything in this cluster is a consequence of that single choice.
The default partitioner takes the murmur2 hash of the key bytes, clears the sign bit, and takes the remainder against the partition count. The partition count is in the divisor, so changing it changes the answer for keys whose hash does not happen to divide the same way. Here is the full derivation for one key, growing a topic from six partitions to twelve.
user-108, which the producer serializes to its eight UTF-8 bytes before hashing.
key = "user-108"user-108 is appended to partition 10, while its entire history sits in partition 4. Two different consumers now own the old and new records for the same entity, and there is nothing coordinating them.
history in partition 4, new records in partition 10 ordering brokenuser-42 hashes to 1459644460, and 1459644460 % 6 = 4 while 1459644460 % 12 = 4. It stays on partition 4, which is what makes this failure so unpleasant: it is silent and it hits only some keys.
user-42: partition 4 before, partition 4 after unaffectedTwo guarantees come from a key, and only these two. Records with the same key always land on the same partition, so they are ordered relative to each other. Records with different keys have no ordering relationship at all, even if one was produced an hour after the other.
| Key choice | Ordered per | Parallelism |
|---|---|---|
| order id | one order | Very high. Millions of distinct orders spread evenly. |
| user id | one user | High, and it lets a consumer keep per-user state locally. |
| tenant id | one tenant | Poor if tenants differ in size. One large customer becomes a hot partition. |
| a constant | everything | None. Total ordering means one partition and one consumer, which is a design smell unless the volume is genuinely tiny. |
| null | nothing | Maximum. The producer fills one partition's batch, then switches, which keeps batches large. |
Hashing distributes keys evenly, not traffic. If one key carries a hundred times the volume of the others, its partition receives a hundred times the load and one consumer instance has to keep up alone. Adding partitions does not help, because the hot key still maps to exactly one of them.
| Fix | Cost | When it fits |
|---|---|---|
| composite key | narrower ordering | Key by tenant plus entity instead of tenant alone. Ordering per entity survives; ordering across the tenant does not. |
| salted key | ordering lost for that key | Append a small random suffix to spread one hot key over several partitions. Only safe when that key's records are independent. |
| custom partitioner | operational complexity | Route the known large tenant to a reserved set of partitions and hash everyone else normally. Explicit and predictable. |
Producers rarely bind first. Work the number out from how fast a single consumer instance can process a record, then leave headroom, because raising the count later is the expensive operation derived above.
The upper bound is not throughput but bookkeeping: every partition is an open file set on three brokers, adds to cluster metadata, and lengthens a rebalance. Tens of thousands of partitions on one cluster is ordinary, hundreds of thousands is where operational strain starts.
# create with headroom rather than growing later kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic orders --partitions 30 --replication-factor 3 # confirm each partition actually receives traffic kafka-run-class.sh kafka.tools.GetOffsetShell \ --bootstrap-server localhost:9092 --topic orders
Almost every follow-up question in a Kafka interview is a delivery question in disguise. The three semantics are not personality types, they are specific settings with specific failure windows, and the two walkthroughs below show exactly where each window opens.
| Guarantee | Producer side | Consumer side | What you accept |
|---|---|---|---|
| At most once | acks=0 or acks=1 with no retries | commit the offset before processing | Records can be lost and never reprocessed. Acceptable for metrics and sampled telemetry, almost nothing else. |
| At least once | acks=all with retries | commit the offset after processing | Records can be delivered twice. This is the default posture and the right answer for most systems, paired with an idempotent consumer. |
| Exactly once | enable.idempotence=true plus transactional.id | isolation.level=read_committed with the offset committed inside the transaction | Throughput cost and a hard boundary: the guarantee holds for Kafka to Kafka work only. |
The producer already prevents its own retries from duplicating, because idempotence is on by default. The duplicate that survives is on the consumer side, and it comes from the gap between doing the work and recording that you did it.
Reverse the order of the same two operations and the duplicate becomes a loss. This is what automatic offset committing can do to you without any explicit configuration choice, because it commits on a timer regardless of where your processing has reached.
Since duplicates are unavoidable in any at least once system, the consumer must make reprocessing harmless. There are three standard ways, and the right one depends on what the side effect touches.
| Technique | Where the state lives | Notes |
|---|---|---|
| natural idempotence | nowhere | The operation is already repeatable, such as setting a value rather than incrementing one. Free when the data model allows it. |
| unique constraint | your database | Insert a row keyed by a business identifier from the record inside the same transaction as the work. The second attempt violates the constraint and is discarded. |
| offset as a version | your database | Store the partition and offset with the row and only apply a record whose offset is higher than the stored one. Works well for state that is overwritten rather than accumulated. |
When a job consumes from one topic and produces to another and the processing genuinely cannot be made idempotent, transactions make the whole cycle atomic. Every one of these settings is required; a partial set gives you no guarantee at all.
# producer enable.idempotence=true transactional.id=enrich-orders-1 # stable per instance acks=all # consumer isolation.level=read_committed enable.auto.commit=false # the transaction commits offsets # topic min.insync.replicas=2 # the loop producer.beginTransaction() producer.send(outputRecord) producer.sendOffsetsToTransaction(offsets, consumerGroupMetadata) producer.commitTransaction()
If Kafka Streams is doing the work, setting its processing guarantee to exactly once configures all of this for you, which is a reasonable thing to say in an interview rather than reciting the settings.
These are the arrangements interviewers are listening for. Each card names the problem, the topic layout that solves it, and the follow-up question that always comes next.
One service writes what happened; several unrelated services react. The writer does not know who reads, and a new consumer can be added later without touching the writer or replaying anything through it, because the history is already in the topic.
# a new group reading all retained history from the beginning kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --group fraud --from-beginning # prove the groups are independent kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
A service cannot atomically commit a database transaction and a Kafka send. Doing both in sequence gives you the dual write problem: either the row is committed and the record is lost, or the record is published for a transaction that then rolled back.
A classic consumer group commits a watermark, so one unprocessable record blocks its whole partition until it is dealt with. Blocking is sometimes correct, but usually the record is moved aside so the partition keeps flowing.
# retry tier and terminal tier, same partition count as the source kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic orders.retry.5m --partitions 6 --replication-factor 3 kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic orders.dlq --partitions 6 --replication-factor 3 # inspect what has been parked kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders.dlq --from-beginning --property print.headers=true
Because reading does not consume, a bug fixed today can be applied to last week's traffic by moving a group's position backwards. This is the single most persuasive argument for a log over a queue, and it is worth stating explicitly in a design discussion.
# 1. rewind an existing group by time, group must be stopped kafka-consumer-groups.sh --bootstrap-server localhost:9092 \ --group billing --topic orders \ --reset-offsets --to-datetime 2026-08-09T00:00:00.000 --execute # 2. shadow run: a brand new group reads history with zero risk kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic orders --group billing-v2 --from-beginning # 3. move back a fixed number of records per partition kafka-consumer-groups.sh --bootstrap-server localhost:9092 \ --group billing --topic orders \ --reset-offsets --shift-by -50000 --execute
A compacted topic keyed by entity identifier holds the latest value for every key indefinitely. A new service can subscribe, read from the beginning to build a complete local view, then stay subscribed to keep it current. This is how configuration, feature flags, and reference data are distributed to many services without a shared database.
# the topic is the source of truth for current state per key kafka-topics.sh --bootstrap-server localhost:9092 \ --create --topic accounts --partitions 12 --replication-factor 3 \ --config cleanup.policy=compact --config min.cleanable.dirty.ratio=0.1 # a service builds its local view by reading all of it kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic accounts --from-beginning --property print.key=true # delete a key: produce the key with a null value (a tombstone) kafka-console-producer.sh --bootstrap-server localhost:9092 \ --topic accounts --property parse.key=true \ --property key.separator=: --property null.marker=NULL
Interviewers often say queue when they mean either. The distinction that matters is whether records are independent work items or an ordered history of something, because it decides the group type and the key.
| Requirement | Use | Why |
|---|---|---|
| Independent jobs, per-item retry, consumers scale past partitions | share group | Per record acknowledgement and delivery counts, with no partition exclusivity. |
| Ordered history per entity, several independent readers | consumer group | Ordering per key and independent offsets per group. |
| Delay or schedule an item for later | not Kafka | There is no per-record delay. Delay tiers are separate topics with a consumer that sleeps, which is coarse. A scheduler or a sorted structure elsewhere fits better. |
| Priority ordering across items | not Kafka | A log has one order, the order of arrival. Priority means separate topics per tier and a consumer that drains them in order. |
The cost of an additional consumer group is the read bandwidth, not extra storage or extra broker state, because every group reads the same file and stores only its own position. Ten teams consuming one topic is normal; ten copies of a topic is a design mistake.
All three read the same segments. Only the analytics group is far enough back to be reading from disk instead of page cache, which is the specific reason a badly lagging consumer degrades everyone: it evicts warm data.
Lag is the number of records between a group's committed offset and the end of the partition. It is the one metric that captures whether the system is keeping up, and its shape tells you which failure you have.
| Shape | Diagnosis | Action |
|---|---|---|
| All partitions rising together | under-provisioned | Add consumer instances up to the partition count, then look at per-record cost. |
| One partition rising alone | hot key or stuck consumer | Check for skew, and check whether that member is failing and rejoining. |
| Lag flat but non-zero | keeping up with a backlog | Nothing urgent. Throughput matches arrival rate but the deficit is not being repaid. |
| Sawtooth | rebalance loop | Processing exceeds the poll interval. Reduce records per poll or raise the interval. |
A Kafka cluster is a set of brokers that hold partition replicas, plus a metadata quorum that decides who leads what. Interviewers probe here to find out whether you have run this or only read about it.
Cluster metadata, meaning which topics exist, which broker leads which partition, and which replicas are in sync, is itself stored in an internal Kafka log replicated by a Raft quorum of controller nodes. The name for this mode is KRaft, short for Kafka Raft.
This replaced the external coordination service that older deployments ran alongside Kafka. If you are asked about ZooKeeper, the accurate answer is that it was removed entirely in the 4.0 release and current clusters do not run it.
| Role | Count | Holds |
|---|---|---|
| controller | 3 or 5 | The metadata log and the elected active controller. An odd number, because it is a Raft quorum. |
| broker | as many as needed | Partition replicas and the client connections. Scales with data and throughput. |
| combined mode | small clusters only | One process serving both roles. Convenient in development, discouraged in production. |
Whenever a member joins or leaves a group, partitions are reassigned. The old protocol stopped every consumer during that reassignment, so one restarting member paused the entire group. Two mechanisms fix this and both are worth naming.
| Mechanism | Effect | When to mention it |
|---|---|---|
| cooperative rebalancing | incremental | Only the partitions that actually move are revoked, so unaffected consumers keep processing throughout. |
| static membership | no rebalance | A member declares a stable instance identifier, so a rolling restart within the session timeout reclaims its own partitions without triggering reassignment. |
| server-side assignment | faster and lighter | The newer consumer group protocol moves assignment computation to the broker, removing the stop-the-world join barrier entirely. |
# the MEMBER-ID and HOST columns show current ownership
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
--describe --group billing --members --verbose
Compression usually takes the raw figure down by a factor of two to five for text-like payloads, so quoting the uncompressed number and then saying you expect compression to reduce it is the honest version. Tiered storage changes this calculation entirely by moving closed segments to object storage, leaving only recent data on broker disk.
Grouped by which failure they prevent, because that is how the follow-up question is usually phrased.
| Setting | Value |
|---|---|
| acks | all |
| min.insync.replicas | 2, with replication factor 3 |
| enable.idempotence | true |
| enable.auto.commit | false, commit after processing |
| unclean.leader.election.enable | false |
| Setting | Value |
|---|---|
| transactional.id | stable per producer instance |
| isolation.level | read_committed |
| consumer-side dedup key | carried in the record, never generated on receipt |
| Setting | Value |
|---|---|
| linger.ms | 5 to 20, the cheapest throughput win |
| batch.size | 64 to 256 KB |
| compression.type | lz4 or zstd |
| fetch.min.bytes | raise it to trade latency for fewer round trips |
| partitions | sized from consumer throughput, with headroom |
| Setting | Value |
|---|---|
| max.poll.records | lower it until a batch finishes well inside the interval |
| max.poll.interval.ms | raise only after checking why processing is slow |
| group.instance.id | set it, for static membership across restarts |
| You want | You configure |
|---|---|
| Ordering per entity | Key by that entity, and never grow partitions in place. |
| Ordering globally | One partition, therefore one consumer. Usually the wrong requirement. |
| No loss on broker failure | acks=all with min.insync.replicas=2 and replication factor 3. |
| No duplicate side effects | Idempotent consumer keyed on a record-carried identifier. |
| Atomic Kafka to Kafka | Transactions with read_committed on the consumer. |
| Per-record acknowledgement | Share group, giving up per-key ordering. |
| Current state per key, forever | Compacted topic keyed by entity identifier. |
Knowing Kafka and interviewing well on Kafka are different skills. This cluster is the second one.
| Question | What Kafka is doing in your answer | The key |
|---|---|---|
| Design a payment system | Durable event log between services, with the outbox pattern so the database write and the published event cannot diverge. | payment id |
| Design a news feed | Fan-out writes as events, consumed by a service that materializes each follower's feed. Replayable when the ranking changes. | author id |
| Design a ride hailing system | Driver location updates as a high volume stream, keyed so one driver's positions stay ordered. | driver id |
| Design a chat system | Persistence and cross-service fan-out of messages, keyed per conversation for ordering. The last hop to the device is a socket, not Kafka. | conversation id |
| Design a metrics or logging pipeline | The buffer that absorbs bursts and feeds several sinks: a warehouse, a search index, and alerting, each as its own group. | null, for spread |
| Design a web crawler | Frontier of URLs to fetch as a queue, which is the case for a share group. Deduplicate before publishing, not after. | domain, to rate limit per host |
| Design an order or ticket system | State changes per order as an ordered log, with a compacted topic holding current state per order. | order id |
| Design a notification service | Decoupling the trigger from delivery, with retry tiers and a dead letter topic for addresses that keep failing. | user id |
| Design a data warehouse ingest | Change data capture from operational databases into compacted topics, then a sink connector into the warehouse. | primary key |
| Design a fraud or anomaly detector | Stream processing with windowed aggregation over the event stream, keyed by the entity being scored. | account id |
| Situation | Better fit |
|---|---|
| Request and response between two services | A synchronous call. A log is not a substitute for an interface that returns a value. |
| Per-message delay or scheduling | A scheduler or a time-ordered structure. Kafka has no per-record timer. |
| Priority ordering | A broker with priority queues, or separate topics drained in order. |
| Low volume task queue in a small system | A database-backed queue or a simpler broker. Kafka's operational cost is real. |
| Point to point routing with complex rules | A message broker with exchanges and bindings. |
| Queries over the data | A database. A log is read sequentially, not searched. |