Apache Kafka is a highly reliable and distributed event-streaming platform. One of its critical features that ensures fault tolerance and high availability is rack awareness. In this blog, we'll explore what rack awareness is, why it's important, and how to implement it in Kafka.

What is Rack Awareness in Kafka?

Rack awareness refers to the capability of Kafka to distribute data replicas across different racks, zones, or data centers in a cluster. A rack can represent:

  • A physical rack in a data center.
  • A logical grouping like availability zones in cloud environments.

By being aware of these groupings, Kafka ensures:

  1. High Availability: Data remains accessible even if a rack or zone goes down.
  2. Fault Tolerance: Replicas are distributed across multiple racks, reducing the risk of data loss.
  3. Load Balancing: Traffic is distributed evenly across racks, improving performance.

How Kafka Uses Rack Awareness

Kafka uses rack awareness during replica assignment. When creating a topic, Kafka assigns replicas of partitions to brokers in such a way that no two replicas of the same partition are in the same rack. This ensures:

  • Even distribution of replicas.
  • Minimised data loss in case of rack failures.

Why is Rack Awareness Important?

Resilience to Rack Failures:

  • Without rack awareness, all replicas of a partition might reside in the same rack. If the rack goes down, the partition becomes unavailable.
  • Rack-aware replication ensures that at least one replica is always available in another rack.

Data Durability:

  • In a rack-aware setup, data loss is minimized since replicas are geographically or logically distributed.

Optimised Disaster Recovery:

  • During disaster recovery, Kafka can recover data more efficiently if replicas are spread across racks.

How to Implement Rack Awareness in Kafka

To enable rack awareness in Kafka, follow these steps:

Step 1: Configure Rack Information for Brokers

Each Kafka broker must be configured with its rack ID. Add the broker.rack property to the server.properties file for each broker:

broker.rack=us-east-1a
  • broker.rack represents the rack or availability zone where the broker is located.

Example values:

  • For physical data centers: rack1, rack2, etc.
  • For cloud zones: us-east-1a, us-east-1b, etc.

Step 2: Update the Kafka Cluster

Restart each Kafka broker after updating its broker.rack property. This makes the broker aware of its rack.

Step 3: Enable Rack-Aware Replica Assignment

Kafka uses rack-aware replica assignment by default. This feature ensures that replicas are spread across different racks.

When creating a topic, specify the desired replication factor:

bin/kafka-topics.sh --create \
    --topic test-topic \
    --partitions 3 \
    --replication-factor 3 \
    --bootstrap-server <broker-address>

Step 4: Verify Rack-Aware Placement

To verify that Kafka has assigned replicas across racks:

  1. Use the kafka-topics.sh tool to describe the topic:
bin/kafka-topics.sh --describe \
    --topic test-topic \
    --bootstrap-server <broker-address>

2. Check the rack placement of replicas in the output. You should see replicas distributed across different brokers and racks.

Example Scenario: Rack Awareness in Action

Cluster Setup

  • 3 racks: us-east-1a, us-east-1b, us-east-1c.
  • 6 brokers:
  • Brokers 1 and 2 in us-east-1a.
  • Brokers 3 and 4 in us-east-1b.
  • Brokers 5 and 6 in us-east-1c.

Topic Configuration

  • Topic: orders
  • Partitions: 3
  • Replication Factor: 3

Replica Assignment

Kafka assigns replicas for each partition as follows:

Partition 0:

  • Leader: Broker 1 (us-east-1a)
  • Replica: Broker 3 (us-east-1b)
  • Replica: Broker 5 (us-east-1c)

Partition 1:

  • Leader: Broker 3 (us-east-1b)
  • Replica: Broker 2 (us-east-1a)
  • Replica: Broker 6 (us-east-1c)

Partition 2:

  • Leader: Broker 5 (us-east-1c)
  • Replica: Broker 4 (us-east-1b)
  • Replica: Broker 1 (us-east-1a)

Benefits Observed

  1. If us-east-1a fails, data is still available in us-east-1b and us-east-1c.
  2. No single rack failure makes the topic unavailable.

Advanced Configurations

1. Custom Partition Assignment

For complete control, you can use Kafka's AdminClient API or tools like kafka-reassign-partitions.sh to manually assign partitions to brokers across racks.

2. Monitoring Rack Awareness

Use monitoring tools like Prometheus, Grafana, or Kafka Manager to:

  • Monitor rack-based distribution of data.
  • Ensure high availability.

Conclusion

Rack awareness in Kafka is a vital feature for achieving high availability, fault tolerance, and resilience in distributed environments. By understanding and implementing rack-aware configurations:

  • You minimize the impact of failures.
  • You ensure optimal resource utilization.

With the steps and examples provided, you can confidently configure rack awareness in your Kafka cluster to make it more robust and reliable.