Cruise Control is the best open-source tool for balancing your Apache Kafka cluster.

It’s used in big companies like Slack, Wise, and NewRelic... 🤩

But how does it work?

To begin with - a Kafka cluster can be imbalanced on many dimensions. This means that it’s impossible to…

You can have extremely high availability or you can have very low resource usage.

You can't have both.

Replication Factor 101

It is a key configuration for availability, durability, and performance. It’s basically a tradeoff between resource usage and availability…

What do you define as a balanced Kafka cluster?

This is a problem I’ve spent 3+ years working peripherally on.

I still find it pretty hard to define.

Some ideas:…

Our paper won the VLDB award for Best Industry Paper!

Kora: A Cloud-Native Event Streaming Platform For Kafka

Not out yet, but trust me when I say it is 🔥🔥🔥

Kudos and thanks to everyone who worked on both the paper and the platform that lead to it!

vldb.org/2023/?conferen…

The most important Kafka concepts in 2 minutes.

a new (spicy) edition of 2 minute streaming is sitting patiently in your inbox :)

• scalability
• simpler deployment
• lower cost
• easier maintenance
• single security model
• more availability in cases where it most matters

All this is coming to Apache Kafka.

Can you guess the feature?

🚨 3AM: wake up by page.
😳 4AM: Realize customer's deployment is deleted.
😥 5AM: Scramble & page teams to find remediation.
😵‍💫 11PM: Contact affected customers.
💀 14 Days Later: Incident fully mitigated.

This should scare you.

Which lesson would have prevented this?

Edu Mínguez | @[email protected] Stanislav Kozlovski Omar lopez viggy28.dev/talks/ Each talk goes in-depth on some of the areas summarized here.