My primary use case for Apache Kafka is replacing ETL and doing data transformations.
Apache Kafka provides scalable, high-throughput, real-time data processing. Appreciated for its open-source nature and integration capabilities, Kafka supports distributed messaging and high-volume handling with essential features like message retention, replication, and partitioning.



| Author info | Rating | Review Summary |
|---|---|---|
| CEO - Founder / Principal Data Scientist / Principal AI Architect at Kanayma LLC | 3.5 | I primarily use Apache Kafka for replacing ETL and data transformations. The classification feature is valuable, but managing Kafka is difficult due to an outdated synchronization component. Despite alternatives like Databricks, we chose Kafka for its cost-effectiveness with high activity. |

My primary use case for Apache Kafka is replacing ETL and doing data transformations.
Kafka allows you to handle huge amounts of data and classify it into different categories. If you have huge amounts of data, Kafka is a very good solution for data classification. When you need to route it in different directions, you have to take a look at the messages that you get, interfile them, and then send them to the correct place. Kafka is a good product to use in the backend.
The feature I find most valuable is the classification feature. Kafka enables you to tag content with a category.
Kafka contains two components. The component that does the synchronization between the rest of the components, that's an older version of the software and it causes all kinds of configuration problems. The Confluent, which is the company that sells a commercial version of Kafka is getting away from that component precisely because of that. Kafka is a nightmare to administer.
In the next release, I would like to see that one troublesome component that causes configuration issues removed.
I have been using Apache Kafka for a couple of years.
The stability of this solution depends on whether it is properly configured. Having said that, Kafka is incredibly complex to configure, set up, administer, and maintain.
My opinion is that Apache Kafka is a scalable solution. In our organization, there are hundreds of thousands of users using Kafka.
The initial setup was extremely complex. In our case, it took a team of 12 two months to deploy.
These systems were installed by somebody else, not me.
I would advise others to schedule a month or two to just set it up and have it up and running.
There are other options. For example, Databricks is a Kafka alternative. We decided to go with Kafka because one of our clients already chose Kafka.
While evaluating, we found out Databricks is more expensive, for the level of activity that Kafka handles (in this case, millions of requests per day). Databricks could do it, but it would be overly expensive.
I would rate Apache Kafka's pricing a seven out of ten, with one being cheap and 10 being very expensive.
Since it has become so popular, large enterprises especially want to do it. For smaller enterprises, Kafka would probably be too expensive because they would have to hire people to maintain it.
I would rate the Apache Kafka solution a seven out of ten.
| Product | Mindshare (%) |
|---|---|
| Apache Kafka | 3.5% |
| Databricks | 7.4% |
| Apache Flink | 6.7% |
| Other | 82.4% |
| Company Size | Count |
|---|---|
| Small Business | 27 |
| Midsize Enterprise | 18 |
| Large Enterprise | 35 |
| Company Size | Count |
|---|---|
| Small Business | 282 |
| Midsize Enterprise | 85 |
| Large Enterprise | 274 |
Apache Kafka is a powerful tool for managing efficient data streams and high volumes of asynchronous messages. Its ease of setup and robust integration options make it popular among industries requiring real-time data streaming and processing. Key features such as message retention and consumer groups cater to demanding applications, while fault-tolerant design ensures reliability. Despite its advantages, Kafka can improve in areas like duplicate management, documentation, and intuitive interfaces. Challenges in configuration and monitoring tools suggest areas for enhancement, alongside reducing complexity and resource dependency.
What are the key features of Apache Kafka?Industry applications for Apache Kafka include real-time data streaming for IoT, big data management, and analytics. In finance, it supports fraud detection and transaction monitoring. Healthcare uses Kafka for patient data handling and logistics leverage its data distribution capabilities to optimize operations. Its ability to manage large-scale asynchronous communication makes it vital across sectors demanding high data throughput and reliability.
Uber, Netflix, Activision, Spotify, Slack, Pinterest