Apache Kafka Reviews

We have a pricing engine where we process transactions published to Apache Kafka, and our consumer application computes the price. We also have notification use cases and state processing use cases. We use Apache Kafka for the normal publish-subscribe model, which is the core capability it offers. The great thing about Apache Kafka is that we can seamlessly scale the application. Based on demand, we can add nodes without shutting down the environment. We can add nodes and increase transactions. If there is a spike in transaction volume for a particular period, we can easily manage the overall infrastructure. It scales horizontally with ease.

High throughput is something we aimed for, and that is the reason why we introduced Apache Kafka. Earlier, we had sequential API calls which were not scaling based on the data. Once we introduced this asynchronous architecture using Apache Kafka, high throughput was observed. That was really helpful in the overall architecture.

Apache Kafka internally handles topic partitions. We design the topics and determine what kind of data storage is stored in those topics. The partition strategy is based on the kind of data model being published. We use all of those capabilities.

Apache Kafka is abundant with a lot of features which only an expert-level person will be able to manage.

Apache Kafka is abundant with features which only an expert-level person will be able to manage due to the high volume and high concurrent expectations. Apache Kafka groups could introduce themes or profiles of configuration to help manage this complexity without needing expertise.

The management process should evolve to be more automated, more understandable, and more user-friendly. For example, with AI modeling for configuration, the management would be more error-free. If templates of definitions could be provided, recommendations could be managed more efficiently.

I have been working with Apache Kafka for almost six years.

For stability, I would rate it a nine.

We can independently scale Apache Kafka. It is fault-tolerant, and most importantly, when multiple vendor applications are defined as part of an overall process, we can publish the data and move on. Otherwise, if it is a seamless API call, all systems must be up and running.

I would rate Apache Kafka an eight for scalability and the ability to scale and expand.

For Apache technical support, I would rate it an eight.

MQ was one solution we used earlier. RabbitMQ is something we used in one place. Message queues were the earlier implementation prior to Apache Kafka.

The initial setup is simple. However, once you set it up, managing it with this kind of volume, different data patterns, and volume expectations is where the complexity comes. The initial setup is simple, but the ongoing management becomes challenging.

From the AWS perspective, the price is on the higher side. However, if you go for Apache Kafka, it is low. From a price perspective, if you are asking about Apache Kafka, I would rate it a nine.

MQ was one solution we used earlier. RabbitMQ is something we used in one place. Message queues were the earlier implementation prior to Apache Kafka.

Apache Kafka Connect is something we use. Within the whole architecture, we look at Apache Kafka Connect and the consumer components. Internally, there are a few rebalancing strategies which must be defined as part of the configuration. Similarly, there are many configuration policies which must be fine-tuned. If I remember correctly, retry, retry interval, and the kind of latency definitions will finally link the overall architecture. We first understand the solution, what kind of payload comes in, and how to do the partitioning. Then that is the way we define the system using Apache Kafka Connect internally. I would give this review an overall rating of eight.

Apache Kafka is usually used for stream data, transactions, high levels of transactions, and orchestration in stream mode data.

Apache Kafka is particularly valuable for stream data processing, handling transactions, managing high levels of transactions, and orchestrating stream mode data.

The interface of Apache Kafka could be significantly better. I started working with Apache Kafka from its early days, and I have seen many improvements. The back office functionality could be enhanced. Scaling up continues to be a challenge, though it is much easier now than it was in the beginning.

I started using the solution in 2018, before COVID.

The open community support for Apache Kafka can be more enriching than traditional support. The knowledge is more readily available, though I have never used technical support for Apache Kafka because it's an open community. With Microsoft, expectations are higher because we pay for a license and have a contract. I want to receive good technical support, which I only need once a month or every six months, and the experience has been unsatisfactory.

Neutral

While there may have been difficulties installing Apache Kafka initially, the process is now much quicker. The networking connection to the cloud can be challenging when integrating with another on-premises system or other cloud providers. In the beginning, the configuration was more complex, but now it is very straightforward.

I participate in Apache Kafka installation and work with the team while managing them. More senior team members can handle installations independently, and the process has become easier for junior colleagues as well.

The number of people involved in deployment varies by project, and sometimes it can be handled by just one person.

We started on-premises, and now we use cloud infrastructure as a service (IaaS). We don't use Apache Kafka directly as Platform as a Service (PaaS), but rather install it ourselves on cloud infrastructure.

The cloud provider varies between AWS and Azure, though we primarily use Azure, depending on what the customer decides for their IaaS requirements.

On a scale of 1-10, I rate Apache Kafka a 9 out of 10.

I primarily use Apache Kafka when I need real-time data from sources to a consumption layer. Streaming data in Apache Kafka involves consuming data from one source to join two disjoint systems. The producer produces data into a topic and the consumer consumes that data in real time. I can then perform real-time analytics or whatever I need.

The replication factor is valuable, and I have experience with this feature. Splitting the topic data into multiple partitions primarily helps with my system scalability. Apache Kafka acts as a broker and supports my data integration process by sitting in between systems. There are other features available as well, such as Kafka Connect or APIs, which allow me to leverage the full capabilities of Apache Kafka for whatever my use case is.

The long-term data storage feature in Apache Kafka depends on the setting, but I believe the maximum duration is seven days.

My use of Apache Kafka depends on the project requirements. I have not worked on it continuously, but I have been using it for the last eight to ten years.

I believe Apache Kafka is a mature product, and I have not encountered any problems because I have not worked in a heavy-loaded system. In the system where I have worked, I did not see any issues. Apache Kafka is a mature product and can handle a massive amount of data in real time for data consumption.

I need to scale up my server for handling high volume with Apache Kafka because the amount of data it can handle depends on server capacity, and servers have their limits. If the solution is not highly available, then there will be challenges. I need to enable my solution with high availability and scalability. Scalability may cause issues in the product if my nodes are full with multiple sources and delivery is slowing down. In such cases, I need to increase my nodes.

I have never spoken or communicated with technical support from Apache.

Apache Kafka is easy to install.

Confluent Kafka is a licensed option, whereas Apache Kafka is free. There is another variant called Confluent Kafka, and recently another solution came to market that I am not aware of, but Apache Kafka is more mature from a use case perspective. That product has solved all the problems which Apache Kafka had earlier. I forgot the new product name, but there is something there. Apache Kafka is indeed mature.

There is something called Kafka Connect that I think will help with integration needs. I can perform both streaming and batch operations with Apache Kafka. I would rate this product a ten out of ten overall.

I worked with Apache Kafka for customers in the financial industry and OTT platforms. They use Kafka particularly for data streaming. Companies offering movie and entertainment as a service, similar to Netflix, use Kafka.

Apache Kafka offers unique data streaming. It allows the use of data in motion, allowing data to propagate from one source to another while it is in motion. This is valuable when data is not simply residing in a database.

In the data sharing space, the performance of Apache Kafka could be improved. The performance angle is critical, and while it works in milliseconds, the goal is to move towards microseconds.

I started working with Kafka about five years ago while at a financial company.

Apache Kafka is stable. Even though enterprises often use the open-source version, there are minimal issues after configuration.

Apache Kafka is very scalable. I would rate its scalability as nine out of ten. Customers have not faced issues with user growth or data streaming needs.

The Apache community provides support for the open-source version. Despite being open-source, extensive documentation is available to resolve issues.

Positive

The initial setup of Apache Kafka is straightforward, around an eight on a scale from one to ten. The deployment process involves configuring the publisher, subscriber, and other parameters. SaaS can be deployed from the cloud in a couple of hours.

Since I work with the open-source version of Kafka, solutions are managed internally with the Apache documentation.

The open-source version of Apache Kafka results in minimal costs, mainly linked to accessing documentation and limited support. Enterprises usually opt for the more cost-effective open-source edition.

For critical business components, it is advisable to use Confluent-managed services for Kafka. However, for non-critical functions, the open-source version is sufficient. 

Overall, I rate Apache Kafka as nine out of ten for its scalability and stability.

We primarily use Kafka for EventStream. We publish messages into the Kafka queues, and then at the other end, we consume those messages to perform tasks.

We have seen a return on investment with respect to improvements in productivity and ease of use, although these benefits are not easily quantified in dollar terms.

Kafka is scalable to any degree we want, and it has several connectors available for integration in multiple languages, making it easier for integration. Additionally, it is 100% stable for our use cases.

The UI used to access Kafka topics can be further improved. It's not very appealing, and there's potential for enhancement in that area.

I have been using Kafka for at least five years in some form or another.

Kafka is 100% stable for my use cases.

Kafka is scalable to a degree that suits our needs.

We never had the opportunity to talk to Apache support as we generally obtained whatever we needed from Kafka.

Positive

We primarily use Kafka and have not worked with other similar solutions.

Our DevOps team takes care of maintaining Kafka, and generally, only one person is needed.

We have experienced ROI in terms of productivity and ease of use, even though it's not quantifiable in dollar terms.

We use managed Kafka, so there is a cost involved. Although Kafka is free as open source, the managed service cost is on a pay-as-you-go basis, which is decent.

I'd rate the solution nine out of ten. 

My main use cases for Apache Kafka are for a big project where Apache Kafka is one of the main components for sending messages and receiving them.

I appreciate that Apache Kafka is fast and secure thanks to implementing it with AWS, allowing me to secure it on a high level. It's fast with a good connection, and the different types of connections are a good thing for me, helping our team that uses it, which is very helpful.

The impact of Apache Kafka's scalability features on my organization and data processing capabilities depends on how many messages each company wants to receive. With a high throughput, it helps to have more brokers and partitions. If you are a company that doesn't need that many messages, I won't say it will help you a lot, but on the other hand, it can change significantly.

I don't actually think about anything they could improve about Apache Kafka, as our use cases using it are more or less on the basic level, so I didn't think about any kind of improvement.

For personal preferences, since we use Managed Kafka in AWS, I would appreciate having some kind of UI integrated into Apache Kafka for connecting to it because using code to connect it is basic, but we can use a UI.

I have been using Apache Kafka for about six months.

I use Apache Kafka topic partitioning feature for my system stability.

This feature of Apache Kafka has helped enhance our system stability when handling high volume data because we have thousands of messages in a small amount of time, so partitioning helps us distribute all the messages that we receive between all partitions, which helps us to be stable.

I have seen some ROI from Apache Kafka, although I can't recall specifics.

I am saying that Apache Kafka has better security than other options, even though I don't know about them because we didn't explore them. We simply knew that Apache Kafka is the base where you should use it, so we went with that.

At this point, I don't have any specific examples to share. I don't actually remember if I have used Apache Kafka Connect for integrating various data sources and sinks within my organization. I currently don't have any examples of how it has benefited my organization.

On a scale of one to ten, I would rate Apache Kafka an eight.

We use Kafka for a stage event-driven process from a process perspective. Our platform is an ID platform, so after registration data is received, it has to be stored from various registration locations. The process includes stages like quality checking, consistency, format, biometric data checking, and so on. We are using both Kafka and ActiveMQ for almost three years now.

Using Kafka has saved us costs compared to proprietary solutions. It has allowed for significant customizations internally, which would have been much costlier with other solutions. We have saved a lot by employing Kafka and only needing basic salary agreements for our experts.

The real-time data processing capability of Apache Kafka is a significant pro. We also find Kafka to be relatively stable under large data volumes and its performance to be consistent. The convenience in setting up after major problems like data center blackouts is a notable feature.

Kafka has some limitations in terms of queue management. Specifically, it lacks the capability to handle larger queues for external system interactions. It would be beneficial if Kafka included more robust, high-capacity queue management features for integration with external systems.

We have been using Kafka for almost three years now.

Kafka is stable even under high data sizes. Even if the data size is at the highest possible capability, Kafka executes relatively stable compared to other solutions we have used. I would rate its stability as a nine out of ten.

Scalability is one of the strong suits of Kafka. Initially, we handled up to five thousand registrations of data items per day. Now we manage up to forty thousand. We have scaled Kafka on the same Kubernetes environment, and it has been very convenient without critical problems.

We have not directly communicated with technical support for Kafka. Instead, we have relied on the open community for advice and employed senior experts to work on the platform.

Positive

We did not use any other solutions before Apache Kafka for these use cases.

The initial setup for Kafka was a bit difficult and required additional configurations. It took us about one to two hours to deploy the whole platform, which includes other components like PostgreSQL database, antivirus, and directory management solutions. Kafka itself would take only a fraction of this time.

Our DevOps team currently consists of four people working directly on this platform. For Kafka, we also involve one person from our infrastructure team, making it a total of five people.

We have definitely seen a return on investment in terms of cost savings. Using Kafka as an open-source solution has saved us a lot compared to proprietary solutions. We have been able to make significant customizations internally at a lower cost.

I would rate the overall cost of using Kafka as a three out of ten, indicating that it is rather affordable, considering the benefits and savings it provides.

We did not evaluate other options or vendors before choosing Apache Kafka. We followed recommendations from other experts.

Kafka is one of the most convenient tools with a fair level of performance. Its stability is impressive, allowing us to recover from data center blackouts quickly. I highly recommend it for similar needs.

I'd rate the solution nine out of ten.

My company uses Apache Kafka to keep some intermediate data in the workflow.

Apache Kafka's most valuable features include clustering and sharding. Though we have not started using Apache Kafka Streams in our company, I have heard that it is one of the good features of the product we plan to use. The good features let you replay and reconsume messages in Apache Kafka, allowing you to have multiple consumer groups. The rebalancing feature of the product is also useful since if one consumer dies, then Apache Kafka does a rebalancing. With Apache Kafka, we use clustering, sharding, and partitioning features in our company.

In Apache Kafka, it is currently difficult to create a consumer. The implementation of Apache Kafka's features, like rebalancing, is possible only when you create a consumer, which is a very difficult task since it is overly complicated. To create a consumer in Apache Kafka, a person needs to have a very strong knowledge of the internal functioning of Apache. I feel that Kaka needs to provide a consumer so that its users don't spend time in the creation of consumers. In general, Apache Kafka must provide users with a more user-friendly UI.

I have been using Apache Kafka for five years. I am just a customer of the solution.

It is a pretty stable solution. Stability-wise, I rate the solution a nine out of ten.

Scalability-wise, I rate the solution an eight out of ten.

The users of the solution are not directly involved with it. Those who use Apache Kafka in our company use it to push orders on the frontend, and then the frontend calls for some microservice, after which the microservice pushes data to Apache Kafka. Around 10,000 to 15,000 people in my organization follow the aforementioned procedure.

I won't be able to comment on the technical support team of Apache Kafka since some other team members in my company were involved with them. In general, my company is satisfied with the technical support team of Apache Kafka. I rate Apache Kafka's technical support an eight out of ten.

Positive

My company started off with Apache Kafka, so they did not use any other solutions previously.

If you are using the latest version of Apache Kafka in which the use of Zookeeper is not required, then it uses the KRaft protocol, which is built into Kafka broker. Since the use of Zookeeper is no longer required, I think the setup phase of dissolution is better than its previous versions. I rate the initial setup of the product somewhere between seven to eight out of ten.

Apache Kafka's initial setup is very straightforward.

The solution is deployed on an on-premises model.

Apache Kafka was deployed in our company within a couple of days.

Three people were involved in the deployment process of Apache Kafka.

I rate Apache Kafka's pricing a five on a scale of one to ten, where one is cheap and ten is expensive. There are no additional costs apart from the licensing fees for Apache Kafka.

During my company's evaluation process for other options from Apache Kafka, we came across RabbitMQ. My company chose Apache Kafka over RabbitMQ since it was one of the market's more popular tools. With Apache Kafka, more materials, support, online technical groups, and forums were available for consumers.

Apache Kafka as a broker tool is a very stable and good product. When you need to create a consumer in any programming language, including Java, Golang or any other programming language, the team involved in the process of the creation of a consumer should have very strong knowledge and expertise in the use of Apache Kafka since it is not at all easy to create a consumer for the product. A highly qualified person with a good amount of experience should also know the internals of the solution, which may not seem too straightforward. Anyone cannot use Apache Kafka easily without proper knowledge or experience. When you use Apache Kafka in your actual application, you need to create some producers and some consumers. To create a consumer, you need to have a very strong understanding of the solution since it is not a process that anyone can manage easily. A company needs to have a very strong team with good technical knowledge to be able to use the product.

I rate the overall solution a nine out of ten.

Currently, I work for an observability company. We stream customer data into our cloud, digest the information, enrich it, transform it, save it, and use on-the-fly aggregation with Kafka. Previously, I worked for a security company doing normal detection using streaming with Kafka. 

I also worked for a company with a data platform based on Kafka, where we ingested clickstream data and enriched it before streaming.

The most valuable feature of Kafka is the Kafka Streams client. Unlike other systems like Flink or Spark Streaming, you don't need a separate engine to do real-time transformations and analytics. The amount of data that can be streamed into the platform and the scalability are also significant benefits.

Kafka requires fine-tuning to find the best architecture, number of nodes, and partitions for your use case. It’s a trial-and-error process with no one-size-fits-all solution. Issues may arise until it’s appropriately tuned. 

While it can scale out efficiently, scaling down is more challenging, making deleting data or reducing activity harder.

I have been working with the Kafka product for more than ten years.

Since Kafka is written in Java, it's not as stable as it should be on the JVM. The stability depends on fine-tuning the system to find the best architecture for your use case. However, the replication factor helps avoid data loss despite the stability issues.

Kafka's architecture allows for scalability by adding nodes and partitions to topics. However, it's not as effective in scaling in, making reducing activity and deleting data harder. 

Scalability can be managed both manually and automatically to meet demands.

I used to work with Spark Streaming and Flink, however, not in the past year.

If you are unfamiliar with Kafka, setting up the cluster can be quite difficult. You need to understand the architecture and components and compute the data volume upfront. For experienced individuals, the setup is less difficult yet still requires preparation.

From a time-saving perspective, onboarding new customers is straightforward, requiring them merely to stream their data into our platform.

We use Apache Kafka, which is open-source, so we don't have fees. I can't comment on ownership costs as I am not responsible for that domain.

Apart from Kafka, I have experience working with Spark Streaming and Flink.

When implementing Kafka, it's important to plan the cluster size upfront to ensure easy scalability. Adding or removing nodes can disrupt the clusters, so proper sizing and planning are key. 

I would rate Kafka as a solution as a nine.