Apache Kafka Reviews

I use Apache Kafka for inter-service communication between components that produce events, and Apache Kafka processes those events asynchronously. Other consumers then listen to those events and consume the particular event.

For the order service, I interact with the payment service. Once an order is placed, an event is triggered and sent to Apache Kafka. Apache Kafka processes that event, and the payment service consumes it.

I use Apache Kafka for inter-service communication, which creates loose coupling communications, and multiple tasks are executed simultaneously without any latency. This supports multithreaded processing of tasks and is part of event-driven architectures used in microservices communication.

We deploy Apache Kafka in our organization's private cloud.

Apache Kafka is a very scalable platform and is extensively used in our applications, particularly in e-commerce applications for asynchronous programming. It provides high throughput and very scalable horizontal scaling. It auto-generates containers, provides fault tolerance, and enables event streaming. Apache Kafka is highly scalable because it provides horizontal scaling by adding brokers to the cluster and partitioning topics for parallel processing. We use consumer groups for parallel processing of events. This distributed architecture is highly scalable and provides loose coupling between services while handling millions of messages per second.

The best feature is that we can monitor events and perform log aggregations and real-time analytics. Multiple consumers can consume the particular event produced by the order service or other services. The best features include high throughput with low latency and support for horizontal scaling. Message partitioning and message retention are also valuable, as Apache Kafka has persistence support that stores events, and once a service is back up, it can consume the event again from Apache Kafka.

Low latency is important, as multiple tasks are executed simultaneously without any latency. It provides high throughput and prevents duplicate transactions. Apache Kafka provides the outbox pattern, which stores the history of events that are missed by consumers. It then schedules background jobs to process the remaining events again.

Apache Kafka is used in distributed architecture and helps with fault tolerance, replication, and message retention. We have consumer groups for parallel processing of events. The ability to replay messages is an advantage. These advantages include replaying messages, dual storage, high scalability, and the ability to handle billions of messages per second.

Apache Kafka is impacting our organization very positively. We deliver client requirements on time, and we are more profitable now. It is very useful in our organization and makes applications more loosely coupled.

I would say it saves time. If there is time saved, we are definitely saving money. We deliver code quality to the client at the appropriate time, and the client is happy with that for the high throughput. The system can handle multiple requests per second.

In our organization, most employees use Apache Kafka technology to save time. It provides high throughput, saves our time, and makes applications more scalable. This is very useful in our organization and gives us more profit since we are using this software to design architecture.

We can improve the high throughput because there are some limitations. We could add something to improve Apache Kafka. Based on my daily usage and analysis, there is a complex setup and management, which is one area requiring improvement. Apache Kafka does not have built-in message delay or scheduling capabilities. Apache Kafka cannot natively schedule messages. Limited message prioritization is also an area requiring improvement. Ordering is limited to a single partition.

Large messages affect performance. Apache Kafka is optimized for many small to medium-sized messages, but large payloads increase network usage, increase disk usage, and slow down producers and consumers.

I rate Apache Kafka eight out of ten instead of ten out of ten because operating an Apache Kafka cluster requires expertise in partitioning, application monitoring, and capacity planning. Self-managed deployment can become complex as the cluster grows. Pro-managed Apache Kafka services significantly reduce the operational overhead.

I have been working in my current field for five years.

Apache Kafka is more stable and is very good technology for asynchronous programming.

The scalability of Apache Kafka, based on my knowledge and experience, is very good and highly scalable. Scalability is one of Apache Kafka's biggest strengths. Apache Kafka is highly scalable and supports horizontal scaling by allowing you to add more brokers to the cluster and increase the number of partitions for a topic. As data volume and traffic grow, producers and consumers can continue to operate efficiently by distributing the workload across multiple brokers and consumer instances. This enables Apache Kafka to handle millions of messages per second with high availability and fault tolerance.

I would rate customer support nine out of ten, and I received a response within an acceptable timeframe, which was very appreciating.

I have not used a different solution previously. From the time I joined the company, I have been using Apache Kafka in our application.

Low latency is important, as multiple tasks are executed simultaneously without any latency. It provides high throughput and prevents duplicate transactions. Apache Kafka provides the outbox pattern, which stores the history of events that are missed by consumers, then schedules background jobs to process the remaining events again.

We purchased Apache Kafka through the AWS Marketplace.

We delivered client requirements on time, and we are more profitable now. Apache Kafka is very useful in our organization and makes applications more loosely coupled.

Based on my understanding for the setup and cost to set up Apache Kafka, I was not responsible for the pricing or licensing decisions. Apache Kafka itself is open source and free to use. Infrastructure costs depend on whether it is self-managed or a managed cloud service. Managed Apache Kafka solutions typically charge based on brokers, storage, and data transfer, while self-managed deployment mainly incurs infrastructure and operational costs.

I attempted to evaluate other options from Apache Kafka, but I was not directly involved in the technology selection process. By the time I joined the project, Apache Kafka had already been chosen as a messaging platform. However, I am familiar with the common alternatives and why Apache Kafka is often preferred for event-driven architectures.

Regarding governance and security for Apache Kafka, it is good. Although Apache Kafka itself is not an artificial intelligence platform, it powers artificial intelligence systems by streaming data such as real-time recommendation engines, fraud detection, chatbots, machine learning feature pipelines, and predictive analytics.

I would say it saves time. If there is time saved, we are definitely saving money. We deliver code quality to the client at the appropriate time, and the client is happy with that for the high throughput. The system can handle multiple requests per second.

My advice to others is that if you are trying to use Apache Kafka, I would recommend it. Apache Kafka is an excellent choice for high throughput, scalable, and fault-tolerant event streaming. My advice would be to first understand its core concepts, design topics and partitions carefully, monitor consumer logs and broker health, and use managed Apache Kafka services if you do not have a dedicated operations team. The key point I would like to add is to learn Apache Kafka fundamentals such as topics, partitions, brokers, consumer groups, and offsets. Choose Apache Kafka only when you need asynchronous programming, asynchronous communication, event-driven communication, or real-time data streaming. I would rate Apache Kafka eight out of ten overall.

We have a pricing engine where we process transactions published to Apache Kafka, and our consumer application computes the price. We also have notification use cases and state processing use cases. We use Apache Kafka for the normal publish-subscribe model, which is the core capability it offers. The great thing about Apache Kafka is that we can seamlessly scale the application. Based on demand, we can add nodes without shutting down the environment. We can add nodes and increase transactions. If there is a spike in transaction volume for a particular period, we can easily manage the overall infrastructure. It scales horizontally with ease.

High throughput is something we aimed for, and that is the reason why we introduced Apache Kafka. Earlier, we had sequential API calls which were not scaling based on the data. Once we introduced this asynchronous architecture using Apache Kafka, high throughput was observed. That was really helpful in the overall architecture.

Apache Kafka internally handles topic partitions. We design the topics and determine what kind of data storage is stored in those topics. The partition strategy is based on the kind of data model being published. We use all of those capabilities.

Apache Kafka is abundant with a lot of features which only an expert-level person will be able to manage.

Apache Kafka is abundant with features which only an expert-level person will be able to manage due to the high volume and high concurrent expectations. Apache Kafka groups could introduce themes or profiles of configuration to help manage this complexity without needing expertise.

The management process should evolve to be more automated, more understandable, and more user-friendly. For example, with AI modeling for configuration, the management would be more error-free. If templates of definitions could be provided, recommendations could be managed more efficiently.

I have been working with Apache Kafka for almost six years.

For stability, I would rate it a nine.

We can independently scale Apache Kafka. It is fault-tolerant, and most importantly, when multiple vendor applications are defined as part of an overall process, we can publish the data and move on. Otherwise, if it is a seamless API call, all systems must be up and running.

I would rate Apache Kafka an eight for scalability and the ability to scale and expand.

For Apache technical support, I would rate it an eight.

MQ was one solution we used earlier. RabbitMQ is something we used in one place. Message queues were the earlier implementation prior to Apache Kafka.

The initial setup is simple. However, once you set it up, managing it with this kind of volume, different data patterns, and volume expectations is where the complexity comes. The initial setup is simple, but the ongoing management becomes challenging.

From the AWS perspective, the price is on the higher side. However, if you go for Apache Kafka, it is low. From a price perspective, if you are asking about Apache Kafka, I would rate it a nine.

MQ was one solution we used earlier. RabbitMQ is something we used in one place. Message queues were the earlier implementation prior to Apache Kafka.

Apache Kafka Connect is something we use. Within the whole architecture, we look at Apache Kafka Connect and the consumer components. Internally, there are a few rebalancing strategies which must be defined as part of the configuration. Similarly, there are many configuration policies which must be fine-tuned. If I remember correctly, retry, retry interval, and the kind of latency definitions will finally link the overall architecture. We first understand the solution, what kind of payload comes in, and how to do the partitioning. Then that is the way we define the system using Apache Kafka Connect internally. I would give this review an overall rating of eight.

Apache Kafka is usually used for stream data, transactions, high levels of transactions, and orchestration in stream mode data.

Apache Kafka is particularly valuable for stream data processing, handling transactions, managing high levels of transactions, and orchestrating stream mode data.

The interface of Apache Kafka could be significantly better. I started working with Apache Kafka from its early days, and I have seen many improvements. The back office functionality could be enhanced. Scaling up continues to be a challenge, though it is much easier now than it was in the beginning.

I started using the solution in 2018, before COVID.

The open community support for Apache Kafka can be more enriching than traditional support. The knowledge is more readily available, though I have never used technical support for Apache Kafka because it's an open community. With Microsoft, expectations are higher because we pay for a license and have a contract. I want to receive good technical support, which I only need once a month or every six months, and the experience has been unsatisfactory.

Neutral

While there may have been difficulties installing Apache Kafka initially, the process is now much quicker. The networking connection to the cloud can be challenging when integrating with another on-premises system or other cloud providers. In the beginning, the configuration was more complex, but now it is very straightforward.

I participate in Apache Kafka installation and work with the team while managing them. More senior team members can handle installations independently, and the process has become easier for junior colleagues as well.

The number of people involved in deployment varies by project, and sometimes it can be handled by just one person.

We started on-premises, and now we use cloud infrastructure as a service (IaaS). We don't use Apache Kafka directly as Platform as a Service (PaaS), but rather install it ourselves on cloud infrastructure.

The cloud provider varies between AWS and Azure, though we primarily use Azure, depending on what the customer decides for their IaaS requirements.

On a scale of 1-10, I rate Apache Kafka a 9 out of 10.

I primarily use Apache Kafka when I need real-time data from sources to a consumption layer. Streaming data in Apache Kafka involves consuming data from one source to join two disjoint systems. The producer produces data into a topic and the consumer consumes that data in real time. I can then perform real-time analytics or whatever I need.

The replication factor is valuable, and I have experience with this feature. Splitting the topic data into multiple partitions primarily helps with my system scalability. Apache Kafka acts as a broker and supports my data integration process by sitting in between systems. There are other features available as well, such as Kafka Connect or APIs, which allow me to leverage the full capabilities of Apache Kafka for whatever my use case is.

The long-term data storage feature in Apache Kafka depends on the setting, but I believe the maximum duration is seven days.

My use of Apache Kafka depends on the project requirements. I have not worked on it continuously, but I have been using it for the last eight to ten years.

I believe Apache Kafka is a mature product, and I have not encountered any problems because I have not worked in a heavy-loaded system. In the system where I have worked, I did not see any issues. Apache Kafka is a mature product and can handle a massive amount of data in real time for data consumption.

I need to scale up my server for handling high volume with Apache Kafka because the amount of data it can handle depends on server capacity, and servers have their limits. If the solution is not highly available, then there will be challenges. I need to enable my solution with high availability and scalability. Scalability may cause issues in the product if my nodes are full with multiple sources and delivery is slowing down. In such cases, I need to increase my nodes.

I have never spoken or communicated with technical support from Apache.

Apache Kafka is easy to install.

Confluent Kafka is a licensed option, whereas Apache Kafka is free. There is another variant called Confluent Kafka, and recently another solution came to market that I am not aware of, but Apache Kafka is more mature from a use case perspective. That product has solved all the problems which Apache Kafka had earlier. I forgot the new product name, but there is something there. Apache Kafka is indeed mature.

There is something called Kafka Connect that I think will help with integration needs. I can perform both streaming and batch operations with Apache Kafka. I would rate this product a ten out of ten overall.

I worked with Apache Kafka for customers in the financial industry and OTT platforms. They use Kafka particularly for data streaming. Companies offering movie and entertainment as a service, similar to Netflix, use Kafka.

Apache Kafka offers unique data streaming. It allows the use of data in motion, allowing data to propagate from one source to another while it is in motion. This is valuable when data is not simply residing in a database.

In the data sharing space, the performance of Apache Kafka could be improved. The performance angle is critical, and while it works in milliseconds, the goal is to move towards microseconds.

I started working with Kafka about five years ago while at a financial company.

Apache Kafka is stable. Even though enterprises often use the open-source version, there are minimal issues after configuration.

Apache Kafka is very scalable. I would rate its scalability as nine out of ten. Customers have not faced issues with user growth or data streaming needs.

The Apache community provides support for the open-source version. Despite being open-source, extensive documentation is available to resolve issues.

Positive

The initial setup of Apache Kafka is straightforward, around an eight on a scale from one to ten. The deployment process involves configuring the publisher, subscriber, and other parameters. SaaS can be deployed from the cloud in a couple of hours.

Since I work with the open-source version of Kafka, solutions are managed internally with the Apache documentation.

The open-source version of Apache Kafka results in minimal costs, mainly linked to accessing documentation and limited support. Enterprises usually opt for the more cost-effective open-source edition.

For critical business components, it is advisable to use Confluent-managed services for Kafka. However, for non-critical functions, the open-source version is sufficient. 

Overall, I rate Apache Kafka as nine out of ten for its scalability and stability.

We primarily use Kafka for EventStream. We publish messages into the Kafka queues, and then at the other end, we consume those messages to perform tasks.

We have seen a return on investment with respect to improvements in productivity and ease of use, although these benefits are not easily quantified in dollar terms.

Kafka is scalable to any degree we want, and it has several connectors available for integration in multiple languages, making it easier for integration. Additionally, it is 100% stable for our use cases.

The UI used to access Kafka topics can be further improved. It's not very appealing, and there's potential for enhancement in that area.

I have been using Kafka for at least five years in some form or another.

Kafka is 100% stable for my use cases.

Kafka is scalable to a degree that suits our needs.

We never had the opportunity to talk to Apache support as we generally obtained whatever we needed from Kafka.

Positive

We primarily use Kafka and have not worked with other similar solutions.

Our DevOps team takes care of maintaining Kafka, and generally, only one person is needed.

We have experienced ROI in terms of productivity and ease of use, even though it's not quantifiable in dollar terms.

We use managed Kafka, so there is a cost involved. Although Kafka is free as open source, the managed service cost is on a pay-as-you-go basis, which is decent.

I'd rate the solution nine out of ten. 

My main use cases for Apache Kafka are for a big project where Apache Kafka is one of the main components for sending messages and receiving them.

I appreciate that Apache Kafka is fast and secure thanks to implementing it with AWS, allowing me to secure it on a high level. It's fast with a good connection, and the different types of connections are a good thing for me, helping our team that uses it, which is very helpful.

The impact of Apache Kafka's scalability features on my organization and data processing capabilities depends on how many messages each company wants to receive. With a high throughput, it helps to have more brokers and partitions. If you are a company that doesn't need that many messages, I won't say it will help you a lot, but on the other hand, it can change significantly.

I don't actually think about anything they could improve about Apache Kafka, as our use cases using it are more or less on the basic level, so I didn't think about any kind of improvement.

For personal preferences, since we use Managed Kafka in AWS, I would appreciate having some kind of UI integrated into Apache Kafka for connecting to it because using code to connect it is basic, but we can use a UI.

I have been using Apache Kafka for about six months.

I use Apache Kafka topic partitioning feature for my system stability.

This feature of Apache Kafka has helped enhance our system stability when handling high volume data because we have thousands of messages in a small amount of time, so partitioning helps us distribute all the messages that we receive between all partitions, which helps us to be stable.

I have seen some ROI from Apache Kafka, although I can't recall specifics.

I am saying that Apache Kafka has better security than other options, even though I don't know about them because we didn't explore them. We simply knew that Apache Kafka is the base where you should use it, so we went with that.

At this point, I don't have any specific examples to share. I don't actually remember if I have used Apache Kafka Connect for integrating various data sources and sinks within my organization. I currently don't have any examples of how it has benefited my organization.

On a scale of one to ten, I would rate Apache Kafka an eight.

We use Kafka for a stage event-driven process from a process perspective. Our platform is an ID platform, so after registration data is received, it has to be stored from various registration locations. The process includes stages like quality checking, consistency, format, biometric data checking, and so on. We are using both Kafka and ActiveMQ for almost three years now.

Using Kafka has saved us costs compared to proprietary solutions. It has allowed for significant customizations internally, which would have been much costlier with other solutions. We have saved a lot by employing Kafka and only needing basic salary agreements for our experts.

The real-time data processing capability of Apache Kafka is a significant pro. We also find Kafka to be relatively stable under large data volumes and its performance to be consistent. The convenience in setting up after major problems like data center blackouts is a notable feature.

Kafka has some limitations in terms of queue management. Specifically, it lacks the capability to handle larger queues for external system interactions. It would be beneficial if Kafka included more robust, high-capacity queue management features for integration with external systems.

We have been using Kafka for almost three years now.

Kafka is stable even under high data sizes. Even if the data size is at the highest possible capability, Kafka executes relatively stable compared to other solutions we have used. I would rate its stability as a nine out of ten.

Scalability is one of the strong suits of Kafka. Initially, we handled up to five thousand registrations of data items per day. Now we manage up to forty thousand. We have scaled Kafka on the same Kubernetes environment, and it has been very convenient without critical problems.

We have not directly communicated with technical support for Kafka. Instead, we have relied on the open community for advice and employed senior experts to work on the platform.

Positive

We did not use any other solutions before Apache Kafka for these use cases.

The initial setup for Kafka was a bit difficult and required additional configurations. It took us about one to two hours to deploy the whole platform, which includes other components like PostgreSQL database, antivirus, and directory management solutions. Kafka itself would take only a fraction of this time.

Our DevOps team currently consists of four people working directly on this platform. For Kafka, we also involve one person from our infrastructure team, making it a total of five people.

We have definitely seen a return on investment in terms of cost savings. Using Kafka as an open-source solution has saved us a lot compared to proprietary solutions. We have been able to make significant customizations internally at a lower cost.

I would rate the overall cost of using Kafka as a three out of ten, indicating that it is rather affordable, considering the benefits and savings it provides.

We did not evaluate other options or vendors before choosing Apache Kafka. We followed recommendations from other experts.

Kafka is one of the most convenient tools with a fair level of performance. Its stability is impressive, allowing us to recover from data center blackouts quickly. I highly recommend it for similar needs.

I'd rate the solution nine out of ten.

My company uses Apache Kafka to keep some intermediate data in the workflow.

Apache Kafka's most valuable features include clustering and sharding. Though we have not started using Apache Kafka Streams in our company, I have heard that it is one of the good features of the product we plan to use. The good features let you replay and reconsume messages in Apache Kafka, allowing you to have multiple consumer groups. The rebalancing feature of the product is also useful since if one consumer dies, then Apache Kafka does a rebalancing. With Apache Kafka, we use clustering, sharding, and partitioning features in our company.

In Apache Kafka, it is currently difficult to create a consumer. The implementation of Apache Kafka's features, like rebalancing, is possible only when you create a consumer, which is a very difficult task since it is overly complicated. To create a consumer in Apache Kafka, a person needs to have a very strong knowledge of the internal functioning of Apache. I feel that Kaka needs to provide a consumer so that its users don't spend time in the creation of consumers. In general, Apache Kafka must provide users with a more user-friendly UI.

I have been using Apache Kafka for five years. I am just a customer of the solution.

It is a pretty stable solution. Stability-wise, I rate the solution a nine out of ten.

Scalability-wise, I rate the solution an eight out of ten.

The users of the solution are not directly involved with it. Those who use Apache Kafka in our company use it to push orders on the frontend, and then the frontend calls for some microservice, after which the microservice pushes data to Apache Kafka. Around 10,000 to 15,000 people in my organization follow the aforementioned procedure.

I won't be able to comment on the technical support team of Apache Kafka since some other team members in my company were involved with them. In general, my company is satisfied with the technical support team of Apache Kafka. I rate Apache Kafka's technical support an eight out of ten.

Positive

My company started off with Apache Kafka, so they did not use any other solutions previously.

If you are using the latest version of Apache Kafka in which the use of Zookeeper is not required, then it uses the KRaft protocol, which is built into Kafka broker. Since the use of Zookeeper is no longer required, I think the setup phase of dissolution is better than its previous versions. I rate the initial setup of the product somewhere between seven to eight out of ten.

Apache Kafka's initial setup is very straightforward.

The solution is deployed on an on-premises model.

Apache Kafka was deployed in our company within a couple of days.

Three people were involved in the deployment process of Apache Kafka.

I rate Apache Kafka's pricing a five on a scale of one to ten, where one is cheap and ten is expensive. There are no additional costs apart from the licensing fees for Apache Kafka.

During my company's evaluation process for other options from Apache Kafka, we came across RabbitMQ. My company chose Apache Kafka over RabbitMQ since it was one of the market's more popular tools. With Apache Kafka, more materials, support, online technical groups, and forums were available for consumers.

Apache Kafka as a broker tool is a very stable and good product. When you need to create a consumer in any programming language, including Java, Golang or any other programming language, the team involved in the process of the creation of a consumer should have very strong knowledge and expertise in the use of Apache Kafka since it is not at all easy to create a consumer for the product. A highly qualified person with a good amount of experience should also know the internals of the solution, which may not seem too straightforward. Anyone cannot use Apache Kafka easily without proper knowledge or experience. When you use Apache Kafka in your actual application, you need to create some producers and some consumers. To create a consumer, you need to have a very strong understanding of the solution since it is not a process that anyone can manage easily. A company needs to have a very strong team with good technical knowledge to be able to use the product.

I rate the overall solution a nine out of ten.