IT Central Station is now PeerSpot: Here's why
Barista Brewing Espresso at Linkedln
Real User
Great horizontal scaling, design with library simplicity
Pros and Cons
  • "Good horizontal scaling and design."
  • "Lacks elasticity and the ability to scale down."

What is our primary use case?

Our primary use case of this solution is for data integration and for real-time data consumption. I'm a senior staff engineer for data and infrastructure and we are customers of Apache. 

What is most valuable?

I love the simplicity of the library and the design as well as the architectural concept which is like horizontal scaling.

What needs improvement?

When compared to other commercial competitors, Kafka doesn't have the ability to scale down, the elasticity is lacking in the product. The other issue for us is the delayed queue, which was available to us in the commercial software but not in Kafka. It's something we use in most of our applications for deferred processing and I know it's available in other solutions. I'd like to see some tooling support and language support in the open source version. 

For how long have I used the solution?

I've been using this solution for four years.

Buyer's Guide
Apache Kafka
August 2022
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: August 2022.
621,327 professionals have used our research since 2012.

What do I think about the stability of the solution?

The stability is good. 

What do I think about the scalability of the solution?

The solution scales horizontally and scales better than its competitors. We have around 400 to 500 microservices consuming this cluster and the company has around 600 employees. We have four different verticals, each with around 100 engineers with 100 to 150 microservices. 90% of the microservices have a touchpoint with Kafka.

How are customer service and support?

I think the community is very good and will respond if you raise a ticket. We also use external third-party libraries that were built in GitHub. It would be good to have some direct support from Apache.

Which solution did I use previously and why did I switch?

Four years ago we were using Rabbit MQ but we switched to Kafka because Rabbit was designed for a very narrow use case. It became difficult for us to run and maintain that server and our client libraries. We had a huge outage, so we shifted to Kafka because of the simplicity in the architecture.

How was the initial setup?

The initial setup was simple although we had a couple of hiccups. It took around a week but that was several years ago and we haven't had any problems since. Our team carried out the deployment and we currently have a few engineers who deal with maintenance. 

What's my experience with pricing, setup cost, and licensing?

We are currently using the open-source version. 

What other advice do I have?

There is room for improvement with this solution so I rate it eight out of 10. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Ravi Kuppusamy - PeerSpot reviewer
CEO and Founder at BAssure Solutions
Real User
Top 5Leaderboard
Plenty of adapters, beneficial for enterprises, and high availability
Pros and Cons
  • "Apache Kafka has good integration capabilities and has plenty of adapters in its ecosystem if you want to build something. There are adapters for many platforms, such as Java, Azure, and Microsoft's ecosystem. Other solutions, such as Pulsar have fewer adapters available."
  • "Pulsar gives more scalability to an even grouping, but Apache Kafka is used more if you want to send something in a time series-based. If this does not matter to you then Pulsar could be more customizable. Apache Kafka is nothing but a streaming system with local storage."

What is our primary use case?

We are building solutions on Apache Kafka for four customers. The customers we have are in various sectors, such as healthcare and architecture.

What is most valuable?

Apache Kafka has good integration capabilities and has plenty of adapters in its ecosystem if you want to build something. There are adapters for many platforms, such as Java, Azure, and Microsoft's ecosystem. Other solutions, such as Pulsar have fewer adapters available.

For how long have I used the solution?

I have been using Apache Kafka for three years.

What do I think about the stability of the solution?

Apache Kafka is stable.

What do I think about the scalability of the solution?

I would recommend Apache Kafka for any enterprise.

The amount of people using the solution depends on the application. However, the starting point is from 6,000 to 7,000 concurrent users.

How are customer service and support?

There is not any support, Apache Kafka is open-source.

Which solution did I use previously and why did I switch?

We have been experimenting with other solutions such as VMware RabbitMQ and Pulsar.

We are going to replace the Apache Kafka solution using Pulsar.

Pulsar gives more scalability to an even grouping, but Apache Kafka is used more if you want to send something in a time series-based. If this does not matter to you then Pulsar could be more customizable. Apache Kafka is nothing but a streaming system with local storage. Apache Kafka fits into many use cases, it's very direct, but if you want more specific use cases and you use Apache Kafka, Pulsar could be considered.

How was the initial setup?

Apache Kafka was simple to install. If you have a complicated clustered production, it takes time. However, for the development, it doesn't take more than one or two hours.

What about the implementation team?

We have approximately two to four technical managers that are deploying and supporting Apache Kafka. A technical manager is necessary.

What's my experience with pricing, setup cost, and licensing?

Apache Kafka is an open-sourced solution. There are fees if you want the support, and I would recommend it for enterprises. There are annual subscriptions available.

What other advice do I have?

Apache Kafka is one of the best open-source solutions that are available today.

I would recommend this solution to others.

I rate Apache Kafka an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Buyer's Guide
Apache Kafka
August 2022
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: August 2022.
621,327 professionals have used our research since 2012.
Assistant Student at a retailer with 5,001-10,000 employees
Real User
Top 10
Reliable solution for processing broker messages from many clients
Pros and Cons
  • "The most valuable feature is the messaging function and reliability."
  • "Something that could be improved is having an interface to monitor the consuming rate."

What is our primary use case?

I have a lot of messages, and we need to process those messages from many clients. Each client takes those messages and processes them.

I'm using the brokerage partner. I'm not storing or maintaining the application on servers. I'm just a client for the Apache Kafka server.

The solution is deployed on-prem.

How has it helped my organization?

Apache Kafka has improved our organization because it's more reliable than Rabbit. That's the whole point for us.

What is most valuable?

The most valuable feature is the messaging function and reliability.

What needs improvement?

Something that could be improved is having an interface to monitor the consuming rate. We use something, but I'm not sure if it's from Apache Kafka, or if it's a borrowed third-party solution. So, the interface for monitoring the processes is an additional feature that could be added.

For how long have I used the solution?

I have been using this solution for two years.

What do I think about the stability of the solution?

The solution is pretty stable compared to Rabbit or other brokers. 

What do I think about the scalability of the solution?

The solution is scalable. We have about 10 departments that use Kafka in various forms. Each department might have 5 or 10 people.

We use the solution all the time. We have consumers that consume messages that come every day because we have clients and customers for the main website. All of those messages go to KAF clients. Our backend departments consume messages from the actions of the final customers.

Which solution did I use previously and why did I switch?

We used Rabbit and we switched to Kafka because it seemed like an upgrade in ability, reliability, and in the consuming process of broker messages.

How was the initial setup?

Implementations took half a year for everyone to learn the solution. It was quite lengthy.

What other advice do I have?

I would rate this solution 9 out of 10.

My advice is to take some time in investigating how to implement the solution.

We used to require about half a year to implement in our organization. Someone who needs to implement Kafka has to be prepared for a quite lengthy process. Don't expect implementation to be completed in a week. It's a little bit longer because it's complex.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Reza Sadeghi - PeerSpot reviewer
Senior Developer at a non-profit with 10,001+ employees
Real User
Top 5Leaderboard
The command line interface is powerful
Pros and Cons
  • "Kafka is an open-source tool that's easy to use in our country, and the command line interface is powerful."
  • "The user interface is one weakness. Sometimes, our data isn't as accessible as we'd like. It takes a lot of work to retrieve the data and the index."

What is our primary use case?

We use Kafka daily for our messaging queue to reduce costs because we have a lot of consumers, producers, and repeat messages. Our company has only one system built on Apache Kafka because it's based on microservices, so all of the applications can communicate using it.

What is most valuable?

Kafka is an open-source tool that's easy to use in our country, and the command line interface is powerful. 

What needs improvement?

The user interface is one weakness. Sometimes, our data isn't as accessible as we'd like. It takes a lot of work to retrieve the data and the index.

For how long have I used the solution?

I've used Kafka for about 10 months.

What do I think about the stability of the solution?

Kafka is stable.

How are customer service and support?

We can't access support because we are in Iran, and many countries prohibit business with Iran. 

Which solution did I use previously and why did I switch?

We used MSMQ on Windows, but we decided to migrate our system to Docker and we wanted to use base Linux, so we move them from Amazon Queue to Kafka.

Apache Kafka has one advantage that sets it apart from other providers. We need to iterate on the messages, but others don't have this feature. Kafka has partitioning, which is useful, so we decided to go with Kafka. 

How was the initial setup?

I rate Kafka 10 out of 10 for ease of setup. It's easy for us because we use Docker, but if you want to use another system like Linux it may be a little challenging

What's my experience with pricing, setup cost, and licensing?

Kafka is free. 

Which other solutions did I evaluate?

Redis has an open-source solution, but I'm not sure about IBM. I haven't researched it. 

What other advice do I have?

I rate Apache Kafka seven out of 10. It's a good solution. They're constantly fixing bugs and adding new features. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Ravi-Patel - PeerSpot reviewer
Assistant Professor at CHAROTAR UNIVERSITY OF SCIENCE AND TECHNOLOGY
Real User
Difficult to configure, lacking automation, but has good community support
Pros and Cons
  • "The valuable features are the group community and support."
  • "The solution can improve by having automation for developers. We have done many manual calculations and it has been difficult but if it was automated it would be much better."

What is our primary use case?

We are in the early stages of testing this solution in our lab as a demo. It is in development and we are not in production at this point.

We are using this solution to relay events when they happen to multiple receivers at once to allow better functionality.

How has it helped my organization?

Apache Kafka has helped our client's online restaurant company by allowing them to take any orders and send the notifications with some other details, such as logic commands, to the different microservices.

What is most valuable?

The valuable features are the group community and support.

What needs improvement?

The solution can improve by having automation for developers. We have done many manual calculations and it has been difficult but if it was automated it would be much better.

For how long have I used the solution?

I have been using this solution for approximately three months.

What do I think about the scalability of the solution?

The solution's scalability is important for our ability to have more throughput from multiple receivers. If we need more throughput it can deliver.

Which solution did I use previously and why did I switch?

We did use other solutions previously but this solution makes things a lot easier.

How was the initial setup?

The installation is fairly easy. Additionally, there is a cloud-based version available if a use case requires it.

What about the implementation team?

We did the implementation ourselves.

What's my experience with pricing, setup cost, and licensing?

The solution is free, it is open-source.

What other advice do I have?

There is a lot of configuration involved in this solution. We have found many configurations that have helped us but it would be beneficial if there was automation. 

I rate Apache Kafka a five out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Reza Gholami - PeerSpot reviewer
Senior Technology Architect at a tech services company with 10,001+ employees
Real User
Top 5
A resilient solution for metrics collection and monitoring
Pros and Cons
  • "Resiliency is great and also the fact that it handles different data formats."
  • "Some vendors don't offer extra features for monitoring."

What is our primary use case?

We use Apache Kafka for financial purposes. Every time one of our subscribed customers is due for an insurance payment, Apache Kafka sends an automated notification to the customer to let them know that their bill is due.

What is most valuable?

Resiliency is great and also the fact that it handles different data formats. There is one data format that's universal across multiple application domains — Avro. It's pretty universal compared to JSON, XML, SQI, and other formats.

What needs improvement?

Some vendors don't offer extra features for monitoring. Some come with Linux for default monitoring. Monitoring is very important. If something is not working properly, then our subscribers won't receive a notification. You then have to trace it back to Kafka and find the glitch or the messaging sequence that hasn't been racked up correctly.

It should support Avro — which handles different data formats — as a default data format. It would be much more flexible if it did.

For how long have I used the solution?

I have been using Apache Kafka for three years.

What do I think about the stability of the solution?

It seems to be quite stable.

What do I think about the scalability of the solution?

Apache Kafka is Scalable. You can actually launch a server node or a broker. Three nodes and Zookeeper (the Kafka server management system) is optimal. If one of them goes down you can automatically launch another one. You can go three servers or brokers back — there's a repetition on each Kafka broker.

How are customer service and technical support?

Apache Kafka is open-source. They don't offer technical support.

What other advice do I have?

On a scale from one to ten, I would give Apache Kafka a rating of eight.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
freelance at a tech services company with 11-50 employees
Real User
Top 5Leaderboard
The solution is flexible, stable, reliable, and robust
Pros and Cons
  • "I like Kafka's flexibility, stability, reliability, and robustness."
  • "Kafka has a lot of monitors, but sometimes it's most important to just have a simple monitor."

What is most valuable?

I like Kafka's flexibility, stability, reliability, and robustness.

What needs improvement?

Kafka has a lot of monitors, but sometimes it's most important to just have a simple monitor. Improvements to Kafka's management would be nice, but it's not so necessary for me. There are a lot of consoles that offer a better view than Kafka. Some are free, and some are paid, but I'm thinking about streaming. For example, if you connect more streams to a component in the same queue, how will it integrate to recognize the flow and the message?

For how long have I used the solution?

I've been using Kafka for more than two years.

What do I think about the scalability of the solution?

Kafka is stable. Defining our user base is hard because Kafka influences the whole company, so you could say around 100 users. Kafka is a core system, so it affects all users we choose to link to the primary key.

Which solution did I use previously and why did I switch?

I previously used IBM MQ. 

How was the initial setup?

The first time we tried to deploy Kafka, it seemed a little complicated, but the second try went better. Sometimes it isn't easy to set up the necessary communication or estimate how many partitions we need. Some applications have a vast amount of data, so we have to consider how to improve the performance and not increase the transfer times.

What other advice do I have?

I rate Apache Kafka nine out of 10. I think it's one of the best tools on the internet.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Solutions Architect at a tech services company with 201-500 employees
Real User
Good support, stable, and it supports a high volume of data
Pros and Cons
  • "The most valuable feature is the support for a high volume of data."
  • "The initial setup and deployment could be less complex."

What is our primary use case?

We are a solution provider and Apache Kafka is being used in our client's company.

What is most valuable?

The most valuable feature is the support for a high volume of data.

What needs improvement?

The initial setup and deployment could be less complex.

Integration is one of the main concerns that we have.

For how long have I used the solution?

We have been using Apache Kafka for two years.

What do I think about the stability of the solution?

Kafka is a stable product.

What do I think about the scalability of the solution?

This is a scalable solution.

How are customer service and technical support?

The technical support is quite good, and we have no problem with it.

Which solution did I use previously and why did I switch?

We also use IBM MQ. It is also a stable product.

IBM MQ is probably easier to deploy than Kafka.

In addition to these, I have also worked with RabbitMQ.

How was the initial setup?

Deploying Kafka is more complex than IBM MQ.

Which other solutions did I evaluate?

My customer has asked me to choose between IBM MQ and Apache Kafka. I will be comparing these two solutions in the near future. My impression is that Kafka is going to better suit my customer, but I have to consider their specific needs before I can be sure.

What other advice do I have?

This is a solution that I may recommend, but its suitability depends on the needs and requirements.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Apache Kafka Report and get advice and tips from experienced pros sharing their opinions.
Updated: August 2022
Product Categories
Message Queue (MQ) Software
Buyer's Guide
Download our free Apache Kafka Report and get advice and tips from experienced pros sharing their opinions.