IT Central Station is now PeerSpot: Here's why

Apache Kafka OverviewUNIXBusinessApplication

Apache Kafka is #2 ranked solution in top Message Queue Software. PeerSpot users give Apache Kafka an average rating of 8 out of 10. Apache Kafka is most commonly compared to IBM MQ: Apache Kafka vs IBM MQ. Apache Kafka is popular among the large enterprise segment, accounting for 75% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 22% of all views.
Apache Kafka Buyer's Guide

Download the Apache Kafka Buyer's Guide including reviews and more. Updated: June 2022

What is Apache Kafka?

Apache Kafka is a distributed streaming platform, with the following capabilities:

  • It lets you publish and subscribe to streams of records. In this respect it is similar to a message queue or enterprise messaging system.
  • It lets you store streams of records in a fault-tolerant way.
  • It lets you process streams of records as they occur.

Apache Kafka gets used for two broad classes of application:

  • Building real-time streaming data pipelines that reliably get data between systems or applications.
  • Building real-time streaming applications that transform or react to the streams of data.
Apache Kafka Customers
Uber, Netflix, Activision, Spotify, Slack, Pinterest
Apache Kafka Video

Apache Kafka Pricing Advice

What users are saying about Apache Kafka pricing:
  • "Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription."
  • "Apache Kafka is free."
  • "The solution is open source; it's free to use."
  • "The price for the enterprise version is quite high. For on-premise, there is an annual fee, which starts at 60,000 euros, but it is usually higher than 100,000 euros. The cost for a project including the subscription is usually between 100,000 to 200,000 euros. The cost also depends on the level of support. There are two different levels of support."
  • "It's a bit cheaper compared to other Q applications."
  • Apache Kafka Reviews

    Filter by:
    Filter Reviews
    Industry
    Loading...
    Filter Unavailable
    Company Size
    Loading...
    Filter Unavailable
    Job Level
    Loading...
    Filter Unavailable
    Rating
    Loading...
    Filter Unavailable
    Considered
    Loading...
    Filter Unavailable
    Order by:
    Loading...
    • Date
    • Highest Rating
    • Lowest Rating
    • Review Length
    Search:
    Showingreviews based on the current filters. Reset all filters
    freelance at a tech services company with 11-50 employees
    Real User
    Top 5Leaderboard
    Open source, granular message retention options, and good third party support
    Pros and Cons
    • "When comparing it with other messaging and integration platforms, this is one of the best rated."
    • "The model where you create the integration or the integration scenario needs improvement."

    What is our primary use case?

    I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

    What is most valuable?

    The most valuable features of this solution is the architectural style of messaging or event streaming. First is important to understand that anything can be represented as an event=message e.g new order, status change, confirmation, information from IoT or from monitoring system. Each message can be transport very quickly from the source (producer) into consumer(s). Messages can be changed in the fly - streaming messaging. Messages can be process exactly at once or at least once. Ordering of messages depends on just configuration setup.  When comparing it with other messaging and integration platforms, this is one of the best rated. Message store is out of the box functionality. Messages are automatically stored based on parameter setup of retention policy. Messages can remain for longer, which can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance window. You don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA or other. One platform for classical messaging, real time messaging, ETL, message streaming.  ETL can be realised with connectors into external systems which also could run in more instances. Exists a lot of ready to use connectors and new ones can be developed.

    What needs improvement?

    The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing.  If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere.  Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs. Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating. It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful. In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment. Having a governance style would also be helpful to understand.  It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

    For how long have I used the solution?

    I have been working with this solution for one year.
    Buyer's Guide
    Apache Kafka
    June 2022
    Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: June 2022.
    608,010 professionals have used our research since 2012.

    What do I think about the stability of the solution?

    It's a stable solution.

    What do I think about the scalability of the solution?

    Kafka is very scalable, which is an important feature of it. Our clients have approximately ten applications in their companies that communicate with Kafka.

    How are customer service and support?

    I have not contacted technical support through Kafka, I communicate directly with Confluence. Confluence is the company that developed the open-source platform and they provide support. The communication is very good and they are very capable of assisting you with all technical inquiries. There are direct contacts that make it easy.

    Which solution did I use previously and why did I switch?

    Previously, I worked with IBM MQ, a different type of messaging platform.

    How was the initial setup?

    For the most part, the initial setup is easy, but if you need a more sophisticated infrastructure or if you have to set up the topics, then you have to be careful and you have to be more knowledgable in Kafka. You will have to know the parameters for the rotations, the size of the message, and the timeouts, as an example. For a developer it is easy, but for an administrator and production, it requires more.

    What's my experience with pricing, setup cost, and licensing?

    Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription.

    What other advice do I have?

    I would recommend trying this solution.  Take the time to understand it because it is a different style when it comes to working with data. I would rate this solution a nine out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: My company has a business relationship with this vendor other than being a customer: partner
    Solution Architect at a manufacturing company with 10,001+ employees
    Real User
    Good performance when a high throughput is required, but they need to implement a portal
    Pros and Cons
    • "The processing power of Apache Kafka is good when you have requirements for high throughput and a large number of consumers."
    • "They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard."

    What is our primary use case?

    I am a solution architect and I used Apache Kafka in this role.

    What is most valuable?

    The processing power of Apache Kafka is good when you have requirements for high throughput and a large number of consumers. 

    What needs improvement?

    They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard. It could be used to do the preprocessing or the configurations, instead of directly doing it on the queues or the topics. If you look at Solace, for example, they have come up with a portal where you don't need to touch these activities. You don't need to access the platform beyond the portal.

    For how long have I used the solution?

    I have used Apache Kafka for between one and one and a half years.

    What do I think about the stability of the solution?

    Apache Kafka is stable.

    What do I think about the scalability of the solution?

    This is certainly a scalable product. There are currently 30 or more people using it but we expect to scale beyond this. It is going to be an enterprise tool within the company.

    How are customer service and technical support?

    I am not directly interacting with the service people at this moment. It is limited for now because we are still exploring and effecting our architecture and design, and deciding how to align it with our existing strategy. There is not much progress in this regard and it will take more time.

    Which solution did I use previously and why did I switch?

    Prior to working with Apache Kafka, there was no messaging queue system. For many projects, they were using the Azure Event Hub, but it was not serving the purpose. So, we started moving towards Kafka, and that's why we have procured Confluent Kafka.

    Several months ago, I stopped working on Apache Kafka. I am now working on Confluent Kafka. It was not my decision to switch solutions.

    My current organization has chosen Confluent Kafka for various reasons. One is that we have a large number of streaming requirements, and Confluent Kafka has one more layer on top of Apache Kafka to do this transformation and connecting with other multiple lane systems.

    There are out-of-the-box features along with the KSQL features. For example, things like fetching the events are kind of query-based. So, that seems to be a good feature for our requirements. That is why we ultimately procured Confluent Kafka.

    For some time, I have also worked with Solace and it has an advantage. Given that my core strength is integration, I work with integration platforms such as MuleSoft, Azure functions, then TIBCO. Based on our requirements, I found that the event-driven APA implementation with Solace was easier.

    Solace also has a top-notch solution for portal management and you register your producers, consumers, and preprocessing logic. All of these things are pretty easy to do. This is an area where Kafka could use some enhancement.

    How was the initial setup?

    I don't think that the initial setup was a complex process.

    Which other solutions did I evaluate?

    MQ messaging systems are not my core strength but for any integration platform where we have a large number of APIs and events, to integrate with an IoT platform, for example, I found Kafka is better than ActiveMQ.

    I'm not getting into in MQTT or other things but comparatively, when you compare ActiveMQ and Kafka, Kafka has done better.

    What other advice do I have?

    I think that many people are using Apache Kafka just as a publishing and subscription model, but I feel that Kafka is better than that. Furthermore, Confluent Kafka is even more than that.

    Confluent Kafka is offering features that are equal to those of a data lake. You can do lots with data, and huge data can be persisted. However, many people are not using that feature. Rather than make use of persistence logic, they are pushing the messages and consuming them. Maybe if people were using it for persistence, they would see the impact or real power of Kafka.

    I would rate this solution a seven out of ten.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Buyer's Guide
    Apache Kafka
    June 2022
    Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: June 2022.
    608,010 professionals have used our research since 2012.
    Principal Technology Architect at a computer software company with 5,001-10,000 employees
    Real User
    Events and streaming are persistent, and multiple subscribers can consume the data
    Pros and Cons
    • "With Kafka, events and streaming are persistent, and multiple subscribers can consume the data. This is an advantage of Kafka compared to simple queue-based solutions."
    • "Kafka's interface could also use some work. Some of our products are in C, and we don't have any libraries to use with C. From an interface perspective, we had a library from the readies. And we are streaming some of the products we built to readies. That is one of the requirements. It would be good to have those libraries available in a future release for our C++ clients or public libraries, so we can include them in our product and build on that."

    What is our primary use case?

    It's a combination of an on-premise and cloud deployment. We use AWS, and we have our offshore deployment that's on-premise for OpenShift, Red Hat, and Kafka. Red Hat provides managed services and everything. We use Kafka and a specific deployment where we deploy on our basic VMs and consume Kafka as well.

    We publish or stream all our business events as well as some of the technical events. You stream it out to Kafka, and multiple consumers develop a different set of solutions. It could be reporting, analytics, or even some data persistence. Later, we used it to build a data lake solution. They all would be consuming the data or events we are streaming into Kafka.

    What is most valuable?

    With Kafka, events and streaming are persistent, and multiple subscribers can consume the data. This is an advantage of Kafka compared to simple queue-based solutions.

    What needs improvement?

    We are still on the production aspect, with our service provider or hyper-scalers providing the solutions. I would like to see some improvement on the HA and DR solutions, where everything is happening in real-time. 

    Kafka's interface could also use some work. Some of our products are in C, and we don't have any libraries to use with C. From an interface perspective, we had a library from the readies. And we are streaming some of the products we built to readies. That is one of the requirements. It would be good to have those libraries available in a future release for our C++ clients or public libraries, so we can include them in our product and build on that.

    For how long have I used the solution?

    We've been using Apache Kafka for the past two to three years.

    What do I think about the stability of the solution?

    Kafka is stable. It's a great product. 

    What do I think about the scalability of the solution?

    We did some benchmarking, but we are still looking further to scale up some of the benchmarking and performances. So far, it meets all our business requirements. We are just developers, so everything goes to the clients, who will deploy it at their scale and use it for their end customers. So were are looking at it from a developer's perspective. Those who are developing the products are working on this.

    How are customer service and support?

    We haven't really contacted technical support, but some of our clients have subscribed to support from the vendors. We generally look for open-source solutions. From there, we try to figure out if there are any issues. There's a good online community where you can ask questions.

    How was the initial setup?

    We were able to deploy and use it with no problems for our use case. We didn't find it so complex. We work with so many applications, databases, Postgres, and so many other things, so we could manage it easily. We deployed Kafka in a few hours. We have an infrastructure team and DevOps. Those teams are pretty capable, and they've completely automated the whole deployment. It always takes time the first time you upgrade any application, not just Kafka. We might discover some issues, such as configuration, parameters, compatibility, etc. Once that becomes standard, it is stable, and then they only need to replicate it to the different environments or different developers groups. We have a sophisticated process.

    What other advice do I have?

    I rate Apache Kafka eight out of 10. There are so many products on the market, so my advice is to consider if Kafka suits your business requirements first. If it's suitable, the next step is to check whether all the technical requirements are met. If everything checks out, I would say that Kafka is a relatively stable, sound, and scalable product, so they can try it out. 

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    Sr Technical Consultant at a tech services company with 1,001-5,000 employees
    Real User
    Top 5Leaderboard
    Effective stream API, useful consumer groups, and highly scalable
    Pros and Cons
    • "The most valuable features are the stream API, consumer groups, and the way that the scaling takes place."
    • "would like to see real-time event-based consumption of messages rather than the traditional way through a loop. The traditional messaging system works by listing and looping with a small wait to check to see what the messages are. A push system is where you have something that is ready to receive a message and when the message comes in and hits the partition, it goes straight to the consumer versus the consumer having to pull. I believe this consumer approach is something they are working on and may come in an upcoming release. However, that is message consumption versus message listening."

    What is our primary use case?

    One of our clients needed to take events out of SAP to stream them through Apache Kafka while applying data enrichment before reaching the consumers.

    How has it helped my organization?

    The solution can handle more speed and has horizontal scalability for both messaging, but more specifically stream processing and data enrichment. By using this solution it can reduce the number of components required in the tech stack. For example, we were taking data events out of SAP and sending them to consumers without having to go through multiple processors that were outside of the KAFKA space. Additionally, we are using Kafka from GoldenGate to propagate database updates in real-time.

    What is most valuable?

    The most valuable features are the stream API, consumer groups, and the way that the scaling takes place. 

    What needs improvement?

    I would like to see real-time event-based consumption of messages rather than the traditional way through a loop. The traditional messaging system works by listing and looping with a small wait to check to see what the messages are. A push system is where you have something that is ready to receive a message and when the message comes in and hits the partition, it goes straight to the consumer versus the consumer having to pull. I believe this consumer approach is something they are working on and may come in an upcoming release. However, that is message consumption versus message listening.

    Confluent created the KSQL language, but they gave it to the open-source community. I would like to see KSQL be able to be used on raw data versus structured and semi-structured data.

    For how long have I used the solution?

    I have been using this solution for approximately one year.

    What do I think about the stability of the solution?

    The solution is stable.

    What do I think about the scalability of the solution?

    I have found the Apache Kafka to be highly scalable

    How are customer service and technical support?

    The project we were working on was open-source, we were using Confluent as support and they were great.

    How was the initial setup?

    Apache Kafka on AWS is a bit complex. There is a third-party company called Confluent and they have the support that makes their installation much easier, especially for the on-premise deployment. You install Apache Kafka alone it can be a little complex compared to other queuing messaging solutions.

    The on-premise deployment takes approximately a few days. The cloud or hybrid deployments including all the permissions, typologies, firewalls, and networking configuration can take weeks for all the accessibility issues to be resolved. However, the delay could have been client-related and not necessarily the solution.

    What about the implementation team?

    We provide the implementation service.

    What's my experience with pricing, setup cost, and licensing?

    Apache Kafka is free. My clients were using Confluent which provides high-quality support and services, and it was relatively expensive for our client. There was a lot of back and forth on negotiating the price.

    Confluent has an offering that has Cloud-Based pricing. There are different packages, prices, and capabilities. The highest level being the most expensive. AWS provides services to their market, for example, to have Kafka running. I do not know what the pricing is and I am fairly confident, Azure and GCP provide similar services.

    What other advice do I have?

    My advice to others wanting to implement this solution is to start with data streaming projects, not simple messaging projects because while it is very good at general-purpose messaging, it is more suited and geared for when you are using it as a streaming solution.

    I rate Apache Kafka an eight out of ten.

    Which deployment model are you using for this solution?

    Public Cloud

    If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

    Amazon Web Services (AWS)
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Senior Consultant at a tech services company with 51-200 employees
    Consultant
    Top 20
    Stable, free to use, and offers good stream processing
    Pros and Cons
    • "The stream processing is a very valuable aspect of the solution for us."
    • "The solution could always add a few more features to enhance its usage."

    What is our primary use case?

    Apache Kafka is used for stream processing, metric and log aggregations, and as a message queue for connecting different microservices.

    What is most valuable?

    The stream processing is a very valuable aspect of the solution for us.

    What needs improvement?

    Due to the fact that the solution is open source, it has a zookeeper dependency. If I could change anything about the solution, it would be that.

    The solution could always add a few more features to enhance its usage.

    For how long have I used the solution?

    I've been with the company for at least one year, which is for how long I've been using the solution.

    What do I think about the stability of the solution?

    The stability of the solution is very good, even for large enterprise-level organizations. It's quite reliable. There aren't bugs or glitches that affect it. The solution doesn't crash.

    What do I think about the scalability of the solution?

    The solution is scalable, however, it's a 50/50 endeavor. It may require some management to build it out.

    How are customer service and technical support?

    The solution is open source, so there isn't technical support per se. The open-source community that surrounds the technology, however, is very good.

    That said, our company provides technical support to our clients if they need it. It's 24/7 support and we try to reply within 20 minutes of receiving a request.

    Which solution did I use previously and why did I switch?

    Some of our clients are using Apache, while others are using other solutions. It depends on the company and its unique requirements.

    How was the initial setup?

    The difficulty or simplicity of the initial setup varies. It really depends on the organization and its requirements and infrastructure.

    Deployment times vary. It can be up to a week in production, however, with some products online, some services can be deployed within minutes.

    When you have already deployed the solution, and it's installed, it doesn't require very much maintenance. If it needs any, my company handles it for our clients. We have an entire team that can work on it.

    What's my experience with pricing, setup cost, and licensing?

    The solution is open source; it's free to use.

    What other advice do I have?

    What happens in our company is a little different. We basically provide services to other companies through Kafka, like our management services. It doesn't necessarily mean we're using the solution ourselves, however, we will be going and deploying Kafka for companies, like a systems integrator.

    The version of the solution is normally 2.4, however, it depends on the requirements. Our cloud providers are always different due to the fact that the countries that we work with are all different. For example, in the US it could Amazon, Azure, or Google. It varies.

    I'd advise other organizations considering using the solution to make sure they understand what the use case is. They need to know what their services will be and if they will be directed to Apache Kafka.

    From a customer perspective, potential companies need to make sure they have an idea of how big it's going to be due to the fact that it's a cluster environment. It needs to be taken care of. Customers will need to know things like what is the message rate is which is coming into Kafka and how they will connect all those different microservices or any services together to Kafka.

    From an infrastructure perspective, it's more of how big of a cluster a company needs. Who would be the producers to produce it, and who's the consumer who's consuming the data are a few questions that need to be asked.

    I'd rate the solution eight out of ten.

    Which deployment model are you using for this solution?

    Public Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Mario Estrada - PeerSpot reviewer
    CTO at Estrada & Consultores
    Real User
    Top 20
    Great scalability with a high throughput and a helpful online community
    Pros and Cons
    • "The solution is very easy to set up."
    • "While the solution scales well and easily, you need to understand your future needs and prep for the peaks."

    What is our primary use case?

    We primarily use the solution for upstreaming messages with different payload for our applications ranging from iOT, Food delivery and patient monitoring. 

    For example for one solution we have a real-time location finding, whereby a customer for the food delivery solution wants to know, where his or her order is on a map. The delivery person's mobile phone would start publishing its location to Kafka, and then Kafka processes it, and then publishes it to subscribers, or, in this case, the customer. It allows them to see information in real-time almost instantly.

    How has it helped my organization?

    Apache Kafka has became our main component on almost all our distributed solutions. It has helped us to delivery fast distributing messages to our customer's applications.

    What is most valuable?

    The solution is good for publishing transactions for commercial solutions whereby a duplicate will not affect any part of the system.

    The solution is very easy to set up.

    The stability is very good.

    There's an online community available that can help answer questions or troubleshoot problems. 

    The scalability of Kafka is very good.

    It provides high throughput.

    What needs improvement?

    Kafka can allow for duplicates, which isn't as helpful in some of our scenarios. They need to work on their duplicate management capabilities but for now developers should ensure idempotent operations for such scenarios.

    While the solution scales well and easily, you need to understand your future needs and prep for the peaks. 

    For how long have I used the solution?

    I've been using the solution for four years so far.

    What do I think about the stability of the solution?

    The stability is excellent. There are no bugs or glitches. It doesn't crash or freeze. It's reliable. 

    What do I think about the scalability of the solution?

    Scaling is not really a problem with Kafka. We have used Kubernetes clusters and it is working very well. It scales up and down, almost automatically almost unnoticeable to the consumers, based upon our configuration. Kafka is just one pod inside of our cluster that scales horizontally.

    We have a couple of customers that also have vertical scaling, meaning that, there's more CPU, more memory available to the Kafka pod.

    How are customer service and technical support?

    For Kafka, we don't actually require support from the company. We usually have people experienced in-house and sometimes we just ask in the community. 

    How was the initial setup?

    The initial setup is easy. The majority of the tools today are really very easy to configure and setup. Docker Containers and Kubernetes, actually, have made life easier for architects as well as developers.

    Nowadays, you just install the container, and then you don't have to really manage the internals at libraries, OS levels, et cetera. You just run the container. Everything is containerized.

    What's my experience with pricing, setup cost, and licensing?

    Apache Kafka is OpenSource, you can set it up in your own Kubernetes cluster or subscribe to Kafka providers online as a service.

    What other advice do I have?

    New users should understand the product capabilities. Often, people will start putting their hands in new products without knowing the capabilities and the disadvantages in specific scenarios. In our case for example, We haven't used Kafka for financial transaction processing, for which we still use IBM MQ, but It really depends upon your knowledge and experience with the product. My advice is to understand the product very well, its pros and cons and work from there.

    Finally I'd rate the solution at a nine out of ten.

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    CEO at a comms service provider with 11-50 employees
    Real User
    Reliable for working with a huge amount of data and has many options for building applications on top of it
    Pros and Cons
    • "The high availability is valuable. It is robust, and we can rely on it for a huge amount of data."
    • "The price for the enterprise version is quite high. It would be better to have a lower price."

    What is our primary use case?

    We deploy it for our customers. The main use case is related to log management and metrics because we are a partner of Elastic Stack, and we usually collect information through Kafka.

    What is most valuable?

    The high availability is valuable. It is robust, and we can rely on it for a huge amount of data. 

    The Kafka Streams capability is also valuable. We get many options to build applications on top of Kafka.

    What needs improvement?

    The price for the enterprise version is quite high. It would be better to have a lower price.

    For how long have I used the solution?

    I have been working with this solution for four or five years.

    What do I think about the stability of the solution?

    It is absolutely stable.

    What do I think about the scalability of the solution?

    It is very scalable. It is easy to scale it. 

    It doesn't matter how many users are using it. The licenses are calculated based on the number of nodes. It is not based on the number of users who are using it. We have between 10 to 20 nodes on average in an organization.

    How are customer service and support?

    It is quite good, but they don't speak Italian. In Italy, we have to provide support in the Italian language. It is a problem for customers to have support in English. This is the reason why we provide direct support to customers.

    How was the initial setup?

    I am into pre-sales and project management. I don't usually install Apache Kafka, but its basic installation seems quite simple.

    Its deployment is usually quite short. Usually, we are able to deploy it in a few days, but data management and application development can take a few months.

    What about the implementation team?

    We have our own team to deploy it. We also take care of its maintenance. We have a team of five or six employees to provide 24/7 support to our customers.

    What was our ROI?

    It depends on the project. For log management projects, the ROI is not very quick, but we have other projects where we used Kafka for high-value applications, and the ROI was very quick. We got an ROI in a few months.

    What's my experience with pricing, setup cost, and licensing?

    The price for the enterprise version is quite high.

    For on-premise, there is an annual fee, which starts at 60,000 euros, but it is usually higher than 100,000 euros. The cost for a project including the subscription is usually between 100,000 to 200,000 euros. The cost also depends on the level of support. There are two different levels of support.

    What other advice do I have?

    Kafka is a really good product. To be able to keep it running in the long term, you need to know very well how it works. You should have good knowledge about it. It isn't about just knowing how to install it because it is quite simple to install it. It is important to have the right knowledge and experience to do a good installation and let it run for a long period. You can also go for someone who has the right experience and knowledge.

    We are very satisfied with Kafka. I would rate it an eight out of 10. It is not perfect, but it is a really good product.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    Salvatore Campana - PeerSpot reviewer
    CEO & Founder at XAUTOMATA TECHNOLOGY GmbH
    Real User
    Top 5Leaderboard
    Allows us to ingest a lot of data and make tech decisions in real time
    Pros and Cons
    • "The stability is very nice. We currently manage 50 million events daily."
    • "The repository isn't working very well. It's not user friendly."

    What is our primary use case?

    We use Apache Kafka to ingest a lot of data in real time that Apache Spark processes, and the result is used for a tech decision in real time – in the IT environment, infrastructure environment, and IOT environment, like for a  manufacturing plant.

    This is an open-source framework. We also sell professional services on this solution and specifically create a business application for customers. 

    The application is called Sherlogic. We have two kinds of customers. We have end-user customers that use the Sherlogic solution, and maybe customers don't know that there is Spark and Kafka in Sherlogic. But we have another kind of customer that uses professional services by Xautomata to create tailor-made applications in analytics and the automation process.

    We use Apache Kafka for our digital cloud.

    What needs improvement?

    To store a large set of analytical data we are using SQL repository. This type of repository works very well but we need specific and high maintenance. The user experience is friendly.

    We are looking for alternative solutions, we tried with noSQL solutions and Confluent specific features but the results were not satisfactory both in terms of performance and usability.

    We are working on automated SQL repository management and maintenance tools in order to increase the democratization of our platform.

    For how long have I used the solution?

    We've been using this solution for a year and a half.

    What do I think about the stability of the solution?

    The stability is very nice. We currently manage 50 million events daily.

    What do I think about the scalability of the solution?

    It's scalable.

    How are customer service and support?

    Support is good. It's typical for an open source application. You can have all the information in a public portal. If you want specific consulting, there is a company that promotes this consulting worldwide called Conduent. Their consulting is quick and they have a lot of know-how.

    How was the initial setup?

    It's very complex, like Spark. 

    Deployment took 50 minutes for all the Kubernetes ports, Spark, Kafka, and other components based on Sherlogic. In 30 minutes, we created an environment using this program to make installation easier.

    What about the implementation team?

    Deployment was done in-house, but we're starting a collaboration with another company and we introduced this company to running this solution. Specifically, we started a collaboration with AWS to promote our platform in a Western marketplace. In this way, it's very easy to use our solution because it is a part of an AWS service, certificated by an engineer.

    What was our ROI?

    The return on investment has been having people dedicated to this solution because it's open source so it hasn't been necessary to invest in licensing or pay a fee. So, internal know-how has been the ROI.

    What's my experience with pricing, setup cost, and licensing?

    It's a bit cheaper compared to other Q applications.

    What other advice do I have?

    I would rate this solution 7 out of 10.

    I would recommend this solution because the queue manager is very fast and stable.

    Which deployment model are you using for this solution?

    Public Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate