Apache Kafka on Confluent Cloud Reviews

It's basically four bands of use cases, where we publish data on Kafka topics and stream it across microservices.

Some of the retention policies of Kafka topics have been most beneficial for data management specifically.

Based on our experience, there are different use cases where data needs to be handled in different ways. Sometimes we want to get rid of it once it has been consumed, or we have to store it for a longer period. 

Kafka provides handy properties that allow us to directly configure the data, whether to keep it or discard it after use.

I feel the streaming speed, the way messages are processed, and some of the topic features like partitions and offset management are quite handy.

There's one thing that's a common use case, but I don't know why it's not covered in Kafka. When a message comes in, and another message with the same key arrives, the first version should be deleted automatically. 

We want to keep only one instance of a message at any given time, the latest one. However, Kafka doesn't have this functionality built-in. It keeps all the data, and we have to manually delete the older versions.

 So, I would like to have only one instance of messages, based on the keys. If the key is the same, there should always be the latest message present instead of all versions of that message.

I have been using it for three years. I use the latest version. I work with v3.6.

I would rate the stability a six out of ten. When it's good, it works fine. But as soon as traffic increases and the number of topics on the cluster exceeds a certain limit, it becomes unresponsive. 

We then have to get rid of Kafka topics, but even that's not easy because the whole site becomes unresponsive. We don't have easy dashboard access to remove unnecessary topics. That's one issue.

I would give it an eight out of ten for scalability. It's scalable, but there's room for improvement in reliability. When we scale up at the pod level, reliability goes down due to mismanagement of offsets, leading to data loss. Then, there is mismanagement when we scale it up, and then there is a point where we want to scale down because traffic is less. 

During scale-down, we also often see data loss. They can work on improving this.

We currently have large enterprise business as our customers.

I would rate my experience with the initial setup of this product, a seven out of ten where one is difficult and ten is easy.

I have not faced any difficulties or challenges while setting this product up. They have proper documentation, so it's easy to go through it and set things up.

It's the cloud solution, so it's deployed on the cloud in our customers' organizations. And they use Confluent Cloud. 

It's taken care of by different teams.

Overall, I would rate it an eight out of ten. 

I would recommend it because everything is well-documented and straightforward.

We can install Kafka directly, but we don't have direct access to the data on Kafka. So it's good to have tools like Kafka Magic or Kafka Tool to access and visualize the data.

Our use case is for real-time data integration. It was a preferred tool for this purpose. Additionally, we employed Azure EventHub, another service, as an indicator for real-time data in a couple of larger programs focused on integrating real-time data and visualization.

Kafka and Confluent Cloud have proven to be cost-effective, especially when compared to other tools. In a recent BI integration program over the past year, we assessed multiple use cases spanning ship-to-shore and various Azure integrations. Our findings revealed that Confluent Kafka performed exceptionally well, standing out alongside Genesys and Azure Event Hubs. While these three are top contenders, the choice among other tools depends on the specific use case and project requirements.

The customer initially used tools like SMQs, FITRA, and Stream for real-time data processing. However, after our recommendation, Confluent Cloud proved to be a superior choice, capable of replacing these three tools and simplifying their data infrastructure. This shift to a single tool, Confluent Cloud, streamlined their operations, making maintenance and management more efficient for their internal projects.

Regarding real-time data usage, there were challenges with CDC (Change Data Capture) integrations. Specifically, with PyTRAN, we encountered difficulties. We recommended using our on-premises Kaspersky as an alternative to PyTRAN for that specific use case due to issues with CDC store configuration and log reading challenges with the iton components.

We implemented this solution approximately a year and a half ago for significant projects.

I would rate the stability 8 out of 10. 

I would rate the scalability 8 out of 10. 

I haven't personally implemented it. The recommendation was made during a consulting program based on a thorough assessment of the tool's capabilities, connectors, and services. However, there hasn't been direct implementation on-site.

While most of the capabilities meet current organizational needs, the pricing is slightly higher. Companies need to consider this aspect in their decision-making process, given the managed and comparatively higher price point. I would rate it an eight due to its capabilities and CDC integrations. It stands out as a top tool that can replace others for real-time data integration processes. The CIBC capability has the potential to handle EDCs effectively. While I assume the support is good for out-of-the-box challenges, I'm not certain about the cost. Overall, I believe it's a strong choice with a rating of eight.

We had a legacy website collecting user data as they logged into the portal. We wanted to capture that information in Snowflake and store it in a mobile app. We used Apache Kafka on Confluent Cloud for real-time data streaming.

It is a single platform to publish any information on any given topic. In case of huge transactions on the web or mobile apps, it helps you capture real-time data and analyze it.

There could be an in-built feature for data analysis.

We have been using Apache Kafka on Confluent Cloud for a year.

I rate the product’s stability a nine out of ten. There is room for improvement as we need to put some effort into setting it up to develop use cases. 

The product is highly scalable.

The technical support services are good.

The initial setup requires effort to deploy in the cloud or local environments. Once the servers and other prerequisites are ready, it is simple. It took us one month to complete the process. However, the approximate deployment time depends on particular use cases.

We implemented the product in-house.

I recommend Apache Kafka on Confluent Cloud and rate it a ten out of ten.

Overall, I think it's a good experience. Apache Kafka can be quite complex and difficult to maintain on your own, so using Apache Kafka on Confluent Cloud makes it much easier to use it without worrying about setup and maintenance.

The solution is expensive. 

I have been using Apache Kafka on Confluent Cloud, the platform's main product, for about three months.

I haven't experienced any major stability problems or bugs.

The tool is scalable. I've found Apache Kafka on Confluent Cloud to be very scalable. We've been able to scale up the volume of data we're handling without any issues with performance.

Setting up a new cluster on Apache Kafka on Confluent Cloud is pretty easy. You can click a few options, and there's also integration with other tools to create the necessary resources. However, there are some caveats to be aware of. The type of networking setup you choose can impact your functionality and visibility in the platform. For example, if you have a public cluster, you'll see more metadata information in the console than a more restricted network deployment.

Regarding pricing, Apache Kafka on Confluent Cloud is not a cheap tool. The right use case would justify the cost. It might make sense if you have a high volume of data that you can leverage to generate value for the business. But if you don't have those requirements, there are likely cheaper solutions you could use instead.

I rate the solution an eight out of ten.