Apache Kafka on Confluent Cloud provides real-time data streaming with seamless integration, enhanced scalability, and efficient data processing, recognized for its real-time architecture, ease of use, and reliable multi-cloud operations while effectively managing large data volumes.
Apache Kafka on Confluent Cloud is designed to handle large-scale data operations across different cloud environments. It supports real-time data streaming, crucial for applications in transaction processing, change data capture, microservices, and enterprise data movement. Users benefit from features like schema registry and error handling, which ensure efficient and reliable operations. While the platform offers extensive connector support and reduced maintenance, there are areas requiring improvement, including better data analysis features, PyTRAN CDC integration, and cost-effective access to premium connectors. Migrating with Kubernetes and managing message states are areas for development as well. Despite these challenges, it remains a robust option for organizations seeking to distribute data effectively for analytics and real-time systems across industries like retail and finance.
What are the key features of Apache Kafka on Confluent Cloud?In industries like retail and finance, Apache Kafka on Confluent Cloud is implemented to manage real-time location tracking, event-driven systems, and enterprise-level data distribution. It aids in operations that require robust data streaming, such as CDC, log processing, and analytics data distribution, providing a significant edge in data management and operational efficiency.
We find that the best features include using the CDC functionality with the connector to take the data from our SQL database and publish it to many consumers. Any changes enable us to easily publish changes about their domain business objects without too much code and work from domain teams. In this way, we can more easily provide a very robust layer of API and events.
The second use case is easier projection of data. We found that many teams were struggling to create projections and read stores with regular event buses, and Apache Kafka on Confluent Cloud helped us because of all sorts of features, such as the log architecture they have, and other features. KSQL also helped us there.
When order is more important, we rely on Apache Kafka on Confluent Cloud.
The benefits that I have seen from having a real-time architecture include better velocity for developers. That is the main one. Instead of developing many of those capabilities in each team, we can rely on Apache Kafka on Confluent Cloud to provide those functionalities we want, and the teams can focus on their own business instead of providing all sorts of APIs and dependencies to other domains, allowing everyone to run faster.
We find that the best features include using the CDC functionality with the connector to take the data from our SQL database and publish it to many consumers. Any changes enable us to easily publish changes about their domain business objects without too much code and work from domain teams. In this way, we can more easily provide a very robust layer of API and events.
The second use case is easier projection of data. We found that many teams were struggling to create projections and read stores with regular event buses, and Apache Kafka on Confluent Cloud helped us because of all sorts of features, such as the log architecture they have. KSQL also helped us there.
I think what I would improve about the solution is the cost, mostly. From my standpoint, it's the cost. From an engineering perspective, it works really well.
There's always room for improvement. One more point is sometimes it's more UI-related issues. Some of the more high-end features are more complicated to execute. But overall, it's a good product.
I have been using Apache Kafka on Confluent Cloud for around a year, maybe two.
When it comes to assessing the impact of the automated scaling features, we don't measure it, but it's part of our technology stack selection criteria - it's pretty much a must today.
We don't want to increase the headcount in our DBA team. They are the ones managing all our databases, queues, and data sources. So for us, having a very thin layer of management is critical, and we sit with other compute. That's very important for us because headcount is the most expensive part.
We looked at other products, specifically other Kafka providers. We have Apache Kafka and AWS. We looked at self-hosting it, but we wanted Apache Kafka on Confluent Cloud.
Neutral
We were looking for specific use cases. We compared different Kafka solutions, not necessarily competitors. We have a message bus already. We wanted the log capability, mostly.
Regarding ROI in any capacity, whether it's savings from employees or cloud, the ROI was very significant. Although, specifically with Apache Kafka on Confluent Cloud, it was a bit more challenging to increase adoption because it's very expensive. So we had to pick and choose where we implemented to make sure that ROI is positive.
I don't remember the exact number because it's been a while since we did a pricing talk, but it was expensive.
They charge per topic and other resources. Because we are very cost sensitive, we want to approve it and make sure people don't just use it unnecessarily.
I would give Apache Kafka on Confluent Cloud a rating of seven out of ten.
For somebody who's shopping around, looking in this space to decide what to purchase, Apache Kafka on Confluent Cloud is a market leader. It's almost the first choice.
Going with AWS Apache was also very compelling to us because it's very quick to enable stuff in AWS and try it. I would start with those, but first understand if this is actually what you need. There are other much cheaper solutions that serve other use cases, and sometimes people can mix those and just pick the wrong product.
Overall, I would rate Apache Kafka on Confluent Cloud a nine out of ten.
