Principal Architect And Management Council Member at a tech services company with 51-200 employees
Real User
2018-07-25T06:10:52Z
Jul 25, 2018
This can be split into two aspects:
- Features and Functionality
- Quality of Service
- Industry support
Within the first, we would evaluate on:
- Support for different Operating System Platforms
- Availability of API across languages, in order of importance - Java, C#, Python, C++
- Conformance to standards - JMS API availability, Queueing standards
- Availability of different addressing schema and messaging styles - Pub-Sub, Producer-Consumer
- Integration and Inter-operability with different types of middle ware - Application Servers, TP Monitors
- Connector availability across different technologies and platforms - with enterprise applications (CRM, ERP, Warehousing), packaged products - Banking platforms like Globus, Finacle, Telco apps like Amdocs, Arbor/BP etc
- Ability to handle different formats - XML, JSON, SAP iDoc
Against the quality of service parameters, the factors to evaluate are
- Performance and scalability - Volumes in terms of number of queues, no of simultaneous messages per queue, number of subscribers/publishers supported
- Availability of commonly agreed upon performance benchmarks
- Security - Authorized access and access control across queues, confidentiality of messages, encryption capability.
- Reliability - Reliable queueing, recoverability of messaging, ability to cluster and create fail-safe message queues, failure cutover timelines
- Stability - little or no degradation over time the messaging server is in running state
- Availability of administration tools to monitor size and performance of queues, ability to quickly identify and recycle lost or dropped messages, auto-healing capabilities
Against industry support parameters, we would evaluate
- Installed base in large enterprises across various business lines and geographies
- Availability of developers
- Participation in industry standard bodies like TOGAF etc
As an architect, my first response is to ask what are your SLA’s for message delivery? They may determine the characteristics and thresholds of the characteristics when selecting a solution – such as resilience, robustness, throughput, scalability and error handling.
Here are the important capabilities I look for in a messaging engine because I support large mission critical applications with billions of messages a day:
1 - Message reliability and precision: Assured delivery where no message is lost or duplicated. For example, a bank cannot lose a check or process the same check twice.
2 - Scalability: Scales to handle billions (possibly even trillions) of messages a day
3 - Robustness: The messaging engine runs forever with no unscheduled production outage
4 - Universal Connectivity: Connects applications and systems together, locally and globally (on-premise, in private and public clouds). Works across all environments and systems, and is portable across clouds, removing cloud provider lock-in
5 - Secured: Secured messages at rest or in transition
6 - Simple development/APIs: Build and quickly deliver business capabilities with a few and simple APIs
7 - Monitoring, logging, and problem determination: Provide easy to use management and monitoring tools
As in all things software, relevance to task, benefits compared to cost and effort, ease of use for developers, documentation and SAMPLE CODE. You aren't going to find one thing that does everything best - go for maximum reliability and your performance will go down. You would not want a job execution queue running off of Kafka - it fills a different niche.
After doing comprehensive research on the different MQ software options available on the market, I came to the conclusion that the top two are IBM MQ and Apache Kafka. Here’s why.
IBM MQ is a rich, highly reliable, and stable messaging platform. Without a doubt, you can feel comfortable that as soon as a message is pushed into your IBM MQ, it will be delivered to its destination without fail. You can even send thousands of messages at the same time without any issue. I think the best part about the solution is that messages never get lost, even if you have server or network problems.
In addition, the solution can be configured quite easily and it has a simple UI. It also has effective authorization management, with minimum effort needed for administration. The storage is flexible and you can easily connect to target systems. Whatsmore is the flexibility of the application is vast and very straightforward and it can do a lot of customizations and parameters. Furthermore, it can be easily integrated with other platforms. It is very easy to access when traveling, and does a fantastic job at securing your data that is at rest or on the move without altering applications.
The only downside I can think of is that IBM MQ is a bit expensive and sometimes IBM lacks quick response times if a support ticket is logged with them.
Apache Kafka comes in at a close second to IBM MQ. First of all, it is open source, has granular message retention options, and also provides good third-party support. I think one of its most valuable features is the architectural style of messaging or event streaming. Messages can be transported very quickly and can be changed very quickly, too. I also like it because messages are automatically stored based on parameter setup of retention policy. In addition, with Apache Kafka, messages can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance and you don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA, or others. If that’s not enough, it has other advantages like great performance, and is easy to integrate with big data technologies. Like IBM MQ, due to its distributed nature, Apache Kafka is capable of operating very quickly and can handle millions of messages every second.
Apache Kafka can have a big memory and/or disk footprint depending on your scenario. Be prepared to delegate resources if your amount of data gets more and more. Kafka is lean by default, but it does require memory (in-memory storage) and disk (offloading) to keep your data. Apache Kafka also has an abundance of options for managing and maintaining queues, which is helpful.
The downside to Apache Kafka is that it is a community-made Java application that looks and feels kind of like it’s from the past century, so you may feel that it is a bit dated.
All in all, both solutions are fantastic but I would say IBM MQ is slightly more powerful than Apache Kafka.
These four pillars -
Developer ExperienceObservabilityPerformance and EfficiencyStabilityNo question at all that Kafka and RabbitMQ dominate the market, but they also hold multiple challenges. If you do want the "big 4" without compromising simplicity and ease of use, I recommend Memphis.dev.
Hello,
I'm working at a university and currently, I'm researching Message Queue (MQ) Software, such as Amazon SQS, Anypoint MQ, IBM MQ and Apache Kafka.
I would like to hear about examples of using asynchronous messages in request/response messaging patterns. What SW would you recommend?
VP of Data and Analytics at a tech services company with 201-500 employees
Oct 18, 2021
I would recommend Apache Kafka as the preferred option.
It is an open-source project and has most of the user community members to contribute/enhance. It is relatively mature.
Hi - this is a really good page to understand more about the differences between message queuing technologies and Kafka: https://developer.ibm.com/arti...
In short, if you need conversational messaging (request/response) or targeted reliable delivery (exactly-once delivery), and you don't need the data stored for historical purposes then message queuing is perfect.
If you want to expose data for insight and status, and you don't have a specific intended recipient for that data, then something like Kafka (or a software product that offers publish/subscribe capabilities) is a good fit.
In terms of products, IBM MQ is the market leader (both in terms of capabilities and the number of customers using the technology in production) for Message Queuing and it also provides support for fine-grained events through Pub/Sub capabilities. Many of the world's leading banks, healthcare organizations, manufacturing businesses, etc use IBM MQ either on-premise in their own private data centers or on the public cloud.
If you want to try it out, here is a really good set of developer materials to get you started and there is a badge at the end to demonstrate that you can create a production-ready messaging solution. It takes about 2 hours to complete: https://developer.ibm.com/lear...
PeerSpot’s crowdsourced user review platform helps technology decision-makers around the world to better connect with peers and other independent experts who provide advice without vendor bias.
Our users have ranked these solutions according to their valuable features, and discuss which features they like most and why.
You can read user reviews for the Top Message Queue (MQ) Software to help ...
I have not used it - sorry
message persistence, able to survive restarts. be easily secured. high available functionality either by being clustered or replicated.
The durability of the messages. The messages must survive a restart. Then stability, performance, ease of administration.
Security, Stability, Reliability, Scalability and Performance
This can be split into two aspects:
- Features and Functionality
- Quality of Service
- Industry support
Within the first, we would evaluate on:
- Support for different Operating System Platforms
- Availability of API across languages, in order of importance - Java, C#, Python, C++
- Conformance to standards - JMS API availability, Queueing standards
- Availability of different addressing schema and messaging styles - Pub-Sub, Producer-Consumer
- Integration and Inter-operability with different types of middle ware - Application Servers, TP Monitors
- Connector availability across different technologies and platforms - with enterprise applications (CRM, ERP, Warehousing), packaged products - Banking platforms like Globus, Finacle, Telco apps like Amdocs, Arbor/BP etc
- Ability to handle different formats - XML, JSON, SAP iDoc
Against the quality of service parameters, the factors to evaluate are
- Performance and scalability - Volumes in terms of number of queues, no of simultaneous messages per queue, number of subscribers/publishers supported
- Availability of commonly agreed upon performance benchmarks
- Security - Authorized access and access control across queues, confidentiality of messages, encryption capability.
- Reliability - Reliable queueing, recoverability of messaging, ability to cluster and create fail-safe message queues, failure cutover timelines
- Stability - little or no degradation over time the messaging server is in running state
- Availability of administration tools to monitor size and performance of queues, ability to quickly identify and recycle lost or dropped messages, auto-healing capabilities
Against industry support parameters, we would evaluate
- Installed base in large enterprises across various business lines and geographies
- Availability of developers
- Participation in industry standard bodies like TOGAF etc
As an architect, my first response is to ask what are your SLA’s for message delivery? They may determine the characteristics and thresholds of the characteristics when selecting a solution – such as resilience, robustness, throughput, scalability and error handling.
Here are the important capabilities I look for in a messaging engine because I support large mission critical applications with billions of messages a day:
1 - Message reliability and precision: Assured delivery where no message is lost or duplicated. For example, a bank cannot lose a check or process the same check twice.
2 - Scalability: Scales to handle billions (possibly even trillions) of messages a day
3 - Robustness: The messaging engine runs forever with no unscheduled production outage
4 - Universal Connectivity: Connects applications and systems together, locally and globally (on-premise, in private and public clouds). Works across all environments and systems, and is portable across clouds, removing cloud provider lock-in
5 - Secured: Secured messages at rest or in transition
6 - Simple development/APIs: Build and quickly deliver business capabilities with a few and simple APIs
7 - Monitoring, logging, and problem determination: Provide easy to use management and monitoring tools
As in all things software, relevance to task, benefits compared to cost and effort, ease of use for developers, documentation and SAMPLE CODE. You aren't going to find one thing that does everything best - go for maximum reliability and your performance will go down. You would not want a job execution queue running off of Kafka - it fills a different niche.
Stability. Routing. Performance. Acknowledging. Clustering. Availability.
No Loss of data.
scale, ease of use, open support (source AND standards). Avoid lock-in. (and be careful that you aren't drawn into the lock-in pool by marketing.
Reliability, scalability, and performance
Stability