VMware Tanzu Data Solutions Reviews

• Does SSL (security)
• Does clustering (stability)
• Integrates with LDAP (management)
• Automatically resends data when a consumer fails
• Automatically routes data
• Excellent spring boot integration
• Multiple programming languages provide excellent integration

With RabbitMQ cluster servicing micro-services, we don't have any downtime and we don't lose any data. We can update and/or upgrade the micro-services without downtime.

• You cannot edit shovels other than by recreating them.
• Routing of data could be more enhanced with a nice GUI. ("IF header.contains(this.thing) THEN data.goesTo(cluster_02)").
• In its current form, you have to recreate a shovel with the same parameters except for the one you want to change. You end up doing more or less a delete/create.
• There is no HTML form where you can click on a shovel and adjust the wrong parameter.
• If I click on a shovel, I get on a page that lists the shovel, but it is not editable. You have to create a shovel and then delete the old one with all the same parameters, except for the one you want to change.
• Temporarily stopping shovels is also not possible in the web interface. I do not know if the CLI version can do it, but if somebody wants to temporarily stop the incoming flow, he or she has to delete the shovel and then recreate it afterwards. This is annoying, to say the least.
• RabbitMQ has to be started before one can define exchanges, queues, and even users with rabbitmqctl. See https://www.rabbitmq.com/man/r...
• This is no problem if one lives in the monolithic server environment. However, if one wanted to make a RabbitMQ Docker-container with a pre-defined set of exchanges, queues, users, and shovels, you have to literally jump start the server. You would have to configure it in the Docker build phase. You would do it like this in the Dockerfile: RUN service start rabbitmq-server && wait 30 && rabbitmqctl add_user mike mikespassword.

I have used RabbitMQ for four years.

We did have stability issues in the past. After shutting it down, the cluster did not start until we deleted some corrupted file. This occurred more than a year ago.

It works as expected, i.e., flawless.

We have not needed any technical support as of yet.

We did not evaluate any previous solutions.

Just enter this command: $ apt-get install rabbitmq-server

It’s open source with paid support.

We looked at Kafka, but we needed the routing as well.

Start it in Docker and use Java Spring Boot or Node.JS with amqplib to connect to it. It has transformed how I think data should flow in an organization.

• The publisher and consumer ACK
• High availability
• Queue mirroring
• Exchanges and topics
• Supported programming languages with well-tested libraries

It provides us with a much better scale. We have never lost a single message with RabbitMQ.

The shared RabbitMQ Cluster has improved stability and maintainability of each application. We only have one message bus now.

I want it to reorder messages in a queue, if possible. If you could reorder messages in a queue directly, then you would not need a sequencer to reorder messages outside of RabbitMQ.

I have used this solution for seven years.

There were no stability issues.

There were no scalability issues.

We haven’t needed to use any support yet.

Initially, we were using the BizTalk and Redis solutions. The reason why we switched over was because we were looking for better support in Celery task management and other programming languages. We were looking for a much more stable and secure solution.

It is very simple to set up for basic usage. Clustering is a bit more complex, but it is also easy to do.

The RabbitMQ open source version works fine for almost all the use cases that I came across.

We looked at the Redis solution, but it was not a good fit for our needs.

Read the documentation and follow best practices. Make sure Erlang is up to date!

The most valuable feature of RabbitMQ is the ability to set up workflows simply with configuration. We had some very complex problems (logging, auditing, sequential and parallel operations) that have been easily solved by inserting a queue in the middle of an existing workflow.

Our software has evolved dramatically over the past 18 months of development.

Major modifications to business logic have been handled easily. This is because each operation that the software performs has been atomic. That was then wired up with other operations via RabbitMQ exchanges/queues,

• RabbitMQ is great, but it depends on the Erlang VM.
• I understand that Erlang is the reason why RabbitMQ is what it is. However, having to install and maintain yet another VM product has been annoying.
• The configuration for RabbitMQ borders on the esoteric. Once we got all of the moving parts working, it’s been a dream. However, it was an effort just to get it going.

We’ve been developing with RabbitMQ for about 18 months now. Our product launch is scheduled for April 1, 2017.

There were stability issues. We originally tried to mirror RabbitMQ servers behind a load balancer. (This is not completely recommended, by the way.)

That suffered from stability issues when network hiccups were a problem.

We ended up moving to a central LDAP authentication with completely disconnected servers, which has been stellar for stability.

We have had no scalability issues at all. We have RabbitMQ servers behind a load balancer, with Queue listeners attached to them.

I have estimated that we will be able to scale quite dramatically without any change to network topology.

RabbitMQ is an open-source project, so the level of tech support is relatively low. The documentation is adequate. There is a decent amount of activity on Stack Overflow and that has answered most of my questions.

I did actually use an open-source plugin and had a lot of communication with the developer. He was responsive and helped me significantly.

Our initial solution was a simple web API. We quickly realized that it would be difficult to maintain.

Scalability and reliability were my primary concerns. HTTP has no reliable delivery built-in, like AMQP does. Changing API versions would have been more difficult if we had stuck with HTTP.

The initial setup was fairly complex. Installation was not difficult, but configuration of the server itself took some work.

I ended up creating code that does the RabbitMQ setup based on a configuration file. This eased the setup dramatically. I also set up a central LDAP server for authentication, which helped. Otherwise, configuration of RabbitMQ was not very straightforward.

Being free and open-source, I have no advice here! Free is a good price!

I looked at other service bus/message queue solutions. In particular, I investigated:

• Azure’s Queue Storage: No real service bus ability without plugins
• AWS’s Simple Notification Service: Not much in the way of service bus capability. It did not allow private queues on the fly.
• A few other open-source message brokers

RabbitMQ seemed the most full-featured option for what we needed.

By all means, try to reduce the amount of up-front configuration of RabbitMQ as much as possible.

At this point, we can spin up a very generic VM with RabbitMQ on it and get it in use immediately. However, that was not the case at the beginning.

RabbitMQ is very flexible, which is good and bad. Once the flexibility is understood, it’s great. Before that, you may be in for a little bit of head-scratching.

• AMQP protocol
• The simplicity to set up a cluster
• All patterns available (Topic, Routing, RPC, etc.)
• Enables you to provide a response to most problems encountered

I introduced RabbitMQ to my company to bring about more scalability and redirect asynchronous tasks outside of the frontal web servers.

• Scalability was provided by the possibility to add on workers behind RabbitMQ. Asynchronous is the keyword for the message broker.
• I started to split our monolith website and remove all the heavy processes from our web servers.
• Dispatching the workload correctly helped to increase the response time.
• The frontal web server must focus on replying and not on doing reports/PDFs. You must have a dedicated server for this.
• The solution is written in Erlang. Erlang was built for telecommunication systems. One of its assets is that it can upgrade a service in production without downtime. That’s a good point!

• Have more features such as being able to replay a sequence of what was received.
• Handle more messages per second.
• Consume fewer resources: NATS can handle millions of requests within a few minutes. RabbitMQ handles hundreds of requests with the same resources (RAM). Finding a way to be more efficient in this aspect would open them up to other markets, like IoT or embedded systems.

I have used this solution for around two years.

There were no stability issues.

There were no scalability issues.

The documentation from the internet is good.

The setup was pretty straightforward.

I have looked at Apache ActiveMQ and Kafka.

This product is fantastic. Before you use it, make sure you analyze your real needs. In the world of message brokers, there is no single solution. Today, you have solutions that are specialized for specific use cases.

For instance, you have the NATS solution which is another great message broker tool. It is focused on handling a tremendous number of topics and messages per second and it consumes few resources.

However, it’s not as resilient as Kafka. Kafka can replay all the messages received from a date range. However, this one can handle fewer topics.

NATS is written in Golang. Golang is the Go programming language, which I love.

The MPP element is crucial, so far as it allows us to query millions of rows at a time, at speed.

The previous data warehouse was built in Oracle. One of the things which has improved in GreenPlum is that we can query millions of rows at speed, without creating lags. We’ve also built far more views; slowly changing dimensions can instantaneously update without creating the issue of having to rebuild tables to reflect new hierarchies, for example.

We found some issues with larger tables that have daily data appended, where after a while this seems to create lag in the query speed. This might just have to do with local knowledge rather than the product itself.

We have a table which is currently contains 27.6m rows and has a daily delta added to it of roughly 16.5k rows per day. While this isn’t particularly large, we have noticed the table begins to perform poorly when queried, in spite of having set up a VACUUM process to be performed weekly. It may be that the VACUUM process needs to be performed more frequently (like daily), but we’ve not yet found the optimal way of maintaining this particular table.

It’s worth saying that this is one table out of over 400 perfectly well performant tables and views in the same database. Hope that helps,

I have used for approximately 30 months.

I have not encountered any deployment, stability or scalability issues.

I have not raised any service issues/tech queries, so I can’t really say.

We used Oracle previously. We based our choice on expertise in our US operation, where we have a GreenPlum expert who provided some amazing use case examples to help us in our selection process.

Implementation was done in-house.

Not within my area I’m afraid, but I understand that this was a very good fit from an ROI point of view

Investigate whether this solution works for you. It is worth creating a rating matrix to compare other similar products, and it is very useful to look deeply at whether the third-generation MPP software might be a good fit.

Some valuable features of this product are:

• Message queuing
• Good support
• Provided scalability and a distributed environment
• Easy usability with NodeJS
• Could easily withstand and pass stress/load testing with more than 10K API calls
• Helped achieve a distributed environment and implement a 3-tier structure
• Helped to develop a highly scalable system by decoupling front end and back end

We had a project where we had huge responses to APIs from the front-end and had to handle such large responses/requests without losing any of them. RabbitMQ efficiently handled this problem by providing message queuing and decoupling our front end and back end.

I would love to see better documentation/demo for few technologies. There is need for better stability in the Windows environment.

I have used this product for around six months.

We encountered a few problems with Windows while clustering and hence we used Linux.

We did not encounter any scalability issues.

I would give the technical support a 7/10 rating.

It involved more of research as how to use RabbitMQ. For clustering, it was little bit complex but I was able to follow the documentation provided.

We did not evaluate any other solution prior to this one.

It gives product support with your technology.

It has solved issues such as queue processing, real-time data processing and decrease count of Cron jobs.

The management of Cron jobs is very hard for me. If you are using RabbitMQ, you may not need a Cron job since you can create a consumer for a queue and listen to the queue in real-time. For example, you need to send an email for your new members but you want to do this from your software. The reason being if you encounter an SMTP error, you should handle it or sending an email from your membership activity can take about 5-10 seconds. If you don't use RabbitMQ, you need to create a Cron job and check for new members to send an email to them. But, if you use RabbitMQ, you do not need a Cron job. You can send a message to your queue and consume it in real-time and then send the email.

I have used this solution for changing stock of products in an e-commerce project. We update the stock of products from RabbitMQ. Some of our clients can change the stock of a product, i.e., either increase or decrease it. They send the stock count to RabbitMQ as a message and our e-commerce platform changes the stock when it gets the message. Consequently, we update the database, cache, search engine service and so on.

There are too many advantages of using this solution. You can scale your processes and update your data in a nearly synchronous way.

RabbitMQ needs 2 additional features:

• It is lacking a good dashboard on the web interface; maybe they can develop a dashboard for monitoring.
• There is no alert mechanism. For example, sometimes consumers may be killed or the input messages in queues are greater than the consumed messages. Thus, I would like them to define a rule for alert; maybe they can develop an alert mechanism.

I have used this product for three or four years.

I have not encountered any stability issues.

I have not encountered any scalability issues either.

Since it is open source, you can resolve your problems either from forums or Stack Overflow in regards to RabbitMQ.

We have used different solutions before. We have tried MySQL as a queue and it was terrible.

It is not complex.

However, if you try sharding or cluster, it is just a little complex because of Erlang.

It is open source.

I did not look into other solutions.

I would recommend this product. It is great and resolves a lot of problems.

RabbitMQ is good in its real-time capability of delivering the sensor data in different topic abstractions. It is quite easy to setup in our sensor network system which involves data monitor agents across geographically distributed organizations.

This tool made our research process easier and allowed our administration (monitoring) process to happen at the same time. We created different queues for different use cases of the same streaming data.

The solution can be improved in terms of how to handle the rolled off data from the queue. Currently, if the consumer does not consume a queue, the data in the queue will eventually overflow and be discarded.

I have used this solution for about a year.

Stability-wise, the consumer needs to configure with caution to make sure the queues never overflow. I have experienced data loss in scenarios in which consumer cases were configured incorrectly.

It would be nice if it could produce automatic load balancing when the number of brokers scales up.

The technical support is generally good in terms of being a nice and active user forum that has good documentation.

The initial setup is not very complex. We just needed to set up several broker instances running on different physical nodes to prevent unavailability. When we needed to add more nodes for more data and queues at a later time, it created a bit of load imbalance issues.

I would suggest that the vendor keep it the same way that it is now.

We chose this solution around 2010, so there weren’t many options at the time. We evaluated the native JMS method and it didn't scale up well.

Keep in mind that the queue structure and the consumer configuration need to keep up with the queuing speed. I would suggest researching other options if you need bulk data publish/subscribe in persistent data storage, e.g., Apache Kafka.