Hi all,
I have a potentially complex many-to-many ETL (Extract, Transform, Load) project.
This is going to take data from MQTT, aggregate it with relatively static data, gather dynamic data from another source (via SNMP) and correlate the output.
The output will be sent to at least 3 destinations: an S3 bucket, a MySQL Database (via SQL commands) and another application (using the API that's available in that application).
Data volumes are potentially high - especially from up to 100,000 MQTT devices, each sending multiple message types.
What is the experience of others in using Node-RED to achieve this kind of thing? Is it performant? What are the gotcha's? What's the learning curve?
I don't want to waste time on what - visually at least - looks like a great solution without seeking the opinion of others.
Thoughts?
Hi @David Collier, a partial answer has been provided by Arthur Alexander Burger (via DM on LI), as follows:
"It’s completely capable, but the number of nodes is the only concern. He had not stated anything regarding messages per second, etc. to know whether the underlying platform would be fine."
@David Collier Sure, You're welcome and I will.
I hope more answers will be added soon by other peers.
Please feel free to modify your original question, once you have any additional information.