Please share with the community what you think needs improvement with Apache NiFi.
What are its weaknesses? What would you like to see changed in a future version?
The challenge with Apache NiFi is that it's not cloud-native. This makes it different from our workflow. The operations are over-complicated and when you build your pipeline, it's a nightmare to follow them. Then, as your pipeline or workflow becomes more complex, the operation of it gets worse. It is not easy to use and it requires a bigger ramp-up than any other solution that we have seen.
There should be a better way to integrate a development environment with local tools. Most of the development is done on the console. For example, in Spark, we can develop on our local desktop and then deploy it to another environment. The integration using this solution is not simple. Eventually, we can implement a local environment on our machines that is web-oriented and we have a browsing console to do it. At times, it is not easy to integrate with other components that could be a part of the entire solution in terms of development. There are issues with stability due to memory. It would be good to include a lock or an alarm to detect or alert you that it needs more resources. If the solution could be integrated more it would repair every part of the flow. Eventually, if we are integrated with other technology, we need to have a notion that we need to plan and have the correct sizing. We could implement an ecosystem that could scale with the requirements. In the next release, I would like to see the support of monitorization with the interface. Also an integrated development environment.
We all know it's really hard to get good pricing and cost information.
Please share what you can so you can help your peers.