What is our primary use case?
Our company uses the solution as a data pipeline. We get information outside the cloud from our factory such as data relating to production. We categorize it, clean it up, and transfer it to a database and data model. From there, we analyze the data using BI and other things. We gather information in data lake products like Microsoft Synapse and Microsoft Data Lake.
We have two to three administrators who use the solution in a quite standard, mainstream way with nothing extreme. They handle administration, security, and development.
It is difficult to define the total number of users because that depends on the number of data factory agents. We built the solution to have a different data factory agent for every customer. For example, if we have ten customers then we have ten users. We hope to increase usage but growth depends on our marketing efforts and how well we sell our products.
What is most valuable?
The data factory agent is quite good and programming or defining the value of jobs, processes, and activities is easy. We have the agent installed on-premises in order to gather information.
The cloud includes all kinds of API connections so we can easily gather information from other services.
The solution seamlessly integrates with the Azure infrastructure.
What needs improvement?
The pricing model should be more transparent and available online. When you start programming, you define the fields, variables, activities, and components but don't know the implication on price. You get a general idea but the more activities you add, the more you pay. It would be better to know price implications up front.
There is a calculator you can run to simulate price but it doesn't help a lot. Practically speaking, you have to build your job and run it to see the exact price implications. This is an issue because you might realize you are paying too much so you have to reprogram or change things.
For how long have I used the solution?
I have been using the solution for three years.
What do I think about the stability of the solution?
The solution is stable with no issues. Stability is rated a nine out of ten.
We did have some breaches, but that was because we misconfigured something. Since we corrected it, we haven't had any issues.
What do I think about the scalability of the solution?
The solution is scalable with no performance issues. We haven't yet reached our limit that would require scaling. Scalability is rated an eight out of ten.
How are customer service and support?
We have discussions with our Microsoft partner all the time.
In the last three years, we have contacted Microsoft directly three or four times. Once was for a general architectural issue and the rest were for the data factory or other items. Each time, we talked together with Microsoft and our partner.
Support gave us answers and solved our issues. Sometimes, we didn't like the answer but we accepted that it was the correct answer.
Support is rated a nine out of ten.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We have not used another solution to this magnitude for real development and production. We work a little bit on Google Cloud.
How was the initial setup?
The initial setup was quite quick. Deployment was fairly simple and took less than a week. The setup got us up and running.
After that, we had to write the implications of the data model and the kinds of activities. We are still doing this today because we make changes all the time.
What about the implementation team?
The initial setup was seamless because we worked with a gold star Microsoft partner. Our side of setup was pretty quiet. We talked with our partner and told them what we needed from a security and monitoring point of view. We had a few high-level discussions from the block diagram perspective. Basically we said we need this or that, and our partner made it happen.
The team included one person from our partner and three in-house team members with varying expertise across data modeling, security, and devops. We always worked with the same person but maybe behind the scenes he talked with coworkers. He did talk several times with Microsoft but we don't really know how many people were involved.
The solution does not require infrastructure maintenance. If we ever have issues, we can use Azure Defender to resolve them. We only make slight changes at the application level.
What was our ROI?
We haven't calculated ROI on a formal level, but the fact is we need the solution. Because of the integration, we save a lot but haven't run exact numbers.
What's my experience with pricing, setup cost, and licensing?
The pricing model is based on usage and is not cheap. Based on our activity, we pay about $2,000 per month.
Pricing is rated a four out of ten.
Which other solutions did I evaluate?
If we didn't have the solution, we would have to find another tool because data pipelines are an essential part of our business.
The biggest advantage to the solution is its integration with the Azure infrastructure that includes the active directory, security, Synapse, Data Lake, Power BI, and the data factory agent.
All of the integration was a big consideration for us. We had general guidelines that said working with one vendor would provide the best integrations. The guideline was to use Microsoft unless there was an issue.
We did not look at a third party or open source even though there are similar tools available.
What other advice do I have?
My best advice is to keep an eye on the pricing because we found out the hard way. Pricing is actually related to the way you use what the solution calls activity. This activity stuff drastically changes the coding to the rate you gather information from your client environment.
So, when marketing guys tell you to gather information every minute, you have to weigh the heavy implications in comparison to collecting data once an hour or day. Programmers and developers designed the solution based on usage activity and building tasks or jobs.
Pay a lot of attention to the pricing implications from the starting point of view. Technically, you can solve all issues but you need to keep an eye on the pricing.
From a technical point of view, the solution is rated an eight out of ten. Because of pricing, the solution's overall rating is downgraded to a seven out of ten.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Microsoft Azure
Disclosure: My company does not have a business relationship with this vendor other than being a customer.