Azure Data Factory Reviews

Our primary use case is for traditional ETL; moving data from the web and data sources to a data warehouse. I've also used Data Factory for batch processing and recently for streaming data sets. We have a partnership with Microsoft and I am a cloud solution architect.

There are a lot of innovative features that Microsoft releases regularly. The UI is easy to navigate and finding information on new features has proven to be quite easy. I like that I can retrieve VTL code pretty quickly without knowing in-depth coding languages like Python. Microsoft has very good support teams that I've dealt with and they are very helpful in resolving problems.

Improvement could be made around streaming data because I feel that the Data Factory product is mainly geared for batch processing and doesn't yet have in-built streaming data processing. Perhaps it's on the way and they are making some changes to help facilitate that. If they were to include better monitoring that would be useful. I'd like to see improved notifications of what the actual errors are.

I've been using this solution for four years. 

The solution is stable. 

The way we've used the product is around streaming data, and that doesn't work well with Data Factory. As loads increase, some of the underlying infrastructure that gets used to process data seems to slow down. This is basically a development product, so in terms of scalability it doesn't have a wide user base, it's only meant for developers and analysts. The number of users will vary from anywhere between one to five people. 

The customer service is excellent. 

I've previously used Microsoft SSIS, which is the last incarnation of Azure Data Factory. I've also used IBM Data Manager and Teradata's Management Loads, so I have quite a bit of experience in this area. One of the major differences is that Azure Data Factory is a SaaS service. The other solutions were in-house and you managed your own infrastructure to run them. What I get from Data Factory is a lot better than what I got for the other products. Azure Data Factory is great because it's evolving day-to-day.

The initial setup is relatively straightforward. That's assuming that when you're creating the Data Factory, you have some knowledge about how to create it. We deployed in-house and it took about ten minutes for the initial creation process. The provisioning itself is not something you have control over because it's a self-service that does what it needs and happens in the background. The infrastructure doesn't require any maintenance, it's all managed on the backend. There is maintenance in terms of your codes, connections, and the like, but that is separate from the infrastructure maintenance.

The Data Factory itself will not give you a return on investment, it's the entire solution that brings a return, I'd say at least 20% to 30% of return for investment over a five-year period. 

Azure Data factory is a pay-as-you-go service so cost depends on the number of connections, how many times each activity in that node is run, and how much data gets moved. There are a number of factors that define the price, but it pairs with your service. You're charged for the amount of data that's moved, but there are no charges for the features you use.

I would definitely recommend this solution. If you decide to implement Data Factory, I suggest reaching out to qualified professionals because there are a lot of moving parts. That said, if you have internally qualified staff, deployment shouldn't be a problem. Apart from a few minor issues, it's pretty reliable with good support and a whole bunch of resources available on the web.

I rate this solution a solid nine out of 10. 

We primarily use the solution in a data engineering context for bringing data from source to sink.

The solution is very comfortable to use. I'm happy with the user interface and dashboards. I'm pretty happy with everything about the solution. 

We haven't had any issues connecting it to other products.

It's a stable product. 

I have not found any real shortcomings within the product.

I've been using the solution for the past year. 

The product has been very stable and reliable. I'd rate the stability nine out of ten. There are no bugs or glitches. It doesn't crash or freeze. 

There is a team of 30 people working on the solution. 

I've connected with technical support a few times. 

They sent a support engineer or a field engineer to us, and he helped us out. 

Positive

I'm not sure about the exact cost of the solution. 

I'm a customer and end-user.

Our company chose to use this solution based on the fact that it is a Microsoft product. We're using a lot of solutions, including Outlook and Teams. We also use Power BI. We try to use Microsoft so that everything is under one umbrella. That way, there is no difficulty with connecting anything. 

It's a good solution to use. There are lots of videos available on YouTube, and it is very easy to learn. It's very easy to perform things on it as well, which is one thing that a product like ThoughtSpot lacks. There is no training needed like Power BI. 

I'd rate the solution nine out of ten.

Our company uses the solution for data ingestion. 

It is beneficial that the solution is written with Spark as the back end. 

The solution is cloud-based and integrates well with other Azure products such as Synapse Analytics.  

There are limitations when processing more than one GD file. 

Data ingestion pipelines sometimes fail because of transient issues that have to do with the cloud network. It takes more than six hours to process or ingest 300,000 records and that is a long time. 

I have been using the solution for two years. 

The solution is new in the market and pretty stable because ADF is a little more codified than AWS. Synapse Analytics adds another tool for data. 

Stability is not quite at the level of Informatica or DataStage. 

The solution is scalable. 

For multi-tenant applications connected to multiple databases, Microsoft recommends a share box and a cell post integration run time. But a run time connecting to multiple sources has limitations and requires multiple shares connecting to your data if you are ingesting it from on-premises. 

Technical support is okay. Support is contracted or partnered with various companies but is fine as a first level. 

Most of the time, technical support has to connect with product engineers who troubleshoot issues. 

The setup is not very complex but requires intake, setting up integration services, and connecting to databases like Oracle before you push it to service.

We implemented the solution in-house. 

The pricing is pay-as-you-go or reserve instance. Of the two options, reserve instance is much cheaper. 

In the cloud, everything is service based and expensive. Users should be knowledgeable enough to maximize the solution. 

For example, it makes no sense to run integration services all day if you are not ingesting data because you pay for that usage. It is important to understand how the product works to manage it accordingly and keep costs down. 

I rate the solution a six out of ten. 

I primarily use Data Factory for data ingestion and B2B transformation.

Data Factory's best features are connectivity with different tools and focusing data ingestion using pipeline copy data.

Data Factory's monitorability could be better. In the next release, Data Factory should include integrations with open-source tools like Air Flow.

I've been working with Data Factory for about a year.

Data Factory is stable.

Data Factory is scalable.

Microsoft's technical support is good, so long as your company has a good relationship with them.

I previously worked with Talend, Matillion, and Fivetran.

Data Factory is expensive.

I would rate Data Factory seven out of ten.

We mainly use this solution to carry out data movement and transformation.

This solution has provided us with an easier, and more efficient way to carry out data migration tasks.

This solution is currently only useful for basic data movement and file extractions, which we would like to see developed to handle more complex data transformations.

We have been using this solution for a year.

We have found this to be a stable solution.

This is an easily scalable product, due to it being cloud-based.

The customer support for this solution is very good. 

The initial setup of this product is straightforward, if you deploy the solution using a template; rather than implementing the solution first, and configuring the features afterwards.

We implemented the product using both in-house staff, and members of a vendor team. The vendor team were very helpful, and gave good advice while we were deploying the solution.

We would recommend this solution as it is very solid and has good security features.

I would rate this solution an eight out of 10.

We use Data Factory for automating ETL processes, data management, digital transformation, and scheduled automated processes. My team has about 11 people, and at least five use Data Factory. It's mostly data engineers and analysts. 

Each data analyst and engineer manages a few projects for clients. Typically, it's one person per client, but we might have two or three people managing and building out pipelines for a larger project.  

It's cloud-based, allowing multiple users to easily access the solution from the office or remote locations. I like that we can set up the security protocols for IP addresses, like allow lists. It's a pretty user-friendly product as well. The interface and build environment where you create pipelines are easy to use. It's straightforward to manage the digital transformation pipelines we build.

Data Factory has so many features that it can be a little difficult or confusing to find some settings and configurations. I'm sure there's a way to make it a little easier to navigate.

In the main ADF web portal could, there's a section for monitoring jobs that are currently running so you can see if recent jobs have failed. There's an app for working with Azure in general where you can look at some segs in your account. It would be nice if Azure had an app that lets you access the monitoring layer of Data Factory from your phone or a tablet, so you could do a quick check-in on the status of certain jobs. That could be useful.

We've been using Azure Data Factory for about three years.

I've been happy with it overall. I don't think we've had any major issues. We've been able to do what we needed, whether connecting to different data sources or setting up different types of transformations and processes. 

It's a cloud solution, so it's inherently scalable. I don't know If we have to raise the limits on resources like clusters and processing power or if it will just automatically scale up. I can't remember offhand. 

We managed the same actions with a combination of tools. We used SFTP servers to move data from one place to another. We used scripts for loading and some other stored procedures or processes for data transformation within a database. It took two or three pieces of technology or systems to manage the same types of operation. Data Factory lets us consolidate those steps into a single pipeline. 

Setting up Azure Data Factory is pretty straightforward. We had an Azure account already, and Data Factory was just something we could add as an extra service. We had to create instances and pipelines, and it took us about two weeks to get our first pipelines scheduled and running. 

We do everything in-house.

We see a return on Data Factory if we compare the time and effort that would be necessary to perform the equivalent processes manually. 

I'm not too familiar with the cost, but I believe we're reasonably happy with what we're paying. My understanding is that the cost of Data Factory is tied to consumption. It depends on the amount of data or the number of pipelines running, and the cost varies from month to month depending on the usage. 

You'll obviously pay more if you're scheduling heavy digital transformation processes to run every hour, but I don't think there are any other hidden costs or anything extra. When you set up a new account, you have a trial period that enables you to create a test pipeline or process that's typical of your use case and then do a benchmark test to see if Data Factory can achieve the efficiency you need. You'll also get some idea of how much the process will cost to run. From there, it's straightforward to do a cost evaluation or comparison to see if it's the right fit for your company. 

We were looking for a single solution, and Data Factory was the first one that interested us. I don't think we looked at many others. We were pretty set on Azure, and Data Factory seemed to fit our needs, so we didn't make a full comparison with the alternatives.

I rate Azure Data Factory nine out of 10. It isn't perfect, but it's solid. Data Factory has improved how we deal with various aspects of Azure. It has always met our needs in terms of the transformations and jobs we want to create and schedule. 

Azure Data Factory is an integration tool, an orchestration service tool. It’s for data integration for the cloud.

The solution is okay.

Some stuff can be better, however, overall it's fine.

The performance and stability are touch and go.

The deployment should be easier.

We’d like the management of the solution to run a little more smoothly.

I’ve used the solution for three to five years.

The solution could be more stable. It’s touch and go. It’s not 100%.

For Azure Data Factory, scalability doesn't mean really too much. However, in some scenarios, you can play with it a little bit.

Azure Data Factory is not for users. Is for engineers, for developers. The end user does not interact with Azure Data Factory. There might be 20 developers on the solution currently.

I don't remember a particular scenario right now where I reached out to support. However, when you work with Azure Services, here and there, you might get into some challenges, and maybe sometimes you reach out to Microsoft. That said, I don't remember a particular scenario right now.

It’s hard to describe the installation. It’s not overly complex or extremely easy.

The point is almost true for all services. If you want to do something simple and quick, then it's just a couple of clicks, and it's there. However, in real production environments, it's not like that. You have to arrange a lot of things. You have to set up a lot of things. You have to configure a lot of things correctly in an automated way. That is totally different than just a couple of clicks. You have to put in the work. If you ask me how easy it is, yeah, it is easy. However, it can also be really, really complicated depending on the scenario.

As far as I know, there isn’t any licensing per se for this solution.

I’d rate the solution eight out of ten overall.

It's a PaaS service. It's a hybrid solution. The cloud provider is Microsoft.

We are not using Azure Data Factory as for users. Rather, we're using it as a process base. We're just using it for orchestration, not for any kind of ETL stuff.

We have plans to increase usage. It's going to take a major role in any kind of traditional data warehousing. It has big potential, especially as a PaaS offering.

There has been improvement in data resilience, in the way that we're moving the data from on-prem to cloud and vice versa.

The most important feature is that it can help you do the multi-threading concepts. It's in Informatica, but the resourcing is quite robust. You can scale up and scale down as per your needs.

There should be a way that it can do switches, so if at any point in time I want to do some hybrid mode of making any data collections or ingestions, I can just click on a button. I can change a switch and make sure a batch can be a streaming process.

I've been using Azure Data Factory for more than two years.

The stability of Azure as a PaaS could be improved.

It's scalable.

I would rate their technical support 3 out of 5. It's not great, but it isn't bad.

The setup is complex. It has nothing to do with the technology but with the design. We were wondering how to leverage the orchestration layer where we are having the Azure Data Factory and how to integrate with the Databricks. That's where we had some challenges in terms of choosing the right product.

You can do deployment in-house. 

I would rate this solution 8 out of 10.

For someone who is looking to use this solution, my advice is to do proper due diligence of your current application, know where your application is fitting, and look for the requirements. It all depends upon the current use case that you have currently in your system.