Azure Data Factory Reviews

Our company uses the solution to extract, transform, and load the data into the database.

Feature-wise, one of the most valuable ones is the data flows introduced recently in the solution.

Currently, our company requires a monitoring tool, and that isn't available in Azure Data Factory. Although the platform displays which pipelines are running, it doesn't offer a monitoring tool that allows for the sequential execution of pipelines and the ability to visualize end-to-end data flow. As such, this feature is currently missing from the platform.

I have been using Azure Data Factory for more than six years. Also, I am an end-user of the solution, and I initially used to work on Azure Data Factory V1. Now, I have switched to Azure Data Factory V2.

It is a stable solution. Stability-wise, I rate the solution an eight or nine out of ten.

Scalability-wise, I rate the solution a seven or eight out of ten. So, scalability can be improved. Also, there are around 150 people in my company using the solution. Moreover, we use the solution daily in our company.

I rate the technical support between eight to nine out of ten.

Positive

Previously, I was using Informatica. My company wanted to shift to a solution that could be deployed on the cloud, so we chose Azure Data Factory.

The solution's initial setup process was easy. On a scale where one is difficult and ten is easy, I rate the initial setup process an eight out of ten. The solution is deployed on the cloud.

Since multiple projects are going on in my organization, there is no uniformity in the time taken to deploy the solution in our company. However, I can say that it only takes a few days to carry out the deployment process.

Our organization has multiple project teams, so each team carries out its deployment process.

To give an average, I would consider that if there are fifty ongoing projects in our company, and if we consider a person from each project, fifty people are needed for the deployment and maintenance process.

The solution's implementation process was done with our in-house team's help.

I cannot comment on the pricing parts since our company's admin team handles it.

Those who want to move to a cloud platform can choose Azure Data Factory since it is the best tool. Since certain improvements are required in the solution, I rate the overall solution an eight out of ten.

I primarily use Data Factory to ingest data. For example, if we need to pull data into our data warehouse from somewhere like Azure Event Hub or salesforce.com.

We have telemetry that streams into an Azure Event Hub, and Data Factory allowed us to move that data from the Event Hub into our data lake and reduce the cost of that compared to the other tooling we were using.

Data Factory's best features are simplicity and flexibility. It's been very easy to set up connections to different types of data sources to pull data into our warehouse.

Data Factory would be improved if it were a little more configuration-oriented and not so code-oriented and if it had more automated features.

I've been using Data Factory for about three years.

I would rate Data Factory's stability eight out of ten.

I would rate Data Factory's scalability eight out of ten.

The initial setup was straightforward, and only one person was required for deployment.

I would rate Data Factory's pricing nine out of ten.

I think Data Factory is a good fit when you need a light, inexpensive way to ingest data. I would rate it eight out of ten.

We use this solution to quickly instantiate certain components in Azure with the aim of having consistent use of certain components and objects. It provides one point of reference rather than having the need to replicate points of references, and then having to keep them in sync. 

It has helped our company by providing consistency. For example, by making sure that the way certain objects and components are defined is consistent throughout every step. If things change at any point, they should be reflected at all points. It's the consistency that makes implementations less buggy and less complex.

The function is the central point of reference and the most valuable thing about Data Factory.

I'd like to see videos on YouTube or the Microsoft site with more detailed implementations. The solution lacks a decent UI that allows us to see what kinds of requests are for what objects and how the population of objects is being requested and compared. Right now we have to look at logs to get an idea of what types of calls the data factory receives in what sequence, for example. It would be nice to be able to see it graphically because we currently have to interpret the logs and then create a graphical representation to have an idea of what's going on. In general, it could be simplified and made more user-friendly.

I've been using this solution for six months.

This solution is stable. 

There are thousands of users so the solution is scalable. 

The customer service is excellent. 

Positive

I previously used a combination of solutions to achieve the same end. Data Factory simplifies things which is why we switched to it. 

The initial setup was complex. The deployment was carried out in-house and we had around 10 people involved in the implementation. 

In terms of time and effort savings, we have a return on our investment. 

It's important to have a good data model before you start using the solution; an idea of the types of data architecture, data objects and components in order to use them. Because of the lack of more user-friendly interfaces, especially for the people debugging the system, I rate this solution eight out of 10.

We primarily use the solution in a data engineering context for bringing data from source to sink.

The solution is very comfortable to use. I'm happy with the user interface and dashboards. I'm pretty happy with everything about the solution. 

We haven't had any issues connecting it to other products.

It's a stable product. 

I have not found any real shortcomings within the product.

I've been using the solution for the past year. 

The product has been very stable and reliable. I'd rate the stability nine out of ten. There are no bugs or glitches. It doesn't crash or freeze. 

There is a team of 30 people working on the solution. 

I've connected with technical support a few times. 

They sent a support engineer or a field engineer to us, and he helped us out. 

Positive

I'm not sure about the exact cost of the solution. 

I'm a customer and end-user.

Our company chose to use this solution based on the fact that it is a Microsoft product. We're using a lot of solutions, including Outlook and Teams. We also use Power BI. We try to use Microsoft so that everything is under one umbrella. That way, there is no difficulty with connecting anything. 

It's a good solution to use. There are lots of videos available on YouTube, and it is very easy to learn. It's very easy to perform things on it as well, which is one thing that a product like ThoughtSpot lacks. There is no training needed like Power BI. 

I'd rate the solution nine out of ten.

I can't go into specifics about the use case for Azure Data Factory, but it's for analytics related to an assessment.

What I like best about Azure Data Factory is that it allows you to create pipelines, specifically ETL pipelines.

I also like that Azure Data Factory has connectors and solves most of my company's problems. I can't recall a case where I couldn't use the solution for solving problems.

I'm also happy about the Azure Data Factory dashboard.

A room for improvement in Azure Data Factory is its speed. Parallelization also needs improvement. As for the rest of the features of Azure Data Factory, I'm happy.

I cannot suggest an additional feature I'd like to see in Azure Data Factory in the future because some of the features aren't available internally because the features undergo security evaluation first, and my organization controls which features would become available to users.

I've been using Azure Data Factory for the last two years.

We're happy with the stability of Azure Data Factory.

Azure Data Factory is scalable, with clusters available on demand. There isn't any issue with scaling the solution.

We have an internal support team and the Azure Data Factory support team. We raise tickets and follow up on those tickets, and on a scale of one to five, we'd rate support as four because sometimes there are delays. Otherwise, we are satisfied with Azure Data Factory support.

My company didn't set up Azure Data Factory as the Azure team did it.

We outsourced the implementation of Azure Data Factory directly to the Azure team.

I have no idea how much Azure Data Factory costs.

We're using AWS apart from Azure Data Factory. We're trying out Palantir Foundry as well. They are the leading service providers in the data analytics and ETL world.

I'm familiar with Palantir Foundry, but my company just recently got the Palantir Foundry license, so I'm still not using it, but checking it for shortcomings.

I have experience with Azure Data Factory, too.

I'm unsure of the exact version of Azure Data Factory, but I'm using the latest version or whatever's available on Azure.

I have a vague figure of users of Azure Data Factory, but it's more than one thousand to one thousand five hundred people.

I'd tell people who want to use Azure Data Factory that Microsoft offers excellent courses, ESI (Enterprise Skill Initiatives). You should register and take the courses. Azure Data Factory is a solution I'd recommend to others.

I'd rate Azure Data Factory as nine out of ten because it has a lot of connectors, even custom connectors, for data onboarding. It can also integrate with Spark notebooks and allows my organization to parallelize code. Azure Data Factory also has provisions for Spark and SQL scripts or any scripts, plus the infrastructure is highly scalable, so it's a nine for me.

My organization is a customer of Azure Data Factory.

Our primary use case for the solution is data integration and we deploy it only on Azure.

When we were integrating the Ports product with our internal data warehouse, we had to update all the reports to our internal data warehouse on the Ports system database. However, they were not given access to the database company, and they dump some files or provide you with them. In one case, they were providing files. In another case, they provided some APIs where you need to call in a batch of thousands of records multiple times. It works very well with Azure Data Factory to pull the records, parse them quickly and post them in the database and data warehouse.

We have found the bulk load feature very valuable.

The only challenge with Azure Data Factory is its exception-handling mechanism. When the record fails, it's tough to identify and log.

We have been using the solution for a year and a half and are currently using the latest version.

The solution is scalable and we intend to further increase its usage in the future.

I rate customer service and support an eight out of ten.

Positive

We previously used different solutions.

The initial setup is straightforward.

The implementation was done in-house.

I cannot comment on licensing costs because I was not involved.

I rate the solution a six out of ten. The solution is good but its exception handling and logging mechanisms can be improved. I advice users considering this solution to go for it especially if their integrations are heavy on the data side.

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

Azure Data Factory didn't bring a lot of good when we were also using Alteryx. Alteryx is user-friendly, while Azure Data Factory uses many resources and has issues with parallel workflows. Alteryx helps you diagnose issues quicker than Azure Data Factory because it's on the cloud and has a cold start debugger.

Azure Data Factory has to wake up whenever you are trying to do testing, and it takes about four to five minutes. It's not always online to do a quick test. For example, if we want to test an Excel file to see if the formatting is correct or why the data-flow or pipeline is failing, we need to wait four to five minutes to get the cold start debugger to run. Compared to Alteryx, Azure Data Factory could be better. Nevertheless, we are using it because we have to.

Initially, when we started using it, we didn't like it because it needed to be more mature and had data-flows, so we used the traditional pipeline. After that, Azure Data Factory introduced the concept of data-flows, and it started to become more mature and look more like Alteryx. Azure Data Factory became more user-friendly when data-flows were introduced.

They introduced the concept of Flowlets, but it has bugs. Flowlets are a reusable component that allows you to create data-flows. We can configure a Flowlet as a reusable pipeline and plug it inside different data-flows, so we don't have to rewrite our code or visual transformation.

If we make any changes in our data-flow, it reverts all our changes to the original state of the Flowlet. It does not retain changes, and we must reconfigure the Flowlets repeatedly. We had these issues three months ago so things might have changed. It works fine whenever we plug it in and configure it in our data-flow, but if we make minor changes to it, the Flowlet needs to be reconfigured again and loses the configuration.

We have used this solution for about a month and a half. It is a cloud-based tool, so there are no versions. It is all deployed on Azure Cloud.

Everything is computed inside the SQL server if we're working with pipelines, so we have to be very careful when designing our solution in Azure Data Factory. Alteryx spoiled us because we never cared how it looked in the backend because all the operations were happening on the Alteryx server. But in Azure Data Factory, they run on the capacity of our data warehouse. So Azure Data Factory cannot run your queries, and it directly sends the query to the instance in the SQL server or data warehouse. So we have to be very careful about how we perform certain operations.

We need to have knowledge of SQL and how to optimize our queries. If we are calling a stored procedure, it joins one table in Alteryx. It is pretty easy, and we just put a joint tool. Suppose we want to do it with a stored procedure in the Azure Data Factory. In that case, we have to be very careful about how we write our code. So that is a challenge for our team because we were not looking into how to optimize their SQL queries when fighting queries from Azure Data Factory to the data warehouse.

In addition, the workflows were running very slow, the performance was bad, and some queries were getting timed out because we have a threshold. So we faced many challenges and had to reeducate ourselves on SQL and query optimization.

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

We do not have experience with customer service and support.

It does not require any installation and is more like software as a service. You need to create an instance of Azure Data Factory in Azure and configure some of the connections to your databases. You can connect to your block storages and some authentication is necessary for Azure Data Factory.

The setup is straightforward. It doesn't take much time, and it's on cloud. It requires a few clicks, and you can quickly set it up and grant access to the developer. Then the developer can go to the link and start developing within their browser.

We have a team that takes care of the cloud infrastructure, so we raise a ticket and request infrastructure, and they just exceed it based on the naming convention with the project name.

We have an entire team that takes care of the cloud infrastructure. So we raise a ticket when we need infrastructure, which is executed based on the naming convention for the project name.

The nature of our solution is not based on ROI because we are building solutions for other functions within the same organization. In addition, due to the large size of our organization and the services we provide, the ROI is not something we consistently track. It's something discussed with the management, so I can't comment on it.

The cost is based on usage and the computing resources consumed. However, since Azure Data Factory connects with so many different functionalities that Azure provides, such as Azure functions, Logic apps and others in the Azure Data Factory pipelines, additional costs can be acquired by using other tools.

We did not evaluate other options because this solution was aligned with out current work environment. 

I rate the solution a seven out of ten. The solution is good and constantly improving, but the concept of Flowlets can be reconfigured to retain the changes we make. I advise users considering this solution to thoroughly understand what Azure Data Factory is and evaluate what's available in the market. Secondly, to assess the nature of the use cases and the kind of products they will be building before deciding to choose a solution.

The primary use case of this solution is to extract ETLS, transform and load data, and organize database synchronization.

The most valuable feature of this solution is the data flow, which is the same SQL server in important service, integration services, which is a very robust and powerful tool to transform data.

The solution can be improved by decreasing the warmup time which currently can take up to five minutes.

I have been using the solution for two years.

The solution is extremely stable.

The solution is scalable.

Previously I used AWS Glue and SSIS.

The initial setup is straightforward.

The implementation was completed in-house and is immediate because it is a native cloud tool.

I have seen an ROI with the time saved migrating data for reports.

The solution's fees are based on a pay-per-minute use plus the amount of data required to process. The more data you process the more CPUs and time is required which increases the cost of using this solution.

I give the solution ten out of ten.

The only thing you need to deploy the solution is to click on publish.

The solution includes a feature that increases the number of processors used which makes it very powerful and adds to the scalability.

We have three people using the solution in our organization and one engineer that maintains it.

I recommend to any potential user to factor in the five-minute warm-up time that is required for each execution.