Azure Data Factory Reviews

It's a PaaS service. It's a hybrid solution. The cloud provider is Microsoft.

We are not using Azure Data Factory as for users. Rather, we're using it as a process base. We're just using it for orchestration, not for any kind of ETL stuff.

We have plans to increase usage. It's going to take a major role in any kind of traditional data warehousing. It has big potential, especially as a PaaS offering.

There has been improvement in data resilience, in the way that we're moving the data from on-prem to cloud and vice versa.

The most important feature is that it can help you do the multi-threading concepts. It's in Informatica, but the resourcing is quite robust. You can scale up and scale down as per your needs.

There should be a way that it can do switches, so if at any point in time I want to do some hybrid mode of making any data collections or ingestions, I can just click on a button. I can change a switch and make sure a batch can be a streaming process.

I've been using Azure Data Factory for more than two years.

The stability of Azure as a PaaS could be improved.

It's scalable.

I would rate their technical support 3 out of 5. It's not great, but it isn't bad.

The setup is complex. It has nothing to do with the technology but with the design. We were wondering how to leverage the orchestration layer where we are having the Azure Data Factory and how to integrate with the Databricks. That's where we had some challenges in terms of choosing the right product.

You can do deployment in-house. 

I would rate this solution 8 out of 10.

For someone who is looking to use this solution, my advice is to do proper due diligence of your current application, know where your application is fitting, and look for the requirements. It all depends upon the current use case that you have currently in your system.

I am a manager of a team that uses this solution.

Azure Data Factory is primarily used for data integration, which involves moving data from sources into a data lake house called Delta Lake.

It's fairly simple to use. The most valuable feature of this solution would be ease of use.

It does not appear to be as rich as other ETL tools. It has very limited capabilities. It simply moves data around. It's not very good after that because it's taking the data to the next level and modeling it.

I have been working with Azure Data Factory for less than a year.

I would say that we are working with the latest version.

The stability of Azure Data Factory is good. The performance is good.

I haven't had to scale this solution as of yet.

We have six people in our company who use this solution.

Increasing the usage is not on our strategy pathway.

I have not contacted technical support. I have not required any yet.

I have had very little contact with Microsoft support, but it's been good.

I have also worked with Talend. I didn't switch products, but rather companies.

Talend is a more robust enterprise-wide solution that can handle everything from start to finish, whereas Azure Data Factory is more of an ingestion tool.

I was not involved with the initial setup.

We are an enterprise that uses an integrator.

It does not require any maintenance, it's simple.

I don't see a cost; it appears to be included in general support. I have been told that you have to be very careful because it can blow out. I have not experienced it yet, but I've been warned that as Azure ingestion increases, the costs can rise.

In my opinion, the price is competitive.

It's a good tool, a good product that does what it's supposed to do well, which is ingesting data from a source to your target, to another cloud, to another source. Just be conscious to monitor your costs.

I would rate Azure Data Factory an eight out of ten.

We are currently using it as an ETL (Extract, Transform, and Load) tool. We are using it to connect to various information providers or, in general, to various sources, to extract data, and then to insert it to our storage devices, databases, or data warehouses.

The two most valuable features of Azure Data Factory are that it's very scalable and that it's also highly reliable.

One area for improvement is documentation. At present, there isn't enough documentation on how to use Azure Data Factory in certain conditions. It would be good to have documentation on the various use cases.

Sometimes, it's really difficult to find the answers to very technical questions regarding certain conditions.

I've been using Azure Data Factory since 2019.

It has been stable so far.  

Azure Data Factory is a very scalable solution. Including internal developers and external consultants working for us, we have about 10-15 people using this solution.

We had been using various ETL tools during the years before moving to the cloud. We picked Azure Data Factory because we were moving towards the Azure cloud.

The initial setup is very easy.

We used a consultant as it was a big project. We had five to six specialists, including both internal and external employees, working on it. It took about about three to six months to complete.

Azure Data Factory is a very easy to use tool. If you want to extract, manipulate, and load data to any type of Azure repository, I recommend this solution. However, I would not recommend it if the manipulation of data is very deep and complicated.

I would rate this solution at eight on a scale from one to ten.

My primary use cases for this solution are integration and connecting to the different data stores where we get data and migration activity, deployment, and integrations into using linked services and deployment models.

This solution has allowed me to quickly get analysis, sales data, supply chain data, and eCommerce data.

The most valuable feature of this solution is that it allows more data between on-premises and cloud solutions. It's also useful for orchestration for complex data flows and allows us to do ETL-based transitions heavily. In addition, it allows us to integrate with other third-party systems and compare features and pricing. Other valuable features include database replication, SQL service products, SLA support, data sharing, vendor lock-in, and support for developer tools.

Areas for improvement would be the product's performance and its mapping of data flow. In addition, some of the optimization techniques are not scalable, some naming connections are not supported, and automated testing is not supported in all cases. In the next release, I would like to see support so we can enhance based on the next-level pipelines, writing from scratch, flexible scheduling, and pipeline activity.

I've been using this solution for about a year.

This solution is very reliable.

This solution is scalable.

I am satisfied with the technical support.

I previously worked with Azure SQL database.

The initial setup was complex, but the deployment only took 30 to 40 minutes.

This product is priced at the market standard, which is good given that the product contains all the available assets.

When selecting services, make sure to choose only those you need in order to reduce your costs. I would rate this solution as seven out of ten.

We generally implement this product for data transformation for our clients. We create the pipelines and provide training before handing it over to them. We generally deal with large-scale organizations. I'm a senior solutions architect. 

I think the main benefit of this solution is the ease of use, especially for companies that have come from an SSIS type of background where they are used to Microsoft tools. 

If you have a very simple pipeline you can use Data Factory for transformations, but it's really for serious analytics. This is an excellent tool for pipeline orchestration; connecting the different components and activities as well as gathering data. It's an orchestration tool, not a transformation tool. The monitoring feature has drastically improved.

Data Factory is embedded in the new Synapse Analytics. The problem is if you're using the core Data Factory, you can't call a notebook within Synapse. It's possible to call Databricks from Data Factory, but not the Spark notebook and I don't understand the reason for that restriction. To my mind, the solution needs to be more connectable to its own services.

There is a list of features I'd like to see in the next release, most of them related to oversight and security. AWS has a lake builder, which basically enforces the whole oversight concept from the start of your pipeline but unfortunately Microsoft hasn't yet implemented a similar feature.

I've been using this solution for five years. 

From what I've seen this is a stable solution. 

The solution is easy to scale keeping in mind that Data Factory doesn't do any computations. We use it mainly to push the computations to Databricks or Synapse. Projects with our clients generally last a few months and only until they go into production. I believe the ability to increase is always there.

We typically do not use customer support, but there were a few cases several years ago as the product was moving to the cloud that things were not so stable and we contacted support services - they were very good. 

When I first started in this field, everything was basically Hadoop on-premise and Hadoop infrastructure. With the increase in cloud integrations, things have changed. Once the big data services got introduced, we were probably one of the few companies in North America that were actually into analytics and big data and we were the first to implement related Microsoft products in Canada.

The initial setup is straightforward. I'm a huge fan and user of CI/CD pipelines and never do deployments manually. It's all automated and deployment takes a few minutes.

Licensing costs of Data Factory are reasonable. The cost is mainly on the Synapse and Databricks side of things because they are the tools where the computations are done and where you need more nodes and servers.

It's important to study the solution before purchasing it. The problem in this market is that because most users are generally not very knowledgeable, they typically fall for services that are not compatible with their use case. Data Factory comes with all the transformations but that doesn't work for serious analytics customers who generally need to resort to Databricks or Synapse which involves training and education. Since it's a new field and everything has just blasted off, it's very hard for people to catch on.

In my opinion, Airflow still ranks as number one but I would rate Data Factory an eight out of 10. 

My primary use case of Azure Data Factory is supporting the data migration for advanced analytics projects. 

One of the most valuable features of Azure Data Factory is the drag-and-drop interface. This helps with workflow management because we can just drag any tables or data sources we need. Because of how easy it is to drag and drop, we can deliver things very quickly. It's more customizable through visual effect. 

Data Factory could be improved by eliminating the need for a physical data area. We have to extract data using Data Factory, then create a staging database for it with Azure SQL, which is very, very expensive. Another improvement would be lowering the licensing cost. 

I have been using this solution for the past year. 

This solution is stable. We are using an Azure subscription, so there is no maintenance or direct updates, it's just always the latest version.

This solution is automatically scalable, since it's in the cloud. At my company, there were more than one thousand people using this solution because we were a big, media-based company. If there are many user requests in the front end application and the system is not responding much or has slow performance, the system will automatically scale up the performance hardware requirements. 

I have contacted technical support. I have never faced an issue like that with Denodo. Fortunately, we got some kind of a tutorial PDF, which helps us to deploy everything quickly. 

Before working with Azure, I worked with Python. In the culture I was working in, there was no integration. We were using Pure Python scripting and Python data manipulation tools. For example, we used Python's pandas library, which we coded to transform and orchestrate the data, which is necessary for the endpoint. It was not at all a visual tool. It took more time than Denodo. 

There is no installation because it's on the cloud. You just log on to the cloud with your subscription credentials, then you can use Data Factory directly. 

I implemented through an in-house team. 

Data Factory is very expensive. We are using an Azure subscription, so Data Factory has no direct updates, it's just always the latest version. Compared to Denodo, Azure is very costly. Azure Framework has multiple services, not only Data Factory. So in the cloud-based solution, if you're selecting a particular service, like Data Factory, you need to pay for each request.

I also use Denodo. Data Factory is like a transformation layer, but we need an additional staging database or a data storage facility, which is very expensive compared to implementing Denodo. So we extracted the data using Data Factory, then created a staging database with Azure SQL, which cost a huge amount since it's a physical data area. In Denodo, we just implement a layer, which is all handled in Denodo, and not a physical storage mechanism. I prefer customizable data solutions because they improve performance, creativity, and are helpful for front end people.

In comparison to Data Factory's drag-and-drop interface, Denodo developers need to create all the unified views by coding, so we have to create SQL queries to execute. With Data Factory, you can quickly drag and drop data or tables, but in Denodo, it takes more time because you need to code and test and all that.

I rate Data Factory an eight out of ten, mainly because you need a staging database. I recommend Azure to others, but it depends on architecture. In Data Factory, there is no virtualization environment, no layer of virtualization to help integration and doing caching mechanisms. Though Data Factory is there, Denodo is going further. 

Our customers use it for data analytics on a large volume of data. So, they're basically bringing data in from multiple sources, and they are doing ETL extraction, transformation, and loading. Then they do initial analytics, populate a data lake, and after that, they take the data from the data lake into more on-premise complex analytics.

Its version depends on a customer's environment. Sometimes, we use the latest version, and sometimes, we use the previous versions.

It is very modular. It works well. We've used Data Factory and then made calls to libraries outside of Data Factory to do things that it wasn't optimized to do, and it worked really well. It is obviously proprietary in regards to Microsoft created it, but it is pretty easy and direct to bring in outside capabilities into Data Factory.

It is very flexible. You can build any features you want.

There is always room to improve. There should be good examples of use that, of course, customers aren't always willing to share. It is Catch-22. It would help the user base if everybody had really good examples of deployments that worked, but when you ask people to put out their good deployments, which also includes me, you usually got, "No, I'm not going to do that." They don't have enough good examples. Microsoft probably just needs to pay one of their partners to build 20 or 30 examples of functional Data Factories and then share them as a user base.

I have been using this solution for the last five years, but probably, the last three years have been significant.

It has been stable. I have not experienced any issues.

It is decent for most things. I'm not sure if it is necessarily intended for large volume and high-speed streams of data. By large, I mean really big, but for pretty much anything that most users would want to do, including ourselves, it is fine. Our clients are large government organizations.

It scales fine within its environment. You can literally throw another Data Factory in or replicate one and do things pretty quickly. So, it is not at all hard to increase your processing footprint, but you have to pay for it. It doesn't end up being quite expensive. Although I haven't really done it, I would suspect that if I did the equivalent in AWS, Azure would be more expensive than AWS because of the way they price data.

They're all right. I would rate them a seven out of 10. They do fine, but there is a lot that they don't do.

I'm not sure if even Microsoft has enough SMEs from a user point of view. They are helpful for getting it set up, making it work, and helping you figure out why it doesn't work. If you want to ask them about something that you are trying to do, they'll try to direct you to a partner, which is fine, but the partners also don't necessarily have an experience. It is Catch-22. There aren't a lot of people out there with Azure experience because Azure started to be in demand only over the last two years.

The customer used a lot of homebrew stuff. They were doing a lot of internal stuff and some Oracle stuff. They were doing things, and they made a workaround and said, "Okay, we'll bring it into Oracle Database, and then we'll do all these things to it." We're like, "Okay, that works, but then you're taking it out of that database and putting it over into the data lake. I don't understand why are you doing that?" That's what they were doing.

It is pretty straightforward. Devil is in the details, but you can easily get up and running in a day with Data Factory. Anybody who is comfortable in Azure can set up Data Factory, but it takes experience to know what it can and can't do or should and shouldn't do.

It is proven, and it works. Make sure you have a well-defined use case and build a quick prototype to ensure that it, in fact, does what you need. Give yourself some benchmarks. That's exactly what we did. We defined the use case, and then we set up Data Factory. We found a couple of things that it didn't do. We figured out a way to work around those things and have it do those things. After that, we confirmed it. It is operational, and it is doing its job. It has been pretty much error-free since then.

It would become easier to use as more people become Azure-capable. If I want to find an AWS SME, I can get tons. They're expensive, but I have them. If I want to find an Azure SME, I usually have to create them. Azure was later to market than AWS. So, there are fewer people who are experts in Azure, and they are in high demand.

I would rate Azure Data Factory a nine out of 10. They just don't have enough good examples out there of things.

I use Azure Data Factory in my company because we are implementing a lot of different projects for a big company based in the USA. We're getting certain information from different sources—for example, some files in the Azure Blob Storage. We're migrating that information to other databases. We are validating and transforming the data. After that, we put that data in some databases in Azure Synapse and SQL databases.

I think it makes it very easy to understand what data flow is and so on. You can leverage the user interface to do the different data flows, and it's great. I like it a lot.

It would be better if it had machine learning capabilities. For example, at the moment, we're working with Databricks and Azure Data Factory. But Databricks is very complex to do the different data flows. It could be great to have more functionalities to do that in Azure Data Factory.

I have been using Azure Data Factory for about one year.

Azure Data Factory is a stable solution.

It's scalable. We're doing a lot of different integrations with a lot of data, and scalability is great.

The initial setup is straightforward. I think that it's so easy to start a project using that technology.

We have a team that's in charge of doing the deployments in Azure in different environments.

I would tell potential users that there are many technologies to do this. For example, if you like to manage big data and do something with it, it would be better to use Databricks.

On a scale from one to ten, I would give Azure Data Factory a nine.