Azure Data Factory OverviewUNIXBusinessApplication

Azure Data Factory is the #1 ranked solution in top Data Integration Tools and #2 ranked solution in top Cloud Data Warehouse tools. PeerSpot users give Azure Data Factory an average rating of 7.8 out of 10. Azure Data Factory is most commonly compared to Informatica PowerCenter: Azure Data Factory vs Informatica PowerCenter. Azure Data Factory is popular among the large enterprise segment, accounting for 70% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 19% of all views.
Azure Data Factory Buyer's Guide

Download the Azure Data Factory Buyer's Guide including reviews and more. Updated: November 2022

What is Azure Data Factory?

Azure Data Factory is a managed cloud service built for extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects. This is a digital integration tool as well as a cloud data warehouse that allows users to create, schedule, and manage data in the cloud or on premises. The use cases of the product include data engineering, operational data integration, analytics, ingesting data into data warehouses, and migrating on-premise SQL Server Integration Services (SSIS) packages to Azure.

The tool allows users to create data-driven workflows for initiating data movement and data transformation at scale. Data can be ingested from disparate data stores via pipelines. Companies can utilize this product to build complex ETL processes for transforming data visually with data flows. Azure Data Factory also offers services such as Azure HDInsight Hadoop, Azure Databricks, Azure Synapse Analytics, and Azure SQL Database. These services are created to facilitate data management and control for organizations, providing them with better visibility of their data for improved decision-making.

Azure Data Factory allows companies to create schedules for moving and transforming data into their pipelines. This can be done hourly, daily, weekly, or according to the specific needs of the organization. The steps through which the data-driven workflows work in Azure Data Factory are the following:

1. Connecting to required sources and collecting data. After connecting to the various sources where data is stored, the pipelines move the data to a centralized location for further processing.

2. Transforming and enriching the data. Once the data is moved to a centralized data store in the cloud, the pipelines transform it through services like HDInsight Hadoop, Azure Data Lake Analytics, Spark, and Machine Learning.

3. Delivering the transformed data to on-premise sources or keeping it in cloud storage sources for usage by different tools and applications.

Azure Data Factory Concepts

The solution consists of a series of interconnected systems that provide data integration and related services for users. The following concepts create the end product for users:

  • Pipelines: A pipeline refers to the logical grouping of activities that performs a unit of work which together perform a task.

  • Mapping data flows: Azure Data Factory lets its users create and manage graphs of data transformation logic for transforming any-sized data. The logic is executed on a Spark cluster, which does not have to be managed or maintained personally by the user.

  • Linked services: The linked services in the tool define the connection to the data source. There are various services used for two main purposes - to represent a data store that the solution supports and to represent a compute resource that can host the execution of an activity.

  • Integration runtime: The integration runtime in the tool provides the bridge between the activity and linked services needed for it.

  • Triggers: There are various types of triggers in the solution, created for different types of events. They determine when a pipeline execution should be initiated.

  • Pipeline runs: Pipeline runs are instantiated by passing the arguments to the parameters that are defined in pipelines, executing the pipelines' work.

  • Control flow: Control flow in Azure Data Factory is an orchestration of pipeline activities.

  • Connect and collect: This serves as the first step of the services that this tool offers. It connects all the required sources of data and processing in order to prepare the data for moving it to a centralized location for further processing. The step eliminates the need for companies to integrate expensive custom solutions for data movement. Through Copy Activity, Azure Blob storage, and Azure HDInsight Hadoop cluster, users can quickly initiate the first step of organizing their data.

  • Transform and enrich: The collected data can be processed or transformed by using the mapping data flows of the product. Data transformation graphs can be executed on Spark without the need to understand its clusters or how programming works.

  • CI/CD and publish: Through Azure DevOps and GitHub clients, the tool can receive full support for CI/CD for their data pipelines, which allows for the development and delivery of ETL processes before publishing the finished product.

  • Monitor: When users have successfully built and deployed their data integration pipelines, the service offers them the option to monitor the scheduled activities and pipelines. This is done through Azure Monitor, API, PowerShell, and health panels on the Azure portal.

Azure Data Factory Benefits

Azure Data Factory offers clients many several benefits. Some of these include:

  • An easy-to-use platform which is suitable for both beginner and expert users, as it offers code-free processes and built-in support.

  • Pay-as-you-go option for clients to pay only for the services that they are using.

  • Powerful tool with more than 90 built-in connectors, which allow companies to ingest on-premise and software as service (SaaS) data quickly.

  • Provided autonomous ETL, which unlocks operational efficiencies and citizen integrators.

  • The tool is designed to handle large volumes of data and provide users with better scalability and performance than classic ETL systems.

  • Azure Data Factory allows users to easily migrate ETL workloads to the solution’s cloud.

  • The solution offers great security for its users, as it provides the option for assigning specific permissions and roles within the organization.

  • Azure Data Factory is highly automated, which allows users to orchestrate their data more efficiently.

  • The platform is a combination of GUI and scripting-based interfaces, which gives users more freedom over data management.

  • The tool provides organizations with the option to rely on Microsoft to fully manage the process. This eliminates the potential need of hiring a third-party expert.

Reviews from Real Users

According to Dan M., a Chief Strategist & CTO at a consultancy, Azure Data Factory is secure and reasonably priced.

A Senior Manager at a tech services company evaluates the tool as reasonably priced, scales well, good performance.

Azure Data Factory Customers

Milliman, Pier 1 Imports, Rockwell Automation, Ziosk, Real Madrid

Azure Data Factory Video

Archived Azure Data Factory Reviews (more than two years old)

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
Business Unit Manager Data Migration and Integration at a tech services company with 201-500 employees
Real User
Offers good flexibility and has good support
Pros and Cons
  • "The flexibility that Azure Data Factory offers is great."
  • "The number of standard adaptors could be extended further."

What is our primary use case?

We use this solution for data integration. We use it to feed operational data into a data warehouse. We also use it for creating connections between applications.

Within our organization, there are a few thousand users of Azure Data Factory.

We believe that the number of customers and usage of this product will extend over the next few years. For this reason, we invest a lot of resources in building skills, and we make sure to hire consultants who know their way around Data Factory.

What is most valuable?

The flexibility that Azure Data Factory offers is great.

What needs improvement?

The number of standard adaptors could be extended further. What we find is that if we develop data integration solutions with Data Factory, there's still quite a bit of coding involved, whereas we'd like to move in a direction with less coding and more select-and-click.

For how long have I used the solution?

I have been using Azure Data Factory for one year.

Buyer's Guide
Azure Data Factory
November 2022
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
654,218 professionals have used our research since 2012.

What do I think about the scalability of the solution?

We don't have any complaints regarding scalability or stability.

How are customer service and support?

I think Microsofts' technical support does a pretty good job. There is a lot of information available on the internet to find out how to use their products. There's also quite an active community. If you really can't find a solution, you can always call Microsoft. Our organization is partnered with Microsoft, so we usually get answers directly from them.

Which solution did I use previously and why did I switch?

We used SSIS. We're still using SSIS. SSIS is an old product. The development of SSIS has more or less stopped and the development is now focused on cloud services — it's the future. Azure Data Factory Is great because it's a cloud service; you do not have to take care of the installation and configuration yourself. The cost buildup is also quite different. I am not sure that's a huge financial advantage yet, but we do believe that it will be in the future.

How was the initial setup?

The initial setup was straightforward.

We didn't have to deploy Azure Data Factory. It's available as an Azure service, so Microsoft takes care of that.

What's my experience with pricing, setup cost, and licensing?

It's a pay-per-use model. So if you run it, it's hardly licensing. The entire cost of Azure is per-use. The price you pay is determined by how much you use it.

What other advice do I have?

I would definitely recommend Azure Data Factory. On a scale from one to ten, I would give this solution a rating of eight. 

If there were a larger amount of automated features, I would give them a higher rating. As I mentioned earlier, if we are working on complex applications, then there is a lot of coding involved. What we hope is that over time, there'll be less coding and more "off the shelf" functionality.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Balan Murugan - PeerSpot reviewer
Azure Technical Architect at a computer software company with 10,001+ employees
Real User
Easy to set up, has many built-in connectors for onboarding data, and offers good support
Pros and Cons
  • "It has built-in connectors for more than 100 sources and onboarding data from many different sources to the cloud environment."
  • "We have experienced some issues with the integration. This is an area that needs improvement."

What is our primary use case?

The primary use case of this solution is for data integration.

What is most valuable?

It has built-in connectors for more than 100 sources and onboarding data from many different sources to the cloud environment.

It has everything we needed.

Being able to post your feedback and queries is a very good feature that is offered by Azure Data Factory.

What needs improvement?

Understanding the pricing model for Data Factory is quite complex. It needs to be simplified, and easier to understand.

We have experienced some issues with the integration. This is an area that needs improvement.

For how long have I used the solution?

I have been using this solution for almost two years.

What do I think about the stability of the solution?

So far it's been stable. We have not experienced any issues.

What do I think about the scalability of the solution?

I haven't put much thought into scalability.

How are customer service and technical support?

I have contacted technical support.

If there are any queries, they have the provision to post them in the Data Factory GUI and we can contact the team directly. This is a very nice feature.

Any feedback or improvements that we post is acknowledged and taken care of.

How was the initial setup?

The initial setup was straightforward. It was not complex.

What's my experience with pricing, setup cost, and licensing?

Pricing is quite complex.

I have had feedback from many people and many of my team members, and they say that it is difficult to understand. 

Understanding the pricing of Data Factory is quite complex. 

It is not presented in a straightforward way.

What other advice do I have?

I would definitely recommend this solution to anyone who is interested in using it.

I would rate Azure Data Factory an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Buyer's Guide
Azure Data Factory
November 2022
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
654,218 professionals have used our research since 2012.
Anil Jha - PeerSpot reviewer
Director D&A at Iris Software Inc.
Real User
Easy to set up and integrates well, but it needs support for custom data delimiters
Pros and Cons
  • "It is easy to integrate."
  • "You cannot use a custom data delimiter, which means that you have problems receiving data in certain formats."

What is our primary use case?

The primary use case is integrating data from different ERP systems and loading it into Azure Synapse for reporting. We use Power BI for the reporting side of it.

We also have customers who are migrating to Azure Data Factory and we are assisting them with making the transition.

What is most valuable?

It is easy to integrate.

I do not foresee any issues with security.

What needs improvement?

I find that Azure Data Factory is still maturing, so there are issues. For example, there are many features missing that you can find in other products.

You cannot use a custom data delimiter, which means that you have problems receiving data in certain formats. For example, there are problems dealing with data that is comma-delimited.

For how long have I used the solution?

I have been using Azure Data Factor for almost one year.

What do I think about the stability of the solution?

The stability is dependent on how you set up your cloud infrastructure, and how you authorize people to make use of it.

What do I think about the scalability of the solution?

I have not seen any issues with respect to scalability, as it is all hosted within the cloud. We have approximately 20 users.

How are customer service and technical support?

I have been in contact with technical support, although most of the time I was told that the feature I was interested in was not yet available. In these cases, they will be implementing the missing features in the future.

Which solution did I use previously and why did I switch?

I use several similar products by different vendors including Talend, Informatica, and Microsoft SSIS. The biggest advantage that Azure has is deployment. However, in others, it is possible to specify custom data delimiters.

How was the initial setup?

The initial setup is pretty simple and it can be deployed in a couple of hours.

What about the implementation team?

I deployed it myself and am also responsible for maintenance.

What other advice do I have?

I would rate this solution a five out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
CTO at a construction company with 1,001-5,000 employees
Real User
Top 20
Fully integrated with the Azure environment and includes support for many connectors
Pros and Cons
  • "The security of the agent that is installed on-premises is very good."
  • "The pricing scheme is very complex and difficult to understand."

What is our primary use case?

We are using this solution to gather information from SCADA systems, analyze it using AI and machine learning, and then sending the results to our users. They receive and view the data using the Power BI interface.

How has it helped my organization?

The integration with the whole environment is smooth and transparent.

What is most valuable?

The reason that we implemented this product is for the full integration with the whole Azure environment. It is very important because you don't have to deal with security, identity flow, or audit flow.

The security of the agent that is installed on-premises is very good. This is another important feature for us.

It comes with a lot of connectors out of the box, including support for SQL, Oracle, SAP, and ODBC.

What needs improvement?

The pricing scheme is very complex and difficult to understand. Analyzing it upfront is impossible, so we just decided to start using it and figure out the costs on a weekly or monthly basis.

For how long have I used the solution?

We have just begun using Azure Data Factory within the past two or three months and it is not fully rolled out yet.

How are customer service and technical support?

We have no issues with technical support. We are working closely with Microsoft partners. There were a couple of problems during the installation that were solved on the fly, and otherwise, we haven't had and trouble.

Which solution did I use previously and why did I switch?

We started with iConduct and as we migrated to the Azure environment, we began the process of switching to the Microsoft Azure Data Factory.

How was the initial setup?

The initial setup was not very complex and the deployment was very quick. We were set up within a matter of days, or perhaps a week.

What about the implementation team?

We received help from one of the Microsoft partners, who handled the installation for us.

What other advice do I have?

To this point, we are still learning the system and have only tried a very simple data flow. At this point, we haven't had any issues.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Suraj Kshirsagar - PeerSpot reviewer
Azure Architect\Informatica ETL Developer at Relativity
Real User
A helpful and responsive GUI, but there are a lot of tasks for which you need to write code
Pros and Cons
  • "The most valuable feature is the ease in which you can create an ETL pipeline."
  • "The support and the documentation can be improved."

What is our primary use case?

I use this primarily for ETL tasks.

What is most valuable?

The most valuable feature is the ease in which you can create an ETL pipeline.

The GUI is very helpful when it comes to creating pipelines. The user interface is also very fast.

The connection to Snowflake is easy. I can store data and transform it during the ETL process before sending it to Snowflake.

What needs improvement?

Azure Data Factory is a bit complicated compared to Informatica. There are a lot of connectors that are missing and there are a lot of instances where I need to create a server and install Integration Runtime.

The support and the documentation can be improved.

There are a lot of tasks that you need to write code for.

For how long have I used the solution?

I have been using Azure Data Factory for about six months.

Which solution did I use previously and why did I switch?

I have experience with Informatica and I find it easier to use. For example, there are a lot of connectors that are directly available. Also, Informatica is able to take incremental copies, but with Azure, you have to write code to do that.

I have also worked with Matillion and Fivetran, and I feel that there are a lot of things that Azure can learn from these products. For example, with Fivetran there are very good connectors for copying data between other solutions. This is unlike Azure, where a lot of the time, I have to build my own logic.

How was the initial setup?

The initial setup is complex.

What other advice do I have?

I would rate this solution a seven out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Data Strategist | Azure Solutions Architect at a tech services company with 11-50 employees
Real User
An easy initial setup with a fast deployment and good technical support
Pros and Cons
  • "On the tool itself, we've never experienced any bugs or glitches. There haven't been crashes. Stability has been good."
  • "If the user interface was more user friendly and there was better error feedback, it would be helpful."

What is our primary use case?

I primarily use the solution for my small and medium-sized clients.

What is most valuable?

So far, I'm quite happy with the solution overall. It's got a lot of tools that I use in my work, and these are items I'm already recommending to my clients. I'm quite happy with it.

What needs improvement?

I'm not sure if I have any complaints about the solution at the moment. There are a few bits and pieces that we would like to see improved. These include improvements related to the solution's ease of use and some quality flash upgrades. However, these are minor complaints. 

If the user interface was more user friendly and there was better error feedback, it would be helpful.

For how long have I used the solution?

I've been working with the solution for three to four years now.

What do I think about the stability of the solution?

On the tool itself, we've never experienced any bugs or glitches. There haven't been crashes. Stability has been good.

What do I think about the scalability of the solution?

I haven't had the ability to scale any of my projects personally. I also wouldn't need to scale too high if I did. I'm not sure if I can speak to aspects of scalability as I've never dealt with it.

How are customer service and technical support?

I've contacted Microsoft technical support in the past. I have to say that I've had a good experience with them. I've been quite satisfied with their level of service.

How was the initial setup?

The initial setup was very straightforward. I wouldn't describe it as complex.

Deployment is fast. It only takes about half an hour at most.

What's my experience with pricing, setup cost, and licensing?

I don't have any information about the pricing. I don't deal with that aspect of the solution.

What other advice do I have?

I typically work with small to medium-sized companies. I'm a consultant, so I give different advice based on what my clients need and what they want to do. However, I would recommend this product. 

I'd rate the solution eight out of ten. All of the issues I have with the solution are very minor, however, it means the solution isn't exactly perfect just yet.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Delivery Manager at a tech services company with 1,001-5,000 employees
Real User
User friendly, cost-effective, and the performance is good
Pros and Cons
  • "It is easy to deploy workflows and schedule jobs."
  • "The setup and configuration process could be simplified."

What is our primary use case?

We are a tech services company and this is one of the tools that we use when implementing solutions for our clients. I am currently managing a team that is working with the Azure Data Factory.

Our clients that use this solution are migrating their data from on-premises to the cloud.

One of our clients is building an integrated data warehouse for all of their data, using this solution. It is used to extract all of the data from different servers and store it into one place.

What is most valuable?

Our clients find that this solution has a very good performance. They like the speed.

It is easy to deploy workflows and schedule jobs. You can just click on the desktop and it works.

What needs improvement?

The setup and configuration process could be simplified.

For how long have I used the solution?

We have been using Azure Data Factory for the past six months.

What do I think about the stability of the solution?

This is a stable product and we're expecting more updates from Microsoft. We have not used more than one terabyte of data so that remains untested, but for one terabyte it works fine.

Development is only done on an occasional basis, but the solution is used every day. If it is streaming data then the process is continuous, otherwise, it is initiated by the user on demand.

What do I think about the scalability of the solution?

This solution is 100% scalable.

We have two clients working with this solution.

How are customer service and technical support?

It is another team who is responsible for contacting technical support.

Which solution did I use previously and why did I switch?

We have used other ETL solutions in the past, and Azure Data Factory is the best one. Compared to SSIS, for example, ADF is easier to use and the performance is better.

Our clients are migrating from on-premises SSIS solutions to the cloud because they want to take advantage of the latest technologies.

How was the initial setup?

The installation is very simple and it doesn't take much time. For us, the deployment took about two days, which does not seem unreasonable for something that is on the cloud. Most of the time is spent waiting for credentials.

Depending on the sources of the data, four people are required for deployment and maintenance. If the sources are SQL databases then it is straightforward and four people can cope with it. If the data is more difficult then we may need more people.

What about the implementation team?

The deployment is done in-house with a technical person with knowledge of the Azure cloud.

What's my experience with pricing, setup cost, and licensing?

This is a cost-effective solution.

Which other solutions did I evaluate?

We have another team that is moving to AWS, but for now, we will continue working with Azure Data Factory. Once they have explored the AWS solution fully, we will compare the two.

What other advice do I have?

Within the next six months, we are planning to enter into the machine learning part of this solution. This is a product that I can recommend.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Balan Murugan - PeerSpot reviewer
Azure Technical Architect at a computer software company with 10,001+ employees
Real User
Has the ability to copy data to any environment
Pros and Cons
  • "From my experience so far, the best feature is the ability to copy data to any environment. We have 100 connects and we can connect them to the system and copy the data from its respective system to any environment. That is the best feature."
  • "The user interface could use improvement. It's not a major issue but it's something that can be improved."

What is our primary use case?

It's an integration platform, we migrate data across hybrid environments. We have data in our cloud environment or on-prem system so we use it for when we want to integrate data across different environments. It was a problem for us to get data from different hybrid environments.

What is most valuable?

From my experience so far, the best feature is the ability to copy data to any environment. We have 100 connectors and we can connect them to the system and copy the data from its respective system to any environment. That is the best feature. 

What needs improvement?

The user interface could use improvement. It's not a major issue but it's something that can be improved. 

It has the ability to create separate folders to organize objects, Data Factory objects. But any time that we created a folder we were not able to create objects. We had to drag and drop into the folder. There were no default options. It was manual work. We offered their team our feedback and they accepted my request.

For how long have I used the solution?

I have been using Azure Data Factory for around one year. 

What do I think about the stability of the solution?

Based on my experience with other products on the market, the stability is good. 

What do I think about the scalability of the solution?

I haven't had much experience with scalability. I know we do have scalability options though. It's used daily. 

There are around 1,000 plus users using this solution in my company. 

It requires two people for maintenance. The administrators are the ones who maintain it and give access to the engineers. They regulate who has privileges. 

How are customer service and technical support?

We have needed to contact their technical support. If we can't find the answers ourselves on the blogs, we contact them with our questions. We get most of the answers we need from the blogs but if not then we can directly speak to the Microsoft team from the Data Factory interface itself, it's really helpful.

Which solution did I use previously and why did I switch?

I have only used Data Factory for the cloud. For on-prem we have used SSIS.

How was the initial setup?

The initial setup was a bit complex but once you understand its setup, it's less complex. There are certain processes that need to be followed. Once you understand the process, it becomes easier to implement.

The implementation took a little less than one day. The planning for the deployment takes around one or two days. 

What about the implementation team?

We had a discussion with the Microsoft team about the data. We discussed how we were going to implement. Based on the discussion we were able to deploy. A Microsoft partner helped us with some parts. 

Which other solutions did I evaluate?

We also evaluated AWS.

What other advice do I have?

The advice that I would give to someone considering this solution is to have some background in data warehousing and ETL concepts. Have the background about data warehousing and ETL that extract, transform, and load. If you have the background you need, you will be successful. If not, then my advice would be to learn a little more about it before using Data Factory.

I would rate Data Factory as an eight out of ten. 

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Team Leader at a insurance company with 201-500 employees
Real User
Complete solution with good interface, workflow, and ease of use
Pros and Cons
  • "It is a complete ETL Solution."
  • "The Microsoft documentation is too complicated."

What is most valuable?

The features that I've found most valuable, in order: That it is a complete ETL Solution, the second one is interface, the third one workflow, and the fourth one ease of use.

What needs improvement?

The only thing that we're struggling with is increasing the competency of my team. So we think that the Microsoft documentation is too complicated.

I would like to see it more connected. I know they're working on the Snowflake data warehouse connector, but more connectors would be helpful.

For how long have I used the solution?

I've been using Azure Data Factory for a few months.

What do I think about the stability of the solution?

It is stable.

What do I think about the scalability of the solution?

Azure Data Factory is a scalable solution.

How are customer service and technical support?

In terms of support, we haven't used the direct support but the community support is very good.

Which solution did I use previously and why did I switch?

We previously used the Oracle Warehouse Builder. The reasons we stopped using it were that Warehouse Builder is at the end of its life-cycle and its slow performance.

How was the initial setup?

The initial setup is complex.

What about the implementation team?

We first implemented it ourselves but we're using a partner now. We thought we could do it ourselves but we needed specialty help.

What's my experience with pricing, setup cost, and licensing?

I would highly recommend Azure Data Factory as it has proof of concept.

What other advice do I have?

We have nine people using Azure Data Factory.

On a scale of one to ten, I give it an eight.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Head of IT at a logistics company with 10,001+ employees
Real User
Helps integrate complex data needs and is easy to use
Pros and Cons
  • "Powerful but easy-to-use and intuitive."
  • "The product could provide more ways to import and export data."

What is our primary use case?

The use cases are more related to logistics, our finance, and back-office activity.  

What is most valuable?

The most valuable part of this product is the ease of use. It is easy to use and rather intuitive. Because it is easy to use, you can do things with it easily. The more things make your work easy, the more valuable they are.  

What needs improvement?

Because I have not really done a really deep benchmark against competitors, I may not be familiar enough with the potential of competing products and capabilities to be able able to say what is missing or should be improved definitively.  

From my perspective, the pricing seems like it could be more user-friendly. Of course, nothing is ever as inexpensive as you want.  

Perhaps one good additional feature would be incorporating more ways to import and export data. It would be nice to have the product fit our service orchestration platform better to make the transfer more fluid.  

For how long have I used the solution?

We started using this product a year ago.  

What do I think about the stability of the solution?

The stability of the product is good.  

What do I think about the scalability of the solution?

The scalability seems okay. As we have only been using it for a short time, it is hard to say more. We are not currently planning to scale usage dramatically at this point but of course we would like to grow. On a scale from one to ten and from what I know, I would say scalability is an eight-out-of-ten. I can't be sure exactly how many people are using the system, but we have hundreds of thousands of users currently. Internally, I would say we use the product often.  

How are customer service and technical support?

I have not had a reason to be in touch with technical support, but I don't know whether others in the organization have been in touch with them. As far as I know, there has been no reason to be.  

How was the initial setup?

The initial setup was not simple and it was not complex. It was in the middle.  

I would say it took two months for the deployment.  

What about the implementation team?

We have a department of developers that implemented the product. The deployment happened before I joined the organization.  

What other advice do I have?

The advice I would give to someone who is looking to implement this product is to understand the IT technology of the product first and why it would be needed. That is the point where you have to start. Next, you have to understand if the product itself fits your organizational needs. That is you have to look at the business requirements and see whether the product really fits the organization and solves the problems while conforming to the business model.  

On a scale from one to ten where one is the worst and ten is the best, I would rate the product overall as an eight-out-of-ten.  

Which deployment model are you using for this solution?

Private Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: My company has a business relationship with this vendor other than being a customer:
PeerSpot user
MayankRakesh - PeerSpot reviewer
Sr. Technology Architect at a tech services company with 10,001+ employees
Real User
Straightforward and scalable but could be more intuitive
Pros and Cons
  • "Data Factory itself is great. It's pretty straightforward. You can easily add sources, join and lookup information, etc. The ease of use is pretty good."
  • "On the UI side, they could make it a little more intuitive in terms of how to add the radius components. Somebody who has been working with tools like Informatica or DataStage gets very used to how the UI looks and feels."

What is our primary use case?

There was a need to bring a lot of CRM and marketing data for some PNL analysis. We are connecting to the Salesforce cloud. In it, there's a specific solution in Salesforce Core CRM for the pharmaceutical industry. We are using the solution to connect to that and we are bringing in the various dimensions and transactions from that data source.

What is most valuable?

Data Factory itself is great. It's pretty straightforward. You can easily add sources, join and lookup information, etc. The ease of use is pretty good. 

They have a lot of other components like a newer monitor, which helps track and generate alerts for any failed jobs and things of that nature, which is helpful.

What needs improvement?

At this point in time, they should work on somehow integrating the big data capabilities within it. I have not explored it, but it would be good if somehow we could call a Spark job or something to do with the Spark SQL within ADS so that we wouldn't need a Spark tested outside.

On the UI side, they could make it a little more intuitive in terms of how to add the radius components. Somebody who has been working with tools like Informatica or DataStage gets very used to how the UI looks and feels. 

In ADS, adding a new table or joining a new table and overriding that with an override SQL that I could customize would be helpful.

Being able to debug from the design mode itself would be helpful.

For how long have I used the solution?

I've been using the solution for one year.

What do I think about the stability of the solution?

In the latest version, the v2 version, the solution is pretty stable. It does not give unknown letters or things like that.

What do I think about the scalability of the solution?

The solution allows you to create reusable components, so it can be scaled pretty easily.

How are customer service and technical support?

Being an IT services company, we have a gold or a platinum partnership with Microsoft. For us, getting the technical support we need is not a big issue. Their community is also pretty active in responding to any issues. It's quite good. We've been satisfied with the level of support that is offered.

How was the initial setup?

We were not actually involved in the initial setup. That was all with the client, so I won't be able to comment on it.

What's my experience with pricing, setup cost, and licensing?

In terms of licensing costs, we pay somewhere around S14,000 USD per month. There are some additional costs. For example, we would have to subscribe to some additional computing and for elasticity, but they are minimal. 

For disaster recovery and readability setups, we did that on Data Lake.

What other advice do I have?

We use the public cloud deployment model.

I'd warn others to ensure that the design should be frozen before you start building because overriding each other's code and managing code takes effort. To avoid or to reduce that effort, ensure that the design is frozen. You can build some configurable code rather than hard-coding everything into the jobs. That's the biggest recommendation.

I'd rate the solution seven out of ten. It's a pretty good solution, but over the past year, I've been limited on the number of cases I have on it. If it had a better user interface and was more intuitive I would have rated it higher.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Microsoft Consultant at a tech services company with 201-500 employees
Consultant
Good support for SAP services and databases, a simple initial setup and good scalability
Pros and Cons
  • "From what we have seen so far, the solution seems very stable."
  • "The solution needs to integrate more with other providers and should have a closer integration with Oracle BI."

What is most valuable?

The support for SAP services and databases, specifically SAP HANA, has been a game-changer for us.

What needs improvement?

It would be helpful if they could adjust the data capture feature so that when there are source-side changes ADF could automatically figure it out.  

The solution needs to integrate more with other providers and should have a closer integration with Oracle BI.

For how long have I used the solution?

I've been working with the solution for just over a year.

What do I think about the stability of the solution?

From what we have seen so far, the solution seems very stable.

What do I think about the scalability of the solution?

The solution is scalable. Right now, we have only three or four people on it. We may increase usage int eh future.

How are customer service and technical support?

We have direct technical support in Israel. In India, we haven't used it yet. However, for other services, the help provided is fine. 

How was the initial setup?

The initial setup was pretty fast and simple. I wouldn't say it was too complex. For deployment, it depends on your environment. In our POC, it didn't take more than two to three hours. In the production environment, however, it takes a bit more time for operations.

What's my experience with pricing, setup cost, and licensing?

In general, we've been satisfied with the pricing. However, since it's a service based on usage, it's really hard to estimate the cost in advance for this product. This makes it hard to quote. It's not so easy to estimate the costs in advance when giving a quote. That makes it difficult to convince some customers because most would like to see a number in advance. 

What other advice do I have?

We primarily use the solution in a hybrid environment. We're Azure partners.

I would suggest others start with a small POC and with a smaller dataset or simple pipeline to test the product before fully implementing it. Users shouldn't hesitate to try it, because there are not so many risks. Practice is the best way to start with this service, as opposed to planning. 

I'd rate the solution eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
DawidDetko - PeerSpot reviewer
Principal Data Architect at Predica
MSP
Very stable, a friendly user interface and easy to set up
Pros and Cons
  • "The user interface is very good. It makes me feel very comfortable when I am using the tool."
  • "The solution should offer better integration with Azure machine learning. We should be able to embed the cognitive services from Microsoft, for example as a web API. It should allow us to embed Azure machine learning in a more user-friendly way."

What is most valuable?

The user interface is very good. It makes me feel very comfortable when I am using the tool.

What needs improvement?

The solution could use some merge statements.

The solution should offer better integration with Azure machine learning. We should be able to embed the cognitive services from Microsoft, for example as a web API. It should allow us to embed Azure machine learning in a more user-friendly way. 

For how long have I used the solution?

I've been using the solution for three years.

What do I think about the scalability of the solution?

The solution is scalable.

How are customer service and technical support?

We've been satisfied with the level of technical support we've received.

How was the initial setup?

The initial setup is very straightforward. The only thing which we had to consider at the beginning was the gateway for the on-premises environment. Otherwise, it's very easy.

What other advice do I have?

We use both the on-premises and cloud deployment models. We typically work with enterprise-level companies.

Azure Data Factory is pretty good but should be considered as an orchestrator, not as an integrated tool. We can use some building components in that tool to orchestrate the entire workflow but if we are thinking about more details, processing, or data modification during the flow, we'd have to consider Azure Databricks or Data Flow for making those calculations or changes. Users will need Azure Data Factory plus third party tools to reach that level of functionally.

I would recommend using Data Factory. I don't have a lot of experience with integration or with integration services, for example, SQL server integration services. However, there are points that should be considered if you are already using SQL server integration services already. You can implement the packages already prepared in Azure Data Factory. It's something that needs to be considered when deciding which technology you are going to use.

I'd rate the solution eight out of ten.

Disclosure: My company has a business relationship with this vendor other than being a customer: Partner.
PeerSpot user
Principal Consultant at a tech services company with 11-50 employees
Consultant
A straightforward solution with a nice interface and ability to integrate with GitHub
Pros and Cons
  • "The solution has a good interface and the integration with GitHub is very useful."
  • "In the next release, it's important that some sort of scheduler for running tasks is added."

What is our primary use case?

We are working on a data warehouse integration which means that I am working on some big data projects. I'm preparing data for the licensing. One of the projects is preparing data in Azure Data Lake, to run some transformation scripts, perform some ETL processing, and to fulfill the stage layer of the data warehouse. It means that I help with ETL use cases.

What is most valuable?

I like the feature that allows you to connect to different sources. You can go through forms and click on them and you don't have to provide the scripts. The solution has a good interface and the integration with GitHub is very useful.

There is also a self-service integration that allows you to run the BTS pages on the SQL server from an SIS. Users have an option to configure this self-service integration engine to run items as a part of a pipeline of the Azure data factory.

What needs improvement?

I think more integration with existing Azure platform services would be extremely beneficial.

In the next release, it's important that some sort of scheduler for running tasks is added.  A built-in scheduling mechanism for running the treasury will be a very helpful improvement.

For how long have I used the solution?

I implement this solution and have been doing so for two years.

What do I think about the stability of the solution?

I've never experienced any issues with the solution's stability.

What do I think about the scalability of the solution?

The scalability of the connected engines makes this solution very scalable.

How are customer service and technical support?

Although I've had to contact support for other Azure products, I haven't had to for this solution, so I can't speak to any experience with technical support directly.

How was the initial setup?

The initial setup is straightforward. It's got a very nice welcome page where everything explained. Depending on the complexity, deployment takes only a few days. You only need one person for the deployment of the solution. This is dependent, however, on how many pipelines you are implementing. Afterward, you would probably only need one person to maintain it.

What about the implementation team?

I'm an integrator, so I work with clients to help them implement the solution.

What's my experience with pricing, setup cost, and licensing?

I'm not sure of the specific cost, but it is monthly and is somewhere in the ballpark of a few hundred Euros. You don't have to pay for extras. Everything is included in the cost.

What other advice do I have?

I don't use Azure Data Factory for my own company; I help clients implement the solution for their companies.

In terms of advice that I would give to others looking to implement the solution, I would say you have to learn. You have to really understand the overall concept. It does not allow you to just click and go.

I would rate this solution ten out of ten.

Disclosure: My company has a business relationship with this vendor other than being a customer: Partner.
PeerSpot user
Consultant at FTS Data & AI
Consultant
Data Flow and Databricks are going to be extremely valuable services
Pros and Cons
  • "This solution will allow the organisation to improve its existing data offerings over time by adding predictive analytics, data sharing via APIs and other enhancements readily."
  • "Data Flow and Databricks are going to be extremely valuable services, allowing data solutions to scale as the business grows and new data sources are added."
  • "The thing we missed most was data update, but this is now available as of two weeks ago."

What is our primary use case?

Used Azure Data Factory, Data Flow (private preview) and Databricks to develop data integration processes from multiple and varied external software sources to an OLTP application Azure SQL database. The tools are impressively well-integrated, allowing quick development of ETL, big data, data warehousing and machine learning solutions with the flexibility to grow and adapt to changing or enhanced requirements. I can't recommend it highly enough.

How has it helped my organization?

This solution will allow the organisation to improve its existing data offerings over time by adding predictive analytics, data sharing via APIs and other enhancements readily. It's as simple as extending the data pipelines with new modules and components. The solution is improving the organisation by offering something the organisation can grow with.

What is most valuable?

Data Flow and Databricks are going to be extremely valuable services, allowing data solutions to scale as the business grows and new data sources are added.

What needs improvement?

Data Flow is in the early stages — currently public preview — and it is growing into a tool that will offer everything other ETL tools offer. There are a few features still to come. The thing we missed most was data update, but this is now available as of two weeks ago. A feature that is confirmed as coming soon is the ability to pass in a parameter and filter, etc.

For how long have I used the solution?

Less than one year.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Azure Data Factory Report and get advice and tips from experienced pros sharing their opinions.
Updated: November 2022
Buyer's Guide
Download our free Azure Data Factory Report and get advice and tips from experienced pros sharing their opinions.