Azure Data Factory Primary Use Case

AS
CTO at a construction company with 1,001-5,000 employees

Our company uses the solution as a data pipeline. We get information outside the cloud from our factory such as data relating to production. We categorize it, clean it up, and transfer it to a database and data model. From there, we analyze the data using BI and other things. We gather information in data lake products like Microsoft Synapse and Microsoft Data Lake. 

We have two to three administrators who use the solution in a quite standard, mainstream way with nothing extreme. They handle administration, security, and development. 

It is difficult to define the total number of users because that depends on the number of data factory agents. We built the solution to have a different data factory agent for every customer. For example, if we have ten customers then we have ten users. We hope to increase usage but growth depends on our marketing efforts and how well we sell our products. 

View full review »
Camilo Velasco - PeerSpot reviewer
CTO at Sosty

The primary use case of this solution is to extract ETLS, transform and load data, and organize database synchronization.

View full review »
PiyushAgarwal - PeerSpot reviewer
Associate Specialist at Synechron

We are currently migrating from on-prem to the cloud, and our on-prem tables are getting data from upstream. We used ADF to build a pipeline to facilitate this migration. A team of 15-20 people currently uses ADF, and more will join once it goes live.

View full review »
Buyer's Guide
Azure Data Factory
March 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
Rama Subba Reddy Thavva - PeerSpot reviewer
Project Lead at Mercedes-Benz AG

I can't go into specifics about the use case for Azure Data Factory, but it's for analytics related to an assessment.

View full review »
Rohit Sircar - PeerSpot reviewer
Integration Solutions Lead | Digital Core Transformation Service Line at Hexaware Technologies Limited

Our primary use case for the solution is data integration and we deploy it only on Azure.

View full review »
Anil Jha - PeerSpot reviewer
Director D&A at Iris Software Inc.

The primary use case is integrating data from different ERP systems and loading it into Azure Synapse for reporting. We use Power BI for the reporting side of it.

We also have customers who are migrating to Azure Data Factory and we are assisting them with making the transition.

View full review »
reviewer826419 - PeerSpot reviewer
CIO, Director at Prosys Infotech Private Limited

The primary use case is to connect to various different data sets and do an EAT into our data warehouse.

View full review »
Anirban Bhattacharya - PeerSpot reviewer
Practice Head, Data & Analytics at a tech vendor with 10,001+ employees

Azure Data Factory can be deployed on the cloud and hybrid cloud. There have been very few deployments on private clouds.

View full review »
Arpita-Mishra - PeerSpot reviewer
Specialist Software Engineer at a financial services firm with 10,001+ employees

I use Azure Data Factory for architecture creation, for example, loading data from Oracle DB to Azure Synapse Analytics, creating facts and dimensions using Azure Data Pipeline, and creating Azure Synapse notebooks for data transformation. 

Another use case for Azure Data Factory is dashboard creation to help customers make informed decisions.

View full review »
GM
Data Architect at World Vision

The current use is for extracting data from Google Analytics into Azure SQL Database as a source for our EDW.  Extracting from GA was problematic with SSIS

The larger use case is to assess the viability of the tool for larger use in our organization as a replacement for SSIS for our EDW and also as an orchestration agent to replace SQL Agent for firing SSIS packages using Azure SSIS-IR.

The initial rollout was to solve the immediate problem while assessing its ability to be used for other purposes within the organization. And also establish the development and administration pipeline process.  

View full review »
Emad Afaq Khan - PeerSpot reviewer
Lead Architect & Scrum Master at a energy/utilities company with 10,001+ employees

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

View full review »
Dan_McCormick - PeerSpot reviewer
Chief Strategist & CTO at a consultancy with 11-50 employees

We use Azure Data Factory for data transformation, normalization, bulk uploads, data stores, and other ETL-related tasks.

View full review »
Davy Michiels - PeerSpot reviewer
Company Owner, Data Consultant at Telenet BVBA

We use the solution for migration. We collect data from SAP and various other sources, including multiple ERP systems. These ERP systems encompassed different versions of SAP, Dynamics, Navision, and Oracle, presenting a considerable challenge for data integration. The objective was to consolidate all data into Azure Data Factory and Data Warehouse, establishing a structured framework for reporting and analytics. The main hurdle encountered was data ingestion, particularly with SAP data, due to its significant volume. Alternative tools such as PolyBase were utilized to expedite the process, as standard SAP APIs were insufficient for loading data into Azure Data Services. Collaboration with an Azure data engineer facilitated the exploration of alternative ingestion methods. 

View full review »
Biswajith Gopinathan - PeerSpot reviewer
Data Analytics Specialist at GlaxoSmithKline

My primary use case of Azure Data Factory is supporting the data migration for advanced analytics projects. 

View full review »
PA
Senior Data Engineer at a tech services company with 1,001-5,000 employees

In my company, we use Azure Data Factory for everything related to data warehousing. Depending on my customer's wants, I will use SSIS or Azure Data Factory. If my customers want Fivetran, I will use it for them. If the customer wants a suggestion from me on what they should use, then I will look at what they have today and their skills. According to the inputs I receive from my customers, I will recommend what makes more sense for a particular customer. I can be called a software agnostic.

View full review »
Gyanendu Rai - PeerSpot reviewer
Senior Tech Consultant at Crowe

Azure Data Factory allows us to provide BI service. We pull the data and put it into Synapse. From there, we create our dimension fact tables that are being used for reporting. 

View full review »
SK
Azure Architect\Informatica ETL Developer at Relativity

I use this primarily for ETL tasks.

View full review »
AS
Solution Architect at a computer software company with 1,001-5,000 employees

We use the product for data warehouses. It helps us to load data to warehouses. 

View full review »
Zubair_Ahmed - PeerSpot reviewer
Senior Consultant at Veraqor

My task involves extracting data from a source, performing necessary transformations, and subsequently loading the data into a target destination, which happens to be Azure SQL Database.

View full review »
PK
BI Technical Development Lead at a energy/utilities company with 10,001+ employees

Our company uses the solution to extract, transform, and load the data into the database.

View full review »
PedroNavarro - PeerSpot reviewer
BI Development & Validation Manager at JT International SA

We use this solution to perform ELTs so that we do not need to keep code within a database.

View full review »
AM
Senior Devops Consultant (CPE India Delivery Lead) at a computer software company with 201-500 employees

Azure Data Factory is an all-in-one solution for ETL in our company.

My company doesn't use the product for development purposes.

I use the solution in my company as an ETL tool and for orchestration.

View full review »
KP
Director of Business Intelligence Analytics at The Gibraltar Group - Insurance Services

We primarily use the solution for data warehousing. I'm integrating data from multiple sources.

View full review »
Kevin McAllister - PeerSpot reviewer
Executive Manager at Hexagon AB

I primarily use Data Factory to ingest data. For example, if we need to pull data into our data warehouse from somewhere like Azure Event Hub or salesforce.com.

View full review »
Aurora Calderon - PeerSpot reviewer
Experienced Consultant at Bluetab

My clients use Data Factory to exchange information between the on-premises environment and the cloud. Data Factory moves the data, and we use other solutions like Databricks to transform and clean up the data. My teams typically consist of three or four data engineers.

View full review »
Charles Nordine - PeerSpot reviewer
Senior Partner at Collective Intelligence

We created data ingestion solutions. We have built interpreters, and we have data factories that pull data from our clients. They submit data via Excel spreadsheets, and we process them into a common homogeneous format.

View full review »
Sunil Singh - PeerSpot reviewer
Lead Engineering at GlobalLogic

Depending on their pipeline, our customers use Azure Data Factory for their ELT or ETL transformation processes.

View full review »
Pavan Yogender - PeerSpot reviewer
Founder and CEO at Zertain

We use Azure Data Factory to set up a data lake information repository.

View full review »
Mano Senaratne - PeerSpot reviewer
Management Consultant at a consultancy with 201-500 employees

As a management consultancy company, we help our clients deploy Azure Data Factory or any other cloud-based solution depending on data integration needs. Regarding how we use Azure Data Factory within our company, we are on the Microsoft Stack, so we use the solution primarily for data warehousing and integration.

View full review »
Vishnu Derkar - PeerSpot reviewer
Sr. Big Data Consultant at a tech services company with 11-50 employees

We primarily use the solution in a data engineering context for bringing data from source to sink.

View full review »
DD
PRESIDENT at a computer software company with 51-200 employees

We use Azure Data Factory to connect to clients' on-premise networks and data sources to bring the data into Azure. Additionally, Azure Data Factory orchestrates data movement and transformations. It can connect to a number of different cloud data sources to bring the information into something, such as a data lake or a formal SQL database. Azure Data Factory has the ability to handle large data workloads and can orchestrate them well.

View full review »
TD
Data engineer at Target

My primary use cases for this solution are integration and connecting to the different data stores where we get data and migration activity, deployment, and integrations into using linked services and deployment models.

View full review »
AS
Senior Manager at a tech services company with 51-200 employees

My primary use case is getting data from the sensors.

The sensors are installed on the various equipment across the plant, and this sensor gives us a huge amount of data. Some are captured on a millisecond basis.

What we are able to do is the data into Azure Data Factory, and it has allowed us to scale up well. We are able to utilize that data for our predictive maintenance of the assets of the equipment, as well as the prediction of the breakdown. Specifically, we use the data to look at predictions for future possible breakdowns. At least, that is what we are looking to build towards.

View full review »
Joaquin Marques - PeerSpot reviewer
CEO - Founder / Principal Data Scientist / Principal AI Architect at Kanayma LLC

We use this solution to quickly instantiate certain components in Azure with the aim of having consistent use of certain components and objects. It provides one point of reference rather than having the need to replicate points of references, and then having to keep them in sync. 

View full review »
Katarzyna Palikowska - PeerSpot reviewer
ETL Developer at Det Norske Veritas

I mainly use Data Factory to load data for ETL processes or to Azure Storage and for testing purposes in our business unit.

View full review »
VT
Solution Architect at Giant Eagle

We use Azure Data Factory for data integration.

View full review »
MG
Lead BI&A Consultant at a computer software company with 10,001+ employees

We had an old, traditional data warehouse. We decided to put it into the cloud and we used Azure Data Factory to reform the EEL process from SQL server integration services to extra data.

View full review »
AN
.NET Architect at a computer software company with 10,001+ employees

I use Azure Data Factory in my company because we are implementing a lot of different projects for a big company based in the USA. We're getting certain information from different sources—for example, some files in the Azure Blob Storage. We're migrating that information to other databases. We are validating and transforming the data. After that, we put that data in some databases in Azure Synapse and SQL databases.

View full review »
TF
General Manager Data & Analytics at a tech services company with 1,001-5,000 employees

The solution is primarily used for data integration. We are using it for the data pipelines to get data out of the legacy systems and provide it to the Azure SQL Database. We are using the SQL data source providers mainly.

View full review »
MR
Sr. Technology Architect at a tech services company with 10,001+ employees

There was a need to bring a lot of CRM and marketing data for some PNL analysis. We are connecting to the Salesforce cloud. In it, there's a specific solution in Salesforce Core CRM for the pharmaceutical industry. We are using the solution to connect to that and we are bringing in the various dimensions and transactions from that data source.

View full review »
Sarath Boppudi - PeerSpot reviewer
Data Strategist, Cloud Solutions Architect at BiTQ

Our primary use case is for traditional ETL; moving data from the web and data sources to a data warehouse. I've also used Data Factory for batch processing and recently for streaming data sets. We have a partnership with Microsoft and I am a cloud solution architect.

View full review »
BS
Chief Analytics Officer at Idiro Analytics

We use Data Factory for automating ETL processes, data management, digital transformation, and scheduled automated processes. My team has about 11 people, and at least five use Data Factory. It's mostly data engineers and analysts. 

Each data analyst and engineer manages a few projects for clients. Typically, it's one person per client, but we might have two or three people managing and building out pipelines for a larger project.  

View full review »
HF
Data engineer at Inicon S.r.l.

Azure Data Factory is an integration tool, an orchestration service tool. It’s for data integration for the cloud.

View full review »
AS
Enterprise Architect at TechnipEnergies

We are not using this product specifically as a data factory. We have taken Synapse Analytics as the entire component for the data warehousing solution. Azure Data Factory is one of the components of that, and we are using it for ETL. 

View full review »
Richard Griffin - PeerSpot reviewer
Manager Data & Analytics at Fletcher Building

I am a manager of a team that uses this solution.

Azure Data Factory is primarily used for data integration, which involves moving data from sources into a data lake house called Delta Lake.

View full review »
RD
Chief Technology Officer at cornerstone defense

Our customers use it for data analytics on a large volume of data. So, they're basically bringing data in from multiple sources, and they are doing ETL extraction, transformation, and loading. Then they do initial analytics, populate a data lake, and after that, they take the data from the data lake into more on-premise complex analytics.

Its version depends on a customer's environment. Sometimes, we use the latest version, and sometimes, we use the previous versions.

View full review »
Abdelmonem Metwally - PeerSpot reviewer
ETL/BI Senior Consultant at Qrious

We mainly use this solution to carry out data movement and transformation.

View full review »
MM
Senior Software Developer at a insurance company with 10,001+ employees

I'm not sure how much information I can provide regarding to some kind of security of my company, but I can tell you that we were migrating integrations from from platform to the other, and the other platform was error data factory.

View full review »
SS
Principal at a tech services company with 51-200 employees

Our company uses the solution for data ingestion. 

View full review »
BM
IT Analyst at a tech vendor with 10,001+ employees

It's a PaaS service. It's a hybrid solution. The cloud provider is Microsoft.

We are not using Azure Data Factory as for users. Rather, we're using it as a process base. We're just using it for orchestration, not for any kind of ETL stuff.

We have plans to increase usage. It's going to take a major role in any kind of traditional data warehousing. It has big potential, especially as a PaaS offering.

View full review »
BM
Azure Technical Architect at a computer software company with 10,001+ employees

It's an integration platform, we migrate data across hybrid environments. We have data in our cloud environment or on-prem system so we use it for when we want to integrate data across different environments. It was a problem for us to get data from different hybrid environments.

View full review »
VH
Network Team Lead at a computer software company with 1,001-5,000 employees

We use the solution to move data from on-premises to the cloud.

View full review »
AK
Director Technology at a computer software company with 10,001+ employees

I primarily use Data Factory for creating pipelines on cloud in terms of integrating multiple cloud services.

View full review »
LB
IT Functional Analyst at a energy/utilities company with 1,001-5,000 employees

We are currently using it as an ETL (Extract, Transform, and Load) tool. We are using it to connect to various information providers or, in general, to various sources, to extract data, and then to insert it to our storage devices, databases, or data warehouses.

View full review »
CarlosAraque - PeerSpot reviewer
Data Warehouse Analyst at a individual & family service with 201-500 employees

I primarily use Data Factory for data ingestion and B2B transformation.

View full review »
AS
CTO at a construction company with 1,001-5,000 employees

We are using this solution to gather information from SCADA systems, analyze it using AI and machine learning, and then sending the results to our users. They receive and view the data using the Power BI interface.

View full review »
GV
Head of IT at a logistics company with 10,001+ employees

The use cases are more related to logistics, our finance, and back-office activity.  

View full review »
BM
Azure Technical Architect at a computer software company with 10,001+ employees

The primary use case of this solution is for data integration.

View full review »
MK
Technical Director, Senior Cloud Solutions Architect (Big Data Engineering & Data Science) at NorthBay Solutions

We generally implement this product for data transformation for our clients. We create the pipelines and provide training before handing it over to them. We generally deal with large-scale organizations. I'm a senior solutions architect. 

View full review »
GS
Senior Consultant at a computer software company with 1,001-5,000 employees

Our primary use case is mainly for ETL and transforming the data and then using it for power VA. So there we are handling multiple projects. It is not just a single thing, but mainly used at the end is data analytics only.

View full review »
HS
Head of Product at Tata Consultancy

When it comes to our business requirements, this solution has worked well for us. However, we have not stretched it to the limit. We use this solution to collect data from multiple data sources and store it in the cloud. We are using it every day and sometimes multiple times a day. 

View full review »
SS
Director of Product Management at EIM solutions

Our primary use case is data loading.

View full review »
TB
Consultant at FTS Data & AI

Used Azure Data Factory, Data Flow (private preview) and Databricks to develop data integration processes from multiple and varied external software sources to an OLTP application Azure SQL database. The tools are impressively well-integrated, allowing quick development of ETL, big data, data warehousing and machine learning solutions with the flexibility to grow and adapt to changing or enhanced requirements. I can't recommend it highly enough.

View full review »
LG
Principal Consultant at a tech services company with 11-50 employees

We use Data Factory in our projects, which we deliver to customers. We have almost five implementations in which we're using Data Factory.

It's cloud-based, but there's the integration runtime, which you can connect to on-premises sources. You can transfer data from on-premises to the cloud. We're integrating on-premise data sources to cloud data-based services, like rescue or finance and so on.

Azure Data Factory is very good for enterprise organizations like banks and international insurance companies.

It's cloud-based.

View full review »
GB
Principal Engineer at a computer software company with 501-1,000 employees

We primarily used this solution for getting data from a client's server, or online data, to an Azure Data Lake. We create pipelines to orchestrate the data flow from source to target.

View full review »
NT
Senior Systems Analyst at a non-profit with 201-500 employees

We use the solution to gather data from four separate operational systems.

View full review »
Sarath Boppudi - PeerSpot reviewer
Data Strategist, Cloud Solutions Architect at BiTQ

I primarily use the solution for my small and medium-sized clients.

View full review »
KS
Director at a tech services company with 1-10 employees

Azure Data Factory is for data transformation and data loading. It works from your transaction systems, and we are using it for our HRMS, Human Resource Capital Management System. It picks up all the transactional data pick and moves into the Azure Data Warehouse. From there, we would like to create reports in terms of our financial positions and our resource utilization project. These are the reports that we need to build onto the warehouse.

The purpose of Azure Data Factory is more about transformations, so it doesn't need to have a good dashboard. But, it has a feeding user interface for us to do our activities and debug actions. I think that's good enough.

View full review »
NK
Delivery Manager at a tech services company with 1,001-5,000 employees

We are a tech services company and this is one of the tools that we use when implementing solutions for our clients. I am currently managing a team that is working with the Azure Data Factory.

Our clients that use this solution are migrating their data from on-premises to the cloud.

One of our clients is building an integrated data warehouse for all of their data, using this solution. It is used to extract all of the data from different servers and store it into one place.

View full review »
LS
Enterprise Data Architect at a financial services firm with 201-500 employees

In my new organization, we are currently analyzing this solution. In my previous organization, I have worked with Azure Data Factory and Databricks.

It is deployed in a virtual private cloud.

View full review »
RK
Business Unit Manager Data Migration and Integration at a tech services company with 201-500 employees

We use this solution for data integration. We use it to feed operational data into a data warehouse. We also use it for creating connections between applications.

Within our organization, there are a few thousand users of Azure Data Factory.

We believe that the number of customers and usage of this product will extend over the next few years. For this reason, we invest a lot of resources in building skills, and we make sure to hire consultants who know their way around Data Factory.

View full review »
LG
Principal Consultant at a tech services company with 11-50 employees

We are working on a data warehouse integration which means that I am working on some big data projects. I'm preparing data for the licensing. One of the projects is preparing data in Azure Data Lake, to run some transformation scripts, perform some ETL processing, and to fulfill the stage layer of the data warehouse. It means that I help with ETL use cases.

View full review »
Buyer's Guide
Azure Data Factory
March 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.