Arpita-Mishra - PeerSpot reviewer
Specialist Software Engineer at a financial services firm with 10,001+ employees
Real User
Top 10
Faster than other solutions, has multiple connectors, and is easy to set up
Pros and Cons
  • "One advantage of Azure Data Factory is that it's fast, unlike SSIS and other on-premise tools. It's also very convenient because it has multiple connectors. The availability of native connectors allows you to connect to several resources to analyze data streams."
  • "There's no Oracle connector if you want to do transformation using data flow activity, so Azure Data Factory needs more connectors for data flow transformation."

What is our primary use case?

I use Azure Data Factory for architecture creation, for example, loading data from Oracle DB to Azure Synapse Analytics, creating facts and dimensions using Azure Data Pipeline, and creating Azure Synapse notebooks for data transformation. 

Another use case for Azure Data Factory is dashboard creation to help customers make informed decisions.

How has it helped my organization?

Compared to the on-premise SSIS, Azure Data Factory has better infrastructure. It also benefits my company because you can scale the solution up or down with different resources.

Azure Data Factory is also on a pay-as-you-go or pay-as-you-use model, which is suitable for the company because my company only pays for its usage or requirement.

The solution is also very user-friendly, and the Azure Data Factory support team responds quickly whenever my team has a loading issue.

What is most valuable?

One advantage of Azure Data Factory is that it's fast, unlike SSIS and other on-premise tools.

It's also very convenient because Azure Data Factory has multiple connectors. It has sixty connectors which you can't find in SSIS. The availability of native connectors allows you to connect to several resources to analyze data streams.

I also like that you can set up your own VM and infrastructure on Azure Data Factory without any help from the IT team because it only requires a single click.

What needs improvement?

What's missing in Azure Data Factory is an Oracle connector. If you want to connect directly to the Oracle database, you must copy and transform the data. There's no Oracle connector if you want to do transformation using data flow activity, so Azure Data Factory needs more connectors for data flow transformation.

Sending out emails after a job is completed is another area for improvement in the tool.

Buyer's Guide
Azure Data Factory
March 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.

For how long have I used the solution?

I've been using Azure Data Factory for three years.

What do I think about the scalability of the solution?

Azure Data Factory is a scalable tool.

Which solution did I use previously and why did I switch?

We used SSIS, but its on-premise version is slower than Azure Data Factory, and Azure Data Factory, infrastructure-wise, is better, so we went with Azure Data Factory.

How was the initial setup?

The initial setup for Azure Data Factory is an eight out of ten.

Manually deploying Azure Data Factory is easy and doesn't take much time, but I'm not sure how long it takes for an automated approach to deployment.

What's my experience with pricing, setup cost, and licensing?

The licensing model for Azure Data Factory is good because you won't have to overpay. Pricing-wise, the solution is a five out of ten. It was not expensive, and it was not cheap. It's in the middle.

What other advice do I have?

I have experience with both Azure Data Factory and SSIS.

I'm using the latest version of Azure Data Factory.

My rating for Azure Data Factory is eight out of ten.

My company is an Azure Data Factory user.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Data Architect at World Vision
Real User
Top 5Leaderboard
The good, the bad and the lots of ugly
Pros and Cons
  • "The trigger scheduling options are decently robust."
  • "There is no built-in pipeline exit activity when encountering an error."

What is our primary use case?

The current use is for extracting data from Google Analytics into Azure SQL Database as a source for our EDW.  Extracting from GA was problematic with SSIS

The larger use case is to assess the viability of the tool for larger use in our organization as a replacement for SSIS for our EDW and also as an orchestration agent to replace SQL Agent for firing SSIS packages using Azure SSIS-IR.

The initial rollout was to solve the immediate problem while assessing its ability to be used for other purposes within the organization. And also establish the development and administration pipeline process.  

How has it helped my organization?

ADF allowed us to extract Google Analytics data (via BigQuery) without purchasing an adapter.  

It has also helped with establishing how our team can operate within Azure using both PaaS and IaaS resources and how those can interact. Rolling out a small data factory has forced us to understand more about all of Azure and how ADF needs to rely upon and interact with other Azure resources.

It provides a learning ground for use of DevOps Git along with managing ARM templates as well as driving the need to establish best practices for CI.  

What is most valuable?

The most valuable aspect has been a large list of no-cost source and target adapters.

It is also providing a PaaS ELT solution that integrates with other Azure resources. 

Its graphical UI is very good and is even now improving significantly with the latest preview feature of displaying inner activities within other activities such as forEach and If conditions.   

Its built-in monitoring and ability to see each activity's JSON inputs/outputs provide an excellent audit trail.

The trigger scheduling options are decently robust.

The fact that it's continually evolving is hopeful that even if some feature is missing today, it may be soon resolved. For example, it lacked support for simple SQL activity until earlier this year, when that was resolved. They have now added a "debug until" option for all activities. The Copy Activity Upsert option did not perform well at all when I first started using the tool but now seems to have acceptable performance.  

The tool is designed to be metadata driven for large numbers of patterned ETL processes, similar to what BIML is commonly used for in SSIS but much simpler to use than BIML. BIML now supports generating ADF code although with ADF's capabilities I'm not sure BIML still holds its same value as it did for SSIS.

What needs improvement?

The list of issues and gaps in this tool is extensive, although as time goes on, it gets shorter. It currently includes:

1) Missing email/SMTP activity

2) Mapping data flows requires significant lag time to spin up spark clusters

3) Performance compared to SSIS. Expect copy activity to take ten times that of what SSIS takes for simple data flow between tables in the same database

4) It is missing the debug of a single activity. The workaround is setting a breakpoint on the task and doing a "rerun from activity" or setting debug on activity and running up to that point

5) OAuth 2.0 adapters lack automated support for refresh tokens

6) Copy activity errors provide no guidance as to which column is causing a failure

7) There's no built-in pipeline exit activity when encountering an error

8) Auto Resolve Integration runtime should never pick a region that you're not using (should be your default for your tenant)

9) IR (integration runtime) queue time lag. For example, a small table copy activity I just ran took 95 seconds of queuing and 12 seconds to actually copy the data. Often the queuing time greatly exceeds the actual runtime

10) Activity dependencies are always AND (OR not supported). This is a significant missing capability that forces unnecessary complex workarounds just to handle OR situations when they could just enhance the dependency to support OR like SSIS does. Did I just ask when ADF will be as good as SSIS?  

They need to fix bugs. For example:

1) The debug sometimes stops picking up saved changes for a period of time, rendering this essential tool useless during that time

2) Enable interactive authoring (a critical tool for development) often doesn't turn on when enabled without going into another part of the tool to enable it. Then, you have to wait several minutes before it's enabled which is time you're blocked from development until it's ready.  And then it only activates for up to 120 minutes before you have to go through this all over again. I think Microsoft is trying to torture developers

3) Exiting the inside of an activity that contains other activities always causes the screen to jump to the beginning of a pipeline requiring re-navigating where you were at (greatly slowing development productivity)

4) Auto Resolve Integration runtime (using default settings) often picks remote regions (not necessarily even paired regions!) to operate, which causes either an unnecessary slowdown or an error message saying it's unable to transfer the volume of data across regions

5) Copy activity often gets the error "mapping source is empty" for no apparent reason. If you play with the activity such as importing new metadata then it's happy again. This sort of thing makes you want to just change careers. Or tools. 

For how long have I used the solution?

I have been using this product for six months.

What do I think about the stability of the solution?

Production operation seems to run reliably so far, however, the development environment seems very buggy where something works one day and not the next. 

What do I think about the scalability of the solution?

So far, the performance of this solution is abysmal compared to SSIS. Especially with small tasks such as copying activity from one table to another within the same database. 

How are customer service and support?

Customer support is non-existent. I logged multiple issues only to hear back from 1st level support weeks later asking questions and providing no help other than wasting my time. In one situation it was a bug where the debug function stopped working for a couple of days. By the time they got back to me, the problem went away. 

How would you rate customer service and support?

Negative

Which solution did I use previously and why did I switch?

We have been and still rely on SSIS for our ETL. ADF seems to do ELT well but I would not consider it for use in ETL at this time.  Its mapping data flows are too slow (which is a large understatement) to be of practical use to us. Also, the ARM template situation is impractical for hundreds of pipelines like we would have if we converted all our SSIS packages into pipelines as a single ADF couldn't take on all our pipelines. 

How was the initial setup?

Initial setup is the largest caveat for this tool. Once you've organized your Azure environment and set up DevOps pipelines, the rest is a breeze. But this is NOT a trivial step if you're the first one to establish the use of ADF at your organization or within your subscription(s). Instead of learning just an ETL tool, you have to get familiar with and establish best practices for the entire Azure and DevOps technologies. That's a lot to take on just to get some data movements operational. 

What about the implementation team?

I did this in-house with the assistance of another team who uses DevOps with Azure for other purposes (non-ADF use). 

What's my experience with pricing, setup cost, and licensing?

The setup cost is only the time it takes to organize Azure resources so you can operate effectively and figure out how to manage different environments (dev/test/sit/UAT/prod, etc.). Also, how to enable multiple developers to work on a single data factory without losing changes or conflicting with other changes.

Which other solutions did I evaluate?

We operate only with SSIS today, and it works very well for us. However, looking toward the future, we will need to eventually find a PaaS solution that will have longer sustainability.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Azure Data Factory
March 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
Emad Afaq Khan - PeerSpot reviewer
Lead Architect & Scrum Master at a energy/utilities company with 10,001+ employees
Real User
Top 10
A good and constantly improving solution but the Flowlets could be reconfigured
Pros and Cons
  • "Azure Data Factory became more user-friendly when data-flows were introduced."
  • "Azure Data Factory uses many resources and has issues with parallel workflows."

What is our primary use case?

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

How has it helped my organization?

Azure Data Factory didn't bring a lot of good when we were also using Alteryx. Alteryx is user-friendly, while Azure Data Factory uses many resources and has issues with parallel workflows. Alteryx helps you diagnose issues quicker than Azure Data Factory because it's on the cloud and has a cold start debugger.

Azure Data Factory has to wake up whenever you are trying to do testing, and it takes about four to five minutes. It's not always online to do a quick test. For example, if we want to test an Excel file to see if the formatting is correct or why the data-flow or pipeline is failing, we need to wait four to five minutes to get the cold start debugger to run. Compared to Alteryx, Azure Data Factory could be better. Nevertheless, we are using it because we have to.

What is most valuable?

Initially, when we started using it, we didn't like it because it needed to be more mature and had data-flows, so we used the traditional pipeline. After that, Azure Data Factory introduced the concept of data-flows, and it started to become more mature and look more like Alteryx. Azure Data Factory became more user-friendly when data-flows were introduced.

What needs improvement?

They introduced the concept of Flowlets, but it has bugs. Flowlets are a reusable component that allows you to create data-flows. We can configure a Flowlet as a reusable pipeline and plug it inside different data-flows, so we don't have to rewrite our code or visual transformation.

If we make any changes in our data-flow, it reverts all our changes to the original state of the Flowlet. It does not retain changes, and we must reconfigure the Flowlets repeatedly. We had these issues three months ago so things might have changed. It works fine whenever we plug it in and configure it in our data-flow, but if we make minor changes to it, the Flowlet needs to be reconfigured again and loses the configuration.

For how long have I used the solution?

We have used this solution for about a month and a half. It is a cloud-based tool, so there are no versions. It is all deployed on Azure Cloud.

What do I think about the stability of the solution?

Everything is computed inside the SQL server if we're working with pipelines, so we have to be very careful when designing our solution in Azure Data Factory. Alteryx spoiled us because we never cared how it looked in the backend because all the operations were happening on the Alteryx server. But in Azure Data Factory, they run on the capacity of our data warehouse. So Azure Data Factory cannot run your queries, and it directly sends the query to the instance in the SQL server or data warehouse. So we have to be very careful about how we perform certain operations.

We need to have knowledge of SQL and how to optimize our queries. If we are calling a stored procedure, it joins one table in Alteryx. It is pretty easy, and we just put a joint tool. Suppose we want to do it with a stored procedure in the Azure Data Factory. In that case, we have to be very careful about how we write our code. So that is a challenge for our team because we were not looking into how to optimize their SQL queries when fighting queries from Azure Data Factory to the data warehouse.

In addition, the workflows were running very slow, the performance was bad, and some queries were getting timed out because we have a threshold. So we faced many challenges and had to reeducate ourselves on SQL and query optimization.

What do I think about the scalability of the solution?

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

How are customer service and support?

We do not have experience with customer service and support.

How was the initial setup?

It does not require any installation and is more like software as a service. You need to create an instance of Azure Data Factory in Azure and configure some of the connections to your databases. You can connect to your block storages and some authentication is necessary for Azure Data Factory.

The setup is straightforward. It doesn't take much time, and it's on cloud. It requires a few clicks, and you can quickly set it up and grant access to the developer. Then the developer can go to the link and start developing within their browser.

We have a team that takes care of the cloud infrastructure, so we raise a ticket and request infrastructure, and they just exceed it based on the naming convention with the project name.

What about the implementation team?

We have an entire team that takes care of the cloud infrastructure. So we raise a ticket when we need infrastructure, which is executed based on the naming convention for the project name.

What was our ROI?

The nature of our solution is not based on ROI because we are building solutions for other functions within the same organization. In addition, due to the large size of our organization and the services we provide, the ROI is not something we consistently track. It's something discussed with the management, so I can't comment on it.

What's my experience with pricing, setup cost, and licensing?

The cost is based on usage and the computing resources consumed. However, since Azure Data Factory connects with so many different functionalities that Azure provides, such as Azure functions, Logic apps and others in the Azure Data Factory pipelines, additional costs can be acquired by using other tools.

Which other solutions did I evaluate?

We did not evaluate other options because this solution was aligned with out current work environment. 

What other advice do I have?

I rate the solution a seven out of ten. The solution is good and constantly improving, but the concept of Flowlets can be reconfigured to retain the changes we make. I advise users considering this solution to thoroughly understand what Azure Data Factory is and evaluate what's available in the market. Secondly, to assess the nature of the use cases and the kind of products they will be building before deciding to choose a solution.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Dan_McCormick - PeerSpot reviewer
Chief Strategist & CTO at a consultancy with 11-50 employees
Real User
Top 10
Secure and reasonably priced, but documentation could be improved and visibility is lacking
Pros and Cons
  • "The most valuable feature of Azure Data Factory is that it has a good combination of flexibility, fine-tuning, automation, and good monitoring."
  • "They require more detailed error reporting, data normalization tools, easier connectivity to other services, more data services, and greater compatibility with other commonly used schemas."

What is our primary use case?

We use Azure Data Factory for data transformation, normalization, bulk uploads, data stores, and other ETL-related tasks.

How has it helped my organization?

Azure Data Factory allows us to create data analytic stores in a secure manner, run machine learning on our data, and easily adapt to changing schema.

What is most valuable?

The most valuable feature of Azure Data Factory is that it has a good combination of flexibility, fine-tuning, automation, and good monitoring.

What needs improvement?

The documentation could be improved. They require more detailed error reporting, data normalization tools, easier connectivity to other services, more data services, and greater compatibility with other commonly used schemas.

I would like to see a better understanding of other common schemas, as well as a simplification of some of the more complex data normalization and standardization issues.

It would be helpful to have visibility, or better debugging, and see parts of the process as they cycle through, to get a better sense of what is and isn't working.

It's essentially just a black box. There is some monitoring that can be done, but when something goes wrong, even simple fixes are difficult to troubleshoot.

For how long have I used the solution?

I have been working with Azure Data Factory for a couple of years.

There is only one version.

What do I think about the stability of the solution?

Overall, I believe the stability has been good, but there have been a couple of occasions when Microsoft's resources needed to be allocated were overburdened, and we had to wait for unacceptable amounts of time to get our slot. It has now happened twice which is not ideal.

What do I think about the scalability of the solution?

There is no limit to scalability.

We only have a few users. One is a data scientist, and the other is a data analyst.

We use it to push up various dashboards and reports, it's a transitional product for transferring, transforming, and transitioning data.

It is extensively used, and we intend to expand our use.

How are customer service and support?

You don't really get that kind of support; it's more about documentation and the community support that is available. I would rate it a three out of five compared to others.

You could call them, and pay for their consulting hours directly, but for the most part, we try to figure it out or look through documentation. 

I think their documentation is lagging because it's not as popular of a tool, there's just not a lot, or as much to fall back on.

How would you rate customer service and support?

Neutral

Which solution did I use previously and why did I switch?

We had only our own tools, and we switched because you get to leverage all of the work done in a SaaS or platform as a service, or however they classify it. As a result, you get more functionality, faster, for less money.

How was the initial setup?

The initial setup is straightforward.

It is a working tool. You can start using it within an hour and then make changes as needed.

We only need one person to maintain the solution; it doesn't take much to keep it running.

It's not a problem; it's a platform.

What about the implementation team?

We completed the deployment ourselves.

What was our ROI?

We have seen a return on investment. I can't really share many details, but for us, this becomes something that we sell back to our clients.

What's my experience with pricing, setup cost, and licensing?

You pay based on your workload. Depending on how much data you process through it, the cost could range from a few hundred dollars to tens of thousands of dollars.

Pricing is comparable, it's somewhere in the middle.

There are no additional fees to the standard licensing fee.

Which other solutions did I evaluate?

We looked at some other tools, such as Databricks, AmazonGlue, and MuleSoft.

We already had most of our infrastructure connected to Azure in some way. So the integration of where our data resided appeared to be simpler and safer.

What other advice do I have?

I believe it would be beneficial if they could find someone experienced in some of the tools that are a part of this, such as Spark, not necessarily Data Factory specifically, but some of those other tools that will be very familiar and have a very quick time for productivity. If you're used to doing things in a different way, it may take some time because there isn't as much documentation and community support as there is for some more popular tools.

I would rate Azure Data Factory a seven out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Davy Michiels - PeerSpot reviewer
Company Owner, Data Consultant at Telenet BVBA
Real User
Top 5
An expensive data tool for migration with Data Catalog

What is our primary use case?

We use the solution for migration. We collect data from SAP and various other sources, including multiple ERP systems. These ERP systems encompassed different versions of SAP, Dynamics, Navision, and Oracle, presenting a considerable challenge for data integration. The objective was to consolidate all data into Azure Data Factory and Data Warehouse, establishing a structured framework for reporting and analytics. The main hurdle encountered was data ingestion, particularly with SAP data, due to its significant volume. Alternative tools such as PolyBase were utilized to expedite the process, as standard SAP APIs were insufficient for loading data into Azure Data Services. Collaboration with an Azure data engineer facilitated the exploration of alternative ingestion methods. 

What is most valuable?

The most important feature is the Data Catalog. We need to define all the data fields we test. It has technical information in the Data Catalog. The main feature is data ingestion in ADF. We also extended it to PurView because PurView is an extension of the Azure data catalog. It can scan metadata. There is a limitation in ADF when setting up the data catalog.

What needs improvement?

Integration with other tools, such as SAP, could be enhanced. It still has challenges when we talk about different types of structured and non-structured datages. Azure Data Factory has data ingestion issues. There are no delays out of the box. We needed a lot of tools to make the ingestion happen because of the data structure and size of the data.

The transformation we needed to do on data was also not so easy. It was also a long process. We had a bit more capabilities for setting up the Data Catalog, but it still didn't solve the problem from the data ingestion.

For how long have I used the solution?

I have been using Azure Data Factory as a consultant for five years.

What do I think about the stability of the solution?

Sometimes, we experienced some instability, mainly on injection.

I rate the solution’s stability as seven out of ten.

What do I think about the scalability of the solution?

I rate the solution’s scalability an eight out of ten.

How are customer service and support?

The support is very good.

How would you rate customer service and support?

Positive

How was the initial setup?

You need to be experienced in deploying the solution. It's not so easy for a business user. Depending on the use case, it takes around six months to get a proof of concept done.

I rate the initial setup a seven out of ten, where one is easy and ten is difficult.

What's my experience with pricing, setup cost, and licensing?

The pricing is visible because you pay for what you do.

The product looks quite expensive because it charges based on the size of the data. If you're not aware, your cost can be very high. If you are experienced, you know that.

I rate the product’s pricing a seven out of ten, where one is cheap and ten is expensive.

What other advice do I have?

I was mainly focusing on ingestion and cataloging. Data engineers were handling data orchestration.

The tool’s maintenance is easy.

There could be a bit more clarity in the pricing structure. It should be understandable for business users. The cost is is becoming too high because users are unaware of the pricing structure. Secondly, the tool should integrate better with other tools like ERP systems.

Overall, I rate the solution a seven out of ten.

Disclosure: My company has a business relationship with this vendor other than being a customer: MSP
Flag as inappropriate
PeerSpot user
Biswajith Gopinathan - PeerSpot reviewer
Data Analytics Specialist at GlaxoSmithKline
Real User
Top 10
Quick delivery due to drag-and-drop interface
Pros and Cons
  • "One of the most valuable features of Azure Data Factory is the drag-and-drop interface. This helps with workflow management because we can just drag any tables or data sources we need. Because of how easy it is to drag and drop, we can deliver things very quickly. It's more customizable through visual effect."
  • "Data Factory could be improved by eliminating the need for a physical data area. We have to extract data using Data Factory, then create a staging database for it with Azure SQL, which is very, very expensive. Another improvement would be lowering the licensing cost."

What is our primary use case?

My primary use case of Azure Data Factory is supporting the data migration for advanced analytics projects. 

What is most valuable?

One of the most valuable features of Azure Data Factory is the drag-and-drop interface. This helps with workflow management because we can just drag any tables or data sources we need. Because of how easy it is to drag and drop, we can deliver things very quickly. It's more customizable through visual effect. 

What needs improvement?

Data Factory could be improved by eliminating the need for a physical data area. We have to extract data using Data Factory, then create a staging database for it with Azure SQL, which is very, very expensive. Another improvement would be lowering the licensing cost. 

For how long have I used the solution?

I have been using this solution for the past year. 

What do I think about the stability of the solution?

This solution is stable. We are using an Azure subscription, so there is no maintenance or direct updates, it's just always the latest version.

What do I think about the scalability of the solution?

This solution is automatically scalable, since it's in the cloud. At my company, there were more than one thousand people using this solution because we were a big, media-based company. If there are many user requests in the front end application and the system is not responding much or has slow performance, the system will automatically scale up the performance hardware requirements. 

How are customer service and support?

I have contacted technical support. I have never faced an issue like that with Denodo. Fortunately, we got some kind of a tutorial PDF, which helps us to deploy everything quickly. 

Which solution did I use previously and why did I switch?

Before working with Azure, I worked with Python. In the culture I was working in, there was no integration. We were using Pure Python scripting and Python data manipulation tools. For example, we used Python's pandas library, which we coded to transform and orchestrate the data, which is necessary for the endpoint. It was not at all a visual tool. It took more time than Denodo. 

How was the initial setup?

There is no installation because it's on the cloud. You just log on to the cloud with your subscription credentials, then you can use Data Factory directly. 

What about the implementation team?

I implemented through an in-house team. 

What's my experience with pricing, setup cost, and licensing?

Data Factory is very expensive. We are using an Azure subscription, so Data Factory has no direct updates, it's just always the latest version. Compared to Denodo, Azure is very costly. Azure Framework has multiple services, not only Data Factory. So in the cloud-based solution, if you're selecting a particular service, like Data Factory, you need to pay for each request.

Which other solutions did I evaluate?

I also use Denodo. Data Factory is like a transformation layer, but we need an additional staging database or a data storage facility, which is very expensive compared to implementing Denodo. So we extracted the data using Data Factory, then created a staging database with Azure SQL, which cost a huge amount since it's a physical data area. In Denodo, we just implement a layer, which is all handled in Denodo, and not a physical storage mechanism. I prefer customizable data solutions because they improve performance, creativity, and are helpful for front end people.

In comparison to Data Factory's drag-and-drop interface, Denodo developers need to create all the unified views by coding, so we have to create SQL queries to execute. With Data Factory, you can quickly drag and drop data or tables, but in Denodo, it takes more time because you need to code and test and all that.

What other advice do I have?

I rate Data Factory an eight out of ten, mainly because you need a staging database. I recommend Azure to others, but it depends on architecture. In Data Factory, there is no virtualization environment, no layer of virtualization to help integration and doing caching mechanisms. Though Data Factory is there, Denodo is going further. 

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Senior Data Engineer at a tech services company with 1,001-5,000 employees
Real User
Top 5
A tool that offers overall efficiency to its users, particularly in the area of data warehousing
Pros and Cons
  • "I can do everything I want with SSIS and Azure Data Factory."
  • "There aren't many third-party extensions or plugins available in the solution."

What is our primary use case?

In my company, we use Azure Data Factory for everything related to data warehousing. Depending on my customer's wants, I will use SSIS or Azure Data Factory. If my customers want Fivetran, I will use it for them. If the customer wants a suggestion from me on what they should use, then I will look at what they have today and their skills. According to the inputs I receive from my customers, I will recommend what makes more sense for a particular customer. I can be called a software agnostic.

How has it helped my organization?

I can do everything I want with SSIS and Azure Data Factory.

What needs improvement?

There aren't many third-party extensions or plugins available in the solution. Adjunction or addition of third-party extensions or plugins to Azure Data Factory can be a great improvement in the tool. Creation of custom codes, custom extensions, or third-party extensions, like Lookup extension, should be made possible in the tool.

I am unsure if Azure Data Factory bridges the gap between on-premises, cloud, and hybrid solutions. I would like to see a version that would work equally well in both on-premises and cloud environments. I would like to see the aforementioned offerings made to customers as valuable alternatives to the old SSIS tool.

For how long have I used the solution?

I have been using Azure Data Factory for many years. I started using the tool since it was called DTS and then, later, SSIS. I currently use Microsoft SQL SSIS 2019.

How was the initial setup?

The solution is deployed on the cloud.

What other advice do I have?

Overall, I rate the solution an eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Gyanendu Rai - PeerSpot reviewer
Senior Tech Consultant at Crowe
Real User
Top 20
Improved flexibility when compared to other solutions
Pros and Cons
  • "I enjoy the ease of use for the backend JSON generator, the deployment solution, and the template management."
  • "I would like to be informed about the changes ahead of time, so we are aware of what's coming."

What is our primary use case?

Azure Data Factory allows us to provide BI service. We pull the data and put it into Synapse. From there, we create our dimension fact tables that are being used for reporting. 

What is most valuable?

The most valuable feature of Azure Data Factory is the improved flexibility compared to SSIS that we previously used for our ETL transformation. I also enjoy the ease of use for the backend JSON generator, the deployment solution, and the template management. All we need to do is create ARM templates.

What needs improvement?

Microsoft is constantly upgrading its product. Changes can happen every week. Every time you open Data Factory you see something new and need to study what it is. I would like to be informed about the changes ahead of time, so we are aware of what's coming.

In future releases, I would like to see Azure Data Factory simplify how the information of logs is presented. Currently, you need to do a lot of clicks and go through steps to find out what happened. It takes too much time. The log needs to be more user-friendly.

For how long have I used the solution?

I have been using Azure Data Factory for two years.

What do I think about the scalability of the solution?

Scalability depends on the use case. 

How are customer service and support?

As far as customer service and support with Azure Data Factory, we are not always satisfied with the response time. However, once they attend to the issue, everything is good.

How would you rate customer service and support?

Positive

How was the initial setup?

All we needed to do was create ARM templates and deployment is easy.

What about the implementation team?

We deployed in-house. For deployment, we use ARM templates that are a part of Azure's deployment strategy. It's not only available for Data Factory, it is built in. It links with DevOps, then the ICD integration.

What other advice do I have?

Azure Data Factory is a good tool. Given that the data platform ecosystem is provided by Microsoft, you know it is good.

I would rate the solution an eight out of ten overall.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Buyer's Guide
Download our free Azure Data Factory Report and get advice and tips from experienced pros sharing their opinions.
Updated: March 2024
Buyer's Guide
Download our free Azure Data Factory Report and get advice and tips from experienced pros sharing their opinions.