Matillion Data Productivity Cloud Reviews

While loading data into Snowflake, I encountered an issue with the key due to the file's large size and a record count in the billions. Loading the data with a Python script was taking a long time, so I decided to explore other options. This is when I discovered Matillion ETL, which I had not heard of before. I learned more about it and used some of its features, including the Material Data Loader, to load the data into Snowflake. Using Matillion ETL, I was able to load around 770 million records in just five to ten minutes. This was a successful use case, and I have also used Matillion ETL for loading data from Amazon S3 to Snowflake and for other data-loading tasks, including connectivity to on-premise servers and different cloud platforms.

I have used on-premise and cloud deployments of this solution.

The most valuable feature of Matillion ETL is the ETL. The solution is open-source which provides advantages, such as good performance and high efficiency. Additionally, it supports three data types which eliminates predefining the data, and we can write script models in Python.

When using the SQL loader type there were not a lot of pre-processing features for the data. For example, if there is a table with twenty columns, but we only want to load ten columns. In that case, we can use a security script to select the specific columns needed. However, if we want to perform extensive pre-processing of the data, I faced some challenges with Matillion ETL. I did not encounter many challenges, but my overall experience is limited as I only have three years of experience.

The solution could improve by adding support for instructed data types.

I have been using Matillion ETL for approximately two years.

Matillion ETL is a stable solution.

We have a large number of people using the solution for our projects.

The scalability of Matillion ETL is good.

I have not used the support from Matillion ETL. My team was able to solve our problems.

Setting up Matillion ETL was straightforward. When connecting to Amazon AWS, all I had to do was pass the connection string, and it would connect easily. The same was true for on-premise servers. Therefore, from a connection standpoint, it was simple to set up.

The time it takes for the implementation depends on the size of the record data. If the data is large then the process will take longer. However, the time it takes is only a few minutes.

The price of Matillion ETL is reasonable.

The decision to use Matillion ETL depends on the specific requirements. If the requirements can be met without Matillion ETL in a short amount of time, then using it would be unnecessary. However, if dealing with large data sets and frequent data migrations to and from the cloud, then Matillion ETL would be a suitable choice.

I have a lot of experience in this field of data and I was able to achieve results with Matillion ETL that I was not able to with the traditional approach. The solution is helpful for large amounts of data.

I rate Matillion ETL an eight out of ten.

I am not specifically into ETL or data pipelines only since I am basically a data architect who is more into designing cloud-based data solutions. My focus is more on understanding the user's use cases and then designing a solution.

When it comes to Snowflake and Matillion ETL, both offer a lot of compatibility and ease of use. The improvement area could be possible if the tool provides better integration capabilities with other ecosystems, including governance tools or data cataloging tools, as it is currently an area where the solution is lacking.

I have been using Matillion ETL for fifteen years.

Matillion ETL runs the codes in Snowflake, which itself is a highly scalable product, and ultimately provides greater scalability due to the combination of both the products.

I did not contact the solution's technical support team because, most of the time, our company could resolve the issues related to the product with the information available online.

I am not able to recall the products that I have used in the past and be able to make a comparison between those tools against Matillion ETL. I have used Fivetran and DBT tools at some point in time, but I am not able to record when I use them. I know that despite using some other products in the past, my company ended up choosing and recommending Matillion ETL since its features did fit into the requirements of my organization.

The product's initial setup phase was easy. Matillion ETL is readily available from Snowflake's marketplace.

Matillion ETL and DBT help our company by making it easy for us to build data pipelines, particularly when Snowflake was used as a data platform.

Matillion ETL helps my company automate the data pipeline and allows us to optimize data workflows.

In the context of the cloud-based data platforms, the good aspect of Matillion ETL stems from the fact that it actually runs code in the platform, which is basically like a push-down approach wherein, though our company builds the pipeline using the tool, the actual execution of those pipelines happens within Snowflake.

The product supports our company's growing data needs.

The product is integrated with Snowflake. Through a straightforward process, Matillion ETL can be degraded with Snowflake.

The integration capabilities of the tool improved our company's overall data analytics pipeline.

I am satisfied with the product interface.

The product has had an impact on my company's team productivity. It is an easy-to-use tool with a simple UI and serves as a no-code ETL solution.

I recommend the tool to those who specifically use Snowflake in their companies.

I rate the tool an eight out of ten.

We use the solution for the ETL pipelines.

I believe the biggest advantage of using Matillion ETL is its speed of development. We don't have to deal with all the details; we can operate on the component level and do a lot with minimal effort. This is the biggest advantage; for example, we can look for particular components, configure them with some metadata, and simply run it. This simplicity and speed of development make it easy to do things quickly. Additionally, the graphical interface makes it easier to visualize something and find the details we are looking for, rather than going through a number of SQL codes and trying to find the issue. Therefore, the speed of development is the biggest advantage of Matillion.

The positive aspect of this solution is that it provides a graphical interface for jobs. Another advantage of Matillion is its extensibility; if something is missing, it can be easily adjusted or custom components can be written. What I really appreciate about Matillion is that it allows us to schedule jobs, so we can track and monitor their execution on a daily basis. I also like how the solution is organized in Matillion; everything is clear and visible, and we can easily access any details we are interested in. Additionally, there is a useful feature for reporting errors, so we don't have to worry about error handling within the jobs themselves. This feature is very convenient as it saves a lot of time and effort.

Matillion ETL has great Git integration that is perfect and convenient to use.

Unlike Snowflake which automatically takes care of upgrading to the latest version and includes additional features, with Matillion ETL we need to do this ourselves. Matillion upgrades the tool quite often, but we need to manually apply it in our environment. This manual process can be done in a few minutes, but it has room for improvement.

Recently, I needed to develop a component that runs queries on Athena, one of the AWS services. Matillion ETL does not have this functionality out of the box, so having an additional component to handle this would be quite convenient. The tool is quite flexible, and there is no source that cannot be easily integrated. The developers are constantly adding new functionality from release to release, responding to market needs. The only thing I was missing at some point was a component for Athena queries.

I have been using the solution for five years.

The solution has been available for a few years now. Initially, there were some issues, but the support was excellent. If something wasn't working, we could quickly get help to resolve the issue. After a few years of using the solution, it has become very stable. We don't have any problems with Marillion ETL; I haven't experienced any surprises. Matillion ETL is very reliable; whatever we develop works.

The solution is scalable due to its cloud environment. This is the beauty of the cloud; if we require a machine with more power, CPU, and memory, we can do it on the fly. We can simply go to the configuration and change the underlying machine, which requires a quick reboot. The new instance is then set up. This is more of a cloud-related feature than a Matillion ETL feature, but it is very easy to scale. If more power is needed, it can be done quickly and easily. It is also important to note that Matillion is usually connected to a database engine, such as Snowflake, AWS Redshift, Azure Synapse, or Databricks. Most of the processing happens on the database side. However, if there is external work such as loading data from S3 or moving data, there is some load on Matillion ETL. But the majority of the work is done on the database side because it is an ELT-like tool. The data is loaded onto the database and then the transformation happens in most cases. It is up to us how we develop it, but usually, the majority of the power is consumed on the database side.

I have had the opportunity to collaborate with the technical support team a few times and have found them to be extremely helpful. They are very responsive, knowledgeable, and adept at understanding the intricacies of our issue. In my experience, the cooperation has been perfect.

Positive

At the time I joined the company I now work for, two years ago, Matillion ETL had already been introduced. We wanted to switch to something nicer and decided to move to the cloud with AWS Redshift. We also wanted to use a graphical interface for development, rather than lots of code snippets, as it would be faster and easier to maintain. With the graphical approach, it is much easier to spot and go to the root of any issues.

The deployment time depends on the configuration, but with the cloud, everything goes smoothly and we can complete it within two hours.

I believe the cost is customer dependent, so depending on the instance we have and what we can have on-premise, we can have an installation in the cloud, or we can use the hub option. This is what Matillion ETL provides directly. A rough estimation of the cost is around 20,000 dollars a month, however, this is dependent on the machine used and how Matillion ETL is used. If there is only an hour or two of processing per day, it is better to use the hub approach as we only pay for the hours used. If there is ongoing processing, then an installation in the cloud is usually the better option. Ultimately, the cost is dependent on the individual case.

I give the solution a nine out of ten. This is one of the best solutions. Matillion ETL is closely integrated with the cloud environment, which is quite common, and thanks to that, we can take advantage of services available in AWS, GCP, and Azure. I have used a few solutions so far and this is one of the best. Everything works as expected, the tool is very intuitive, the monitoring is very well-developed, and the Git integration is great. From a developer or architect's point of view, it is quite intuitive and nice to use. Matillion ETL is one of the best.

We currently have 15 developers using the solution in our organization.

Depending on the specific case and requirements; when deciding, I would take into consideration what other options are available. From an end-user perspective, I really like Matillion ETL; it is comfortable to work with and easy to set up and maintain. The solution is not perfect, but there are no other similar solutions. The support is also very good and the integration with Git is quite nice, so it is quite flexible. Even if something is not supported out of the box, we can customize components or include Python code. It also depends on the amount of data to be processed, what kind of data it is, and the underlying database engine. There are cases where Matillion ETL makes sense, but there may be cases where it is not recommended. The solution is quite flexible, with the Snowflake dedicated version or Databricks, so we can decide which underlying data warehouse to use. If the company is GCP-related, they may not want to use Redshift, but there is an option for Bitquery. If developers prefer to use Spark, the Databricks version of Matillion ETL would be a nice option. I have been using the solution for a while and I feel comfortable with it. Compared to other tools like DataStage and Informatica, I can say a lot of good things about Matillion ETL, so it is quite convenient for us.

We are using Matillion ETL for extracting and integrating the data from different applications, such as SQL, and other data sources.

The most valuable feature of Matillion ETL is its ease of use. If you have had some experience with other solutions, such as Snowflake, the use of this solution will be simple.

The cost of the solution is high and could be reduced.

I have been using Matillion ETL for approximately six months.

Matillion ETL is a highly stable solution. We are using a stable version of the solution.

I rate the stability of Matillion ETL a nine out of ten.

The solution has limited scalability. However, for concurrent tasks, the solution has been scalable enough for our needs.

I rate the scalability of Matillion ETL an eight out of ten.

The support from the vendor could be better. The speed of the support could improve.

I rate the support of Matillion ETL a six out of ten.

Neutral

I have used Snowflake. However, I am beginning the using these types of tools.

The implementation of Matillion ETL is straightforward. The length of time it takes for the deployment of the solution typically can be completed in a few hours.

We did the deployment of the solution with some assistance.

The price of Matillion ETL is expensive.

For those who want a cloud-based analytics platform, I would recommend this solution.

I rate Matillion ETL an eight out of ten.

It is cloud native and designed to run on cloud warehouses. There is compatibility with many of the cloud data warehouses, as well as Snowflake, and any SQL data warehouse. It is not compatible with other ETL products.

The interface is very intuitive.

I am looking forward to seeing the expansion of the source range for their data loader product. However, I think they have done a very good job of incorporating a lot of different data sources.

I have been working with Matillion ETL for four years now.

Matillion ETL is one hundred percent stable.

I have found that Matillion ETL is infinitely scalable.

Technical support is excellent.

The initial setup was very easy.

I think it is cost conscious. It used to be very cheap and they have more recently bumped up the pricing, so it is competitive now. I would not call it cheap anymore, but it is certainly competitive.

I would rate Matillion ETL a nine out of ten it is a very good product.

I am using Matillion ETL for Snowflake. We are doing a migration project from SAP BODS  
to Matillion for Snowflake. We are migrating all the BODS data flows, workflows, to Matillion jobs.

The customer wanted to move ahead from the Oracle database and SAP BODS. They wanted to move into the cloud for their various data integration and as part of their digitalization. The customer wanted to move to a cloud-based solution. They preferred Matillion ETL for Snowflake.

The loading of data is the most valuable feature of Matillion ETL.

There are certain functions that are available in other ETL tools which are still not present in Matillion ETL. It would be good to have more features.

I have used Matillion ETL within the past 12 months.

The performance of Matillion ETL is good.

The solution is scalable.

Companies can receive a return on investment by using it.

I would recommend Matillion ETL for any cloud-based operations. 

I rate Matillion ETL an eight out of ten.

We are a consulting company. We provide services to our customers. You can use this solution for every use case for which you use an on-premise solution. You can use it for cloud data warehousing in the cloud. You can use it with other cloud databases such as Snowflake or Synapse. We are using its latest version.

The simplicity of this tool is nice. It has a good graphical user interface.

You can also do a lot of generic stuff in the tool. If there is good connectivity to a cloud database, such as Snowflake, and you can have a lot of Snowflake functionality in the tool.

It can have multi-environment support. We should be able to deploy it in different environments. 

Its integration with SAP connection is not so nice, which should be improved. It can also support an on-prem database.

We have been partnering with them for one year.

It is stable.

It is scalable like your cloud solution. We have five to ten customers of this solution.

Their technical support is very good. We are a partner, so we have a different channel. From our point of view, it is very good.

There is no installation. Everything is web-based or browser-based.

Its price depends on what you expect. You pay on a monthly basis, but there is a possibility to have special contracts depending on the installation.

I would recommend this solution. We plan to keep providing it.

I would rate Matillion ETL an eight out of ten.

We're populating a data warehouse, which is the primary focus, but we are also using it to populate our data lake with unstructured data.

It's very intuitive and it's super fast once the data's in the cloud. It's simple and visual, as well. I was self-taught. It's so intuitive and easy to use you can actually just teach yourself how to use it. They also have very good support with their YouTube tutorials. Those show you how to do various things if you need help figuring out aspects of the solution.

Ideally, I would like it to integrate with Secrets Manager as well as the AWS. 

I haven't had any issues with stability.

It's highly scalable. It takes upon itself the Redshift scalability, so it's very good.

We'll be accommodating more data sources going forward. Up until now, it's been a pilot, but we've basically got about another 22 data sources to bring on board. This has been a very successful proof of concept and we will definitely continue using the Matillion going forward.

Technical support has been very helpful. They've got quite an efficient help desk and they've always come back to me with good technical answers within 24 hours, which is very acceptable.

We had previously used a homegrown ETL system and used a combination of IBM Cognos and ClickView as the presentation layout. They both had their own ETL steps inside them. What we're doing now is we're taking all of the data and moving it into one central data lake warehouse, which is sitting in Redshift, and controlling all the ETL in one place, as well. We're looking to switch off the legacy products now.

Essentially, I want all my data sitting in AWS and I wanted one product that could handle everything, that also needed to be lightweight. I didn't want to go with one of these big Informatica's or other similar tools that are out there, which are all licensed based and very expensive. I wanted something lightweight that was easy to use and relatively cheap.

The initial setup is very straightforward. You can set your first job up in literally minutes. We had our first jobs up almost instantly.

For deployment, the latest version that's just come out integrates with Git, so you can do all of your conversions just using the native Redshift code commit, which is the Git repository. You can actually just do it and it's turnkey. It's very easy. You don't need any people, actually. For maintenance, ideally, we would need about a couple of people. 

We have a small team doing all of the data requirements.

We handled the implementation themselves. That's the beauty of the product. We didn't need to get any external help.

It probably is a bit early to measure ROI because at the moment we are having to run all of our processes in parallel and a lot of our existing infrastructure has hidden costs. It definitely will be a lot cheaper going forward and we'll be able to show that. Once we're able to switch off all our legacy servers we'll see very clearly that there's an ROI. 

With on-prem servers, there are always hidden costs, but moving to the cloud makes billing and any extra costs very transparent.

The solution is very cheap. You're paying $2.50 an hour and if you set your service up, which you can do, you're not getting charged. Currently, our ETL process is just an overnight process that runs for about an hour. I can start and stop my server just for an hour if I want to and spent $2.50 a day for an ETL solution. There are no additional costs. It plugs into the AWS marketplace. It's very easy to manage that cost. And if you don't want to use it for a while, you just switch off the EC2 instance and you're not getting billed.

We did have a look briefly at Informatica, but it was just too industrial and over the top in terms of price.

Also, Matillion was the only tool that was fully integrated into Redshift. The other tools were basically bolt-ons onto the cloud, but they weren't using the full extent of the card processing capabilities. For this tool, Matillion is actually using the full processing capabilities of Redshift, which includes what they call Massively Parallel Processing on all of their nodes. It's actually a cloud-native product. It was built specifically for Redshift and that was the main difference. There are other tools we could have used, but they required separate EC2 instances to be commissioned and they just didn't seem to integrate as well into the cloud space.

We're using the latest enterprise version of the solution with the public cloud deployment model.

They're constantly improving themselves. I'm quite pleased with the pace of the number of changes that are coming out all the time.

It's a great product and I think it's very competitive. I would recommend the solution.

I would rate the solution nine out of ten.

• ETL process
• Data warehouse

It has improved the costs of managing my customer’s data.

• Analytics
• Integration with SAP and AWS.

It needs integration with more data sources. I would also like a better UI because it is complicated to manage it.

It makes loading and transforming data on Snowflake very fast, easy, and affordable. It is also stable.

It can scale up/down Snowflake warehouses. My client’s size is a mid-sized environment.

It takes less than five minutes to set up and delivers results. It is much quicker than traditional ETL technologies. It loads data into Snowflake from dozens of sources, including S3 and RDS, multiple databases, and APIs.

Purchasing this solution through the AWS Marketplace was quick and easy. 

It was procured through the AWS Marketplace because it keeps things simple. They offer retail-like checkout and bill through your existing Amazon Web Services account.

The AWS pricing and licensing are a cost-effective solution for data integration needs.

It is perfect for when you need your data to never leave your environment using a robust tool. It also ensures the best security practices are deployed within your AWS account.

I have also integrated this solution with Microsoft Azure, but the best performance is with AWS.

• It enabled an enterprise data-warehouse to be set up and operated, quickly and cheaply.
• The pipeline UI provides a means to present solutions to analysts and non-tech management for review and agreement.

• It works well with AWS Redshift: I have used Informatica, SnapLogic, and Talend and they do not work well with Redshift.
• Mid-pipeline data sampling: Without changes to pipelines, one can quickly and easily track down errors.
• Automatic database-object verification: A fundamental part of how Matillion works is ensuring objects, tables, columns, views, and other metadata are ready and available for use if, and when, a pipeline is started.
• Run-time parameters: These enable pipelines to be organized and modularized with minimum effort.
• An array of source-data components: Pulling data from wide tables, such as SFDC, can be setup in seconds. Using Sqoop to pull data to S3, for example, can take hours or even weeks to get right. It is worth noting that Matillion does not cost more if you need more source-data components (SFDC, Marketo, Google AdWords, RDS, MongoDB, etc.).
• A wide array of in-database DDLs and UDFs accessible from ETL are able to process unstructured data easily, without having to resort to EMR/Hadoop solutions.
• Integration with other AWS services: It can use Amazon SQS or SNS very easily to extend capabilities, such as doing micro-batch imports (near real-time updates) from source-systems.
• Python components using Boto and other libraries running on Matillion’s EC2: Matillion can push data from a data warehouse, via REST APIs, to target systems like DynamoDB and Marketo.

Compared to the likes of traditional ETLs, like Informatica, SnapLogic, and Talend, or even raw Python scripts, this product needs no improvement, as it is so much better.

Any new product like this has teething problems that get solved pretty quickly in the next release. Better user documentation with more examples would be helpful, especially in areas with run-time parameters or JavaScript inserts.

There have been some issues with stability over the first year, but Matillion support is very responsive. I have allowed them to log into our system on occasion.

There are no issues with scalability if one strictly does all transformations in-database, using Redshift’s DDL/SQL.

All the ‘heavy-lifting’ is done by Redshift, as it is MPP. Simply adding more nodes deals with scalability. It is worth noting that Matillion does not cost more if you add more Redshift nodes.

If one uses Python components (as opposed to UDFs), one may encounter scalability issues.

The CPU utilization in WatchTower, of Matillion’s single EC2 (it is not, itself, MPP), will peak. Therefore, it is best to keep a close watch over what your data engineers are doing with Python components.

I would give technical support a rating of 5/5.

We used Informatica, SnapLogic, and Talend. They do not work well with Redshift and they cost more. They do not understand MPP and much of what they do is outside of Redshift, i.e., not in-database.

You need to put them on a bigger EC2 or buy multiple licenses and have multiple EC2s to manage, in order to get scalability.

The initial setup was very straightforward, as it’s all done from the AWS Marketplace. A wizard steps you through the process of setup. Due to Matillion’s clean and clear architecture, there is not much to configure before one is up and running.

Regardless of the quantity of your data, the size of your cluster, or variety of source systems, the price of Matillion is the same.

The only variable that changes what you pay Matillion is the size of your data engineering team.

• If your team is just one or two people, then you can just use the [t2.medium @ $1.37/hr]
• If you have a bigger team, you will need [m4.large @ $2.74/hr] or even [m4.xlarge @ $5.48/hr].

As soon as you can, lock in the yearly discounted price with Matillion, as your level of support availability will increase.

We evaluated Informatica, SnapLogic, Talend, Sqoop, and pure Python scripts. Don’t go with any of these if your data can be categorized as any two of the following: volume, variety, and velocity.

• Experiment and test it ASAP
• Watch all the videos from Matillion
• Join their webinar series
• Talk with existing users

Remember, what Matillion does is simply orchestrate the required jobs. It stands back and lets Redshift do what it is good at.