IBM InfoSphere DataStage Reviews

I'm an enterprise and information architect, and we're a customer of IBM. I personally don't use the product but assist companies in the market do for making the integration picture and design.

ETL is the most valuable feature, extract, transform and load. 

I think that performance monitoring could be improved. I know that my colleagues don't give good monitoring. I'm not sure if it's because of the product or because they don't do it normally, but performance monitoring is an issue. I also believe integration with the cloud is not so clear. It's typically a heavy system that people install on-premise. You can install it in the cloud, but it's not so straightforward. You don't find a lot of information unless you go to the IBM cloud. I think IBM is behind in cloud strategy, we would like to put it in the cloud, but there isn't much information about that. 

There are three things that could improve - the cloud, monitoring and cloud integration. It's a solid product but not a modern one and of course it depends what you're looking for. 

I've been using this product for 15 years. 

In the company I'm working with now you can see it's scalable.

We've been using the product a long time and hardly need technical support, which is great. We have very few problems.

I've never done an installation but I believe it's relatively complex, IBM usually is. 

I think DataStage is a product that one should look at as a good candidate in this segment.

I would rate this product a seven out of 10.

My customers use it. I don't use this solution. It's used to extract data from several different sources, as well as to store data and load it into a data warehouse.

The most valuable feature is the product's versatility to inject data.

The interface needs improvement. It is really too technical. That is the main problem.

In the next release, they could offer more connectors with the new database, especially cloud databases.

We've been using this solution for five or six years.

It is very stable.

It depends on project and the customer. Mostly, we have three or four developer users. Maybe three people maintain the solution.

Their technical support is very good.

We used Microsoft Talend previously.

The initial setup is quite complex. Actually, it is probably very quick to deploy, but a difficult project. Typical projects about ETL or extracting some information take dozens of days.

Try to do a lot of training before beginning the project.

I would rate this solution as seven out of ten because I think there are some problems with the design of the interface. They should also develop new connectors. The solution is quite complex for people and you need a lot of training to use it.

My company is a consulting firm and I'm a managing partner. We consult for large size companies in the telecommunications, banking and insurance sectors. We partner with IBM. We're not using the latest version of DataStage but one of the more recent ones. I use the product for a migration project. It's a DB2 to Oracle migration project and the source database is mainframe. I need DataStage for that purpose.

I think the ETL tools are the most valuable features. It has an IBM tool, a friendly UI and it makes things more comfortable. A second good feature is that after you make some ETLs, migrations source to target, DataStage is capable of providing details and extracting data. The push down feature is also a valuable feature. 

The price would be the first thing I would want to change. Reduced cost would allow more customers to choose the product. It's quite expensive in relation to the cost of other similar solutions. I think it would also be helpful if the product was more adaptable to other platforms and vendors. I would also like to see an improvement in support. 

I've been using the product for more than two years. 

The product is stable. 

The product is scalable.

We do sometimes have issues in terms of support with regard to IBM products in this country. It's difficult to get what we want and it could really be improved and made more efficient. 

I recall that the setup was quite difficult. We had to call in some IBM people. It's possible that they have improved that aspect. I know there's a difference in installation depending on whether it's on-prem or cloud. It's possible that the cloud version is easier to install but I don't have experience with that. 

I would rate InfoSphere an eight out of 10. 

It is in the environment of our client, who is a large Russian bank. They are in the top 20, as of August, and have the re-maintenance project of their data warehouse solution based on IBM technologies. They use IBM BWD, a banking data model, on Netezza and DataStages in ETL tools. It is a native case.

We are using the on-premise deployment model.

Our main goal of this project is to increase the efficiency of the usage of this solution and help the bank to get money from the data.

The data lineage report can be filtered for reporting. The reports are user-friendly and take less time to find what you need.

It is a powerful tool with parallel data streams.

The previous project was based on Microsoft SQL. It moved huge amounts of data from different data sources and DataStage to a middle stage, then moved it to Netezza. This created a bottleneck in the solution. We are trying to streamline it and create ETL processes. These will take data exactly from the data sources and move them to Netezza without using of a middle database. The volume of data is quite detailed. We are talking about records in the tens to hundreds of millions. 

We would be happy to see in next versions the ability to return several parameters from jobs. Now, jobs can return just one parameter. If they could return several parameters, that would be great.

We would be happy if the IBM could give us more tolerance for bad networks or VPN channels, as this happens from time to time.

It would be great if we could use more than one SQL operator in the Source DB connector stage. Currently, in the target DB connection stage, we can use several SQL operators, but in the Source DB connector stage we can use only one. It would be better if we could use several.

Data Vault is become more popular. It would be great if it appeared in the newest versions.

I would like them to have more database procedures.

It is quite stable. I haven't seen any pop up errors. It works properly.

They fixed some bugs in version 11.5.02. It works well now.

It is quite scalable. 

DataStage is okay, but the problem of scalability is with another component of the solution (Netezza). The main problem is with the client version of Netezza. IBM stops to support it, then they tell us that we need move to the next version of Netezza. However, the price is too high for the client and we need to look for another platform. 

The client thinks that Datastage can stay in place with another platform.

There are not more than five data analysts and administrators using DataStage because it works at night with ETL processes. Therefore, end users are not using it. The several people who maintain and administer it are the users. 

We have two data specialist who work with it. From the bank, there are about five people who use it.

I haven't used the technical support.

Our client previously used SSIS from Microsoft. They also used Oracle. However, they did not have a special solution for ETL. Ten years ago, they used another data warehouse solution which used XML files as a transport layer.

DataStage is a directly specialized ETL tool which has instruments built for the ETL process as a stream. It can visualize and can track the ETL process, integrating it with the data governance catalog along with other IBM instruments. Previous solutions, except for SSIS, were just a number of scripts which created a process like peer-to-peer. It wasn't a centralized ETL tool with centralized ETL governance.

It was straightforward technically.

Three to four years ago, they decided to start a new data warehouse project. They were working with another Ukrainian company, which engineered this solution. However, the solution hadn't made it to production because of some problems between the understanding of IT and business. They tried to move it to production several times. After that, they decided to do some technical audits for this solution. They& asked us to come and see the solution, then write the audit report, which we did. Then, they asked us what to do with these problems, and this is when we began to help them.

All the components were already in place. We changed it a bit, tweaked the ETL processes, and changed some structures in the data warehouse. This solved the current needs of their business. 

The deployment is continuous. We are working on this project currently. It should take another year. At the moment, we have some Agile processes, in which we are finding new business needs. We try to understand them, then deploy the current user story.

The main problem of this project is they are trying to move the old solution to production in order to begin getting return on investment.

There were no problems with the licensing model for the bank.

It is the best solution in the IBM environment. It uses IBM data models, such as data quality tools.

DataStage itself is a very powerful tool and you have a lot of transformations that you can do. In comparison to Informatica, you can run very complex transactions on it. It's a precise and powerful environment. And when you have ETLs and you all your data documented on the data manager and you have the rest of management from IBM itself on InfoSphere, it's very powerful, especially when you use the whole suite. It helps with end-user technologies and gives you better imaging. 

It's powerful in administration mode. It works well when you're using solutions like DataConnect, IBM CDC, etc.

DataStage works better with Linux operating systems when the application services are hosted on a Linux system equipment, but it's powerful on Windows too.

Also, no matter what language you use, you can always transform the data.

I'm not sure about the latest versions, however, as I'm on 11.3, which is about five years old.

I really like this tool, but the administration should be on the same client application because a lot of administration features are not on the client-side, and they usually need to have administrative access. It's quite complicated to force IT, teams, to have separate administrative access from the developers. 

The platform also needs more stability. It caches a lot. It crashes on the application servers that the host allows on the platform.

The solution needs better online tools for data, or for sourcing data on the internet. They have InfoSphere exchange but it's not as useful for DataStage. 

The stability is related to the operating system your company uses. It crashes occasionally when you're hosting the application servers, but only in the servers on the operating system. With Linux, because Linux itself is very robust, it crashes less. For example, we have a telecom with 40 million users and using Linux it crashed maybe two times a year. However, when the solution was hosted on the Windows platform, it crashed two to three times a month. 

The crashes are related to memory and if it's not automated, you have to deal with it manually. On Windows, you have to release the cache memories manually, but on Linux, you don't have to do that, which is why you get less crashing. 

The solution is very scalable, but at a certain point, it consumes a lot of resources. In order to scale, you need a lot of memory and a lot of people.

The quality of technical support you can expect usually depends on the region. In Egypt, it was not great, but in Jordan, Dubai, and Kuwait, it was good. That was a couple of years ago. I'm not using the solution right now, so I can't say for sure if this is still the case.

The solution has one of the most straightforward setups. It's even easier than Office.

The last version I interacted with was 11.3 because the later versions were cloud-based and usually our customers didn't want to use the solution on the cloud.

In terms of advice, I would give to anyone trying to implement the solution is this: you to have accurate sizing. Clients always do the sizing wrong and they need more experience to get the sizing right. Setting up the environments takes sizing into account but it usually makes a lot of problems if the sizing is poor when it starts to operate. Then you have re-implement and it will require an increase in resources that will change your budget. 

I would rate this solution nine out of ten.

We implement this solution for our customers. The majority of them are Enterprise companies.

This solution is very easy to use because you can design to compile and to run.

The most valuable feature is the ability to transfer information via notes.

The documentation and in-application help for this solution need to be improved, especially for new features. By comparison, in Talend, there is help available for all of the features.

One of my clients has a problem using this solution with MongoDB.

In the next release of this solution, I would like to see the ability to copy and paste schemas. It would be very good because as it is now, you have to save the schema to a repository and then re-load it. It can be done in Talend, but in DataStage, it is not as good.

This is a stable solution. You have to be careful when you install a service pack because sometimes it causes problems. There may be a second service pack to solve problems that were introduced by the first one.

Scaling this solution is not difficult. When you first install you chose what components you need. My clients are enterprise companies, with at least five hundred or a thousand employees.

The are several technical support teams, and the quality of support depends on where the customer is situated. Normally, technical support answers quickly, but it can be improved.

The initial setup is not easy. You have to be an expert to install DataStage.

Sometimes I get calls from clients who ask me to install this solution because their Unix administrator is not able to do it. You have to configure your OS, Database, Web server, and more. There are a lot of things to install.  

If you are not experienced then it is not possible to install.

Small and medium-sized companies cannot afford to pay for this solution.

This solution is for larger companies. Smaller businesses use Talend.

This is a good product, but there is room for improvement.

I would rate this solution an eight out of ten.

We primarily use the solution for ETL jobs and movement of data.

The solution has improved the time it takes to perform tasks related to batch applications.

The parallel job scan has been a very valuable feature.

The solution should be more user-friendly.

The solution is quite stable.

I've never investigated the scalability, but I can say that it only scales on the machine. I can have three machines running in parallel. We have about 20 people using the solution, and they are mostly developers and admins. We mainly use the solution for APL jobs.

On a scale of one to ten, I would give technical support and eight.

We were mostly using ETLs on mainframe jobs.

It's a straightforward process. In terms of deployment from the data stage, it takes about a week.

We use an integrator to assist with implementation.

The advice I would give to others is to make sure they define a framework for development and for management. This could be very useful for the future of the product in the company.

I would rate the entire solution eight out of ten. I really like DataStage. The product fits our requirements perfectly.

We are changing the product now, however, to a cloud-based approach for DataStage.

Complex data integration projects which require integration from multiple data sources.

I have worked during many implementations using DataStage. All of the projects that I worked on have been successful. This is due mainly to the strict discipline around best practices, and by following a set of standards and templates designed to reduce complexity and improve automation, including strong reference architecture.

• Its parallel processing capability allows you to go through extremely large data sets in no time at all, if you do your job right. 
• Highly customizable: Allowing you to handle multiple data latencies (scheduled batch, on-demand, and real-time) in the same job. 
• High scalability: Start small and go big with the same job. You just need to adjust the configuration file, no need to recompile.
• Strong metadata management: Business, technical, and process metadata can all be managed from a single place.
• Ease of integration with other tool sets: Easily supports APIs (or build your own) to support data streaming (or batched) from other systems.
• Data Quality Management from within the tool: Supporting data sampling, including profiling of data, directly from the development canvas.
  

High-cost of ownership: They could take a page from open source software, such as Talend.

Working with some of the big data components is good, but I can see improvements are needed, such as native support for Spark and HBase.

No issues.

No issues.

Support is always good.

Have used quite a few ETL tools in my job.

• Ab Initio: Even pricier, but has a highly competent ETL tool. It is complete, but hard to use. 
• Informatica: Not as flexible and does not support the same level of complexity in its maps.
• Talend: It is a good tool suite, extensive, but can be cumbersome to cite all its pieces.
• ODI: For the Oracle centric world.
• SSIS: Week when compared to any of the above tool sets.

Depends on type of environment that is being installed. I have seen fairly simple to overly complex initial setups due to the environment, not due to the tool.

Both vendor and in-house team implementations:

IBM has top-notch support and tool services along with other partners as well. Depending on the partner, this can go from installation and configuration to solution development, etc.)

Most in-house teams that I have seen tend to have have good developers, but not always good architects. Like most every data integration project, if you do not have a strong architecture, your solution will eventually fail.

Depends on the project.

Have done many ETL tool evaluations based on client requirements. DataStage has always been in the top-three. It may not have been selected due to different weights being used for different sections of the evaluation for different clients, but it has always been in the top-three consistently.

If you have the budget and your solution requires industrial/enterprise strength data integration, this product is always a good choice.