erwin Data Intelligence by Quest Reviews

Our primary use case is that we want to enable self-service for different business teams to be able to find different data. We are using erwin Data Intelligence platform to enable data literacy and to enable different users to be able to find the data by using the data catalog.

It can be hosted on-premise or in the cloud. We chose to run it in the cloud because the rest of our analytics infrastructure is running in the cloud. It made natural sense to host it in the cloud.

I represent IT teams, and a lot of times different business teams want to do data analysis. Before using erwin Data Intelligence Suite, they used to constantly come to IT teams to understand things like how is the data is organized and what type of queries or table they should use. It used to take a lot of my team's time to answer those questions. Some of those questions were pretty repetitive. With erwin Data Intelligence Suite, they can now do a self-service. There is a business user portal using which they can search different tables. They can do the search in different ways. If they already know the table name, they can just directly search for that table name, and they will find the definition of each column there, and that would help them in understanding how to use that table. In some cases, they may not know the exact table name, but they may know, for example, a business metric. In such a case, they can search by using a business metric, and, inside the tool, they can link those business metrics to the underlying tables from which these metrics get calculated. They can get to the table definitions through that route as well. This is helping all of our business analysts to do the self-service analytics, and, at the same time, we can enforce some governance around it. Because we enabled self-service for different business analysts, it has improved the speed. It has easily reduced at least 20% of the time that my IT team had to spend answering questions from different business teams. The benefit is probably even more for business teams, and I think they are faster by at least 30% in terms of being able to get the data that they need and perform their analysis based on that. I would expect at least 25% savings in time.

It has a big impact in terms of the transparency of the data. Everybody is able to find the data by using the catalog, and they can also see how the data is getting loaded through different tables. It has given a lot of transparency. Based on this transparency, we were able to have a good discussion about how the data should be organized so that we can collaborate better with the business in terms of data organization. We were also able to change some of our table structures, data models, etc.

By using the data catalog, we have definitely improved in terms of maturity as a data-driven decision-maker organization. We are now getting to a level where everybody understands the data. Everyone understands how it is organized, and how they can use this data for different business decisions. The next level for us would be to go and use some of these advanced features such as AI Data Match.

In terms of the effect of the data pipeline on our speed of analysis, understanding the data pipeline and the data flow is helpful in identifying a problem and resolving it quickly. A lot of times there is some level of ambiguity, and businesses don't understand that how the data flows. Understanding the data pipeline helps them in quickly identifying the problems. They can solve the identified problems and bottlenecks in the data flow. For example, they can identify the data set that is required for a specific analysis and then bring in the data from another system. 

In terms of the money and time that the real-time data pipeline has saved us, it is hard to quantify the amount in dollars. In terms of time, it has saved us 25% time on the analysis part.

It has allowed us to automate a lot of stuff for data governance. By using Smart Data Connectors, we are automatically able to pull metric definitions from our reporting solution. We are then able to put an overall governance and approval process on top of that. Whenever a new business metric needs to be created, the data stewards who have Write access to the tool can go ahead and create those definitions. Other data stewards can peer-review their definitions. Our automated workflow then takes that metric to an approved state, and it can be used across the company. We have definitely built a lot of good automation with the help of this tool.

It definitely affects the quality of data. A lot of times, different teams may have different levels of understanding, and they might have different definitions of a particular metric. A good example is the customer lifetime value. This metric is used by multiple departments, but each department can have its own metric definition. In such a case, they will get different values for this metric, and they won't make consistent decisions across. If they have a common definition, which is governed by this tool, then everybody can reference that. When they do the analysis, they will get the same result, which leads to a better quality of decision-making across the company.

It affects data delivery in terms of making correct decisions. Ultimately, we are using all of this data to get some insights and then make decisions based on that. It is not so much of the cost but more of the risk that it affects.

We are a consultant for erwin Data Intelligence by Quest and provide the service to our customers for data access discovery.

The standard data connectors for automation, metadata, harvesting, and ingestion are easy to use.

The solution enables us to deliver data pipelines faster and with a 20 to 30 percent reduction in cost.

erwin provides an immediate view of the technical details required to manage our data landscape. Data transparency is increasing which helps our IT operations.

It delivers up-to-date and detailed data lineage which is important.

erwin helps us with data discovery and understanding of our entire organization's data.

The solution provides visibility into our organization's data for our IT, data governance, and business users which we require to build the compound report.

The asset discovery and collaboration provided by the data quality feature is good.

Its ability to affect data users' confidence levels when they are utilizing data is admirable.

erwin's capacity to tackle challenges associated with data quality and offer the required information for users to make well-informed decisions is effective in overseeing the regional database and metadata.

The Data Intelligence suite helps us manage all our data governance activities around setting up metadata, data lineage, business clusters, and business metadata definitions. These are all managed in this tool. There is definitely extended use of this data set, which includes using the metadata that we have built. Using that metadata, we also integrate with other ETL tools to pull the metadata and use it in our data transformations. It is very tightly coupled with our data processing in general. We also use erwin Data Modeler, which helps us build our metadata, business definitions, and the physical data model of the data structures that we have to manage. These two tools work hand-in-hand to manage our data governance metadata capabilities and support many business processes.

I manage the data architecture plus manage the whole data governance team who designed the data pipelines. We designed the overall data infrastructure plus the actual governance processes. The stewards, who work with the data in the business, set up the metadata and manage this tool everyday end-to-end.

The benefit of the solution was the adoption of a lot of business partners using and leveraging our data through our governance processes. We have matrices of how many users have been capturing and using it. We have data consultants and other data governance teams who are set up to review these processes and ensure that nobody is really bypassing them. We use this tool in the middle of our work processes for utilization of data on the tail-end, letting the business do self-service, and build our own IT things.

When we manage our data processes, we know that there are some upward sources or downstream systems. We know that they could be impacted based on some changes coming in from the source or some related to the lineage and impact analysis that this tool brings to the table. We have been able to identify system changes which could impact all downstream systems. That is a big plus because IT and production support teams are now able to use this tool to identify the impact of any issues with the data or any data quality gaps. They can notify all the recipients upfront with the product business communications of any impacts.

For any company mature enough to have implemented any of these data governance rules or principles, these are the building blocks of the actual process. The criticality is such because we want the business to self-service. We can build data lakes or data warehouses using our data pipelines, but if nobody can actually use the data to be able to see what information they have available without going through IT sources, that defeats the whole purpose of doing this additional work. It is a data platform that allows any business process to come in and be self-service, building their own processes without a lot of IT dependencies.

There is a data science function where a lot of critical operational reporting can be done. Users leverage this tool to be able to discover what information is available, and it's very heavily used.

If we start capturing additional data about some metadata, then we can define our own user-defined attributes, which we can then start capturing. It does provide all the information that we want to manage. For our own processes, we have some special tags that we have been able to configure quickly through this tool to start capturing that information.

We have our own homegrown solutions built around the data that we are capturing in the tool. We build our own pipelines and have our own homegrown ETL tools built using Spark and cloud-based ecosystems. We capture all the metadata in this tool and all the transformation business rules are captured there too. We have API-level interfaces built into the tool to pull the data at the runtime. We then use that information to build our pipelines.

This tool allows us to bring in any data stewards in the business area to use this tool and set up the metadata, so we don't have to spend a lot of time in IT understanding all the data transformation rules. The business can set up the business metadata, and once it is set up, IT can then use the metadata directly, which feeds into our ETL tool.

Impact analysis is a huge benefit because it gives us access to our pipeline and data mapping. It captures the source systems from which the data came. For each source system, there is good lineage so we can identify where it came from. Then, it is loaded into our clean zone and data warehouse, where I have reports, data extracts, API calls, and the web application layer. This provides access to all the interfaces and how information has been consumed. Impact analysis, at an IT and field levels, lets me determine:

• What kind of business rules are applied. 
• How data has been transformed from each stage. 
• How the data is consumed and moved to different data marts or reporting layers. 

Our visibility is now huge, creating a good IT and business process. With confidence, they can assess where the information is, who is using it, and what applications are impacted if that information is not available, inaccurate, or if there are any issues at the source. That impact analysis part is a very strong use case of this tool.

We are a large company, and we purchase a lot of small, medium, and large companies and roll them into our IT. As a result, we have a lot of challenges with mapping all of these systems. We brought in erwin Data Intelligence by Quest to automate some of the data transformations and get a head start on it.

We map corporate entities to these business units, which means that we deal with a lot of different data sources. The biggest use case we have is mapping things like an older mainframe-type database to SQL Server, which erwin does really well. We're super impressed with it.

erwin's staff were great to work with in terms of customization. For example, if we told them that we'd like customized connectors to generate DDL and SSIS packages, then in a couple of weeks they would usually have those features put in. They were very flexible and willing to make the changes on the fly. We had good direct access to the development staff, and they were great to work with.

The biggest impact for us is that erwin generates DDL extremely quickly. We're able to pull in metadata, map it to a target, generate DDL to create the tables, and generate SSIS packages. Previously, especially going back 10 to 15 years ago, hundreds of hours had to be spent to manually perform these tasks. This solution completely automates it and gets it 90% done. We can then pass it off to a developer to create the items in SSIS.

Data Intelligence enables us to provide deeper technical insight into our enterprise data warehouse while democratizing the solution and data. 

For more than 10 years, we had built our data systems without detailed documentation. We finally determined that we needed to improve our data management, and we chose Data Intelligence Suite (DIS) based on our past experience using erwin Data Modeler. After researching DIS, we also discovered other desirable features, such as the Business Glossary and Mind Map features that link various assets.

Data Intelligence has provided more profound insights into legacy data movements, lineages, and definitions in the short term. We have linked three critical layers of data, providing us with an end-to-end lineage at the column level.

Our long-term plans include adding other systems to complete the end-to-end picture of the data lineage. We also intend to better utilize the Business Glossary and Mind Map features. This will require commitment from a planned data governance program, which may still be a year or more into the future.

The use cases were for a large federal government agency with 10 smaller agencies that handle all of the metadata for that agency, and all of that metadata is sensitive PHI or PII. It includes Social Security numbers and all of the metadata for the provider, and beneficiary health records. The purpose of the agencies is to prevent fraud, waste, and abuse of American taxpayers' money.

One of the biggest use cases is to do mappings, manual and auto mappings, and data lineage. The data is used by the agency for prosecution when they find fraud and waste.

At a very high level, what the Medicare or Medicaid services want is the ability to ingest their metadata so that there is transparency. They also want it to be up to date, and the ability to interpret it, both technically and for business use cases, meaning business terms, policies, and rules.

They want end-users with different levels of technical acumen to be able to find information easily and quickly. A data store would be a quasi-technical person, like me, who understands enough to retrieve information or a lineage.

A business user would be either a congressman or congresswoman or a project manager who would see visual representations. erwin has a lot of really good data visualization capabilities.

The use cases include being able to quickly look at data and evaluate it on many levels, from the granular level of a column, table, or even a view, to a zoomed-out level to see how many of a certain table or column are in a data set from each agency.

Another use case is to take the data out of legacy tools, like Excel spreadsheets. And in some cases, agencies are still using a mainframe from the 1960s where some of that data is stuck.

The Standard Data Connectors we used were for Snowflake, RedShift, SQL, IBM, and others. All of the standard data connectors worked. One problem that our team ran into was that some of the applications didn't really do the best job of grooming and maintaining their data. One particular system had 1 million tables, which meant there were a couple of million columns. The size of the data was an issue, but the data connectors worked. There were no APIs used, just database connectors.

In terms of seeing the technical details needed to manage the data landscape, when you log in to erwin it's broken down into modules. One of them is Metadata Manager, and that is one of the things I liked about it. It's broken down according to the work you need to do. With Metadata Manager, you immediately see all of your systems and, in our case, The Centers for Medicare & Medicaid Services had many systems. And in the left-hand panel, there was a really good user interface to expand your systems. You can see your environments and what's in them, and then you see your tables, columns, views, and anything else.

In the center of the UI, you can do your work, such as run a lineage, mind map, or look at an impact analysis. It's set up well visually, and it's also set up like old-school computer science with correct folders.

Another work area module, called Mapping Manager, is where you do all of your mapping. It gives you a mirror view of everything that's in your systems and environments, and you can work with that metadata on your mappings. You can also export and publish your mappings and, once you've done your mappings, you can go back into Metadata Manager and run an impact analysis and look at your mind map.

The third module for business users is the Business Glossary Manager where you can create your business terms, policies, and rules, assign them and see how many are spread across which environments. It gives you a visual in addition to the folder structure.

These modules are the strength of erwin's Data Intelligence Suite. People who are non-technical can learn about data governance using this tool, like I did. The tool we're now using instead of erwin, requires too much searching and linking things, like you're using Facebook.

We use Data Intelligence as a metadata repository for our search target systems and to define metadata. We also use the tool to define the mapping between the search and the targets. It enables us to track the flow of data between systems, design data flows, and share flow implementation with developers. Our end-users seldom access Data Intelligence. We use other tools to provide end-users access to our metadata repository. Data Intelligence is used internally for projects and users with project-oriented roles.

Data Intelligence allows us to automate multiple tasks we had previously done manually, such as restructuring the metadata for our purposes, setting up ETL flows, and defining the data tables we create. It also enables us to standardize our approach and our technical processes.

The ability to automate metadata harvesting and ingestion from common industry sources is pretty nice. Data Intelligence helps us better understand the systems we use. We automatically connect to a specific system. For example, we might connect our Oracle Snowflake databases to Teradata Cloud and automatically ingest all the metadata. We transform it and browse through what we can use. Automation also simplifies the mapping processes. We don't need to recreate anything manually. 

Data Intelligence's standard data connectors for the metadata manager are easy to use. We can connect the systems and have everything in place. 

The automation capabilities help us create ETL flows and map the tables in our system. It frees up our staff who would otherwise need to spend time generating all those pieces manually. Data Intelligence lets us connect to those reports and change the metadata automatically. We get a picture of the target lineage, so we can check the dependencies of one data object on another.

The lineage functionality that erwin offers is only a recommendation, and it isn't fully validated. We cannot trust shared data because anyone can modify a work in progress. Based on the latest information in erwin, we can make high-level assumptions and trust the data in the process.

Our project must be validated, and erwin is a non-validated tool. We can only use outcomes that we can validate. We can check the data lineage and see the potential data flows from sources to targets. However, we cannot fully track the data that we have there.

Data Intelligence saves time on data discovery and helps us understand our data through standardization. It helps us connect to data services and simplifies the process. That's one of the significant benefits of using the Data Intelligence suite. It's difficult to estimate how much time it saves us. In the early stages of a project, we need to spend a lot of time on integration. Data Intelligence simplifies and standardizes the mapping. It's hard to say how much because I would need to compare the time spent manually generating the metrics in Excel versus doing it in Data Intelligence. 

Initially, we had a data warehouse built on a digital data platform. The first task was to perform data fixes and optimize scripts to prepare data for our Teradata databases. We had to adapt the data types, indexes, etc., and generate the codebase. 

Most of our users are product developers. I'm unsure how many there are, but I believe it's fewer than a hundred. The operations team typically isn't digging deeper into the solution. This is related to how the project is organized. The operations team is primarily interested in what is already in production, whereas Data Intelligence is mostly for software development. 

We have erwin two environments: testing and production. The data fixes are performed in the testing erwin environment. Upgrades are also implemented in the test erwin environment. 

Data Intelligence creates a single source of truth for all of our metadata. This solution is better for data warehousing, but the metadata features speed up our development work. It's easy to create and manage mappings because we can export them to Informatica and pick up the work where we left off.

We are using the connectors for Snowflake and our data warehouse. The data connectors work well. We've never had any bugs or other issues when new versions of the connectors are released. 

The solution allows us to deliver data pipelines faster and cheaper. The alternative is to write the code down from scratch, so it's almost 30 percent faster.