Coming October 25: PeerSpot Awards will be announced! Learn more
Nurit Sherman - PeerSpot reviewer
Content Operations Manager at PeerSpot (formerly IT Central Station)
  • 8
  • 58

What's the best way to trial reporting tools?

We all know that it's important to conduct a trial and/or proof of concept as part of the buying process. 

Do you have any advice for the community about the best way to conduct a trial or PoC? How would you conduct a trial effectively? 

What mistakes should be avoided?

PeerSpot user
10 Answers
Alexandra  Misios - PeerSpot reviewer
Business Intelligence Coach at Wipro
Real User
26 October 21

I would say that, in order to properly assess a new reporting tool, you should do a PoC, for sure. Try to connect the tool to as many data sources available in your company, see if it fits your expectations regarding connectivity, performance, use of resources.

Also, it would be a good point to check on the learning material available. See the educational offer, check the documentation available freely, see if support is offered and at what costs.

Also, you should take care of portability, see if it fits both on premises and on cloud, check on the devices it is suitable for.

See how security is implemented and if it fits your organizational structure.

Bob Gill - PeerSpot reviewer
Principle Architect at a tech services company with 51-200 employees
28 August 18

Hi Rhea,
This is a great question. Just like Alberto, this is my opinion, so please take it with a grain of salt. Here are some tips I would suggest for any group looking at purchasing reporting tools.

* Define your current business challenge - What kind of decisions is this reporting supposed to support? ...number of users involved? are they organized into teams? ...what are the steps in the business process?

* Decide on some metrics in key categories - If you decide on some metrics before looking at all the alternatives, it can help make comparison conversations more clear. Here are some examples

A) Ease of Use - {Training required, Usable from Excel, Usable from Web, Usable from Mobile, ...}
B) Aesthetics - {Visuals, Responsiveness, Look and Feel}
C) Functionality - {Ease of development, Business rules, change management, Output}
D) Support - {Community Support, Vendor Support, Partner Support, Inhouse Support}
E) Effectiveness - {Solves the business problem, Flexibility for change, Transparency, Accountability}

* Ask for a proof of concept - Vendors are happy to build a proof of concept with your own data. They know showing you your own data will prove (or reject) the idea of the software helping the organization. After the vendor builds a proof of concept, ask multiple users from the organization to play with it. for mistakes to be avoided, here are some common pitfalls I've seen folks encounter.

* My buddy uses is, so it must work - Please don't assume that something that worked for someone else works for your organization. Test all the key functionality in the PoC or at least talk through it with your vendor/partner.

* We need something now, so let's just do this quickly - By skipping through the selection process, there is so much risk added. If you don't have the time to invest in the selection process, just wait.

* We want everything in the PoC - PoCs are not meant to be full solutions with everything. Ask for the most important features to be developed as part of the PoC. Even if it does have everything, you'll want to participate in the development process to ensure you know how to change things without having to go back to the vendor.

Alberto Guisande - PeerSpot reviewer
Director at Decision Science
Real User
Top 5Leaderboard
28 August 18

Hi Rhea,
This is my personal opinion and shouldn't be taken as a best practice manual, but regarding PoCs, the better way I found so far:
- Having very clear what you want to "test" in the tool. Lets say that your "actual problem" with data is volume, please, don't test only volume handling (you risk not to assess other functionalities that may become the "new problem". in the future).
- Ask the vendor/partner to help you with the PoC. This is the better way to avoid trouble solving specifics on the tool, other than testing it, and can be very frustrating, biasing your judgement on the tool.
- Use a specific business case/business question to tackle the PoC and involve the business user in the PoC (at the end, he is the SME)
- Do not hold ANY questions to the vendor, he is there to sell, but once the buying decision is made, he should be there to provide you support (and enablement, and training, and etc..

Hope this helps you.

it_user856365 - PeerSpot reviewer
VP Business Development and Technology at Exago
29 August 18

Hi All,

Excellent answers below. A few things I would add. Full disclosure...I am on the vendor side of an embedded BI platform. I ran our pre-sales engineering program for several years. Hoping that sharing a few things from the vendor's perspective can help you find the right platform.

First off, the comments about using your own data, coming up with well-defined requirements, including example reports and dashboards, etc. are all spot on. Make sure you know what your "must have's" are versus your "nice-to-have's". No software can be all things to all people.

One thing I didn't notice mentioned already is to evaluate the *team* you'll be working with as well. Entering into a new vendor relationship is just like any other, and you'll want to make sure you feel comfortable working with the team. Everything is bright and glowy and sunshine-y during the sales process. But sometime down the road, you'll need something. Do you feel like the vendor is vested in your success? Or are they just trying to close a deal?

If you haven't done so already, I'd make sure to check out online reviews of both the product and the company on sites like Gartner Peer Reviews, G2 Crowd and Glassdoor. Down the road the people you are working with will be just as important as the features of the software. Do you want to be supported by a bunch of people who hate coming to work every day?

By the same token, talk to reference customers. They will of course be hand-picked by the vendor so take their responses with a grain of salt. But don't be afraid to ask about vendor weaknesses as well. If they tell you there aren't any, that should be suspect.

Overall be realistic. Every vendor solution will have its trade-offs and if someone appears to be perfect and promises you the earth the moon and the stars something's amiss. At the end of the day you want to find the right combination of tradeoffs that will provide you the most value with the least pain.

As someone who's been through a number of these I'm happy to talk with you directly if I can be of any further help. I'm easily findable on LinkedIn.

Jhornber - PeerSpot reviewer
Director, BI & Analytics at a leisure / travel company with 10,001+ employees
Real User
28 August 18

All good guidance so far! A few things I'd add:

1) Before you even start a POC, put together a core set of requirements, and ask vendors to complete an RFI (Request for Information) - basically a check list of capabilities as they pertain to your use cases. This will help you identify early show-stoppers and rule out some vendors from the beginning should they not be able to provide some of the core functionality you're looking for. For example, when we last evaluated tools, automated report generation and distribution was a requirement, that several vendors could not meet. Likewise, more specifically, we needed a tool that could display Image thumbnails within a table. Again, many vendors were unable to do this and we were thus able to cross them off our list without out any more time evaluating the solution.

2) Echoing what Bob said, don't rely only on the vendor to prove something can be done by going away and doing it for you. Be sure you fully understand what it takes to do it yourself. i.e. Is it more or less out-of-the-box functionality, or does it require a lot custom coding, extension building, etc? A lot of solutions can accomplish nearly anything with enough time and technical expertise, but you probably don't want to rely on that for your primary use cases. Look for something that meets most of your core requirements while still enabling rapid dev and deployment.

3) As much as possible, make sure you're testing it under real-world conditions to properly gauge performance. I agree that data volume shouldn't be the only thing you assess, but do use one of your largest data sets rather than a sample and make sure the POC is taking place in an environment that has relative parity with your production environment. A lot of products demo well on a sample data set and vendor's architecture, but fall short once fully deployed. Performance is so crucial to adoption. You can build all the insightful, cool data tools in the world, but if they're slow, nobody's going to use them.

Managing Partner at a insurance company with 10,001+ employees
Real User
28 August 18

I’ve always approached BI/Analytics tools evaluations/POCs with the mindset that the POC needs to demonstrate the tools ability to satisfy the requirements of both the IT organization and business teams ( who will be the primary consumers of the insights generated out of the tool and interacting with the technology most frequently). This is especially important when the POC is focused on tools for self-service data discovery and visualization. I have always started the process with a jointly defined set of evaluation criteria (evaluation matrix) that is weighted and can be stored as part of the POC evaluations. Another best practice I have used is to take actual business use cases and incorporate them as part of the POC. These are usually of simple to moderate complexity, can be implemented quickly in the POC, and can demonstrate business value. I have also made sure to incorporate IT specific use cases such as data connectivity, metadata capture, account setup/creation, security, and administration. Each of these can be tied to the evaluation matrix. I’ve found that this really demonstrates which tool best fits the needs of the organization.

I’ve also relied frequently on the vendors to create the POC environments. This eliminates the need for internal IT to provision temporary infrastructure and can significantly accelerate the evaluation process.

What mistakes should be avoided?

The biggest pitfall I’ve seen is not including the business in the process. Another mistake is to not validate the fit with the existing technical landscape and systems portfolio.

Find out what your peers are saying about Microsoft, Tableau, IBM and others in Reporting Tools. Updated: September 2022.
633,572 professionals have used our research since 2012.
ArtourAslanian - PeerSpot reviewer
BI Architect/Cognos Solution Architect/ETL Design Architect at a media company with 201-500 employees
Real User
21 September 18

Most of a reporting tools are exact same.
Details is a subject to add to a question about what is better to use?

Major steps to select a right product:

1 Estimate how many potential users you have
2 Estimate how much you gonna pay for a license
This will help to narrow down a list of products offered.

3 If possible - check with a vendor a report about at least 5 gb size to see a reports performance

4 Figure out which back end can be used and how effectively as of a connectivity to use a tool as well as a price of a different back end if required together with a migration price.

5 Figure out a culture level of users if you are going to implement reporting as a self serve. Who at least can work with Excel and knows what is a pivot table?

6 Agree on a trial test for a most advanced users - again using a mid size data sets.

7 Figure out about how complicated is a security configuration and its requirements.

8 Does this product requires an administrative 24/7 maintenance? How long takes a refresh or reboot?

9 What is a back compliance policy? as new version upgraded -> may have crap all existing stuff

10 Working in a cloud environment is too different from a local net?

11 .. many other things basically related to a reports functionality and advanced capabilities.


Sanam Sarwar - PeerSpot reviewer
Software Developer at a tech services company
Real User
29 August 18

First of all check the reporting tool requirements and if it works with your database or not. Install all the necessary frameworks and then make a connection to your database to see if it works or not. After the connection is made you can make a sample report with a test query.

RaviLagu - PeerSpot reviewer
Senior Manager at Indiabulls Housing Finance Limited
Real User
29 August 18

What's the best way to trial reporting tools?

The best way to trial reporting tools is by doing a hands-on PoC at your office. The people in charge of making reports should be able to recreate simple reports on their own in the tools alongside an expert from the company.

Do you have any advice for the community about the best way to conduct a trial or PoC? How would you conduct a trial effectively?
My advice would be to recreate 4/5 reports that are made manually currently in the BI tool. We found out certain features were lacking when we tried to recreate the same report in different tools. For our internal comparison, we used SAS VA, Power BI, Qlik and Tableau. We created the same report in all 4 tools starting with the absolute raw data. We created all calculated columns/flags in the tool itself.

What mistakes should be avoided?
Do not go by the demos the Pre-Sales guy shows you while demonstrating the product. The data used has gone a lot of transformation and the data you have may not be as clean as theirs. Always use your data (dummy/masked) for the PoC.

Data Analyst with 201-500 employees
04 September 18

To reiterate what is already been stated, have a strong business case established to use for the PoC. Don't make it up as you go along. Also, recommend having a scoring methodology/matrix for each tool you are testing. This takes some of the subjectivity out of the mix so you can base your decision on how each tool meets your needs.

We recently went through this process and having the scoring really helped.

Related Questions
Frank Bianchi - PeerSpot reviewer
Chief Sales and Marketing Officer at AdminaHealth
Jan 27, 2022
Hi peers, Aside from dashboards, there are some use cases that require a fair amount of detailed data, with customizable sort and display fields which can vary by account.  This is in the insurance, healthcare, benefits space in particular.   While advanced visualization features help on the dashboard side of the equation, I would like to get the opinion on the best tools for this detailed, ...
2 out of 5 answers
Alan Sutton - PeerSpot reviewer
Senior Manager at a tech services company with 1-10 employees
25 January 22
XLCubed (now owned by Fluence Technologies Inc) is the market-leading Reporting and Dashboard tool for any Excel-connected data.  With entry-level pricing and straightforward feature explanations via online training materials, XLCubed can be implemented as either a standalone cloud-based BI tool or as part of today's world-beating Financial Consolidation and Reporting solution.
Jose Barbosa - PeerSpot reviewer
CEO at Finanblue
25 January 22
Hi Mr. @Frank Bianchi, Yes, you can do that but only on Professional or Enterprise Plan.  Also, you can link to Power BI and generate some different views.
PeerSpot user
User with 10,001+ employees
Jan 16, 2017
We are looking to replace our ad hoc reporting capabilities that we currently do with MainFrame Focus (not WebFocus!) with the more modern QlikView product.   We are creating a Pros and Cons list for QlikView, but I'd also like to hear from anyone who may have gone through with this type of conversion effort.  We have a small group of focus power-users who would be now using QlikView to crea...
2 out of 38 answers
03 January 17
my company has done 1000+ Qlik projects and I am doing with qlik for 9.5 years.. Please write me with details at or call me at +91-9312667720
PeerSpot user
Project Leader at a energy/utilities company with 1,001-5,000 employees
03 January 17
If you ar planning to use QlikView for ad hoc reports I suggests the following recomendations: Do not use of the ETL capabilities of Qv, just to avoid high dependency on the product. Beware of the abuse of .qvd files to store big data files, they are not trustworthy at all. Remember that Qv is a Dashboarding Tool but not a complete reporting tool. If it´s possible, try to use QlikSense instead of QlikView to avoid excessive consulting.
Related Categories
Download Free Report
Download our free Reporting Tools Report and find out what your peers are saying about Microsoft, Tableau, IBM, and more! Updated: September 2022.
633,572 professionals have used our research since 2012.