I evaluated Business Objects, Tibco Spotfire, Tableau, Qlikview, Sisense Prism, and Pentaho.

I just completed an evaluation of (and ultimately a final selection from) several vendors and identified about a dozen high level evaluation criteria (using Gartner capabilities as a starting point, and adding those specific to our focus). We were primarily concerned with an efficient, competent dashboarding tool with a secondary focus on ad hoc analysis.

Keeping in mind that my focus was first and foremost a competent dashboard tool suited to our particular needs and environment, here are the criteria and associated descriptions I used to complete our evaluation and selection. In addition to these criteria, budget was of course a consideration, as well as Vendor support and the overall support community (i.e. free help!). 

The vendors I was evaluating were Business Objects (already used at our company for flat report building but not much else), Tibco Spotfire, Tableau, Qlikview, Sisense Prism, and Pentaho.

1. Easily combine a mix of data sources – Our data landscape contains many sources and an array of source types. We need a tool that can pull a variety of sources (e.g. SQL Server, CSVs, etc.) together in an easy, efficient manner, and generate visualizations and dashboards that reflect the combined data.
2. Broad range of Visualizations– The tool should, out of the box, have an array of visualizations beyond the basic pie, bar and line charts, as it is important that we are able to select the most effective visualization for a particular measure.
3. Responsive Data Engine – Users need to be able to slice, dice and filter data, without the tool taking more than a few seconds to refresh the visualizations and dashboard accordingly.
4. Low development time/quick turnaround of dashboards - There will be a number of dashboards needed in 2014, and it’s likely that requests for additional pages, visualizations, and entire dashboards will increase as more of the business is exposed to what is released. The dashboard tool selected should facilitate the rapid design, development and deployment of new dashboards and enhancements to those already deployed.
5. Scheduled refreshes and publishing - It is necessary for the refresh of dashboards to be automated and occur in conjunction with the update of other BI components, so as to have data up to date and ready for the business at the start of the work day.
6. Considerable design control and theme application – The tool selected should allow a high degree of control over layout, colors, fonts, and overall look and feel of the entire dashboard and individual visualizations. This is necessary to incorporate data visualization best practices and align with company style guides where and when appropriate.
7. Central portal & cross-platform (Mac/PC/Mobile)– Type of device or OS should not impede users from accessing the dashboard. Dashboards should be accessed via web browser and be HTML5 based.
8. Client install not required, web friendly – Access to deployed dashboards should not require the installation of software on a user’s desktop. (thick client/desktop software for development is acceptable)
9. Intuitive interaction, enabled data discovery – The tool selected should employ components such as gauges, sliders, and checkboxes, and enable users to analyze the data by interacting directly with a visual representation of it. Navigation through a dashboard and its pages should be easy and intuitive.
10. Quick response to filtering, drilldowns, and data manipulation – The use of filters and other components to manipulate data should result in a rapid refresh of the visualized data. A tool that takes more than a few seconds to respond is likely to have a low adoption rate.