2014-03-25T12:43:00Z
Ariel Lindenfeld - PeerSpot reviewer
Director of Community at PeerSpot
  • 8
  • 10

Do you have an Evaluation Criteria Matrix for APM or other software you can share with the community?

9
PeerSpot user
9 Answers
it_user96747 - PeerSpot reviewer
User at a tech services company with 51-200 employees
Real User
2014-03-27T05:50:25Z
Mar 27, 2014

I would be interested to learn more about this. Randy has raised some good points.

Search for a product comparison in Application Performance Management (APM)
it_user9288 - PeerSpot reviewer
Consultant at a consultancy with 51-200 employees
Consultant
2014-03-26T20:35:29Z
Mar 26, 2014

All valid points above.
In our case we find ourselves sometimes in the position to evaluate this on behalf of our customers.
A point which I do not think has been mentioned in the above comments is the maturity of the organisation. Some organisations are very immature / old school with regards to APM, and should therefore start with the minimum and get value from that. More mature organisations might get more value from more complete / complex solutions.

it_user69357 - PeerSpot reviewer
Project Manager with 501-1,000 employees
Vendor
2014-03-26T14:12:35Z
Mar 26, 2014

Gartner has a the APM Magic Quadrant and the newly released NPMD Quadrant. They have some good qualifications...

I define that APM side as Agent Instrumentation within JAVA and .NET, while the new NPMD is the Network and Application combo...

Disclaimer: I work for Fluke Networks

it_user98046 - PeerSpot reviewer
Director at Riverbed Technology
Real User
2014-03-26T12:52:10Z
Mar 26, 2014

I would suggest these primary criteria:
1. How straightforward is the product to set up and instrument your applications? Remember that you'll probably be adding applications constantly, so your team should be able to understand and execute the installation and configuration process for new applications as they come online. Invest in a rich test environment that looks and acts like production, where you can try instrumenting a new application before deploying it.
2. Does it give you visibility into the applications you care about? Some applications don't let you instrument critical sections of their startup sequence, but once up and running, you shouldn't have major blind spots -- it should see each application tier and the network segments that connect them.
3. How much overhead does it consume? The product shouldn't use "much", but how much depends on how much headroom you have on your systems. It's worth a little extra memory or CPU to get visibility, but plan for it.
4. Does the product help you hunt down the root cause of the problem, or does it just tell you "you have a problem" without getting you any closer to fixing it? Does it make the nature of the problem clear enough for your team to take action?
5. What's the signal-to-noise ratio? Does the product highlight real problems that need immediate attention, and separate these from the things it's nice to know but that aren't immediately actionable?

Disclaimer: I work for Riverbed

it_user96747 - PeerSpot reviewer
User at a tech services company with 51-200 employees
Real User
2014-03-26T11:53:32Z
Mar 26, 2014

Hello Rusty and Bill....
Enteros Upbeat and load2test are very simple uses - very agile methodology techniques for load testing. It provides comprehensive diagnostic reports, real time performance sampling and reporting, low level tracing and containment command and DBA function library. With Grid2Go it provides proactive problem identification and remediation.

Ariel Lindenfeld - PeerSpot reviewer
Director of Community at PeerSpot
Community Manager
2014-03-26T11:09:50Z
Mar 26, 2014

Very interesting points Randy. I would be interested to hear from others if they feel the same way or if they've successfully used a matrix to make their decision.

Learn what your peers think about ITRS Geneos. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
656,474 professionals have used our research since 2012.
it_user69183 - PeerSpot reviewer
Systems Engineer at a healthcare company with 1,001-5,000 employees
Vendor
2014-03-25T18:02:59Z
Mar 25, 2014

Interesting approach Bill, I'll have to think about it.

Rusty, I don't have an evaluation matrix either - frankly I've never needed to build a matrix to make a decision, but they are useful to explain or defend a choice. I'm more used to testing, reading and finding strengths and weaknesses. Make sure features are more than a checkbox - even a single numerical value can be problematic as any given feature may have relative strengths that may not condense to a single number.

Keeping this in mind I start with needs - what do I, or you need a monitoring solution to do? I look for stable and extensible either by the vendor or users. Ease of upgrade and recovery, of course. (Early versions of Solar Winds suffered greatly here, the current product is easy to manage.) Auto discovery, naturally. Performance is a must, as is methods of extending monitoring. Platform choice may be a consideration; forcing a unique platform in an environment may not be the best idea. While on the subject I look for endpoint support too. Does it recognize and integrate with, say your virtual environment, and can it peer deep into the databases?

Then I consider the console. Most all are web consoles but how are the views built? It has to be easy or you'll never do the update maintenance. Are they configurable for different users or groups, and can you view multiple pages at the same time? (This is great for a NOC; a wall of displays from a single server can provide an overview of just about everything.) Most people are graphical, so live maps are helpful. Recursive, or linked maps are moreso. Can a user drill down from a high level alert down to the problem area and understand what the issue is? Bonus points for the ability to actually make changes in the impacted system based of course on access rights. Finally, is the console open (standard web browser) or specific to a platform? Tablets and smartphones are here, now and mobility is incredibly useful when tracking down a point problem that requires physical access and a console view.

Next consider alerting. Dull, boring stuff but it has to be done. What alert options are available? How about out of band alerts? Can you sent to groups, cascade to others when no response is given? How about consolidating alerts through dependencies? One big alert when a trunk line goes down is MUCH better than a thousand alerts where the root cause is buried, although it does take effort to map out the dependencies. Then, what can you alert on? Is programmatic alerting available to add sophisticated responses?

Finally, reports. I don’t see much use for static reports printed and distributed, but that depends on the environment. Much more useful to have a mix of standard reports, configurable reports and free form reports that can be configured by users. Solar Winds suffered here in the past; the report writer was only on the Orion server itself. It remains there at the moment, but there are some reports now configurable from the web console. This is a huge annoyance finally being addressed.

So where does this leave the matrix? Pretty complex, as befits a complex tool. Each feature needs a relative value for importance and execution. If I were to create one I would split it into 4 parts - platform, console, alerts and reports, then consider changing the matrix as systems evolve. I would also suggest some further discussion; I doubt I have captured everything just yet.

it_user95673 - PeerSpot reviewer
Developer at a comms service provider with 501-1,000 employees
Vendor
2014-03-25T13:04:50Z
Mar 25, 2014

I cannot share a matrix, but I do have a general philosophy that helped me get started determining the value and best in breed solutions. There are three axes of functionality that you define for yourself:
1. Internally facing features (supports your infrastructure) vs Externally facing features (supports customers)
2. Availability of core features vs Availability for extension & integration
3. Standard Features vs Disruptive Features
Some solutions will be strong in one but not the other. Look for those that align with what you value.

Rusty Isaac - PeerSpot reviewer
Vice President at PeerSpot
Vendor
2014-03-25T12:49:55Z
Mar 25, 2014

I would be interested to get one too. Please share, thanks!

Related Questions
ML
Configuration Management Analyst at MRI Network Inc.
Nov 23, 2022
Hi Peers,  We have microservices and non-microservices to monitor. We are considering Instana, New Relic, and Grafana Cloud for this task. We want to monitor apps and the underlying machines that run on them to troubleshoot as well as monitor and alert. Any experience in using these tools would give me a more clear picture than months of researching with POCs. Can anyone shed some light on th...
See 1 answer
TS
CEO at Rufusforyou LLC
Nov 23, 2022
I have worked with a lot of APM tools like Instana, New Relic, and Grafana, I prefer Instana because it is easy to install, and the configuration is very simple but more importantly, it provides what I need to find a solution to a problem. With the integration of Turbonomic, we save a lot on cloud costs of well due cost AI and automation. New Relic and Grafana are also great, but I love the integration with IBM APM it provides us with a lot of information. The problem with Grafana for us was the lack of DB2 support only in an enterprise version possible. Regards Tjeerd Saijoen CEO Rufusai                     
AH
Software Engineer at Thomson Reuters
Mar 15, 2022
Hi peers, Which solutions are the leaders of Gartner MQ for Application Performance Management in 2021? What do you think of their choice of companies? Thanks!
2 out of 7 answers
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Dec 10, 2021
Hi @Tjeerd Saijoen, @Ravi Suvvari, @mehrab hussain ​and @Ravi Khanchandani, Can you please assist here? What are your thoughts on the recent APM MQ? Thanks!
Shibu Babuchandran - PeerSpot reviewer
Regional Manager/ Service Delivery Manager at ASPL INFO Services
Dec 11, 2021
Hi, Looks like the 4 players below are in the Leaders position: -Dynatrace-AppDynamic-New Relic-Datadog Anyone with a new update, please kindly share.
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jun 20, 2022
If you are considering DevOps tools for your business, you want to be sure you choose the right solution. DevOps software is a rapidly growing market, and while it can be hard to know which tools you really need, researching all of your options and making the right choice does not have to be a daunting task. To help you choose the best DevOps solution, here are some tips: Develop a common...
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jun 20, 2022
Hi PeerSpot community members, This is a fresh-from-the-oven Community Spotlight for you. Here, we've summarized and selected the latest posts (professional questions, articles and discussions) by PeerSpot community members. Check them out! Also, please share with us your feedback and suggestions by commenting below! Trending See what is trending at the moment and chime in to discuss! ...
Deena Nouril - PeerSpot reviewer
Tech Blogger
Jun 17, 2022
PeerSpot’s crowdsourced user review platform helps technology decision-makers around the world to better connect with peers and other independent experts who provide advice without vendor bias. Our users have ranked these solutions according to their valuable features, and discuss which features they like most and why. You can read user reviews for the top Application Performance Management (...
2 out of 3 comments
Abbasi Poonawala - PeerSpot reviewer
Chief Enterprise Architect at Bank in KSA/JPMORGAN
Jun 16, 2022
How about pricing strategy?  Why do top banks in MENA still prefer Dynatrace over New Relic or AppDynamics?🤔 
Abbasi Poonawala - PeerSpot reviewer
Chief Enterprise Architect at Bank in KSA/JPMORGAN
Jun 16, 2022
@Abdulla Pathan, I appreciate it if you can share more insights & your valuable experience around APM adoption @LearningMate. 
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jul 11, 2022
Hi community members, As usual, this new Community Spotlight shares with you the latest articles, questions and trending discussions from your peers. Trending See what is trending at the moment and chime in to discuss! Top 8 Extended Detection and Response (XDR) Tools 2022 Would you recommend replacing Cisco ASA Firewall with Fortinet FortiGate FG 100F due to cost reasons? What is the...
See 2 comments
Ravi Suvvari - PeerSpot reviewer
Performance and Fault-tolerance Architect with 1,001-5,000 employees
May 30, 2022
Good very informative
Jairo Willian Pereira - PeerSpot reviewer
Information Security Manager at a financial services firm with 5,001-10,000 employees
Jul 11, 2022
Analyze the wave of product at Gartner Hype Cycle. EDR was good in the past. After that, MDR joined the hype and now, XDR is the trend. Wait for more in a couple of months and (sic) know the ZDR!
Shibu Babuchandran - PeerSpot reviewer
Regional Manager/ Service Delivery Manager at ASPL INFO Services
Nov 1, 2021
What Is AIOps? AIOps is the practice of applying analytics and machine learning to big data to automate and improve IT operations. These new learning systems can analyze massive amounts of network and machine data to find patterns not always identified by human operators. These patterns can both identify the cause of existing problems and predict future impacts. The ultimate goal of AIOps is...
See 2 comments
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Sep 14, 2021
Great article, @Shibu Babuchandran! Thank you for sharing your knowledge with the community!
Jairo Willian Pereira - PeerSpot reviewer
Information Security Manager at a financial services firm with 5,001-10,000 employees
Nov 1, 2021
I always like this order a lot: "Consider People and Process" and only after, Technology.
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jun 20, 2022
How to Choose the Best DevOps Tools for your Business
If you are considering DevOps tools for your business, you want to be sure you choose the right s...
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jun 20, 2022
Community Spotlight #16
Hi PeerSpot community members, This is a fresh-from-the-oven Community Spotlight for you. Here, ...
Download Free Report
Download our free ITRS Geneos Report and get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
DOWNLOAD NOW
656,474 professionals have used our research since 2012.