2018-06-13T10:29:00Z
Nurit Sherman - PeerSpot reviewer
Content Operations Manager at PeerSpot (formerly IT Central Station)
  • 8
  • 90

What's the best way to trial APM solutions?

We all know that it's important to try out software as part of the buying process. Do you have any advice for the community about the best way to conduct a trial or POC? How do you conduct a trial effectively? Are there any mistakes to avoid?

11
PeerSpot user
11 Answers
Chandan-Kumar - PeerSpot reviewer
Associate Director at a financial services firm with 10,001+ employees
Real User
Top 10
2021-10-20T14:31:17Z
Oct 20, 2021

The best way to trail APM is to consider your most unstable application or environment. 


Download all the incidents, changes, problems tickets from the ITSM tool.


Give the vendor to analyze and deploy the plan. Also, ask them about problem-solving strategies. Also, provide similar information to Application owners, business owners, and end-users for the services on the apps or application direct users. Build 5 - 6 use cases and deploy the tool to achieve the use case objective.


It is always better that all team members about the solutions are trained and know how to use and navigate the tool.


Make sure all the business users, apps users, and end-user understand the objective and goal of the deployment.


Ask the vendor to propose the solution irrespective and independent of the integration vendor dependency.


APM is a big and costly solution, so best to go with the vendor's recommendation to see the max output.


Do not go for perpetual licenses, start with subscriptions base or terms bases or time base licenses for evaluation and test.


Best to have ROI thinking solution to how, when, why, what and where answer did, after your testing. 


Use the subscription base or termbase licenses till the time of ROI.


Give similar objectives to the vendor.

Search for a product comparison in Application Performance Management (APM)
Hani Khalil - PeerSpot reviewer
Service Assurance , Senior Manager at a computer software company with 1,001-5,000 employees
Real User
Top 5
2021-10-18T06:13:54Z
Oct 18, 2021

One of the main steps is to define your cases and problems that you need to solve by using an APM solution and then do a full POC for one service end-to-end to see the results.

SR
Business Consultant at Tata Consultancy Services
Real User
2018-06-21T16:31:28Z
Jun 21, 2018

Your questions are about 'how to trial effectively' and 'avoid mistakes' so will assume that after a lot of research, you have narrowed down the tools that fit your organization's specific needs. Also assumed is that you will be able to procure free trial license for the time you want which almost all tool vendors are happy to comply.

- First task is to form a small team with lead(s) to own the evaluation effort. This can be tricky but most organizations have top tech folks at various levels who are open to change in tool direction. For APM tool solution trials, team members should come from engineering, performance, capacity planning and development/devops. Engineering include platform, network, infrastructure and service desk SMEs. Team would be a handful with hands on ability to install,configure, script, test, analyze and present findings.

- Next task is identifying sandbox environment for trials and naturally would not be production or replication environments. Performance or staging would be ideal and the footprint of tool coverage across the CIs (Configuration items) should be minimal but representative of the APM operating environment.

- A load / performance test is almost always a requirement for APM tool evaluation and should be used for trial. Any existing tool would suffice as the objective is APM tool trial not test tool capability.

- Some criteria for effective evaluation are :
a. that resolved issues could be captured by the new tool when the fixes are removed (satisfies current capability)
b. one or more unresolved issues can be unearthed during the poc and corroborated (new capability)
c. time to resolution meets or exceeds current state
d. tool instrumentation is easy and dynamic or dynamic enough
e. tool footprint on CIs are known and scalable
f. tool integrates well with ITSM, platform specific tools(e.g. vm, SAN, DB)

- Some guidelines to avoid mistakes during APM tool trial:
a. focusing too narrowly on application areas can make choice look better than what it actually is
b. not differentiating monitoring and debug instrumentation can overstate bau efforts
c. during trials, it may be difficult to determine the false positives and negatives received/not received at ITSM so have to keep an eye the number and quality; it is difficult to be objective
d. all IT teams have a chance to independently review from their vantage point

The suggestions here are not on how to select an APM tool.

CM
Senior Computer Performance Specialist at a government with 1,001-5,000 employees
Real User
2018-06-19T15:05:27Z
Jun 19, 2018

Because there are so many APM solutions available, you will need to do some preliminary research first. Consider your current APM solution that is in your environment. Review articles and evaluations such as those found in Gartner’s software evaluations. Make note of the good and bad of the tool being used now in your environment. From your preliminary research, narrow your perspective trial candidates to perhaps 5 possible solutions. Sign up for the trail offers and take advantage of the demos and WebEx’s that will engage the support and sales staff of the products. Typical you want to setup any downloaded software in a test or developer environment. If the trail is agentless, then you should have no problem. Consider the most problematic applications that you currently have in your current APM solution to be the ones to focus on in the trail or demo.

Obviously there are a number of other things to consider i.e. firewall rules, authentication, security, etc., but I hope this post help to answer your question.

ES
Special Source Operations Head with 201-500 employees
User
2018-06-19T07:26:47Z
Jun 19, 2018

We conducted a POC within our development/test environment where we ran a select set of UAT test cases manually after we had installed the APM solution. This so as to assure ourselves that the APM solution would not have any negative impact to our application’s functionality. I would have preferred to have run a complete regression test and would suggest that be done where possible and especially if the regression test suite has already been automated via an automated testing tool such as HPE UFT or Selenium. I would watch out for the ability to monitor and improve the performance of batch jobs as different APM solutions provide support for batch jobs to different degrees. Lastly, if you are selecting an APM solution for the enterprise, you might want to look into how the APM solution should integrate with the ITMS solution as well as any IPM (infrastructure performance monitoring) solutions and also whether it would be suited to monitoring the different technologies your applications are implemented upon.

KP
Solutions Architect at Tata Consultancy Services
Real User
2018-06-19T09:27:18Z
Jun 19, 2018

- Identify the critical application(s)
- If you use an APM already, figure out the shortcomings and/or identify new requirements which are not fulfilled currently by already-in-use APM
- Having an APM in production environment has its own side effects. So, build your performance environment if you don't have one. Is it OK for your organization to run APM in production on a trial version? I strongly believe answer is NO, but it is still an option
- Simulate the production load in PT environment which is a key challenge as at many places identifying the right user mix is always an issue
- Enable application or configuration changes
- Get the new APM configured in your PT environment
- Configure dahsboards with relevant metrics as you require
- Run few performance tests including stress, production like load test
- Look at the dashboards. Are you convinced? Do you require any further tweaks or new metrics? If yes, get the help of product team or tool expert and make sure you are satisfied

Are you happy with the tool? If yes, go ahead with the commercial talks with the APM vendor.

Find out what your peers are saying about Datadog, Dynatrace, New Relic and others in Application Performance Management (APM). Updated: November 2022.
653,522 professionals have used our research since 2012.
NT
Manager with 1,001-5,000 employees
User
2018-06-19T05:05:32Z
Jun 19, 2018

Best way to conduct a trial or POC for APM.

- For the POC, identify YOUR solved issues to see if the proposed APM solution can identify and pinpoint the problem. Duplicate the issue to see the proposed solution can find the problem.

- Make sure the solution can be integrated to your environment. Different organizations have different operational environments. Make sure it meets the expectations for all concerned departments.

- Load test. The best load test is the production, real world, load test. MOST enterprise organizations won't allow this, and this is where it matter the most.

- Talk to the technical advisors directly. Understand that some APM solutions will offer different methods of delivery or solution. For example, SaaS, On-Premises, On/Off Premises. Make sure you know YOUR environment first so that you can best explain your requirements.

- Don't mix Sales and Technical teams together. Make sure the right persons are communicating with each other in order to minimize miscommunication or misunderstanding. I've had sales team overselling their product just to make a sale. I've had sales team lacking technical details. Sales team are there to sell and negotiate pricing, not give technical advice. In the end, everything will be contractual.

- Don't be shy to ask for product limitations, this will come after discussing in detail with their technical advisors about your environment and requirements (this can be contractual), good technical advisors will tell you of product limitations and possible workarounds. Good technical advisors will want to have good reputation, thus will always give the best honest answers, and they are the ones you want to be with.

Dharmendra-Singh - PeerSpot reviewer
Specialist Operations(Windows) at HCL Technologies
Consultant
2018-06-19T08:56:25Z
Jun 19, 2018

As per my experience, if you are able to show the number of concurrent user hits at URLs and pagewise response time as well as include these things in reports format in POC.

It will be very helpful if we segregate the code, database, and infrastructure by going straight method after integration with unified infrastructure management.

it_user233283 - PeerSpot reviewer
Associate Director at Capgemini
Real User
2018-06-19T06:57:13Z
Jun 19, 2018

The best method of shortlisting a tool is to list down all your requirements in a sheet, give weight to every requirement based on its criticality for your business and then comparing the tools based on your requirements. The tool which gives you best result in terms of meeting all your critical or must have requirements is the one most suitable for your business.

Chandan-Kumar - PeerSpot reviewer
Associate Director at a financial services firm with 10,001+ employees
Real User
Top 10
2018-06-19T05:29:26Z
Jun 19, 2018

Best way to complete POC, is to select 2 - 3 application and expected parameters or measurement type.

Michael Sydor - PeerSpot reviewer
Founder and Solution Architect at The APM Practice, LLC
Real User
2018-06-18T17:48:46Z
Jun 18, 2018

You need to experience 'living with' the software - so it needs to be a trial exercise in your environment, driven by your people and stakeholders. The most important element is to exercise an application via load testing, in order to see what a 'real' application is doing, in normal and high load situations. If you don't have sufficient licenses to generate a heavy load, it is often enough to run the app on undersized hardware.

I would also recommend you make a modest investment in "APM best Practices - Realizing Application Performance Management" https://www.amazon.com/APM-Best-Practices-Application-Professionals/dp/1430231416/ref=sr_1_1?ie=UTF8&qid=1529343857&sr=8-1&keywords=APM+best+practices

Chapter 6 lays out all of the considerations for a successful pilot exercise - plus you get additional insights in how to justify, deploy and operate an APM solution. The best practices described are vendor-neutral.

Related Questions
ML
Configuration Management Analyst at MRI Network Inc.
Nov 23, 2022
Hi Peers,  We have microservices and non-microservices to monitor. We are considering Instana, New Relic, and Grafana Cloud for this task. We want to monitor apps and the underlying machines that run on them to troubleshoot as well as monitor and alert. Any experience in using these tools would give me a more clear picture than months of researching with POCs. Can anyone shed some light on th...
See 1 answer
TS
CEO at Rufusforyou LLC
Nov 23, 2022
I have worked with a lot of APM tools like Instana, New Relic, and Grafana, I prefer Instana because it is easy to install, and the configuration is very simple but more importantly, it provides what I need to find a solution to a problem. With the integration of Turbonomic, we save a lot on cloud costs of well due cost AI and automation. New Relic and Grafana are also great, but I love the integration with IBM APM it provides us with a lot of information. The problem with Grafana for us was the lack of DB2 support only in an enterprise version possible. Regards Tjeerd Saijoen CEO Rufusai                     
AH
Software Engineer at Thomson Reuters
Mar 15, 2022
Hi peers, Which solutions are the leaders of Gartner MQ for Application Performance Management in 2021? What do you think of their choice of companies? Thanks!
2 out of 7 answers
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Dec 10, 2021
Hi @Tjeerd Saijoen, @Ravi Suvvari, @mehrab hussain ​and @Ravi Khanchandani, Can you please assist here? What are your thoughts on the recent APM MQ? Thanks!
Shibu Babuchandran - PeerSpot reviewer
Regional Manager/ Service Delivery Manager at ASPL INFO Services
Dec 11, 2021
Hi, Looks like the 4 players below are in the Leaders position: -Dynatrace-AppDynamic-New Relic-Datadog Anyone with a new update, please kindly share.
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jun 20, 2022
If you are considering DevOps tools for your business, you want to be sure you choose the right solution. DevOps software is a rapidly growing market, and while it can be hard to know which tools you really need, researching all of your options and making the right choice does not have to be a daunting task. To help you choose the best DevOps solution, here are some tips: Develop a common...
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jun 20, 2022
Hi PeerSpot community members, This is a fresh-from-the-oven Community Spotlight for you. Here, we've summarized and selected the latest posts (professional questions, articles and discussions) by PeerSpot community members. Check them out! Also, please share with us your feedback and suggestions by commenting below! Trending See what is trending at the moment and chime in to discuss! ...
Deena Nouril - PeerSpot reviewer
Tech Blogger
Jun 17, 2022
PeerSpot’s crowdsourced user review platform helps technology decision-makers around the world to better connect with peers and other independent experts who provide advice without vendor bias. Our users have ranked these solutions according to their valuable features, and discuss which features they like most and why. You can read user reviews for the top Application Performance Management (...
2 out of 3 comments
Abbasi Poonawala - PeerSpot reviewer
Chief Enterprise Architect at Bank in KSA/JPMORGAN
Jun 16, 2022
How about pricing strategy?  Why do top banks in MENA still prefer Dynatrace over New Relic or AppDynamics?🤔 
Abbasi Poonawala - PeerSpot reviewer
Chief Enterprise Architect at Bank in KSA/JPMORGAN
Jun 16, 2022
@Abdulla Pathan, I appreciate it if you can share more insights & your valuable experience around APM adoption @LearningMate. 
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jul 11, 2022
Hi community members, As usual, this new Community Spotlight shares with you the latest articles, questions and trending discussions from your peers. Trending See what is trending at the moment and chime in to discuss! Top 8 Extended Detection and Response (XDR) Tools 2022 Would you recommend replacing Cisco ASA Firewall with Fortinet FortiGate FG 100F due to cost reasons? What is the...
See 2 comments
Ravi Suvvari - PeerSpot reviewer
Performance and Fault-tolerance Architect with 1,001-5,000 employees
May 30, 2022
Good very informative
Jairo Willian Pereira - PeerSpot reviewer
Information Security Manager at a financial services firm with 5,001-10,000 employees
Jul 11, 2022
Analyze the wave of product at Gartner Hype Cycle. EDR was good in the past. After that, MDR joined the hype and now, XDR is the trend. Wait for more in a couple of months and (sic) know the ZDR!
Shibu Babuchandran - PeerSpot reviewer
Regional Manager/ Service Delivery Manager at ASPL INFO Services
Nov 1, 2021
What Is AIOps? AIOps is the practice of applying analytics and machine learning to big data to automate and improve IT operations. These new learning systems can analyze massive amounts of network and machine data to find patterns not always identified by human operators. These patterns can both identify the cause of existing problems and predict future impacts. The ultimate goal of AIOps is...
See 2 comments
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Sep 14, 2021
Great article, @Shibu Babuchandran! Thank you for sharing your knowledge with the community!
Jairo Willian Pereira - PeerSpot reviewer
Information Security Manager at a financial services firm with 5,001-10,000 employees
Nov 1, 2021
I always like this order a lot: "Consider People and Process" and only after, Technology.
Related Articles
Janet Staver - PeerSpot reviewer
Tech Blogger
Jun 20, 2022
How to Choose the Best DevOps Tools for your Business
If you are considering DevOps tools for your business, you want to be sure you choose the right s...
Evgeny Belenky - PeerSpot reviewer
Director of Community at PeerSpot (formerly IT Central Station)
Jun 20, 2022
Community Spotlight #16
Hi PeerSpot community members, This is a fresh-from-the-oven Community Spotlight for you. Here, ...
Download Free Report
Download our free Application Performance Management (APM) Report and find out what your peers are saying about Datadog, Dynatrace, New Relic, and more! Updated: November 2022.
DOWNLOAD NOW
653,522 professionals have used our research since 2012.