There are so many monitoring systems in every company that provide alerts. This is what's called - alert fatigue phenomena. Moreover, most alerts are handled manually which means it takes a long time and cost a lot of money to resolve an event. I think companies should evaluate the remediation part of monitoring systems... What is there to instantly and automatically resolve problems that come up instead of just alerting.
Full disclosure I am the COO at Correlsense.
2 years ago I wrote a post just about that - "Why APM project fails" - I think it can guide you through the process of the most important aspects of APM tools.
Take a look - feel free to leave a comment:
In order to evaluate/benchmark APM solutions, We can based on the 5 dimension provided by Gartner:
1. End-user experience monitoring: the capture of data about how end-to-end application
availability, latency, execution correctness and quality appeared to the end user
2. Runtime application architecture discovery, modeling and display: the discovery of the
various software and hardware components involved in application execution, and the array of
possible paths across which those components could communicate that, together, enable that
3. User-defined transaction profiling: the tracing of events as they occur among the components
or objects as they move across the paths discovered in the second dimension, generated in
response to a user's attempt to cause the application to execute what the user regards as a
logical unit of work
4. Component deep-dive monitoring in an application context: the fine-grained monitoring of
resources consumed by and events occurring within the components discovered in the second
5. Analytics: the marshalling of a variety of techniques (including behavior learning engines,
complex-event processing (CEP) platforms, log analysis and multidimensional database
analysis) to discover meaningful and actionable patterns in the typically large datasets
generated by the first four dimensions of APM
In other side, we tried to benchmark internally some APM solutions based on the following evaluation groups:
Technologies and framework support
Central PMDB (Performance Management DataBase)
Service modeling and monitoring
Performance analysis and diagnostics
Dashboard and Visualization
Setup and configuration
and we got interresting results
Most vendors have similar transaction monitoring capabilities so I look at the End user experience monitoring features to differentiate. Not only RUM (mobile and web) but also Active Monitoring through synthetics.
Hi As per exp.. customer always look how much faster application its open, how many users are accessed in a faster way...how are then got an errors....application page load....end user request load on application ....how the server resource utilized the application....is there any more resource are need to run the application
In my opinion, an APM solution should demonstrate the following characteristics:
Ease of use. The solution should be easy to use from installation, configuration and maintenance/support.
Fulfill the business need. There are so many solutions/vendors out in the marketplace that claim to have the RIGHT solution for you. After carefully evaluating some of the solutions, you find that they do not satisfy your business need and promises to have the additional functionality in their future roadmap. Why wait?
Integration. An APM solution should have the ability integrate with various other tools/systems without disrupting your current environment.
Cost effective. In most cases, "you get what you pay for". Free isn't always the best solution for your business need, as well as paying a substantial price tag for a solution and still does not give you the value you need.
Application Performance Management is a shared concern. APM tools should be used, easily, by all people involved in the build and run process, to share a common view of which part of the application/system is used and how it impacts the whole performance of the service according to the user/customer point of view.
My scope for a monitoring system is the capability to monitor:
- FC switches
- Storage (SAN, TapeLibrary,...)
These are the main components where the monitoring system has to get as much data as possible out of the systems.
Here is the HA of the monitoring system very important.
We have tested CACTI and Nagios and SCOM but we are the most happy with OpManager of ManageEngine. because the correlation with the other components is so easy and logic we prefer and recommend the solutions of ManageEngine.
- Server Monitoring capabilities
- Ease for user i.e. easily configurable and end-user experience
- Alerts and events management
- Dashboards available so you can visualize the stats
- Analyse performance and diagnose problem
- Must be under budget
We mostly recommend Nagios and Cacti as tools to our client.
These posts hit my main points on the nose.
Bob_J " 1. End-to-end transaction visibility 2. Scalability 3. Low/no over-head 4. Ease of use "
reviewer142791 - "Being able to see EVERY transaction. No sampling or just high water marks. If you don't have information on all of your data, you can't effectively benchmark and trend performance over time."
I would also look for transaction topology mapping. I picture paints a 1000 words in expressing urgency.
When you want to evaluate an APM Solution in my option the most important aspect is
1) What are my organization needs and if that solution partialy complies to the needs.
If we know for what we need the APM Solution we go for the other things:
2) Monitor End-to-end transactions
3) Integration with monitoring tools
Usualy after implementing the APM Solution the client wants an unified solution that agregates the data from the APM and from the monitoring tool.
4) Synthetic transactions
Every organisation has it's own needs. Sometimes the client doesn't choose the solution they really need because of the price, and in time they discover that their choise is not fullfilling it's purpose and they need to buy other solution.
6) Event correlation
It is ideal to correlate the events from a server with the events from the APM. The correlation helps the administrators to solve the issue ASAP when you have all the data.
7) Easy to use
If the solution is easy to use, it gives the administrator the time to develope new rules and it give him the ability to learn the solution faster.
8) Ability to analyse application code for flaws
If you see errors in the APM from your applications, usually the application owner is asking what it will be the cost for troubleshooting and for error remediation. If the APM is able to provide the flaws, than the cost of repair is much lower than expected.
1. The APM solution getting implemented should not have an additional overhead on the system performance in terms of server resources. ( Very Important aspect)
2. The APM solution should be compatible with the technology /software you are trying to use.
Well, the most important: the solution covers your application stack and your ecosystem, and this happens at the implementation and in the roadmap of your company. Then you should decide having in count: easy of use, data integrated from multiple sources, transactional view and in-depth root cause analysis available. All platforms of this type performs well and are based in very similar concepts. After a pilot or PoC you will have more light in this field.
1. Low Overhead
2. Ability to use it in production, test and development
3. No Sampling or just turning it on when there is something suspicious going on -this is to late to find the root cause
4. Ability to follow exactly end to end transactions from the click/touch to the database
5. Real User Monitoring for Web, Mobile Apps and Rich Clients
6. Ability to give APM data to different audience in a convenient way. eg. CEO can view high level data on a tablet, Developer can deep dive on stored data a long a bug report, operators can have their personalized view of the system on the ops screens
One suite which would cover
1. End-to-end business/audit transaction flow irrespective of technology stack/Integrated components,
2. NFRs/Infrastructure/capacity metrics and their management as well
4. Easy configurable/manageable.
5. Plugin availability for integration for customization.
6. Of Course budgetary/cost saving in long run.
By default APM tool/suite should provide evidence/root cause of performance bottleneck and notify reaching threshold of configured KPIs.
1. How the tool affects application load is most important. An APM tool is useless if it slows down the production environment.
2. Ease of installation.
3. Ability to drill down into the application stack for root cause analysis.
4. Alerts on performance and errors.
5. Ability to customize the data stream.
1> Monitoring application performance with the help of APM introscope.
2> Being able to see EVERY transaction. No sampling or just high water marks. If you don't have information on all of your data, you can't effectively benchmark and trend performance over time
Nothing can be more important than end user visibility.
If it wasn't for the end user, what metrics would have any value since we talk about Application Performance Management?
The judges of the performance is the end users - period. It can never be the tools!
In a APM solution, there needs to be a direct correlation of what a user does to what resources that particular action demands from the infrastructure. Of course the time spent in those resources is also valuable.
For one, the APM tool needs to be stable and be able to notify me when it is unable to complete the configured monitoring. After that, it is important to be able to monitor log message occurrences for the software that my company produces.
I am not an actual user of these products. I am a consultant that offers comparative research support for clients with vested, potential investment or consumer interest in them or at times, for the developers themselves.
End User Experience along with its operation time,slow operations and the application performance.
For the web based applications,I think we should look the TCP and HTTP errors which are basic parameters.
For SSL Based applications,it is TCP and SSL errors.
Knowing what you wil do with results. Match them with technical and functional requirements and think about what information you would need if you need to troubleshoot the applictaion chain. Then look for a tool that supports this.
1. Identify problems and pinpoint root-cause.
2. Understand your application or website’s readiness before go-live.
3. Shorten application release cycles.
4. Understand and optimize your end-users’ journey.
5. Understand outside forces affecting your application and/or website.
6. Simplify deployment and management of your APM platform.
scalability and flexibility come first, in order for an Organization to -proactively- runs its business and -effectively- adapt to this fast growing business environment. Then comes all the other obvious APM capabilities and benefits customers expect, some listed above.
technologies (big data, apps, mobile, cloud, etc) have been quickly evolving... an organization can only run its IT Operations and be more proactive if it relies on APM that are built on 3year old (or less) technology stack, so the APM is flexible, scalable and can adapt to any business and technological changes, in an ever increasing big data, mobile, cloud and highly integrated environment.
most other APMs that are built on 5-15year old technology stack may have more "analytics" pre-configured, but it takes weeks, months to add more capabilities to them and make them scalable and flexible to have these legacy systems keep up the pace with this ever/quickly changing world.
take a look at:
Germain APM (Application Performance Management)
Germain CRT (Code Review Tool)
Enable Complex Distributed Application Architecture Discovery.
Enable Business Transaction and Internal Call Discovery, bottleneck and latency discovery.
Enable Business and IT Customisable Dashboard build.
Enable Alerting based on Pattern / Baselining.
A Way To Be Pro-Active whether used in QA Environnement, or Efficient on Production issues analysis.
A Must Have In Today, RealTime, Distributed, Multi-tiers Applications
Ability to discover flow transaction, capture all business transaction. Easy to use and to deploy. Also have customize dashboard, as my user comes from different role (management, it operation, application owner, business development, etc)
Monitoring application performance with the help of APM
Visibility to see All transaction. Not a sampling or just high water marks.
The ability to see what the End user is experiencing is most important.
Problem solving not finger pointing
Thanx Ariel for starting interesting APM ongoing forum, lot of valuable info/inputs from our peer's couldn't stop liking those valuable comments. I would personally suggest ease of APM solution which supports multiple diversified Apps ,DB's,O/S all in one shot for complete system including Upstream/Down stream systems and produces transaction level reports which can be easily drilled down and can be understood by Non technical management teams also.
- each transaction must be monitored and analyzed
- the solution has to correlate traffic generated by a user with its true EURT measure
- any kind of metrics has to be suppored by its pcap, so the system has to be able to extract automatically the pcap related to the analyzed detail
- great reporting and dashboarding
1. Doing what it has promised, helping root cause analysis
2. End-2-end transaction monitoring (end user+data center) without sampling
3. Discovery of application topology
4. Not affecting apps that are monitoring