Hi everyone,
I'm looking to purchase an APM solution, and looking for some feedback.
I've shortlisted AppDynamics, New Relic, Dynatrace, Datadog, as well as some others like Big Panda and Corelogic that might be used to bolt on as value add. I have experience with AppDynamics, Datadog and Dynatrace but not New Relic - I'm leaning towards New Relic though.
I want to hear how you did your discovery/research, and what were some deciding factors when you finally picked a solution (E.g.: New Relic has xyz but the fact that Datadog gave us 1 and 2 and sort of has xyz made us go with Datadog).
Some background:
I have background in DevOps, Java and Cloud. Theoretically, my group is DevOps but we're still developing in this area. I'm looking to modernize and automate more, and get rid of all the noise. With whatever solution I end up choosing, I want to improve monitoring and do app tracing for developers. I also need predictive analysis. We're currently doing everything on-prem but we have plans to move to the cloud, so the solution needs to support both on-prem and cloud.
I've put together a survey to help me with researching the tools. I'd really appreciate if you could take a few minutes to fill in the survey to help me out: APM tool survey
Thanks!
There are many factors and we know little about your requirements (size of org, technology stack, management systems, the scope of implementation). Our goal was to consolidate APM and infra monitoring. We maintain critical processing on our mainframe so there was a desire to include this in our transaction trace. Due to a highly mature ELK implementation, we are not trying to incorporate log analytics into solution buy may consider in the future. We had AppD, Dynatrace, New Relic, and CA Wily all in house at the time of our evaluation. We eliminated Datadog due to a lack of real user monitoring and AppD based on experience and licensing. Between Dynatrace and New Relic, Dynatrace won based on the automation, integrated AI, support for "old" techs, and confidence we could eliminate multiple APM and infra monitoring tools.
I would not include products like BigPanda, MoogSoft, in this analysis. They are not monitoring solutions but event correlation solutions. You will need additional monitoring products to capture data and feed them. Having said that if you cannot consolidate tools you will likely need to purchase an event solution to make sense of all the alarms. We did evaluate these products but with Dynatrace AI did not feel the business value was there for the investment.
Here's a quick pro/con list on Dynatrace & New Relic from our analysis.
New Relic Pros: Insights is an awesome product and capability. Lots of capabilities and plugins to extend data collection. The APM dashboard is aesthetically pleasing and intuitive. Good training and documentation are available to support the product.
New Relic Cons: Requires lots of manual configurations to implement and support. Insights product requires an investment of time to achieve value. Licensing is a nightmare as there is virtually no transparency in what you are being charged for. Lack of solution to consolidate alerts across implementation other than significant investment in insights to manually achieve this.
Dynatrace Pros: Very simple to implement and maintain with out of the box automation which supports modern (cloud/Kubernetes) and "old" (mainframe). In-app chat is helpful. High integration of infra and APM data for full-stack observability and engineering. Topology and trace discovery is more reliable than other products or our CMDB. Synthetics are easy to set up for any user. AI-assisted problem analysis on the trace discovery streamlines troubleshooting. AI includes "events" in an analysis like VMotion, deployment events. Have not done yet but looking to leverage monitoring as code for a fully integrated and automated delivery pipeline. See keptn.sh open source project.
Dynatrace Cons: User SQL lacks some functions of NRQL for user analysis. Host, process, and service data is not available to query within the product. Alarm processing lacks some granular controls. The Plug-in library is less robust.
Good luck with your decision!
@reviewer1352679 thanks for weighing in with such a detailed answer for @SystemsEngineer234z :)
We are currently going through a paper-based analysis to select an Enterprise APM solution.
Our Contenders are
1. Dynatrace
2. Cisco(AppDynamics)
3. Broadcom DX-APM
Shortlisted based on existing relationships with other products and services they provide.
We discounted New Relic- despite their growing capability - as they are yet to enter the enterprise APM solution scene.
With regards to your response "We eliminated Datadog due to a lack of real user monitoring and AppD based on experience and licensing .." :
Will you be willing to expand on Appd - what was your experience and issues w.r.t licensing. These could help us with our evaluation. Much appreciated. Regards Adrian
@Adrian Antonypillai Easiest deployments? this is easily Dynatrace. You deploy the oneagent on the host and much of the rest is taken care of for you. There is a lot of automation offered by the solution for deployment and maintenance. For example, you don't need to configure variables in you JVM startup or select which libraries (java vs .net...) need to be loaded.
Decommissioning Introscope? The biggest transition for your clients/users is navigation in the user interface. Plan on training for users. Each product is backed by different "models" and transition from one tool to the other is a learning experience. Our transition from Introscope was easy as there were so many features not in APM, teams were eager. After training depends on use case. If you are just monitoring application instances it's just a matter of removing the references to Introscope in the application startup. If you pump in data, the API will obviously change and would be a bit more work. From what you state, I'm assuming it is monitoring of applications and you will be awe struck with the simplicity of Dynatrace vs Introscope.
Transitioning Introscope config? By default Dynatrace is instrumented with alarms using baselines. Simply by installing you get a good base of monitoring. The additional visibility provided out of the box should make you re-evaluate any alarming you had in Introscope. Because it was in Introscope, don't think you must migrate it. Beg benefit is Dynatrace comes with AIOps so alarms are handled differently. You can extend monitoring with JMX and custom alarms as necessary. Of everything I like about Dynatrace, the alarm configuration lacks some flexibility I would like. You should think about how your are going to manage processing alarms for each team. Ideally automate your alarm configuration via the API to handle the configuration as code.
How long does migration take? To remove Introscope, the middleware team wrote a script to remove from the application start up files. That took minutes to run. To deploy Dynatrace, you install the agent on the host and restart the app. So the time to deploy depends on your deployment automation and application restart time. Again, I would suggest prior to migrating, you have engaged the application developers/support team and they have been introduced prior to a non-production roll out. Deployment is so easy, training is your bottle-neck.
Physical to virtual to cloud to kubernetes? We have all of the above plus mainframe, thick client apps and SaaS. The product is used across all these types of implementations and a big reason Dynatrace was selected. They have the benefit of existing in the "legacy" physical server world and was re-designed for the cloud. My $.02. If you are on-prem only, choice is between Dynatrace and AppD. If you are cloud only, choice is between Dynatrace and New Relic. If you are heterogenous, it is only Dynatrace.
The key is to have a holistic view over the complete infrastructure, the ones you have listed are great for APM if you need to monitor applications end to end. I have tested them all and have not found one capable to give me solutions they all report you have a problem but you need to figure out what the problems is. So we have decided to develop our own AI and it is not only monitoring OSI layer 1 -3 but layer 1-8 as well it give you a solution what has been wrong based on delays in transactions combined with security. We have deployed this now across healthcare in the USA as well Airports but it is working on every industry and working in a SaaS solution. But if I have a chose between the tools you mentioned, Dynatrace is the best choose it is easy to install and has a great interface. Also it has almost no configuration.
Could you please share your requirements ? There are a lot tools can be added to the list. I spent almost 6 months to test and check many tools then I select eG enterprise.
Thanks