New Relic Reviews

Over the years I’ve come to rely on information radiators during testing to get immediate (or as quick as possible) feedback from the systems I’m testing.

Firebug, log files, event logs and many other sources of information are all very useful to a tester. They can give you insights in to what is happening in the system under test.

We’ve just taken this a step further by rolling out NewRelic on our test servers.

NewRelic is what’s termed a “Application Management Solution”.

I’ve been talking about this internally as a system that can give us three distinct insights:

• User Experience Information
• Server Information
• Product Performance Information

I’ve probably over simplified the tool and doing an injustice but it allows me to clearly explain the value we’re seeing from it.

User Experience Information

NewRelic gives us all sorts of data around how the experience is for end users when they use our product.

We can use this to ascertain how our product is being experienced by our customers, but we can also use it to understand how the experience is stacking up for our testers.

If we are testing and we observe a slow down we can check whether it really was a product slow down using NewRelic and more importantly; what’s actually happening on the stack.

We can use NewRelic to work out what browsers are being used across all of our environments. We can see the test coverage we have across browsers and we can also see what browsers our own business use from our pre-production test environments (where we test all kits before live deploy).

We can also then see which browsers are faster than others. We can see which versions are used and which browser is our most heavily used. Interesting stuff to help guide and tune our testing.

Server Information

NewRelic monitors the actual servers giving all sorts of information such as memory, CPU, process usage etc etc. This is great information on our test servers, especially during perceived slow downs or during a load test.

We have other mechanisms for measuring this also so this is the least used function in NewRelic when testing.

Product Performance Information

For me, this is the greatest information tools like NewRelic offer; they show you what the product is actually doing.

It includes what pages are being dished, how fast are they being dished, where they may be slow (in the DOM? Network?), what queries are being run, what part of the code is running them and how often they are being called.

When we dig around in the data we can find traces that NewRelic stores which give an amazing level of detail about what the product is/was doing when the trace was run.

It’s going to become a testers best friend.

In a nutshell what it allows us to do is provide an accurate picture of what the product is doing when we are testing. This means we can now log supremely accurate defect reports including traces and metrics about the product at the moment any bugs were foud.

The programmers can also dig straight in to any errors and be given the exact code that is generating the error.

We can see which queries are running meaning that if we encounter an error, a slow down or something worth digging in to we have the details to hand.

It’s still early days using the tool but already we’ve had deep insight in to how the product runs in our environments which I’ve never been able to get from just one place.

It’s immediate also. Test – check NewRelic – move on.

Imagine how powerful this could be on your live systems too.

Imagine the richness of information you could retrieve and imagine how fast you could get to the root cause of any problems. It’s powerful stuff. Expect to hear further posts on how tools like this can inform tests, provide a depth of supporting information and provide help to performance testing.

Some notes:

• There are alternatives to NewRelic.
• It’s still early days but tools like this are proving invaluable for accurate and timely troubleshooting and information gathering.
• I’m not affiliated to NewRelic in any way – I’m just a fan.

Introduction

New Relic has become an industry standard for web applications performance measurement in Rails community. Simply adding and configuring newrelic_rpm gem for your Rails application gives you a great overview of its performance with an adequate level of details.
There are still a few improvements you can do to make use of some cool and extremely useful features of New Relic monitoring. They will help you to save time and efforts when fixing performance issues, scaling application, refactoring, etc.
So what should you do to make fast and effective use of New Relic?

Record your deploys

Always record deploys with your capistrano or chief scripts. This will save you time when looking for reasons for performance improvements or drawbacks.
A detailed information about recording deploys using capistrano can be found in official documentation. You can also use simple API deployment notification described here.
Either way it is important to add a clear description of deploy along with a tag or branch name.

Note that Pro users can also see ‘Change report’ for each deploy with a detailed overview of performance metrics changes (response time, throughput, apdex etc.).

Setup additional metrics before you start

It is important to setup some additional custom metrics right after configuring and enabling New Relic before you have experienced performance issues. You can read more about custom metrics collection here.
While you can add traces to almost any method in your code, they are commonly used for tracking important/critical parts of your code. We strongly recommend you to add traces for:

• application critical parts of code;
• external services communication (redis, RPCs, external APIs, DNS resolve etc.);
• encoding/decoding methods (some json encode methods implementations can be slow for large objects).

These traces are useful even within internal network. Drawbacks may be caused by network lags or hosting provider problems.

Here are some useful examples of custom traces that you can easily add to your initialisation pipeline (e.g. application.rb).

Resque

Resolv::DNS

Redis

Setting up additional metrics have two main goals:

• measuring performance of specific parts of code;
• when investigating slow requests with a lot of ‘Application code’ segments these additional traces can be used as benchmarks.

Additional metrics will give you more detailed view of what and when is going on in your application.

Note: New Relic gem implementation already has traces for Net::HTTP methods.
Caution: Don’t be zealous on adding custom metrics, clean obsolete ones.
Collecting too many metrics can impact the performance of your application. It is strongly recommended to keep the total number of custom metrics under 2000.

Setup Server Monitoring

To look beyond application itself New Relic has introduced Server Monitoring tool.
It is critically important to setup Server Monitoring for all servers in production and even staging hosting scheme. It will help you to measure your environment performance and state.

Note. New Relic is including Server Monitoring for free for every subscription level.
You can find all required information about installation and configuration of Server Monitoring tool here.

Setup custom views and dashboards

In most cases application performance strongly depends on application state and users activity. It is also helpful to see how your application performance correlates with business metrics, database activity, internal services state etc. New Relic gives you an opportunity to setup custom metrics, group and organise them with views and dashboards.
Common useful custom metrics you should define for your applications are:

• number of active users;
• users activity metrics;
• number of active application’s core entities (e.g. active tables for poker application);
• number of connections/calls for internal/external services (some have specific thresholds);
• database additional metrics.

Official documentation will help you to find more about creating and managing custom views and custom dashboards.

You can collect any custom metric and setup a specific custom view/dashboard in order to have general view on how they correlate.
Along with monitoring, custom dashboards and views also:

• helps you to understand the reason of performance changes;
• gives you a vision of future load changes depending on custom metrics;
• provides an ability to predict critical thresholds for load values (e.g. active users);
• helps you to improve application architecture by showing bottlenecks of internal infrastructure.

Note: The Custom Dashboards feature is available with New Relic Pro and higher.
It is useful to track only those custom metrics that have impact on overall application performance.

Setup background jobs/tasks monitoring

Always trace your background jobs performance. While in most cases they doesn’t directly impact on server response time they still can impact on user’s feedback and overall application performance.

Currently the Ruby agent supports only Delayed::Job and Resque.
Read more about monitoring Ruby background processes and daemons here. Make sure you have enabled and configured everything correctly.

Within one of our products we have employed an actor based architecture where actual action is executing asynchronously in a separate thread (we use JRuby). In such complex cases, it is useful to add traces for these actions and mark them as real web transactions (controller actions).

To complete such tasks or customise your add_method_tracer options please check API documentation.
For more detailed information about available options please read perform_action_with_newrelic_trace API documentation here. The most important options of tracer methods are:

• :category — defines action type (available options: [:controller, :task, :rack, :uri]);
• :name — action name (will be used as last part of metric name);
• :params — call context parameters;
• :class_name — first part of metric name. Default is a current class.

Here is a small example of custom background task implementation using JRuby thread actors.
ApplicationController:

Scheduler::WorkerActor:

So in this case all enqueued tasks and controller actions appear in ‘Web transactions’ together.

Summary

New Relic is a solid monitoring solution, especially for Rails developers.
Make sure you have enabled and configured all basic features described in this article.
And moreover — always learn your tools.

Explained by example

The best way to explain how New Relic can help, is by example.

A few weeks back, one of the magento based webstes I manage had some serious delays at checkout. After 'submit payment' was clicked, some users reported that the site could take up to 40 seconds (and in some instances just timed out) to respond back with the resulting page. Yikes! This was just unaccepatble.

After spending nearly 5 hours tracking this down (without New Relic), and not getting a clear view of the reasons, and not being able to replicate the issue, I signed up to New Relic, and hooked the site up.

And within an hour I had the reasons, and a fixed was placed on the site.

New Relic 'Key Transactions'

New Relic has this great new feature called ' Key Transactions'. This feature gives you the ability to mark any transaction as important, and it will appear in a new menu called 'Transactions'. This feature makes it a lot easier to monitor the transactions you are specifically interested in. In my case, I marked the checkout process (/onestepcheckout/index/index) as a Key Transaction.

Ok, so now I can monitor the actual checkout process specifically, and I am ready to collect information on what the hell is going on.

I placed a test order. And yep, I could not replicate the issue, all looked good, and I got the result page within a few seconds.

But all was not lost. All I had to do was wait for New Relic to collect details about every checkout, and I coud analyze the results.

It did not take long to finally find the reason behind the intermitted delay.

The trace from New Relic for a transaction that took more than 20 seconds to complete:

as easy as that, I now had the reason for the delay: Email mailing list subscriptions at checkout!

The reason I could not replicate this issue was because my user was already subscribed to the mailing list, thus, at checkout I would never got the 'subcribe to mailing list' checkbox.

Now that I knew what was going on, I could easily relicate the issue, implement a fix, and eliminate the delay at checkout.

Easy as.

The conclusion

Without the help of New Relic, it would have been near impossible task to track this issue down. New Relic not only saved me hours of frustration, but saved me hours of time, and allowed me to quickly, effectively target, and fix the problem at hand.

Simply put: New Relic is a developers best friend.

Valuable Features:

Room for Improvement:

Other Advice:

Valuable Features:

Room for Improvement:

Other Advice: