[Home] [Blog] [Contact] - [Talks] [Bio] [Customers]
twitter linkedin youtube github rss

Patrick Debois

Monitoring Wonderland Survey - Moving up the stack Application and User metrics

While all the previously described metric systems have easy protocols, they tend to stay in Sysadmin/Operations land. But you should not stop there. There is a lot more to track than CPU,Memory and Disk metrics. This blogpost is about metrics up the stack: at the Application Middleware, Application and the User Usage.

To the cloud

Application Metrics

Maybe grumpy sysadmins have scared the developers and business to the cloud. It seems that the space of Application metrics, whether it’s Ruby, Java , PHP is being ruled today by New Relic In a blogpost New Relic describes serving about 20 Billion Metrics A day.

It allows for easy instrumentation of ruby apps, but they also have support for PHP, Java, .NET, and Python

Part of their secret of success is the easy at how developers can get metrics from their application by adding a few files, and a token.

Several other cloud monitoring vendors are stepping into arena, and I really hope to see them grow the space and give some competition:

Some other complementary services, popular amongst developers are:

Check this blogpost on Monitoring Reporting Signal, Pingdom, Proby, Graphite, Monit , Pagerduty, Airbrake to see how they make a powerful team.

User tracking Metrics - Cloud

Clicks, Page view etc …

Besides the application metrics, there is one other major player in web metrics. Google Analytics

I found several tools to get data out of it using the Google Analytics API

With google Analytics there is always a delay on getting your data;

If you want to have realtime statistics/metrics checkout Gaug.es http://get.gaug.es :

A/B Testing

Haven’t really gotten into this, but well worth exploring getting metrics out of A/B testing

Page render time

Another important to track is the page render time. This is well explained in the Real User Monitoring- Chapter 10 - Complete Web Monitoring - O’Reilly Media

Again Newrelic provides RUM : Real User Monitoring. See How we provide real user monitoring: A quick technical review for more technical info

Who needs a cloud anyway

Putting your metrics into the cloud can be very convenient , but it has downsides:

  • most tools don’t have way to redirect/replicate the metrics they collect internally
  • that makes it hard to correlate with your internal metrics
  • it’s easy to get metrics in, but hard to get the full/raw data out again
  • it depends on the internet , duh, and sometimes this fails :)
  • or privacy or the volume of metrics just isn’t possible to put it out in the cloud

Application Metrics - Non - Cloud

In his epic Metrics Anywhere, Codahale explains the importance of instrumenting your code with metrics. This looks very promising as this is really driven from the developers world:

Java

Or you can always use JMX to monitor/metrics from your application

And with JMX-trans http://code.google.com/p/jmxtrans you can feed jmx information into Graphite, Ganglia, Cacti/Rrdtool,

Other

Esty style: StatsD

To collect various metrics, Etsy has created StatD https://github.com/etsy/statsd a network daemon for aggregating statistics (counters and timers), rolling them up, then sending them to graphite.

There have been written clients in many languages php, java, ruby etc..

Other companies have been raving about the benefits of StatsD and for example Shopify has completely integrated it in their environment

It’s incredible to see the power and simplicity of this; I’ve created a simple Proof of Concept to extract the statsd metrics on ZeroMQ in this experimental fork

MetricsD https://github.com/tritonrc/metricsd tries to marry both Etsy’s statsD and the Coda Hale / Yammer’s Metrics Library for the JVM and puts the data into Graphite. It should be drop-in compatible with Etsy’s statsd, although with added explicit support for meters (with the m type) and gauges (with the g type) and introduce the h (histogram) type as an alias for timers (ms).

User tracking - Non Cloud

Clicks, Page view etc …

Here are some Open Source Web Analytics libraries. These are merely links, haven’t investigated it enough, work in progress

Another tool worth mentioning for tracking endusers is HummingBird - http://hummingbirdstats.com/ . It is NodeJS based an allows for realtime web traffic visualization. To send metrics is has a very simple UDP protocol.

A/B Testing

At Arrrrcamp I saw a great presentation on A/B Testing by Andrew Nesbitt(@teabass. Do watch the video to get inspired!

He pointed out several A/B testing frameworks:

And presented his own A/B Testing framework: Split - http://github.com/andrew/split

It would be interesting to integrate this further into traditional Monitoring/Metrics tools. View metrics per new version/enabled flags etc… In a Nutshell food for thought.

Page render time

For checking the page render time, I could not really found Open Source Alternatives.

There is a page by Steve Sounders about Episodes http://stevesouders.com/episodes/paper.php. Or you can track your Apache logs with Mod Log I/O

Conclusion

It’s exciting to see the cross over between both development, operations and business. Up until now only New Relic has a very well integrated suite for all metrics. Hope the internal solutions catch up.

Now that we have all that data, it’s time to talk about dashboards and visualization. On to the next blogpost.

If you are using other tools, have ideas, feel free to add them in the comments.