You are hereHow to use Google Analytics to Monitor Your Servers (or anything else!)

How to use Google Analytics to Monitor Your Servers (or anything else!)


By steve - Posted on 17 July 2014

Most people in tech know that you can use Google Analytics to monitor website performance. App developers and website hackers are probably aware that you can monitor your users' use of your apps for your own custom events. However, very few people know that you can use Google Analytics to monitor your own Windows and Linux (and anything else) servers' disk space, performance, and any other metric you want!

One of the best sites I've found for monitoring uptime of my services is Up Time Robot. They do a great job monitoring access to my websites and critical services like SMTP for email and SSH or whatever you like. I can even make sure a certain phrase (like "user login") is appearing on a certain web page! But that only works for internet-facing services.

Many of my cloud service providers (like Rackspace and Amazon, and Digital Ocean) have monitoring services for my servers they are running, but that doesn't help for my client's servers behind the firewall.

After just a little fiddling around, I've come up with an easy way of monitoring key metrics on all of my servers using "Universal" Google Analytics...

The idea of this little project is that you can send a simple HTTP POST command to a GA server and and it will record it. Then, later, you can see the results of all this data collection.

What I'm doing is using the Google Analytics Collection API for Measurement Protocol to track arbitrary data points. As of July 2014, this is described here in the documentation.

I'm going to develop a very simple example that tracks disk space used on a Linux server using two different parameter types of the Measurement Protocol. I'll create a bash script that checks a volume's disk space and submit the data to Google once a day using curl (I'll tell you why I'm limiting this to once a day, a little later).

Here are the overall steps set this up:

  • Set up a Google Analytics account.
  • Set up a "site", which can represent your server.
  • Enable Universal Analytics, if it isn't.
  • Enable ecommerce, if it isn't.
  • Create a script that POSTS information to GA.
  • Test the script until you see something show on GA's real-time reporting.
  • Schedule the script to run once a day.
  • Create a custom report to show your values.
  • Watch your server history grow!

I'm not going to cover the first 4 steps as there is plenty of information about this on the Googles.

Let's talk about the script, though. Here's a simple script that posts one value you want to track to GA (note that the curl lines are wrapped for readability but should each be all one line):


#!/bin/bash
# Tell GA about key metrics on the server
myTid=UA-XXXXXXX-1
myCid=1
myDiskWarningLevel=80
# Disk space used
# Use df to tell us how much disk space%
# is used on the system drive
myMetric=$( df -Pl|grep '^/dev'|awk '{print $5}'|sed 's/%//');
# ecommerce needs a unique transaction ID
myTI=$(date +%s);
# Example 1: Use ecommerce parameter data collection
curl -d "v=1&tid=$myTid&cid=$myCid&t=item&ti=$myTI&in=DiskPercentUsed&iq=$myMetric" http://www.google-analytics.com/collect
# Example 2: Use event parameter data collection
curl -d "v=1&tid=$myTid&cid=$myCid&t=event&ec=System&ea=Disk&el=PercentUsed&ev=$myMetric" http://www.google-analytics.com/collect
# Example 3: Issue a conditional warning event
if [ $myMetric -gt $myDiskWarningLevel ]; then
  curl -d "v=1&tid=$myTid&cid=$myCid&t=event&ec=SystemDisk&ea=DiskLow&el=LowDisk$myMetric&ev=$myMetric" http://www.google-analytics.com/collect
fi

Replace the value of myTid with your GA tracking ID. This can be the same as your website's ID, but it has to be an existing "property" to work. Go to Administration/Property Settings/Property Settings to find yours.

So here's what we are doing...

Example 1

We are using the ecommerce functionality to create an "item" transaction that has a quantity that is the value we care about. We can report on the quantity or average quantity later. Our item name (in) is DiskPercentUsed and the quantity is the percent used that we collected earlier.

Example 2

Here we are using the Event functionality to log an event that has the values we care about. We've logged an event category of System, event area of Disk, event label of PercentUsed, and an event value of the percent that was used. We have a lot more categorizing we can do with the Event stuff, but we are much more limited in GA on what we can do with values. We can total them and we can average them and that is about it.

Example 3

The 3rd example is a case where I only want to log the exception. When the disk used gets over 80% in my example, an event will be logged. This is handy for logging special events and alerts. I'm logging the same data as in Example 2, but I'm flagging with a label that has the value and including the value in the value. Later, we'll see how we can trigger an email or text when this event happens.

Take this sample and modify it for your situation. If you want to run it each day on Ubuntu or similar systems, save it as google-analytics-update, put it in /etc/cron.daily and chmod it to 755. If you don't know what that means, you shouldn't be trying this!

You can run it one time: /etc/cron.daily/google-analytics-update

If you don't get an error (you will probably get a line that says GIFs0230 or whatever), you logged your first event.

Now, the way GA works is that it will COUNT most events. If you run this script with events and it logs a GA event, it will log one event with the disk space as of that moment. If you run it again, you'll double the value for the "total" for the data, and things get complicated. This is a simple example for a simple exercise, so don't run it lots of times and then wonder why your disk% is 1000%! Run it once a day and you're fine!

Getting Your Analytics

So now we are recording things, so let's go to GA and see what is happening.

If you go to GA/Real-time/Events, you'll see a window that goes back 30 minutes and shows your test. There's a link called Events (Last 30 min) that has the details of your tests.

For events, you can go to Behavior/Events/Overview and see what is happening. It looks like this:

Extra credit: For a more interesting graph, I changed the process to run EVERY HOUR! Then I created a report that compares Unique Events (24 per day) and Avg Value. So, if my disk space example was giving 31%, then 32%, etc., I could see the impact over time and have more data points. This, however, will mess up the the "default" presentation of events where it tries to total the values. Having my total of 24 hours x 31% as my default graph value of 744 is pretty worthless! This is why for my demo/example, I suggest only running the disk check each day. That way, your graph will show your percentage usage over time as it grows or shrinks.

If you are trying the ecommerce example, you can go to Reporting/Conversions/Ecommerce and poke around there.

Alerts

So, we are collecting data every day about different events on our server. So what if our server gets to 80% of the space used, do we want to wait until we see that to know that? Nah. We want to know right away!

Here's how:

  • Go to GA
  • Go to Admin
  • Go to your account/property/view
  • Scroll down to Custom Alerts
  • Click "+New Alert"
  • Give it a name: System disk low warning
  • Pick a period (Day)
  • Check "Send me an email"
  • Set up the text message alert for your phone, if you want
  • Create the conditions:
    • Event Category/Matches exactly/SystemDisk
    • Alert me when/Event Value/Is greater than/79
  • Click "Save Alert"

You should test this by forcing the value to be more than what you put (79) and running the script that posts the event. You should get an email and your optional text.

What Else?

Anything you can track that is event based works well for this kind of thing. How about a motion sensor on your driveway hooked up to a raspberry pi that logs an event when a car goes by? How about the temperature in your office?

The sky's the limit! Well, there are some limits for what GA will allow you to log and how many times per month, but at (currently) 10 million hits per month per property, you should be OK as long as you avoid a "runaway" script!

Summary

What have we learned? Well, GA is not a great place to stick "readings" per se if you want to do straight graphing, but it is a great place to put events. You really are putting a round peg in a square hole, if you are really stuck in the readings mode. However, if you are willing to play and averages mean something to you, you really can use GA to record anything you want for any "event" based monitoring.

Did this help you? You can help me!


Did you find this information helpful? You can help me back by linking to this page, purchasing from my sponsors, or posting a comment!


+One me on Google:


Follow me on twitter: http://twitter.com/mojocode







Comments



Affiliation Badges