Greg's network link statistics

One of the less pleasant consequences of moving from Echunga to Dereel in July 2007 was that I lost my ADSL line and had to resort to other methods. After a disastrous attempt to use Telstra's NextG service, I got a satellite connection with IPStar. This was the most unreliable network connection I have ever had. In December 2010, I moved to Internode's NodeMobile Data service.

This page bases on earlier pages that I wrote for the DSL and satellite connections.

The graphs show:

Link status. This is the connectivity to the other end of the 3G link, measured by an ICMP echo (ping) packet once a minute. The value is a reciprocal function of the ping time, 0.5 / ping_time. Thus 5 represents a ping time of 100 ms, and 1 represents 500 ms. 0 indicates a link failure.
Network status. This is the number of remote systems responding to a ICMP echo (ping) packet at the same time as the link status ping. I currently ping 5 systems which I frequently access: freefall.FreeBSD.org, w3.lemis.com, www.auug.org.au, ozlabs.org and ftp.NetBSD.org. Normally this value should be 5. If it's less, the script retries every second until full connectivity is restored.
“TCP speed”, a reciprocal function of the time it takes to transfer a very small “web page” (four bytes) from my external web server. The values shown are reciprocal seconds: 0.5 is 2 seconds, 2 is 0.5 seconds, etc. This graph doesn't directly show the loss of TCP connectivity: as long as the “speed” is above 0, I have connectivity. When the connectivity goes away, the “speed” drops to 0.

This speed was of particular interest with the satellite link. I'm not sure it makes much sense over 3G, but we'll see.

The graphs were made with gnuplot, and I'm not very happy with the smoothing. In particular, the right-hand side of smoothed graphs has too much influence, and it's easy to get the false impression that the link status or TCP speed have changed significantly in the last few minutes. If anybody can point me at a way to fix this problem, I'd be grateful.

Click on the graphs for a 1600x1200 version.

Detailed outage information

With the satellite link I had continual outages, 1685 over a period of 3 years, or one every 15 hours. This section will monitor the 3G link for similar outages, though I hope they'll be much better. I have a program that evaluates the outage information and produces detailed outage information. Here's the information on the past 10 outages, along with the overall statistics:

Start time End time  Duration   Badness        from                    to
                     (seconds)

1367059674 1367105457  45783	  0.013	# 27 April 2013 20:47:54 28 April 2013 09:30:57
1367204688 1367204701     13	  0.036	# 29 April 2013 13:04:48 29 April 2013 13:05:01
1367549800 1367549818     18	  0.010	#  3 May 2013 12:56:40  3 May 2013 12:56:58
1367919005 1367925216   6211	  0.010	#  7 May 2013 19:30:05  7 May 2013 21:13:36
1368277179 1368277224     45	  0.010	# 11 May 2013 22:59:39 11 May 2013 23:00:24
1368447434 1368447469     35	  0.021	# 13 May 2013 22:17:14 13 May 2013 22:17:49
1368644089 1368644103     14	  0.018	# 16 May 2013 04:54:49 16 May 2013 04:55:03
1369109109 1369109124     15	  0.008	# 21 May 2013 14:05:09 21 May 2013 14:05:24
1369116593 1369116617     24	  0.482	# 21 May 2013 16:09:53 21 May 2013 16:10:17
1369231474 1369231492     18	  0.031	# 23 May 2013 00:04:34 23 May 2013 00:04:52

Total 556 outages, total time 855890 seconds (9 days, 21:44:50)
Average time between outages:	136139 seconds (1 days, 13:48:59)
Average duration:		1539 seconds (00:25:39)
Availability:			98.87%

“Badness” is an attempt to quantify the effect. It's the reciprocal of the number of seconds per hour that the link was up between failures (i.e. 3600 / uptime).

And here is the summary information for the past 10 days with a less than 100% record:

Date        Outages   Duration  Availability    Date
                      (seconds)
1366725600	  1	   134	 99.84%	# 24 April 2013
1366984800	  1	 11526	 86.66%	# 27 April 2013
1367157600	  1	    13	 99.98%	# 29 April 2013
1367503200	  1	    18	 99.98%	#  3 May 2013
1367848800	  1	  6211	 92.81%	#  7 May 2013
1368194400	  1	    45	 99.95%	# 11 May 2013
1368367200	  1	    35	 99.96%	# 13 May 2013
1368626400	  1	    14	 99.98%	# 16 May 2013
1369058400	  2	    39	 99.95%	# 21 May 2013
1369231200	  1	    18	 99.96%	# 23 May 2013

The dates in the left columns are in UNIX time_t format to ease further processing. I also have lists of all individual outages, or summaries per day.

The monitor scripts

The scripts that do this monitoring are:

The script linkcheck performs the periodic pings to check link and network status. It writes the results to a log file for later processing.
linkcheck invokes tcpcheck to check the TCP connection, since it could take forever. tcpcheck writes its own log file.
The main script takes a raw gnuplot script and modifies it to create a total of 16 plots: four times (4 hours, 24 hours, one week and forever), two views (smoothed and unsmoothed) and two sizes (540x360 and 1600x1200). I chose 540x360 to fit two plots comfortably side-by-side on my screen; YMMV.

Download the scripts

I'm still adapting the scripts from the satellite versions. If you're interested, come back in a few days, or send me mail.