User login

Blogs

16

Sep

2011

This was another week spent mostly on assignments. One of which was setting up a VOIP system. The lab includes a set of virtual machines connected to 4 Cisco IP phones. It got quite interesting in the Interactive Voice Response part. I didn't realise it was that easy to setup, especially considering Asterisk comes with a bunch of sample audio files.

I haven't started writing my 520 report yet. I plan on starting this weekend and working a lot on it next week because I should have a break before we get the next set of assignments.

13

Sep

2011

Spent some time refactoring portions of the state machine generation code
so that it is a lot more readable and sections requiring work are more
obvious. Significantly cut down the duration of a verification run by
being smarter about writing output and removing some very slow tree
accesses. Started to generate test data on a wider scale to get a better
variety of locations and times.

Put some time into making new AMP packages and putting together a bit of
documentation for a possible new test deployment. Could be interesting to
get some feedback from people outside of the project about how well they
think it works and what options they would like to see.

13

Sep

2011

Spent this week deploying all my nagios/cacti monitoring stuff to all the WAND machines. Found a few bugs that I fixed along the way. Also I had to create a few groups in nagios to separate processing servers to servers that we actually care about load on so I don't get hundreds of emails per day.

You can find all the new graphs of WAND servers and switches here: https://secure.wand.net.nz/cacti/

12

Sep

2011

Libtrace 3.0.12 has been released.

This release adds a new tool called tracetopends which can be used to identify the endpoints that are contributing the most traffic in a trace. We've also improved the general performance of the protocol decoding code and fixed a few obscure bugs in that area as well.

The full list of changes in this release can be found in the libtrace ChangeLog.

You can download the new version of libtrace from the libtrace website.

12

Sep

2011

It was a busy week in terms of tests and assignments. I spent most of the
week trying to catch up with those. I did a bit of planning for the chapters
in my 520 report in more detail.

12

Sep

2011

Further analysis of the data from the last scamper run of 100 addresses
was carried out. A program was written to investigate warts data for
paths that diverge from each other and then reconnect at a later point,
excluding the per flow cases that have already been found. Some of these
patterns were found and are likely to represent per destination load
balancing.

The same data was also tested for the presence of per packet load balancing.

The WAND wiki was also updated with further experimental results.

12

Sep

2011

Continued working on the new libtrace paper. Most of the content is now complete and now running a whole series of performance tests using my various test programs. The results so far are pretty promising - libtrace is looking a clear winner in terms of runtime.

Added a new tool to libtrace called tracetopends which reports the busiest endpoints in a given trace, where an endpoint can be defined as a MAC, IPv4 or IPv6 address. Hopefully it should make Chris happy.

Started doing some prep for a new libtrace release to go with the paper. Updated the ChangeLog and documented the new tools properly.

06

Sep

2011

Spent this week working on my monitoring system for WAND. Started to produce documentation on adding a server to icinga and cacti. Started working on a bash script that will eventually monitor all our RAIDs exposing its data to icinga over SNMP, in the process of writing it I found a few degraded RAID arrays, oops... Also worked on graphing io statistics with another script. Started to standardise everything so its easy to add new machines.

Also had 520 conference on Wednesday which was good to see the WAND talks finally after missing the practice, good work guys and good luck with writing your reports. Also attended Matthew's lectures as part of his job interviews which he pulled off well despite a few technicial difficulties, has increased the amount of time I have spent playing pacman recently.

06

Sep

2011

I spent last week prodding libflowinfo and making it ready for extracting features of the waikato 8 trace set. I threw out autoconf since it'd broken and I couldn't be bothered figuring out with what and replaced it with CMake. I fixed several bugs in the process and added support for measuring statistics about advertised TCP windows. I also got code from Shane to measure MSS over a flow.

I think that I have a decent amount of metrics now, and apart from possibly doing an FFT over some of the data I'm collecting it's getting pretty good.

The next step will be to run the software over the waikato traces and then run my anomaly detection software over the outputs.

libflowinfo isn't too far from a state where it could potentially be released if anyone thinks that's a good idea. It'd need some documentation written and copyright information added to the files, as well as two extra tests added to the CMake files.

06

Sep

2011

Bit of a disrupted week this week with lots of presentations going on,
including the annual honours student conference. Well done to those that
presented.

Worked through some examples of what the clusterer was doing to figure out
why it was generating the clusters that it was. Added a few short cuts in
that avoided running the clusterer and just did "the right thing" when
dealing with very small clusters (size 1 or 2) as that had the most poor
results. Also allowed the clusterer to naively explore a few different
results sets to try to find a more optimum number of clusters that may
have been hidden past a local maxima.

Also strengthened some checks to prevent links being merged that would
obviously mess up the distributions of packet sizes. Had to regenerate my
baseline data again after making these changes and then confirm that they
helped - overall accuracy dropped slightly but the number of false
positives dropped dramatically which I'm pleased with.