User login

Blogs

29

Nov

2011

After a fantastic trip through parts of Asia and Europe, I'm happy to say that my thesis has been approved to progress to oral examination. However a number of changes (two of which are quite major) have been asked for, so I am currently working on these. Due to uni closing so early this year, my oral examination won't be until the new year. This is a little disappointing as I wanted to have it all wrapped up by Christmas but at least I will have some extra time to make the changes and refine the thesis.

28

Nov

2011

I added reading in ASN data to my programme, from files from Team Cymru's website, and used a formula Perry gave me for converting ASNs to colours, so now graphs are coloured by ASN.

Assuming every node with an unknown address was a new node was resulting in 80% of the nodes in the graph being unkown, since the same routes are taken many times. So I made it not display unknown nodes.

Brendon got some new data that checks more times for an address before giving up, which reduced the number of unknown nodes, but it still has around 60% unknown nodes.

Gephi can group nodes according to an attribute into one larger mode, so I played around with that, but it leads to awkward looking results, which I havent been able to work out how to overcome.

28

Nov

2011

Continued looking into properties of sleeper traffic, primarily the rate at which sleeper traffic quantities degrade as the host continues to be idle. This has proved a bit tricky to visualise well, but finally managed to come up with what I think should be a useful graphing approach. This did require a lot of battling with R, though.

The fixed version of NAVL was not available last week, but I was able to continue looking at cases where PACE was able to identify traffic that libprotoident could not. Brad set me up with a Windows VM so that I could download various apps and capture traffic while using them, so that I can confirm PACE's classifications and add or update libprotoident's rules so that we can match the traffic as well. This meant I got to have a bit of fun playing Second Life and hanging out in chatrooms....

Started moving towards a new release of libprotoident, seeing as I've now added or updated the rules for quite a few protocols.

25

Nov

2011

This week I spent most of my time completing the php scripts that will be used to generate the necessary JSON/php configuration files for the NetMapJs version of the Karen Weather Map. The data in the configuration files that defines the map is based on the Karen topology data stored in a database, which is used to generate the current Karen Weather Map configuration files. This proved to be a little more challenging than expected since some of the information in the current karen weather map is hard-coded.

I also played around with FLOT (Javascript library that works in collaboration with JQuery to produce graphs), since I will be working on adding a graphing feature to NetMapJs. The idea is when a link is hovered (or clicked, double clicked..etc), a graph will be produced to show the maximum throughput between the two link just like the current weather map.

25

Nov

2011

Spent this week working primarily on WAND hard drives. I have started to monitor our 3ware RAIDs and software MD arrays. By using metrics from each drive such as reallocated sectors and temperature I hope to be able to quickly find failing drives and source replacements before they die.

Another project I have been working on this week is building a new voodoo based on Debian Squeeze. I have a very early work-in-progress on my desk. Things I hope to add to voodoo this time round is: network sound (pulseaudio), network file system (AFS? NFSv4?) and a kerberos relm which seems to be required by all modern network filesystems these days.

25

Nov

2011

This week I wrote a number of scamper modes for testing load balancing behaviour.

I also ran the ICMP ID test using the driver I wrote for this purpose. Paths found to have load balancing of this type were further analysed with other modes, in addition to a rerun of the original test.

21

Nov

2011

After seeing the graphs Chris made with a small number of destinations and
a single testing site, I found a few more sources of addresses that are
used within New Zealand (though not necessarily assigned to NZ
organisations) and started generating a new test data set. The new dataset
will have data for 28,000 /24s from four sources once collection
completes. Will need to have a look and see if this resolution is
appropriate.

Also put together a bunch of sample data and databases for William to use
with the new weathermap. It mimics the KAREN set up, so hopefully he can
use that to write scripts to deal with getting the data into the right
json format.

Spent some time picking up the smtp state machine documentation again,
trying to get some images of example trees to show the merging process.

21

Nov

2011

I converted my graph file producing programme to c++ so it can take in warts files as inputs instead of text files, and so it can store addresses as Scamper addresses, allowing it to deal with ipv6. I had some larger sample data from Brendon and tried producing graphs with that using graphviz sfdp, but it didnt cope well with more than 1000 nodes, plus it tended to produce pretty ugly graphs..

So I had a go with Gephi, which, surprisingly, handled the data without any problems at all. Or at least without any more problems than it has handling small graphs.

I played around a bit with the algorithms in Gephi, and managed to produce something which I think reasonably elegantly demonstrates the data.

Currently I am trying to match the addresses to ASNs, so I can use that to colour the nodes.

21

Nov

2011

I carried out MDA analysis using UDP, TCP and ICMP packets, on 100 addresses. IP header length was also varied for these modes using timestamp, record route and stream identifier, making 9 more tests. The resulting trace data was tested for per flow and per packet load balancing.

A driver was written for carrying ICMP ID MDA tests and if per flow load balancing is found further tests are carried out. A small run was carried out to test the driver, and then a larger run was initiated.

I also wrote a report on the state of my load balancing research.

21

Nov

2011

I continued working on the php scripts I started last week after getting Brendon to install the missing RRD-php module. I then spent a good amount of time getting NetMapJs to update based on the data in the example RRD file I received last week.

This involved me creating a new JSON structure which differs to the one used to define the map (as it only contain values that is being updated. e.g. traffic_out, bandwidth), creating a php script which defines the configuration of the node data, and modifying some JavaScript in NetMapJs to deal with the new JSON file.