User login

Blogs

26

Sep

2011

Generated some new object traces from recent Waikato and ISP traces to
test how well an SMTP state machine built using an hour of SMTP flows
would work. Across 17 traces from various times and locations from the
last 2 years it matches on average 86% of SMTP flows, though accuracy for
individual traces ranges from 55-97%. Will have to look into what is
causing some of the flows to be mismatched so poorly in some cases.

Made fixes to the ampcentral distribution/install scripts to include more
missing javascript files. Packaged up a few more components that were
missing from the new deployment and investigated some of the small issues
that had been reported. Added more documentation to individual AMP tests
describing parameters and how the tests should be used.

Made a few updates to the KAREN weathermap to bring it back up to date
with changes in the network. Got stuck for a couple of hours with bogus or
nonexistent data being generated by RRDbot when the RRD target was a
symlink. Not sure why this would happen.

26

Sep

2011

Finally finished all my libtrace performance tests and now have a completed draft of the paper done. If anyone feels like reading over it and offering feedback, let me know.

Starting looking at some traces I took last week at our ISP capture point. The initial results suggest that the proportion of P2P traffic has dropped compared with earlier in the year, possibly due to the introduction of the Copyright Amendment Act. These results are the subject of a blog post on the WAND website. Now starting to investigate this further.

Spent some time profiling the pcapint: and int: live capture formats for libtrace. Found a couple of bugs that I was able to fix as well as some inefficiencies in the way that we apply BPF filters.

23

Sep

2011

I managed to fall ill again on Monday which knocked me out for a few days. Started actually writing my final 520 report at the end of this week. Very momentous!

Also did a little hacking on vulnerable machines (in a virtual environment) for Comp518. I have managed to get root access on one of the machines so far. I should be able to on two other machines too based on some application versions.

23

Sep

2011

I managed to fall ill again on Monday which knocked me out for a few days.
Start actually writing my final 520 report at the end of this week.
Very momentous!

Also did a little hacking on vulnerable machines (in a virtual environment)
for Comp518. I have managed to get root access on one of the machines so
far. I should be able to on two other machines too based on some application
versions.

23

Sep

2011

Hi,
This week my supervisor was away so I spent time reading papers, taking
notes and adding new papers to my bibliography database.

Stephen Eichler

22

Sep

2011

On the first of September this year, the New Zealand Government's Copyright Amendment Act (more colloquially known as the "Skynet law") came into effect. Briefly, the Act promises harsh penalties for Internet users who download copyrighted content illegally, culminating in the cancellation of their Internet account. This law unsurprisingly received a lot of media attention in New Zealand and there were conflicting accounts as to whether the law was having any effect on traffic levels (http://arstechnica.com/tech-policy/news/2011/09/nz-traffic-down-as-three...).

I therefore decided that this called for a quick spot of Internet measurement. I used a passive monitor that we have located inside the core network of a New Zealand ISP to capture traces of several days worth of traffic in early September. I've now started running the traces through an analysis program based on libprotoident to investigate the application protocols being used by the ISP customers, with a particular focus on P2P (which is what the Act is targeting).

The first graphs I produced turned out to be very interesting. This graph shows the inbound (i.e. originating from hosts outside the ISP's customer ranges) traffic mix for the September trace set, broken down by application category.

As a comparison, this graph shows the same traffic mix for a trace set captured from the same ISP in January this year.

We see that the proportion of traffic that is P2P (the orange segment) has decreased quite noticeably in the September dataset compared with earlier in the year. It is hard to say for certain whether this is a direct consequence of the new law, but this is a promising result nonetheless. Certainly it is enough to encourage me to start looking into this a bit further - expect more updates soon as I get more results!

20

Sep

2011

Put together new distribution tarballs for the ampcentral code that took a
lot longer than expected. It has mostly grown organically in various web
directories (with changes going back into SVN), but has not been installed
fresh anywhere in a long time. Updated the install scripts and automake
scripts to include the new files and to work properly with newer versions
of php, postgres, etc.

Generating new data for me to test on turned out to be a slow process, so
spent some time investigating why that was. After some time spent with
valgrind I tracked down the main culprit - extracting objects from a days
worth of Waikato trace now takes less than 10 minutes on chasm, down from
20 hours!

While that was running I started to better organise my thoughts on the
state machine processing into a more formal technical report.

19

Sep

2011

In order to test if ICMP ID is part of the flow id, scamper was modified
to vary this field as flow id and ICMP seq as packet id. 100 addresses
were run using this program. Once the warts dump output was obtained, a
perl program was used to check for nodes with more than one link.

Papers from the citation lists of some key papers have been studied and
entered in my citation database. I have started reading these, taking
notes and entering key words.

Stephen Eichler

19

Sep

2011

Released libtrace 3.0.12 on Monday.

Continued running performance tests for the libtrace paper. Downloaded a few pcap traces from the MAWI archive and found that libtrace was surprisingly slow at processing them. After doing some profiling, I realised the problem was actually with my test program, which was not the most efficient. Because the processing thread was so slow, it was not spending enough time reading data that the decompression thread had written into the I/O buffer. This meant that the buffer filled up and the decompression came to a halt temporarily.

Fixing the test program solved the problem, but this raises an interesting point: the effectiveness of libtrace I/O is directly tied to the user's ability to write an efficient program. It'd be nice if this was not such a major factor.

Started looking into sFlow on Friday, with an eye towards developing a way to read and write sFlow raw packet records (possibly with libtrace).

Went to Auckland on Wednesday morning to chat with Jason and Josh from Vineyard Networks. Very interesting meeting and definitely looks like there will be some opportunities to work together in the future, especially if we have students keen on getting involved in application identification.

18

Sep

2011

Just a quick update. We are half-way through our trip, currently in
Belgium drinking lots of nice beers. Next destination is Amsterdam. No
word on my thesis yet which is a little disappointing but at least I
don't need to worry too much yet. Hope all is well and that the RWC
isn't driving everyone too mad!