User login

Blogs

04

Sep

2017

Implemented basic route refresh functionality in the BGP disaggregated router, and wrestled with exabgp to find out how to pass through the messages I required to do so. Also spent some time chasing down what looked like bugs in the topology module, but was actually a broken data file that didn't correctly describe the layout of the network.

Had another attempt at getting my chromium/youtube test working. It works fine when I build it within the chromium source tree alongside their example headless applications, but otherwise fails. It appears to be linking against a lot of object files deep inside chromium, as well as the headless static library (which I thought should contain everything needed to build a headless application?), as well as all the normal shared libraries. Back into the too-hard basket until they sort their stuff out or I have some more time to push through this.

Spent a lot of time reading about different approaches to HA/resilience, what sort of information nodes often pass around and how they go about sharing state (or avoiding sharing state).

04

Sep

2017

Finished up writing and testing the new address family selection options in AMP and making sure that all of the tests work properly when they are set. Changing the way the config files worked to allow globally setting options (but able to be overridden at test level) meant there were a few more edge cases than anticipated.

Started thinking and writing about how we might go about making the BGP disaggregated router more resilient, and what situations may arise that it will need to deal with.

04

Sep

2017

Spent some more time trying to fix the chromium/youtube test to get around segfaulting when javascript is present on a page, without any success yet.

Started work on improving manual IPv4/IPv6 address selection in AMP tests. Previously you could specify addresses to use for each address family, but generally couldn't control which family would be used (that was up to DNS). The plan is now to allow enabling/disabling individual address families globally or per test, and not require that a particular address be specified (but still allowing it if desired).

04

Sep

2017

Ranked every libprotoident protocol by observed bytes in three trace sets (from 2012, 2015 and 2017) as part of an attempt to demonstrate the relative transience of application protocols -- i.e. how protocols can appear, die off, surge or plummet in popularity. Some of the ranks didn't quite make sense, so I had to go back and validate a few of the rules. There were a couple of errors, which meant that I had to re-run the ranking analysis once I had fixed them.

Helped Brad with some stress testing of the Endace probe. This had the unfortunate side effect of making Vision unusable for a few days afterwards, but hopefully the process was helpful to Endace in the long run.

Continued working on a method of automating myself.

28

Aug

2017

This week I spent some time looking at comparing the Python3 and Python2.7 versions of the disaggregator to see which version offers better performance and is more memory efficient. The Python 3 version of the disaggregator uses less memory and also seems to be a bit faster. This is now the versions used in the master branch of the repo.

Most of the week I worked on adding IPv6 support to the disaggregator. I extended the prefix class to support IPv6 prefixes and then extended the BGP prefix parsing to retrieve the advertised and withdrawn IPv6 prefixes. Have also added parsing of the ExaBGP negotiated messages to retrieve the negotiated MP-BGP NLRI families for the disaggregator peers. The code was modified to uses these negotiated families when parsing the update prefixes. This will make it easier to deal with other MP-BGP NLRI families that will be supported later on.

More unit-tests and Mininet unit-test cases were also added to check the new modifications are working. At the end of the week, I also started looking at ways to get the topology information of the network dynamically.

28

Aug

2017

Back at work on Wednesday after a couple of weeks away. Spent most of my week catching up on emails and preparing for a STRATUS workshop coming up on Monday.

Did manage to spent a little bit of time looking at unknown traffic on the Waikato capture point again. Added a couple of new protocols: Smite and Fliggy. I've also found what appears to be another IP sharing "tool" similar to IPSharkk -- this one looks like it is installed as malware, so the network user is probably unaware that their machine is being used to proxy other people's traffic. Will try to dig a little more into this next week.

08

Aug

2017

Spent some time thinking and reading about ways to improve resilience and reliability of the BGP project, and how multiple instances of it could interact. Wrote up some failure cases and some ideas around how they can be dealt with.

Had another look at the tcp_info struct and how much of the web10g style information has made its way into the kernel. There is almost enough there now to do interesting things around reporting possible causes of slowness in throughput tests, though no probes in any of our deployments are running new enough kernels to do so. Will put this aside for a little while until kernels are updated or I get some ideas how to use the information that is available now.

Picked up the Chrome/youtube test again now that the headless mode is part of the main branch and included in the current Ubuntu packages. I have my test program building and running using the libraries in the package, though still have to link against a static headless Chromium library that doesn't get distributed. The test runs, but crashes if there is javascript on the webpage that it loads. Unsure so far what I'm missing that could be causing this.

08

Aug

2017

Finished updating the disaggregator and method unit tests to work with Python 3. Most of the week I have also spent on updating the simulation tools to run with Python 3 and have updated to the new ExaBGP (version 4.0.2).

I have also worked on the automatic unit tests which will make sure that the disaggregator behaves accordingly when running.

31

Jul

2017

Continued researching and writing for my application protocol paper. Added quite a lengthy background which summarises some of the key events and trends in the history of application protocols which will add a lot of context to my paper.

Also kept investigating new application protocol patterns that continue to appear on the Waikato passive monitor. Added another 5 new protocols this week, so progress continues to be made.

31

Jul

2017

This week I finished the simulation tools which allow building and simulating network topologies in Mininet. I spent some time working on testing the disaggregator in the simulation framework to make sure everything is working correctly.

While testing, I have found and fixed several bugs and issues. For example, there were some problems with parsing the BGP route updates for communities and also some behavioural problems when parsing the communities from the config file filters. The code has been modified to store and process communities as a list of integer tuples. The config file accepts entry of communities in several different formats. I have also addressed an issue that was preventing updates to be processed from peers that do not generate End-of-RIB markers. The dissaggregator now parses a peers Open message to check if they are going to send EoR markers, if not we will no longer defer the table updates until we see a marker for these peers.

I have also spent some time looking at various BGP performance testing tools and on unit testing the code using a similar approach to Faucet. I have also worked on modifying to code to make it Python 3 compatible. Currently, I am in the processes of further testing the code using Python 3 and addressing some issues that I have found.