User login

Blogs

06

Jan

2017

Libprotoident 2.0.10 has been released.

This release includes rules to match new traffic patterns for many of the protocols that we introduced in the 2.0.9 release. We've also added two new protocols: BACnet and Maxicloud.

This release also no longer treats TCP keepalive packets as payload-bearing.

The full list of updated protocols can be found in the new libprotoident ChangeLog.

Download libprotoident 2.0.10 here!

19

Dec

2016

Tidied up and documented the FSM extraction code, so that I'll be able to remember how it works when I start working on it again in earnest next year.

Finished the matrix layout / selection changes and merged them back into develop. Hopefully we will get a chance to roll these out early next year once Brendon builds some new packages.

I had to run a test capture for a few days last week to make sure that some changes Richard had made to libtrace had not broken DAG and RT inputs. Ran the resulting traces through libprotoident to see if there are any new protocols worth investigating. Managed to make a few improvements to the rules for existing protocols to catch a few cases that we were missing but otherwise nothing particularly exciting cropped up.

16

Dec

2016

This week I tested my second AMP test and ironed out the remaining issues and bugs. Because all the options where set in my TCP ack packet, if it was dropped due to one of the options I wouldn’t know why so I added a fall back which would test options individually if a few preconditions where meet. I also separated my tests into their own files to make it easier to see what was going on and make extensions easier as well as a generally refactoring of the code.

15

Dec

2016

Spent most of the week fighting with WSGI to get URI components containing slashes to properly pass through the routing and arrive at my code. Double escaping them will hide them enough from WSGI that the slashes aren't interpreted as a separator so that I can get a correct site name. Updated all the views, javascript and templates where a site/mesh name is used to be properly escaped.

Started adding a rudimentary email alerting system to netevmon to send emails when event groups cross a size/importance threshold. It's been a while since I looked at this code, so it's been a bit of a learning process to find the best place to do so.

12

Dec

2016

In Wellington for STRATUS forum on Monday. Had a few interesting chats -- definitely a lot of people out there interested in anomaly detection in a variety of contexts.

Continued refining my FSM generation code. Managed to get rid of most of the obviously incorrect transitions in my test cases now. There's still a bit of work to do in terms of tidying up some orphaned states that are left over as a result of the code realising they are redundant and trying to choose better start states, but my main focus before the end of the year will be tidying up the code and making sure it is sufficiently documented so I'll be able to pick it up again in the new year.

Fixed a bunch of small problems with amp-web and NNTSC that we've known about for a while. Started working on replacing the matrix selection tabs with dropdowns and combining related "tabs" into a single matrix type, e.g. http duration and http page size are combined into a single "http" matrix with the ability to change the metric using a dropdown.

09

Dec

2016

This week I worked towards getting an amp test complete that will test a list of TCP options to a location, seeing which options are accepted or if the packet is dropped all together. With the idea being that if this was done from many points in the network you could infer if a middle box is tampering with the options.

07

Dec

2016

Wrote a short program to read from a RabbitMQ queue and dump the data as a CSV file. It had been requested for a particular site to get access to the data in other tools, and I think it's a helpful example for others wanting to do things with the data outside of what our tools provide.

Finished updating the site configuration web interface to show basic information about whether a site has a certificate or not (if using the amppki package), and allowing signing of the certificate in simple cases. Created a new user to run this web code. Found and fixed a few amppki user interface bugs as well, such as expired certs counting as duplicates (and triggering the simple protection code) when trying to revoke them.

Built and deployed some new packages for ampy and ampweb. Spent way too long trying to fix some issues caused by versioning of python modules where the Debian version is too old. Quietened a few unnecessary messages during package install.

06

Dec

2016

Continued working on the automatic FSM generation code. Managed to implement my earlier solution to the problem of loop recognition -- the core principle is that every candidate sequence is subjected to being transformed into its own suffix tree, which we can use to identify repeated patterns within the candidate itself. I've also placed an upper limit on the sequence length of candidates extracted from my original suffix tree, so that we do not waste time dealing with candidates that are obviously too long to represent a single action.

Once I had that working, I added some pydot code to generate visual representations of my state machines. Not only does this give me something to show people, it is also very handy for spotting incorrect transitions introduced by my code. Spent the rest of the week chasing up various incorrectness in my machines and testing across a handful of different input datasets.

06

Dec

2016

Over the last two weeks I've been making the final modifications to the TTP validator. Thanks to Brad for having a look and making and number of suggestions and setting up the hosting for this. This process involved turning of a couple of features that were unsafe and removing dependency on the ryu library as well as a number of other consistency fixes. It is now available online at http://wand.nz/ttp-validator/.

I've taken some time to fix an number of issues in libtrace. Including modifying the behaviour of trace_interrupt to stop all traces, not only live ones, this is more consistent. Updating DPDK in libtrace to support the newest versions; Richard Cziva provided an initial patch for DPDK. I extended this to include packaged versions of the DPDK which will be included in future releases of Debian/Ubuntu. I also started initial work into fixing an issue with pstop() and the ring format on older kernel versions which is related to stopping non-parallel formats, that are being split internally within libtrace.

28

Nov

2016

Finished and merged the scheduling REST interface after spending some time making return codes more consistent and implementing the last few missing functions. I can still see some of this changing to fit in with the way the web interface works, but it's done for now. Also fixed some minor bugs in ampweb where some sites weren't available in dropdowns.

Started adding the ability to display certificate status for each site in the web interface, and perform some very simple management there without having to use the command line tools. Split out some of the code from the command line tools into a library to be shared between them and the web interface.

Updated the ampy packaging scripts to bring the database schema up to date with the new changes when the package is updated. Also built new test packages for amppki to allow testing the certificate additions to the web interface.

Spent some time working with Jayden to help get him up to speed with AMP, and how one might go about writing a new test.