User login

Blogs

05

Oct

2016

Spent some time working on the test scheduling web interface to try to make the flow clearer for tests that don't require a hostname as a target (i.e. the HTTP test). Tests that don't require a hostname target will now hide that option and display the other options required. Still need to do some work around creating meshes of non-hostname targets.

Updated some more documentation to fill in gaps or bring it up to date after making changes. Based on feedback I changed some default values for configuration options, making some of them compulsory rather than defaulting to values that were not particularly useful.

Made various other smaller fixes - the traceroute test now uses the same library functions to receive and timestamp results as the other tests do, python ampsave code was tidied up slightly and made more consistent, CentOS packaging scripts were updated to build new packages, etc.

Finally got the amplet2 code up on GitHub.

05

Oct

2016

Continuing working on the paper. From feedback I've added some graphs showing the traits of individual switches, as this was something I only briefly mentioned but people have found interesting. I've also been working through updating the written results to include the x930 and I've been replacing any references to a vendor with the switch model within the text as performance characteristics are going to be linked to individual products.

Submitted my 6 monthly PhD report this week, spent a bit of time reviewing how far I have got, and revising the best next steps. I have a relatively naive algorithm currently on paper to begin implementing, but I expect it to be pluggable to some degree. Allowing me to turn on and off features and compare the results of them.

03

Oct

2016

Finished the draft of my NNTSC paper. Got some initial feedback from Brendon which I've been able to incorporate into the paper.

Still not entirely happy with Influx-NNTSC and netevmon running on the same machine, as the combined memory usage will push skeptic's current hardware to its limit. Experimented with running netevmon on a separate VM just to make sure that a remote event database does actually work, so we at least have the option of moving netevmon onto its own dedicated machine.

Finished my implementation of the imprecise pattern mining algorithm. Starting working on a more homegrown algorithm for detecting repeated sequences of syscalls within a larger trace, based on existing techniques for using a suffix tree to find repeated substrings within strings.

28

Sep

2016

I spent last week writing a rough first-cut draft of the RheaFlow paper to be submitted to SOSR. I intend to finish the draft this week and send it out for reviews and comments. I will spend the rest of the week, revising the draft and sifting away unnecessary bits.

26

Sep

2016

Found and fixed a few small bugs that had shown up in my recent testing packages, such as one where the DNS loss timeout was set too large (and causing the test to be killed if any of the targets failed to respond). Also spent some time looking at the ASN lookup errors in my logs and added further logging around that to try to track down what was causing them - so far it appears to be the server having issues, not us.

Created a github repository for the amplet2 code and started adding documentation. Cleaned up the source to the manpages so that I can generate nicer markdown from them to put in the wiki. The code should hopefully be added in the very near future too, just waiting to test a few things in the latest client so I can make a new release at the same time. Made a few last minute tidy-ups to the repository ahead of this, making sure no autogenerated files remain and that I've removed unrelated content (e.g. libwandevent and librabbitmq CentOS spec files).

Updated CentOS spec files for the amplet2 client and new librabbitmq. Built and deployed new CentOS and Debian Wheezy packages onto a couple of test clients to test over the weekend.

23

Sep

2016

Returned to my half-written NNTSC paper with an eye towards submitting it to PAM in a few weeks. Paper is now around 75% finished, including a couple of nice diagrams showing the NNTSC architecture and the database schema. Space is starting to get a bit tight, so I'll have to revisit some of my earlier writing and cleanse it of unnecessary waffle.

With the help of an explanation from Harris, I've been able to decipher the temporal property mining algorithms. Managed to implement the simple version this week, which seems to be doing the right thing, and started working on a
more complicated variant that allows for some imperfections in the source data (e.g 9/10 times a close follows an open, but every now again someone forgets to call close before opening something else).

21

Sep

2016

I added code to provide interswitch link configuration for RheaFlow and performed some tests to confirm it is working fine. Based on the code, RheaFlow will now support an arbitrary number of switches all connected to each in a full mesh. I also played around with the Atrium SDN router to examine its features and make comparisons to RheaFlow for the paper to be submitted to the SOSR conference.

This week, I'll be working on a rough draft of the RheaFlow paper to be submitted to SOSR.

20

Sep

2016

I'm working on the packet in and out paper to submit to SOSR.

I've been working through a number of Matthew's suggestions, focusing on tidying up the things I don't expect to change first. I've tidied up the graphs a little more and re-arranged the key to better fit within the graph and not overlap other bars. I've also tweaked the colour of the target rate lines to ensure this is the same across graphs. Removed underscores from some words. I've tidied up some tables to make them clearer. I've also started rewriting/reorganising parts as needed.

19

Sep

2016

Found and fixed a bug in the test scheduling while dealing with some user queries around test scheduling. The default value for test frequency (used if not explicitly specified) had the wrong units and so caused tests to be scheduled more frequently than intended. Also fixed a couple of places where wrapping could occur, and wrote some unit tests to cover those code paths.

Removed an unused option and related code paths from the traceroute test that weren't adding a lot except some hideous looking code. While looking at this I tightened up the timers being used for sending/timing out packets to make sure that timeouts were correctly based on the oldest outstanding probe, and that probes were being sent as close to the desired rate as possible. Also added some knobs for changing how many targets are probed at once, and now randomise the initial TTL probed to try not to hammer any nearby hops quite so hard.

Removed some old and unused files from the amplet2 repository. Updated more documentation to be accurate with the current state of things.

19

Sep

2016

Kept tinkering with my mock skeptic install. I was a little concerned about the memory usage of anomaly_ts so I went back over some previous work I did to work out relative accuracy rates of each detector under a variety of different parameter settings to try and find good settings for each detector that used a minimal amount of stored history.

Spent a bit of time reading over some papers on mining temporal properties from sequences of function calls. The algorithms that these people are using are a bit tricky to decipher -- the explanation is a bit terse and I don't really have the background in the area to fill in the gaps -- so hopefully Harris will be able to get further than I did.

Continued building FSMs for common syscall patterns. Started working with the user study data which is not at all well covered by my existing FSMs. This appears to be mostly because of various Gnome / X processes and widgets that are continuously polling and receiving events. The syscalls generated by these processes drowns out everything else, so it is hard to find the actions that the users actually performed during the study.

Arranged travel and accommodation for my upcoming trip to IMC.