In Januari I first wrote about my prototype code for the OISF engine. The first thing I started with when creating the code was the threading. The current code can run as a single thread or with many threads. In my normal testing I run with about 11 threads, 10 of which handle packets, 1 is a management thread.

The basic principle in the threading is that a packet is always handled by one thread at a time only. The reason for this is that it saves a lot of locking issues. If there is more than one thread, the engine can handle multiple packete simultaniously.

All functionality is created in what I call threading modules. Such a module is run in a thread. Threads can have one or more of the modules running. Examples of these modules are Decoding, Detection, Alerting, etc. I intend to make these modules plugins in the future so that 3rd party modules can be loaded without recompiling the codebase.

The threading model works both in a parallel and serial way. The parallel way can be used to have multiple threads doing the same jobs, for example have 2 threads both acquire packets, decoding, detection and alerting. The serial way of threading works differently. In that case a thread has a limited number of tasks (e.g. Decoding) and if it’s done with a packet it passes the packet on to the next thread (that for example does Detection). Both methods can be combined, which I use by default: I have 1 packet aquiring thread, 2 decoding, 2 detection, 1 verdict (I’m using nfq), and a few alerting and active response threads.

Between the (serial) threads queue’s are used to transfer the packet from one thread to another. A queue can contain multiple packets. In the above example, the nfq packet acquire thread can read packets from the queue as fast as it can and put them in it’s queue. The 2 decoding threads then get packets from this queue as fast as they can. Then they put them in the next queue where they are picked up again, etc.

Using the queue’s code paths can also be determined. It’s possible for example to have IPv4 packets be handled by different threads than IPv6 packets. Or packets with alerts differently from packets that didn’t have alerts.

One big challenge is that this is all extremely complex & configurable. Threads have to created, queue’s, queue handlers, CPU affinity can be set per thread, threading modules need to be assigned to threads, etc. I think power users & apliance builders would be interested in having all these options, but for casual users it’s probably (way) too complex to be bothered with. So some reasonable defaults need to be created, maybe in the form of having a number of pre-configure profiles.