Tag Archives: nfqueue

Snort_inline load balancing

Posted on by

Dave Remien of NitroSecurity created a patch that “implements a relatively simple form of (IPV4) load balancing” between multiple Snort_inline processes using Nfqueue. Here is what it does:

1. Load balancing. The bottom half of the source and dest addresses are added together, and mod’d with the number of “load-balancing” snorts you desire to run. This means that traffic stays with a particular snort, so that state is maintained.

2. Because you can run many snorts (presumably on many CPUs), you can now take advantage of that super-hooty 16way box and those 10 gig NICs you just got your hands on…

3. The snort at the “magicqueue” is a back-up snort – normally no traffic is sent to it, but in the event that one of the “load-balancing” snorts isn’t present, the traffic for that “load-balancing” snort gets sent to the “backup” snort.

And even more, read that here.

I haven’t tried this myself, but it sure looks cool. Try it by checking out the latest SVN code for Snort_inline. Then follow the instructions from README.NFQUEUE.

Big thanks to Dave Remien & NitroSecurity for this contribution!

Improving Snort_inline’s NFQ performance

Posted on by

When using Snort_inline with NFQ support, it’s likely that at some point you’ve seen messages like these on the console: packet recv contents failure: No buffer space available. When the messages are appearing Snort_inline slows down significantly. I’ve been trying to find out why.

There are a number of setting that influence NFQ performance. One of them is the NFQ queue maximum length. This is a value in packets. Snort_inline takes an argument to modify the buffer length: –queue-maxlen 5000 (note: there are two dashes before queue-maxlen).

That’s not enough though. The following settings increase the buffer that NFQ seems to use for it’s queue. Since I’ve set it this high, I haven’t been able to get a single read error anymore:

sysctl -w net.core.rmem_default=’8388608′
sysctl -w net.core.wmem_default=’8388608′

The values are in bytes. The following values increase buffers for tcp traffic.

sysctl -w net.ipv4.tcp_wmem=’1048576 4194304 16777216′
sysctl -w net.ipv4.tcp_rmem=’1048576 4194304 16777216′

For more details see this page: http://www-didc.lbl.gov/TCP-tuning/linux.html

Setting these values fixed all my NFQ related slowdowns. The values probably work for ip_queue as well. If you use other values, please put them in a comment below.

Thanks to Dave Remien for helping me track this down!

Multiple Snort_inline processes with Vuurmuur

Posted on by

One of the cool things of the Snort_inline project is the support for NFQUEUE. NFQUEUE is the new queuing mechanism to push packets from the kernel to userspace so a userspace program can issue a verdict on it. What makes NFQUEUE cooler than it’s predecessor ip_queue is that it supports multiple queue’s. This means that there can be more than one Snort_inline process inspecting and judging traffic. The challenge is to make sure that each Snort_inline instance sees all traffic belonging to a certain connection so Snort_inline can do stateful inspection on it. Luckily, Vuurmuur makes it very easy.

Normally an ‘accept’ rule in Vuurmuur looks like this:

accept service http from local.lan to world.inet options log

The NFQUEUE equivalent of this rule is:

nfqueue service http from local.lan to world.inet options log,nfqueuenum=”1″

To have ftp handled by another Snort_inline instance, just add a new rule:

nfqueue service ftp from local.lan to world.inet options log,nfqueuenum=”2″

Easy, no? 🙂 Vuurmuur creates the iptables rules that are required. It uses some advanced connmark-fu for it, so the right Snort_inline process receives all packets from a connection. It uses the helper match to make sure related connections are handled by the right queue, such as the ftp data channel. Of course you also need Snort_inline to be ready for it. See this post for more info on that.

The Snort_inline configuration part takes some work. You have to setup your init scripts to start all instances, setup different configs, logging to different locations. You will need multiple Barnyard’s and if using Sguil multiple snort_agent.tcl instances. When updating the rules you need to take care of the multiple processes as well. As said, it takes some work, but it’s rewarding. You can for example setup an extra Snort_inline instance for testing purposes only. Send all traffic from a certain IP to it to try out new rules, config changes, etc. I have set it up to have separate processes monitor my dmz and my lan.

What is possible as well, but not with Vuurmuur so far, is to have a form of poor man’s load balancing by sending new connections to one of multiple processes. This could be done by making use of the ‘ipt_statistics’ iptables module (fmr ipt_random). This allows a rule to be activated only some percent of the time. By using some more connmark-fu it’s possible to have multiple Snort_inline instances to handle different connections of the same type of traffic. I’ll add support for that to a future Vuurmuur release.

Vuurmuur developments

Posted on by

Last weeks I’ve spend many hours on my Vuurmuur Firewall project. First I’ve been improving the code to prepare for a new release. I’ve added NFQUEUE support to Vuurmuur, so I could use it with nfnetlink enabled Snort_inline. Also the connection killing has been improved. The rules limit options were extended, to allow more flexibility.

Second, with the great help of Adi Kriegisch, I’ve been working on setting up a new build server for Debian and Ubuntu packages. Credits mostly go to Adi, who did most of the work and hosts the server. So many thanks to Adi! The new build server supports all version of Debian from Sarge up and of Ubuntu from Dapper and up.

Third, I have been busy setting up a new site for Vuurmuur. It is based on the Trac software and replaces both the static site at sourceforge.net and the tikiwiki based wiki. The new site can be found at http://www.vuurmuur.org/

Last but not least, and the reason for this post, is that today I’ve released a new version of Vuurmuur. The version, 0.5.73, is the first non-test release since January. This has taken a lot longer than I wanted, but work and other projects and of course ‘real life’ have taken much of the time from Vuurmuur. I don’t think this will change any time soon, unless someone will join the project to contribute some serious amount of coding time, or someone wants to fund me to do so.

On the roadmap (which is a nice feature of Trac btw) one can see the future directions of Vuurmuur. Ideas and comments are welcome. So are bug reports of course 🙂

Compiling Snort_inline with NFQUEUE support on Ubuntu

Posted on by

I needed to setup the right libraries for Snort_inline development on my fresh Ubuntu Feisty installation, so I decided to write down the procedure for those who think compiling Snort_inline from source is hard. 🙂

Make sure you have build-essential package installed. This makes sure you have a compiler and development packages for glibc and other important libraries. I’m installing the libraries from source to get the latest versions because the latest versions are more stable and perform better than the versions included in Feisty. I’m installing them into /usr because some programs like them there best.

Download the library libnfnetlink version 0.0.25 from http://ftp.netfilter.org/pub/libnfnetlink/

$ tar xvfj libnfnetlink-0.0.25.tar.bz2
$ cd libnfnetlink-0.0.25/
$ ./configure --prefix=/usr
$ make
$ sudo make install

Next download libnetfilter_queue version 0.0.13 from http://ftp.netfilter.org/pub/libnetfilter_queue/

$ tar xvfj libnetfilter_queue-0.0.13.tar.bz2
$ cd libnetfilter_queue-0.0.13
$ ./configure --prefix=/usr
$ make
$ sudo make install

Okay, that sets up the Netfilter libraries.

Next is the installation of libdnet. Do not use the dnet packages from Debian and Ubuntu because this is not the same library as we need for Snort_inline.

Download libdnet 1.11 from http://libdnet.sourceforge.net/

$ tar xzvf libdnet-1.11.tar.gz
$ cd libdnet-1.11
$ ./configure --prefix=/usr
$ make
$ sudo make install

Download snort_inline 2.6.1.5 from http://snort-inline.sourceforge.net/

$ tar xvzf snort_inline-2.6.1.5.tar.gz
$ cd snort_inline-2.6.1.5
$ ./configure --enable-nfnetlink
$ make
$ sudo make install

Test if it works (as root):

# iptables -A OUTPUT -p icmp -j NFQUEUE
# iptables -A INPUT -p icmp -j NFQUEUE
# /usr/local/bin/snort_inline -v -Q -H0

Then ping some host and you should see Snort_inline display the packets. If it doesn’t work, it might be that the kernel modules needed for nfqueue are not yet loaded. Using ‘lsmod’ check if ‘nfnetlink’ and ‘nfnetlink_queue’ are loaded. If not, load them manually.

After this, setup Snort_inline as normal.

Vuurmuur NFQUEUE support

Posted on by

Vuurmuur supported the QUEUE target for a while already, even though it needed a little bit of a hack to handle the state. This is because the iptables ruleset Vuurmuur creates is quite simple: after a few general protection rules it starts by accepting traffic with the state established. Since there is no way to say ‘queue established traffic that was queued before’ in iptables I decided to use traffic marking to distinguish between traffic to be queued or accepted. But there was a problem with this approach. I didn’t want to cripple the marking of traffic for other purposes, such as traffic shaping and routing, so I decided to use mark-ranges to either queue or accept:

/sbin/iptables -t filter -A FORWARD -m mark –mark 0x0/0xff000000 -m state –state ESTABLISHED -j ACCEPT

/sbin/iptables -t filter -A FORWARD -m mark –mark 0x1000000/0xff000000 -m state –state ESTABLISHED -j QUEUE

The reason I am explaining all this is that while this works fine for the QUEUE target with a single queue, it won’t work as well for NFQUEUE. One of the new things with NFQUEUE is that there are up to 65536 different queues. The above solution won’t scale for numbers like this.

I could just settle for supporting a few queues so the marking would still work. However, this conflicts with my design goals for Vuurmuur. It is the goal to put as little artificial limitations in the usage of Vuurmuur as possible. A second goal is that Vuurmuur should be able to run on standard distribution kernels. So I’m not interested in adding obscure features that require patching of the kernel and the iptables userland tools. For this purpose, I consider Debian Stable to be the standard.

For the NFQUEUE support I decided to repeat the mark trick using the newer connmark target and match. The advantage of connmark is that you only have to set it once for a connection, as opposed to for every packet with mark. Additionally connmark works separately from mark, so marking the traffic for other purposes still works. Because connmark wasn’t supported by many distributions at the time the QUEUE support was added, I didn’t use it before. At this time only the NFQUEUE support uses connmark, so if your system doesn’t support it nothing will change.

The Vuurmuur NFQUEUE support is now available in the Vuurmuur SVN repository.

Snort_inline updated to 2.6.1.4 in SVN

Posted on by

After moving, which went fine, I now finally have some real coding time again. The last week I have been updating and fixing various parts of Snort_inline. The most important change was the update to Snort version 2.6.1.4, which contains security fixes. William also found an issue with the Stream4inline code. The issue was that the memcap that the admin sets to limit the amount of memory used by stream4 wasn’t properly enforced.

Other fixes that are done is that Snort_inline in nfqueue mode now properly honors signals and also no longer needs the libipq library and headers. There are few changes that will be committed soon. One is an issue that clamav can sometimes return an error when parsing malformed file. Until now the spp_clamav preprocessor would issue a FatalError and cause Snort_inline to die. This is obviously not desirable so the patch makes sure that Snort_inline no longer dies and gives the admin an option to either drop or pass traffic that can’t be inspected.

Last but not least there will be a fix to the nfqueue code that appears to solve the ‘stuck packet problem’ we were seeing under heavy load. A number of people are testing my patch currently so if all goes well that will be commited soon as well.

Checking out the latest code is done with the following command:

svn co https://snort-inline.svn.sourceforge.net/svnroot/snort-inline/trunk