Monthly Archives: December 2010

One year of (public) Suricata

Posted on by

Today exactly one year ago we released the first public version of Suricata, tagged 0.8.0. It was the first beta version. Six months later we released Suricata 1.0.0, the first stable release. Since then we’ve been doing 3 more releases: 1.0.1, 1.0.2 and 1.1 beta 1.

It has been an very exciting year, with a lot of press and community interest for our project. Also, a lot of work has been done in the past year. I already wrote that our performance has increased a lot.

There have been over a thousand commits to our source code management system (git), more than all previous development together. Or as git shows it: “548 files changed, 193714 insertions(+), 39606 deletions(-)”. Quite impressive. Almost 20 developers have contributed code, some paid by OISF, some from our consortium members, some from the community.

Exactly one year ago, our codebase had a size of 120k lines of code. Today, we’re looking at 269k lines of code. Admittedly this includes our 2264 unittests (up from 1191), but still a large increase. In short in 2010 we’ve shown to the world we’re building a exciting and thriving project!

For 2011 we are in great shape. We have a stable 1.0.2 release and a promising upcoming 1.1 release. Emerging Threats (and also Emerging Threats Pro) has a tuned and optimized Suricata ruleset. Additionally, one of the 1.1 goals is to continue to fully support VRT’s ruleset.

Soon we’ll be starting work on Suricata 2.x, so there are exciting times ahead!

Happy New Year everybody! 🙂

Listening on multiple interfaces with Suricata

Posted on by

A question I see quite often is, can I listen on multiple interfaces with a single Suricata instance? Until now the answer always was “no”. I’d suggest trying the “any”-pseudo interface (suricata -i any), with an bpf to limit the traffic or using multiple instances of Suricata. That last suggestion was especially painful, as one of the goals of Suricata is to allow a single process to process all packets using all available resources.

Last week I found some time to look at how hard adding support for acquiring packets from multiple interfaces would be. Turned out, not so hard! Due to Suricata’s highly modular threading design, it was actually quite easy. I decided to keep it simple, so if you want to add multiple interfaces to listen on, just add each separately on the command line, like so: suricata -i eth0 -i eth1 -i ppp0. This will create a so called “receive thread” for each of those interfaces.

I’ve added no internal limits, so in theory it should possible to add dozens. I just tested with 2 though, so be careful. Normally the thread name in logs and “top” for the pcap receive thread is “ReceivePcap”. This is still true if a single interface is passed to Suricata. In case more are passed to Suricata, thread names change to “RecvPcap-<int>”, e.g. RecvPcap-eth0 and RecvPcap-eth1. Untested, but it should work fine to monitor multiple interfaces from different types. Suricata sets the data link type in the interface-specific receive thread.

If you’re interested in trying out this new feature, there are a few limitations to consider. First, no Windows support yet. I hope this can be addressed later. Second, the case where two or more interfaces (partly) see the same traffic is untested. The problem here is that we’ll see identical packets going into the engine. This may (or may not, like I said, it’s untested) screw up the defrag, stream engines. Might cause duplicate alerts, etc. Addressing this is something that would probably require keeping a hash of packets so we can detect duplicates. This is probably quite computationally intensive, so it may not be worth it. I’m very much open to other solutions. Patches are even more welcome 🙂

So, for now use it only if interfaces see completely separate traffic. Unless you’re interested to see what happens if you ignore my warnings, in that case I’d like to know! The code is available right now in our current git master, and will be part of 1.1beta2.

Merry xmas everyone!

Suricata 1.1 beta 1 released

Posted on by

Today we’ve released Suricata 1.1 beta 1, the first beta of the upcoming Suricata 1.1 release. The official release announcement is here on the OISF website.

The main focus of the new release has been to improve performance and to add support to the features the new ET/ETpro ruleset needs. ET and ETpro have rulesets specially tuned and geared for Suricata. We’re still missing some new rule keywords that are used by VRT, so in the 1.1 beta 2 release we’ll address that.

Other than that, I got quite a few patches waiting. We’ll be improving stream reassembly, inline mode, prelude output, and numerous other things.

Like always, please give this a try and let us know how it works for you!

Suricata development update

Posted on by

The last months we’ve been working hard on improving Suricata. So hard actually, that we’ve drifted a bit from our original goal of doing a 1.0.3 “maintenance” release. Instead, the new release will be 1.1beta1. The change to 1.1 is to indicate the large number of changes, the beta1 is to … indicate the large number of changes 🙂

As you may know, Will Metcalf moved on to join Qualys. A significant loss to our project as Will was one of our founding members and is hard to replace in his role as QA lead. Not having a full time QA person on the team right now is a reason for us to decide we’re in need of a beta cycle for the next release.

So… what kind of improvements are we talking about?

  • Improved parsers, especially the DCERPC parser.
  • New keyword support: http_raw_header, http_stat_msg, http_stat_code.
  • Much improved fast_pattern support, including for http_uri, http_client_body, http_header, http_raw_header.
  • A new default pattern matcher, Aho-Corasick based, that uses much less memory.
  • Lots of small performance updates, including SSE3, SSE4.1 and SSE4.2 optimizations.
  • The signature bitmask prefiltering I wrote about before.
  • We support the reference.config supplied by ET(pro) and VRT now.

So… performance?!

Lots of mention of performance in this list. Did it improve? Yes! As some of you may have read, Npulse has demonstrated 10 Gbps IDS support for Suricata using Napatech (PDF) hardware support. This was on fast hardware, but nothing outrageous. To be honest, I didn’t expect to get there yet. But they did it. Based on a slightly modified Suricata 1.0.1 and about 7k signatures. Our own testing has shown that the code has improved quite a bit since then: ranging from 25% to 67% more packets per second throughput. Btw, native Napatech support is expected to go into our code base sometime in the next few weeks.

Whats left?

We have two major areas where we want more improvement. The first is the inline mode. Due to Suricata’s HTTP and other protocol parsers working statefully on top of the stream reassembly engine, currently all work is done on ack’d data. This means dropping attacks based on keywords such as http_uri is hard. We’re planning a number of changes to the stream engine to address this. More on that in a future post. The second area is the rule language. At this point we still miss a number of keywords to properly support mostly VRT signatures. Keywords like file_data.

Whats next?

The current git master is pretty much what Suricata 1.1beta1 is going to be. The actual release is planned for next week, probably Tuesday or Wednesday. If you can, help us out by trying it and report any issue to us!