Tag Archives: william metcalf

Suricata development update

Posted on by

The last months we’ve been working hard on improving Suricata. So hard actually, that we’ve drifted a bit from our original goal of doing a 1.0.3 “maintenance” release. Instead, the new release will be 1.1beta1. The change to 1.1 is to indicate the large number of changes, the beta1 is to … indicate the large number of changes 🙂

As you may know, Will Metcalf moved on to join Qualys. A significant loss to our project as Will was one of our founding members and is hard to replace in his role as QA lead. Not having a full time QA person on the team right now is a reason for us to decide we’re in need of a beta cycle for the next release.

So… what kind of improvements are we talking about?

  • Improved parsers, especially the DCERPC parser.
  • New keyword support: http_raw_header, http_stat_msg, http_stat_code.
  • Much improved fast_pattern support, including for http_uri, http_client_body, http_header, http_raw_header.
  • A new default pattern matcher, Aho-Corasick based, that uses much less memory.
  • Lots of small performance updates, including SSE3, SSE4.1 and SSE4.2 optimizations.
  • The signature bitmask prefiltering I wrote about before.
  • We support the reference.config supplied by ET(pro) and VRT now.

So… performance?!

Lots of mention of performance in this list. Did it improve? Yes! As some of you may have read, Npulse has demonstrated 10 Gbps IDS support for Suricata using Napatech (PDF) hardware support. This was on fast hardware, but nothing outrageous. To be honest, I didn’t expect to get there yet. But they did it. Based on a slightly modified Suricata 1.0.1 and about 7k signatures. Our own testing has shown that the code has improved quite a bit since then: ranging from 25% to 67% more packets per second throughput. Btw, native Napatech support is expected to go into our code base sometime in the next few weeks.

Whats left?

We have two major areas where we want more improvement. The first is the inline mode. Due to Suricata’s HTTP and other protocol parsers working statefully on top of the stream reassembly engine, currently all work is done on ack’d data. This means dropping attacks based on keywords such as http_uri is hard. We’re planning a number of changes to the stream engine to address this. More on that in a future post. The second area is the rule language. At this point we still miss a number of keywords to properly support mostly VRT signatures. Keywords like file_data.

Whats next?

The current git master is pretty much what Suricata 1.1beta1 is going to be. The actual release is planned for next week, probably Tuesday or Wednesday. If you can, help us out by trying it and report any issue to us!

OISF engine development update

Posted on by

The last month has been crazy busy. Development of the engine is progressing nicely. My own role has been assigning tasks to our coders, guiding them, reviewing their work, integrating it and of course write code. We currently have nine people coding, not all full time though, and are still looking for more coders.

Progress has been made on a number of things: we have many more decoders, threading updates, a stats subsystem, stream tracking and reassembly, a L7 protocol parser framework and many more unittests. We’re working on OpenCL hardware accelaration, although we’re running into driver issues, so that may take some time before it’s usable.

On the QA side Will Metcalf is busy setting up an automated test rig, doing daily tests runs of our unittests on various platforms and with different compiler settings and such. When that is done pcap based tests and live traffic testing is next.

We have set up a number of “working group” mailinglists that discuss different subjects such as a configuration language and a rule language. Most are still ongoing, however the configuration language discussion seems to have come to a conclusion.

For the configuration language the discussion has settled on using YAML, a structured but still nicely editable format. It has many language bindings, so I hope management tools will be built for it later.

Other discussions, such as about the ip reputation, are still ongoing. You are very welcome to share your ideas with the group.

Like stated above, we’re still looking for coders. If you are a C coder and you’re interested in working with and for us, send us your resume!

Snort_inline 2.6.1.5 released

Posted on by

Finally, after many months of development and testing, Snort_inline 2.6.1.5 has been released. It’s the first stable release in almost a year and also the first stable release based on Snort 2.6. William sent the announcement:

snort_inline-2.6.1.5 released

List,

I know it has been a long time since we have had a non-beta release,
but what can I say? Victor and I have both been busy in our personal
and professional lives. If you have been running the version of code
in SVN, there are no major updates with this release other than a
memleak fix for stream4inline. I don't think this gets said often
enough, so I would like to thank Sourcefire for all the hard work they
put into snort and the snort rule sets for which I and the rest of the
community greatly benefit.

Regards,

Will

snort_inline-2.6.1.5
http://snort-inline.sourceforge.net/download.html

Differences between snort in inline mode and snort_inline
http://www.inliniac.net/blog/?p=74

Go and get it! 🙂