detecting: malloc(-1) or malloc(0xffffffff)

In Suricata we’re often not printing malloc errors. The reason is that we’re not willing to print such errors based on (attacker controlled) traffic. So often such cases are silently handled.

We came across a bug though, where a integer underflow led to -1/0xffffffff being passed to malloc. Luckily, malloc just failed by returning NULL, and this return was properly handled. Still, passing such a large value to malloc is a bug, so I would like to catch it.

This turned out to be trickier than I thought.

valgrind says: ==10274== Warning: silly arg (-1) to malloc(). However, it does not count it as an error. So calling valgrind with –error-exitcode=255 (my usual choice) doesn’t work.

glibc’s malloc responds to an environment variable, MALLOC_CHECK_ (I tried values 0 to 7), but this didn’t catch it at all.

AddressSanitizer also detects it as something non-fatal: ==18885== WARNING: AddressSanitizer failed to allocate 0xffffffffffffffff bytes. Not fatal like the usual errors, and no exit code.

tcmalloc, preloaded through LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.0, also just prints something to stderr: tcmalloc: large alloc 0 bytes == (nil) @

I also tried electric-fence, however this failed to work for me altogether.

As I wanted this check to be part of my QA, I needed an automated check. In this case however, I saw no other way than to just inspect the stderr output of one of the tools above. My choice was tcmalloc, as it’s fast and doesn’t require compile time options.

Suricata Development Update

SuricataWith the holidays approaching and the 1.4.7 and 2.0beta2 releases out, I thought it was a good moment for some reflection on how development is going.

I feel things are going very well. It’s great to work with a group that approaches this project from different angles. OISF has budget have people work on overall features, quality and support. Next to that, our consortium supporters help develop the project: Tilera’s Ken Steele is working on the Tile hardware support, doing lots optimizations. Many of which benefit performance and overall quality for the whole project. Tom Decanio of Npulse is doing great work on the output side, unifying the outputs to be machine readable. Jason Ish of Emulex/Endace is helping out the configuration API, defrag, etc. Others, both from the larger community and our consortium, are helping as well.

QA

At our last meetup in Luxembourg, we’ve spend quite a bit of time discussing how we can improve the quality of Suricata. Since then, we’ve been working hard to add better and more regression and quality testing.

We’ve been using a Buildbot setup for some time now, where on a number of platforms we do basic build testing. First, this was done only against the git master(s). Eric has then created a new method using a script call prscript. It’s purpose is to push a git branch to our buildbot _before_ it’s even considered for inclusion.

Recently, with cooperation of Emerging Threats, we’ve been extending this setup to include a large set of rule+pcap matches that are checked against each commit. This too is part of the pre-include QA process.

There are many more plans to extend this setup further. I’ve set up a private buildbot instance to serve as a staging area. Things we’ll be adding soon:
– valgrind testing
– DrMemory testing
– clang/scan-build
– cppcheck

Ideally, each of those tools would report 0 issues, but thats hard in practice. Sometimes there are false positives. Most tools support some form of suppression, so one of the tasks is to create those.

We’ve spend some time updating our documents regarding contributing to our code base. Please take a moment to a general contribution page, aimed at devs new to the project.

Next to this, this document describes quality requirements for our code, commits and pull requests.

Suricata 2.0

Our roadmap shows a late January 2.0 final release. It might slip a little bit, as we have a few larger changes to make:
– a logging API rewrite is in progress
“united” output, an all JSON log method written by Tom Decanio of Npulse [5]
app-layer API cleanup and update that Anoop is working on [6]

Wrapping up, I think 2013 was a very good year for Suricata. 2014 will hopefully be even better. We will be announcing some new support soon, are improving our training curicullum and will just be working hard to make Suricata better.

But first, the holidays. Cheers!