Monthly Archives: February 2016

Fuzzing Suricata with pcaps

Posted on by

Yesterday I wrote about fuzzing Suricata with AFL. Today I’m going to show another way. Since early in the project, we’ve shipped a perl based fuzzer called ‘wirefuzz’. The tool is very simple. It takes a list of pcaps, changes random bits in them using Wiresharks editcap and runs them through Suricata. Early in the project Will Metcalf, who wrote the tool, found a lot of issues with it.

Since it’s random based fuzzing, the fuzzing is quite shallow. It is still a great way of stressing the decoder layers of Suricata though, as we need to be able to process all junk input correctly.

Lately we had an issue that I thought should have been found using fuzzing: #1653, and indeed, when I started fuzzing the code I found the issue within an hour. Pretty embarrassing.

Another reason to revisit is Address Sanitizer. It’s great because it’s so unforgiving. If it finds something it blows up. This is great for fuzzing. It’s recommended to use AFL with Asan as well. Wirefuzz does support a valgrind mode, but that is very slow. With Asan things are quite fast again, while doing much more thorough checking.

So I decided to spend some time on improving this tool so that I can add it to my CI set up.

Here is how to use it.

git clone https://github.com/inliniac/suricata -b dev-fuzz-v3.1
cd suricata
git clone https://github.com/OISF/libhtp -b 0.5.x
bash autogen.sh
export CFLAGS="-fsanitize=address"
./configure --disable-shared --sysconfdir=/etc
make
mkdir fuzzer
# finally run the fuzzer
qa/wirefuzz.pl -r=/home/victor/pcaps/*/* -c=suricata.yaml -e=0.02 \
    -p=src/suricata -l=fuzzer/ -S=rules/http-events.rules -N=1

What this command does is:

  • run from the source dir, output into fuzzer/
  • modify 2% of each pcap randomly while making sure the pcap itself stays valid (-e=0.02)
  • use the rules file rules/http-events.rules exclusively (-S)
  • use all the pcaps from /home/victor/pcaps/*/*
  • return success if a single pass over the pcaps was done (-N=1)

One thing to keep in mind is that the script creates a copy of the pcap when randomizing it. This means that very large files may cause problems depending on your disk space.

I would encourage everyone to fuzz Suricata using your private pcap collections. Then report issues to me… pretty please? 🙂

*UPDATE 2/15*: the updated wirefuzz.pl is now part of the master branch.

Fuzzing Suricata with AFL

Posted on by

AFL is a very powerful fuzzer, that tries to be smarter than random input generating fuzzers. It’s cool, but needs a bit more baby sitting. I’ve added some support to Suricata to assist AFL.

Here’s how to get started on fuzzing pcaps.

mkdir ~/tmp/fuzz
git clone https://github.com/inliniac/suricata -b dev-afl-v5
cd suricata
git clone https://github.com/OISF/libhtp -b 0.5.x
bash autogen.sh
export CFLAGS="-fsanitize=address"
export AFLDIR=/opt/afl-1.96b/bin/
export CC="${AFLDIR}/afl-gcc"
export CXX="${AFLDIR}/afl-g++"
./configure --disable-shared --sysconfdir=/etc --enable-afl

The configure output should show:
Compiler: /opt/afl-1.96b/bin//afl-gcc (exec name) / gcc (real)

make

# create tmp output dir for suricata
mkdir tmp/

# test the command to be fuzzed
src/suricata --runmode=single -k none -c suricata.yaml -l tmp/ \
    -S /dev/null \
    -r /opt/afl-1.96b/share/afl/testcases/others/pcap/small_capture.pcap

# start the fuzzer
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i /opt/afl-1.96b/share/afl/testcases/others/pcap/ -o aflout -- \
    src/suricata --runmode=single -k none -c suricata.yaml -l tmp/ \
    -S /dev/null -r @@

AFL should start running:

afl

Couple of things to keep in mind:

  • the above list assumes you have a /etc/suricata/ set up already, including a reference.config and classification.config
  • don’t skip the test step or you risk that AFL will just fuzz some basic error reporting by Suricata
  • the used ‘dev-afl-v5’ branch makes fuzzing faster and more reliable by disabling random, threading and a few other things
  • src/suricata –build-info should show the compiler is afl
  • keep your test cases small, even then runtime is going to be very long. AFL takes the input and modifies it to find as many unique code paths as possible

 

Fuzzing rules and YAMLs

For fuzzing rules and YAMLs the compilation steps are the same.

To fuzz rules, create a directory & test input:

mkdir testrules
echo 'alert http any any -> any any (content:"abc"; sid:1; rev:1;)' \
    > testrules/rules.txt

# test command
src/suricata -c suricata.yaml -l tmp/ --afl-parse-rules -T \
    -S testrules/rules.txt

# run AFL
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i testrules/ -o aflout -- \
    src/suricata -c suricata.yaml -l tmp/ --afl-parse-rules \
    -T -S @@

Finally, YAMLs:

mkdir testyamls/
cp suricata.yaml testyamls/

# test command
src/suricata -l tmp/ --afl-parse-rules -T -S testrules/rules.txt \
    -c testyamls/suricata.yaml

# run AFL
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i testyamls/ -o aflout -- \
    src/suricata -l tmp/ --afl-parse-rules \
    -T -S testrules/rules.txt -c @@

Note that the default YAML is HUGE for this purpose. It may be more efficient to use a sub set of it.

I plan to create some wrapper scripts to make things easier in the near future. Meanwhile, if you have crashes to report, please send them my way!