Follow up on Sguil securtiy

In the discussion about my post about Sguil security there have been a number of ideas and general thoughts. I’d like to write about them here to we can further discuss them. There seems to be consensus on that when a sensors is rooted, there is nothing we can do to prevent injection of bogus data as long as it isn’t malformed.

Having the agent authenticate itself is a solution, but it relies on the agent credentials to remain secret. So when a webserver is rooted the attacker will have access to the credentials as they will be stored on the webserver itself. So this approach does provide an extra layer of defense but local roots aren’t uncommon, so it remains risky. It may still be worth the effort though.

One idea that came up was a proxy, in two forms. A Sguil proxy filtering the Sguil agent-server protocol is one of them. This would be positioned between the Sguil server and the agent to prevent the attacker going after the server directly. The proxy would be able to filter on the communications and could maybe trottle the rate of events to protect agains DoSsing the server. Maybe it could also block an agent completely is it’s sending malformed commands. Such a proxy could be positioned on the border of a monitored and management network.

David Bianco suggested a different proxy-like setup in which the actual sensor would receive the raw data over the network in some (secure) way from the webserver. This way the sensor talking to the Sguil server wouldn’t have run on the webserver. I think this is a good idea for the webservers. It does add complexity, because there will have to be some form of communications between the sensor and the webserver. At the same time this could simplify things as a single sensor box can deal with multiple webservers. In this case the credentials of the agent won’t be stored at the webserver, which is another plus. With this idea, much relies on how the webserver-sensor communications would be implemented.

Both ideas won’t be able to prevent an attacker owning the webserver to insert bogus data. Also in both cases the proxy can and probably will get a target itself.

Even though the risk increases with agents running on webserver, it also exists on other types of sensors. If a hole is found in Snort, Snort_inline, Sancp or other tools Sguil uses, the effects could be the same.

More ideas and suggestions are welcome!

Thoughts on Sguil security

Sguil is build using a server and sensors. Traditionally the sensors are passive monitoring agents running Snort and a few other tools. Best practice was (and still is) to separate the management network of these sensors and server from the monitored network(s). This way it would be fairly hard for an attacker to get a shot at the Sguil server.

Sguil of course, would be a extremely interesting target for hackers. It contains so much info about the monitored network. Also, it has realtime access to all network traffic. A hacker may also be interested in shutting Sguil down to avoid detection.

Securing Sguil is therefore very important. Sguil has a number of defenses. It separates agent and client access so an agent cannot issue commands a client only should issue. The clients need to authenticate to the server and Sguil provides the option to do both the agent-server and the client-server communications over SSL. Finally Sguil also has the option to only allow certain ipaddresses to connect. This can be set both for agents and clients separately.

With my Modsec2sguil agent, a part of these defenses go overboard. Modsec2sguil is likely to be run on webservers directly, and therefore the separation of a monitored network and a management network is no longer possible. Webservers tend to get hacked a lot more often than IDS systems, so this is an additional risk. A user getting access to the webserver is able to see that there is a Sguil agent active and will be able to connect to the Sguil server directly.

When all security of Sguil is in place the only thing this attacker will be able to do is act as an agent to Sguil. No authentication is required (or possible). The risk here is that the attacker can send bogus events to flood the analyst and Sguil itself. Possibly even DoSsing the server. By sending malformed data, the attacker could also try to crash the server.

So what can be done about it? Well I really can’t think of much other than to add authentication for agents in addition to the clients. This would provide an extra hurdle for the attacker because he has to get the right credentials. However, these credentials will have to be in the agent configuration on the sensor, so if the attacker manages to escalate his privileges he will get access to the credentials and this defense will fail. Still, it’s another layer of defense, so I think it’s better than nothing.

I’m very much interested in hearing other opinions about this!

Using Modsec2sguil for HTTP transaction logging revisited

Recently I wrote about the idea to log all HTTP transactions into Sguil using my Modsec2sguil agent. I’ve implemented this in the current 0.8-dev5 release and it works very well. All events go into Sguil smoothly and I’ve not experienced slowdowns on the webserver. I’ve been running it for almost a week now, like to share the first experiences here.

I find it to be quite useful. When receiving an alert, it is perhaps more interesting to see what else was done from that ipaddress than to see what was blocked (unless you are suspecting a false positive of course). One area I find to be useful is when I’m creating rules against comment spam on this blog. By seeing all properties of a spam message I can create better rules. For example on broken user-agents or weird codes inserted into the comment field of WordPress.

It’s easy to search and filter on HTTP response codes because the code is a part of the RT message. For example, when searching for all HTTP 500 error codes, add the following ‘WHERE’ clause to a query:

WHERE event.signature like “%MSc 500%”

This works quite fast although you best limit the query on properties like date and port as well. To get all the HTTP code 500 alerts from the last days do something like:

WHERE event.timestamp > ‘2007-08-18’ AND (event.dst_port = 80 OR event.dst_port = 443) AND event.signature like “%MSc 500%”

One thing that is disappointing is the inabillity to search in the event payloads stored in the database. Technically it’s possible to create mysql queries that search for certain strings, but this process is so slow that it’s hardly usable in practice. The problem here is that the database field containing the payload is not indexed. I’ll show the query I used here (ripped from David Bianco’s blog)

WHERE event.timestamp >= ‘2007-08-18’ AND (event.dst_port = 80 OR event.dst_port = 443) AND data.data_payload like CONCAT(“%”, HEX(“Mozilla/5.0”), “%”)

If you know a more efficient query, please let me know!

Using Modsec2sguil for HTTP transaction logging

Modsec2sguil is currently configured to send alerts to Sguil. ModSecurity can be configured to log any event or transaction, including 200 OK, 302 Redirect, etc. Modsec2sguil distinguishes between alerts and other events by only processing HTTP codes of 400 and higher. Since 0.8-dev2 there is a configuration directive to prevent certain codes, such as 404, from being treated as an alert.

Now I have the following idea. Since ModSecurity can log all events with details of request headers, response headers and POST message body, it may be interesting to just send all these events to Sguil. They should not be appearing as alerts, but having them in the database can perhaps be interesting. I know using flow data and full packet captures the same data can be accessed, but having it in the database makes querying it a lot easier and longer available.

Possible problems are mostly the performance hit the webserver may take for sending all these events to Sguil and the storage requirements in Sguil’s database. I estimate the events are about 1kb in size on average, so on a busy site this may cause the database to grow very rapidly. Of course this behavior would be optional so it can be disabled.

Any thoughts on this idea?

First Modsec2sguil release for Sguil 0.7-CVS

I just uploaded a new version of Modsec2sguil. I’ve been working on it the last weeks to get it updated to Sguil 0.7. The scripts are changed all over the place. This is because in the 0.7 framework, my scripts would no longer be a replacement for Barnyard only talking to the sensor_agent on the localhost, instead now it would become a full agent talking to the Sguil server directly.

This brings some challenges. First the connection can be going over the internet, or another untrusted network, so the agent needs ssl support. Second, since the connection may be unreliable we need to be able to detect and deal with lost connections. Next to this I wanted to be able to run without superuser privileges.

The new version of modsec2sguil supports it all, and more:

  • Converted into a real agent for Sguil 0.7 (no more barnyard replacement)
  • Agent can drop privileges
  • Agent can daemonize
  • Pinging the server is supported
  • The agent reconnects to the server if the connection is lost
  • Agent supports SSL for the connection to the server
  • A sguil-compatible configuration file is now used
  • A debug mode was added

So if you run Sguil 0.7-CVS and ModSecurity, go check it out at http://www.inliniac.net/modsec2sguil/

Last but not least, the agent contains a SguilAgent.pm Perl library. I hope it enables developers to easily create Perl agents for Sguil. If you need help with that, please let me know!

Migrating a Sguil server from 0.6.1 to 0.7.0 (CVS)

Today I finally restored my server that used to host my blog, mail server and sguil server. The sguil server was still at 0.6.1 so this was a good time to see how a migration procedure would work (the earlier 0.7.0 test were done with a newly setup server). I haven’t been able to find documentation about this procedure, but it looks very straightforward, so I think I did it all right.

Like everyone always does, I started by creating a backup (you do make backups before upgrades, don’t you? 😉 )

Next I got the latest version of Sguil from CVS. I made cvs place the directory in the /usr/local/ directory. In /usr/local/sguil/server/sql_scripts I saw a tcl script named ‘update_0.7.tcl’. As the name looked very promising, I decided to try it. I ran the script as a normal user, not as root.

$ ./update_0.7.tcl

This script is used for upgrading from Sguil Version 0.6.x
to Sguil Version 0.7.x only.

Use this script at your own risk. Be sure to back
up your data before proceeding!!
Do you want to continue? (y/N)

This looks good, let’s continue.

Database password:

First the script asks for the password of your database. Note that you need to enter the root password for your MySql database, not that of the sguil user.

Connecting to database…Success.
Trying to use database sguildb…Success.
Sguild DB Versions: 0.11
The DB schema needs to be updated. Do it now ([y]/n)?:

Yes please.

Path to update_sguildb_v11-v12.sql [./update_sguildb_v11-v12.sql]:

The default path is valid, so I pressed enter.

Updating database sguildb……………………..Success.
Getting a list of current Snort sensors…Success.

Okay, so far so good. This took only a few moments.

WARNING: The next step is important. Please make sure you understand the concept before continuing.

The functions of the old sensor agent have been split out into separate agents (snort_agent, pcap_agent, sancp_agent, and pads_agent). Each agent requires its own sensor id (sid). Older Sguil installs used the same sid for Snort alerts and SANCPflows. The separation of these agents also allows you to place agents on different pieces of hardware.

Net names are used to correlate data between these agents. For example, when an analyst requests the pcap associated with hosts from a specific alert, sguild will use the net name to determine which pcap agent to make the request too (each agent registers its net name when it connects).

This next process will create a new sid for any sensors and also prompt you for a net name to assign to them. Make sure you remember these net names as you will need them when you configure the agents. Net names should be simple descriptors like Corp_Ext_Net, DMZ, or web_farm.

** Press <Enter> when you are ready to continue **

I’m ready to go!

sid | hostname | net_name | agent_type
========================
1 | vuurmuur | |
2 | wiki-ids-01 | |
3 | eagle | |
4 | wikiweb01 | |

Please enter a net name for vuurmuur:

Nice sensor names huh? 🙂 The sensor ‘vuurmuur’ is located at my home, so thats how I will call the net_name: ‘home’.

The net name for vuurmuur will be set to home.
Is this correct (y/n)?

Yes sir!

Updating net name and agent type for vuurmuur…Success.
Checking for SANCP data from this sensor…Success.
Found SANCP data for the sensor vuurmuur with an ID of 1.
Adding agent information to the sensor table…Success.
Updating SANCP data to reflect new sid (5). This could take a bit…Success.

This took about 7 minutes.

sid | hostname | net_name | agent_type
========================
1 | vuurmuur | home | snort
2 | wiki-ids-01 | |
3 | eagle | |
4 | wikiweb01 | |
5 | vuurmuur | home | sancp

Please enter a net name for wiki-ids-01:

This IDS sensor is located in Utrecht, so I’m naming the net_name ‘utrecht’. This time the process finished in about 2 seconds. Of the sensors ‘eagle’ and ‘wikiweb01’ the former went in ‘home’, the latter in ‘utrecht’. Both are ModSecurity sensors and thus have no associated sancp data. The update script properly detected this:

No SANCP data was found for the sensor eagle with an ID of 3.
No SANCP data was found for the sensor wikiweb01 with an ID of 4.

Okay, this script is done:

** Finished. The DB has been upgraded. **

So far it all went very smooth. Let’s have a look at the configuration. I’m using my 0.6.1 sguild.conf, which is fine although it needs a few adjustments.

Because my sguil installation is now at a different location I needed to change the library path setting:

# Path the sguild libs
set SGUILD_LIB_PATH /usr/local/sguil/server/lib

There is also a new directive in the configuration file:

# Where to store DB LOADable files until loaderd can put them in the DB
set TMP_LOAD_DIR /sguild_data/load

I also created this directory.

Next I couldn’t think of anything more to do, so I started sguild. And it worked! Client could connect, sensors could connect (ModSecurity sensors are not yet updated because Modsec2Sguil doesn’t support 0.7 yet) so as far as I can see, the upgrade process is complete and successful! Nice work Bamm! 🙂

Update on Sguil 0.7-CVS client on Ubuntu Feisty

A short time ago I wrote about how the Sguil 0.7-CVS client can be installed on Ubuntu Feisty. Since then there has been a change to Sguil that changes the requirements a bit. Because of this the standard tcllib package in Feisty is no longer usable. It provides tcllib 1.8 while Sguil now needs 1.9. Luckily, we can use the tcllib package from the upcoming Ubuntu release called ‘Gutsy’. It can be found here: http://packages.ubuntu.com/gutsy/interpreters/tcllib

I installed it with this command:

$ sudo dpkg –install tcllib_1.9.dfsg1-1_all.deb

This made Sguil 0.7-CVS work again on my system.

Sguil 0.7-CVS client on Ubuntu Feisty

I just got a new workstation that I’m setting up today with Ubuntu Feisty 7.04. When setting up the Sguil client from CVS I needed to install the following packages (including dependencies, but apt-get takes care of that):

tcl8.4
tclx8.4
tcllib
tk8.4
iwidgets4

After this it ran but looked horrible because of some ugly font that was used. I found that for my use the following fonts look good:

standard: Bitstream Vera Sans 12
fixed: Bitstream Vera Sans Mono 12

Both fonts are installed by default and can be selected in the Sguil client by opening the ‘File’ dropdown menu and choosing ‘Change Font’.

Sguil 0.7 CVS installation on Debian Etch

Sguil 0.7 is getting shape quite nicely. One of the most interesting new features is the splitting up of different types of agents and the option to create ‘net groups’. This are groups of agents that Sguil considers part of the same network. You can use this to spread the agents over multiple servers, but still use it from Sguil as if it was one single sensor. For example, this way you can easily create a Snort sensor and a separate full content logging capture server. When you request the full content for a Snort event in Sguil, it will know that it needs to request the packet data from the capture server. This way you can also have multiple Snort agents without the need for capturing the same sancp and full content data over and over again.

David Bianco has written a very nice guide for installing Sguil 0.7 on Redhat Enterprise 4. I used this guide to install the server and sensor on a Debian Etch installation. The main difference is that I used Debian packages where ever possible. These packages could be used:

mysql-server
p0f
tcpflow
tcllib
mysqltcl
tcltls
tcl8.3
tclx8.3

Important: do not use the tcl8.4 package. It is not compatible with Sguil and will produce the following message:

ERROR: This version of tcl was compile with threading enabled. Sguil is NOT compatible with threading.
SGUILD: Exiting…

You can get Sguil 0.7 CVS by checking out the latest CVS version:

cvs -d:pserver:anonymous@sguil.cvs.sourceforge.net:/cvsroot/sguil login
cvs -d:pserver:anonymous@sguil.cvs.sourceforge.net:/cvsroot/sguil co sguil

I will update Modsec2sguil soon!

Modsec2sguil 0.7 released

I’ve just released version 0.7 of Modsec2sguil, the set of perl scripts to feed ModSecurity alerts to the Sguil NSM system. The main change of this release is that it adds support for alerts produced by ModSecurity 2.x, while 1.9.x remains to be supported. Next to this the conversion between ModSecurity’s severity and Snort’s priority was fixed, so alerts should show up in the right pane in Sguil again.

Please give this release a try and let me know how it works for you!

Download it here: http://www.inliniac.net/files/modsec2sguil-0.7.tar.gz