Migrating a Sguil server from 0.6.1 to 0.7.0 (CVS)

Today I finally restored my server that used to host my blog, mail server and sguil server. The sguil server was still at 0.6.1 so this was a good time to see how a migration procedure would work (the earlier 0.7.0 test were done with a newly setup server). I haven’t been able to find documentation about this procedure, but it looks very straightforward, so I think I did it all right.

Like everyone always does, I started by creating a backup (you do make backups before upgrades, don’t you? ;-) )

Next I got the latest version of Sguil from CVS. I made cvs place the directory in the /usr/local/ directory. In /usr/local/sguil/server/sql_scripts I saw a tcl script named ‘update_0.7.tcl’. As the name looked very promising, I decided to try it. I ran the script as a normal user, not as root.

$ ./update_0.7.tcl

This script is used for upgrading from Sguil Version 0.6.x
to Sguil Version 0.7.x only.

Use this script at your own risk. Be sure to back
up your data before proceeding!!
Do you want to continue? (y/N)

This looks good, let’s continue.

Database password:

First the script asks for the password of your database. Note that you need to enter the root password for your MySql database, not that of the sguil user.

Connecting to database…Success.
Trying to use database sguildb…Success.
Sguild DB Versions: 0.11
The DB schema needs to be updated. Do it now ([y]/n)?:

Yes please.

Path to update_sguildb_v11-v12.sql [./update_sguildb_v11-v12.sql]:

The default path is valid, so I pressed enter.

Updating database sguildb……………………..Success.
Getting a list of current Snort sensors…Success.

Okay, so far so good. This took only a few moments.

WARNING: The next step is important. Please make sure you understand the concept before continuing.

The functions of the old sensor agent have been split out into separate agents (snort_agent, pcap_agent, sancp_agent, and pads_agent). Each agent requires its own sensor id (sid). Older Sguil installs used the same sid for Snort alerts and SANCPflows. The separation of these agents also allows you to place agents on different pieces of hardware.

Net names are used to correlate data between these agents. For example, when an analyst requests the pcap associated with hosts from a specific alert, sguild will use the net name to determine which pcap agent to make the request too (each agent registers its net name when it connects).

This next process will create a new sid for any sensors and also prompt you for a net name to assign to them. Make sure you remember these net names as you will need them when you configure the agents. Net names should be simple descriptors like Corp_Ext_Net, DMZ, or web_farm.

** Press <Enter> when you are ready to continue **

I’m ready to go!

sid | hostname | net_name | agent_type
========================
1 | vuurmuur | |
2 | wiki-ids-01 | |
3 | eagle | |
4 | wikiweb01 | |

Please enter a net name for vuurmuur:

Nice sensor names huh? :) The sensor ‘vuurmuur’ is located at my home, so thats how I will call the net_name: ‘home’.

The net name for vuurmuur will be set to home.
Is this correct (y/n)?

Yes sir!

Updating net name and agent type for vuurmuur…Success.
Checking for SANCP data from this sensor…Success.
Found SANCP data for the sensor vuurmuur with an ID of 1.
Adding agent information to the sensor table…Success.
Updating SANCP data to reflect new sid (5). This could take a bit…Success.

This took about 7 minutes.

sid | hostname | net_name | agent_type
========================
1 | vuurmuur | home | snort
2 | wiki-ids-01 | |
3 | eagle | |
4 | wikiweb01 | |
5 | vuurmuur | home | sancp

Please enter a net name for wiki-ids-01:

This IDS sensor is located in Utrecht, so I’m naming the net_name ‘utrecht’. This time the process finished in about 2 seconds. Of the sensors ‘eagle’ and ‘wikiweb01′ the former went in ‘home’, the latter in ‘utrecht’. Both are ModSecurity sensors and thus have no associated sancp data. The update script properly detected this:

No SANCP data was found for the sensor eagle with an ID of 3.
No SANCP data was found for the sensor wikiweb01 with an ID of 4.

Okay, this script is done:

** Finished. The DB has been upgraded. **

So far it all went very smooth. Let’s have a look at the configuration. I’m using my 0.6.1 sguild.conf, which is fine although it needs a few adjustments.

Because my sguil installation is now at a different location I needed to change the library path setting:

# Path the sguild libs
set SGUILD_LIB_PATH /usr/local/sguil/server/lib

There is also a new directive in the configuration file:

# Where to store DB LOADable files until loaderd can put them in the DB
set TMP_LOAD_DIR /sguild_data/load

I also created this directory.

Next I couldn’t think of anything more to do, so I started sguild. And it worked! Client could connect, sensors could connect (ModSecurity sensors are not yet updated because Modsec2Sguil doesn’t support 0.7 yet) so as far as I can see, the upgrade process is complete and successful! Nice work Bamm! :)

Migrating from ModSecurity 1.9.4 to 2.0.4

ModSecurity 2 has been out for a while now, and although I have played with it some, I never found some time to upgrade my own servers. The upgrading generally went quite smooth, even though ModSecurity 2 changed quite a bit.

First of all there are now 5 phases where you can filter. Actually, one of them only applies to the logging, so you can filter in 4 phases. The phases are headers and body for both request and response traffic. Filtering on specific URIs can be done in phase 1 (request headers), while inspecting a POST payload requires phase 2 (request body).

Next, some shortcuts where removed. In 1.9.4 there was a variable called POST_PAYLOAD, that enabled the user to match against payloads from POST requests easily. Now there is REQUEST_BODY, but since that can be part of non-POST requests as well, you have to use:

SecRule REQUEST_METHOD “POST” chain
SecRule REQUEST_BODY “evil”

instead of:

SecFilterSelective POST_PAYLOAD “evil”

One other change is visible above already. The keyword to create a rule has been changed from SecFilterSelective to SecRule. Many rules can be converted by just replacing the keyword, but certainly not all. A simple find/replace should not be done without a manual review!

I use a number of custom rules to protect certain parts of my server, so I needed to convert my rules. For most of them it was simply enough to replace SecFilterSelective with SecRule. For a few I had to replace the OUTPUT_STATUS variable with the RESPONSE_STATUS variable, as it is called now.

For one rule however, I had quite some problems to get it running correctly. This was the rule in 1.9.4 syntax:

# block wp-login.php
SecFilterSelective REMOTE_ADDR “!10.1.1.1″ chain
SecFilterSelective REQUEST_URI “/wp-login.php”
log,deny,redirect:http://www.inliniac.net/nologin.html

This rule makes sure only 10.1.1.1 can open the login page, everyone else is redirected to a simple html page containing a ‘Access Denied – Logins disabled’ message. I converted it to the following:

SecRule REMOTE_ADDR “!10.1.1.1″ chain
SecRule REQUEST_URI “/wp-login.php”
log,deny,redirect:http://www.inliniac.net/nologin.html

Guess what? It didn’t work. I’ve spend quite some time trying all kinds of variations of the rule, and finally I found out what the issue is. In 1.9.4 the rule actions, like deny, redirect etc could be in the final rule of a series of chained rules. With 2.0.4 this doesn’t work correctly. So when I changed the rules to the following, it worked:

# block wp-login.php
SecRule REMOTE_ADDR “!10.1.1.1″ “chain,phase:1,log,deny,redirect:http://www.inliniac.net/nologin.html&#8221;
SecRule REQUEST_URI “/wp-login.php$”

I haven’t looked into this further to find out whether this is a bug or a feature.

The last thing that was interesting is the modsec2sguil script. There have been some changes to the alert files. So expect a new version of the script soon!