Modsec2sguil is currently configured to send alerts to Sguil. ModSecurity can be configured to log any event or transaction, including 200 OK, 302 Redirect, etc. Modsec2sguil distinguishes between alerts and other events by only processing HTTP codes of 400 and higher. Since 0.8-dev2 there is a configuration directive to prevent certain codes, such as 404, from being treated as an alert.
Now I have the following idea. Since ModSecurity can log all events with details of request headers, response headers and POST message body, it may be interesting to just send all these events to Sguil. They should not be appearing as alerts, but having them in the database can perhaps be interesting. I know using flow data and full packet captures the same data can be accessed, but having it in the database makes querying it a lot easier and longer available.
Possible problems are mostly the performance hit the webserver may take for sending all these events to Sguil and the storage requirements in Sguil’s database. I estimate the events are about 1kb in size on average, so on a busy site this may cause the database to grow very rapidly. Of course this behavior would be optional so it can be disabled.
Any thoughts on this idea?
Inliniac 
Cool, so now Sguil can accept Web logs in addition to Web security events. 🙂 I’m all for extra information if your setup can handle it. Exporting Web logs via another method is usually preferred, I imagine?
The extra information is indeed what I had in mind. What this approach would have in advance above the normal webserver logs, is that request headers, response headers and the post payload are also available in the event Sguil would receive.
This is a great idea from a forensic/IR perspective. Any time a valid alert is identified and you start the IR process, the first thing that customers ask is “What else did this person do?” Without the full audit trail of HTTP transactions, you have no visibility into what they did before or after the alert. This will allow you to quickly do session reconstruction by simply querying for the IP address. Better yet, you could query based on a sessionid/cookie value to uniquely rebuild sessions. For this to work, however, it seems that the agent script would need to be able to parse the data into proper fields.
It is true though that there is a performance hit when you set the modsecurity audit engine to on (meaning log everything) vs. what most people use which is the relevantonly setting (which will only generate an audit log entry when a rule matches). There are some optimizations that could be made to lighten the load somewhat such as to filter out “static” content requests for images.
Even with some performance hurdles, this is exciting stuff 🙂