In 2.1beta2 the long awaited SMTP file extraction support for Suricata finally appeared. It has been a long development cycle. Originally started by BAE Systems, it was picked up by Tom Decanio of FireEye Forensics Group (formerly nPulse Technologies) followed by a last round of changes from my side. But it’s here now.
- a MIME decoder
- updates to the SMTP parser to use the MIME decoder for extracting files
- SMTP JSON log, integrated with EVE
- SMTP message URL extraction and logging
As it uses the Suricata file handling API, it shares almost everything with the existing file handling for HTTP. The rule keyword work and the various logs work automatically with SMTP as well.
Trying it out
To enable the file extraction, make sure that the MIME decoder is enabled:
app-layer: protocols: smtp: enabled: yes # Configure SMTP-MIME Decoder mime: # Decode MIME messages from SMTP transactions # (may be resource intensive) # This field supercedes all others because it turns the entire # process on or off decode-mime: yes # Decode MIME entity bodies (ie. base64, quoted-printable, etc.) decode-base64: yes decode-quoted-printable: yes # Maximum bytes per header data value stored in the data structure # (default is 2000) header-value-depth: 2000 # Extract URLs and save in state data structure extract-urls: yes
Like with HTTP, SMTP depends on the stream engine working correctly. So this page applies https://redmine.openinfosecfoundation.org/projects/suricata/wiki/File_Extraction, although of course the HTTP specific settings are irrelevant to SMTP.
Troubleshooting (SMTP) file extraction issues should always start here: https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Self_Help_Diagrams#File-Extraction-and-Logging-Issues
Enabling the SMTP logging is simple, just add ‘smtp’ to the list of types in your EVE config, like so:
# Extensible Event Format (nicknamed EVE) event log in JSON format - eve-log: enabled: yes filetype: regular #regular|syslog|unix_dgram|unix_stream filename: eve.json # the following are valid when type: syslog above #identity: "suricata" #facility: local5 #level: Info ## possible levels: Emergency, Alert, Critical, ## Error, Warning, Notice, Info, Debug types: - alert: # payload: yes # enable dumping payload in Base64 # payload-printable: yes # enable dumping payload in printable (lossy) format # packet: yes # enable dumping of packet (without stream segments) # http: yes # enable dumping of http fields - http: extended: yes # enable this for extended logging information # custom allows additional http fields to be included in eve-log # the example below adds three additional fields when uncommented #custom: [Accept-Encoding, Accept-Language, Authorization] - dns - tls: extended: yes # enable this for extended logging information - files: force-magic: no # force logging magic on all logged files force-md5: no # force logging of md5 checksums #- drop - smtp - ssh # bi-directional flows #- flow # uni-directional flows #- newflow
As a bonus, the MIME decoder also extracts URL’s from the SMTP message body (not attachments) and logs them in the SMTP log. This should make it easy to post process them. Currently only ‘HTTP’ URLS are extracted, starting with ‘
http://‘. So HTTPS/FTP or URLs that don’t have the protocol prefix aren’t logged.
Naturally, if you’re using SMTP over TLS or have STARTTLS enabled, as you should at least on public networks, none of this will work.
Please help us test this feature!
stable 4.0.4,file extration function of SMTP,FTP,SMB and NFS protocol,failed.
After repeated testing, only the files transferred in the HTTP protocol traffic can be extracted. The other four protocols, such as FTP, NFS, SMB and SMTP, all failed to be extracted.