Linux Tomorrow

13 Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures)

There is currently a lot of spam where the spam "information" is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.

13.1 Create a Folder and Download the Script

Create a folder for sanesecurity and download and give the script the proper permission.

apt-get install curl

mkdir /usr/src/sanesecurity
cd /usr/src/sanesecurity
wget http://www.sanesecurity.co.uk/clamav/ss-msrbl.txt
mv ss-msrbl.txt /usr/bin/ss_update.sh
chmod +x /usr/bin/ss_update.sh

Edit ss_update.sh and change the following variables to match your installation:

clam_sigs="/var/lib/clamav"

The variable clamav_sigs contains the path to the directory where your ClamAV signatures are stored.

clam_user="clamav"

Now we run the update script to check if the download works:

./ss_update.sh

The result should look similar to this:

=================================
SaneSecurity SCAM Database Update
=================================
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 116k 100 116k 0 0 65448 0 0:00:01 0:00:01 --:--:-- 139k

==================================
SaneSecurity PHISH Database Update
==================================
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 179k 100 179k 0 0 216k 0 --:--:-- --:--:-- --:--:-- 216k

==========================
MSRBL SPAM Database Update
==========================
Number of files: 1
Number of files transferred: 1
Total file size: 228436 bytes
Total transferred file size: 228436 bytes
Literal data: 228436 bytes
Matched data: 0 bytes
File list size: 33
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 101
Total bytes received: 228579
sent 101 bytes received 228579 bytes 26903.53 bytes/sec
total size is 228436 speedup is 1.00

===========================
MSRBL IMAGE Database Update
===========================
Number of files: 1
Number of files transferred: 1
Total file size: 550503 bytes
Total transferred file size: 550503 bytes
Literal data: 550503 bytes
Matched data: 0 bytes
File list size: 35
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 103
Total bytes received: 550688
sent 103 bytes received 550688 bytes 157368.86 bytes/sec
total size is 550503 speedup is 1.00

Now we a add the script to the root crontab to be run once a day:

crontab -e

Add the following line at the end of the root crontab:

00 04 * * * /usr/bin/ss_update.sh &> /dev/null

14 GreyListing with Postfix-gld

14.1 Installing Postfix-gld

apt-get install postfix-gld

cd /usr/src
wget http://www.gasmi.net/down/gld-1.7.tgz
tar xvfx gld-1.7.tgz
cd gld-1.7

Create MySQL Database:

mysql –u root –p

mysql> create database gld_db
mysql> GRANT ALL ON gld_db.* TO gld_user@localhost IDENTIFIED BY ‘gld_password’;
mysql> flush privileges;

Import tables.mysql:

mysql –u gld_user –p gld_db < /path/to/tables.mysql

Import table-whitelist.sql:

mysql –u gld_user –p gld_db < /path/to/table-whitelist.sql

You will have to enable it by configuring that in the /etc/default/gld:

vi /etc/default/gld

#/etc/default/gld
ENABLED=1

14.2 Configuration

Edit /etc/gld.conf according to your needs. I'm using the following settings:

vi /etc/gld.conf

# Config file for gld
# TCP Port gld should listen to (default is 2525)
#
PORT=2525
# Shall we bind only to loopback ? (0=No,1=Yes) (default is 1)
LOOPBACKONLY=1
# The list of networks allowed to connect to us (default is everybody)
CLIENTS=127.0.0.1/32
# The user used to run gld (default value is no user change)
USER=postfix-gld
# The group used to run gld (default value is no group change)
GROUP=postfix-gld
# Maximum simultaneous connexions (default is 100)
MAXCON=100
# How many seconds we should wait before accepting a mail that is in greylist (default is 60)
MINTIME=60
# Shall we use lightgrey option ? (0=No,1=Yes) (default is 0)
# The lightgrey option, mask the last octet of IP addresses
# and thus we greylist only C classes (/24) instead of individual IPs.
LIGHTGREY=0
# Shall we use the mxgrey algorithm ? (0=No,>0=Yes) (default is 0)
# the mxgrey algorithm is a variation of the greylist algorithm.
# When this is enabled, we allow all incoming mails from an IP address
# whatever source/destination email as long as this IP has been greylisted
# at least X time and succeded the mail resend .
#
# Example:
# The IP 1.2.3.4 sends an email from src@domain.com to user@yourdomain.com
# We greylist this mail as this IP is not yet in database and send a 450 SMTP code
# After some time, the IP re-send the mail from src@domain.com to user@yourdomain.com
# We update the db.
# Some time after the ip 1.2.3.4 sends an email from john@domain.com to fred@yourdomain.com
# We will accept this mail without any greylisting, as this ip already succeded a greylist test
# and thus seems to be a valid smtp server and not a spammer .
#
# The advantage of this method, is that it reduce the re-send time due to greylisting to
# x mail per server instead of one mail per destination .
#
# The value you provide in MXGREY is the minimum number of succesful greylists
# before accepting all mails from this MX. higher the number is, harder is to get in.
#
# This algortihm replace the old LIGHTGREYDOMAIN which was available prior version 1.6
#
MXGREY=1
# Shall we use the whitelist table ?  (0=No,1=Yes) (default is 1)
# If set to yes, then the table 'whitelist' is looked up
# each time postfix request the server
# if the email/domain/ip is in the whitelist, then the response
# will be 'dunno' .
# In the whitelist table, you can set the following values:
# an email: ie john@foo.tld
# a domain: ie @bar.tld
# an IP   : ie 1.2.3.4
# a subnet: ie 1.2.3
#
WHITELIST=1
# Shall we use a DNS based whitelist ? (default is no)
# To activate it, the line must be uncommented
# and the value set to the domain of the DNS whitelist.
# for example, if DNSWL is set to toto.com and we get a mail from ip a.b.c.d
# then gld will DNS lookup d.c.b.a.toto.com
# and if found allow the ip without greylisting it.
#DNSWL=toto.com
# Shall we send a 'dunno' in case of error (mysql down,....) (0=No,1=Yes) (default is 1)
# Normaly, if an error occur, the server is supposed to close the connection
# and thus postfix will return a 450 Server configuration error
# if this parameter is set to 1, then the server will return 'dunno'
# and thus let postfix decide the fate of the mail.
ERRACCEPT=1
# Shall we log to the syslog (0=No,1=Yes) (default is 1)
SYSLOG=1
# If we use syslog, which facility shall we use (default is mail)
# it can only be one of the following facilities:
# daemon mail local0 local1 local2 local3 local4 local5 local6 local7
FACILITY=mail
# The Message that we display in case of reject (default is "Greylisted")
# If you want another SMTP return code than the default 450, just put it at
# the beginning of the message, ie: 451 You have been greylisted by gld ...
# If you don't provide any SMTP code, the default 450 will be used by postfix
# WARNING: if you set a custom smtp code make sure it's a 4XX code.
# if you don't provide a 4XX code, gld will ignore it and send the default 450.
# Be also warned that if you set a custom code, gld will not use defer_if_permit anymore
# but direct supplied code to postfix .
MESSAGE=Service temporarily unavailable, please try later
# Training mode activated ? (0=No,1=Yes) (default is 0)
# If activated, gld will do all the work but will always reply dunno to postfix
# and thus, will never greylist any mail.
# This feature is useful for testing gld performances without greylisting any mail
TRAINING=0
# SQL INFOS (defaults are localhost,myuser,mypasswd,mydb)
#
SQLHOST=localhost
SQLUSER=gld_user
SQLPASSWD=gld_password
SQLDB-gld_db

Edit /etc/postfix/main.cf and add the following to smtpd_recipient_restrictions:

vi /etc/postfix/main.cf

check_policy_service inet:127.0.0.1:2525

Do a

tail –f /var/log/mail.log

and check your log for the following:

Apr 28 09:07:03 server1 gld: Greylist activated for recipient= sender= ip=

You can set up a cron job to keep your database clean. Below is the gld command usage.

Usage: gld [-c n|-C n|-k n|-K n|-h|-v]
gld -c n : clean the database for ALL entries not updated since n days
gld -C n : show what the -c option would do, without doing it
gld -k n : clean the database for entries not updated since n days with only one hit
gld -K n : show what the -k option would do, without doing it
gld -i : show some database informations
gld -d : enable debug mode
gld -v : display version
gld -h : display Usage

15 Logwatch Statistical Reporting (Optional)

Logwatch is a customizable log analysis system. Logwatch parses through your system's logs for a given period of time and creates a report analyzing areas that you specify, in as much detail as you require.

We will be using Logwatch to give us daily reports for mailscanner. This is a way for us to see how effective mailscanner really is.

Install Logwatch:

apt-get install logwatch

Edit the /usr/share/logwatch/default.conf/logwatch.conf and set the options:

vi /usr/share/logwatch/default.conf/logwatch

Mail To = youremailaddress
Service = mailscanner

Test Logwatch:

logwatch

It should generate a log file and email it to the email you specified.

16 Automatically Add A Disclaimer To Outgoing Emails With alterMIME (Optional)

This tutorial shows how to install and use alterMIME. alterMIME is a tool that can automatically add a disclaimer to emails. In this article I will explain how to install it as a Postfix filter on Ubuntu.

16.1 Installing alterMIME

alterMIME can be installed as follows:

apt-get install altermime

Next we create the user filter with the home directory /var/spool/filter filter - alterMIME will be run as that user:

useradd -r -c "Postfix Filters" -d /var/spool/filter filter
mkdir /var/spool/filter
chown filter:filter /var/spool/filter
chmod 750 /var/spool/filter

Afterwards we create the script /etc/postfix/disclaimer which executes alterMIME. Ubuntu's alterMIME package comes with a sample script that we can simply copy to /etc/postfix/disclaimer:

cp /usr/share/doc/altermime/examples/postfix_filter.sh /etc/postfix/disclaimer
chgrp filter /etc/postfix/disclaimer
chmod 750 /etc/postfix/disclaimer

Now the problem with this script is that it doesn't distinguish between incoming and outgoing emails - it simply adds a disclaimer to all mails. Typically you want disclaimers only for outgoing emails, and even then not for all sender addresses. Therefore I've modified the /etc/postfix/disclaimer script a little bit - we'll come to that in a minute.

Right now, we create the file /etc/postfix/disclaimer_addresses which holds all sender email addresses (one per line) for which alterMIME should add a disclaimer:

vi /etc/postfix/disclaimer_addresses

user1@example.com
user2@example.org
user3@example.net

Now we open /etc/postfix/disclaimer and modify it as follows (I have marked the parts that I've changed):

vi /etc/postfix/disclaimer

#!/bin/sh
# Localize these.
INSPECT_DIR=/var/spool/filter
SENDMAIL=/usr/sbin/sendmail

####### Changed From Original Script #######
DISCLAIMER_ADDRESSES=/etc/postfix/disclaimer_addresses
####### Changed From Original Script END #######

# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69

# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15

# Start processing.
cd $INSPECT_DIR || { echo $INSPECT_DIR does not exist; exit
$EX_TEMPFAIL; }

cat >in.$$ || { echo Cannot save mail to file; exit $EX_TEMPFAIL; }

####### Changed From Original Script #######
# obtain From address
from_address=`grep -m 1 "From:" in.$$ | cut -d "<" -f 2 | cut -d ">" -f 1`

if [ `grep -wi ^${from_address}$ ${DISCLAIMER_ADDRESSES}` ]; then
/usr/bin/altermime --input=in.$$ \
               --disclaimer=/etc/postfix/disclaimer.txt \
               --disclaimer-html=/etc/postfix/disclaimer.txt \
               --xheader="X-Copyrighted-Material: Please visit http://www.company.com/privacy.htm" || \
                { echo Message content rejected; exit $EX_UNAVAILABLE; }
fi
####### Changed From Original Script END #######

$SENDMAIL "$@" 

Next we need the text file /etc/postfix/disclaimer.txt which holds our disclaimer text. Ubuntu's alterMIME package comes with a sample text that we can use for now (of course, you can modify it if you like):

cp /usr/share/doc/altermime/examples/disclaimer.txt /etc/postfix/disclaimer.txt

Finally we have to tell Postfix that it should use the /etc/postfix/disclaimer script to add disclaimers to outgoing emails. Open /etc/postfix/master.cf and add -o content_filter=dfilt: to the smtp line:

vi /etc/postfix/master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
-o content_filter=dfilt:
[...]

At the end of the same file, add the following two lines:

[...]
dfilt     unix    -       n       n       -       -       pipe
flags=Rq user=filter argv=/etc/postfix/disclaimer -f ${sender} -- ${recipient}

Restart Postfix afterwards:

/etc/init.d/postfix restart

That's it! Now a disclaimer should be added to outgoing emails sent from the addresses listed in /etc/postfix/disclaimer_addresses.

Congratulations!

You should now have a complete working SpamSnake.

Here are some Mailwatch screenshots: