Linux Tomorrow

Author : Rocky

10 Install and Configure SPF

The postfix-policyd-spf-perl package depends on the Mail::SPF and the NetAddr::IP Perl modules.

We need to download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:

cd /usr/src
wget http://www.openspf.org/blobs/postfix-policyd-spf-perl-2.005.tar.gz
tar xvfz postfix-policyd-spf-perl-2.005.tar.gz
cd postfix-policyd-spf-perl-2.005
cp postfix-policyd-spf-perl /usr/lib/postfix/policyd-spf-perl

Then we edit /etc/postfix/master.cf and add the following stanza at the end:

vi /etc/postfix/master.cf

policy unix - n n - - spawn
user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl

(The leading spaces before user=nobody are important so that Postfix knows that this line belongs to the previous one!)

Then open /etc/postfix/main.cf and search for the smtpd_recipient_restrictions directive. You should have reject_unauth_destination in that directive, and right after reject_unauth_destination you add check_policy_service unix:private/policy like this:

vi /etc/postfix/main.cf

[...]
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination,check_policy_service unix:private/policy
[...]

or like this:

[...]
smtpd_recipient_restrictions =
[...]
reject_unauth_destination
check_policy_service unix:private/policy
[...]

It is important that you specify check_policy_service AFTER reject_unauth_destination or else your system can become an open relay!

Then restart Postfix:

/etc/init.d/postfix restart

That's it already.

11 Install and Configure FuzzyOcr

apt-get install netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl imagemagick tesseract-ocr

Download and install the latest FuzzyOCR devel version from http://fuzzyocr.own-hero.net/wiki/Downloads:

cd /usr/src/
wget http://users.own-hero.net/~decoder/fuzzyocr/fuzzyocr-3.5.1-devel.tar.gz

Unpack FuzzyOCR and move all FuzzyOcr* files and the FuzzyOcr directory (they are all in the FuzzyOcr-3.5.1/ directory) to /etc/mail/spamassassin:

tar xvfz fuzzyocr-3.5.1-devel.tar.gz
cd FuzzyOcr-3.5.1/
mv FuzzyOcr* /etc/mail/spamassassin/
wget http://www.gbnetwork.co.uk/mailscanner/FuzzyOcr.words -O /etc/mail/spamassassin/FuzzyOcr.words

We will be storing the image hashes in a mysql database to improve on performance such that images that we have already scanned do not get scanned again as OCR is a resource intense activity.

11.1 Create MySQL Database

The sql script creates the database and tables and adds a user fuzzyocr with the password fuzzyocr:

mysql -p < /etc/mail/spamassassin/FuzzyOcr.mysql

Change the password:

mysqladmin -u fuzzyocr -p fuzzyocr newpassword

11.2 MailWatch Fix

Do the following to prevent an error in MailWatch:

vi /etc/mail/spamassassin/FuzzyOcr.pm

Change 'use POSIX;' to 'use POSIX qw(SIGTERM);'

11.3 FuzzyOcr Configuration

FuzzyOCR's configuration file is /etc/mail/spamassassin/FuzzyOcr.cf. In that file almost everything is commented out. We open that file now and make some modifications:

vi /etc/mail/spamassassin/FuzzyOcr.cf

Put the following line into it to define the location of FuzzyOCR's spam words file:

focr_global_wordlist /etc/mail/spamassassin/FuzzyOcr.words

/etc/mail/spamassassin/FuzzyOcr.words is a predefined word list that comes with FuzzyOCR. You can adjust it to your needs.

Next change:

# Include additional scanner/preprocessor commands here:
#
focr_bin_helper pnmnorm, pnminvert, pamthreshold, ppmtopgm, pamtopnm
focr_bin_helper tesseract

to

# Include additional scanner/preprocessor commands here:
#
focr_bin_helper pnmnorm, pnminvert, convert, ppmtopgm, tesseract

Finally add/enable the following lines:

# Search path for locating helper applications
focr_path_bin /usr/local/netpbm/bin:/usr/local/bin:/usr/bin

focr_preprocessor_file /etc/mail/spamassassin/FuzzyOcr.preps
focr_scanset_file /etc/mail/spamassassin/FuzzyOcr.scansets

focr_digest_db /etc/mail/spamassassin/FuzzyOcr.hashdb
focr_db_hash /etc/mail/spamassassin/FuzzyOcr.db
focr_db_safe /etc/mail/spamassassin/FuzzyOcr.safe.db
focr_minimal_scanset 1
focr_autosort_scanset 1
focr_enable_image_hashing 3
focr_logfile /var/log/FuzzyOcr.log

#Mysql Connection#
focr_mysql_db FuzzyOcr
focr_mysql_hash Hash
focr_mysql_safe Safe
focr_mysql_user fuzzyocr
focr_mysql_pass password
focr_mysql_host localhost
focr_mysql_port 3306
focr_mysql_socket /var/run/mysqld/mysqld.sock

This is what the FuzzyOCR developers say about image hashing:

"The Image hashing database feature allows the plugin to store a vector of image features to a database, so it knows this image when it arrives a second time (and therefore does not need to scan it again). The special thing about this function is that it also recognizes the image again if it was changed slightly (which is done by spammers). "

11.4 Test FuzzyOCR

cd /usr/src/FuzzyOcr-3.5.1/samples
spamassassin --debug FuzzyOcr <> /dev/null

You see the following:

[14808] info: FuzzyOcr: Found Score <9.000> for Exact Image Hash
[14808] info: FuzzyOcr: Matched [1] time(s). Prev match: 16 sec. ago
[14808] info: FuzzyOcr: Message is SPAM. Words found:
[14808] info: FuzzyOcr: "price" in 1 lines
[14808] info: FuzzyOcr: "company" in 1 lines
[14808] info: FuzzyOcr: "alert" in 1 lines
[14808] info: FuzzyOcr: "news" in 1 lines
[14808] info: FuzzyOcr: (6 word occurrences found)
[14808] dbg: FuzzyOcr: Remove DIR: /tmp/.spamassassin14808JZSvHBtmp
[14808] dbg: FuzzyOcr: Processed in 0.104555 sec.

12 Apply Relay Recipients

The following directions are meant for people using Microsoft Exchange 2000 or Microsoft Exchange 2003.

This page describes how to configure your mail gateway to periodically get a list of valid recipient email addresses from your Exchange system. By doing this, you can configure your server to automatically reject any email addressed to invalid addresses. This will reduce the load on your exchange server, since it no longer has to process non-delivery reports, and it will reduce the load on your postfix server since it won't have to perform spam and virus scanning on the message.

12.1 Install Dependencies

Install the perl module Net::LDAP:

perl -MCPAN -e shell
install Net::LDAP

12.2 Create the Get Email Address Script

Create and edit the script:

vi /usr/bin/getadsmtp.pl

Copy and paste the code below into this new file.

#!/usr/bin/perl -T -w

# This script will pull all users' SMTP addresses from your Active Directory
# (including primary and secondary email addresses) and list them in the
# format "user@example.com OK" which Postfix uses with relay_recipient_maps.
# Be sure to double-check the path to perl above.

# This requires Net::LDAP to be installed.  To install Net::LDAP, at a shell
# type "perl -MCPAN -e shell" and then "install Net::LDAP"

use Net::LDAP;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant ( "LDAP_CONTROL_PAGED" );

# Enter the path/file for the output
$VALID = "/etc/postfix/relay_recipients";
open VALID, ">$VALID" or die "CANNOT OPEN $VALID $!";

# Enter the FQDN of your Active Directory domain controllers below
$dc1="domaincontroller1.example.com";
$dc2="domaincontroller2.example.com";

# Enter the LDAP container for your userbase.
# The syntax is CN=Users,dc=example,dc=com
# This can be found by installing the Windows 2000 Support Tools
# then running ADSI Edit.
# In ADSI Edit, expand the "Domain NC [domaincontroller1.example.com]" &
# you will see, for example, DC=example,DC=com (this is your base).
# The Users Container will be specified in the right pane as
# CN=Users depending on your schema (this is your container).
# You can double-check this by clicking "Properties" of your user
# folder in ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=Users,DC=example,DC=com
# which would be $hqbase="cn=Users,dc=example,dc=com"
# Note:  You can also use just $hqbase="dc=example,dc=com"
$hqbase="cn=Users,dc=example,dc=com";

# Enter the username & password for a valid user in your Active Directory
# with username in the form cn=username,cn=Users,dc=example,dc=com
# Make sure the user's password does not expire.  Note that this user
# does not require any special privileges.
# You can double-check this by clicking "Properties" of your user in
# ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=user,CN=Users,DC=example,DC=com
# which would be $user="cn=user,cn=Users,dc=example,dc=com"
# Note: You can also use the UPN login: "user\@example.com"
$user="cn=user,cn=Users,dc=example,dc=com";
$passwd="password";

# Connecting to Active Directory domain controllers
$noldapserver=0;
$ldap = Net::LDAP->new($dc1) or
$noldapserver=1;
if ($noldapserver == 1)  {
$ldap = Net::LDAP->new($dc2) or
   die "Error connecting to specified domain controllers $@ \n";
}

$mesg = $ldap->bind ( dn => $user,
                   password =>$passwd);
if ( $mesg->code()) {
 die ("error:", $mesg->error_text((),"\n"));
}

# How many LDAP query results to grab for each paged round
# Set to under 1000 for Active Directory
$page = Net::LDAP::Control::Paged->new( size => 990 );

@args = ( base     => $hqbase,
# Play around with this to grab objects such as Contacts, Public Folders, etc.
# A minimal filter for just users with email would be:
# filter => "(&(sAMAccountName=*)(mail=*))"
      filter => "(& (mailnickname=*) (| (&(objectCategory=person)
                 (objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))
                 (&(objectCategory=person)(objectClass=user)(|(homeMDB=*)
                 (msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact))
                 (objectCategory=group)(objectCategory=publicFolder) ))",
       control  => [ $page ],
       attrs  => "proxyAddresses",
);

my $cookie;
while(1) {
# Perform search
my $mesg = $ldap->search( @args );

# Filtering results for proxyAddresses attributes
foreach my $entry ( $mesg->entries ) {
 my $name = $entry->get_value( "cn" );
 # LDAP Attributes are multi-valued, so we have to print each one.
 foreach my $mail ( $entry->get_value( "proxyAddresses" ) ) {
  # Test if the Line starts with one of the following lines:
  # proxyAddresses: [smtp|SMTP]:
  # and also discard this starting string, so that $mail is only the
  # address without any other characters...
  if ( $mail =~ s/^(smtp|SMTP)://gs ) {
    print VALID $mail." OK\n";
  }
 }
}

# Only continue on LDAP_SUCCESS
$mesg->code and last;

# Get cookie from paged control
my($resp)  = $mesg->control( LDAP_CONTROL_PAGED ) or last;
$cookie    = $resp->cookie or last;

# Set cookie in paged control
$page->cookie($cookie);
}

if ($cookie) {
# We had an abnormal exit, so let the server know we do not want any more
$page->cookie($cookie);
$page->size(0);
$ldap->search( @args );
# Also would be a good idea to die unhappily and inform OP at this point
  die("LDAP query unsuccessful");
}
# Add additional restrictions, users, etc. to the output file below.
#print VALID "user\@domain1.com OK\n";
#print VALID "user\@domain2.com 550 User unknown.\n";
#print VALID "domain3.com 550 User does not exist.\n";

close VALID;

Next set the permissions on the file to allow it to be executed:

chmod 500 /usr/bin/getadsmtp.pl

Edit the file to customize it for your specific domain. Since the file is read only, you will need to use :w! to save the file in vi.

1. Set $dc1 and $dc2 to the fully qualified domain names or IP addresses of 2 of your domain controllers.

2. Set $hqbase equal to the LDAP path to the container or organizational unit which holds the email accounts for which you wish to get the email addresses.

3. Set $user and $passwd to indicate which user account should be used to access this information. This account only needs to be a member of the domain, so it would be a good idea to setup an account specifically for this.

12.3 Run the Script

Try running the script. If it works correctly, it will create /etc/postfix/relay_recipients Note that if your postfix server is separated from your active directory controllers by a firewall, you will need to open TCP port 389 from the postfix server to the ADCs. At this point, you can update your /etc/postfix/main.cf to relay_recipient_maps. You will also have to postmap the file to create the database.

getadsmtp.pl

At this point, you may want to edit /etc/postfix/relay_recipients and edit out any unwanted email addresses as this script imports everything.

12.4 Create the Table

postmap /etc/postfix/relay_recipients

Finally, you may want to set up a cron job to periodically update and build the /etc/postfix/relay_recipients.db file. You can set up a script called /usr/bin/update-relay-recipients.sh: (Optional)

vi /usr/bin/update-relay-recipients.sh

#!/bin/sh

/usr/bin/getadsmtp.pl
cd /etc/postfix
postmap relay_recipients

Don't forget to make sure the following is in your /etc/postfix/main.cf file:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Run crontab to add this script to the scheduled jobs:

crontab -e

Now add the following lines to the bottom of the file. Note that this cron job will run every day at 2:30 AM to update the database file. You may want to run yours more frequently or not depending on how often you add new email users to your system.

# syncronize relay_recipients with Active Directory addresses
30 2 * * * /usr/bin/update-relay-recipients.sh