Background
Postfix ( http://www.postfix.org/ ) is the popular Sendmail replacement that is fast, secure, and easy to administer.
Postfix also makes an outstanding email security gateway or "mail firewall", shielding fragile Intranet mail systems such as WOE (Windows/Outlook/Exchange) or Lotus Notes from direct contact with the Internet.
Postfix's functionality can be extended by filtering mail through external programs. By using external filters, Postfix can do content filtering including scanning for viruses and MS Outlook exploits, and identifying spam.
This paper demonstrates how to use the filtering capabilities of Postfix to perform high-level content filtering including:
* Dropping potentially hostile attachments such as .EXE and .VBS files
* Preventing MS Outlook from auto-executing unknown attachments
* Preventing exploits embedded in HTML email and MIME headers
* Disabling "web bugs" that spammers use to track recipients
* Identifying spam based on message content
The method shown here is multi-purpose: it is suitable for local mail (where the email is stored on the Postfix server itself) and also suitable for when Postfix is used as a "mail firewall" to relay mail to an internal mail system. Unlike endpoint solutions like Outlook plugins that must be installed and managed on each workstation. This method also has the advantage of applying one spam and content filtering policy for all users. Users can also share one auto-whitelist database, reducing the chance that legitimate mail will be labelled as spam.
However, if you require unique settings for each user it may be better to filter mail via a mail delivery agent such as procmail or maildrop (http://www.courier-mta.org/maildrop/) instead of using the method shown here.
The software
We extend the capabilities of Postfix using two best-of-breed open source programs that perform content filtering and spam identification:
Anomy Sanitizer
Anomy Sanitizer ( http://mailtools.anomy.net/ ) is an open source email content filter that disables all the hostile content of email messages that anti-virus programs ignore: potential buffer overflows, MIME exploits, web bugs, dangerous HTML tags, and other problems that have made MS Outlook such a security nightmare.
Anomy Sanitizer is written in Perl. It filters SMTP messages checking for most known exploits and hostile file attachments. It can remove attachments, rename unknown file types, "defang" exploitable HTML tags, fix MIME headers and generally cleans up malformed email.
Cleaning up malformed email is essential: virus scanners have a history of missing attachments in malformed email. Attackers regularly alter MIME structures in email so malicious content becomes invisible to antivirus but still works when the user opens it in the email client
SpamAssassin
SpamAssassin ( http://spamassassin.apache.org/ ) is a mail filter that attempts to identify junk e-mail ("spam"). It examines the content of mail messages looking for key phrases and other identifiers commonly used by spammers.
SpamAssassin uses a scoring system: messages are tagged as spam only when they have enough spam characteristics in total. When combined with other features this results in very few false positives. In our experience, a properly managed SpamAssassin installation correctly identifies 95% to 98% of spam with less than 1% false positives.
SpamAssassin doesn't block spam. Instead, it tags messages as probable spam by changing the message headers and (optionally) the Subject line. This is wise: no automated system can recognize spam with complete certainty... deciding "what is spam" is a judgement call. All automated spam filters will produce some false positives (wanted e-mail mistakenly tagged as spam) and false negatives (spam not identified as such).
SpamAssassin identifies messages that are probable spam, but leaves the choice of what to do with it up to you. Most organizations instruct their users how to add rules to their e-mail software to delete identified messages or, better yet, move them to a folder for later review. Others choose to use an additional filter automatically delete extremely high scoring messages or put them in a holding area for further review by an administrator.
In our opinion, the best approach is to tag low scoring messages as possible spam and deliver it to the user so they can make the final call. For high scoring spam, sideline it for review by an administrator or automatically delete it.
The method shown here only tags suspected spam. No automated deletion or sidelining is performed. However, we have a collection of alternative filter script that will sideline or delete suspected spam available at http://advosys.ca/examples/filter-misc/.
Preparing the server
Prerequisites
Before you begin, here is what you will need:
1. A Unix-type server such as Linux or BSD. The method shown here has been used on Solaris, Debian Linux, Ubuntu Linux, RedHat Linux and FreeBSD.
2. Postfix version 2.0.1 or higher (type postconf mail_version at the Unix command line to see the version number). This method does not work with Postfix 1.1.x.
3. At least Perl 5.005_03 or higher. Type perl -v at the command line to see the version number. Perl 5.6 or higher is recommended, but it is best to stick with the version of Perl your system provides by default.
NOTE: Your server must already have Postfix installed and working. Please do not try to follow these instructions until you first have a basic installation of Postfix working.
Creating a filter account
To use external filtering with Postfix, first add a new a Unix group on the server named "filter" (use groupadd or addgroup as appropriate to your system).
Next, add a new user account named "filter" on the server and make it a member of group "filter". Set the account's home directory to /var/spool/filter. This will be the account used by the filtering scripts.
No other user should belong to group "filter". For now, give the filter account a password and a valid login shell. You should be able to use the "su" command to login as user filter. This will make troubleshooting easier. Don't worry... later we will disable logins for the account to improve security.
Installing required Perl modules
Also, download and install the following Perl modules from CPAN ( http://search.cpan.org/ ):
* MIME::Base64
* MIME::QuotedPrint
* Net::DNS
* DB_File
If you've set up the CPAN module in your Perl installation, the easiest way to add these modules is to use CPAN to download, build and install automatically:
perl -MCPAN -e shell
o conf prerequisites_policy ask
install MIME::Base64
install MIME::QuotedPrint
install Net::DNS
install DB_File
quit
Installing Anomy Sanitizer
Download the latest version of Anomy Sanitizer from http://mailtools.anomy.net/
Install it by unpacking the tar file into a suitable directory on your mail server. We recommend using directory /usr/local/anomy but any other directory will work. After unpacking, it's a good idea to read the instructions and notes in file sanitizer.html in the anomy directory.
Anomy configuration
In directory /usr/local/anomy, create a file named anomy.conf and add your anomy configuration rules. The specific settings you choose depends on your particular e-mail policies and the type of internal e-mail software you are protecting. The "real world configuration" shown in the Anomy documentation is a good starting point.
Below is a configuration we've used on mail gateways to protect Windows-Outlook-Exchange users. It defangs HTML and MIME exploits, plus drops all executable attachments (we strongly recommend dropping executable attachments as a general policy: see our paper "E-mail policies that prevent viruses" at http://advosys.ca/papers/mail-policies.html).
Download: anomy.conf
Once the config file is in place, change ownership of the entire anomy directory tree to owner "root", group "filter". For example:
chown -R root:filter /usr/local/anomy
Change permissions on the anomy directory tree so it is not world readable. For example:
chmod 0750 /usr/local/anomy
Installing SpamAssassin
Some Linux distributions provide packaged versions of SpamAssassin and Anomy Sanitizer as RPM or DEB files. Usually these packaged versions are out of date, move files from the default locations and make other subtle changes. To avoid problems, we recommend removing packaged versions of SpamAssassin and installing manually as described below.
Refer to your system documentation for details on how to use its package manager (e.g.: on Redhat Linux, the command is rpm. On Solaris, use pkginfo).
The SpamAssassin documentation describes how to install and configure that software. As described above, if you have the CPAN module set up on your server, it can download and install SpamAssassin automatically. For example:
perl -MCPAN -e shell
o conf prerequisites_policy ask
install Mail::SpamAssassin
quit
SpamAssassin configuration
SpamAssassin is installed "out of the box" with a good set of spam identification rules. You can specify your own settings in file /etc/mail/spamassassin/local.cf.
It's a good idea to leave local.cf unchanged until you get things working and have a feel for how the default rules work. Later on you can fine tune SpamAssassin settings in local.cf as required.
However, we recommend making one change to local.cf right away: Whitelist well-known senders so their mail will never be identified as spam:
Manual whitelists
You should whitelist the e-mail addresses of well-known legitimate senders to avoid the chance of them being misidentified by the SpamAssassin default rules. Add "whitelist_from" settings to file /etc/mail/spamassassin/local.cf for each important client, mailing list and other known spam free senders. For example:
whitelist_from director_8345@ hotmail.com # whitelist one sender
whitelist_from *@advosys.ca # whitelist entire domain
whitelist_from *@securityfocus.com
Configuring filtering in Postfix
You must have Postfix installed and accepting mail the way you want. To preserve your mental health, do not attempt to add filtering and get a first-time installation of Postfix working at the same time. Many have tried, all have failed. Be wise: install and configure Postfix first, and verify it is working the way you want (accepting mail for your domain(s), relaying to and from internal servers, and so on). Getting Postfix running the first time is complex enough without also trying to add filtering.
The After queue content filter README (http://www.postfix.org/FILTER_README.html) in the Postfix documentation describes ways to add external content filtering to Postfix. Here we use a variation of the "Simple content filtering" method described in that file: plug a shell script into Postfix by modifying the master.cf file.
Creating the filter script
The following method pipes each message sent to it through Anomy Sanitizer, then through SpamAssassin. The command-line "sendmail" compatibility command that is part of Postfix is then used to queue the result for final delivery.
Note: Modify the file locations specified in this file to match the locations on your server. File locations vary depending on your flavour of Unix/Linux/BSD and other factors.
Download: filter.sh
Place the filter.sh script into the same directory you installed Anomy Sanitizer (e.g.. /usr/local/anomy ). The script should have the following permissions:
-rwxr-x--- owner=root group=filter
Create a temporary directory
The script needs a directory to store temporary files. Create a directory that is writable by group "filter". For example:
mkdir /var/spool/filter
chown root:filter /var/spool/filter
chmod 0770 /var/spool/filter
The directory should be located on a partition large enough to store a few incoming mail messages. On most systems, the /var partition is the best choice. Whatever directory you choose, change the INSPECT_DIR setting in filter.sh to match.
Send a test message
Before activating all this in Postfix, it's wise to first try sending a few test messages from the command line.
As root, use the "su" command to change to user filter. For example:
su filter
Change to the /var/spool/filter directory if not there already. Use your favorite text editor to create a file named "test.txt" containing the following:
From:
To:
Hi there. This is a test message.
(The blank line between the "To:" and "Hi there" is required).
Send the above test message through the filter script as follows:
cat test.txt | /usr/local/anomy/filter.sh -f tester -- root
If things are working, no error messages should be printed and the shell prompt should return. Check the mail for account "root": you should have received the test message.
SpamAssassin may emit a warning that it "cannot create user_prefs". Ignore this warning.
If you see error messages, such as "cannot execute..." or "cannot find...", or the message arrives but is blank, double check what you just typed and try again. Depending on your system, the error messages should lead you to where the error is... in the anomy program, the filter.sh shell script, or file permissions. Incorrect file and directory permissions or wrong pathnames in the filter script are the most frequent cause of problems... check everything twice!
The Anomy filter can be debugged by temporarily changing the value of $ANOMY_LOG in filter.sh from /dev/null to an actual file, such as /tmp/anomy.log (make sure the user filter has write access to that file), then set feat_log_inline = 1 in file anomy.conf. Remember to change the settings back to the original when you are finished troubleshooting.
Resolve all the errors before proceeding... if you can't send mail manually using this method, Postfix won't be able to send any either.
Creating a recipient map
We need to create a recipient access map so Postfix will know which e-mail domains to filter. Without this map, Postfix will not filter any e-mail, inbound or outbound, through SpamAssassin and Anomy.
Change to directory /etc/postfix and create a new file named filtered_domains. Add something like to following to the file:
# Filter only mail addressed to local domains:
example.com FILTER filter:dummy
example.net FILTER filter:dummy
otherdomain.ca FILTER filter:dummy
Of course, replace the above domains with your own e-mail domain names. You need one line for each domain you receive mail for.
If you have a domain you receive mail for but don't want filtered, simply don't list it in the file: Postfix will still accept mail for the domain and deliver it, unfiltered.
Save the file and use command postmap filtered_domains to create the corresponding data file.
Changing main.cf
In file /etc/postfix/main.cf look for the setting smtpd_recipient_restrictions
That setting may not be present or may be commented out, depending on your particular installation of Postfix. Add the setting or if it already exists, edit it similar to the following:
smtpd_recipient_restrictions =
check_recipient_access $default_database_type:/etc/postfix/filtered_domains
...existing settings, if any...
reject_unauth_destination
* If there is already a check_recipient_access setting present, do not replace it... add the above as a second setting, but place it after any other check_recipient_access settings.
Here is an example of what the smtpd_recipient_restrictions setting could look like it it's entirety. Your setting does not have to be the same... this is just an example:
smtpd_recipient_restrictions =
reject_non_fqdn_sender
reject_non_fqdn_recipient
permit_mynetworks
reject_unknown_sender_domain
check_recipient_access $default_database_type:/etc/postfix/recipient_access
check_recipient_access $default_database_type:/etc/postfix/filtered_domains
check_sender_access $default_database_type:/etc/postfix/sender_access
reject_unknown_recipient_domain
reject_rbl_client zen.spamhaus.org
reject_unauth_destination
Save the changes to main.cf and proceed to the next step.
Changing master.cf
Add the following to the bottom of file /etc/postfix/master.cf:
filter unix - n n - - pipe
flags=Rq user=filter argv=/usr/local/anomy/filter.sh -f ${sender} -- ${recipient}
Save the changes to master.cf and use command postfix reload to tell Postfix to re-read the configuration files.
Testing it out
Postfix should now be filtering messages it receives for your local domains through both Anomy Sanitizer and Spamassassin.
Test it by sending a few messages with MIME attachments through Postfix addressed to recipients in your domain. The Postfix log file should show each the message being received twice: once by the SMTP daemon with "recipient address", then again by the command-line "sendmail" compatibility program.
Each message should also now have SpamAssassin headers such as X-Spam-Status:.
Tightening security
Now that things are working, disable interactive logins for user "filter". Allowing logins for accounts used only by scripts and daemons is a dangerous security hole.
To disable logins, change the shell for user "filter" to /bin/false or some other invalid shell (use the chsh or usermod commands, or edit /etc/passwd). On Linux and Solaris, you can also "lock" the account using "passwd -l filter".
Separating inbound mail from outbound
The configuration described above filters messages based on the recipient domain in each e-mail message. This prevents outbound mail from being filtered through Anomy and SpamAssassin.
Local e-mail (e.g.. someone in "example.com" sending to another person with an "example.com" address) will also not be filtered, provided two conditions are met:
1. Your smtpd_recipient_restrictions setting lists permit_mynetworks before it lists check_recipient_access (the restrictions list shown above illustrates this).
2. The sender's IP address is within the range specified by the $mynetworks setting in your main.cf file.
If your organization has remote users who send mail to your domain from outside your $mynetworks range (e.g.. laptop travelers using an ISP dialup, home telecommuters, etc.), mail from them will be filtered.
To prevent this, you might be tempted to just expand your $mynetworks range to include the external ISPs. However, that makes your mail server an open relay... allowing spammers on those ISPs to freely abuse your mail server.
A better method is to using SASL authentication: external users require a username and password to send mail through your server. The Postfix documentation page ( http://www.postfix.org/docs.html ) lists several HOWTOs on adding SASL authentication to Postfix. Most versions Linux that include Postfix provide additional packages to enable SASL.
Once SASL is enabled, list permit_sasl_authenticated right after permit_mynetworks in smtpd_recipient_restrictions and senders who use a username and password to connection to the server will be able to bypass the content filtering and mail relay controls.
A word about performance
The method shown here is an easy and reliable way to filter messages with Postfix. However, performance suffers because each e-mail message has the overhead of invoking a shell, starting the Perl interpreter, and creating a temporary file.
The file creation overhead can be greatly reduced by mounting directory /var/spool/filter as a memory filesystem ("tmpfs" in Linux and Solaris). These filesystems are thousands of times faster than physical disk and are ideal for short-lived temp files.
SpamAssassin performance can be improved by using spamd, the "daemonized" version of that program ( http://svn.apache.org/repos/asf/spamassassin/branches/3.2/spamd/README ). See our collection of alternative scripts, below, for a spamd version of filter.sh.
Ideally the Anomy program should also have a daemon version, but no such version exists yet. Tricks like compiling sanitizer.pl using the perlcc compiler in Perl 5.6+, or using Persistent Perl ( http://daemoninc.com/PersistentPerl/ ) helps a little.
However, even without tmpfs or daemon versions of the programs, the method described above has been able to filter upwards of 50,000 messages per day on one dual-processor Sun e250 server with 512MB memory. A Linux implementation on a single-cpu 750Mhz Pentium II platform with 512MB demonstrated similar performance levels.
Other filtering scripts
It's easy to change the filter.sh shell script to suit your own needs. Each email message is a standard text file so basic shell programming or Perl can be used to process it as you wish.
We've written a few alternative filtering scripts and made them available here:
http://advosys.ca/examples/filter-misc/
The scripts are fully functional and demonstrate ways to extend the functionality of Postfix filtering. Feel free to experiment with the scripts and adapt them for your own use.
0 comments: