...making Linux just a little more fun!

An Anti-Spam Solution for Your Home or Office Network By Sandro Mangovski

An Anti-Spam Solution for Your Home or Office Network

So here is my work, step-by-step so you can follow it. I hope it will help you to build your own anti-spam system, or just give you the guidelines of how to make your own idea come to life.

When I started working on this problem I decided to build a solution from scratch and reinstall or replace most of software which I had been using before. So I picked the new tools: Exim MTA, SpamAssassin, Anomy mail sanitizer, and vm-pop3d. I downloaded all the software and started building it, which is your first step also.

Step 1.

Exim MTA: You download Exim .tar.gz package from www.exim.org and unpack it to some directory. At the time of writing this document, the latest Exim version is 4.24. Cd to the directory which has been created (for example, exim-4.24). Next, you copy the file src/EDITME to Local/Makefile, but before that you need to fill in some information to src/EDITME. The following is the least you should set up:

BIN_DIRECTORY=/usr/exim/bin
CONFIGURE_FILE=/usr/exim/configure
EXIM_USER=eximusr

I created this user just for Exim and I suggest you do this also. Of course, the username doesn't have to be the same.

I also set up FIXED_NEVER_USERS=root for security precautions. This file is also very well commented, so if you need some other options, it is not hard to find out how to set it up; but this configuration should do just fine for your office network.

If you are going to build the Exim monitor, a similar configuration process is required. The file exim_monitor/EDITME must be edited appropriately for your installation and saved under the name Local/eximon.conf.

If you are happy with the default settings described in exim_monitor/EDITME, then Local/eximon.conf can be empty, but it must exist.

After the preinstall configuration you'll need to compile the software. Commands make and make install should do the trick. After this, you do some post install configuration and you'll be almost done. Open the file /usr/exim/configure with your favorite editor and change

domainlist local_domains = @

to

 domainlist local_domains = @ : localhost : foo.bar

or some other domain you want exim to deliver to locally. Configuring Exim with virtual domains is beyond the scope of this document, but still we'll touch on it at the end of this document. What you need do next is to get Exim to run whenever the computer boots up. I prefer doing this from inetd. To do this, add the following line to /etc/inetd.conf:

smtp stream tcp nowait eximusr /usr/sbin/tcpd /usr/exim/bin/exim -bs

Here eximusr is the user which was set in the EXIM_USER variable in src/EDITME

Now restart inetd and telnet to your machine on port 25. You should get a line like this:

220 enterprise ESMTP Exim 4.24 Fri, 28 Nov 2003 20:03:32 +0100

which indicates that all went Ok :) This is all you have to do with exim for now.

Anomy mail sanitizer: Unpack the .tar.gz package from mailtools.anomy.net in some directory (e.g. /usr/local is mine) and cd to anomy/bin. Then run ./sanitizer.pl. You will probably get some error message but that is ok. Probably the error message will be something about missing a Perl module, and we will come back to it later. If you do get any errors, leave it like that for the moment and read on.

SpamAssassin: Unpack the .tar.gz package and cd to the newly created directory. There are two ways of installation. The first one is shorter and saves you some minor difficulties. Type following set of commands in the shell as root:

perl -MCPAN -e shell
o conf prerequisites_policy ask
install Mail::SpamAssasin
quit

This way installs SpamAssasin using module CPAN. If this is your first time using that module, after the first command you will be asked a series of questions for CPAN configuration.

This is the second way:

perl Makefile.PL
make
make install

When you run spamc or spamd you may get the same problem as with anomy. Don't worry, just read on. Now we will explain the missing module error. If your error message is something like missing module, for example HTML/Test.pm, then install it using MCPAN; or go to the www.cpan.org and search for the module HTML::Test and download it. If you need to install the Perl module, it is not very difficult. Unpack the .tar.gz package and cd to the newly created directory. Type the following set of commands as root:

perl Makefile.PL
make
make test
make install

Now since you know how to install Perl modules, you can also fix that anomy missing modules errors. When installing modules you may have same error as with SpamAssasin or anomy because Perl modules may need other modules to work. So once again go to www.cpan.org and start over until you resolve all requirements. For example, on my system I needed to install the following modules for both anomy and SpamAssasin: HTML::Parser, HTML::Tree, HTML::Tagset, HTML::Sanitizer, MIME::Base64, Test::Harness, Test::Simple, Digest::MD5 and Devel::CoreStack. Unfortunately, I don't remember which modules were prerequisites of the others, so you will have to read README files or get error messages until everything is installed.

There is one more thing to do with SpamAssasin. Since spamc is just client for spamd we need to make sure that spamd is running when mail is passed through spamc. Just add spamd --daemonize to your init script.

vm-pop3d: Unpack .tar.gz source from www.reedmedia.net and cd into the newly created directory (Do you see a pattern here?).Type following commands as root:

./configure
make
make install

Now we have to make vm-pop3d running when computer starts.Add this line to your /etc/inetd.conf file:

pop3 stream tcp nowait root /usr/sbin/tcpd /usr/local/sbin/vm-pop3d

Restart inetd and telnet to localhost on port 110. A line similar to this:

+OK POP3 Welcome to vm-pop3d 1.1.6 <14665.1070049711@enterprise> means you are at the end of Step 1.

If you had any problems with building which I didn't describe browse through the documentation or ask on Usenet or try to figure a way yourself like I had to do in the following step, which is configuring of all the sotfware we built.

Step 2.

A little introduction and a small request from me. Read the Exim documentation parts about mail delivery, routers, transports just to have some background before we start working. I'm still going to write something about it here. So, when Exim receives mail it goes from router to router until it is accepted. When a message is accepted, the router calls its transport to handle the message. If the message isn't delivered after it went through transport, it goes once again through more routers until it is accepted and delivered, or until an “undeliverable” error message is generated. That is the shortened version of the story. So, if you read carefully you might have concluded that the order of the transports which are listed in the configuration file of exim is irrelevant, but the order of routers is important.

Now you have to get your hands dirty. Open /usr/exim/configure with your favorite editor and add this in routers section before routers who handle local delivery (after the dnslookup router).

# MAIL SCAN ROUTER
mail_scan_router:
no_verify
check_local_user
condition = "${if !eq{$header_X-I-came-to:}{scary devil's monastery}{1}}"
driver = accept
transport = mail_scan_transport

This router will be run only if message doesn't contain X-I-came-to:scary devil's monastery header, or by other words only when it first arrives. With that condition we disabled the router loop that would have been created without it. Now we have to add the transport which this router calls when the condition is met. So add this anywhere in the transports section (remember, the order of transports is irrelevant).

#MAIL SCAN TRANSPORT
mail_scan_transport:
driver = pipe
command = /usr/exim/bin/exim -bS
use_bsmtp = true
transport_filter = /usr/exim/mail.sh
home_directory = "/tmp"
current_directory = "/tmp"
user = mail
group = mail
log_output = true
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =
headers_add = X-I-came-to: scary devil's monastery

This transport handles the message through transport filter and adds an X header, which in combination with a condition in the router disables any infinite filtering/spamchecking loops.

Now let's write mail.sh. This is script which enables us to run both anomy sanitizer and SpamAssasin within single transport. It goes like this:

#!/bin/bash
cd /usr/local/anomy/bin
./sanitizer.pl | /usr/local/bin/spamc
cd /tmp
#end of script.

Maybe all this cd-ing seems strange, but I had some errors when running anomy from outside of its directory. Anyway, copy this code into a file. Save it in /usr/exim and make sure it has permissions which enable user mail to run it. For example my permissions look like this:

-rwxr-xr-x root root.

Now a little bit more on the Exim configuration. When SpamAssassin scans a message it adds X-Spam-Status header to it. We will use that to check whether the message is spam and to decide where it should be delivered to. Add this just before local_delivery router (remember that routers order is relevant)in your Exim configuration file.

#SPAM DELIVERY ROUTER
spam_delivery_router:
condition="${if eq {${length_3:$header_X-Spam-Status:}}{Yes}{1}}"
check_local_user
driver=accept
transport = spam_delivery

So, if first 3 characters of X-Spam-Status: header are Yes that means that the message is spam and we will use spam_delivery transport. Otherwise message goes to normal local delivery. Now, add this to transports section of configuration file:

spam_delivery:
driver = appendfile
create_directory
file=/var/spool/virtual/spam.foo/$local_part
delivery_date_add
envelope_to_add
return_path_add
group = mail
mode = 0660

This means that, for example, messages for local user sandro ( sandro@localhost) are delivered to /var/spool/virtual/spam.foo/sandro. Make sure that directories virtual and spam.foo look like this when you do ls -l in their parent directories:

drwxrwsrwx 3 mail mail 4096 Stu 27 19:05 virtual
drwxrwxrwx 2 mail mail 4096 Stu 28 21:08 spam.foo

Of course I don't need to remind you to restart inetd after these changes.

Now you can see what these two dirs are for. They are for delivering spam to local users. Yes, we will create virtual domain spam.foo for vm-pop3d (pop3 daemon if you haven't got it by now) so our users will be able to read their spam. Why, you ask? Because in my case many users complain about missing newsletters, commercials etc. Mail in /var/spool/virtual/spam.foo in my case gets deleted weekly (a simple script in cron.weekly) because system's resources are limited and we wouldn't want to waste them on spam more than we have to, do we?

Ok, now to configure vm-pop3d. We don't need to do anything for local users' “real” mails, but we need to do for spam. Each local user will get an account for spam.foo virtual domain. MUA configuration will then be slightly different than for the “real” mailbox. For example if a user has the local username vms then his username for spam mailbox is vms@spam.foo or vms:spam.foo. Of course passwords don't have to be the same for these two mailboxes. Note that here we have a similar “as it can be” concept like Yahoo(tm) Bulk mail folder

Now, let's create that spam accounts. Create directory /etc/virtual which needs to look like this:

drwxr-xr-x 3 root root 4096 Stu 25 21:22 virtual.

It is not critical that the permissions are exactly the same as here, but vm-pop3d must be able to read the directory. So, if you don't like these, play with them and come up with some other combination. I just say this because I don't want you to get the impression that my way is the only right way.

Now under that directory create directory spam.foo with same permissions. That directory will contain passwd file for our virtual domain. We will create that file with a Perl script which i got from vm-pop3d authors' website. Here is that script:

#!/usr/bin/perl
$name = $ARGV[0];
@salt_chars = ('a'..'z','A'..'Z','0'..'9');
$salt = $salt_chars[rand(62)] . $salt_chars[rand(62)];
$passwd = crypt ($ARGV[1], $salt);
print "$name:$passwd\n";

Now save this script in a file and make sure it is executable by root, or whoever is mail admin.

Script is used in a way ./script_file_name username password >> /path_to_passwd/passwd. For example:

enterprise:/etc/virtual# pop3passwd mosor uncrackable >> spam.foo/passwd

Now you can add a username for spam.foo domain for all your local users, and the vm-pop3d configuration is finished.

What comes next is some SpamAssasin fine tuning. Open /etc/mail/spamassassin/local.cf and change report_safe 1 to report_safe 0.

If this option is set to 1, if an incoming message is tagged as spam, instead of modifying the original message, SpamAssassin will create a new report message and attach the original message as a message/rfc822 MIME part (ensuring the original message is completely preserved, not easily opened, and easier to recover. If this option is set to 2, then original messages will be attached with a content type of text/plain instead of message/rfc822. This setting may be required for safety reasons on certain broken mail clients that automatically load attachments without any action by the user. This setting may also make it somewhat more difficult to extract or view the original message.If this option is set to 0, incoming spam is only modified by adding some "X-Spam-" headers.

Another thing which is important when working with spam is efficient learning technique. Man page is the best resource for SpamAssasin learning strategy (I urge you to read it, and all the references inside), but it all comes down to this: the more the filter learns, the better it gets. The technical side of learning is this:

sa-learn --spam path_to/message_file

or

sa-learn --ham path_to/message_file.

Of course --spam is for spam mails and --ham for mails which are not spam. It is equally important to let filter learn both spam and ham mails.

And that is it. You have a working anti-spam system, congratulations.

There are a few more things. If you don't have a permanent Internet connection, fetchmail is the way you'll retrieve your mails. So let's configure fetchmail. Unpack .tar.gz package and cd to the newly created directory (Now I am even boring myself). Type as root:

./configure
make
make install

Now you need to configure fetching mail for the users. Each user needs to have .fetchmailrc file in his home directory. A simple .fetchmailrc file looks like this:

poll pop.iskon.hr proto pop3 username "amangovs" password "Unkrakkable" is "sandro" here nokeep expunge 3

Fetchmail's configure file format is excellently explained in man pages, so I strongly advise you to read them.

Now for little more fine tuning. We need to go to the /etc/aliases file and set some aliases. The most important alias is postmaster which you must set, but I advise you to spend a minute more and set them all.

Since receiving mail has by now been fully covered, now here is something about sending it. You might know it already but I'll still mention it. If your network has a permanent connection to internet and you have a fully qualified domain name (this could be dynamic DNS, too), with an MX DNS entry your users can use your mail server as their smtp server. Otherwise (if your mail is retrieved via fetchmail), users should put their provider's smtp server in their MUA (mail user agent).

Now as I promised here is a little something about virtual domains in Exim.

local_delivery:
driver = appendfile
file = /var/spool/virtual/$domain/$local_part
delivery_date_add
envelope_to_add
return_path_add
group = mail
mode = 0660

If you change your local_delivery transport in Exim configuration file to this Exim would deliver message for some_username@some_domain.com to /var/spool/virtual/some_domain.com/some_username. Now what you have to do more is add some_domain.com in domainlist local_domains = @ : localhost : foo.bar: domain1.com : domain2.com : some_domain.com (remember this?) and configure vm-pop3d just as you configured it for spam.foo domain. Please, remember to check permissions, it will save you some time.

That's all folks! It is a pleasure writing for you :)

Sandro Mangovski.

None provided.