UC San Diego SearchMenu

Using the Mail Filter Utility

The Mail Filter Utility may be used for many different purposes, including managing spam emails and even settin up auto-replyers!

Overview of spam

Most people are used to getting junk mail delivered to their front door by the post office. So, most people were not surprised when junk email, spam, started getting delivered to their electronic mailboxes. However, unlike the paper products that cost money to create and deliver, spam is almost free to send, and has proliferated to the point of forcing some users to abandon their email addresses and seek unlisted, unknown addresses. Whenever you give out your email address on the Internet—sign up for a newsletter, enter a contest, buy from an online store, display your address on a web page—your address gets picked up by advertisers seeking to make a profit. At some point, some programmers decided enough was enough, and they started to work on solutions to the spam problem.

At UC San Diego, we use Spam Assasin, a program created for everyone to use for free. Our Mail Filter Utility harnesses the power of Spam Assassin, alongside another powerful and flexible tool for manipulating messages called Procmail, without forcing you to learn the ambiguities required to configure them by hand.

Most of this article is aimed at people using the advanced options of the Mail Filter Utility. Those who wish to enable simple spam filtering need only log in and press a button, without worrying about "the man behind the curtain."

Definitions and ideas you need to know

When a message is received from off campus for a UC San Diego email address, it first passes through a central mail server before being passed on to its final destination. One of the things this central mail server does is pass the message to Spam Assassin for spam grading, regardless of whether or not you are using Mail Filter Utility. Spam Assassin assigns a numerical grade, which is based on how many spam-like features the program found in the message. The message is then forwarded to its final destination, either an account on a UC San Diego-managed mail server, or an off-campus email address.

Since this utility works with ACMS mail servers, if you have mail forwarded to a department mail server or an off-campus mail server, this utility will not affect how you receive your messages. If your mail is not forwarded to another address, then before it is delivered to your mail inbox, your mail server will first check for a set of rules about how messages should be handled. This set of rules is what you are editing with this utility.

The set of rules is made up of what are called recipes. When a message is received, the mail server will look at each recipe in the order that it is listed in a file on your mail server. This utility generates and edits that file, allowing you to create new recipes and change their order within the recipe list.

Some information you need to know about mail folders

Both of the advanced options, the spam filter and the simple mail filters, and many of the pre-written recipes, give you the option to have mail moved to a mail folder separate from the mail inbox where mail is delivered by default. If you chose this option, the new mail folder will appear in your list of folders on Webmail the first time a message is delivered to that folder. In addition, if you use the Unix-based program Pine, a file will be created where that program looks for folders telling Pine where these messages are delivered. This will also only appear after the first message has been delivered to that folder.

Logging In

The Mail Filter Utility needs to connect to the mail server twice, once to retrieve the old recipe file and once to deposit the new one. Use the same user name, password, and server as you use to check your email. Once you have logged in, your web browser will keep track of your password, so be sure to close your web browser before you leave the computer.

Creating Recipes

Spam Filtering Options

Enable Spam Filtering

If you would like to temporarily disable any actions having to do with spam filtering, but want to save your settings, you can un-check the box labeled "Enable Spam Filtering". The settings will still show up in your list of mail filters, but will have no effect on your mail. Also note that, once you have set up spam filtering, you can use the button for enabling or disabling quick spam filtering without losing your settings.

Spam Filter name

The name you give your spam filter has no effect on how your mail is handled; it is for you to help remember which filter handles spam when you are looking at your list of mail filters.

Spam Sensitivity, and the Two Levels of Spam Filtering

When a message is checked by Spam Assassin, it looks through the message for various characteristics of spam that make a message more likely to be spam than a message not bearing those marks. Each time it finds something that is common in spam messages, it adds a small number to that message's spam score. Depending on the type of messages you receive, you may wish to set these messages higher (if many messages that are not spam are being marked as spam) or lower (if much of your spam is not being marked). Most people find that any message with a score higher than 8 is almost certain to be spam, and that messages with a score higher than 5 are usually spam.

To give more flexibility in dealing with messages that may or may not be spam, all of the options associated with spam filtering are available twice, once for messages that are "Probably spam", meaning that the spam filter saw lots of spam-like things in a message, and one for messages that are only "Possibly spam", meaning that the spam filter saw some, but not a lot, of spam-like features.

You will want your sensitivity for 'Probably Spam' to be a higher number than the level you choose for 'Possibly spam', usually between 8-10, so that only messages that are almost certain to be spam are handled there. Messages that have a score above 4 or 5 have a good chance of being spam, and thus can be labeled as such.

Since the messages marked as 'Probably Spam' have a very high chance of being spam, it is usually a good idea to send these messages to a folder separate from your mail Inbox, and review them only occasionally to make sure that no important messages were mis-marked. Again, the higher spam sensitivity level you choose, the less likely a legitimate message is to be marked as spam, but the less likely a spam message is to be marked as spam. Messages that are only "probably spam" should usually be delivered to your mail inbox anyways, and given a cursory review.

The options presented when you first begin to configure Spam Assassin are suitable for most users.

Changing the Subject

For many users it is enough to change the subject of a message to start with "*****SPAM*****", so that they can easily recognize which messages are not likely to be worth reading, and which ones should be reviewed to see if they are indeed spam. If you are using Eudora or Outlook, since you will not be able to use separate mail folders on the mail server, this will be the only way to distinguish what is probably spam from what is probably legitimate.

Dealing With Spam

If a message is marked as spam, there are two options for handling the message. The first is to deliver the message to your Mail Inbox, despite being spam. It is usually wise to do this for a while, until you are sure that everything is working correctly. If you elect to have messages delivered to another mail folder, you will be able to read your messages using ACS-Webmail and Pine without any additional configuration.

Cutting down on spam buildup

To make sure that you do not go over your quota where your spam is being stored, there is an option to periodically check for any messages that were marked as spam and delete them. You can specify how frequently this occurs. When the check is run, all messages older than that frequency that are also marked as spam are deleted. For instance, if you specify that every 7 days a check should be run to delete old spam, any message that is BOTH older than 7 days AND marked as spam will be deleted, once a week.

What to do when a message is NOT spam

If you find that messages from a particular person, or about a particular subject are always being marked as spam when they are not, you can make sure that that message gets delivered without being marked as spam by setting up a Simple Recipe (described below) to "Whitelist" the person. (A "Whitelist" is a list of a email address that are never spam). For example, if I make many purchases from Ebay.com, but all my bid confirmation messages are being marked as spam, I would set up a simple recipe called "Ebay Whitelist" and tell it to check the email address and make sure that it contains the phrase "bidconfirm@ebay.com", and say that such a message should be delivered immediately to my mail inbox. After I create this recipe, I would then use the arrows in the list of recipes to make sure that this list appears before my Spam Assassin recipe.

Pre-Written Recipes

A number of recipes have been pre-written, so that you will only need to fill in a few details to create a functional recipe.

It is important to think about where in the list of recipes these should be placed in order to work correctly.

Whitelist: Always deliver messages from someone

With this tool, you can make sure that messages from particular people are never marked as spam. Just enter in the box a list of email addresses whose messages are never spam.

This recipe should be somewere above the spam filter recipe in your list of recipes.

Blacklist: Always delete email from someone

With this tool, you can make sure that messages from particular people don't bug you ever again. Just enter in the box a list of email addresses and choose whether their messages should always be deleted or whether they should just be dumped in another folder.

The blacklist should usually be the very first recipe. If an email address in the blacklist is found, the message will be deleted or delivered and no recipes after it will be run.

Vacation Notice: Automatically reply to messages

If you won't be able to respond to your mail for a while, you can use this recipe to let people know you might not get their messages for a while.

Whenever a message arrives, it will reply with a message of your choosing. Type your message in the large box, enter in how many days the Vacation Notice should run, and decide whether someone sending more than one message should be alerted to your absense every time.

You should place this recipe after any recipes that delete unwanted mail (such as the spam filter, a blacklist, or the duplicate filter), or after any recipes that handle mail from "non-people" (such as mailing lists), since you probably do not want to send your vacation notice to these emails. Whether or not a vacation notice is sent, recipes after this one will continue to be run.

Forward a copy to another address

If you have another email address where you read your mail, and you would like to be able to read them from BOTH places, use this tool to send a copy of all your newly arriving messages to another email address.

If you would like your mail to be delivered ONLY to one other email address, use the Mailbox Destination Tool. If you change your mail destination using that tool, you will be unable to use ACS-Webmail to read your mail, and any mail filters you have set up with this site (including spam filters) will be ignored.

You should place this recipe after any recipes that delete unwanted mail (such as the spam filter, a blacklist, or the duplicate filter), or after any recipes that handle mail from "non-people" (such as mailing lists), since you probably do not want to forward these kinds of messages. After forwarding the message, recipes after this one will continue to be run.

Mailing List Sorter

If you have signed up for many mailing lists, it may be convenient for messages from each list to be automatically sorted into their own folder.

Using some complicated logic, each message is scanned for indications that it is a mailing list. If it appears to be a mailing list, the list name will be guessed, and the message will be delivered to a mail folder with that name, appended to a prefix of your choosing. You can also use this to redirect those "This message is intended for everone in the following majors/minors" messages to a separate folder.

Sometimes mailing lists are spammed just like normal email addresses, so you may want to place this recipe after your spam filter. If experience shows that messages bound for the mailing list are marked as spam, you will want to either move this recipe before the spam filter, or add the mailing list's address to the Whitelist. If a message is deemed to be from a mailing list and is delivered to its list's folder, no further recipes will be run.

Large Message Sorter

One of the easiest ways to go over quota quickly is to receive a few large messages that you don't realize are large. This recipe will filter any messages larger than a certain size into a folder in your home directory, where you have a much larger quota. Tell it how large of messages should be stored elsewhere, and what the name of the folder should be.

This recipe should be the last recipe in your recipe list, so that the spam filter has a chance to delete large spam, and any mailing list messages are apropreatly filed away.

Duplicate Message Deleter

If you are plagued with receiving the same message over and over again, often when you are subscribed to many mailing lists with interests in common, you can use this recipe to delete or set aside any message that comes twice.

This should be one of the first recipes in your recipe list.

Simple Recipes

Recipe name

For the purposes of this utility, each recipe is given a name. This name has no affect on the operation of the recipe; it is only to help you remember what it does when you are looking at the list of recipes you have created. If you create many recipes, you will eventually create one recipe that you want to test messages with before any other recipe. For instance, suppose you receive messages from many mailing lists, and you want these messages saved to a special folder. If you are an active person on the list, however, messages may be sent to you directly, so you might create a recipe that moves all messages with your address in the 'To:' field directly to your mail inbox. Setting it up like that helps sort out the every-day messages from those of most interest to you. Giving the recipes names which help you clearly remember their function will help you know that the one that delivers messages directly to your inbox should go before the one that moves all the messages from the list to the list's folder. If they have names like "Recipe 1" and "Recipe 2", this becomes difficult. The more recipes you have, the greater the necessity for clarity.

Conditions: When to do something

Being accurate is necessary to creating a filtering system that works for you, and nowhere is accuracy more important than in specifying conditions. Each recipe can have up to two conditions. Make sure that at least one of the 'enable' boxes is checked; if none are checked then no conditions will be created. Since an action is done as long as no conditions are unmet, the action for such a recipe would always be done.

The first thing to consider for a recipe is in which part of the message to look for a word or phrase. This depends on what you are trying to filter out. If you want to stop a particular person from sending email to you, for instance, you would want the recipe to search through the email's From address. If you wanted to do something to messages about a particular topic, say, a web site you wrote where you compare batteries, which may come from many different people, you might have a recipe search in the subject line for "battery comparison". Or, if the subject line is not sufficient, you may want it to search the entire message. You may wish to make sure that something happens to every mail sent personally to you (as opposed to mass-mailings to many people). You could then look in the 'To' field of a message for your own email address.

When you enter something to search for, there are many ways to interpret what you enter. One way is to look for the exact phrase. With this option, if you search for "UCSD Anime Club", the condition would only be met if the message contains exactly that (with the exception of capital letters; for instance, 'ucsd anime club' would meet the condition as well). By setting it to look for 'at least one of these words' that phrase, a message would only need to have 'UCSD' in it to satisfy the condition; messages with 'anime' or 'club' would equally satisfy the condition. In contrast, setting it to look for 'all of these words' will require that the message contain 'UCSD', 'anime', and 'club', but they do not need to be near one another or in any particular order. The last option, 'none of these words', means that if the message has any of the words 'UCSD', 'anime', or 'club' anywhere in the message it will NOT satisfy the condition, and will not take the action for that recipe.

An overview: We're searching for "UCSD Anime Club"
  Action will be taken with the condition...
If the message contains this... 'exact' 'at least 1' 'all' 'none'
"This is the newsletter of the UCSD Anime Club." Yes Yes Yes No
"This is the newsletter of the Anime Club at UCSD" No Yes Yes No
"The first annual meeting of the Odd-ballians club will be this Thursday" No Yes No No
"Welcome to Aardvark Mail service" No No No No

One problem with University email addresses is receiving messages intended for the former account holder. For example, when Jeremy Smith got his email account, he was assigned jsmith@ucsd.edu, but found that he was getting emails intended for someone named Julius Smith. Jeremy could set up a recipe that would affect all messages bound for Julius Smith by creating a condition that looked through the whole message for the exact phrase 'Julius Smith'.

Actions: What to do

There are four options for what to do to a message that satisfies all the conditions in a recipe. They are 1) to delete the message 2) to forward the message, 3) to move the message to a folder, and 4) to deliver immediately to your mail inbox.

Deleting a message should probably be avoided in most case unless you are VERY sure that the recipe will catch only messages you want deleted. This is because deleting a message is permanent; no record will remain anywhere of what the message contained, unless the sender saved a copy on their own computer. Use this option at your own risk!

Forwarding a message can be useful if you tend to check, for example, different email messages in different places. You could set up recipes to forward emails from friends to an address you check at home, and leave school emails in your school mailbox.

To add or not to add?

When you are finished creating a recipe, click "Add this recipe". This will not change your settings on your mail server; it will only update a temporary copy of your mail filters. You have to click "Log out, saving changes" from the main screen before changes will be saved.

Custom Recipes

If you know how to use Procmail recipes, entering a custom recipe gives you the flexibility to do whatever you want. You are prompted for a recipe name, and just as with a simple recipe the name has no effect on the operation of the recipe; it is to make sorting the recipes easier. You are also given a text field into which you can enter whatever Procmail commands you desire. There is no verification of any sort; your mail will be filtered according to whatever you specify. You should only do this if you are very familiar with Procmail commands.

Saving your recipes—or not

To log out, you can click either of two buttons. 'Log out, saving changes' will copy the file containing your filters to your mail server and delete the temporary copy you have been working on. 'Log out, DO NOT save changes' will just delete the temporary copy, and any changes you have made will be lost. If you do not log out, no changes will be saved! Also, if you hide the advanced options, your changes will not be saved unless you change enable or disable the quick spam filtering option.

Changing your recipe order

Remember that the higher a filter is listed, the sooner it will have a chance to filter a message — and if one filter does something to a message, no other filters will get a chance to do something to a message. So, if you want to pass messages through a particular filter before some other filter, use the up arrows to move the message higher in the list.