Sendmail

1. Introduction to sendmail

It's been said that you aren't a real Unix system administrator until you've edited a sendmail.cf file. It's also been said that you're crazy if you've attempted to do so twice.

sendmail is an incredibly powerful mail program. It's also incredibly difficult to learn and understand. Any program whose definitive reference (sendmail, by Bryan Costales and Eric Allman, published by O'Reilly) is 1,050 pages long scares most people off. Information on the sendmail reference is contained in the bibliography at the end of this book.

Fortunately, new versions of sendmail are different. You no longer need to directly edit the cryptic sendmail.cf file; the new version provides a configuration utility that will create the sendmail.cf file for you based on much simpler macro files. You do not need to understand the complex syntax of the sendmail.cf file; the macro files don't require you to. Instead, you need only list items, such as the name of features you wish to include in your configuration, and specify some of the parameters that determine how that feature operates. A traditional Unix utility called m4 then takes your macro configuration data and mixes it with the data it reads from template files containing the actual sendmail.cf syntax, to produce your sendmail.cf file.

In this chapter we introduce sendmail and describe how to install, configure and test it, using the Virtual Brewery as an example. If the information presented here helps make the task of configuring sendmail less daunting for you, we hope you'll gain the confidence to tackle more complex configurations on your own.

2. Installing sendmail

The sendmail mail transport agent is included in prepackaged form in most Linux distributions. Installation in this case is relatively simple. Despite this fact, there are some good reasons to install sendmail from source, especially if you are security conscious. The sendmail program is very complex and has earned a reputation over the years for containing bugs that allow security breaches. One of the best known examples is the RTM Internet worm that exploited a buffer overflow problem in early versions of sendmail. We touched on this briefly in Chapter 9. Most security exploits involving buffer overflows rely on all copies of sendmail on different machines being identical, as the exploits rely on data being stored in specific locations. This, of course, is precisely what happens with sendmail installed from Linux distributions. Compiling sendmail from source yourself can help reduce this risk. Modern versions of sendmail are less vulnerable because they have come under exceedingly close scrutiny as security has become a more widespread concern throughout the Internet community.

The sendmail source code is available via anonymous FTP from ftp.sendmail.org.

Compilation is very simple bceause the sendmail source package directly supports Linux. The steps involved in compiling sendmail are:

You need root permissions to complete the installation of the resulting binary files using:

You have now installed the sendmail binary into the /usr/sbin directory. Several symbolic links to the sendmail binary will be installed into the /usr/bin/ directory. We'll talk about those links when we discuss common tasks in running sendmail.

3. Overview of Configuration Files

Traditionally, sendmail was set up through a system configuration file (typically called /etc/mail/sendmail.cf, or in older distributions, /etc/sendmail.cf, or even /usr/lib/sendmail.cf ) that is not anything close to any language you've seen before. Editing the sendmail.cf file to provide customized behavior can be a humbling experience.

Today sendmail makes all configuration options macro driven with an easy-to-understand syntax. The macro method generates configurations to cover most installations, but you always have the option of tuning the resultant sendmail.cf manually to work in a more complex environment.

3. Overview of Configuration Files

Traditionally, sendmail was set up through a system configuration file (typically called /etc/mail/sendmail.cf, or in older distributions, /etc/sendmail.cf, or even /usr/lib/sendmail.cf ) that is not anything close to any language you've seen before. Editing the sendmail.cf file to provide customized behavior can be a humbling experience.

Today sendmail makes all configuration options macro driven with an easy-to-understand syntax. The macro method generates configurations to cover most installations, but you always have the option of tuning the resultant sendmail.cf manually to work in a more complex environment.

4. The sendmail.cf and sendmail.mc Files

The m4 macro processor program generates the sendmail.df file when it processes the macro configuration file provided by the local system administrator. Throughout the remainder of this chapter we will refer to this configuration file as the sendmail.mc file.

The configuration process is basically a matter of creating a suitable sendmail.mc file that includes macros that describe your desired configuration. The macros are expressions that the m4 macro processor understands and expands into the complex sendmail.cf syntax. The macro expressions are made up of the macro name (the text in capital letters at the start), which can be likened to a function in a programming language, and some parameters (the text within brackets) that are used in the expansion. The parameters may be passed literally into the sendmail.cf output or may be used to govern the way the macro processing occurs.

A sendmail.mc file for a minimal configuration (UUCP or SMTP with all nonlocal mail being relayed to a directly connected smart host) can be as short as 10 or 15 lines, excluding comments.

4.1. Two Example sendmail.mc Files

If you're an administator of a number of different mail hosts, you might not want to name your configuration file sendmail.mc. Instead, it is common practice to name it after the host-vstout.m4 in our case. The name doesn't really matter as long as the output is called sendmail.cf. Providing a unique name for the configuration file for each host allows you to keep all configuration files in the same directory and is just an administrative convenience. Let's look at two example macro configuration files so we know where we are heading.

Most sendmail configurations today use SMTP only. It is very simple to configure sendmail for SMTP. Example 18-1 expects a DNS name server to be available to resolve hosts and will attempt to accept and deliver all mail for hosts using just SMTP.

Example 18-1. Sample Configuration File vstout.smtp.m4

A sendmail.mc file for vstout at the Virtual Brewery is shown in Example 18-2. vstout uses SMTP to talk to all hosts on the Brewery's LAN, and you'll see the commonality with the generic SMTP-only configuration just presented. In addition, the vstout configuration sends all mail for other destinations to moria, its Internet relay host, via UUCP.

Example 18-2. Sample Configuration File vstout.uucpsmtp.m4

If you compare and contrast the two configurations, you might be able to work out what each of the configuration parameters does. We'll explain them all in detail.

4.2. Typically Used sendmail.mc Parameters

A few of the items in the sendmail.mc file are required all the time; others can be ignored if you can get away with defaults. The general sequence of the definitions in the sendmail.mc is as follows:

VERSIONID

OSTYPE

DOMAIN

FEATURE

Local macro definitions

MAILER

LOCAL_* rulesets

We'll talk about each of these in turn in the following sections and refer to our examples in Example 18-1 and Example 18-2, when appropriate, to explain them.

4.2.1. Comments

Lines in the sendmail.mc file that begin with the character are not parsed by m4, and will by default be output directly into the sendmail.cf file. This is useful if you want to comment on what your configuration is doing in both the input and output files.

To allow comments in your sendmail.mc that are not placed into the sendmail.cf, you can use the m4 divert and dnl tokens. divert(-1) will cause all output to cease. divert(0) will cause output to be restored to the default. Any output generated by lines between these will be discarded. In our example, we've used this mechanism to provide a comment that appears only in the sendmail.mc file. To achieve the same result for a single line, you can use the dnl token that means, literally, "starting at the beginning of the next line, delete all characters up to and including the next newline." We've used this in our example, too.

These are standard m4 features, and you can obtain more information on them from its manual page.

4.2.2. VERSIONID and OSTYPE

The VERSIONID macro is optional, but is useful to record the version of the sendmail configuration in the sendmail.cf file. So you'll often encounter it, and we recommend it. In any case, be sure to include:

This is probably the most important definition. The OSTYPE macro causes a file of definitions to be included that are good defaults for your operating system. Most of the definitions in an OSTYPE macro file set the pathnames of various configuration files, mailer program paths and arguments, and the location of directories sendmail uses to store messages. The standard sendmail source code release includes such a file for Linux, which would be included by the previous example. Some Linux distributions, notably the Debian distribution, include their own definition file that is completely Linux-FHS compliant. When your distribution does this, you should probably use its definition instead of the Linux default one.

The OSTYPE definition should be one of the first definitions to appear in your sendmail.mc file, as many other definitions depend upon it.

4.2.3. DOMAIN

The DOMAIN macro is useful when you wish to configure a large number of machines on the same network in a standard way. It you're configuring a small number of hosts, it probably isn't worth bothering with. You typically configure items, such as the name of mail relay hosts or hubs that all hosts on your network will use.

The standard installation contains a directory of m4 macro templates used to drive the configuration process. This directory is usually named /usr/share/sendmail.cf or something similar. Here you will find a subdirectory called domain that contains domain-specific configuration templates. To make use of the DOMAIN macro, you must create your own macro file containing the standard definitions you require for your site, and write it into the domain subdirectory. You'd normally include only the macro definitions that were unique to your domain here, such as smart host definitions or relay hosts, but you are not limited to these.

The sendmail source distribution comes with a number of sample domain macro files that you can use to model your own.

If you saved your domain macro file as /usr/share/sendmail.cf/domain/vbrew.m4, you'd include definitions in your sendmail.mc using:

4.2.4. FEATURE

The FEATURE macro enables you to include predefined sendmail features in your configuration. These sendmail features make the supported configurations very simple to use. There are a large number, and throughout this chapter we'll talk about only a few of the more useful and important ones. You can find full details of the features available in the CF file included in the source package.

To use any of the features listed, you should include a line in your sendmail.mc that looks like:

where name is substituted with the feature name. Some features take one optional parameter. If you wish to use something other than the default, you should use an entry that looks like:

where param is the parameter to supply.

4.2.5. Local macro definitions

The standard sendmail macro configuration files provide lots of hooks and variables with which you can customize your configuration. These are called local macro definitions. Many of them are listed in the CF file in the sendmail source package.

The local macro definitions are usually invoked by supplying the name of the macro with an argument representing the value you wish to assign to the variable the macro manages. Again, we'll explore some of the more common local macro definitions in the examples we present later in the chapter.

4.2.6. Defining mail transport protocols

If you want sendmail to transport mail in any way other than by local delivery, you must tell it which transports to use. The MAILER macro makes this very easy. The current version of sendmail supports a variety of mail transport protocols; some of these are experimental, others are probably rarely used.

In our network we need the SMTP transport to send and receive mail among the hosts on our local area network, and the UUCP transport to send and receive mail from our smart host. To achieve this, we simply include both the smtp and uucp mail transports. The local mail transport is included by default, but may be defined for clarity, if you wish. If you are including both the smtp and the uucp mailers in your configuration, you must always be sure to define the smtp mailer first.

The more commonly used transports available to you using the MAILER macro are described in the following list:

local

This transport includes both the local delivery agent used to send mail into the mailbox of users on this machine and the prog mailer used to send messages to local programs. This transport is included by default.

smtp

This transport implements the Simple Mail Transport Protocol (SMTP), which is the most common means of transporting mail on the Internet. When you include this transport, four mailers are configured: smtp (basic SMTP), esmtp (Extended SMTP), smtp8 (8bit binary clean SMTP), and relay (specifically designed for gatewaying messages between hosts).

uucp

The uucp transport provides support for two mailers: uucp-old, which is the traditional UUCP, and uucp-new, which allows multiple recipients to be handled in one transfer.

usenet

This mailer allows you to send mail messages directly into Usenet style news networks. Any local message directed to an address of news.group.usenet will be fed into the news network for the news.group newsgroup.

fax

If you have the HylaFAX software installed, this mailer will allow you to direct email to it so that you may build an email-fax gateway. This feature is experimental at the time of writing and more information may be obtained from https://www.vix.com/hylafax/.

There are others, such as the pop, procmail, mail11, phquery, and cyrus that are useful, but less common. If your curiosity is piqued, you can read about these in the sendmail book or the documentation supplied in the source package.

4.2.7. Configure mail routing for local hosts

The Virtual Brewery's configuration is probably more complex than most sites require. Most sites today would use the SMTP transport only and do not have to deal with UUCP at all. In our configuration we've configured a "smart host" that is used to handle all outgoing mail. Since we are using the SMTP transport on our local network we must tell sendmail that it is not to send local mail via the smart host. The LOCAL_NET_CONFIG macro allows you to insert sendmail rules directly into the output sendmail.cf to modify the way that local mail is handled. We'll talk more about rewrite rules later on, but for the moment you should accept that the rule we've supplied in our example specifies that any mail destined for hosts in the vbrew.com domain should be delivered directly to the target hosts using the SMTP mail transport.

5. Generating the sendmail.cf File

When you have completed editing your m4 configuration file, you must process it to produce the /etc/mail/sendmail.cf file read by sendmail. This is straightforward, as illustrated by the following example:

This command invokes the m4 macro processor, supplying it the name of two macro definition files to process. m4 processes the files in the order given. The first file is a standard sendmail macro template supplied with the sendmail source package, the second, of course, is the file containing our own macro definitions. The output of the command is directed to the /etc/mail/sendmail.cf file, which is our target file.

You may now start sendmail with the new configuration.

6. Interpreting and Writing Rewrite Rules

Arguably the most powerful feature of sendmail is the rewrite rule. Rewrite rules are used by sendmail to determine how to process a received mail message. sendmail passes the addresses from the headers of a mail message through collections of rewrite rules called rulesets The rewrite rules transform a mail address from one form to another and you can think of them as being similar to a command in your editor that replaces all text matching a specified pattern with another.

Each rule has a lefthand side and a righthand side, separated by at least one tab character. When sendmail is processing mail it scans through the rewriting rules looking for a match on the lefthand side. If an address matches the lefthand side of a rewrite rule, the address is replaced by the righthand side and processed again.

6.1. sendmail.cf R and S Commands

In the sendmail.cf file, the rulesets are defined using commands coded as Sn, where n specifies the ruleset that is considered the current one.

The rules themselves appear in commands coded as R. As each R command is read, it is added to the current ruleset.

If you're dealing only with the sendmail.mc file, you won't need to worry about S commands at all, as the macros will build those for you. You will need to manually code your R rules.

A sendmail ruleset therefore looks like:

6.2. Some Useful Macro Definitions

sendmail uses a number of standard macro definitions internally. The most useful of these in writing rulesets are:

$j

The fully qualified domain name of this host.

$w

The hostname component of the FQDN.

$m

The domain name component of the FQDN.

We can incorporate these macro definitions in our rewrite rules. Our Virtual Brewery configuration uses the $m macro.

6.3. The Lefthand Side

In the lefthand side of a rewriting rule, you specify a pattern that will match an address you wish to transform. Most characters are matched literally, but there are a number of characters that have special meaning; these are described in the following list. The rewrite rules for the lefthand side are:

Match exactly zero tokens

Match zero or more tokens

Match one or more tokens

Match exactly one token

$=x

Match any phrase in class x

$~x

Match any word not in class x

A token is a string of characters delimited by spaces. There is no way to include spaces in a token, nor is it necessary, as the expression patterns are flexible enough to work around this need. When a rule matches an address, the text matched by each of the patterns in the expression will be assigned to special variables that we'll use in the righthand side. The only exception to this is the , which matches no tokens and therefore will never generate text to be used on the righthand side.

6.4. The Righthand Side

When the lefthand side of a rewrite rule matches an address, the original text is deleted and replaced by the righthand side of the rule. All tokens in the righthand side are copied literally, unless they begin with a dollar sign. Just as for the lefthand side, a number of metasymbols may be used on the righthand side. These are described in the following list. The rewrite rules for the righthand side are:

$n

This metasymbol is replaced with the n'th expression from the lefthand side.

$[name$]

This metasymbol resolves hostname to canonical name. It is replaced by the canonical form of the host name supplied.

$(map key $@arguments $:default $)

This is the more general form of lookup. The output is the result of looking up key in the map named map passing arguments as arguments. The map can be any of the maps that sendmail supports such as the virtusertable that we describe a little later. If the lookup is unsuccessful, default will be output. If a default is not supplied and lookup fails, the input is unchanged and key is output.

$>n

This will cause the rest of this line to be parsed and then given to ruleset n to evaluate. The output of the called ruleset will be written as output to this rule. This is the mechanism that allows rules to call other rulesets.

$#mailer

This metasymbol causes ruleset evaluation to halt and specifies the mailer that should be used to transport this message in the next step of its delivery. This metasymbol should be called only from ruleset 0 or one of its subroutines. This is the final stage of address parsing and should be accompanied by the next two metasymbols.

$@host

This metasymbol specifies the host that this message will be forwarded to. If the destination host is the local host, it may be omitted. The host may be a colon-separated list of destination hosts that will be tried in sequence to deliver the message.

$:user

This metasymbol specifies the target user for the mail message.

A rewrite rule that matches is normally tried repeatedly until it fails to match, then parsing moves on to the next rule. This behavior can be changed by preceding the righthand side with one of two special righthand side metaymbols described in the following list. The rewrite rules for a righthand side loop control metasymbols are:

This metasymbol causes the ruleset to return with the remainder of the righthand side as the value. No other rules in the ruleset are evaluated.

This metasymbol causes this rule to terminate immediately, but the rest of the current ruleset is evaluated.

6.5. A Simple Rule Pattern Example

To better see how the macro substitution patterns operate, consider the following rule lefthand side:

This rule matches "Zero or more tokens, followed by the < character, followed by one or more tokens, followed by the > character."

If this rule were applied to brewer@vbrew.com or Head Brewer < >, the rule would not match. The first string would not match because it does not include a < character, and the second would fail because matches one or more tokens and there are no tokens between the <> characters. In any case in which a rule does not match, the righthand side of the rule is not used.

If the rule were applied to Head Brewer < brewer@vbrew.com >, the rule would match, and on the righthand side would be substituted with Head Brewer and would be substituted with brewer@vbrew.com.

If the rule were applied to < brewer@vbrew.com > the rule would match because matches zero or more tokens, and on the righthand side would be substituted with the empty string.

6.6. Ruleset Semantics

Each of the sendmail rulesets is called upon to perform a different task in mail processing. When you are writing rules, it is important to understand what each of the rulesets are expected to do. We'll look at each of the rulesets that the m4 configuration scripts allow us to modify:

LOCAL_RULE_3

Ruleset 3 is responsible for converting an address in an arbitrary format into a common format that sendmail will then process. The output format expected is the familiar looking local-part@host-domain-spec.

Ruleset 3 should place the hostname part of the converted address inside the < and > characters to make parsing by later rulesets easier. Ruleset 3 is applied before sendmail does any other processing of an email address, so if you want sendmail to gateway mail from some system that uses some unusual address format, you should add a rule using the LOCAL_RULE_3 macro to convert addresses into the common format.

LOCAL_RULE_0 and LOCAL_NET_CONFIG

Ruleset 0 is applied to recipient addresses by sendmail after Ruleset 3. The LOCAL_NET_CONFIG macro causes rules to be inserted into the bottom half of Ruleset 0.

Ruleset 0 is expected to perform the delivery of the message to the recipient, so it must resolve to a triple that specifies each of the mailer, host, and user. The rules will be placed before any smart host definition you may include, so if you add rules that resolve addresses appropriately, any address that matches a rule will not be handled by the smart host. This is how we handle the direct smtp for the users on our local LAN in our example.

LOCAL_RULE_1 and LOCAL_RULE_2

Ruleset 1 is applied to all sender addresses and Ruleset 2 is applied to all recipient addresses. They are both usually empty.

6.6.1. Interpreting the rule in our example

Our sample in Example 18-3 uses the LOCAL_NET_CONFIG macro to declare a local rule that ensures that any mail within our domain is delivered directly using the smtp mailer. Now that we've looked at how rewrite rules are constructed, we will be able to understand how this rule works. Let's take another look at it.

Example 18-3. Rewrite Rule from vstout.uucpsmtp.m4

We know that the LOCAL_NET_CONFIG macro will cause the rule to be inserted somewhere near the end of ruleset 0, but before any smart host definition. We also know that ruleset 0 is the last ruleset to be executed and that it should resolve to a three-tuple specifying the mailer, user, and host.

We can ignore the two comment lines; they don't do anything useful. The rule itself is the line beginning with R. We know that the R is a sendmail command and that it adds this rule to the current ruleset, in this case ruleset . Let's look at the lefthand side and the righthand side in turn.

The lefthand side looks like: $* < @ $* .$m. > $*.

Ruleset 0 expects < and > characters because it is fed by ruleset 3. Ruleset 3 converts addresses into a common form and to make parsing easier, it also places the host part of the mail address inside <>s.

This rule matches any mail address that looks like: 'DestUser < @ somehost.ourdomain. > Some Text'. That is, it matches mail for any user at any host within our domain.

You will remember that the text matched by metasymbols on the lefthand side of a rewrite rule is assigned to macro definitions for use on the righthand side. In our example, the first matches all text from the start of the address until the < character. All of this text is assigned to for use on the righthand side. Similarly the second in our rewrite rule is assigned to , and the last is assigned to .

We now have enough to understand the lefthand side. This rule matches mail for any user at any host within our domain. It assigns the username to $1, the hostname to , and any trailing text to . The righthand side is then invoked to process these.

Let's now look at what we're expecting to see outputed. The righthand side of our example rewrite rule looks like: $#smtp $@ $2.$m. $: $1 < @ $2.$m. > $3.

When the righthand side of our ruleset is processed, each of the metasymbols are interpreted and relevant substitutions are made.

The metasymbol causes this rule to resolve to a specific mailer, smtp in our case.

The resolves the target host. In our example, the target host is specified as $2.$m., which is the fully qualified domain name of the host on in our domain. The FQDN is constructed of the hostname component assigned to from our lefthand side with our domain name (.$m.) appended.

The metasymbol specifies the target user, which we again captured from the lefthand side and had stored in .

We preserve the contents of the <> section, and any trailing text, using the data we collected from the lefthand side of the rule.

Since this rule resolves to a mailer, the message is forwarded to the mailer for delivery. In our example, the message would be forwarded to the destination host using the SMTP protocol.

7. Configuring sendmail Options

sendmail has a number of options that allow you to customize the way it performs certain tasks. There are a large number of these, so we've listed only a few of the more commonly used ones in the upcoming list.

To configure any of these options, you may either define them in the m4 configuration file, which is the preferable method, or you may insert them directly into the sendmail.cf file. For example, if we wished to have sendmail fork a new job for each mail message to be delivered, we might add the following line to our m4 configuration file:

The corresponding sendmail.cf entry created is:

The following list describes common sendmail m4 options (and sendmail.cf equivalents):

confMIN_FREE_BLOCKS (MinFreeBlocks)

There are occasions when a problem might prevent the immediate delivery of mail messages, causing messages to be queued in the mail spool. If your mail host processes large volumes of mail, it is possible for the mail spool to grow to such a size that it fills the filesystem supporting the spool. To prevent this, sendmail provides this option to specify the minimum number of free disk blocks that must exist before a mail message will be accepted. This allows you to ensure that sendmail never causes your spool filesystem to be filled (Default: 100).

confME_TOO (MeToo)

When a mail target such as an email alias is expanded, it is sometimes possible for the sender to appear in the recipient list. This option determines whether the originators of an email message will receive a copy if they appear in the expanded recipient list. Valid values are "true" and "false" (Default: false).

confMAX_DAEMON_CHILDREN (MaxDaemonChildren)

Whenever sendmail receives an SMTP connection from a remote host, it spawns a new copy of itself to deal with the incoming mail message. This way, it is possible for sendmail to be processing multiple incoming mail messages simulatanenously. While this is useful, each new copy of sendmail consumes memory in the host computer. If an unusually large number of incoming connections are received, by chance, because of a problem or a malicious attack, it is possible for sendmail daemons to consume all system memory. This option provides you with a means of limiting the maximum number of daemon children that will be spawned. When this number is reached, new connections are rejected until some of the existing children have terminated (Default: undefined).

confSEPARATE_PROC (ForkEachJob)

When processing the mail queue and sending mail messages, sendmail processes one mail message at a time. When this option is enabled, sendmail will fork a new copy of itself for each message to be delivered. This is particularly useful when there are some mail messages that are stuck in the queue because of a problem with the target host (Default: false).

confSMTP_LOGIN_MSG (SmtpGreetingMessage)

Whenever a connection is made to sendmail, a greeting message is sent. By default, this message contains the hostname, name of the mail transfer agent, the sendmail version number, the local version number, and the current date. RFC821 specifies that the first word of the greeting should be the fully qualified domain name of the host, but the rest of the greeting can be configured however you please. You can specify sendmail macros here and they will be expanded when used. The only people who will see this message are suffering system administrators diagnosing mail delivery problems or strongly curious people interested in discovering how your machine is configured. You can relieve some of the tedium of their task by customizing the welcome message with some witticisms; be nice. The word "EMSTP" will be inserted between the first and second words by sendmail, as this is the signal to remote hosts that we support the ESMTP protocol (Default: $j Sendmail $v/$Z; $b

8. Some Useful sendmail Configurations

There are myriad possible sendmail configurations. In this space we'll illustrate just a few important types of configuration that will be useful in many sendmail installations.

8.1. Trusting Users to Set the From: Field

It is sometimes useful to overwrite the From: field of an outgoing mail message. Let's say you have a web-based program that generates email. Normally the mail message would appear to come from the user who owned the web server process. We might want to specify some other source address so that the mail appears to have originated from some other user or address on that machine. sendmail provides a means of specifying which systems users are to be entrusted with the ability to do this.

The use_ct_file feature enables the specification and use of a file that lists the names of trusted users. By default, a small number of system users are trusted by sendmail (root, for example). The default filename for this feature is /etc/mail/trusted-users in systems exploiting the /etc/mail/ configuration directory and /etc/sendmail.ct in those that don't. You can specify the name and location of the file by overriding the confCT_FILE definition.

Add FEATURE(use_ct_file) to your sendmail.mc file to enable the feature.

8.2. Managing Mail Aliases

Mail aliases are a powerful feature that enable mail to be directed to mailboxes that are alternate names for users or processes on a destination host. For example, it is common practice to have feedback or comments relating to a World Wide Web server to be directed to "webmaster." Often there isn't a user known as "webmaster" on the target machine, instead it is an alias of another system user. Another common use of mail aliases is exploited by mailing list server programs in which an alias directs incoming messages to the list server program for handling.

The /etc/aliases file is where the aliases are stored. The sendmail program consults this file when determining how to handle an incoming mail message. If it finds an entry in this file matching the target user in the mail message, it redirects the message to wherever the entry describes.

Specifically there are three things that aliases allow to happen:

They provide a shorthand or well-known name for mail to be addressed to in order to go to one or more persons.

They can invoke a program with the mail message as the input to the program.

They can send mail to a file.

All systems require aliases for Postmaster and MAILER-DAEMON to be RFC-compliant.

Always be extremely aware of security when defining aliases that invoke programs or write to programs, since sendmail generally runs with root permissions.

Details concerning mail aliases may be found in the aliases(5) manual page. A sample aliases file is shown in Example 18-4.

Example 18-4. Sample aliases File

Whenever you update the /etc/aliases file, be sure to run the command:

to rebuild the database that sendmail uses internally. The /usr/bin/newaliases command is a symbolic link to the sendmail executable, and when invoked this way, behaves exactly as though it were invoked as:

The newaliases command is an alternative and more convenient way to do this.

8.3. Using a Smart Host

Sometimes a host finds mail that it is unable to deliver directly to the desired remote host. It is often convenient to have a single host on a network take on the role of managing transmission of mail to remote hosts that are difficult to reach, rather than have each local host try to do this independently.

There are a few good reasons to have a single host take on mail management. You can simplify management by having only one host with a comprehensive mail configuration that knows how to handle all of the different mail transport types, such as UUCP, Usenet, etc. All other hosts need only a single tranport protocol to send their mail to this central host. Hosts that fill this central mail routing and forwarding role are called smart hosts. If you have a smart host that will accept mail from you, you can send it mail of any sort and it will manage the routing and transmission of that mail to the desired remote destinations.

Another good application for smart host configurations is to manage transmission of mail across a private firewall. An organization may elect to install a private IP network and use their own, unregistered IP addresses. The private network may be connected to the Internet through a firewall. Sending mail to and from hosts in the private network to the outside world using SMTP would not be possible in a conventional configuration because the hosts are not able to accept or establish direct network connections to hosts on the Internet. Instead, the organization could elect to have the firewall provide a mail smart host function. The smart host running on the firewall is able to establish direct network connections with hosts both on the private network and on the Internet. The smart host would accept mail from both hosts on the private network and the Internet, store them in local storage and then manage the retransmission of that mail to the correct host directly.

Smart hosts are usually used when all other methods of delivery have failed. In the case of the organization with the private network, it would be perfectly reasonable to have the hosts attempt to deliver mail directly first, and if that fails then to send it to the smart host. This relieves the smart host of a lot of traffic because other hosts can directly send mail to other hosts on the private network.

sendmail provides a simple method of configuring a smart host using the SMART_HOST feature; when implementing it in the Virtual Brewery configuration, we do exactly this. The relevant portions of our configuration that define the smart host are:

The SMART_HOST macro allows you to specify the host that should relay all outgoing mail that you are unable to deliver directly, and the mail transport protocol to use to talk to it.

In our configuration we are using the uucp-new transport to UUCP host moria. If we wanted to configure sendmail to use an SMTP-based Smart Host, we would instead use something like:

We don't need to specify SMTP as the transport, as it is the default.

Can you guess what the LOCAL_NET_CONFIG macro and the rewrite rule might be doing?

The LOCAL_NET_CONFIG macro allows you to add raw sendmail rewrite rules to your configuration that define what mail should stay within the local mail system. In our example, we've used a rule that matches any email address where the host belongs to our domain (.$m.) and rewrite it so that it is sent directly to the SMTP mailer. This ensures that any message for a host on our local domain is directed immediately to the SMTP mailer and forwarded to that host, rather than falling through to our smart host, which is the default treatment.

8.4. Managing Unwanted or Unsolicited Mail (Spam)

If you've subscribed to a mailing list, published your email address on a web site, or posted an article to UseNet, you will most likely have begun to receive unsolicited advertising email. It is commonplace now for people to scour the net in search of email addresses to add to mailing lists that they then sell to companies seeking to advertise their products. This sort of mass-mailing behavior is commonly called spamming.

The Free On-line Dictionary of Computing offers a mail-specific definition of spam as:[1]

2. (A narrowing of sense 1, above) To indiscrimately send large amounts of unsolicited e-mail meant to promote a product or service. Spam in this sense is sort of like the electronic equivalent of junk mail sent to 'Occupant.'

In the 1990s, with the rise in commercial awareness of the net, there are actually scumbags who offer spamming as a 'service' to companies wishing to advertise on the net. They do this by mailing to collections of e-mail addresses, Usenet news, or mailing lists. Such practises have caused outrage and aggressive reaction by many net users against the individuals concerned.

Fortunately, sendmail includes some support for mechanisms that can help you deal with unsolicited mail.

8.4.1. The Real-time Blackhole List

The Real-time Blackhole List is a public facility provided to help reduce the volume of unsolicited advertising you have to contend with. Known email sources and hosts are listed in a queryable database on the Internet. They're entered there by people who have received unsolicited advertising from some email address. Major domains sometimes find themselves on the list because of slip-ups in shutting down spam. While some people complain about particular choices made by the maintainers of the list, it remains very popular and disagreements are usually worked out quickly. Full details on how the service is operated may be found from the home site of the Mail Abuse Protection System at https://maps.vix.com/rbl/.

If you enable this sendmail feature, it will test the source address of each incoming mail message against the Real-time Blackhole List to determine whether to accept the message. If you run a large site with many users, this feature could save a considerable volume of disk space. This feature accepts a parameter to specify the name of the server to use. The default is the main server at rbl.maps.vix.com.

To configure the Real-time Blackhole List feature, add the following macro declaration to your sendmail.mc file:

Should you wish to specify some other RBL server, you would use a declaration that looks like:

8.4.2. The access database

An alternative system that offers greater flexibility and control at the cost of manual configuration is the sendmail access_db feature. The access database allows you to configure which hosts or users you will accept mail from and which you will relay mail for.

Managing who you will relay mail for is important, as it is another technique commonly employed by spamming hosts to circumvent systems such as the Real-time Blackhole List just described. Instead of sending the mail to you directly, spammers will relay the mail via some other unsuspecting host who allows it. The incoming SMTP connection then doesn't come from the known spamming host, it instead comes from the relay host. To ensure that your own mail hosts aren't used in this way, you should relay mail only for known hosts. Versions of sendmail that are 8.9.0 or newer have relaying disabled by default, so for those you'll need to use the access database to enable individual hosts to relay.

The general idea is simple. When a new incoming SMTP connection is received, sendmail retrieves the message header information and then consults the access database to see whether it should proceed to accept the body of the message itself.

The access database is a collection of rules that describe what action should be taken for messages received from nominated hosts. The default access control file is called /etc/mail/access. The table has a simple format. Each line of the table contains an access rule. The lefthand side of each rule is a pattern used to match the sender of an incoming mail message. It may be a complete email address, a hostname, or an IP address. The righthand side is the action to take. There are five types of action you may configure. These are:

OK

Accept the mail message.

RELAY

Accept messages from this host or user even if they are not destined for our host; that is, accept messages for relaying to other hosts from this host.

REJECT

Reject the mail with a generic message.

DISCARD

Discard the message using the $#discard mailer.

### any text

Return an error message using as the error code (which should be RFC-821 compliant) and "any text" as the message.

An example /etc/mail/access might look like:

This example would reject any email received from friends@cybermail.com, any host in the domain aol.com and the host 207.46.131.30. The next rule would accept email from postmaster@aol.com despite the fact that the domain itself has a reject rule. The last rule allows relaying of mail from any host in the linux.org.au domain.

To enable the access database feature, use the following declaration in your sendmail.mc file:

The default definition builds the database using hash -o /etc/mail/access, which generates a simple hashed database from the plain text file. This is perfectly adequate in most installations. There are other options that you should consider if you intend to have a large access database. Consult the sendmail book or other sendmail documentation for details.

8.4.3. Barring users from receiving mail

If you have users or automated processes that send mail but will never need to receive it, it is sometimes useful to refuse to accept mail destined for them. This saves wasted disk-space storing mail that will never be read. The blacklist_recipients feature, when used in combination with the access_db feature, allows you to disable the receipt of mail for local users.

To enable the feature, you add the following lines to your sendmail.mc file, if they're not already there:

To disable receipt of mail for a local user, simply add his details into the access database. Usually you would use the entry style that would return a meaningful error message to the sender so they know why the mail is not being delivered. This feature applies equally well to users in virtual mail domains, and you must include the virtual mail domain in the access database specification. Some sample /etc/mail/access entries might look like:

8.5. Configuring Virtual Email Hosting

Virtual email hosting provides a host the capability of accepting and delivering mail on behalf of a number of different domains as though it were a number of separate mail hosts. Most commonly, virtual hosting is exploited by Internet Application Providers in combination with virtual web hosting, but it's simple to configure and you never know when you might be in a position to virtual host a mailing list for your favorite Linux project, so we'll describe it here.

8.5.1. Accepting mail for other domains

When sendmail receives an email message, it compares the destination host in the message headers to the local host name. If they match, sendmail accepts the message for local delivery; if they differ, sendmail may decide to accept the message and attempt to forward it on to the final destination (See Section 8.4.2" earlier in this chapter for details on how to configure sendmail to accept mail for forwarding ).

If we wish to configure virtual email hosting, the first thing we need to do is to convince sendmail that it should also accept mail for the domains that we are hosting. Fortunately, this is a very simple thing to do.

The sendmail use_cw_file feature allows us to specify the name of a file where we store domain names for which sendmail accepts mail. To configure the feature, add the feature declaration to your sendmail.mc file:

The default name of the file will be /etc/mail/local-host-names for distributions using the /etc/mail/ configuration directory or /etc/sendmail.cw for those that don't. Alternatively, you can specify the name and location of the file by overriding the confCW_FILE macro using a variation on:

To stick with the default filename, if we wished to offer virtual hosting to the bovine.net, dairy.org, and artist.org domains, we would create a /etc/mail/local-host-names that looks like:

When this is done, and assuming appropriate DNS records exist that point those domain names to our host, sendmail will accept mail messages for those domains as though they were destined for our real domain name.

8.5.2. Forwarding virtual-hosted mail to other destinations

The sendmail virtusertable feature configures support for the virtual user table, where we configure virtual email hosting. The virtual user table maps incoming mail destined for some user@host to some otheruser@otherhost. You can think of this as an advanced mail alias feature, one that operates using not just the destination user, but also the destination domain.

To configure the virtusertable feature, add the feature to your sendmail.mc configuration as shown:

By default, the file containing the rules to perform translations will be /etc/mail/virtusertable. You can override this by supplying an argument to the macro definition; consult a detailed sendmail reference to learn about what options are available.

The format of the virtual user table is very simple. The lefthand side of each line contains a pattern representing the original destination mail address; the righthand side has a pattern representing the mail address the virtual hosted address will be mapped to.

The following example shows three possible types of entries:

In this example, we are virtual hosting three domains: bovine.net, dairy.org, and artist.org.

The first entry redirects mail sent to a user in the bovine.net virtual domain to a local user on the machine. The second entry redirects mail to a user in the same virtual domain to a user in another domain. The third example redirects all mail addressed to any user in the dairly.org virtual domain to a single remote mail address. Finally, the last entry redirects any mail to a user in the artist.org virtual domain to the same user in another domain; for example, julie@artists.org would be redirected to julie@red.firefly.com.

Notes

9. Testing Your Configuration

The m4 command processes the macro definition files according to its own syntax rules without understanding anything about correct sendmail syntax; so there won't be any error messages if you've gotten anything wrong in your macro definition file. For this reason, it is very important that you thoroughly test your configuration. Fortunately, sendmail provides a relatively easy way of doing this.

sendmail supports an "address test" mode that allows us to test our configuration and identify any errors. In this mode of operation, we invoke sendmail from the command line, and it prompts us for a ruleset specification and a destination mail address. sendmail then processes that destination address using the rules specified, displaying the output of each rewrite rule as it proceeds. To place sendmail into this mode, we invoke it with the -bt argument:

The default configuration file used is the /etc/mail/sendmail.cf file; you can specify an alternate configuration file using the -C argument. To test our configuration, we need to select a number of addresses to process that will tell us that each of our mail-handing requirements are met. To illustrate this, we'll work through a test of our more complicated UUCP configuration shown in Example 18-2

First we'll test that sendmail is able to deliver mail to local users on the system. In these tests we expect all addresses to be rewritten to the local mailer on this machine:

This output shows us how sendmail processes mail addressed to isaac on this system. Each line shows us what information has been supplied to a ruleset or the result obtained from processing by a ruleset. We told sendmail we wished to use rulesets 3 and 0 to process the address. Ruleset 0 is what is normally invoked and we forced ruleset 3 because it is not tested by default. The last line shows us that the result of ruleset 0 does indeed direct mail to isaac to the local mailer.

Next we'll test mail addressed to our SMTP address: isaac@vstout.vbrew.com. We should be able to produce the same end result as our last example:

Again, this test passed. Next we'll test mail to our UUCP style address: vstout!isaac.

This test has also passed. These tests confirm that any mail received for local users on this machine will be properly delivered irrespective of how the address is formatted. If you've defined any aliases for your machine, such as virtual hosts, you should repeat these tests for each of the alternate names by which this host is known to ensure they also work correctly.

Next we will test that mail addressed to other hosts in the vbrew.com domain is delivered directly to that host using the SMTP mailer:

We can see that this test has directed the message to the SMTP mailer to be forwarded directly to the vale.vbrew.com host and specifies the user isaac. This test confirms that our LOCAL_NET_CONFIG definition works correctly. For this test to succeed, the destination hostname must be able to be resolved correctly, so it must either have an entry in our /etc/hosts file, or in our local DNS. We can see what happens if the destination hostname isn't able to be resolved by intentionally specifying an unknown host:

This result is very different. First, ruleset 3 returned an error message indicating the hostname could not be resolved. Second, we deal with this situation by relying on the other key feature of our configuration, the smart host. The smart host will is to handle any mail that is otherwise undeliverable. The hostname we specified in this test was unable to be resolved and the rulesets determined that the mail should be forwarded to our smart host moria using the uucp-new mailer. Our smart host might be better connected and know what to do with the address.

Our final test ensures that any mail addressed to a host not within our domain is delivered to our smart host. This should produce a result similar to our previous example:

The results of this test indicate that the hostname was resolved, and that the message would still have been routed to our smart host. This proves that our LOCAL_NET_CONFIG definition works correctly and it handled both cases correctly. This test was also successful, so we can happily assume our configuration is correct and use it.

10. Running sendmail

The sendmail daemon can be run in either of two ways. One way is to have to have it run from the inetd daemon; the alternative, and more commonly used method is to run sendmail as a standalone daemon. It is also common for mailer programs to invoke sendmail as a user command to accept locally generated mail for delivery.

When running sendmail in standalone mode, place the command in an rc file so it starts at boot time. The syntax used is commonly:

The -bd argument tells sendmail to run as a daemon. It will fork and run in the background. The -q10m argument tells sendmail to check its queue every ten minutes. You may choose to use a different queue to check time.

To run sendmail from the inetd network daemon, you'd use an entry like:

The -bs argument here tells sendmail to use the SMTP protocol on stdin/stdout, which is required for use with inetd.

The runq command is usually a symlink to the sendmail binary and is a more convenient form of:

When sendmail is invoked this way, it processes any mail waiting in the queue to be transmitted. When running sendmail from inetd you must also create a cron job that runs the runq command periodically to ensure that the mail spool is serviced periodically.

A suitable cron table entry would be similar to:

In most installations sendmail processes the queue every 15 minutes as shown in our crontab example, attempting to transmit any messages there.

11. Tips and Tricks

There are a number of things you can do to make managing a sendmail site efficient. A number of management tools are provided in the sendmail package; let's look at the most important of these.

11.1. Managing the Mail Spool

Mail is queued in the /var/spool/mqueue directory before being transmitted. This directory is called the mail spool. The sendmail program provides a means of displaying a formatted list of all spooled mail messages and their status.

The /usr/bin/mailq command is a symbolic link to the sendmail executable and behaves indentically to:

The output displays the message ID, its size, the time it was placed in the queue, who sent it, and a message indicating its current status. The following example shows a mail message stuck in the queue with a problem:

This message is still in the mail queue because the destination host IP address could not be resolved.

We can force sendmail to process the queue now by issuing the /usr/bin/runq command.

The runq command produces no output. sendmail will begin processing the mail queue in the background.

11.2. Forcing a Remote Host to Process its Mail Queue

If you use a temporary dial-up Internet connection with a fixed IP address and rely on an MX host to collect your mail while you are disconnected, you will find it useful to force the MX host to process its mail queue soon after you establish your connection.

A small perl program is included with the sendmail distribution that makes this simple for mail hosts that support it. The etrn script has much the same effect on a remote host as the runq command has on our own. If we invoke the command as shown in this example:

we will force the host vstout.vbrew.com to process any mail queued for our local machine.

Typically you'd add this command to your PPP ip-up script so that it is executed soon after your network connection is established.

11.3. Analyzing Mail Statistics

sendmail collects data on mail traffic volumes and some information on hosts to which it has delivered mail. There are two commands available to display this information, mailstats, and hoststat.

11.3.1. mailstats

The mailstats command displays statistics on the volume of mail processed by sendmail. The time at which data collection commenced is printed first, followed by a table with one row for each configured mailer and one showing a summary total of all mail. Each line presents eight items of information:

A sample of the output of the mailstats command is shown in Example 18-5.

Example 18-5. Sample Output of the mailstats Command

This data is collected if the StatusFile option is enabled in the sendmail.cf file and the status file exists. Typically you'd add the following to your sendmail.cf file:

To restart the statistics collection, you need to make the statistics file zero length:

and restart sendmail.

11.3.2. hoststat

The hoststat command displays information about the status of hosts that sendmail has attempted to deliver mail to. The hoststat command is equivalent to invoking sendmail as:

The output presents each host on a line of its own, and for each the time since delivery was attempted to it, and the status message received at that time.

Example 18-6 shows the sort of output you can expect from the hoststat command. Note that most of the results indicate successful delivery. The result for earthlink.net, on the other hand, indicates that delivery was unsuccessful. The status message can sometimes help determine the cause of the failure. In this case, the connection timed out, probably because the host was down or unreachable at the time delivery was attempted.

Example 18-6. Sample Output of the oststat Command

The purgestat command flushes the collected host data and is equivalent to invoking sendmail as:

The statistics will continue to grow until you purge them. You might want to periodically run the purgestat command to make it easier to search and find recent entries, especially if you have a busy site. You could put the command into a crontab file so it runs automatically, or just do it yourself occasionally.