Configuring mail services (revision 1)

Introduction

This page describes how Alexis Huxley installed and configured various mail services.

Warning: Postfix’s chroot environment

Postfix’s chroot environment contains copies of system files, which are not updated when using the postfix command to control the service. Therefore always using the systemctl command, otherwise a lot of head scratching ensues!

Prologue

  1. On the firewall close the mail related ports.
  2. Shutdown the existing mail server, if there is one.
  3. Rsync over users’ mail directories from old storage to new storage.

Certificates

Acquiring proper certificates is outside the scope of this procedure.

  1. In order to use SSL/TLS, create a self-signed certificate by running: 
    openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout /etc/ssl/private/mail.key -out /etc/ssl/certs/mail.pem

    and answer the questions. E.g.:

    Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Bayern
Locality Name (eg, city) []:Ismaning
Organization Name (eg, company) [Internet Widgits Pty Ltd]:pasta.net
Organizational Unit Name (eg, section) []: network
Common Name (e.g. server FQDN or YOUR name) []:
    <fqdn-mail-clients-use-to-connect-to-mail-server>
Email Address []:<my-personal-email-address>

    You can set the organisation name and unit name to yourself instead, if you want, e.g.:

    Organization Name (eg, company) [Internet Widgits Pty Ltd]:Alexis Huxley
Organizational Unit Name (eg, section) []: person

    If the common name does not match the name that clients use to connect to the mail server then mutt will complain:

    WARNING: Server hostname does not match certificate
  2. Make sure the key is readable only by root by running: 
    ls -ld /etc/ssl/private

    Expect output like this:

    drwx--x--- 2 root ssl-cert 4096 Nov  2 14:39 /etc/ssl/private
  3. If clients on the internet will use MX records to identify the mail server for your domain, then configure your DNS accordingly.
  4. If you have followed this procedure in order to regenerate a certificate then restart the dovecot service.

Dovecot

We set up Dovecot first because it provides user information to Postfix.

  1. Run: 
    apt-get install dovecot-core dovecot-imapd
  2. Stop dovecot with: 
    service dovecot stop
  3. Entirely replace /etc/dovecot/dovecot.conf with the following: 
    #  Don't listen on IPv6
listen = *

#  Encryption
ssl = required
ssl_cert = </etc/ssl/certs/mail.pem
ssl_key = </etc/ssl/private/mail.key
disable_plaintext_auth = no

#  Services provided to other programs
service auth {
    unix_listener /var/spool/postfix/private/auth {
        group = postfix
        mode = 0660
        user = postfix
    }
}

#  Protocols 
protocols = " imap"

#  Misc
mail_privileged_group = mail
mail_location = maildir:/var/mail/maildir/%u:INDEX=/var/mail/indexes/%u
  4. If you want to store mail on NFS but indexes on local disk in an environment where only one machine will access the NFS mount (I do!):
    1. Add the following to /etc/dovecot/dovecot.conf: 
      #  single-server NFS setup
mmap_disable = yes
mail_fsync = always
mail_nfs_storage = no
mail_nfs_index = no
    2. NFS mount the /var/mail/maildir. For me, this meant:
      1. on the storage server, add an entry to /etc/exports and run exportfs -av
      2. on the LDAP server, add a suitable entry to the auto.staging map (see here for details)
      3. on the mail server, replace /var/mail/maildir with a symlink to ../../staging/mail/maildir.
    3. Verify that the maildir and indexes directories exist and have permissions: 
      # ls -lL /var/mail/
drwxrwsrwt 3 root mail 4096 Jul 15 16:29 indexes
drwxrwsrwt 5 root mail   45 Sep  1  2018 maildir
#
  5. If you want to have Dovecot automatically add a Trash and Sent folder to mailboxes (I don’t!) then add the following to /etc/dovecot/dovecot.conf: 
    protocol imap {
    mail_plugins = " autocreate"
}
plugin {
    autocreate = Trash
    autosubscribe = Trash
    autocreate2 = Sent
    autosubscribe2 = Sent
}
  6. If you want to have Dovecot authenticate users and for Dovecot’s LDA to get their home directories from /etc/passwd or from NIS (I did but not any longer) then:
    1. Add the following to /etc/dovecot/dovecot.conf: 
      #  User and home directory lookup from files/NIS
userdb {
    driver = passwd
}

#  Authenticate via /etc/pam.d/dovecot, which will read files/NIS
passdb {
  driver = pam
  args = dovecot
}
  7. If you want to have Dovecot authenticate users and for Dovecot’s LDA to get their home directories from LDAP (I do!) then:
    1. Add the following to /etc/dovecot/dovecot.conf: 
      #  User and home directory lookup from LDAP
userdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

#  Authenticate via LDAP
passdb {
  driver = ldap
  args = /etc/dovecot/dovecot-ldap.conf
}
    2. Create /etc/dovecot/dovecot-ldap.conf containing: 
      #  Without these two, dovecot will log messages complaining
#  explicitly about these two
base = ou=Users,dc=pasta,dc=net
uris = ldap://192.168.1.21/

#  Without these two, dovecot syslogs "dovecot: imap-login: 
#  Aborted login (auth failed, ...): user=, method=PLAIN, 
#  rip=..., lip=..., TLS, session="
auth_bind = yes
auth_bind_userdn = uid=%u,ou=Users,dc=pasta,dc=net

#  If you want to set any additional user-specific attributes
#  (e.g. mail quota) then set ALL NEEDED user-specific 
#  attributes plus the additional ones.
user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid
    3. If the mail directories are to be automounted, then make sure there is a suitable entry in the automounter map; see here for details.
  8. Stop systemd managing the sockets: 
    systemctl mask dovecot.socket
  9. Restart stuff: 
    service dovecot restart
  10. From a machine on the same network check that you can read your mail via IMAP.
  11. From a machine outside the network check that you can read your mail via IMAP.

Postfix

  1. Run: 
    apt-get --purge install postfix postgrey
service postfix stop
  2. marille: Set some environment variables that will be used later in this procedure: 
    MY_FQHN=$(hostname -f)
MY_ROOTRCPT=alexis@pasta.net
MY_OTHERNAMES="mail.pasta.freemyip.com,pasta.freemyip.com"   #  Note comma-separated
RELAY_HOSTNAME=smtp.gmail.com
RELAY_AUTHENTICATE=yes
RELAY_LOGIN=alexishuxley
RELAY_PASSWD=<my-gmail-password>
  3. Edit /etc/postfix/main.cf and insert the following basic configuration: 
    ######################################################################
#
#  WHO AM I AND WHAT INTERFACES DO I LISTEN ON?
#
######################################################################

#  myhostname is mainly used to derive other stuff
myhostname = MY_FQHN
#  myhostname must be set and cannot be derived from $myorigin. On 
#  the other hand, myorigin defaults to $myhostname. Therefore set 
#  myhostname explicitly and leave myorigin unset. mydomain defaults
#  to the last parts of $myhostname.
#myorigin = ...
#mydomain = ...
#  Which interfaces to listen on for incoming mails.
inet_interfaces = all

######################################################################
#
#  FROM WHOM DO WE ACCEPT MAILS?
#
######################################################################

#  http://jimsun.linxnet.com/misc/restriction_order_prelim-03.txt says:
#
#	It was noted earlier that a match stops further processing of 
#       an access list, and of the restriction stage that "called" it.
#       But what does a restriction return when there's no match?  
#       Well, it returns "DUNNO."  ("I don't know, somebody else 
#       decide.")
#
#  and earlier says:
#
#	Postfix' restriction stages are as follows, and are processed
#       in the following order:
#	
#		smtpd_client_restrictions
#		smtpd_helo_restrictions
#		smtpd_sender_restrictions
#		smtpd_recipient_restrictions
#		smtpd_data_restrictions
#	
#	regardless of the order in which they're listed in main.cf.
#	
#	...
#	
#	Each restriction stage must evaluate to "OK" or "DUNNO" for 
#       processing to continue with the next stage.
#
#  So we're going to define a series of *_restrictions, which will be 
#  applied according to the above rules.
#
#  In fact, we need only one restriction: smtpd_recipient_restrictions.

#  Later we say that we accept mails from clients connecting to us from 
#  $mynetworks. Let that be derived from our interfaces' CIDRs.
mynetworks_style = subnet

#  Later we say that we accept mails from clients that do SASL 
#  authentication. Let us allow clients to authenticate this way 
#  (this still doesn't mean we'll accept their mails, only that we
#  allow them to authenticate) and say how we validate their attempts
#  to authenticate.
smtpd_sasl_auth_enable = yes 
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

#  In addition, we can perform some simple checks on the client's 
#  self-introducing 'HELO'. From what I've read, 'HELO' seems pretty
#  pointless  but at the same time a lot of spammers either omit this
#  or set it to the that name of the server the client is connecting 
#  *to*! I'm going to try commenting this out to see what happens! It 
#  seems to be not very clear what happens when a client doesn't match
#  any of the restrictions. More info at
#  http://www.unixwiz.net/techtips/postfix-HELO.html.
#smtpd_helo_required = yes
#smtpd_helo_restrictions = permit_sasl_authenticated

#  And finally here is the list of who we will accept mails from
smtpd_recipient_restrictions = permit_sasl_authenticated, 
    permit_mynetworks, 
    reject_unauth_destination, 
    reject_rbl_client zen.spamhaus.org, 
    reject_rbl_client bl.spamcop.net, 
    check_policy_service inet:127.0.0.1:10023, 
    permit

######################################################################
#
#  IN-BOUND ENCRYPTION
#
######################################################################

smtpd_use_tls = yes
#  Offer encryption, but do not require it (requiring would contravene
#  RFC2487)
smtpd_tls_security_level = may
smtpd_tls_cert_file = /etc/ssl/certs/mail.pem
smtpd_tls_key_file = /etc/ssl/private/mail.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
#  Poodle protection
smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3
smtpd_tls_protocols = !SSLv2 !SSLv3

######################################################################
#
#  OUT-BOUND ENCRYPTION
#
######################################################################

#  Gmail requires next two lines; others probably don't care
smtp_use_tls = yes
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
#  Poodle protection
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3
smtp_tls_protocols = !SSLv2 !SSLv3
#  Our relay (gmail.com) requires us to to SASL authwenticate.
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options =

######################################################################
#
#  ADDRESS REWRITING
#
######################################################################

#  If an address matches *.$mydomain then change it to $mydomain. 
masquerade_domains = $mydomain
#  Which addresses to we masquerade?
masquerade_classes = envelope_sender, header_sender, 
    header_recipient, envelope_recipient
#  Adjust incomplete addresses for mails generated locally.
local_header_rewrite_clients = permit_mynetworks
#  Mail redirection for mail we've not yet decided is remote or local
#  (alias_maps is only for mail we've decided is for local delivery).
virtual_alias_maps = hash:/etc/postfix/virtual

######################################################################
#
#  WHICH DOMAINS ARE REMOTE AND WHICH ARE LOCAL?
#
######################################################################

#  Local is localhost, our private domain name and various public 
#  names we might be known as.
mydestination = localhost, $mydomain, MY_OTHERNAMES

######################################################################
#
#  HOW TO DO REMOTE DELIVERY
#
######################################################################

relayhost = RELAY_HOSTNAME
#  why?
smtp_host_lookup = native     

######################################################################
#
#  HOW TO DO LOCAL DELIVERY
#
######################################################################

#  Note aliases are consulted by LDA (so never consulted if everything
#  relayed)
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mailbox_size_limit = 0
mailbox_size_limit = 0
mailbox_command = /usr/lib/dovecot/deliver

######################################################################
#
#  MISCELLANEOUS
#
######################################################################

recipient_delimiter = +
inet_protocols = ipv4
append_dot_mydomain = no
compatibility_level = 2
#debug_peer_level = 10
#debug_peer_list = 80.187.84.82
  4. Run: 
    rm -f /etc/mailname
echo "root: MY_ROOTRCPT" > /etc/aliases
> /etc/postfix/virtual
> /etc/postfix/sasl_passwd
[ $RELAY_AUTHENTICATE = yes ] && \
    echo "RELAY_HOSTNAME RELAY_LOGIN:RELAY_PASSWD" \
    >> /etc/postfix/sasl_passwd
  5. Replace markers: 
    perl -pi -e "s/MY_FQHN/$MY_FQHN/g; \
             s/RELAY_HOSTNAME/$RELAY_HOSTNAME/g; \
             s/RELAY_AUTHENTICATE/$RELAY_AUTHENTICATE/g; \
             s/RELAY_LOGIN/$RELAY_LOGIN/g;
           \ s/RELAY_PASSWD/$RELAY_PASSWD/g; \
             s/MY_OTHERNAMES/${MY_OTHERNAMES// /,}/g; \
             s/MY_ROOTRCPT/${MY_ROOTRCPT/@/\\@}/g" \
    /etc/postfix/main.cf /etc/aliases \
    /etc/postfix/virtual /etc/postfix/sasl_passwd
  6. Update some databases that use the files that had markers: 
    newaliases
postmap hash:/etc/postfix/virtual
postmap hash:/etc/postfix/sasl_passwd
service postfix stop
service postfix start
  7. If you need to change your DNS database to point other systems on the network to this mail server, then do so now.
  8. If you have rules on your firewall for incoming mail to update, then update them now.
  9. When sending personal mail from work, it may be that work’s mail servers block access to any remote host on port 25. Therefore:
    1. Edit /etc/postfix/master.cf and uncomment the line: 
      smtps     inet  n       -       y       -       -       smtpd
    2. Rerun: 
      service postfix restart
    3. Open port 465 on the firewall.
    4. Android K-9 needs to be told to connect to the SMTP server using STARTTLS on port 465.
  10. Do lots of tests!
  11. If appropriate, add an entry for webmaster to /etc/aliases and run: 
    newaliases

Mailman

  1. Install a basic https webserver as follows:
    1. Run: 
      apt-get install apache2
a2enmod cgid remoteip ssl
a2ensite default-ssl
a2dissite 000-default
    2. Edit /etc/apache2/sites-available/default-ssl.conf to contain only: 
      <IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        <FilesMatch "\.(cgi|shtml|phtml|php)$">
                SSLOptions +StdEnvVars
        </FilesMatch>
        <Directory /usr/lib/cgi-bin>
                SSLOptions +StdEnvVars
        </Directory>
#        ScriptAlias /cgi-bin/mailman/ /usr/lib/cgi-bin/mailman/
#        Alias /pipermail/ /var/lib/mailman/archives/public/
#        Alias /images/mailman/ /usr/share/images/mailman/
#        <Directory /usr/lib/cgi-bin/mailman/>
#            AllowOverride None
#            Options ExecCGI
#            AddHandler cgi-script .cgi
#            Require all granted
#        </Directory>
#        <Directory /var/lib/mailman/archives/public/>
#            Options FollowSymlinks
#            AllowOverride None
#            Require all granted
#        </Directory>
#        <Directory /usr/share/images/mailman/>
#            AllowOverride None
#            Require all granted
#        </Directory>
    </VirtualHost>
</IfModule>
    3. Edit /etc/apache2/ports.conf and comment out: 
      #Listen 80
    4. Configure logging of client IPs on backend vhosts.
    5. Run: 
      systemctl restart apache2
  2. Run: 
    apt-get install mailman
newlist mailman

    and follow the prompts regarding created the required ‘mailman’ mailing list. (Stupidly, mailman won’t work if this list does not exist.)

  3. Edit /etc/mailman/mm_cfg.py and set the following variable (values shown are thise that my site requires): 
    DEFAULT_URL_PATTERN = 'https://%s/cgi-bin/mailman/'
DEFAULT_EMAIL_HOST = 'pasta.freemyip.com'
DEFAULT_URL_HOST = 'mail.pasta.freemyip.com'
  4. Uncomment the commented-out section in /etc/apache2/sites-available/default-ssl.conf and run: 
    systemctl reload apache2
  5. Edit /var/www/html/index.html to contain only: 
    <html>
<body>
<ul>
<li><a href="/cgi-bin/mailman/listinfo/mailman">Mailman subscribe</a></li>
<li><a href="/cgi-bin/mailman/admin/mailman">Mailman admin</a></li>
<li><a href="/cgi-bin/mailman/listinfo/filmnight">FilmNight subscribe</a></li>
<li><a href="/cgi-bin/mailman/admin/filmnight">FilmNight admin</a></li>
</ul>
</body>
</html>
  6. If there is a front-end reverse proxy vhost, then add a stanza like this to its configuration: 
    ProxyPass / https://<backend>/
ProxyPassReverse / https://<backend>/
  7. Check that the web interface is accessible by visiting http://<default-url-host>/cgi-bin/mailman/listinfo (the links there won’t work yet).
  8. Important note: if <mailserver-host> differs from <default-url-host> then http://<mailserver-host>/cgi-bin/mailman/listinfo shows no lists, whereas the first URL shows the lists! In my own case, this means I cannot access http://fusilli/cgi-bin/mailman/listinfo, but must set up the proxy (see below) and then access: https://mail.pasta.freemyip.com/cgi-bin/mailman/listinfo.
  9. If you have mailing lists to migrate then migrate each of them as follows:
    1. Create the list by running: 
      newlist <list-name>

      and follow the prompts.

    2. Add suitable entries to the file /var/www/html/index.html.
    3. On the old mailserver determine the directories to copy across by running: 
      find /var/lib/mailman -name <list-name>* -type d -prune
    4. Copy those directories across.
  10. Make a test posting.

See also