Cleaning up how Postfix and Dovecot connect to LDAP

Introduction

A passwd entry is a database record that contains some basic information about a user. Traditionally this information included the user’s password, which is why a passwd entry is called a passwd entry, but these days the password itself is typically stored somewhere more secure.

Consequently, passwd entries can be used for listing or validating users but cannot be used for authenticating users.

Both Dovecot and Postfix must deal with this dichotomy.

This article:

  • describes the very misaligned approaches that Dovecot and Postfix take to address this dichotomy, with special attention to using LDAP as the database backend
  • suggests a more aligned approach
  • mentions various error messages encountered along the way and how to resolve them
  • discusses a couple of other unrelated questions that came up along the way

A common Dovecot+Postfix configuration with LDAP support

On networks with real Unix accounts for each mail user, a common Postfix+Dovecot configuration is:

  1. Dovecot validates and authenticates users via LDAP
  2. Dovecot offers an LDAP user authentication service to Postfix
  3. Postfix authenticates users by using the service provided by Dovecot
  4. Postfix validates users via LDAP (using OS-level LDAP services)

The first and last points need some clarification.

Firstly, Dovecot may need to look up the user’s passwd entry or authenticate a user for several reasons:

  • check if the user is authorised to read the mail that the user is asking Dovecot to retrieve via IMAP
  • maybe to get the user’s UID so that it can set the owner of the files into which it moves mails

We tell Dovecot to use LDAP as follows:

mandala# fgrep -B 1 -A 1 ldap /etc/dovecot/dovecot.conf 
userdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}
--
passdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

and we tell Dovecot the LDAP connection parameters as follows:

mandala# egrep -v '^ *(#|$)' /etc/dovecot/dovecot-ldap.conf 
base = ou=Users,dc=pasta,dc=net
uris = ldaps://ldap.pasta.freemyip.com/
auth_bind = yes
auth_bind_userdn = uid=%u,ou=Users,dc=pasta,dc=net
tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt
...

With this information, Dovecot can establish a TCP connection to the LDAP server and make its query.

Additionally, we tell Dovecot how to hone its query and how to interpret the results:

...
user_filter = (&(objectClass=posixAccount)(uid=%n))
user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid
mandala#

Secondly, Postfix may need to look up the user’s passwd entry or authenticate a user for several reasons:

  • to check if a user exists
  • to check what is in the user’s .forward file
  • to authenticate a remote user who is attempting to relay mail via Postfi
  • maybe to save incoming mail into an INBOX in the user’s home

In the common Dovecot+Postfix configuration outlined earlier, Postfix delegates looking up a user’s password entry to either NSS or PAM and delegates authenticating a user to Dovecot.

The delegation to NSS or PAM are very similar, consisting of a series of functions calling other functions calling other functions with the last function connecting to an LDAP server, submitting a query, fetching results and passing them back up the call stack. I am going to describe the NSS route because I am more familiar with it and it is easier to follow.

The C standard libary contains a few functions for looking up passwd entries:

We would expect that the Postfix source code contains calls to some of these and we can confirm this by running:

mandala# dpkg -L postfix | xargs nm -D 2>/dev/null \
    sort -u | grep getpw
U getpwnam@GLIBC_2.2.5
U getpwnam_r@GLIBC_2.2.5
U getpwuid@GLIBC_2.2.5
U getpwuid_r@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
U getpwuid@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
mandala#

That command is listing the contents of the postfix package – including directories, binaries, man pages, READMEs – and searching all of them for calls to functions containing ‘getpw‘ in their names. That search is obviously going to produce errors when it looks for calls inside directories, man pages and READMEs, so we discard any error messages. We are left with:

  • getpwnam() retrieves a passwd entry for a specified username
  • getpwuid() retrieves a passwd entry for a specified numeric user ID
  • The *_r() functions provide the same functionality but are more suited to threaded environments

We will follow the route taken by getpwnam(), but similar routes would apply to the other functions.

We have already established that Postfix calls getpwnam() and we can reasonably assume that it does this for the reasons described above. Note that “reasonably assume”; using nm to examine the functions that a program or library can call does not prove that the program or library actually calls a particular function, but it does strongly suggest that – in some circumstances – it does.

So then getpwnam() looks in /etc/nsswitch.conf to see where passwd entries could be stored and finds this:

mandala# grep passwd /etc/nsswitch.conf 
passwd: files ldap
mandala#

So then getpwnam() checks if the user is mentioned in the local /etc/passwd file by delegating to this function:

mandala# nm -D /lib/x86_64-linux-gnu/libnss_files.so \
    | grep getpwnam
0000000000006770 T _nss_files_getpwnam_r@@GLIBC_PRIVATE
mandala#

and then – assuming the user it is looking up is not listed in /etc/passwd because it is listed in LDAP instead – it checks in LDAP by delegating to this function:

mandala# nm -D /lib/x86_64-linux-gnu/libnss_ldap.so.2 \
    | grep getpwnam
0000000000006900 T _nss_ldap_getpwnam_r@@EXPORTED
mandala#

nss_ldap_getpwnam_r() delegates to nslcd(8),the local LDAP name service daemon, which does LDAP queries on behalf of local processes:

mandala# ps -C nslcd
PID TTY TIME CMD
1075 ? 00:00:01 nslcd
mandala#

nss_ldap_getpwnam_r() probably communicates with nslcd through one of these sockets:

mandala# lsof -p 1075 -E | grep unix
nslcd 1075 nslcd 5u unix 0x0000000092a56bcc 0t0 18974 type=DGRAM ->INO=11198 229,systemd-j,6u 1,systemd,122u
nslcd 1075 nslcd 6u unix 0x00000000fb6e0ef7 0t0 18989 /var/run/nslcd/socket type=STREAM
mandala#

nslcd looks – or rather it looked when it was first started – in /etc/ldap/ldap.conf to get the LDAP connection parameters:

mandala# egrep -v '^ *($|#)' /etc/nslcd.conf 
uid nslcd
gid nslcd
uri ldaps://ldap.pasta.freemyip.com/
base dc=pasta,dc=net
tls_cacertfile /etc/ssl/certs/ca-certificates.crt
mandala#

and connects to the LDAP server and forwards it the query.

The LDAP server replies to nslcd, which replies to nss_ldap_getpwnam_r(), which replies to getpwnam(), which replies to Postfix.

So what’s wrong with that and how can we fix it?

  • Postfix delegates some LDAP communication (i.e. user authentication) to Dovecot, but not all of it (i.e. user validation), meaning that two different stacks need to be configured
  • Dovecot does not at all delegate LDAP communication
  • meaning that in total three different stacks need to be configured

Can it be simplified?

  • Can Postfix and Dovecot be configured to access LDAP via NSS?
  • Can Postfix and Dovecot be configured to access LDAP directly?
  • Can Postfix be configured to delegate all LDAP communication to Dovecot?

We will consider each of these options in the next sections.

Can Postfix and Dovecot be configured to access LDAP via NSS?

I would guess that making Postfix not delegate user authentication to Dovecot, by removing these lines from /etc/postfix/main.cf:

smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

would probably be enough to make it use NSS for user authentication instead.

However, making Dovecot use NSS is a bigger problem. The userdb documentation leads to the passwd authentication documentation, which suggests it is possible, by setting:

userdb {
    driver = passwd
    args = blocking=no
}

but the same page says that passwd cannot be used for passdb, because of the splitting out of the password into /etc/shadow.

There is the alternative nss driver, which is probably better (because probably the passwd driver goes straight to /etc/passwd, whereas nss will go to /etc/nsswitch.conf). It is documented here  but that documentation says that a service parameter is required, like this:

userdb {
    driver = nss
    args = service=ldap
}

But we do not want to tell Dovecot to use LDAP; we want to tell it to use NSS (and for nss to look in /etc/nsswitch.conf and discover that I want it to use LDAP). I could not find any documention regarding what other args = server=whatever options were possible. If we omit args=service=ldap  and duplicate the stanza for the passdb, i.e.:

userdb {
    driver = nss
}

passdb {
    driver = nss
}

then starting up mutt results in Dovecot complaining:

Dec 15 09:42:20 mandala dovecot: auth: Fatal: Unknown passdb driver 'nss'

This passdb page lists (rather obtusely) the possible passdb drivers. The equivalent of userdb/passwd seems to be passdb/shadow but the shadow authentication page says that can be problematic and that driver pam is preferred.

The PAM page makes for pessmistic reading but the file it refers to – /etc/pam.d/dovecot, which goes on to include /etc/pam.d/common-auth – actually looks quite suitable:

mandala# egrep -v '^(#|$)' /etc/pam.d/common-auth 
auth [success=2 default=ignore] pam_unix.so nullok
auth [success=1 default=ignore] pam_ldap.so minimum_uid=1000 use_first_pass
auth requisite pam_deny.so
auth required pam_permit.so
mandala#

But rather inconsistently there is no pam driver for the userdb database. The closest seems to the nss driver. So let’s try:

userdb {
    driver = nss
    #args = service=ldap
}

passdb {
    driver = pam
    args = failure_show_msg=yes
}

which causes:

Dec 15 10:02:39 mandala dovecot: auth: Fatal: Unknown userdb driver 'nss'

So NSS is not an option in Debian 11’s Dovecot for either userdb or passdb.

What other userdb drivers would force Dovecot to delegate to the OS rather than going directly to LDAP? The userdb page does not list pam as a possibility but it does say:

The user and password databases … may be the same or they may be different depending on your needs.

If we try userdb/pam, then that fails with:

Dec 15 10:06:07 mandala dovecot: auth: Fatal: Unknown userdb driver 'pam'

So I conclude that both Postfix and Dovecot cannot be configured to access LDAP via NSS.

Can Postfix and Dovecot be configured to access LDAP directly?

We revert Dovecot to our original configuration:

userdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

passdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

with /etc/dovecot/dovecot-ldap.conf specifying how to connect to the LDAP server, hone queries and filter results, as discussed earlier.

Can we make Postfix talk directly to the LDAP server in the same way?

The postconf(5) man page does not mention accessing LDAP directly.

So I conclude that both Postfix and Dovecot cannot be configured to access LDAP directly.

A digression

While reading postconf(5) I saw this:

smtpd_relay_restrictions …

With Postfix versions before 2.10, the rules for relay permission and spam blocking were combined under smtpd_recipient_restrictions, resulting in error-prone configuration. As of Postfix 2.10, relay permission rules are preferably implemented with smtpd_relay_restrictions, so that a permissive spam blocking policy under smtpd_recipient_restrictions will no longer result in a permissive mail relay policy.

which looks like it might be useful!

Currently my smtp_recipient_restrictions are:

smtpd_recipient_restrictions = 
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

But this could be split this into:

smtpd_relay_restrictions = 
    permit_sasl_authenticated,
    permit_mynetworks,
    reject

and:

smtpd_recipient_restrictions = 
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    reject_unverified_recipient,
    check_policy_service inet:127.0.0.1:10023,
    permit

Note that the reject_unverified_recipient necessitates Postfix delegating to Dovecot and will result in a mail being rejected if the local part of the address does not match any known user (in LDAP).

Can Postfix be configured to delegate all LDAP communication to Dovecot?

I googled ‘postfix validate local address dovecot’ and that led here, which says:

With Dovecot 2.0 you can also use LMTP and the Postfix setting “reject_unverified_recipient” for dynamic address verification. It is really nice because Postfix does not need to query an external datasource (MySQL, LDAP…).

which sounds like what I want! I.e. stop Postfix from using (OS-level) LDAP and get it to use Dovecot.

Dovecot’s page about LMTP says:

LMTP uses the same settings as LDA ( See Common configuration), as specified in conf.d/15-lda.conf in example configuration. There is also a bit of extra configuration in conf.d/20-lmtp.conf.

I installed dovecot-lmtp package.

No settings are applied by default:

mandala# egrep -v '^ *(#|$)' /etc/dovecot/conf.d/15-lda.conf 
protocol lda {
}
mandala# egrep -v '^ *(#|$)' /usr/share/dovecot/conf.d/20-lmtp.conf
protocol lmtp {
}
mandala#

I.e. no settings were applied by default.

I modified dovecot.conf:

protocols = " ... lmtp"

service lmtp {
    unix_listener /var/spool/postfix/private/dovecot-lmtp {
        group = postfix
        mode = 0600
        user = postfix
    }
}

and also in /etc/postfix/main.cf:

# This was the old setting
#mailbox_command = /usr/lib/dovecot/deliver
# This is the new setting 
mailbox_transport = lmtp:unix:private/dovecot-lmtp

I restart Dovecot and Postfix.

My tests are as follows:

  • mutt/imap: can Mutt open an IMAP connection to the Dovecot server, authenticate itself and read my inbox?
  • cercis/mpcdf: can a remote SMTP client (cercis) authenticate itself with the Postfix server and upload a mail for a remote address (mpcdf) and is that mail delivered succesfully?
  • mpcdf/alexis: can a remote SMTP client (mpcdf) send mail to a valid user on my mail server (alexis@my-mail-server’s-internet-facing-name)

The test results were:

  • mutt/imap: ok
  • cercis/mpcdf: failed: 
    Dec 15 11:45:04 mandala postfix/smtpd[10119]: NOQUEUE: reject: RCPT from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]: 554 5.7.1 Service unavailable; Client host [88.111.68.216] blocked using zen.spamhaus.org; https://www.spamhaus.org/query/ip/88.111.68.216; from=<root@cercis.freemyip.com> to=<alexis.huxley@mpcdf.mpg.de> proto=ESMTP helo=<cercis.freemyip.com>
  • mpcdf/alexis: skipped

The digression revisited

By removing the two reject_rbl_client directives in the setting of smtpd_recipient_restrictions, I established that this mail that should be relayed is not being checked against smtpd_relay_restrictions, but rather it is checked against smtpd_recipient_restrictions, which is wrong: since (a) cercis is sending mail to alexis.huxley@mpcdf.mpg.de and (b) that domain is not in mydestination, then the relay rule should apply.

I found somebody asking for help with the same problem but no solution is offered.

Debian 11’s) postconf(5) suggests that that person and I are right to be puzzled:

smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination)

Access restrictions for mail relay control that the Postfix SMTP server applies in the context of the RCPT TO command, before  smtpd_recipient_restrictions. See SMTPD_ACCESS_README, section “Delayed evaluation of SMTP access restriction lists” for a discussion of evaluation context and time.

Note that “before”.

Googling ‘smtpd_recipient_restrictions processed before smtp_relay_restrictions‘ sent me to this page, which includes:

smtpd_relay_before_recipient_restrictions (default: see “postconf -d” output)

Evaluate smtpd_relay_restrictions before smtpd_recipient_restrictions. Historically, smtpd_relay_restrictions was evaluated after smtpd_recipient_restrictions, contradicting documented behavior.

Background: the smtpd_relay_restrictions feature is primarily designed to enforce a mail relaying policy, while smtpd_recipient_restrictions is primarily designed to enforce spam blocking policy. Both are evaluated while replying to the RCPT TO command, and both support the same features.

This feature is available in Postfix 3.6 and later.

but:

mandala# postconf -d | grep smtpd_recipient_restrictions
mandala#

and indeed, by Postfix is too old to have this setting:

mandala# dpkg -l | grep postfix
ii postfix 3.5.6-1+b1 amd64 High-performance mail transport agent
mandala#

So the meaning of the two directives is:

  • smtpd_recipient_restrictions: restrictions for local and relayed mail
  • smtpd_relay_restrictions: additional restrictions only for relayed mail

Are there any other smtp_*_restrictions that might be helpful? Perhaps something like smtp_notrelay_restrictions?

mandala# man 5 postconf | grep '^smtpd_[^ ]*_restrictions'
smtpd_client_restrictions (default: empty)
smtpd_data_restrictions (default: empty)
smtpd_end_of_data_restrictions (default: empty)
smtpd_etrn_restrictions (default: empty)
smtpd_helo_restrictions (default: empty)
smtpd_recipient_restrictions (default: see postconf -d output)
smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination)
smtpd_sender_restrictions (default: empty)
mandala#

smtpd_client_restrictions would apply to google, not to a mail sent by a spammer to alexishuxley@gmail.com, so I think it is not useful.

smtpd_sender_restrictions would not help as senders are commonly faked.

How about I revert to using only smtpd_recipient_restrictions but now with the understanding that it’s also for relayed mail?

# Restrictions for local *and* relayed mail.
smtpd_recipient_restrictions = 
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unverified_recipient,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

# No *additional* restrictions for relaying.
#smtpd_relay_restrictions =

I ran my tests:

  • mutt/imap: ok
  • cercis/mpcdf: ok
  • mpcdf/alexis: failed: 
    Dec 15 13:48:07 mandala postfix/smtpd[13116]: NOQUEUE: reject: RCPT from a1962.mx.srv.dfn.de[194.95.232.160]: 450 4.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis@pasta.freemyip.com> User doesn't exist: alexis@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<a1962.mx.srv.dfn.de>

It looks like lmtp needs to be told either to drop the ‘@’ sign and fully qualified domain name (FQDN) when it checks if the user is in LDAP, or it needs to be told what FQDNs are valid.

Googling ‘dovecot lmtp drop domain’ turned up a question on stackexchange,  which suggests either to set auth_username_format in /etc/dovecot/conf.d/10-auth.conf:

auth_username_format = %Ln

or to set user_filter in /etc/dovecot/dovecot-ldap.conf to set:

user_filter = (&(objectClass=posixAccount)(uid=%n)

The auth_username_format’s documentation says:

auth_username_format

Default: %Lu
Values: String

Formatting applied to username before querying the auth database.
You can use the standard variables here.

Examples:

%Lu – lowercases the username
%n – drops the domain if one was supplied
%n-AT-%d – changes the “@” symbol into “-AT-” before lookup

This translation is done after the changes specified with the auth_username_translation setting.

and the user_filter’s documentation  says:

user_filter

Default: <empty>

Values: String

Filter for user lookup (userdb lookup). See also LDAP Backend Configuration
Below variables can be used.

Variable Long name Description
%u %{user} username
%n %{username} user part in user@domain, same as %u if there’s no domain
%d %{domain} domain part in user@domain, empty if user there’s no domain

See Config Variables for full list

Example:
user_filter = (&(objectClass=posixAccount)(uid=%u))

The former looks better because it takes effect slightly earlier. So I added the following to dovecot.conf:

auth_username_format = %Ln

and restarted Dovecot and re-ran my tests:

  • mutt/imap: ok
  • cercis/mpcdf: ok
  • mpcdf/alexis: failed 
    Dec 15 14:19:10 mandala postfix/smtpd[13394]: NOQUEUE: reject: RCPT from c1962.mx.srv.dfn.de[194.95.238.160]: 450 4.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis@pasta.freemyip.com> User doesn't exist: alexis@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<c1962.mx.srv.dfn.de>

Since that did not work, I reverted that change to dovecot.conf.

It looks like lmtp is still looking up the wrong user. Perhaps auth_username_format does not apply when the username is coming from Postfix rather than from an IMAP client?

So then I tried the alternative. I changed the following in  dovecot-ldap.conf:

#user_filter = uid=%u
user_filter = (&(objectClass=posixAccount)(uid=%n))

and restarted Dovecot and re-ran my tests:

  • mutt/imap: ok
  • cercis/mpcdf: ok
  • mpcdf/alexis: failed 
    Dec 15 14:24:07 mandala postfix/smtpd[13436]: NOQUEUE: reject: RCPT from a1962.mx.srv.dfn.de[194.95.232.160]: 450 4.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis@pasta.freemyip.com> User doesn't exist: alexis@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<a1962.mx.srv.dfn.de>

Since that did not work, I reverted the change to dovecot-ldap.conf.

After some googling, I found a post on the Dovecot mailing list that described running doveadm -u user <user>  to test how Dovecot handles different addresses. So I commented out the setting of auth_username_format altogether, restarted Dovecot and ran:

mandala# doveadm user -u alexis@pasta.freemyip.com
userdb lookup: user alexis@pasta.freemyip.com doesn't exist
mandala#

This error was to be expected with the changes I just made.

So then I set:

auth_username_format = %Ln

restarted Dovecot and run the doveadm command and now it works:

mandala# doveadm user -u alexis@pasta.freemyip.com
userdb: alexis@pasta.freemyip.com
user : alexis
home : /home/alexis
uid : 1000
gid : 1000
mandala#

So although when I tested this setting of auth_username_format above, all my tests failed, I do now know that I am on the right track!

Perhaps private/dovecot-lmtp says alexis@pasta.freemyip.com not only in the message but also when it talks to the LDAP server?

As a stupid test, on my LDAP server orzo, I cloned the uid=alexis entry to uid=alexis@pasta.freemyip.com:

orzo# shelldap 
~ > cd ou=Users 
ou=Users,~ > ls
- uid=alexis
- uid=cercis
- uid=guest
- uid=repomaster
- uid=suzie
ou=Users,~ > cp uid=alexis uid=alexis@pasta.freemyip.com
Success
ou=Users,~ >

and then I retried (no need to restart Dovecot), but it made no difference: the test still failed.

Reading further in the same link I ran:

mandala# doveadm auth test alexis@pasta.freemyip.com
Password: 
passdb: alexis@pasta.freemyip.com auth succeeded
extra fields:
user=alexis
original_user=alexis@pasta.freemyip.com
mandala#

At this point I felt quite stuck.

Can the Dovecot mailing list help?

Hi, I’m trying to set up Postfix +Dovecot, with Postfix delegating to Dovecot for authentication (with LDAP backend), local user validation (again with LDAP backend) and local email delivery. It’s partly working but ultimately Dovecot reports “User doesn’t exist” with the user mentioned having user@domain format:

Dec 15 15:24:49 mandala postfix/smtpd[13945]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 450 4.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis@pasta.freemyip.com> User doesn't exist: alexis@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>

(pasta.freemyip.com is my mailserver’s internet-facing name and name for which I have a LetsEncrypt SSL certificate; mandala.pasta.net is its name on my pasta.net home network; alexis.huxley@mpcdf.mpg.de is from where I sent a test mail, via mailservers at dfn.de.)

However, if I just ask doveadm to check if the user exists then it is okay:

mandala# doveadm user -u alexis@pasta.freemyip.com
userdb: alexis@pasta.freemyip.com
user : alexis
home : /home/alexis
uid : 1000
gid : 1000
mandala#

Similarly with auth test:

mandala# doveadm auth test alexis@pasta.freemyip.com
Password: 
passdb: alexis@pasta.freemyip.com auth succeeded
extra fields:
user=alexis
original_user=alexis@pasta.freemyip.com
mandala#

My IMAP client can connect without problems.

As can external clients that authenticate in order to relay mails.

I had already added two fixes to drop the @domain from the user name; this in the dovecot-ldap.conf

user_filter = (&(objectClass=posixAccount)(uid=%n))
pass_filter = (&(objectClass=posixAccount)(uid=%n))

and this in dovecot.conf:

auth_username_format = %n

I believe that *either* of those fixes makes doveadm work and *should* make Dovecot work too [1] but still it does not.

The relevant parts of main.cf are:

...
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
...
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unverified_recipient,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit
...
mailbox_transport = lmtp:unix:private/dovecot-lmtp
...

(I know about the preferred separation between smtpd_recipient_restrictions and smtpd_relay_restrictions, but this is Postfix 3.5.6 and that consults smtpd_recipient_restrictions before smtpd_relay_restrictions, and is not overrulable, making smtpd_relay_restrictions almost pointless [2] [3].)

I’ve reach the tearing-my-hair-out stage. Could anybody advise please?

Version and config info below.

Many thanks!

Alexis

[1] https://serverfault.com/questions/658703/postfix-%E2%86%92-dovecot-lmtp-user-does-not-exist-uiddomain
[2] https://list.postfix.users.narkive.com/DoyVr83m/postfix-3-2-0-during-the-processing-of-the-relays-applies-the-rules-from-smtpd-relay-restrictions
[3] from http://www.postfix.org/postconf.5.html:
smtpd_relay_before_recipient_restrictions (default: see “postconf -d” output)

Evaluate smtpd_relay_restrictions before smtpd_recipient_restrictions. Historically, smtpd_relay_restrictions was evaluated after smtpd_recipient_restrictions, contradicting documented behavior.

Background: the smtpd_relay_restrictions feature is primarily designed to enforce a mail relaying policy, while smtpd_recipient_restrictions is primarily designed to enforce spam blocking policy. Both are evaluated while replying to the RCPT TO command, and both support the same features.

This feature is available in Postfix 3.6 and later.

mandala# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 11 (bullseye)
Release: 11
Codename: bullseye
mandala# dpkg -l | egrep '(postfix|dovecot)'
ii dovecot-core 1:2.3.13+dfsg1-2 amd64 secure POP3/IMAP server - core files
ii dovecot-imapd 1:2.3.13+dfsg1-2 amd64 secure POP3/IMAP server - IMAP daemon
ii dovecot-ldap 1:2.3.13+dfsg1-2 amd64 secure POP3/IMAP server - LDAP support
ii dovecot-lmtpd 1:2.3.13+dfsg1-2 amd64 secure POP3/IMAP server - LMTP server
ii postfix 3.5.6-1+b1 amd64 High-performance mail transport agent
mandala# dovecot -n
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-9-amd64 x86_64 Debian 11.1 ext4
# Hostname: mandala.pasta.net
auth_username_format = %n
disable_plaintext_auth = no
listen = *
mail_location = maildir:/var/mail/maildir/%u:INDEX=/var/mail/indexes/%u
mail_privileged_group = mail
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
protocols = imap lmtp
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
ssl = required
ssl_cert = </etc/letsencrypt/live/mail.pasta.freemyip.com/fullchain.pem
ssl_key = # hidden, use -P to show it
userdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
mandala# 
mandala# egrep -v '^ *(#|$)' /etc/dovecot/dovecot-ldap.conf
base = ou=Users,dc=pasta,dc=net
uris = ldaps://ldap.pasta.freemyip.com/
auth_bind = yes
auth_bind_userdn = uid=%u,ou=Users,dc=pasta,dc=net
user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid
user_filter = (&(objectClass=posixAccount)(uid=%n))
pass_filter = (&(objectClass=posixAccount)(uid=%n))
mandala# 
mandala# egrep -v '^ *(#|$)' /etc/postfix/main.cf
myhostname = mandala.pasta.net
inet_interfaces = all
mynetworks_style = subnet
smtpd_sasl_auth_enable = yes 
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_recipient_restrictions =
permit_sasl_authenticated,
permit_mynetworks,
reject_unverified_recipient,
reject_rbl_client zen.spamhaus.org,
reject_rbl_client bl.spamcop.net,
check_policy_service inet:127.0.0.1:10023,
permit
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.pasta.freemyip.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.pasta.freemyip.com/privkey.pem
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3
smtpd_tls_protocols = !SSLv2 !SSLv3
smtp_use_tls = yes
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3
smtp_tls_protocols = !SSLv2 !SSLv3
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options =
masquerade_domains = $mydomain
masquerade_classes = envelope_sender, header_sender, 
header_recipient, envelope_recipient
local_header_rewrite_clients = permit_mynetworks
virtual_alias_maps = hash:/etc/postfix/virtual
mydestination = localhost, $mydomain, mail.pasta.freemyip.com,
pasta.freemyip.com
relayhost = smtp.gmail.com
smtp_host_lookup = native 
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mailbox_size_limit = 0
mailbox_transport = lmtp:unix:private/dovecot-lmtp
recipient_delimiter = +
inet_protocols = ipv4
append_dot_mydomain = no
compatibility_level = 9999
message_size_limit = 102400000
mandala#

Does this configuration necessitate virtual users?

Before I posted that it dawned on my that ‘virtual’ (a term bandied about in the Postfix documentation), could refer to users that do not have corresponding Unix accounts, which is the case when I stop nslcd. However, after more thought, I concluded:

  1. virtual was going to be much too complicated (there are so many different directives for main.cf that contain ‘virtual’!)
  2. probably I would have to tell Postfix how to contact LDAP itself (or at OS level) and one of the points of this whole procedure was to get Postfix to always delegate interactions with LDAP to Dovecot
  3. telling Postfix to delegate to Dovecot only for user validation and user authentication is probably not sufficient (perhaps Postfix wants to examine a user’s .forward file and so wants to know where ~user is).

So I think virtual users is not a viable solution.

Which part of Postfix is it that is complaining that it cannot find a user?

The last error message was:

Dec 16 09:40:17 mandala postfix/smtpd[1512]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 550 5.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: User unknown in local recipient table; from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>

It’s smtpd deciding that there is no local user! The recipient restriction that says to ask dovecot/lmtp did not reject the mail, so Dovecot understood that this is a local user (or a mail to be relayed, which would not cause it to reject it). Googling that message ‘User unknown in local recipient table’ turns up some Postfix documentation that says:

As of Postfix version 2.0, the Postfix SMTP server rejects mail for unknown recipients in local domains (domains that match $mydestination or the IP addresses in $inet_interfaces or $proxy_interfaces) with “User unknown in local recipient table”. This feature was optional with earlier Postfix versions.

So smtpd is validating the local user. With nslcd stopped it fails but it nslcd running it succeeds.

Reading further at that link:

The local_recipient_maps parameter specifies lookup tables with all names or addresses of local recipients. A recipient address is local when its domain matches $mydestination, $inet_interfaces or $proxy_interfaces. If a local username or address is not listed in $local_recipient_maps, then the Postfix SMTP server will reject the address with “User unknown in local recipient table”.

So this means that smtpd did not find ‘alexis’ in local_recipient_maps. So what is local_recipient_maps? I do not define it in main.cf and the default is:

mandala# postconf -d local_recipient_maps
local_recipient_maps = proxy:unix:passwd.byname $alias_maps
mandala#

So that explains it! getent passwd alexis fails when nslcd is not running and succeeds when it is. So what should I set local_recipient_maps to avoid it delegating to the OS?

postconf(5) says of local_recipient_maps:

If this parameter is non-empty (the default), then the Postfix SMTP server will reject mail for unknown local users.

To turn off local recipient checking in the Postfix SMTP server, specify “local_recipient_maps =” (i.e. empty).

So should I set local_recipient_maps = (i.e. without a value)? Well, yes, I think it is safe to do this: remember that the smtpd_recipient_restrictions already say that if an invalid local user is recipient then the mail is to be rejected, so I think it is safe to say here that we just accept all mails (that are in $mydestination). So I added:

local_recipient_maps =

and re-ran my test:

  • mutt/imap: skipped
  • cercis/mpcdf: skipped
  • mpcdf/alexis: failed:
Dec 16 10:13:49 mandala postfix/local[1752]: warning: error looking up passwd info for alexis: No such file or directory

Okay, so the local transport agent is complaining now, presumably because it cannot establish some attribute of user alexis that it can establish when nslcd is running; the home directory of the user is the most probable.

But why is the local agent not handing the mail to Dovecot? I have in my main.cf:

#mailbox_command = /usr/lib/dovecot/deliver
mailbox_transport = lmtp:unix:private/dovecot-lmtp

(mailbox_command was my original one; mailbox_transport was what I added earlier in this procedure.)

Regarding mailbox_transport postconf(5) says:

Optional message delivery transport that the local(8) delivery agent should use for mailbox delivery to all local recipients, whether or not they are found in the UNIX passwd database.

Whoah! Well, that sounds reasonable, right? But obviously the local transport agent is encountering a problem before it passes the mail on to Dovecot. Perhaps we can get Postfix to pass the mail on to Dovecot earlier, i.e. before the local transport agent does this home lookup (if that is what it is that causes the error) or perhaps we can get Postfix to use Dovecot as the transport!

I believe I need a local_transport setting (if it is possible to set it to one thing) or local_transports_map (if it is possible to set it to a list). Indeed local_transport exists and can I copy the value from mailbox_transport (and discard mailbox_transport altogether as that is only used by the Postfix local transport), i.e.:

#mailbox_command = /usr/lib/dovecot/deliver
#mailbox_transport = lmtp:unix:private/dovecot-lmtp
local_transport = lmtp:unix:private/dovecot-lmtp

I re-ran my tests:

  • mutt/imap: ok
  • cercis/mpcdf: ok
  • mpcdf/alexis: ok

The main question posed by this article is now addressed, but along the way some other questions came up.

Can recipient restriction ordering be improved?

smtpd_recipient_restrictions is set to:

smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unverified_recipient,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

which leads to a couple of questions:

Mails sent from my work email (mpcdf) to alexis2@pasta.freemip.com (not a valid address) should be rejected by smtpd because of the reject_unverified_recipient directive, but mails sent from cercis (a remote SMTP client that authenticates) to the same address would be accepted by smtpd because the permit_sasl_authenticated directive is earlier in the list, but then later rejected because when Postfix passes the mail to dovecot/lmtp in its role as the local delivery agent, dovecot/lmtp will reject it, which presumably will trigger a bounce that could have been avoided, right?

Let’s verify:

Dec 16 10:35:16 mandala postfix/smtpd[2565]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 10:35:16 mandala postgrey[243]: action=greylist, reason=new, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=alexis2@pasta.freemyip.com
Dec 16 10:35:16 mandala postfix/smtpd[2565]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>
Dec 16 10:35:16 mandala postfix/smtpd[2565]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8

Okay, that was a bit unfortunate that postgrey greylisted it, but oddly lmtp did reject it, ah, ok, I get it! postgrey got its message logged first, but actually ran second. We know the latter because we can see the order of directives in the setting of smtpd_recipient_restrictions. Err … but in that case why was postgrey even called at all?

I checked postconf(5) smtpd_recipient_restrictions and it does not even mention check_policy_service! I am sure that – perhaps in an earlier version – it was allowed. check_policy_service is mentioned in the section for smtpd_client_restrictions (that’s client, not recipient). The man page says to also see the SMTPD_POLICY_README document. Let’s do that, to see if there is any mention of sucj checks being done in parallel or for backward compatibility still allowed to be declared in the recipient restrictions but actually done during the client restriction phase (CONNECT-time, rather than RCTP TO-time?).

I did not find any reference in SMTPD_POLICY_README to what happened.

However, after a couple of tests, the postgrey program had got used to the ‘alexis2’ address and did not interfere, leaving it to lmtp to reject it:

Dec 16 11:00:27 mandala postfix/smtpd[2612]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 11:00:28 mandala postgrey[243]: action=pass, reason=triplet found, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=alexis2@pasta.freemyip.com
Dec 16 11:00:28 mandala postfix/smtpd[2612]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>
Dec 16 11:00:28 mandala postfix/smtpd[2612]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8

Interestingly, I did not immediately see a bounce mail at MPCDF; just in case it does arrive it was subject ‘test404’). So the right thing happened there.

But what about when I sent it from cercis, which will do SASL authentication and so be permitted due to smtpd_recipient_restrictions? Indeed:

Dec 16 11:04:36 mandala postfix/smtpd[2618]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:04:36 mandala postfix/smtpd[2618]: ED21A42CC: client=88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216], sasl_method=PLAIN, sasl_username=cercis
Dec 16 11:04:37 mandala postfix/cleanup[2622]: ED21A42CC: message-id=<20211216100435.F40A9769@cercis.freemyip.com>
Dec 16 11:04:37 mandala postfix/qmgr[2507]: ED21A42CC: from=<root@cercis.freemyip.com>, size=611, nrcpt=1 (queue active)
Dec 16 11:04:37 mandala postfix/smtpd[2618]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Dec 16 11:04:37 mandala dovecot: lmtp(2624): Connect from local
Dec 16 11:04:37 mandala postfix/lmtp[2623]: ED21A42CC: to=<alexis2@pasta.freemyip.com>, relay=mandala.pasta.net[private/dovecot-lmtp], delay=0.32, delays=0.19/0.01/0.07/0.05, dsn=5.1.1, status=bounced (host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command))
Dec 16 11:04:37 mandala dovecot: lmtp(2624): Disconnect from local: Client has quit the connection (state=READY)
Dec 16 11:04:37 mandala postfix/cleanup[2622]: 3F63F4587: message-id=<20211216100437.3F63F4587@mandala.pasta.net>
Dec 16 11:04:37 mandala postfix/bounce[2625]: ED21A42CC: sender non-delivery notification: 3F63F4587
Dec 16 11:04:37 mandala postfix/qmgr[2507]: 3F63F4587: from=<>, size=2762, nrcpt=1 (queue active)
Dec 16 11:04:37 mandala postfix/qmgr[2507]: ED21A42CC: removed
Dec 16 11:04:38 mandala postfix/smtp[2626]: 3F63F4587: to=<root@cercis.freemyip.com>, relay=smtp.gmail.com[173.194.76.109]:25, delay=1.7, delays=0.02/0.02/0.59/1.1, dsn=2.0.0, status=sent (250 2.0.0 OK 1639649078 l26sm4024502wms.15 - gsmtp)
Dec 16 11:04:38 mandala postfix/qmgr[2507]: 3F63F4587: removed

The mail was accepted for delivery and smtpd closed the connection with the client, so the client thinks it is going to be delivered. Only later does lmtp complain that it cannot save the mail into alexis2’s INBOX and postfix/bounce sends a bounce to root@cercis.freemyip.com. Ok, it will not be able to delivery that, but that is smtp.gmail.com’s problem to deal with.

But could we avoid that unnecessary bounce by re-ordering the restrictions so that reject_unverified_recipient appears before permit_sasl_authenticated? But does reject_unverified_recipient cause mails that are to be relayed to be rejected because they do not match local users?

Let’s try:

smtpd_recipient_restrictions =
    reject_unverified_recipient,
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

cercis–>alexis2@pasta.freemyip.com:

Dec 16 11:11:27 mandala postfix/smtpd[2811]: connect from unknown[141.98.10.220]
Dec 16 11:11:27 mandala postfix/smtpd[2811]: warning: unknown[141.98.10.220]: SASL LOGIN authentication failed: Invalid authentication mechanism
Dec 16 11:11:27 mandala postfix/smtpd[2811]: disconnect from unknown[141.98.10.220] ehlo=1 auth=0/1 quit=1 commands=2/3
Dec 16 11:12:19 mandala postfix/smtpd[2811]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:12:19 mandala postfix/smtpd[2811]: NOQUEUE: reject: RCPT from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<root@cercis.freemyip.com> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<cercis.freemyip.com>
Dec 16 11:12:19 mandala postfix/smtpd[2811]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=7/9

That looks good. Interestingly, cercis mail logs said:

Dec 16 11:12:18 cercis postfix/pickup[8528]: B52D876F: uid=0 from=<root>
Dec 16 11:12:18 cercis postfix/cleanup[8930]: B52D876F: message-id=<20211216101218.B52D876F@cercis.freemyip.com>
Dec 16 11:12:18 cercis postfix/qmgr[1130]: B52D876F: from=<root@cercis.freemyip.com>, size=387, nrcpt=1 (queue active)
Dec 16 11:12:19 cercis postfix/smtp[8932]: B52D876F: to=<alexis2@pasta.freemyip.com>, relay=mail.pasta.freemyip.com[188.192.9.35]:25, delay=1, delays=0.07/0.04/0.84/0.1, dsn=4.1.1, status=deferred (host mail.pasta.freemyip.com[188.192.9.35] said: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command) (in reply to RCPT TO command))

Note the status=deferred! So the mail queue … err … yes, the mailq command shows the mail sitting on cercis waiting for another attempt.

Okay, so that test looks good now, but what about when cercis wants to relay mails through my mail server?

cercis–>mpcdf:

Dec 16 11:18:08 mandala postfix/smtpd[2828]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:18:08 mandala postfix/smtpd[2828]: 924B6420C: client=88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216], sasl_method=PLAIN, sasl_username=cercis
Dec 16 11:18:08 mandala postfix/cleanup[2835]: 924B6420C: message-id=<20211216101808.16E9376E@cercis.freemyip.com>
Dec 16 11:18:08 mandala postfix/qmgr[2809]: 924B6420C: from=<root@cercis.freemyip.com>, size=611, nrcpt=1 (queue active)
Dec 16 11:18:08 mandala postfix/smtpd[2828]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Dec 16 11:18:10 mandala postfix/smtp[2838]: 924B6420C: to=<alexis.huxley@mpcdf.mpg.de>, relay=smtp.gmail.com[173.194.76.108]:25, delay=2.1, delays=0.07/0.03/0.64/1.3, dsn=2.0.0, status=sent (250 2.0.0 OK 1639649890 m36sm4207359wms.25 - gsmtp)
Dec 16 11:18:10 mandala postfix/qmgr[2809]: 924B6420C: removed

So the right thing happened: ‘alexis.huxley’ is not a local user but did not cause a rejection because reject_unverified_recipient is only checking when the domain matches something in $mydestination. Does postconf(5) confirm that is what reject_unverified_recipient does?

It is not clear because it says this is delegated to the verify(8) Postfix subsystem and I did not read that far. But what it did say was this:

The unverified_recipient_reject_code parameter specifies the numerical response code when an address is known to bounce (default: 450, change into 550 when you are confident that itis safe to do so).

Since Dovecot and Postfix have their own LDAP configuration, can I remove the OS’s LDAP configuration?

I have deleted much of the original progress log stored here as it was very incoherent and complicated by use of PCMS. A summary is presented instead.

Without the following line in /etc/ldap/ldap.conf:

TLS_CACERT /etc/ssl/certs/ca-certificates.crt

mail does not work. But it looks like I can tell Dovecot directly about the certificate authorities instead; I added this to dovecot-ldap.conf:

tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt

and re-commented-out the line in /etc/ldap/ldap.conf, at which point there are no ldap processes or files outside of Dovecot.

    Is it only the Postfix local delivery agent that understands aliases?

    /etc/aliases specifies that mail to root should be sent to alexis.

    I ran tests:

    • mpcdf—>root@pasta.freemyip.com: greylisted (okay, I suspect)
    • alexis@pasta—>root: ok

    which worked!

    main.cf contains:

    alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

    which raises some questions:

    • are both needed?
    • why do they work at all now that Postfix’s local transport is not involved?

    postconf(5) does not really clarify this but does suggest that the local transport agent is still involved. I think I need simply to test it but removing things.

    First I send a few mails from work to root@pasta.freemyip.com to get past the greylisting. Then by setting:

    #alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliasesxxx

    I was able to show that the newaliases command reads alias_database:

    mandala# newaliases
postalias: fatal: open /etc/aliasesxxx: No such file or directory
mandala#

    so we should set:

    alias_database = hash:/etc/aliases

    in order that the newaliases command works and that the aliases file is where we expect it to be. However:

    mandala# postconf -d alias_database
alias_database = hash:/etc/aliases
mandala#

    I.e. if that is what we are going to set it to then we do not need to set it explicitly at all. So we will comment alias_database out.

    Regarding alias expansion, is it enough to set alias_database?

    I had not realised it but I still had this entry:

    virtual_alias_maps = hash:/etc/postfix/virtual

    and:

    mandala# cat /etc/postfix/virtual
root alexis@pasta.net
mandala#

    So I think that that is why it worked!

    When I commented out virtual_alias_maps and uncommented back in the alias_database and alias_maps then mail to root@pasta.freemyip.com failed as follows:

    Dec 16 14:34:51 mandala postfix/smtpd[21012]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 14:34:52 mandala postgrey[250]: action=pass, reason=triplet found, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=root@pasta.freemyip.com
Dec 16 14:34:52 mandala postfix/smtpd[21012]: 337CD420D: client=b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 14:34:52 mandala postfix/cleanup[21017]: 337CD420D: message-id=<20211216133449.u4ckhgnkjhnmtlvt@damson.rzg.mpg.de>
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 337CD420D: from=<alexis.huxley@mpcdf.mpg.de>, size=1393, nrcpt=1 (queue active)
Dec 16 14:34:52 mandala postfix/smtpd[21012]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Dec 16 14:34:52 mandala dovecot: lmtp(21019): Connect from local
Dec 16 14:34:52 mandala postfix/lmtp[21018]: 337CD420D: to=<root@pasta.freemyip.com>, relay=mandala.pasta.net[private/dovecot-lmtp], delay=0.37, delays=0.33/0.01/0.01/0.03, dsn=5.1.1, status=bounced (host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <root@pasta.freemyip.com> User doesn't exist: root@pasta.freemyip.com (in reply to RCPT TO command))
Dec 16 14:34:52 mandala dovecot: lmtp(21019): Disconnect from local: Client has quit the connection (state=READY)
Dec 16 14:34:52 mandala postfix/cleanup[21017]: 4896444DC: message-id=<20211216133452.4896444DC@mandala.pasta.net>
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 4896444DC: from=<>, size=3529, nrcpt=1 (queue active)
Dec 16 14:34:52 mandala postfix/bounce[21020]: 337CD420D: sender non-delivery notification: 4896444DC
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 337CD420D: removed
Dec 16 14:34:53 mandala postfix/smtp[21021]: 4896444DC: to=<alexis.huxley@mpcdf.mpg.de>, relay=smtp.gmail.com[173.194.76.108]:25, delay=1.3, delays=0.02/0.02/0.57/0.74, dsn=2.0.0, status=sent (250 2.0.0 OK 1639661693 n4sm5038645wrc.1 - gsmtp)
Dec 16 14:34:53 mandala postfix/qmgr[21010]: 4896444DC: removed

    It is interesting that lmtp did not reject the mail for smtpd but it did reject it when it came to try to save it in the inbox, which, because smtpd had already hung up the connection, meant a bounce was triggered.

    So now we know this:

    • with the default values for alias_database, alias_maps and virt_alias_maps and with root: alexis in /etc/aliases (and newaliases having been run) mail to root is not delivered, implying that lmtp does not read aliases
    • newliases reads alias_database to find out what to rebuild
    • with virtual_alias_maps set then Postfix redirects mail saving it locally (and we knew ages ago that Postfix would redirect it remotely using this file)

    So a configuration comes to mind:

    alias_database = hash:/etc/aliases 
alias_maps =
virt_alias_maps = hash:/etc/aliases

    This way /etc/aliases becomes not only for local mail, newaliases knows to regenerate it and we do not create unnecessary extra maps.

    It did not work: mail to root@pasta.freemyip.com was rejected. Changing virt_alias_maps back to /etc/postfix/virtual, which contained:

    mandala# cat /etc/postfix/virtual
root alexis@pasta.net
mandala#

    worked.

    Hmm … so either virt_alias_maps wants the maps to have entries of form user@domain (not just user) or it is unhappy about sharing its value with alias_database and alias_maps.

    Let’s set it back to hash:/etc/aliases but put fully qualified right hand sides in. That worked!

    So now let’s extend the test with this in aliases:

    root: alexis@pasta.net
www-data: root
test: alexis.huxley@mpcdf.mpg.de

    and try sending mails to those three addresses from outside:

    • root: ok
    • www-data: ok (after greylisting)
    • test: ok (after greylisting)

    So this configuration works:

    virtual_alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases

    But I think that alias_database should be redundant, so I set:

    alias_database =

    and I re-ran my testts:

    • root: ok
    • www-data: ok
    • test: ok

    Conclusions

    Postfix can be made to delegate both validation and authentication to Dovecot!