Can sendmail forward e-mail immediately instead of queueing?

Question

Some of the e-mail passing through my server is forwarded to external accounts.

Unfortunately, my upstream SMTP-server is very picky about spam -- and rejects some of the legitimate messages as such. When this happens to the forwarded mail, I get the bounces (as the postmaster) -- not the originators.

I understand, that this is because sendmail queues the messages locally, disconnects from the relay, and only then proceeds to forward them further. If the further forwarding breaks for any reason -- such as because the next relay misidentifies the message as spam -- my sendmail is left to hold the pieces.

Can things be configured so that the forwarding begins immediately instead (as soon as the forwarding destination is determined)? The status -- success or failure -- can then be communicated directly to the previous relay still on the line...

If sendmail can not do it, can any other MTAs? Thanks!

score 4 · Accepted Answer · edited Oct 07 '21 at 07:34

No, it's not possible as it's not implemented with any wide spread SMTP software; you would have to program your own SMTP server that supports this kind of behavior, which would be out of scope on Serverfault. In this answer I explain, why all MTAs have implemented the SMTP protocol very similarly, using queue, and how that is the best way to accomplish all requirements of the protocol.

A mail transport agent MTA always either denies a message or accepts and queues it, based on its own settings. Then, it's relayed or delivered from the queue.

That's because

there can be both permanent and temporary errors. If the MTA can't connect the nexthop immediate, it'll try again later and bounces only if the delay reaches the limit set. Neither can it wait for another MTA to respond before closing the connection, as it may have other messages to deliver first.
there can be several recipients. While a client can simply list all recipients at once with RCPT TO commands, the message can be finally delivered to several other servers, of which some can be available now and some later. Furthermore, the MTA can't open all these connections at once during the initial connection and wait for their responses. There's no practical reason to have totally different workflow for messages with a single recipient.
it should always be clear which MTA currently has the responsibility for delivering the message. (This has been explained by examples in MadHatter's answer.)

That's just how SMTP was designed. Rather than syntactical requirement for the connection commands this leads to very similar architectures; Sendmail, Postfix and even MS Exchange has separate components for sending and receiving mail.

The SMTP server component receives mail and adds it to the queue.
Then, separate SMTP client tries to send it further to other MTAs, or if a recipient is local, the message can be saved to a file or passed to a mail delivery agent MDA, e.g. Procmail.

The requirement still comes from the SMTP specification; RFC 5321 2.1 on SMTP model basic structure:

Fully-capable SMTP implementations, including the relays used by these less capable ones, and their destinations, are expected to support all of the queuing, retrying, and alternate address functions discussed in this specification. In many situations and configurations, the less-capable clients discussed above SHOULD be using the message submission protocol (RFC 4409) rather than SMTP.

And a bit further:

In other words, message transfer can occur in a single connection between the original SMTP-sender and the final SMTP-recipient, or can occur in a series of hops through intermediary systems. In either case, once the server has issued a success response at the end of the mail data, a formal handoff of responsibility for the message occurs: the protocol requires that a server MUST accept responsibility for either delivering the message or properly reporting the failure to do so (see Sections 6.1, 6.2, and 7.8).

score 3 · Answer 2 · answered Aug 08 '17 at 08:28

I think Esa's answer is excellent, but I'm going to disagree with some of it. I think that what you want is possible, but it's a bad idea, and it won't help you. As he says, RFC5321 s2.1 notes that

once the server has issued a success response at the end of the mail data, a formal handoff of responsibility for the message occurs: the protocol requires that a server MUST accept responsibility for either delivering the message or properly reporting the failure to do so

In the case that server B receives a message from server A and delivers it on to server C, I don't see that this bars B from waiting to confirm receipt to A until it has had confirmation of receipt from C - which is what you're asking for. But the problem is that between two servers 250 OK is atomic (either the receiver has accepted it, and so is responsible for delivery, or it hasn't, and so the sender remains responsible), while between three it isn't.

Consider the case where client A is unintentionally disconnected after it has sent the mail but before it's had the 250 OK, while B is delivering it on to C. C then confirms receipt from B with its 250 OK, so B knows that C has it. But A doesn't, so A must still be responsible, and must continue to redeliver to B. If there is some systematic communications issue between A and B (eg, one of those lovely firewalls that think it's their job to mess with the contents of SMTP conversations), this could result in a very large number of copies of the same message being delivered.

Moreover, sendmail already does what you think it doesn't: in the event of failed delivery to C, it will try to pass the buck back to A. This generally only fails where A is malicious, and either lies in the envelope-From (to whom such notification should be made) or doesn't run a mail server at all. In such cases the mail must be delivered to B's postmaster, for B is responsible for delivery (having said 250 OK to A, but not had it from C), cannot deliver to C (it's tried and got a 5xx permanent failure) and cannot hand back to A (because A has made this impossible) so has no-one else to give it to. Here's an example I got this morning:

Date: Tue, 8 Aug 2017 02:53:55 +0100
From: Mail Delivery Subsystem <MAILER-DAEMON@lory.teaparty.net>
To: mailman-bounces@teaparty.net
Subject: Returned mail: see transcript for details
Parts/Attachments:
   1   Shown     17 lines  Text
   2   Shown    407 bytes  Message, "Delivery Status"
   3   Shown     15 KB     Message
   3.1           10 KB     Application
----------------------------------------

The original message was received at Tue, 8 Aug 2017 02:53:53 +0100
from localhost [IPv6:::1]

   ----- The following addresses had permanent fatal errors -----
<redacted@googlemail.com>
    (reason: 550-5.7.1 Unauthenticated email from aol.com is not accepted due to domain's)

   ----- Transcript of session follows -----
... while talking to gmail-smtp-in.l.google.com.:
>>> DATA
<<< 550-5.7.1 Unauthenticated email from aol.com is not accepted due to domain's
<<< 550-5.7.1 DMARC policy. Please contact the administrator of aol.com domain if
<<< 550-5.7.1 this was a legitimate mail. Please visit
<<< 550-5.7.1  https://support.google.com/mail/answer/2451690 to learn about the
<<< 550 5.7.1 DMARC initiative. 53si155585wrc.260 - gsmtp
554 5.0.0 Service unavailable

    [ Part 2: "Delivery Status" ]

Reporting-MTA: dns; lory.teaparty.net
Received-From-MTA: DNS; localhost
Arrival-Date: Tue, 8 Aug 2017 02:53:53 +0100

Final-Recipient: RFC822; redacted@googlemail.com
Action: failed
Status: 5.7.1
Remote-MTA: DNS; gmail-smtp-in.l.google.com
Diagnostic-Code: SMTP; 550-5.7.1 Unauthenticated email from aol.com is not accepted due to domain's
Last-Attempt-Date: Tue, 8 Aug 2017 02:53:55 +0100

    [ Part 3: "Included Message" ]

Date: Tue, 08 Aug 2017 01:53:46 -0000
From: rmusic80proof@aol.com
To: redacted@stphilipschurch.org.uk
[...]

Note how the original email purports to come from an aol.com address. How, then, am I not trying to deliver the failure report back to them? Because they lied in their original SMTP transaction:

Aug  8 02:53:51 lory sendmail[9457]: v781rmjA009457: from=<rmusic80proof@stphilipschurch.org.uk>, size=14095, class=0, nrcpts=1, msgid=<150215722683.22237.12283849532100059916@37.114.157.178>, proto=SMTP, daemon=MTA-v6, relay=[37.114.157.178]

It's my fault that I haven't yet set up SPF for that particular domain of mine (stphilipschurch.org.uk), but since I haven't, nothing stops me from accepting that lie - and then I'm stuck with an undeliverable message which cannot be returned to sender, for the sender is malicious and uninterested (being connected via an ISP in Azerbaijan).

tl; dr: sendmail already does what you're asking for, when it can. What you want to do won't help it when it can't, and will create problems. Don't do it.

Can sendmail forward e-mail immediately instead of queueing?

2 Answers2