GMail Seems To Ignore The Return-Path Header Defined By The CFMail FailTo Attribute
After a conversation I saw unfold between Jose Galdamez and Delon in the comments on my ColdFusion CFPOP blog entry, I was inspired to play around with ColdFusion's CFMail FailTo attribute. I've been using CFMail since I start using ColdFusion but, I've never actually used the FailTo attribute. Despite the seemingly obvious name, I don't think I ever even knew what function the FailTo attribute served.
The ColdFusion documentation says that the CFMail FailTo attribute sets the "reverse-path". After some research, I found out that by "reverse-path", the documentation actually meant to say "return-path", which is the address to which the email should be returned if the email cannot be delivered to its intended target (To Attribute). The FailTo attribute defines the single address placed in the "Return-Path" header of the outgoing email. This can be defined either through the FailTo attribute or through the use of a CFMailParam tag to explicitly define the header:
<cfmailparam
name="return-path"
value="failto@mydomain.com"
/>
To experiment with this CFMail feature, I figured I would start off using my GMail account complete with the plus-style (+) addressing to define the various CFMail attributes:
<!--- Set up the SMTP server settings. --->
<cfset smtpSettings = {
server="smtp.gmail.com",
username="ben@bennadel.com",
password="****************",
port="465",
usessl="true"
} />
<!---
Send out the email to a non-existent address. Here, we are
going to use a personal GMail account in conjunction with
the ReplyTo and FailTo email headers.
--->
<cfmail
to="nat@natalie-gets-naughty.com"
from="ben+from@bennadel.com"
replyto="ben+replyto@bennadel.com"
failto="ben+failto@bennadel.com"
subject="CFMail FailTo Test"
type="html"
attributecollection="#smtpSettings#">
<p>
This is a CFMail FailTo Test
</p>
<p>
To: nat@natalie-gets-naughty.com
</p>
<p>
ReplyTo: ben+replyto@bennadel.com
</p>
<p>
FailTo: ben+failto@bennadel.com
</p>
</cfmail>
As you can see here, I am using both the ReplyTo and FailTo CFMail attributes. In each case, as well as with the From attribute, I am using a plus-style address to help me differentiate the defined values within the email header. While I don't particularly care about the ReplyTo attribute in this case, I am using it as a sort of control in my experiment; if GMail properly sets the ReplyTo and not the FailTo (or From for that matter), I'll know it has nothing to do with the plus-style addressing, but rather with some other unknown limitation.
When I run the above code, which sends an email to a non-existent address (nat@natalie-gets-naughty.com), I do indeed get a bounce back, but not to the intended address. Rather than bouncing back to ben+failto@bennadel.com, GMail has sent it to directly to ben@bennadel.com. When I look at the outgoing email header, this is what I see:
Return-Path: <ben@bennadel.com>
Received: from ........
Date: Tue, 13 Apr 2010 08:15:10 -0400 (EDT)
From: ben@bennadel.com
Reply-To: ben+replyto@bennadel.com
To: nat@natalie-gets-naughty.com
Message-ID: <18680524.141271160910577.JavaMail.SYSTEM@smtp.gmail.com>
Subject: CFMail FailTo Test
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Mailer: ColdFusion 8 Application Server
As you can see here, the Reply-To header successfully used my plus-style address (ben+replyto@bennadel.com). But, both the Return-Path header and the From header completely ignored the plus-style address. In fact, after some more experimentation, I found that GMail completely ignored any value I put in either the FailTo attribute or the CFMailParam tag as well as the From attribute. It appears that no matter what values I supply, GMail uses my SMTP username as both the From and the Return-Path values.
From what I have read, the concept of a FailTo address seems very powerful; but, it also seems to depend too-much on the rules of the outgoing SMTP server. Of course, I'm basing that statement almost entirely on a small set of GMail-driven tests. It is definitely possible that the way GMail handles the Return-Path / From headers is unique to it [GMail] and not a typical complication caused by any other SMTP servers.
Want to use code from this post? Check out the license.
Reader Comments
FWIW, a while ago we got hit with a barrage of spam using a fake From but a legitimate Failto/Reply address. Basically spammers would use our server to bounce off spam. Now that the google captcha is broken, I can see how they might have turned that feature off.
@Mischa,
I am not sure I understand what you mean. Are you saying that spammers were sending email and using your mailing address as the FailTo?
You are stating that GMail removes the FailTo when "sending" through GMail, right? (It should work if you send through another service to a GMail account.)
We send email to GMail accounts and they get bounced back to the FailTo address, but we don't use GMail to send our messages. (Check their TOS as this may be part of their policy.)
We use Surgemail and we have an option to rewrite/drop certain mail headers for security reasons. It's a very robust mail program (multi-platform, clusterable, etc) and I've been able to integrate CF to work with it to manage domain Aliases, process bounces and send messages.
I have a separate mailbox where I instruct Internet mail servers to send bounce messages to, thanks to FailTo. A scheduled task goes through this mailbox to take action on accounts in our database where the email address isn't working.
We've had emails sent to our FailTo mailbox from gmail.com so I think they do use return-path for bouncing back emails. Try sending your emails from a non-Gmail account as James suggests.
Sorry Ben... say I work at CompanyX.com... spammers would send:
from: cheapmedz@fakedomain.com
to: RandomlyPickedName@CompanyX.com
failto: ben@bennadel.com
delivery fails because the random name doesn't exist, our mailserver would fail the mail to you @ bennadel.com and the spam message would be attached. They basically used our "good" (ahem) reputation to relay their spam. Needless to say, we turned that off. Hope that makes sense!?
Completely agree Ben on your point it depends on the recipients SMTP server.
I've developed a CF mass mailer over the last year or so and in testing and doing lareg email sends, we have found that most servers do use the failTo address but some use the replyTo to send their bounces to. It's annoying as the whole point we are using the failTo is, like above, catch bounces in a pop account. These are then processed, the address gained via regEx from the bounce body and marked "Bounced" in the database.
Obviously this isn't 100% reliable though if we are not catching all bounces :(
Processing bounces is made even more difficult when mail servers bounce the message and strip the original "To" email address. We started adding extra headers in an attempt to track this but sometimes they are removed too. Some mailservers also suppress the body of the message, so the opt-out tracking code is removed. AOL is worse when it comes to blocking email due to suspected spam. They silently delete the message without notifying the recipient or sender. We've had to subscribe to their SCOMP service... but even then they remove to "To" email address for security purposes.
If you aren't doing it now, I recommend adding this to your CFMail tag:
<CFMAILPARAM Name="X-Subscribed-As" Value="#Email#">
If your message is going to be sent to many users, add a bulk parameter:
<CFMAILPARAM Name="Precedence" Value="bulk">
If you have an unsubscribe link, add this:
<CFMAILPARAM Name="List-Unsubscribe" Value="<http://www.domain.com/unsusbcribe/>">
@James
Without sounding stupid what to those extra mail params actually do? i.e. what benefit if any do they provide?
We use the code below to retrieve the bounced email which seems to to work pretty well but, admittedly, isn't perfect by any stretch:
<cfset myIndex = REFindNoCase("([a-zA-Z_0-9-])*([.]?([[:alnum:]_-]+)*)?@([[:alnum:]\-_]+\.)+[a-zA-Z]{2,4}",body,1,"True")>
<cfif myIndex.len[1] GT 0>
<cfset Email = trim(Mid(body,myIndex.pos[1],myIndex.len[1]))>
<cfif ReFindNoCase("([a-zA-Z_0-9-])*([.]?([[:alnum:]_-]+)*)?@([[:alnum:]\-_]+\.)+[a-zA-Z]{2,4}",Email) EQ 1>
"Email" is in theory the bounced email addre
</cfif>
</cfif>
@James,
I just tried using something called Postmark, which seems to be pretty decent. In production, we use SmarterMail, which I think should support the FailTo feature (but am not I've tested it just yet).
@Gary,
I'll be trying to test this with other, non-Gmail origins.
@Mischa,
Ahh, I see what you're saying now. Spammers are sleezy! They really make everyone else's life harder.
@Tom,
Some email clients use the ReplyTo for bounce backs? That is super lame. That doesn't even make sense. It seems like this is exactly what the Return-Path header was designed for.
@James,
That's timely that you bring up the additional headers - I was just experimenting with Postmark (as I said above) and it looks like Gmail provides an "Unsubscribe" link in the meta-recipients list. I had never seen anything like this before. I guess this is the header it is looking for. That's really cool.
As far as all kinds of information getting stripped out, I've read a lot about this. That is why the plus-style addressing in GMail seemed to awesome - I figured I could append some sort of unique ID to the failto address:
failto+<customerID>@domain.com
That just would have made life easy!
@Tom
How can you parse email addresses from a bounced message that has had them removed? Adding the extra "X-Subscribed-As" header sometimes allows the email address to be identified when the bouncing server removes the standard "FROM" address or when the email address is forwarded to a third-party email address that is inactive (and not in your mailing list.)
Regarding bulk, using it should cause out-of-office repliers to not send automatic replies. More info here:
https://mail.google.com/support/bin/answer.py?answer=81126#format
For info on list-unsubscribe header:
http://www.list-unsubscribe.com/
I played around with this a bit more, this time using SmarterMail as my SMTP server and GMail as my FailTo account.
www.bennadel.com/blog/1902-Using-GMail-s-Plus-Style-Addressing-To-Track-Bounce-Back-Email-Meta-Data.htm
It works better in that the FailTo / Return-Path header is set properly. But, I could *not* get this to work if the target email address did not exist. It seems to require a valid mail server on the other end of the equation or nothing ever ended up in the FailTo account.
Clearly this is not always true as I was able to get Postmark to track non-existent domains.
The failto address seems to work correctly now, unless a display name is added e.g. failto="Fred <fred@example.com>"
Adding a display name to the failto attribute does work with GoDaddy though, and I am left wondering if it is jusy Godaddy's server being lenient, or if I shouldn't be adding a display name to the failto anyway.
So, is the header "Problems-to" no longer valid? I also add "return-path" via cfmailparam.