Monitoring The ColdFusion Mail Server With GMail And A Scheduled Task
Every now and then, my ColdFusion mail server breaks. I don't know why; it seems to be completely random. But, every now and then, it cannot connect to the SMTP host and it simply stops sending out emails. I typically am not aware that this has happened until I happen notice that someone has left a comment on my blog for which I never got an email alert. At that point, I'll check my mail server to find out that the mail hasn't actually worked in days and I have dozens of undelivered mail stubs.
My ColdFusion mail server stopped working again a few days ago and this time, I decided it was time to put a process in place to help me stay on top of the situation. I created a scheduled task that runs every hour and monitors the undelivered mail folder (Undelivr) on the server. When the number of items in the undelivered mail folder gets above, say 3 files, the ColdFusion server shoots me an email to let me know something is wrong.
But, if the mail server is down, I can't just send myself an email alert in the normal fashion! If I did that, my alert would end up in the undelivered mail folder along with everything else. As such, I have to use a 3rd party mail server to have ColdFusion send the alert. In this case, since ColdFusion 8+ now has SSL support for CFMail, I have decided to go with Google's GMail. I set up a GMail account for my ColdFusion server and wired up this script:
<!--- Create a directory path to the undelivered mail folder. --->
<cfset undeliveredMailDirectory = "C:\ColdFusion8\Mail\Undelivr\" />
<!--- Query for the undelivered mail items in this directory. --->
<cfdirectory
name="mailStubs"
action="list"
directory="#undeliveredMailDirectory#"
sort="dateLastModified DESC"
/>
<!---
Check to see how many undelivered files we have. If we
have more than the given threshhold, we need to alert the
server administrator that something has crapped-out in the
mail server.
--->
<cfif (mailStubs.recordCount gt 3)>
<!---
Too many emails have failed! Something is breaking on
the server - might need a restart. Send email - of
course, since MAIL SERVER IS BROKEN, we cannot use the
configured mail server. Rather, we have to use a 3rd
party mail service; in this case, Google Mail (GMAIL).
--->
<cfmail
to="ben@bennadel.com"
from="coldfusion@bennadel.com"
subject="ColdFusion Mail Server Not Working (#mailStubs.recordCount# Undelivered Messages)"
type="html"
server="smtp.gmail.com"
username="****"
password="****"
port="465"
usessl="true">
<h1>
ColdFusion Mail Server Has Probably Failed
</h1>
<p>
The following emails are sitting in the undelivered
mail folder on the server:
</p>
<ul>
<cfloop query="mailStubs">
<li>
#mailStubs.name# -
#dateFormat( mailStubs.dateLastModified, "mm/dd/yyyy" )#
at
#timeFormat( mailStubs.dateLastModified, "hh:mm TT" )#
</li>
</cfloop>
</ul>
<p>
You might want to consider restarting the service.
</p>
</cfmail>
</cfif>
<!--- Mail sent! --->
Done.
As you can see, if there are more than 3 files in the undelivered mail folder, I am using the CFMail tag and the GMail service to send a list of undelivered files to myself.
Of course, this makes the assumption that the CFMail tag will work once the mail service is broken. But, since I don't actually know what is breaking when the mail stops working, it's definitely possible that the CFMail tag won't even work when routing through the GMail server. Who knows? I'll just have to see what happens the next time my mail server decides to give up on life.
I know that there are services that will help you monitor stuff like this (server uptime and what not); but, I figured this was, if nothing else, a chance to try using the CFMail tag with Google's GMail service.
Want to use code from this post? Check out the license.
Reader Comments
Forgive me for asking, but you are sending an email if you determine that email is not working?
@Ernest,
Right; the hope is that when the email stops working, I can leverage the 3rd party service - GMail - to send out the alert.
Sorry for not making that clear - my primary web service is *not* GMail.
This happens to us as well Ben. More often than not it is at 2:00am prompting a call from support with open tickets on unreceived email.
I am not convinced that the <cfmail> tag will work though when the service is down. But I am going to give this a shot as well.
Thanks!
-Robert
To be on the safe side, you might want to skip the mail spool entirely for the alert email that's going out via Gmail. Add spoolEnable="false" to the cfmail tag and it will post the email directly to the GMail server during page processing.
@Robert,
Yeah, I am not 100% convinced either. Typically, when the CFMail stops working, the error I get in the logs is, "Could not connect to SMTP host". So, my **hope** is that the real problem is with the mail service, not with ColdFusion.
Of course, that could just be a pipe dream :)
And, to make matters a bit scary, I just checked my Log file and I see I have gotten that error like 300 times in the last 3 minutes. But, no undelivered mail... I assume that is someone trying to hack my server and use my mail service??
I wish I knew more about server administration.
@Sean,
Ahhh, awesome tip! Will be updating that right now.
Ben - innovative way to keep an eye on the mail folders!
I would also recommend watching the spool folder as well. On occasion, the mail delivery subsystem in CF will completely die. Mail will get written to the spool folder but will never get delivered or sent to the undeliverable folder. When that happens, you're hosed until you restart CF.
I use a custom written app that looks at both the spool and undelivr every 5 minutes and that works really well. Had it setup so that it can FTP into remote servers and check them.
When the cfmail is not sending, at this time shall we use the "failto" attribute, which will indicate about this. Hope it makes sense
thanks...
@Tom,
Using an FTP account is pretty clever! I hadn't even thought of that kind of approach to reaching the server.
@CF Fan,
The problem is that I think the FailTo attribute requires the mail server to be functioning properly. I think if the mail fails because the mail service is messed up, FailTo will likely also fail.
In my experience, it's usually just CF's mail spooler that's hosed, not the SMTP server. Stopping/restarting the mail spool usually does the trick without having to restart CF altogether. There's a sample script in a HoF thread here: http://www.mail-archive.com/cf-talk@houseoffusion.com/msg344187.html
@Debbie,
Hmm, thanks for the tip. I was not even aware that the Spooler was a service unto itself. To tell the truth, I don't actually know much about mail in general. I will be sure to read that thread.
@a cf fan: the failto address is used for directing bounced emails to a specific account. I'm not sure that it would help in this situation as the message would need to be sent out by the smtp server before it could fail.
you could, of course, go a little OTT and use Java to open a direct SMTP port to your server.
I've used this in the past for email verification:
http://cookbooks.adobe.com/post_Verify_an_email_address_using_the_SMTP_protocol__a-16332.html
Whilst this code stop just after the triplet it should be easily extendable to include the data command which would actually send an email.
@Rob,
Very cool! I had no idea you could even do that. I'll have to take a closer look at your CFC.
I had a similar problem last year. I was sending mass email to our client list. ColdFusion's spooler was trickling the mail over to our SMTP server (a separate server) a few at a time. I'm using <cfmail> with a query attribute set.
When the SMTP server got backed up, then ColdFusion would assume they were undeliverable, and move them to the appropriate server - at least that's what I think was happening. I wrote a Scheduled Task that would, once an hour, move EVERYTHING from the Undeliverable folder back into the spool folder - simple file moves. I still have to periodically monitor the undeliverable folder for items going out with bad email addresses. For the most part though, it became a hands-off problem instead of a all-hands-on-deck problem.
@Brian,
Yeah, I've definitely run into that kind of situation before where I simply move the files from undeliverable to spool and just works :) So frustrating.
We're still running on a CF7 server, and our biggest problem at the time I was having the issues with cfmail we were regularly having to reboot the server. Since then we've migrated most of our ODBC data sources off to web services running on a different server, and the problem with cfmail has vanished. My working theory is that once ODBC got to a certain level of locks, it started to freeze the coldfusion service, which led to all kinds of problems.
Lots of activity on this topic lately. I wonder if something else is going on? Just in the last week I've had to restart the spool service twice... and I've never had to do that before. I think that MAYBE the service was restarted twice since I installed CF8 all those years ago.
Thanks for the code Ben!
@Brian,
I've only ever had all of this stuff located on one machine, so it's definitely possible that the machine gets overloaded. I wish I knew more about actual server setup caveats.
@Michael,
Who knows, right? It's so frustrating why things start breaking all of a sudden. I never used to have FTP problems; now, often times, my FTP cannot connect on the first try. Never changed anything.
What about a free tool for 1 mail server:
http://www.mxtoolbox.com/services_servermonitoring.aspx
Instead of using CFMAIL to send a notification email, why not setup your gmail account as your backup mail server in CF Admin?
@Dawesi,
Thanks, I'll have to take a look at that; I have not seen it before.
@Carlos,
I would say that is ok IF the ColdFusion mail service would eventually come back online; but, in my experience, once the CF mail server craps out, it never comes back on it's own. Since it seems to die permanently (until the server / service is restarted), the idea of a backup makes me think the problem will just go unnoticed.
@Dawesi,
This free tool looks great to monitor the smtp server however it would not catch the CF Mail Spool lock up. (afaict)
A little tidbit, if you are using monitoring and you see that your CF has taken a crap on sending message but now leaves you with your Undelivr folder filled with messages. After restarting the Spool service you can run the command from within your Coldfusion installation folder /Mail do "mv Undelivr/* Spool/". Coldfusion will then try and resend all of those undelivered messages.
@Travis,
Good tip. Typically, I manually move the mail files over to the other file. Sounds like that could be a niec .bat file to create.
Now wouldn't this notification be best lined up to an SMS gateway of some sort?
That way you can get instant notifications to a mobile device.
@David,
You could definitely hook this up to an SMS service. Using things like Twilio make that kind of connectivity really easy to implement these days. It just comes down to how much notification you want. For me, since my personal mail server is not "mission critical", I don't want to get text messages at 4AM :)
However, if this was for a client on my production server, then yeah - getting more immediate notification is probably a great idea.
We have been running into this problem for a couple years now; while running both CF 6 MX and our upgraded CF 8 instance.
I found this script a while back and it has been so amazingly helpful, thank you Ben!
I started getting significantly more "can't connect to SMTP server" errors after moving our mail host to a new provider with a new SMTP server/address. I'm working on it by tweaking the Connection timeout settings, but I am wondering if anyone has had better luck by disabling the "Maintain connection to server" option. This way, the CF instance would make a fresh connection to the SMTP server each time. Would this degrade performance significantly?
I also receive the SMTP server relay timeout exceeded error more often now. Our host suggested lowering the HELO response time, but I'm not sure if this is possible with CF8, any ideas? I saw a post about editing the jvm.config, but I'd rather not have to do this.
I also have a support ticket into Adobe to see if they have any tips or ideas.
One more thing, as I wrote this, I got notified my CF instance had stopped sending mail, ugh, what a pain.
Thanks Ben and CF community!
Hi Ben,
Thanks for the great post, I love your blog!
I have a question. It seems that the "from" and the "bounce" parameters are being replaced by the gmail account addresss.
Is there a workaround that?
If anyone is interested I have done a cfrespooler extension for the cfadmin which automatically handles your undelivered mail.
I will also be adding a feature to this which will monitor the spool folder for problems and will first try resetting spool and then send out an alert either via CFMAIL wihtout using the spool or by SMS.
http://cfrespooler.riaforge.org/
eremiya, there is an option in your gmail settings to add nick names/aliases, if you use this then you should be able to send emails FROM those addresses as well.
Pretty nice post. I simply stumbled upon your blog and wished to mention that I have truly enjoyed browsing your blog posts.
Note sure if you've ever come across something like this but I've had the same issue with mail not sending. I created a task that runs hourly checking the file count in the mail directory. If it exceeds my count I reset the service and off they go without me having to do anything =)
<CFIF #oFileCount.RECORDCOUNT# GT 100>
<CFSET sFactory = CreateObject("java","coldfusion.server.ServiceFactory") />
<CFSET MailSpoolService = sFactory.mailSpoolService />
<CFSET MailSpoolService.stop() />
<CFSET MailSpoolService.start() />
</CFIF>
Thnaks for your posting this blog. I bookmarked it and will forward to my friend he is doing the same.