Making Sure Scheduled Tasks Don't Overlap In ColdFusion
Yesterday, I explored the use of the ThrowOnTimeout attribute of ColdFusion's CFLock tag. I had always assumed that settings this attribute to False would cause ColdFusion to skip over the CFLock tag body upon timeout; it was nice to finally confirm that this assumption was accurate. In the closing notes to that post, I had mentioned that one place that I use this approach was to prevent ColdFusion scheduled tasks from overlapping their execution. I figured I would write a quick follow-up on that idea.
Most of the time, ColdFusion scheduled tasks don't do too much processing; sometimes, however, they do a lot of processing! This is especially true for any kind of FTP of mail-based data processing. In such cases, it is possible that the duration of a given scheduled task will be longer than the interval in which it executes. In such cases, I often don't want a scheduled task to execute while its previous incarnation is still running. To create this behavior without having to rely on any globally accessible application flags, I use ColdFusion's CFLock tag.
By wrapping my entire scheduled task's entry point in an exclusive, named lock, I can prevent parallel execution attempts from interfering with one another.
<!---
When executing a scheduled task, I use a CFLock tag to
prevent multiple calls to the scheduled task from overlapping.
By using a 1-second timeout, this scheduled task will be
skipped if it is still running.
NOTE: Do not use a timeout of "0" - that only delegates the
timeout setting to the ColdFusion administrator.
--->
<cflock
name="myTask_#getCurrentTemplatePath()#"
type="exclusive"
timeout="1"
throwontimeout="false">
<!---
Here is where you would execute your scheduled task
algorithm. For demo puroposes, we're just going to
pause to simulate hard work.
--->
<cfthread
action="sleep"
duration="#(3 * 1000)#"
/>
<!--- Output debug comment. --->
Task Executed!<br />
<br />
</cflock>
<!--- Output confirmation comment. --->
Scheduled task complete.<br />
In the above code, there's a few things to take note of; first, I am using a named lock. This keeps the lock specific to this scheduled task and this scheduled task only. As part of the name, I use the current template path; just as with your ColdFusion Application name, this approach allows for like-named scheduled tasks in different applications to act independently. Remember, named locks are ColdFusion-instance-specific, not application-specific.
This combination of exclusive access, a one second timeout, and the directive to not throw an error upon timeout allows ColdFusion to skip over this CFLock tag if it is currently executing in a previous thread. I use a one second timeout rather than a zero second timeout because a zero second timeout doesn't actually mean zero seconds; rather, it simply defers the timeout duration to whatever setting is defined within the ColdFusion administrator.
Sometimes, I have individual URLs for my scheduled tasks; sometimes, I use a single-point of entry for all of my scheduled tasks (either via subsequent CFHTTP or CFThread execution); in either case, this approach works well - you might just have to adjust where you place your CFLock tags. If you use a single-point of entry, you don't want unrelated tasks to affect each other's execution.
While I know this approach is overkill for most types of scheduled tasks, I have found it to be useful to do just out of habit. Something about it just makes me feel more comfortable - I know that I'm not going to be overloading the server too much processing. Anyway, I thought this might be helpful to someone.
Want to use code from this post? Check out the license.
Reader Comments
Hi Ben thanks for this post. I've run into this issue recently whereby a large travel client has lots of external synchronization processes running on 15 minute timers to keep traveller data up to date. With more and more travellers being added sometimes up to 5,000 traveller profiles may be synchronized in bulk and this will take longer than the 15 min interval, causing the task to run again over the top.
Also this sync url used by the scheduled tasks can also be manually called from the CMS to perform traveller sync's where necessary and this adds even more complexity as they may attempt the sync over the top of the scheduled background sync???
I'll give this method a go and hopefully this fixes my issue :)
Oh just wondering if you knew of a way to monitor a currently running scheduled task or thread so i could throw useful information back to the UI to show that a sync is currently still executing?
Cheers!
@Phil,
This approach has been great for me! I use it in scheduled tasks - I even use it inside CFThread if it has to do with long-running tasks that can be invoked before they have completed. Really saves the day.
As far as monitoring the scheduled tasks, I guess you would need something like FusionReactor for that; I am not too familiar with that stuff.
Good idea Ben, thanks!
@Erik,
Thanks!
Hey Ben,
This place is a great resource for myself and team. Thanks!
I have a question, we use SeeFusion and we're getting some hung processes (ST that run every 15 seconds) with an exclusive cflock so once it hangs, any future process won't.
Ben, thank you for this information! I tried it and it works.
Have a nice day!
Dear Ben, excellent piece of code.
my question, think you wd prepare printouts for a web-shop. deliverynotes and invoices.
and this just runs from the same printout.cfm
just controlled by a variable doctype IN or DN.
so we want to prevent IN running multiple and DN multiple; BUT printout.cfm?doctype=IN and
printout.cfm?doctype=DN is wanted to be parallel for 24/7. wd the code be "different" enough as using those variable... to have success with your cflock example???
best regds & thanks.
claus