Experimenting With Try / Catch / Retry Semantics In Lucee CFML 5.3.7.47
The other day, when I was looking something up in the Lucee CFML documentation, I came across a page that outlined the CFRetry tag. This tag will jump the control flow back up to the CFTry ingress, re-running the volatile execution pathway. I'm not sure I've seen this tag before. And, I don't actually "retry" much code in my ColdFusion applications. But, every now and then, I'll have some "exponential" back-off logic around a remote system call. As such, I wanted to see how my traditional approach to this would compare and contrast with the retry semantics in Lucee CFML 5.3.7.47.
Normally, when I want to retry a volatile call to a remote system (a remote API or a database), I'll either explicitly wrap the call in a loop; or, I'll wrap the call in a proxy which implicitly applies the loop. Code for this might look something like this:
<cfscript>
// NOTE: I'm using an IIFE (Immediately-Invoked Function Expression) so that I can
// use the local scope with my variable declarations.
(() => {
// Rather than relying on the maths to do back-off calculations, this range of
// values provides an explicit set of back-off times (in milliseconds). This
// collection also doubles as the number of attempts that we should execute
// against the remote system before giving up (indicated by a final zero).
var backoffDurations = [
100,
200,
400,
1000,
0 // Indicates that the last timeout should be recorded as an error.
];
for ( var backoffDuration in backoffDurations ) {
try {
return( simulateRemoteCallError() );
} catch ( any error ) {
// If the remote system call was not successful AND we still have a non-
// zero back-off time to consume, sleep the thread briefly and try making
// the call again.
// --
// NOTE: In this implementation the retry is implicitly generated by way
// of the FOR-LOOP that we are currently in.
if ( backoffDuration ) {
sleep( applyJitter( backoffDuration ) );
} else {
rethrow;
}
}
}
})();
echo( "Call was successful." );
// ------------------------------------------------------------------------------- //
// ------------------------------------------------------------------------------- //
/**
* I apply a +/- 20% jitter to the given value so that can vary the timing in which
* we call a remote system.
*
* @value I am the timing value be adjusted.
*/
public numeric function applyJitter( required numeric value ) {
var jitter = ( randRange( 80, 120 ) / 100 );
return( fix( value * jitter ) );
}
/**
* I simulate a remote client call that breaks.
*/
public void function simulateRemoteCallError() {
systemOutput( "Remote call simulation (about to error)...", true, true );
throw( type = "Oops.I.Did.It.Again" );
}
</cfscript>
Here, I have an array of sleep-durations, backoffDurations, over which I am iterating in order to apply my retry logic. For each non-zero value in the array, I sleep() the current thread and then just let the CFLoop tag naturally re-enter the try / catch block.
I like this approach a lot because it's very explicit and not clever in any way. Which means, it's easy to follow and easy to maintain over the long term.
Now, let's try refactoring this ColdFusion demo to use the CFRetry tag. This time, instead of leaning on the CFLoop tag, I'm going to use the retry tag to re-enter the try / catch block:
<cfscript>
// NOTE: I'm using an IIFE (Immediately-Invoked Function Expression) so that I can
// use the local scope with my variable declarations.
(() => {
// Rather than relying on the maths to do back-off calculations, this range of
// values provides an explicit set of back-off times (in milliseconds). This
// collection also doubles as the number of attempts that we should execute
// against the remote system before giving up (indicated by a final zero).
var backoffDurations = [
100,
200,
400,
1000,
0 // Indicates that the last timeout should be recorded as an error.
];
try {
return( simulateRemoteCallError() );
} catch ( any error ) {
// If the remote system call was not successful AND we still have a non-zero
// back-off time to consume, sleep the thread briefly and try making the call
// again (using the explicit "retry" statement).
if ( local.backoffDuration = arrayShift( backoffDurations ) ) {
sleep( applyJitter( backoffDuration ) );
retry;
}
rethrow;
}
})();
echo( "Call was successful." );
// ------------------------------------------------------------------------------- //
// ------------------------------------------------------------------------------- //
/**
* I apply a +/- 20% jitter to the given value so that can vary the timing in which
* we call a remote system.
*
* @value I am the timing value be adjusted.
*/
public numeric function applyJitter( required numeric value ) {
var jitter = ( randRange( 80, 120 ) / 100 );
return( fix( value * jitter ) );
}
/**
* I shift the first value off of the given collection and return it. If the array is
* empty, a null value is returned.
*
* @collection I am the collection being mutated.
*/
public any function arrayShift( required array collection ) {
if ( collection.isDefined( 1 ) ) {
var value = collection.first();
collection.deleteAt( 1 );
return( value );
}
}
/**
* I simulate a remote client call that breaks.
*/
public void function simulateRemoteCallError() {
systemOutput( "Remote call simulation (about to error)...", true, true );
throw( type = "Oops.I.Did.It.Again" );
}
</cfscript>
As you can see, I'm still using the concept of the explicit backoffDurations sleep-values. But, there's no CFLoop tag to apply them. Instead, I keep shifting values off the front of the array, sleeping, and then calling retry.
The outcome of both of these demos is exactly the same. And, I'm trying to decide if the retry logic in the latter demo feels any cleaner. Part of me likes the fact that the retry tag is maybe a little more "clever". But, as I get older, I've come to fear that "clever" is really just an alias for "hard to maintain."
I'm also not crazy about the fact that I used a non-native array function - ArrayShift() - in order to keep the code less verbose. Shifting a value off an array, at least in ColdFusion, is not nearly as easy as a for-in loop that iterates over the values.
I'm honestly torn on this one. Now that I know there is a CFRetry tag in Lucee CFML, I'll see where I can apply it. And perhaps just seeing it more will make it feel less clever. Then again, I really don't have much retry logic in my applications (probably much less than I should).
Want to use code from this post? Check out the license.
Reader Comments
I had a use case for this once. CFWheels uses an ORM and populating a user object for a "New User" form might look something like this...
However, sometimes...for reasons I could never figure out...it might sometimes fail to create the model the first time. So, if it failed, I would retry once. That seemed to do the trick. Again, not sure why it sometimes failed. And the solution definitely felt hacky. But, it worked.
@Chris,
Sometimes, you just have to get work done, and "understanding" why something happens isn't necessary :D I actually have something similar in my code - I have a MongoDB query that I have to retry. I have no idea why it fails sometimes; but the retry fixes it in like 99.99% of cases. And, not getting the error is good enough for me, ha ha.
@Ben
Exactly! Sometimes ideal takes a back seat to practical. Thank you for having my back on that. In my case, it's particularly mind boggling because while I have many instances/example of that pattern being used, it only intermittently failed in one specific place. At least I didn't have to wrap all instances in try...catch...retry traps. 🙏
Oooo... Ghost in the machine
@Samuel,
That's how they get you 😂