Using Nested Locks To Synchronize Background Data Cleanup In ColdFusion
As I'm building out the Dig Deep Fitness MVP, I'm having to implement functionality that I might ordinarily implement in a more robust fashion given better resources (ie, when someone else is paying for the servers). For example, I would normally use Redis to build a one-time token service. But, when writing the same functionality exclusively in ColdFusion, I have to get a little more low-level when implementing the locking (that Redis would normally apply). Specifically, I wanted to think about how to handle locking when I have a background process that needs to clean-up and expunge expired data.
Most one-time use tokens are both single-use and volatile; meaning, they expire and auto-invalidate even if no one ever attempts to verify them. When using Redis, any key can be given a TTL (Time to Live), which will handle the expiration automatically. But, if I need to implement the same concept in ColdFusion, I must have a background process (such a Scheduled Task), that runs periodically to remove the expired data.
What this means is that I might end up with two different types of race conditions:
Two concurrent user requests examining the same data.
One user request and one background request examining the same data.
In order to keep things predictable and bug-free, we need to use CFLock to synchronize data access. But, our two different race conditions above deal with two different scopes of data. I think what I can do here is use two different nested locks to handle the two different scopes.
The outer lock will deal with the background pruning of all expired tokens. And, the inner lock will deal with a user verifying a single token. Of course, we don't want the outer lock to single-thread all access, so it's going to be a read-only lock. Here's the snippet of my verifyToken() method:
component {
/**
* I verify that the given token exists and is still valid. Verifying the token
* implicitly causes the one-time-use token to be deleted.
*/
public boolean function verifyToken( required string token ) {
// All code that might REMOVE a token is contained within this named lock. In this
// case, since we are only dealing with a single token, I'm going to use READONLY
// so that this doesn't become a single-threaded process UNLESS the background
// pruning method is being called, in which case, no single-token verification can
// take-place until the EXCLUSIVE pruning lock is released.
lock
name = "TokenService.PruneTokens"
type = "readonly"
timeout = 3
{
// In order to make sure that we don't allow concurrent requests to verify the
// same token twice, we must single-thread the verification of the token.
lock
name = "TokenService.VerifyToken.#token#"
type = "exclusive"
timeout = 3
{
// ... verify given token value ...
} // END: Single Token Lock.
} // END: All Token Lock.
}
}
As you can see, the very first thing we do here is enter a read-only lock, TokenService.PruneTokens. On their own, read-only locks have no performance overhead; so, this shouldn't create any unnecessary bottlenecks across user-facing. Then, internally, I enter a standard exclusive lock that is scoped to the given token that I need to verify for the user (note that the name of the lock incorporates the token value).
The background process that performs the actual pruning of expired tokens calls a different method, pruneTokens(), which enters the same outer lock, but this time with an exclusive access right:
component {
/**
* I prune all expired tokens.
*/
public void function pruneTokens() {
// All code that might REMOVE a token is contained within this named lock. In this
// case, since we are examining ALL TOKENS, we want the lock to be EXCLUSIVE. This
// way, we can be sure that this is the only process that has WRITE permissions to
// delete tokens during the time of execution. All of the SINGLE-TOKEN threads,
// which have read-only access to this named lock will have to block-and-wait
// until this thread is done.
lock
name = "TokenService.PruneTokens"
type = "exclusive"
timeout = 3
{
// ... remove any expired tokens ...
}
}
}
Because this exclusive CFLock tag is using the same name as the readonly lock in the previous method, the following outcomes should take place (if I am correctly understanding locking dynamics):
The
pruneTokens()method will block-and-wait for any requests currently going through theverifyToken()method (due to the outer lock). Once those requests have completed, thepruneTokens()method will enter the exclusive lock.Once the
pruneTokens()method obtains its exclusive lock, all newverifyToken()requests will block-and-wait on the pruning process to complete before entering the read-only outer lock.
I believe that this locking approach should give us optimal throughput while also synchronizing code in a predictable fashion. In a perfect world, I'd be using Redis to manage expirations / time-to-live for me. But, while I have to implement my own background process to purge expired data, I think nested CFLock tags will serve me well.
Want to use code from this post? Check out the license.
Reader Comments
When building an MVP or any app that has not reached breakaway scale, aka most business and personal apps its important to embrace "The Majestic Monolith". This means many things, but here is means keep all your functionality in one code base, one deploy step etc.
The primary outcome of "The Majestic Monolith" here is speed and consistently of new features, most project die due to lack of product velocity. Your method above for validating and expiring One-Time-Tokens is simple and fast and I think it will suit your project well for some time.
I love CFML. I'm very proficient in it and I'm able to express and idea and deploy the feature quickly. I've yet to find a task that I can not do with CFML. One side note, I like to keep app configuration, native to the app as much as possible, so I can do an x-copy deploy.
How do you intent to set up the scheduled tasked to "prune" the tokens? I've created a wrapper around schedule, which allows the app to initiate the scheduled tasks on app startup and at a later step verify the task is still present and running.
I have a separate internal tool that is an extension to the schedule wrapper which is inspired by the rails tool DelayedJob. I call it cfDelayedJon and it lets you run arbitrary cfscript on a schedule or at some point in the future. Both, just work well with the ""The Majestic Monolith" concept: with out of the box lucee / adobe cfml
@Peter,
I'm 💯 right there with you! There's no need to add complexity until complexity is warranted! For now, keeping all the code in the app is going to suit me just fine.
Initially, I was going to use a scheduled task to prune the tokens in the background. And, I was hoping I could just configure it in the
Application.cfc. I'm pretty sure I can do that in Lucee CFML; but, I can't remember if I can do that in Adobe ColdFusion.But, in relation to the MVP, I'm actually going even simpler. In my
getToken()method for provisioning new tokens, I'm just going to call thepruneTokens()method as the very first step:In the future, calling
pruneTokens()here might be a performance bottleneck. But, until we actually get a lot of volume, it will be basically instantaneous. Especially since I'm not constantly generating one-time tokens; but, am only generating them in certain workflows.Just trying to lean hard into the MVP mindset.
If anyone is curious, this is kind of off-topic, or rather topic-adjacent, but I ended up using the one-time token concepts here to power a passwordless login system in my Dig Deep Fitness app:
www.bennadel.com/blog/4473-building-a-magic-link-passwordless-login-in-coldfusion.htm
Post A Comment — ❤️ I'd Love To Hear From You! ❤️
Post a Comment →