Calculating A Consistent Cache-Friendly Expiration Date For Signed-URLs In Lucee 5.3.2.77
Yesterday, I shared the results of a performance experiment in which I started caching cryptographically signed-URLs in Redis using Lucee 5.3.2.77. The signed-URLs in the experiment where for a CDN (Content Delivery Network) that pulled-through from Amazon S3. And, after posting yesterday, I wasn't sure if I've ever looked at the mechanics of a generating cache-friendly, CDN-friendly signed-URLs in a ColdFusion application. As such, I wanted to quickly look at how I produce these URLs for our CDN using a bucketed date-scheme in Lucee 5.3.2.77.
When using a signed-URL in conjunction with a CDN (Content Delivery Network), just signing the URL is insufficient. This is because every URL will, essentially, end up creating a unique URL; and, unique URLs are the enemy of caching. In order for a cache - which is what a CDN is - to be effective, it has to serve requests for URLs that don't change with high frequency.
Signed-URLs are generally "unique" because they entail some sort of expiration date that changes based on the current time-stamp. As such, a signed-URL generated right now will be different than the same URL generated 5-seconds from now.
To make signed-URLs cache and CDN friendly, we have to "bucket" the expiration dates. That is, every signed-URL that is generated within some window of time has to use the same exact expiration date. This way, the signed-URLs will be consistent for that window of time, allowing subsequent requests (to the signed-URL) to become "cache hits" in the CDN:
Essentially, every signed-URL that is generated with a window gets bucketed by that window, using an expiration date that represents a single point-in-time relative to that window.
In a ColdFusion application, we can achieve this bucketing with a little date-math. For example, if we want to bucket signed-URLs by month, we can just use the 1st-day of the following month as the expiration date. This way, every signed-URL that is generated in a given month will be valid until the end of that month. So, a signed-URL that is generated (for a specific resource) on Aug-1 will be valid until Sep-1. And, the same signed-URL that is generated on Aug-31 will be valid until Sep-1.
In an application experience, the relatively short expiration at the end of the "window" is OK because subsequent requests to the application will just generate URLs for the next "window". However, we have to be aware that not all signed-URLs are consumed directly within the ColdFusion application. For example, a signed-URL that is used to render an image in an email won't have the luxury of making subsequent requests to the application. As such, a signed-URL in an email may no longer be valid by the time the user opens said email.
To make signed-URLs more flexible for this kind of extra-application consumption, we can do one of two things:
- We can generate different kinds of signed-URLs for external consumption.
- We can give signed-URLs a "rolling window".
In this case, when I say "rolling window", what I mean is that signed-URLs that are generated in the back-half of the "window" use a more-distant-future expiration date. So, in a monthly bucket, a signed-URL that is generated on Aug-1 will be valid until Sep-1; but, the same signed-URL that is generated on Aug-20 will be valid until Oct-1 - one month more than the previous signed-URL:
I tend to go with Option 2 because it keeps the application logic simple without dramatically changing the rules around signed-URLs.
To see this logic in action, I've created a Lucee CFML file that loops over almost "two windows" of time and generates expiration dates for each day within that window:
<cfscript>
/**
* I return the "monthly" bucket for the given date (defaults to now). This helps with
* expiration dates on signed-URLs that will be consumed as the origin of a CDN pull-
* through.
*
* @input I am the date being used to calculate the bucket.
*/
public date function getMonthlyBucket( date input = now() )
cachedWithin = "request"
{
var inputYear = input.year();
var inputMonth = input.month();
var inputDay = input.day();
// To calculate the bucket for the input date, we need to roll back to the
// beginning of the month. And then, roll forward one month. This will place the
// bucket on the 1st of the next month.
var bucket = createDate( inputYear, inputMonth, 1 ).add( "m", 1 );
// Rolling forward to the next month will help with consistent in-app URLs, where
// the URLs are being GENERATED ON EACH REQUEST. However, not all URLs are
// consumed in-app. Some URLs are consumed externally, such as in an email.
// Because of this, we may want to roll-forward an ADDITIONAL MONTH if we are
// close to the end of the current bucket. This way, an email with embedded
// images that is sent at the END OF THE BUCKET will continue to work for at
// least two-weeks (roughly) before they expire.
if ( inputDay > 15 ) {
bucket = bucket.add( "m", 1 );
}
return( bucket );
}
// ------------------------------------------------------------------------------- //
// ------------------------------------------------------------------------------- //
fromDate = createDate( 2019, 8, 1 );
// Let's examine the bucket calculation over a 50-day period.
cfloop(
index = "i",
from = 0,
to = 50,
step = 1
) {
loopDate = fromDate.add( "d", i );
// Get the monthly bucket for the given date. Normally, this would just be
// assumed to be "now"; but, for the sake of this demo, we're going to provide
// an explicit test date.
bucketDate = getMonthlyBucket( loopDate );
echoLine(
loopDate.dateFormat( "short" ),
" => ",
bucketDate.dateFormat( "short" )
);
}
// ------------------------------------------------------------------------------- //
// ------------------------------------------------------------------------------- //
/**
* I echo each argument, in turn, and then add a line-break at the end.
*/
public void function echoLine() {
arguments.each(
( value ) => {
echo( value & " " );
}
);
echo( "<br />" );
}
</cfscript>
As you can see, in the getMonthlyBucket() function, dates that fall in the first half of the month use the end-of-the-month as the expiration date. But, dates that fall in the latter half of the month use the end-of-the-next-month as the expiration date. And, when we run this ColdFusion code, we get the following output:
8/1/19 => 9/1/19
8/2/19 => 9/1/19
8/3/19 => 9/1/19
8/4/19 => 9/1/19
8/5/19 => 9/1/19
8/6/19 => 9/1/19
8/7/19 => 9/1/19
8/8/19 => 9/1/19
8/9/19 => 9/1/19
8/10/19 => 9/1/19
8/11/19 => 9/1/19
8/12/19 => 9/1/19
8/13/19 => 9/1/19
8/14/19 => 9/1/19
8/15/19 => 9/1/19
8/16/19 => 10/1/19 (Notice the rolling window)
8/17/19 => 10/1/19
8/18/19 => 10/1/19
8/19/19 => 10/1/19
8/20/19 => 10/1/19
8/21/19 => 10/1/19
8/22/19 => 10/1/19
8/23/19 => 10/1/19
8/24/19 => 10/1/19
8/25/19 => 10/1/19
8/26/19 => 10/1/19
8/27/19 => 10/1/19
8/28/19 => 10/1/19
8/29/19 => 10/1/19
8/30/19 => 10/1/19
8/31/19 => 10/1/19
9/1/19 => 10/1/19
9/2/19 => 10/1/19
9/3/19 => 10/1/19
9/4/19 => 10/1/19
9/5/19 => 10/1/19
9/6/19 => 10/1/19
9/7/19 => 10/1/19
9/8/19 => 10/1/19
9/9/19 => 10/1/19
9/10/19 => 10/1/19
9/11/19 => 10/1/19
9/12/19 => 10/1/19
9/13/19 => 10/1/19
9/14/19 => 10/1/19
9/15/19 => 10/1/19
9/16/19 => 11/1/19 (Notice the rolling window)
9/17/19 => 11/1/19
9/18/19 => 11/1/19
9/19/19 => 11/1/19
9/20/19 => 11/1/19
As you can see, expiration dates for signed-URLs will be consistent for the duration of a "rolling window". This will make the signed-URLs cache-friendly, leading to many more "Cache Hits" than "Cache Misses".
When going from a "window" to a "rolling window", little changes from an in-application standpoint. That is, signed-URLs that are generated at the end of the "rolling window" will be different than signed-URLs that are generated one-day later. The real difference here is that the signed-URLs become safer to consume in an external context. By using a rolling window, every signed-URL is guaranteed to work for a time-period that is at least half of the window. This lowers the chances that a user will experience a broken image in an email, even if said email is being opened a few days after it was sent.
Typically, when you are generating a signed-URL, you are providing access to a resource that would otherwise be inaccessible. As such, the type of resource and the security concerns of the application need to be taken into account when developing a URL-generation strategy. But, when you are fronting a secure resource with a CDN (Content Delivery Network), you have to take steps to make URLs more consistent; otherwise, the CDN becomes nothing more than an expensive network hop. By bucketing the expiration date of signed-URLs, you can strike a nice balance between security and performance in a Lucee application.
Want to use code from this post? Check out the license.
Reader Comments