Ask Ben: Simple Data Caching In ColdFusion
I have a website where I am trying to generate a member of the day, is there anyway to cache data for a set amount of time, I know you can cache a query, just wondering if there are other methods for caching. How would you address this?
This title, Simple Data Caching in ColdFusion, is a little misleading for the post. When is comes to "smart" data caching, there really is no easy way to do it. Easy caching is simply throwing a data value into a persisting scope, like APPLICATION, and then just reading directly from the scope going forward. The moment your need data points to timeout or expire, these rules become much much more complex.
There are a good number of open source projects out there that deal with smart data caching; but, I will try to give you a really simple overview of time-based caching. This cannot handle all the advanced features of some of the stuff out there, but it would be fine for some basic stuff like cached output generation. If you want some more general information and more ColdFusion-based tools, check out Tyson Vanek's presentation, Are You "Cashing In" on Caching?
In order to make our data-caching lives easier, we don't want to have to deal with the caching directly. We want some sort of go-between that handles calls from the code and maintains the data cache without complicating our lives (as much as possible). To handle that, we are going to create a small ColdFusion component, SimpleCache.cfc. This object will handle the internal storage of data and only has a few methods:
Init() :: Any
Initializes the ColdFusion component. We don't need to pass anything in to our SimpleCache.cfc.
GetData( Key ) :: Any
Gets the cached data stored at the unique key. If no data exists at the given key, the SimpleCache component throws an exception.
HasData( Key ) :: Boolean
Determines whether or not the data cache has a value stored at the given unique key.
SetData( Key, Data, ExpirationDate ) :: Void
Stores the given data at the given unique key in such a way that it will no longer exist after the given expiration date.
Not too much going on here, but there a few highly important points. For starters, at no point are well telling the SimpleCache.cfc where to store the data. The SimpleCache.cfc ColdFusion component operates under the assumption that it will be cached within some persistent scope (ex. APPLICATION); then, based on that assumption, it caches the data internally and since it is being cached, all of its internal data is being cached. Remember, it is a best practice for a component to not know how or where it is being stored; we want it to become reusable and system-independent.
Furthermore, all data is stored by unique key. If you try to cache two different points of data with the same key (even if they are saved from different files), they will overwrite each other.
When we store data, we need to pass in an expiration date. This gives us the flexibility to use either a specific expiration date or a calculated expiration date using a "timeout" value. More on this later.
One super important thing to notice here, and part of why smart caching is never easy, is that fact that GetData() will throw an exception if the requested data does not exist. Yes, there is a HasData() method which will check the cache for key existence; but, realize that with time-sensitive storage, calling a HasData() before calling GetData() does NOT ensure that the cached data actually exists at the time GetData() is called.
We will cover the SimpleCache.cfc usage in a minute, but let's first take a quick look at the code that is responsible for SimpleCache.cfc ColdFusion component:
<cfcomponent
output="false"
hint="Handles simple time-based data caching.">
<!--- Set up instance variables. --->
<cfset VARIABLES.Instance = StructNew() />
<!--- This will be the structure to hold our cached data. --->
<cfset VARIABLES.Instance.Data = StructNew() />
<cffunction
name="Init"
access="public"
returntype="any"
output="false"
hint="Returns an initialized component.">
<!--- Return This reference. --->
<cfreturn THIS />
</cffunction>
<cffunction
name="GetData"
access="public"
returntype="any"
output="false"
hint="Returns the given data item from the data cache (will throw exception if the requested data does not exist).">
<!--- Define arguments. --->
<cfargument
name="Key"
type="string"
required="true"
hint="The unique key for this data entry."
/>
<!--- Update the data cache. --->
<cfset VARIABLES.UpdateCacheData(
ARGUMENTS.Key
) />
<!---
Now that the cache has been updated, simply return
the given data item. If the data does not exist,
this will throw an error.
--->
<cfreturn VARIABLES.Instance.Data[ ARGUMENTS.Key ].Data />
</cffunction>
<cffunction
name="HasData"
access="public"
returntype="boolean"
output="false"
hint="Checks to see if the given data item exists in the cache.">
<!--- Define arguments. --->
<cfargument
name="Key"
type="string"
required="true"
hint="The unique key for this data entry."
/>
<!--- Update the data cache. --->
<cfset VARIABLES.UpdateCacheData(
ARGUMENTS.Key
) />
<!---
Now that the cache has been updated, simply return
the existence of the given key in the cache.
--->
<cfreturn StructKeyExists(
VARIABLES.Instance.Data,
ARGUMENTS.Key
) />
</cffunction>
<cffunction
name="SetData"
access="public"
returntype="void"
output="false"
hint="Sets the data in the cache.">
<!--- Define arguments. --->
<cfargument
name="Key"
type="string"
required="true"
hint="The unique key for this data entry."
/>
<cfargument
name="Data"
type="any"
required="true"
hint="The data being stored at the given key."
/>
<cfargument
name="ExpirationDate"
type="numeric"
required="true"
hint="The date on which this data will expire and be removed from the cache."
/>
<!--- Set the local scope. --->
<cfset var LOCAL = StructNew() />
<!--- Create the cache item. --->
<cfset LOCAL.Item = StructNew() />
<cfset LOCAL.Item.Data = ARGUMENTS.Data />
<cfset LOCAL.Item.ExpirationDate = ARGUMENTS.ExpirationDate />
<!--- Put item in cache. --->
<cfset VARIABLES.Instance.Data[ ARGUMENTS.Key ] = LOCAL.Item />
<!--- Return out. --->
<cfreturn />
</cffunction>
<cffunction
name="UpdateCacheData"
access="private"
returntype="void"
output="false"
hint="Checks to see if the given data tiem needs to be removed from the cache (and removes it if necessary).">
<!--- Define arguments. --->
<cfargument
name="Key"
type="string"
required="true"
hint="The unique key for this data entry."
/>
<!---
Check to see if the data item has expired. This
will be true if the set expiration date is less
than Now().
--->
<cfif (VARIABLES.Instance.Data[ ARGUMENTS.Key ].ExpirationDate LTE Now())>
<!---
This data item has expired. Remove it from
the data cache.
--->
<cfset StructDelete(
VARIABLES.Instance.Data,
ARGUMENTS.Key
) />
</cfif>
<!--- Return out. --->
<cfreturn />
</cffunction>
</cfcomponent>
That code is quite simple. Very little going on; there's just a private scope to which we are setting and getting small structs of data that have expiration date/time stamps. The complicated part of caching is the interaction with the caching mechanism. Because, we have the potentially to throw errors on GetData() and because there is no good way to ensure locking (as this is a globally accessible component), the best we can do is abstract the SimpleCache.cfc interaction.
This can be done in different ways depending on what kind of data needs to be cached and how that data needs to be accessed. For example, if you needed to cache queries, you might create a special CFC that decorates a Data Access Object (DAO) or Service Object and intercepts methods calls. One of the easiest kinds of caching, which is what we will discuss, is generated HTML content caching.
In generated HTML content caching, we want to generate all or part of an HTML page and then cache it so that it doesn't need to be generated for each request. To handle this kind of caching more gracefully, I have created a very simple ColdFusion custom tag that handles the complicated interaction between the calling code and the SimpleCache.cfc.
Here is an example of a ColdFusion page that uses my SimpleCache.CFM custom tag:
<cfoutput>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>Simple Data Caching Example in ColdFusion</title>
</head>
<body>
<h1>
Simple Data Caching Example in ColdFusion
</h1>
<p>
Current Timestamp:
#TimeFormat( Now(), "hh:mm:ss TT" )#
</p>
<!---
This data will be cached using a timeout. This data
will be persisted for 5 minutes.
--->
<cf_simplecache
key="exampleA"
timeout="#CreateTimeSpan( 0, 0, 5, 0 )#"
cache="#APPLICATION.Cache#">
<p>
Cached Timestamp (Timeout):
#TimeFormat( Now(), "hh:mm:ss TT" )#
</p>
</cf_simplecache>
<!---
This data will be cached using an explicit expiration
date and will be persisted until February 1st.
--->
<cf_simplecache
key="exampleB"
expirationdate="02/01/2008"
cache="#APPLICATION.Cache#">
<p>
Cached Timestamp (Expiration):
#TimeFormat( Now(), "hh:mm:ss TT" )#
</p>
</cf_simplecache>
</body>
</html>
</cfoutput>
As you can see, there are two ways to use this tag. Both tags take the unique key identifier for the data point and a reference to the SimpleCache.cfc instance, but one tag uses the Timeout attribute, which takes a time span, and one tag takes an expiration date, which uses a hard coded date/time stamp. Both tag-usages work by storing the generated content of ColdFusion custom tag with a calculated expiration date.
By running the page a few times, we get the following output:
Simple Data Caching Example in ColdFusion
Current Timestamp: 01:52:02 PM
Cached Timestamp (Timeout): 01:51:46 PM
Cached Timestamp (Expiration): 01:51:46 PM
Notice that the cached time stamps are a few seconds behind the current timestamp. The caching has executed correctly. But, realize that this isn't just a matter of not showing the new content - the SimpleCache.cfm ColdFusion custom tag is NOT executing the nested tags if the cached data is still available. This is where the performance gains come from.
The code for the SimpleCache.cfm custom tag is very straightforward:
<!--- Check to see which mode this tag is executing in. --->
<cfswitch expression="#THISTAG.ExecutionMode#">
<cfcase value="Start">
<!--- Define tag attributes. --->
<cfparam
name="ATTRIBUTES.Key"
type="string"
/>
<cfparam
name="ATTRIBUTES.Timeout"
type="numeric"
default="#CreateTimeSpan( 0, 0, 30, 0 )#"
/>
<cfparam
name="ATTRIBUTES.ExpirationDate"
type="numeric"
default="0"
/>
<cfparam
name="ATTRIBUTES.Cache"
type="any"
/>
<!---
In the beginning of the tag, we just want to try
to get the data before the contents of the tag
execute. However, this will throw an error if
the data doesn't exist. If so, we want the tag
to fully execute.
--->
<cftry>
<!--- Try to output the cached data. --->
<cfset WriteOutput(
ATTRIBUTES.Cache.GetData( ATTRIBUTES.Key )
) />
<!---
Exit out of tag. This will make sure that the
contents and code enclosed in this ColdFusion
custom tag do not execute.
--->
<cfexit method="exittag" />
<!--- Catch any errors that happened. --->
<cfcatch>
<!---
There was an error getting the data.
Therefore, the cache was not updated
and we need to let the tag fully
execute.
--->
</cfcatch>
</cftry>
</cfcase>
<cfcase value="End">
<!---
Now that the tag has fully executed, let's store the
generaged output in the cache. If the user provided
an expiration date, we will use that, otherwise, we
will calculate expiration based on the timeout.
--->
<cfif NOT Val( ATTRIBUTES.ExpirationDate )>
<!---
Calculate expiration date and store it back into
the attributes scope.
--->
<cfset ATTRIBUTES.ExpirationDate = (Now() + ATTRIBUTES.Timeout) />
</cfif>
<cfset ATTRIBUTES.Cache.SetData(
ATTRIBUTES.Key,
THISTAG.GeneratedContent,
ATTRIBUTES.ExpirationDate
) />
</cfcase>
</cfswitch>
Not much going on, right? I have chosen to pass in a reference to the APPLICATION.Cache object. This is so that the system is less coupled to the architecture of the custom tag itself. However, you could easily have just referred to APPLICATION.Cache within the custom tag and not worried about passing it in. I don't think it is a best practice one way or the other because I don't feel that custom tags need to be so cohesive.
If you want to see the Application.cfc that is responsible for creating and caching the SimpleCache.cfc instance itself, here it is:
<cfcomponent
output="false"
hint="Set up the application configuration and handle application-level events.">
<!--- Set application settings. --->
<cfset THIS.Name = "SimpleCacheDemo" />
<cfset THIS.ApplicationTimeout = CreateTimeSpan( 0, 0, 5, 0 ) />
<!--- Set page request settings. --->
<cfsetting
showdebugoutput="false"
requesttimeout="10"
/>
<cffunction
name="OnApplicationStart"
access="public"
returntype="boolean"
output="false"
hint="Handles the application configuration and initialization.">
<!---
Create a simple data cache instance and persist
it in the APPLICATION scope.
--->
<cfset APPLICATION.Cache = CreateObject(
"component",
"SimpleCache"
).Init()
/>
<!--- Return out. --->
<cfreturn true />
</cffunction>
<cffunction
name="OnRequestStart"
access="public"
returntype="boolean"
output="false"
hint="Handles pre-page processing of each request.">
<!---
Check to see if we need to re-initialized
the application.
--->
<cfif StructKeyExists( URL, "reset" )>
<!--- Re-initialized application. --->
<cfset THIS.OnApplicationStart() />
</cfif>
<!--- Return out. --->
<cfreturn true />
</cffunction>
</cfcomponent>
Like I was saying, smart caching is not an easy thing to implement; and, there is sooo much more that can be done than what's covered in the brief overview I have given here. But, notice that by using a simple CFC and a simple ColdFusion custom tag, we can make a complex piece of functionality much easier than you might first expect.
Sorry that I didn't quite touch fully on the problem at hand (member of the day), but I hope this helps give a little insight into the wonderful world of data caching.
Want to use code from this post? Check out the license.
Reader Comments
Hi Ben,
This is great way of Data caching approach in ColdFusion. Please give me some ideas on how to bring the bookmark link at the bottom of our site using ColdFusion like add to Del.icio.us , Add to Digg in your blog
@Akbar,
I am not sure what you are asking?
I am seeing lot of bookmarks links like del.icio.us, Digg etc... At your site at the bottom of every page so can you tell me that how can we show this bookmark links at our site at the end of every message is there a way in ColdFusion to implement this?
@Akbar,
I hand-coded my social bookmarking links, however, there are plug-ins and Javascript files that you can easily include that create those links for you. For example:
http://sharethis.com/
http://www.addtobookmarks.com/
Good luck.
Thanks Ben.
Hi,
Im working on a report where i need to pull out the working hours spent on a particular company's work
I have 40 different companies on the list, and say that person aa has worked on 5 companies and person bb worked on another 5. It is possible that in future person aa works on the other 5/10 companies.
Now I want to Pick the company names and the individual hours on them for person aa and similarly for person bb, whichever company they work on out of the 40 listed companies, on any given day with the correspondent hours of work on them individually.
Im taking out the number of hours worked on the company through a pivot for 10 different people.
I have created a table which gives me the result of number of hours spent by 10 different people on 40 different companies.
I want to only pick the name of the company (with a daily variance of name and hours) and the number of hours worked by a person on a given day and paste it on a differet sheet in the same excel.
How do I do it?
The result required is in the under mentioned format
1st day's work for aa
Company No of Hours
A 1
B 2.5
C 3.4
2nd day's work for aa
Company No of Hours
E 2.1
F 0.8
S 1.78
Hi Ben,
It seems Coldfusion does not implement an init function like I expect it to...
Here's what I do.
in Application.cfm:
<cfset APPLICATION.objects.db = CreateObject('component','components.db').init(dsn_users='users',dsn_guestbook="guestbook",dsn_irama="irama",maxStringLength=25)>
in components.db:
<cffunction name="init" access="public" output="no" returntype="components.db" >
<cfargument name="dsn_users" required="false" default="users">
<cfargument name="dsn_guestbook" required="false" default="guestbook">
<cfargument name="dsn_irama" required="false" default="irama">
<cfargument name="maxStringLength" required="false" default="25">
<cfset VARIABLES.dsn.users="#arguments.dsn_users#">
<cfset VARIABLES.dsn.guestbook="#arguments.dsn_guestbook#">
<cfset VARIABLES.dsn.irama="#arguments.dsn_irama#">
<cfset VARIABLES.maxStringLength="arguments.dsn_maxStringLength">
<cfreturn this>
</cffunction>
When I later call a method in this cfc and refer to VARIABLES.dsn.users Coldfusion says
"Element DSN.USERS is undefined in VARIABLES."
So there is no instance data!
But when I do this:
<cfset VARIABLES.dsn.users="users">
<cfset VARIABLES.dsn.guestbook="guestbook">
<cfset VARIABLES.dsn.irama="irama">
<cfset VARIABLES.maxStringLength="25">
<cffunction name="init" access="public" output="no" returntype="components.db" >
<cfreturn this>
</cffunction>
everything works as expected nd VARIABLES.dsn.users is known further on. Is it not possible to store arguments passed to a method in the Variables scope of a cfc?
Thanks for your time,
Marc
...Solved,
Just before where I do a call to the function in the cfc I did a cfinvoke and called on the resultvar of that operation. The init() method had not been invoked.
I did this because Godaddy is not supporting Createobject() so I had to rewrite parts of the app and obviously didn't bother to rewrite all...
:S
Sorry to bother,
Marc
@Marc,
Glad you got that figured out :)
Can I use this to cache queries?
@Matt,
You should be able to cache anything that you can put in a variable (so, Yes, ColdFusion queries could be cached this way).
Ben, this may be what I'm looking for. thanks for the great info.
Ben, first thanks for all of these great tutorials. I think the use of the custom tag to break the flow of generating new data is a great idea, but from what I can tell, THISTAG.GeneratedContent will only store a string.
In this case, variable bar contains text, not a parsed XML object, which is what is being returned. Objects of other datatypes must be converted with wddx or json.
I think a possible alternative would be to include a cf_cache sub tag that would allow data to be passed directly in as any type, but I don't know for sure as I've never had a reason to create nested custom tags before. I am hoping it would be optional too, so GeneratedContent could still be used if wanted. So I am looking forward to diving in and I am thinking something along the lines of
I know already that this could be done with cfcatch, but it just wouldn't be as pretty..
Hi Ben any suggestions on how to improve the following:
<cffunction name="CacheUsers" access="public" output="no" returntype="void">
<cfquery name="Application.DB.Users" datasource="database">
SELECT *
FROM users
</cfquery>
</cffunction>
This function is called on application start or when user table is modified.