Learning ColdFusion 9: IsNull() And Working With NULL Values
Before ColdFusion 9, there was never really a sense of NULL values in ColdFusion. If you had a NULL value returned from a Java method, it simply destroyed the variable into which it was stored. If you had a NULL value come back from a query, it was presented as an empty string. While that hasn't really changed, ColdFusion 9 now has a method that might make dealing with null values a bit easier. The new IsNull() method takes a variable and returns a boolean as to whether or not that variable is null.
Seeing as this is a completely new concept in ColdFusion 9, I think we need to do some thorough testing. To make this testing easier, I defined a ColdFusion function that returns VOID (NOTE: Remember, VOID is NULL in the ColdFusion world):
<cffunction
name="returnNull"
access="public"
returntype="void"
output="false"
hint="I simply return void / null.">
<!--- Return null. --->
<cfreturn />
</cffunction>
As you can see, this method returns without a given value. This will return a NULL result. With this in place, I ran my first test:
<!--- Get a null value. --->
<cfset request.value = returnNull() />
<!--- Test to see if value is null. --->
Null: #isNull( request.value )#<br />
<!--- Test to see if the key exists. --->
StructKeyExists: #structKeyExists( request, "value" )#<br />
Our variable, REQUEST.Value, contains the NULL result of the ReturnNull(). And, when we run the above code, we get the following output:
Null: YES
StructKeyExists: NO
That's pretty cool. But, I wondered if ColdFusion NULL values were any different than Java NULL values. As such, in my next test, I decided to see how this worked with JavaCast():
<!--- Get a Java null value. --->
<cfset javaNull = javaCast( "null", 0 ) />
<!--- Test to see if value is null. --->
Null: #isNull( javaNull )#<br />
<!--- Test to see if the key exists. --->
StructKeyExists: #structKeyExists( variables, "javaNull" )#<br />
Here, I am using JavaCast() to create a Java NULL value. And, when we run the above code, we get the following output:
Null: YES
StructKeyExists: NO
Pretty slick! When I saw this result, I started to think - if the StructKeyExists() keeps returning NO, then how I am even referring to the variable being tested? I mean, it seems like a bit of a weird chicken-and-egg situation: a variable needs to exist to be referred to but I need to check a variable which I might not be able to refer to because it might not exist? After thinking about this for a second, I tried referring to a completely made up variable:
<!--- Test to see if value is null. --->
Null: #isNull( request.whatIsUpMyMan )#<br />
<!--- Test to see if the key exists. --->
StructKeyExists: #structKeyExists( request, "whatIsUpMyMan" )#<br />
Here, I'm referring to a variable name which I didn't even create previously. And, when we run the above code, we get the following output:
Null: YES
StructKeyExists: NO
So, I guess we can pass any kind of variable name into this new IsNull() method. Because this action would traditionally trigger an exception, I wonder if it is doing some sort of behind-the-scenes error handling. And, if so, can we pass array-notation values to it. Let's try using array notation with a struct:
<!--- Test to see if value is null. --->
Null: #isNull( request[ "whatIsUpMyMan" ] )#<br />
<!--- Test to see if the key exists. --->
StructKeyExists: #structKeyExists( request, "whatIsUpMyMan" )#<br />
Unfortunately, this time, we get the following ColdFusion error:
Element whatIsUpMyMan is undefined in a Java object of type class coldfusion.runtime.RequestScope.
This doesn't work with a struct, but, what about an array with an undefined value:
<!--- Create an array of simple vaules. --->
<cfset data = [ 1, 2, 3 ] />
<!--- Destroy one of the values. --->
<cfset data[ 2 ] = returnNull() />
<!--- Check the existence of that array index. --->
Null: #isNull( data[ 2 ] )#
This time we get the ColdFusion error:
Element 2 is undefined in a Java object of type class coldfusion.runtime.Array.
OK, so we can't use array notation with structs or arrays. It looks like we can only use valid "variable names" or those that would pass the "variablename" validation in ColdFusion's CFParam tag. But, I wonder, are there any limitations to the domain of the base object in question? Meaning, does this work only on ColdFusion objects? Or, can I use this on Java objects as well? Let's have a look:
<!--- Create a java object. --->
<cfset javaObject = createObject(
"java",
"java.lang.StringBuffer"
) />
<!--- Check for a null key. --->
Null: #isNull( javaObject.madeUpKey )#<br />
Here, I am creating a Java object and then referring to an undefined property of it. And, when we run this code, we get the following output:
Null: YES
Ok cool - so it looks like we can use this with any type of object as long as it's stored in a ColdFusion variable.
But, what about queries? Null values have always been massaged by ColdFusion, presenting themselves as strings. I wanted to see if that has changed and IsNull() might work with queries where NULL values are present:
<!--- Create our query object. --->
<cfset friends = queryNew( "name", "cf_sql_varchar" ) />
<!--- Add two rows. --->
<cfset queryAddRow( friends, 2 ) />
<!--- Set a valid value only on the second row. --->
<cfset friends[ "name" ][ 2 ] = javaCast( "null", 0 ) />
<cfset friends[ "name" ][ 2 ] = javaCast( "string", "Tricia" ) />
<!--- Loop over the query to check for null values. --->
<cfloop query="friends">
Null( #friends.currentRow# ): #isNull( friends.name )#<br />
</cfloop>
After we manually construct our query, our first record value should be null and our second record value should contain, "Tricia." When we run the above code, we get the following output:
Null( 1 ): NO
Null( 2 ): NO
It looks like IsNull() does not interact with query values in any special way - NULL column data is still presented as empty strings.
As a final test, I wondered how IsNull() interacted with scope searching, which has always been a huge part of ColdFusion. As you all know, when you refer to an unscoped variable in ColdFusion, the engine searches through a number of scopes to find it. For example, if you refer to "foo", ColdFusion will search through the VARIABLES, FORM, URL, COOKIE, ARGUMENTS, LOCAL, Query, and a number of other scopes looking for it. So, the question becomes, is IsNull() smart enough to know that. To test this, let's try using one of the most common scope searching scenarios - the FORM scope:
<!--- Store a value in the FORM scope. --->
<cfset FORM.tricia = "Awesome!" />
<!---
First, to test, let's just output the value to make sure
ColdFusion knows how to find it (searching scopes).
--->
Tricia: #tricia#<br />
<!---
Now, let's check to see if the form value is null according
to InNull().
--->
Null: #isNull( tricia )#
As you can see, we are simply referring to Tricia without the FORM scope. When we run the above code, we get the following output:
Tricia: Awesome!
Null: YES
Hmm, ok. While at first, you might consider this a bug, it actually isn't. Because of the way ColdFusion searches through scopes, we can refer to the value contained within FORM.Tricia without using the FORM scoping; however, we're not truly referring to the variable, "tricia". As such, I think the above behavior is what is to be expected. That said, be sure you fully understand scope searching before you use IsNull() on variables without scopes. Even better, always scope your non-page-scope variables!
Ultimately, it looks like ColdFusion 9's new IsNull() method is nothing more than a short-hand notation for StructKeyExists(). But, I'm OK with that. Most of what I've blogged about so far in ColdFusion 9 has been about fairly small changes that will result in large performance and contentment increases among ColdFusion programmers. I believe that IsNull() is no different and will, in fact, being hugely appreciated.
Want to use code from this post? Check out the license.
Reader Comments
I admit I was excited to see this new function in CF9. Too bad it doesn't really add any new capability. The potential for misunderstanding of what IsNull() does outweighs any cool factor as a StructKeyExists() substitute. I doubt that I will use it.
As usual, a thorough exploration of the question, Ben!
Ben,
in oracle land a null gets treated like empty string. It's different between databases. Maps quite nicely to CFML.
Not having nulls actually reduces the amount of code you have to write. Over on the FDO project something similiar came up and there was a lot of extra null checking handling code needed.
somewhat related I wanna see automatic null handling for CFQUERYPARAM
http://cfbugs.adobe.com/cfbugreport/flexbugui/cfbugtracker/main.html#bugId=78911
When I see isNull() in your post title, first thing comes to my mind is the query null values. But it's too bad to know that this new function does not do any thing special for that, or actually there is no benefit to use it.
But even I was asking myself, what if you tried to use isDefined() in your examples, will it return some thing different than structKeyExists().
@Ben, seems pretty easy to explain (this is sheer speculation because I can't be bothered to fire up a Java disassembler):
Variable reference syntax ('obj.key') and associative-array reference syntax ('obj["key"]') map to 'obj.get(key)' which returns null if the Map object does not contain the given key, or if the Map object is assigned a value to the given key, and the value is null.
'StructKeyExists(obj, key)' syntax basically maps to the Java 'obj.get(key) != null'.
'IsNull' syntax (this is *not* a function-call!) looks at Variable reference syntax (not associative-array reference syntax) passed to it, splits it on the final '.' into (obj, key), and then does 'not StructKeyExists(obj, key)'.
Java and most programming languages make a clear distinction between 'the variable does not exist' and 'the value does not exist'. Because ColdFusion has for so long made no such distinction between 'the variable does not exist' and 'the value does not exist', you will see extraordinary confusion and inconsistencies surrounding null values assigned to existent variables.
@zac, the lack of null in ColdFusion tends to *increase* the amount of code people have to write. You use null every time you use '<cfargument required="false" />'. You use null every time you use 'if(!StructKeyExists(obj, key)) obj[key] = someValue;'. You just don't see the null handling directly - but depending on the type of code you write, you do plenty of null handling.
@Mark, @Ameen,
While is seems to be mere short hand for StructKeyExists() (what I believe Justice is confirming), I still think it's a more useful and more intuitive approach to checking value. The intent, after all, of IsNull() is going to be more directly apparent to readers than StructKeyExists().
@Zac,
I agree that not having NULLs, at least when dealing with a database, does decrease the amount of code.
@Justice,
I think there has always been a difference between variables and values in ColdFusion, although it might not be that evident. You learn very quickly when dealing with Custom Tags that you can't use value-based references for names, but only variable names (as would pass the CFParam "variablename" validation). But, unless you are dealing with evaluate-later-references, yes, there is very little difference between values and variables in ColdFusion.
@Ben, there certainly is a distinction between variables and values. But is there a distinction between the existence of a variable and the existence of the value the variable is intended to contain? Recall that, in ColdFusion, variables must contain values in order to exist as variables: storing null in a variable destroys the variable; you cannot use 'var x;' at the top of a function; etc. The reason for this seems to be a rather obvious combination of an initial design choice that variables must contain values coupled with a simple internal implementation using java.util.Map.get(object), which returns null to signify either a null has been assigned to the key, or that the key has never been assigned to at all, or that the key has been assigned to in the past but has subsequently been deleted.
The design decision is not necessarily bad: there are certainly number of languages (O'Caml, Haskell) requiring all variables to have values. Although these languages are strikingly different from ColdFusion in so many ways, that one particular feature in common to both these languages and ColdFusion may provide benefit to one side and may impair the other side (or may benefit both, or impair both).
The important thing to keep in mind is that all scopes and structs are *just* Java hashtables. Both nonexistent keys and keys with nonexistent values (null) assigned to them will, in ColdFusion, mean that the corresponding ColdFusion variable does not exist. That ultimately determines all of the following: how IsDefined works, how StructKeyExists works, how referencing a variable using struct-dot-key notation works, how referencing a variable using associative-array notation works, and how the new IsNull function works.
@Ben, to me, thinking this might be null value detection in queries, the purpose of IsNull() was actually counterintuitive.
StructKeyExists(), on the other hand, actually does exactly what it says it does.
@Justice,
I see what you're saying.
@Mark,
That's a good point. But, I think once you do know what it can be used for, I think you will find it useful.
If I remember right, Hibernate makes CRUD decisions (primarily, Save vs. Update) decisions based on null values for Primary Keys. I would assume that adding a more direct reference to null-tests was necessary for ORM integration.
@David, you don't do CRUD with Hibernate. (I don't know what happens with ColdFusion/Hibernate.) Hibernate is simply not an ActiveRecord type of ORM, like Rails and Django have.
Hibernate determines whether an object is persistent (currently attached to a Session, and possibly related to rows that are currently in the database) or transient (not currently attached to a Session, and certainly not related to any rows) based upon certain characteristics of the object identifier property. Hibernate uses this internally to determine, whenever the Session is flushed, whether to send out an INSERT or an UPDATE SQL statement for that object.
Keep in mind that, when you use Hibernate, you do *not* explicitly tell it to update a given row in the database from the new properties of some object. That's simply the opposite of how it works. Hibernate automatically tracks all of your objects behind the scenes, and, whenever the Session is flushed, *automatically* inserts or updates rows based upon what you have done to the objects. When you call Hibernate's save method, you are instructing Hibernate that a given transient object is to become a persistent object, *not* to run an insert statement (Hibernate will figure that much out, when the Session is flushed).
Whenever you work with objects with Hibernate, you are saving transient objects *to the Session* or removing persistent objects *from the Session*, not to or from the database. In fact, if you save an object to the Session and then subsequently remove it from the Session, Hibernate will not issue any statements for INSERTing or DELETEing any rows related to that object to or from the database!
@Justice.
Noted and appreciated.
Would have been nice of them to make an isEmpty() like PHP. Could stop using Len(Trim())
From "Cold Fusion", I thought it was related to Physics.
Great post Ben. I always find my answers in Ben's blogs. Good job
Hi Ben - thorough stuff as usual. Although it's an old post I stumbled across it while scratching my head a bit about these functions.
It's all pretty clear no but the thing that threw me in the first place was this. If you assign for example some java function that returns NULL to a variable then that variable just doesn't get set. For example if I set
session.user = getPageContext().getRequest().getRemoteUser();
And there is no user, then
structKeyExists(session,'user') = NO
isDefined('session.user') = NO
isNull('session.user') = YES
However if you CFDUMP the session structure 'user' is there, value "undefined" and it also appears for example in the CF debugging output. I guess "undefined" is a clue, but I don't understand why CF should still show the key - it's confusing, at least to me
@Ben, great job of 'splainin as usual. I figured I'd share this point though:
We have a udf called IsNull (I know bad idea, but that's what whomever named it,did) that worked fine on CF8. But when our infrastructure folks updated one of the load balanced servers to CF9 the thing blew up (cuz there is a built in isnull func)! Started getting pretty descriptive errors and customers yelling. Of course, once we changed the name of the udf, things were back to normal.
So anybody out there using IsNull for a udf name, change it now before moving to cf9.
-roger
@Richard,
Yes, good point. Visually speaking, the key is in the structure. You will also see this in the Arguments collection in function with non-passed, optional arguments; although, that problem can even present in a slightly different way:
www.bennadel.com/blog/1430-ColdFusion-ARGUMENTS-Keys-Always-Exist-Even-When-Not-Required.htm
... where the key exists in a key-list, but is not considered to be "part" of the structure.
@Roger,
I had that same problem going from CF6 to CF7 I think. I had a function called isUserLoggedIn(), which then became part of the core function list. We had to update dozens of applications on that server. Luckily, things like this are typically a easy fix with an extended-Find on the server. Just a bit of a pain and some down time.
Ultimately, it looks like ColdFusion 9's new IsNull() method is nothing more than a short-hand notation for StructKeyExists().
Pardon me for being so late to the discussion - and I apologize if the point has already been covered - but I believe isNull() is slightly more awesome than structKeyExists(). isNull() will return a true value even when structKeyExists() raises an error. Suppose you want to check to see if a certain struct in the session scope contains a certain key. isNull() will return false if it does, but it will even return true if the struct does not exist in the first place. structKeyExists() will throw an error if the struct does not exist.
<cfset session.the_struct = {} />
<!--- Note the misspelling; this will not throw an error --->
<cfoutput>#isNull(session.the_sturct.myKey)#</cfoutput>
<!--- Throws an error --->
<cfoutput>#structKeyExists(session.the_sturct, "myKey")#</cfoutput>
You might find this functionality useful or not; if not, just keep using structKeyExists(). Hope this helps.
@DCS,
Good point - structKeyExists() will definitely balk if you try to test multiple layers deep. I don't think I ever checked this against isNull(); but from what it sounds like, you have confirmed this. Good insight!
I've scimmed through the comments here and couldn't find the use that I found for it:
var someEntity = EntityLoadByPk('myEntity', myPk);
if( IsNull(someEntity) ){
// do something
}
When EntityLoad and EntityLoadByPk are expected to return a single entity bean and don't find one, they return NULL.
Dominic
@Dominic,
Very nice call. And, for anyone else who is reading this, I believe if you load entities without a PK:
EntityLoad( myEntity, primaryKey )
... and no entity is returned, I *believe* that an empty array is returned... but not 100% sure offhand.
@Richard, @Ben
Regarding [undefined] keys showing up in a cfdump of a struct, this only plays well with certain structs.
Ones I tested with success were:
Session, Application, Cookie, Caller, Arguments, & user defined structs.
Ones I tested with failure were:
Server, Client, URL, Form, Variables, Request, Local
Client actually barfs when trying to assign a Null. I'm disappointed that Variables won't let me create a null variable. I have places where it would be handy to have a null variable around as a placeholder in calling a UDF with a variety of optional parameters.
I currently solve this by using an argumentList or pass in a struct.
Going back to comments by @Justice, it seems that introducing isNull() would force the issue of "distinction between the existence of a variable and the existence of the value the variable is intended to contain."
Is there any difference between isNull(variable) & isDefined("variable")? It depends on if that scope can have null keys.
Variables.null = returnNull();
c.null = returnNull();
Null: #isNull(Variables.null)#
Defined: #isDefined("Variables.null")#
Null: #isNull(c.null)#
Defined: #isDefined("c.null")#
I do not believe that they (currently Adobe) defined Null properly or sufficiently. I love working with CF, but it seems that I'm always having to work around a 'not completely' implemented CF method. RegEx testing being in that list.
-Dan