Adobe ColdFusion Parses JSON Into Non-Ordered Structs
In ColdFusion, an ordered struct (aka linked struct) allows keys to be iterated in the same order in which they were defined. This is hugely valuable for workflows that involve MongoDB queries and Hashed Message Authentication Codes (HMAC). But, one quirky behavior that I came across today is that when Adobe ColdFusion parses JSON strings, it parses the object notation into an unordered struct. This means that you lose the "orderedness" through the serialization and deserialization workflow.
To be clear, I'm not saying that this bug. In fact, if you look at the JSON specification, it specifically calls out objects as being unordered:
An object is an unordered set of name/value pairs.
But, this behavior diverges from Lucee CFML and from JavaScript (the grand-daddy of JSON implementations).
To see this in action, let's look at a small demo. In the following ColdFusion code, I'm creating an ordered struct in which the keys are being defined in reverse alphabetical order. Then, I'm running said struct through the serialization workflow and rendering the result:
<cfscript>
product = server.keyExists( "lucee" )
? "Lucee CFML"
: "Adobe ColdFusion"
;
// Define original struct as ORDERED - KEYS are defined in REVERSE alphabetical order.
value = [
c: "1st",
b: "2nd",
a: "3rd"
];
// Output original struct.
writeOutput( "<h1>#product#</h1>" );
writeOutput( "<h2>Pre-Serialization</h2>" );
writeDump( value );
// Output struct after SERIALIZATION WORKFLOW.
writeOutput( "<h2>Post-Serialization</h2>" );
writeDump( deserializeJson( serializeJson( value ) ) );
// Output serialized values.
writeOutput( "<br /><hr />" );
writeOutput( serializeJson( value ) );
</cfscript>
Here's what we get when we run this CFML code in the two ColdFusion runtimes:
Notice that in Lucee CFML, the struct parsed from the JSON string is an ordered struct with a key-order that matches the original struct definition. In Adobe ColdFusion, however, the struct parsed from the JSON string an unordered struct with a key-order that is reverse of the original struct definition.
JSON and Hyrum's Law
Earlier in this post, I mentioned that Adobe ColdFusion's behavior is not a bug because the JSON specification identifies objects as an unordered set of key-value pairs. As such, Adobe's implementation is consistent with the specification.
But, that brings me to Hyrum's Law:
With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.
Hyrum's law essentially says that an API is bound by its actions not its promises. And, if an API does something in a particular way, that behavior inherently becomes part of the API's contract regardless of whether or not that behavior is explicitly defined.
If we look at the language that puts the "JavaScript" in "JavaScript Object Notation", we can see that key-order is maintained during the serialization life-cycle:
Notice that in the Chrome Dev Tools, the key-order of the object is maintained through the JSON.stringify() / JSON.parse() life-cycle. Granted, this is just one JavaScript runtime. But, the same behavior can be observed in Safari's JavaScript runtime and the Node JavaScript runtime (which I think might be what Chrome is using anyway).
From Hyrum's perspective, one could argue that if this is what JavaScript runtimes have been doing since the beginning of JSON—maintaining key-order—then this behavior has become a de facto part of the JSON specification, regardless of what the JSON specification actually states. Which would make Adobe's implementation buggy.
But, that argument cuts both ways. Meaning, if this is how Adobe ColdFusion has always implemented its JSON serialization life-cycle, then Adobe's behavior has become a de facto part of the Adobe ColdFusion language specification, even if the documentation states nothing about key-order.
In fact, it's completely reasonable to believe that there are systems out there that have logic (perhaps something related to a replayable integration test artifact) that somehow depends upon the key-sorting behavior that Adobe ColdFusion has implemented. And, if Adobe were to suddenly change this behavior (to be more in alignment with the JSON world at large), it would become a breaking change for some legacy ColdFusion applications.
Aside: This is kind of how I came across this behavior. I was building some logic to see if a subset of struct-keys for an in-memory model had diverged from the same data that was stored on disk. But, when reading the data off of disk, the order of the keys no longer matched the order of the in-memory keys.
Software maintenance is complicated!
Want to use code from this post? Check out the license.
Reader Comments
I would argue that if you need a struct to be ordered...you should not be using a struct, you should be using an array.
I would also argue that getting values by the position of the key in the struct rather than the key name is doing it wrong.
@Scott,
I think it's just a matter of understanding the tools at hand. If an unordered struct is created, then I agree, you should not depend on the order of the keys. But, if a system intentionally generates an ordered struct in a given workflow, I think it's completely reasonable to leverage the feature of that data-type.
I think it really comes down to whether or not the code is intentional and clear about what kind of struct is being produced.
@Ben Nadel,
Meh...ordered or not, I am NEVER going to get values from a struct based on the position of the key in the struct (nor can I think of a use case for doing so).
Which brings me to two interesting thoughts:
How would you even get the value of a key based on the position of that key in a struct?
Assuming its part of the solution to the question above, is the value returned from
structKeyList()different between the two examples you shared?I recently (6 months ago?) discovered ordered structs. I love 'em! But not for "important" order, but so that when I dump the objects they look like the order I built them in.
I would say that ACF should, however, convert to using ordered structs with deserializeJSON(), since it could be done that way. Probably just hasn't been touched in forever.
It would be interesting if you had two unordered structs in an array, at least currently they'd come back similar. Even if they were serialized as [1]{c,b,a} and [2]{a,c,b}.
From memory, this change was partly inspired whilst we started to implement
.CFConfig.jsonsupport.As for compat concerns, there is no guarantee about the order of an unordered struct, even if it seems to be consistent
This conversation inspired me to share another use of ordered structs: creating a very simple fixed-size cache:
www.bennadel.com/blog/4677-using-an-ordered-struct-as-a-fixed-size-cache-in-coldfusion.htm
Since the order of the key-insertion matches the order of key-iteration, you always know that the first key is the oldest key:
.keyArray().first()This allows you to maintain a cache size constraint without having to store yet another piece of information about the cache - the cache itself stores that for us.
All to say, I think ordered structs are a very valuable part of the language. And, as long as you know that you're dealing with an ordered struct, it should be embraced like any other data-type.
@Zac,
To be clear, I prefer Lucee's implementation. And, I think it more closely matches the expectation in the JavaScript community (despite what the JavaScript specifications say). And, presumably, JavaScript will never change since the web bends-over-backwards to never break backwards compatibility (see "SmooshGate").
Micha and I are glad you like it!
I've been using JSONUtil since ColdFusion 8/9. I modified the deserializeJSON method to explicitly generated ordered structs so that the struct keys are in the same order as the original JSON string.
https://github.com/cfcommunity/jsonutil
The reason for this? We save data in JSON format and this approach enables us to see the data in the same order as it was originally saved and compare it against a third-party API response or perform browser-based diffs on a string.
@James,
That use-case is actually kind of what I was intending as well: to compare one version of the payload to another. I thought that would be an easy win - to just compare the JSON outputs. But once one of them went through the serialization life-cycle, the orders stopped lining up. That said, I also ran into other issues with some numbers being decimals. It's messy business :)
Post A Comment — ❤️ I'd Love To Hear From You! ❤️
Post a Comment →