Consuming Sparse, Unpredictable "omitempty" Golang JSON Payloads Using Null Coalescing In Lucee CFML 5.3.7.48
At InVision, I'm working on yet another "remonolithification" project, this time merging a Go service into my ColdFusion monolith. As part of this subsumption, I have to write CFML code that consumes the JSON (JavaScript Object Notation) payload being returned from a different Go service. I have basically no Go experience; so, this endeavor has been comically challenging given the simplicity of the service that I'm tearing down. It turns out, in Go, you can use an omitempty flag in your deserialization process to make your return payloads wildly unpredictable. To translate the sparse, unpredictable, and potentially missing data into a predictable ColdFusion format, I'm using the null coalescing operator (aka, the "Elivs" Operator) in Lucee CFML 5.3.7.48.
First, a huge shout-out to Bryan Stanley, who has taken time out of his very busy schedule to walk me through some Go code; and to hear me rant excessively about the aspects of Go that make no sense to me as a ColdFusion developer. Such as why on earth you would want to make your JSON payloads harder for other services to consume?! You are a true hero!
At first, I thought subsuming this Go service would require little more than a few simple HTTP calls followed by a few simple deserializeJson() calls. But, when I first tried to implement this naive approach, my ColdFusion code started blowing up due to missing Struct keys. When I dug into the Go code, I noticed that the vast majority of keys have an omitempty notation attached to them. They all looked something like this (pseudo-code):
type PrototypeType struct {
ID string `json:"id,omitempty"`
Name string `json:"name"`
UserID int `json:"userId"`
CompanyID int `json:"companyId,omitempty"`
Archived bool `json:"archived,omitempty"`
Background *ColorType `json:"background,omitempty"`
Device *DeviceType `json:"device,omitempty"`
}
type ColorType struct {
R int `json:"r"`
G int `json:"g"`
B int `json:"b"`
}
type DeviceType struct {
Appearance *AppearanceType `json:"appearance,omitempty"`
ID string `json:"id,omitempty"`
Height int `json:"height,omitempty"`
Width int `json:"width,omitempty"`
Scaling string `json:"scaling,omitempty"`
ScaleToFitWindow bool `json:"scaleToFitWindow,omitempty"`
}
type AppearanceType struct {
Type string `json:"type,omitempty"`
IsRotated bool `json:"isRotated,omitempty"`
Skin string `json:"skin,omitempty"`
}
Notice that most of the json annotations on these complex objects have omitempty. According to the Go documentation on JSON:
The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.
So basically what this means is that when I call this Go service and it returns JSON, the shape of that JSON data is going to be completely different depending on how many "empty" values happen to be in the serialized structure.
It seems unfortunate that an API would have an unpredictable data model; but, thankfully, I can use the null coalescing operator in ColdFusion, ?:, to cast undefined values into the appropriate "empty" values that Go stripped out:
cfml_value = ( unpredictable_go_value ?: empty_value )
It makes my ColdFusion translation layer rather verbose; but, at least it's mostly straightforward and leaves me with a data structure that I can depend on. Here's an example of how I'm handling this in my Lucee CFML code:
<cfscript>
// This represents the sparse JSON payload that might come back from a Go service.
// The "omitempty" flags on the serialization definitions mean that any "empty" value
// will be completely removed from the resultant JSON payload.
goModel = {
name: "My Prototype",
userId: 1234,
companyId: 5678,
background: {
r: 123,
g: 0,
b: 227
},
device: {
id: "d7-landscape",
appearance: {
type: "mobile"
}
}
};
dump(
label = "Golang Model",
var = goModel
);
dump(
label = "Lucee CFML Model",
var = translatePrototypeModel( goModel )
);
// ------------------------------------------------------------------------------- //
// ------------------------------------------------------------------------------- //
/**
* I translate the sparse, unpredictable, potentially missing Go model for
* AppearanceType into a predictable ColdFusion model.
*
* type AppearanceType struct {
* Type string `json:"type,omitempty"`
* IsRotated bool `json:"isRotated,omitempty"`
* Skin string `json:"skin,omitempty"`
* }
*
* @appearanceModel I am the Go model being translated.
*/
public struct function translateAppearanceModel( struct appearanceModel ) {
return({
type: ( appearanceModel.type ?: "" ),
isRotated: ( appearanceModel.isRotated ?: false ),
skin: ( appearanceModel.skin ?: "" )
});
}
/**
* I translate the potentially missing Go model for ColorType into a predictable
* ColdFusion model (a 6-digit HEX string).
*
* type ColorType struct {
* R int `json:"r"`
* G int `json:"g"`
* B int `json:"b"`
* }
*
* @colorModel I am the Go model being translated.
*/
public string function translateColorModel( struct colorModel ) {
if ( isNull( colorModel ) ) {
return( "" );
}
var r = formatBaseN( colorModel.r, 16 ).lcase();
var g = formatBaseN( colorModel.g, 16 ).lcase();
var b = formatBaseN( colorModel.b, 16 ).lcase();
return(
right( "0#r#", 2 ) &
right( "0#g#", 2 ) &
right( "0#b#", 2 )
);
}
/**
* I translate the sparse, unpredictable, potentially missing Go model for
* DeviceType into a predictable ColdFusion model.
*
* type DeviceType struct {
* Appearance *AppearanceType `json:"appearance,omitempty"`
* ID string `json:"id,omitempty"`
* Height int `json:"height,omitempty"`
* Width int `json:"width,omitempty"`
* Scaling string `json:"scaling,omitempty"`
* ScaleToFitWindow bool `json:"scaleToFitWindow,omitempty"`
* }
*
* @deviceModel I am the Go model being translated.
*/
public struct function translateDeviceModel( struct deviceModel ) {
return({
appearance: translateAppearanceModel( deviceModel.appearance ?: nullValue() ),
id: ( deviceModel.id ?: "" ),
height: ( deviceModel.height ?: 0 ),
width: ( deviceModel.width ?: 0 ),
scaling: ( deviceModel.scaling ?: "" ),
scaleToFitWindow: ( deviceModel.scaleToFitWindow ?: false )
});
}
/**
* I translate the sparse, unpredictable Go model for PrototypeType into a predictable
* ColdFusion model.
*
* type PrototypeType struct {
* ID string `json:"id,omitempty"`
* Name string `json:"name"`
* UserID int `json:"userId"`
* CompanyID int `json:"companyId,omitempty"
* Archived bool `json:"archived,omitempty"`
* Background *ColorType `json:"background,omitempty"`
* Device *DeviceType `json:"device,omitempty"`
* }
*
* @prototypeModel I am the Go model being translated.
*/
public struct function translatePrototypeModel( required struct prototypeModel ) {
return({
id: ( prototypeModel.id ?: "" ),
name: prototypeModel.name,
userID: prototypeModel.userId,
companyID: ( prototypeModel.companyId ?: 0 ),
isArchived: ( prototypeModel.archived ?: false ),
backgroundColor: translateColorModel( prototypeModel.background ?: nullValue() ),
device: translateDeviceModel( prototypeModel.device ?: nullValue() )
});
}
</cfscript>
As you can see, in order to manage the translation of data, I have to recurse down through the data structures, mapping each Go value onto the appropriate ColdFusion value. For values that are defined, they come through as is; but, if the value is missing, I have to default it to the "empty" value that Go removed. Like I said above, it's verbose. But, when we run this ColdFusion code we get the following output:
As I stated above, I don't really have any Go experience. As such, I can't speak to why a Go API would want to return an unpredictable JSON structure. For me, that just seems to make the response data harder to consume. It feels a little bit like a premature optimization of data-size over the network (at the cost of developer ergonomics). That said, at least I can easily - if not pedantically - marshal those return payloads into something I can safely consume in Lucee CFML 5.3.7.48.
Want to use code from this post? Check out the license.
Reader Comments
I love the elvis operator. It's so useful. Depending on the situation I'll use it instead of cfparam. As in:
form.name = form.name ?: "";
form.zip = form.zip ?:"";
@Hugh,
100% It's a great operator. Though, I also do use the
<cfparam>tag for a lot of the form-value defaulting. That's what I love about ColdFusion - so many powerful options!@All,
Over the weekend, I realized that I accidentally discovered that my mental model for the safe navigation operator was incomplete. I always thought it tested the left operand only. However, it appears to test both the left and right operands when short-circuiting:
www.bennadel.com/blog/4017-the-safe-navigation-operator-checks-both-left-and-right-operands-in-coldfusion.htm
What this means is that most of my use of the Elvis operator in this post can actually be replaced with safe navigation operators. So, for example, this:
translateAppearanceModel( deviceModel.appearance ?: nullValue() )... can actually become just this:
translateAppearanceModel( deviceModel?.appearance )... because this expression will return
nullif eitherdeviceModelordeviceModel.appearanceis undefined.