XML Posts Without A UTF-8 Encoding May Lead To 400 Bad Request Errors

Last week, Ryan Jeffords and I spent a solid hour debugging some API interactions between ColdFusion and Recurly. The API requests required the posting of XML content using CFHTTP and CFHTTPParam. The API calls worked 99% of the time; however, every now and then, one would come back with a "400 Bad Request" status code response. After banging our heads together, we finally figured out that the cause of the 400 Bad Request was the high ASCII values embedded in some people's names.

Outside of the Byte-Order-Mark (BOM) and leading white space, I've never had an issue parsing XML in ColdFusion. As such, it never occurred to me that character encoding would be cause for concern. And, to be frank, character encoding has always been a bit of a blind spot for me. The Recurly API, on the other hand, seems to require explicit encoding when XML content resides outside the standard 256 ASCII characters.

At first, we were passing in XML bodies like this:

<?xml version="1.0"?>
<data>
	<message>
		Here is a high ascii characters: [#chr( 9786 )#].
	</message>
</data>

This XML body contains a high ASCII value but no character encoding. This would parse absolutely fine in ColdFusion (cause ColdFusion rocks!); but, this was giving us a 400 Bad Request response when posted to Recurly. Once we added the character encoding, the API requests went through perfectly:

<?xml version="1.0" encoding="utf-8"?>
<data>
	<message>
		Here is a high ascii characters: [#chr( 9786 )#].
	</message>
</data>

Notice that we added:

encoding="utf-8"

... to the meta-data at the beginning of the XML document.

Slowly, I am beginning to wrap my head around character encoding. And, mostly, that just means explicitly specifying UTF-8 whenever possible, whether that's in an XML body, a CFContent tag, a HTML5 Meta tag, or a database column definition. In this case, we needed to include a UTF-8 encoding with our XML API requests in order to prevent XML parsing errors (400 Bad Request responses).

Want to use code from this post? Check out the license.

Short link: https://bennadel.com/2290