Generating The Content-MD5 Checksum For The Amazon S3 REST API Using ColdFusion
Recently, I've been playing around with the Amazon Simple Storage Service (S3) API. In my previous post, I was able to upload images to an S3 storage bucket without providing any content checksum. For certain methods, however, such as the multi-object delete, Amazon S3 requires that the Content-MD5 checksum be posted along with the resource request. Getting the Content-MD5 checksum took me a little bit of trial and error; so, I figured I would post it here for my own future reference (as well as for anyone else who might get tripped up).
In ColdFusion 9 and earlier, ColdFusion didn't really provide any features that were geared towards cryptographic hashing or binary hashing. With ColdFusion 10, however, ColdFusion added the new hmac() function for generating Message Authentication Codes; and, it augmented the functionality of the existing hash() function to accept binary values. Both of these functions can be used to greatly reduce the complexity of an Amazon S3 request.
However, since these features were only added in ColdFusion 10, I'm posting code that demonstrates both ColdFusion 9 and ColdFusion 10 approaches.
<cfscript>
// I generate the Amazon S3 request authentication signature using
// the given secret key and the given input.
string function generateSignature(
required string key,
required string input
) {
// Create the specification for our secret key.
var secretkeySpec = createObject( "java", "javax.crypto.spec.SecretKeySpec" ).init(
charsetDecode( key, "utf-8" ),
javaCast( "string", "HmacSHA1" )
);
// Get an instance of our MAC generator.
var mac = createObject( "java", "javax.crypto.Mac" ).getInstance(
javaCast( "string", "HmacSHA1" )
);
// Initialize the Mac with our secret key spec.
mac.init( secretkeySpec );
// Hash the input (as a byte array).
var hashedBytes = mac.doFinal(
charsetDecode( input, "utf-8" )
);
// Return the hashed bytes as Base64.
return(
binaryEncode( hashedBytes, "base64" )
);
}
// I generate the Amazon S3 request authentication signature using
// the new hmac() function found in ColdFusion 10.
string function generateSignatureWithColdFusion10(
required string key,
required string input
) {
// Unfortunateyl, it looks like the hmac() function can only
// return values in HEX. As such, we'll have to convert it
// from hex to binary, and then encode the binary as Base64.
var hexEncoding = hmac( input, key, "HmacSHA1", "utf-8" );
return(
binaryEncode(
binaryDecode( hexEncoding, "hex" ),
"base64"
)
);
}
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// I generate the base64-encoded 128-bit MD5 digest of the
// message content that is being posted to Amazon S3.
string function generateContentMD5( required any content ) {
// Get our instance of the digest algorithm.
var messageDigest = createObject( "java", "java.security.MessageDigest" )
.getInstance( javaCast( "string", "MD5" ) )
;
// Create the MD5 hash (as a byte array).
var digest = messageDigest.digest( content );
// Return the hashed bytes as Base64.
return(
binaryEncode( digest, "base64" )
);
}
// I generate the base64-encoded 128-bit MD5 digest of the
// message content using the new hash() functionality present
// in ColdFusion 10.
string function generateContentMD5WithColdFusion10( required any content ) {
// Unfortunateyl, it looks like the hash() function can only
// return values in HEX. As such, we'll have to convert it
// from hex to binary, and then encode the binary as Base64.
var hexEncoding = hash( content, "MD5", "utf-8" );
return(
binaryEncode(
binaryDecode( hexEncoding, "hex" ),
"base64"
)
);
}
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// Includ our Amazon S3 crendentials (for testing).
include "../credentials.cfm";
// Read in the file that we are going to post up to Amazon S3.
// This will be the BODY of our PUT and the basis for our
// Content-MD5 generation.
content = fileReadBinary( expandPath( "./helena.jpg" ) );
// The target resource for our upload to Amazon S3.
resource = "/testing.bennadel.com/signed-urls/helena.jpg";
// Get the current HTTP date - requied for all Amazon S3 requests.
currentTime = getHttpTimeString( now() );
// Generate the Base64-encoded Content-MD5 value.
contentMD5 = generateContentMD5WithColdFusion10( content );
// The content-type will be stored as meta-data with the S3
// resource and then will be used to serve up the resource later.
contentType = "image/jpeg";
// Set up the part of the string to sign.
stringToSignParts = [
"PUT",
contentMD5,
contentType,
currentTime,
resource
];
// Generate the Base64-encoded signature.
signature = generateSignatureWithColdFusion10(
aws.secretKey,
arrayToList( stringToSignParts, chr( 10 ) )
);
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// Set up our Amazon S3 RESTful API request.
put = new Http(
method = "put",
url = "https://s3.amazonaws.com#resource#"
);
put.addParam(
type = "header",
name = "Authorization",
value ="AWS #aws.accessID#:#signature#"
);
put.addParam(
type ="header",
name ="Content-Length",
value = arrayLen( content )
);
put.addParam(
type = "header",
name = "Content-MD5",
value = contentMD5
);
put.addParam(
type = "header",
name = "Content-Type",
value = contentType
);
put.addParam(
type = "header",
name = "Date",
value = currentTime
);
put.addParam(
type = "body",
value = content
);
// Make the acutal request and get the result.
result = put.send().getPrefix();
// Output the status code.
writeOutput( "Status: " & result.statusCode );
</cfscript>
Only the first half of the code is really relevant - it defines the ColdFusion user defined functions (UDFs) for generating the request signature and the Content-MD5 checksum. The second half of the post is simply consuming those functions for a single RESTful API request.
Data type and encoding conversions are not necessarily second-nature for me (yet!). So, the above code did take a little trial and error. I hope this helps someone.
Want to use code from this post? Check out the license.
Reader Comments
Hi,
Just been doing some work with Amazon Market Web Services and I found this works:
<cfreturn binaryEncode((BinaryDecode(Hash(charsetDecode( arguments.content, "utf-8" )),'hex')),"base64")/>
Note the charsetDecode. Amazon MD5 the bytearray of the string.
Regards,
Adam
Thanks for this! Specifically for lines 45 and following. I was staring at the hmac() function, trying to figure out why it didn't return what I expected. I followed one of my standard debugging procedures, which is to read all of your blog posts that touch on whatever my current issue is, and there the answer was (again).
Really nice article.... many things to know about