Uploading Files To Amazon S3 Using A Form Post And ColdFusion
Recently, I've been doing a lot of experimenting with ColdFusion and Amazon's Simple Storage Service (S3). Up until now, this has primarily involved using ColdFusion as a proxy to upload files and provide pre-signed (query-string authenticated) URLs. As I've been doing this exploration, a few people have mentioned that Amazon S3 allows web-users to upload files directly to an S3 bucket using a regular HTTP Form POST. This sounds intriguing, so I figured I would give it a try.
When uploading to Amazon S3 using a Form POST, you still need ColdFusion (or some other server-side language) in order to generate the authorization signature for the request. The point of the Form POST is not to remove the need for a server-side infrastructure; rather, it is intended only to remove the need to use your server a proxy for the upload.
If the upload to S3 is successful, Amazon will redirect the request to the "success url" you define in the form data. If the upload fails, however, there is no redirect. Instead, Amazon returns its typical XML-encoded error response.
When posting to Amazon S3 using a form POST, you can only upload one file at a time. And, you need to provide a policy for the upload. This policy describes the request data and all the constraints that Amazon needs to apply to the various form fields. I don't have a super in-depth understanding of how this policy works - my demo was cobbled together using the example provided on the Amazon Web Services blog.
That said, the following script prepares an upload policy and then presents the user with an HTTP Form for the upload. You may notice that both the policy and the form data seem to indicate that an image will be uploaded. It is important to understand that this defines the meta data that will be stored with the resource on Amazon S3 and in no way limits the type of file that can be uploaded. If a user selects a non-image file, it will be uploaded all the same; it will simply be stored with an illogical content-type value.
<cfscript>
// Include our Amazon S3 credentials.
include "../credentials.cfm";
// Set up the Success url that Amazon S3 will redirect to if the
// FORM POST has been submitted successfully.
// NOTE: If the form post fails, Amazon will present an error
// message - there is not error-based redirect.
successUrl = (
"http://" & cgi.server_name &
getDirectoryFromPath( cgi.script_name ) & "success.cfm"
);
// The expiration must defined in UCT time.
expiration = dateConvert( "local2utc", dateAdd( "h", 1, now() ) );
// NOTE: When formatting the UTC time, the hours must be in 24-
// hour time; therefore, make sure to use "HH", not "hh" so that
// your policy don't expire prematurely.
policy = {
"expiration" = (
dateFormat( expiration, "yyyy-mm-dd" ) & "T" &
timeFormat( expiration, "HH:mm:ss" ) & "Z"
),
"conditions" = [
{
"bucket" = aws.bucket
},
{
"acl" = "private"
},
{
"success_action_redirect" = successUrl
},
[ "starts-with", "$key", "/form-post/" ],
[ "starts-with", "$Content-Type", "image/" ],
[ "content-length-range", 0, 10485760 ] // 10mb
]
};
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// The policy will be posted along with the FORM post as a
// hidden form field. Serialize it as JavaScript Object notation.
serializedPolicy = serializeJson( policy );
// Remove up the line breaks.
serializedPolicy = reReplace( serializedPolicy, "[\r\n]+", "", "all" );
// Encode the policy as Base64 so that it doesn't mess up
// the form post data at all.
encodedPolicy = binaryEncode(
charsetDecode( serializedPolicy, "utf-8" ) ,
"base64"
);
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// To make sure that no one tampers with the FORM POST, create
// hashed message authentication code of the policy content.
// NOTE: The hmac() function was added in ColdFusion 10.
hashedPolicy = hmac(
encodedPolicy,
aws.secretKey,
"HmacSHA1",
"utf-8"
);
// Encode the message authentication code in Base64.
encodedSignature = binaryEncode(
binaryDecode( hashedPolicy, "hex" ),
"base64"
);
</cfscript>
<!--- ----------------------------------------------------- --->
<!--- ----------------------------------------------------- --->
<!--- Reset the output buffer and set the page encoding. --->
<cfcontent type="text/html; charset=utf-8" />
<cfoutput>
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<title>
Uploading Files To Amazon S3 Using A Form Post And ColdFusion
</title>
</head>
<body>
<h1>
Uploading Files To Amazon S3 Using A Form Post And ColdFusion
</h1>
<form
method="post"
action="https://#aws.bucket#.s3.amazonaws.com/"
enctype="multipart/form-data">
<input type="hidden" name="AWSAccessKeyId" value="#aws.accessID#" />
<input type="hidden" name="key" value="/form-post/${filename}" />
<input type="hidden" name="Content-Type" value="image/*" />
<input type="hidden" name="acl" value="private" />
<input type="hidden" name="success_action_redirect" value="#htmlEditFormat( successUrl )#" />
<!--- Base64-encoded policy and request signature. --->
<input type="hidden" name="policy" value="#encodedPolicy#" />
<input type="hidden" name="signature" value="#encodedSignature#" />
<p>
Select your file:
</p>
<!---
NOTE: The file upload must be the LAST field in the
form post and must be called "file".
--->
<p>
<input name="file" type="file" size="40" />
</p>
<p>
<input type="submit" value="Upload Image" />
</p>
</form>
</body>
</html>
</cfoutput>
Once the file is processed, Amazon will redirect the browser to the URL defined in the "success_action_redirect" form field. When doing this, Amazon will add three query string parameters for:
- bucket
- key
- etag
The "key" value is the resource key at which the uploaded document was stored. This value may or may not start with a leading "/" depending on how your form fields were configured. The HTTP Form POST upload will work with or without the leading "/" on your resource key; however, if you don't provide in the form, you'll have to add it to the key in the query string parameter.
To test that the upload was successful, my "success_action_redirect" page created a pre-signed (query string authenticated) URL and rendered the uploaded image using an HTML IMG tag:
<cfscript>
// When Amazon S3 redirects the FORM POST to the success URL,
// it will pass the following parameters in the URL.
// NOTE: The etag value is quoted (ie, "abc123").
param name="url.bucket" type="string";
param name="url.key" type="string";
param name="url.etag" type="string";
// ------------------------------------------------------ //
// ------------------------------------------------------ //
// Include our Amazon S3 credentials.
include "../credentials.cfm";
// Now that we have the resource and the bucket to which it was
// posted, we can construct a full URL and generate a pre-signed,
// authorized URL.
resource = ( "/" & url.bucket & url.key );
// The expiration is defined as the number of seconds since
// epoch - as such, we need to figure out what our local timezone
// epoch is.
localEpoch = dateConvert( "utc2local", "1970/01/01" );
// The resource will expire in +1 day.
expiration = dateDiff( "s", localEpoch, ( now() + 1 ) );
// Build up the content of the signature (excluding Content-MD5
// and the mime-type).
stringToSignParts = [
"GET",
"",
"",
expiration,
resource
];
stringToSign = arrayToList( stringToSignParts, chr( 10 ) );
// Generate the signature as a Base64-encoded string.
// NOTE: Hmac() function was added in ColdFusion 10.
signature = binaryEncode(
binaryDecode(
hmac( stringToSign, aws.secretKey, "HmacSHA1", "utf-8" ),
"hex"
),
"base64"
);
// Prepare the signature for use in a URL (to make sure none of
// the characters get transported improperly).
urlEncodedSignature = urlEncodedFormat( signature );
</cfscript>
<!--- Reset the output buffer and set the page encoding. --->
<cfcontent type="text/html; charset=utf-8" />
<cfoutput>
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<title>
Your Upload To Amazon S3 Was Successful!
</title>
</head>
<body>
<h1>
Your Upload To Amazon S3 Was Successful!
</h1>
<p>
<img src="https://s3.amazonaws.com#resource#?AWSAccessKeyId=#aws.accessID#&Expires=#expiration#&Signature=#urlEncodedSignature#" />
</p>
</body>
</html>
</cfoutput>
This is a pretty cool feature, but I am not sure how often I would use it. Once you upload a file to your server, transferring it to Amazon S3 should be negligible (considering that most hosting providers have massive internet backbones). I guess it will depend on how much (if any) post-upload processing you will need to do in your specific workflow. Definitely this is something that warrants further exploration.
Want to use code from this post? Check out the license.
Reader Comments
Plus you are putting a lot of trust, and your AWS account ID, in the hands of a client, and trusting it to upload to the right place with the right policy.
I'm not sure that is a good plan; if I edit the HTML page can I can do a public upload to the bucket and use your account as an anonymous image hosting service, for instance...
@Tom,
Luckily, it's not quite that easy. Since the Policy (that you submit via a hidden form field) is signed using your AWS "secret key", you can't mess with the form fields without violating the policy (or, really, the Signature which is based on the policy).
It seems a number of AWS interactions actually force you to put your Access ID out in public; for example, the generating of pre-signed (query-string authenticated) URLs makes you put your Access ID right in the URL that you are generating.
I don't know much about cryptography; but, I think the Access ID is kind of like your "public key", which can be shared without problem.... so long as you make sure no one has your "secret key."
@Ben,
Ahh, I missed the signing step. That's not so bad then.
Any suggestions how to make this work with CF9.1?
I guess this part is causing errors
// NOTE: The hmac() function was added in ColdFusion 10.
hashedPolicy = hmac(
encodedPolicy,
aws.secretKey,
"HmacSHA1",
"utf-8"
);
Thanks for your awesome blog. Very helpful! Used it many times :)
@Jan,
Try taking a look at my Crypto.cfc library:
https://github.com/bennadel/Crypto.cfc
It has a number of hashed message authentication code methods that will deal with various hashes.