Using Plupload To Upload Files Directly To Amazon S3 Using PUT And Pre-Signed (Query String Authenticated) URLs
Out of the box, Plupload hard-codes the "POST" method when it uploads a file to the remote server. This is fine if you want to upload files to Amazon S3 using a form upload policy; but, it falls short if you want to upload files to Amazon S3 using a pre-signed URL. Pre-signed (aka, query string authenticated) URLs require the PUT method. After talking to Jonathan Rowny about this the other day, we decided to see if Plupload could be patched to use the PUT method. And, as Jonathan discovered, it turns out it can. As such, I wanted to take my recent Plupload Data-URI demo and refactor it to use pre-signed URLs instead of a form upload policy.
View this project on my GitHub account.
When switching from a POST to a PUT, we need to have control over two hard-coded aspects of the Plupload library:
Method: This is always hard-coded to be POST, but we need it to be PUT for use with pre-signed URLs.
Content-Type: When multipart mode is turned off, the Content-Type header is hard-coded to be "application/octet-stream." But, for use with pre-signed URLs, we need the content-type to match the content-type that was used to generate the pre-signed URL signature.
Making these changes to the Plupload library is actually quite easy; it's like 8 lines of code. First, you need to set defaults for the "method" and the "content_type" in the base settings hash. Then, you need to replace the hard-coded values with references to the uploader's settings. This way, the user can provide settings that can redefine the HTTP method and the Content-Type header.
To demonstrate how small this change is, here's a git-diff of the original file and my patched file.
CAUTION: Please note that I am not aware of the full repercussions of changing this code. Meaning, these changes may cause bugs or other issues in edge-cases and error handling. That said, I have tested this in Firefox, Chrome, and Safari and it seems to work quite nicely.
Once these changes were in place, I had to alter my server-side code and my client-side to produce and then consume these new settings. On the server, I'm creating a "settings" object that gets passed back to the client. The client then uses this settings object to update the Plupload uploader configuration on a per-file basis.
Ultimately, switching from a Form Upload Policy to a Pre-Signed URL was a fairly transparent change. Obviously, the server needs to know what values to produce and the client needs to know how to apply them; but, this only creates tight-coupling between two files: the storage component (that generates the upload settings) and the uploader directive (that applies the upload settings). Every other piece of code simply passes around this "settings" object, blissfully unaware of the implementation details.
Switching from a Form Upload Policy to a Pre-Signed URL doesn't actually reduce any complexity; you still have to pass around a settings object and you still need to alter the uploader configuration on a per-file basis. The benefit of patching Plupload, to allow for the PUT method, is simply that you can now consume Plupload in greater variety of ways. Also, in fairness, generating a pre-signed URL is a bit simpler than generating a form upload policy. But again, that difference can be quite well encapsulated.
Reader Comments
I have patched plupload with your patch in order to put videos on vimeo using the new API. To get working, it's important set the multipart setting of plupload to false. Hopping it will help some poeple.
@Frshog,
That's very cool that you were able to get that working. And, just to be clear, having to set the multipart to false is a requirement of Amazon S3, not of Plupload (I don't think). It's been a little while since I've looked at this, but multipart is the encoding type of request. With it, multiple values get sent in the same request; without it (ie, set to false), the request is sent as a single binary object, which is S3 is expecting.
... I think :)