Consuming Plupload Data URIs In An AngularJS Application
I've been using Plupload as my HTML5 uploader in AngularJS for a long time. I've even looked at the mechanics of extracting a Base64-encoded image preview from a Plupload file; but, I've never really noodled on how to best consume those data URIs in the context of a real AngularJS application. This blog post is my first attempt at turning data URIs into a cacheable resource that I can use throughout my application as I navigate to different views.
View this project on my GitHub account.
When thinking about integrating base64-encoded data URIs, I had three big questions:
- How do I cache them?
- How do I consume them?
- How and when do I load the actual image object (ie, the remote one)?
As a base for this exploration, I start with my global-uploader project and then refactored it. First, I turned the globalUploader service into a feature that communicated via promises rather than $broadcast() events. This way, the calling context would have an easier time hooking into the lifecycle of a given file upload.
Then, I had to figure out who was responsible for extracting the data URIs. Ultimately, I ended up putting that logic in the globalUploader service. This felt like a decent choice because it could be easily integrated within a single promise workflow:
- Pass file to globalUploader -> returns Promise.
- Notify (Optional) -> base64-encoded data URI.
- Resolve -> File has been uploaded.
- Reject -> File upload has failed.
In this case, I'm using the optional "notify" event to deliver the base64-encoded data URI to the calling context. Of course, this somewhat precludes (though not entirely) the ability to use the notify event to report uploaded progress, as in "46% uploaded." With that in mind, I'd be happy to move the data URI extraction to a separate service altogether.
Once I had the data URI, I had to figure out where to store it so that it could be consumed across different Views within the AngularJS application. I ended up creating a dataUriCache service. This service maintains an internal hash of image URLs mapped onto data URIs:
cache[ imageURL ] : data URI
I chose to cache the data URIs based on the remote URLs that they represent because it setup the dataUriCache as place where I could easily preload the remote images. The dataUriCache service exposes two methods for accessing the underlying data URIs:
- .get( imageUrl )
- .replace( imageUrl )
Both of these methods return the underlying data URI; however, the .replace() method will also try to load the remote Image in the background. And, once the remote image is loaded (and resides within the browser's local cache), the dataUriCache service will automatically evict the base64-encoded data URI from the cache, thereby freeing up the memory.
app.factory(
"dataUriCache",
function( $timeout ) {
// I store the cached data-uri values. Each value is intended to be cached at a
// key that represents the remote URL for the data item.
var cache = {};
// I define the time (in milliseconds) that a cached data-uri before it is
// automatically flushed from the memory.
var cacheDuration = ( 120 * 1000 );
// I store the eviction timers for the cached data-uri.
var timers = {};
// Return the public API.
return({
set: set,
get: get,
has: has,
remove: remove,
replace: replace
});
// ---
// PUBLIC METHODS.
// ---
// I cache the given data-uri under the given key (which is intended to be a URL
// that represents the remote version of the data).
// --
// CAUTION: The data is not kept around indefinitely; once cached, it will be
// flushed from the cache within a relatively short time period. This way, the
// browser doesn't get bloated with data that is not going to be accessed.
function set( key, dataUri ) {
// Normalize the key so we don't accidentally conflict with built-in object
// prototype methods and properties.
cache[ key = normalizeKey( key ) ] = dataUri;
$timeout.cancel( timers[ key ] );
timers[ key ] = $timeout(
function clearCache() {
console.warn( "Expiring data-uri for %s", key );
delete( cache[ key ] );
// Clear the closed-over variables.
key = dataUri = null;
},
cacheDuration,
// Don't trigger digest - the application doesn't need to know about
// this change to the data-model.
false
);
return( dataUri );
}
// I get the data-uri cached at the given key.
// --
// NOTE: Returns NULL if not defined.
function get( key ) {
return( cache[ normalizeKey( key ) ] || null );
}
// I determine if a data-uri is cached at the given key.
function has( key ) {
return( normalizeKey( key ) in cache );
}
// I remove the data-uri cached at the given key.
function remove( key ) {
console.warn( "Evicting data-uri for %s", key );
$timeout.cancel( timers[ key = normalizeKey( key ) ] );
delete( cache[ key ] );
}
// I return the data-uri cached at the given key. But, the remote object,
// represented by the cache-key (which is intended to be a URL) is loaded in the
// background. When (and if) the remote image is loaded, the cached data-uri is
// evicted from the cache.
function replace( key ) {
loadRemoteObject( key );
return( get( key ) );
}
// ---
// PRIVATE METHODS.
// ---
// I load the remote object represented by the given key (which is intended to be
// a URL). This will cache the object in the local browser cache, at which point
// the data-uri is evicted from the cache.
function loadRemoteObject( key ) {
angular.element( new Image() )
.on(
"load",
function handleLoadEvent() {
console.info( "Remote object loaded at %s", key );
// Now that the image has loaded, and is cached in the browser's
// local memory, we can evict the data-uri.
remove( key );
// Clear the closed-cover variables.
key = null;
}
)
.on(
"error",
function handleErrorEvent() {
// Clear the closed-cover variables.
key = null;
}
)
.prop( "src", key )
;
}
// I normalize the given key for use as cache or timer key.
function normalizeKey( key ) {
return( "url:" + key );
}
}
);
As you can see, in addition to the .replace() method, the dataUriCache service will also auto-evict data URIs after a period of inactivity. This way, they don't stick around indefinitely, taking up a massive amount of browser memory.
I didn't want to make the globalUploader service responsible for caching the data URIs; after all, the global uploader could be used for more than just uploading Images. As such, I made the calling context responsible for taking the data URI (from the Promise-based notify event) and caching it in the dataUriCache service. In this case, that's the HomeController:
app.controller(
"HomeController",
function( $scope, $rootScope, $q, imageService, globalUploader, dataUriCache, _ ) {
// I hold the list of images to render.
$scope.images = [];
// I am the ID of the currently-selected image.
$scope.selectedImageID = null;
// Pull the list of images from the remote repository.
loadRemoteData();
// ---
// PUBLIC METHODS.
// ---
// I close the current image detail.
$scope.closeImage = function() {
$scope.selectedImageID = null;
};
// I delete the given image.
$scope.deleteImage = function( image ) {
$scope.images = _.without( $scope.images, image );
// NOTE: Assuming no errors for this demo - not waiting for response.
imageService.deleteImage( image.id );
};
// I process the given files. These are expected to be mOxie file objects. I
// return a promise that will be done when all the files have been processed.
$scope.saveFiles = function( files ) {
var promises = _.map( files, saveFile );
return( $q.all( promises ) );
};
// I open the detail view for the given image.
$scope.openImage = function( image ) {
$scope.selectedImageID = image.id;
};
// ---
// PRIVATE METHODS.
// ---
// I apply the remote data to the local view-model.
function applyRemoteData( images ) {
$scope.images = augmentImages( images );
}
// I augment the image for use in the local view-model.
function augmentImage( image ) {
return( image );
}
// I aument the images for use in the local view-model.
function augmentImages( images ) {
return( _.each( images, augmentImage ) );
}
// I load the images from the remote resource.
function loadRemoteData() {
imageService.getImages()
.then(
function handleGetImagesResolve( response ) {
applyRemoteData( response );
},
function handleGetImagesReject( error ) {
console.warn( "Error loading remote data." );
}
)
;
}
// I save a file-record with the same name as the given file, then pass the file
// on to the application to be uploaded asynchronously.
function saveFile( file ) {
var image = null;
// We need to separate our promise chain a bit - the local uploader only cares
// about the image RECORDS that need to be "saved." The local uploader doesn't
// actually care about the global uploader, as this doesn't pertain to it's
// view-model / rendered state.
var savePromise = imageService.saveImage( file.name )
.then(
function handleSaveResolve( response ) {
$scope.images.push( image = augmentImage( response.image ) );
// NOTE: Pass response through chain so next promise can get at it.
return( response );
},
function handleSaveReject( error ) {
alert( "For some reason we couldn't save the file, " + file.name );
// Pass-through the error (will ALSO be handled by the next
// error handler in the upload chain).
return( $q.reject( error ) );
}
)
;
// Now that we have our "save promise", we can't hook into the global uploader
// workflow - sending the file to the uploader and then waiting for it to be
// done uploading. The global uploader doesn't know files from Adam; as such,
// we have to tell what the app to do when the global uploader has finished
// uploading a file.
savePromise
.then(
function handleSaveResolve( response ) {
return(
globalUploader.uploadFile(
file,
response.uploadSettings.url,
response.uploadSettings.data,
// Have the uploader extract the data-uri for the image. This
// will be made avialable in the .notify() handler. If this
// is omitted, only the resolve/reject handlers will be called.
true
)
);
}
)
.then(
function handleUploadResolve() {
// Once the file has been uploaded, we know that the remote binary
// can be reached at the known imageUrl; but, we need to let the
// server know that.
// --
// NOTE: This could probably be replaced with some sort of t S3 /
// Simple Queue Service (SQS) integration.
imageService.finalizeImage( image.id );
image.isFileAvailable = true;
},
function handleUploadReject( error ) {
// CAUTION: The way this promise chain is configured, this error
// handler will also be invoked for "Save" errors as well.
alert( "For some reason we couldn't upload one of your files." );
},
function handleUploadNotify( dataUri ) {
// The notify event means that the uploader has extracted the image
// binary as a base64-encoded data-uri. We can use that in lieu of
// a remote image while the file is still being uploaded. By sticking
// this in the dataUriCache() service, we can also use it in the
// detail view, if the file still has yet to be uploaded.
image.imageUrl = dataUriCache.set( image.imageUrl, dataUri );
// Since we're using the data-uri instead of the imageUrl, we can
// think of the image as being "available" for our intents and purposes.
image.isFileAvailable = true;
}
)
.finally(
function handleFinally() {
// Clear closed-over variables.
file = image = promise = null;
}
)
;
// Return the promise for the initial save - does not include the physical file
// upload to Amazon S3.
return( savePromise );
}
}
);
If you look at the .saveFile() method, you can see where I interact with the globalUploader service. I pass-in the file and get a promise back. Then, in the "notify" event, I take the base64-encoded data URI and cache it based on the current image's remote URL. This ends up being a perfect place to do so because I can also copy the data URI into the current image and flag the image's file as being "available" for rendering. This will allow the home View to render the thumbnails almost instantly (depending on the size of the image).
Once the data URI is cached in the dataUriCache service, I can start to consume it across multiple views. In my demo, I can click on a thumbnail and get a detail-view overlay. This detail view makes a new AJAX request to the server for the given image's data and then renders the full-size image.
The DetailController also injects the dataUriCache; and, when the image record is loaded, it checks the dataUriCache to see if it contains a relevant data URI. If it does, it uses that to immediately render the image. And, if it doesn't, it uses the remote URL:
app.controller(
"DetailController",
function( $scope, $exceptionHandler, imageService, dataUriCache ) {
// I am the image object that is being viewed.
$scope.image = null;
// Load the image data from the server.
loadRemoteData();
// ---
// PRIVATE METHODS.
// ---
// I apply the remote data to the local view-model.
function applyRemoteData( image ) {
$scope.image = augmentImage( image );
// Just because we have the image, it doesn't mean that remote image binary
// is actually available. But, we may have a data-uri version of it cached
// locally. In that case, let's consume the data-uri as the imageUrl, which
// will allow us to render the image immediately.
if ( dataUriCache.has( image.imageUrl ) ) {
// The data-uri cache has two modes of access: either you get the data-
// uri; or, you get the data-uri and the remote object is loaded in the
// background. We only want to use "replace" if we know that the remote
// image is available; otherwise, we run the risk of caching the wrong
// binary in the browser's cache.
if ( image.isFileAvailable ) {
// Get the data-uri and try to load the remote object in parallel.
// --
// NOTE: Once the remote object is loaded, the data-uri cache is
// automatically flushed.
image.imageUrl = dataUriCache.replace( image.imageUrl );
} else {
// Just get the data-uri - don't try to load the remote binary; this
// will allow the data-uri to stay in-memory for a bit longer.
image.imageUrl = dataUriCache.get( image.imageUrl );
// Since we're using the data-uri in lieu of the remote image, we can
// flag the file as available.
image.isFileAvailable = true;
}
}
}
// I add the additional view-specific data to the image object.
function augmentImage( image ) {
var createdAt = new Date( image.createdAt );
// Add a user-friendly data label for the created timestamp.
image.dateLabel = (
createdAt.toDateString().replace( /^(Sun|Mon|Tue|Wen|Thr|Fri|Sat)/i, "$1," ) +
" at " +
createdAt.toTimeString().replace( / GMT.+/i, "" )
);
return( image );
}
// I load the selected image from the remote resource.
function loadRemoteData() {
// CAUTION: Inherited property - selectedImageID.
imageService.getImage( $scope.selectedImageID )
.then(
function handleGetImagesResolve( response ) {
applyRemoteData( response );
},
function handleGetImagesReject( error ) {
console.warn( "Image data could not be loaded." );
// CAUTION: Inherited method.
$scope.closeImage();
}
)
;
}
}
);
Here, we can see the two different approaches to data URI access. If the image's remote file is available, it doesn't necessarily mean that we want to delay the rendering of the image. If the data URI is in memory, we want to use that, regardless, to render the image instantly. But, if the remote file is available, we'd also like to get the dataUriCache service to load the image in the background and then evict the data URI. That's why the DetailController uses .replace() if the remote image is available and .get() if it is not.
I kept thinking about ways to try and encapsulate this logic in some sort of image SRC directive or image component. And, if the only logical step was to map URLs onto data URIs, that would have been hella easy. The blocker for that approach is the "isFileAvailable" flag on the image; we don't simply map the data URI onto the remote image URL and call it a day - we also flag the file as being "available" when we do so. This allows the View to conditionally include the IMG tag in the page. If I tried to move all of that logic into a "generic" SRC directive, I would have been overly complex and too tightly coupled to the various images within the application.
Right now, I think I have a pretty good separation of responsibilities in this demo. The globalUploader takes care of saving the file; the dataUriCache takes care of caching data URIs and remote images (in the browser's cache); and, the Controllers take care of marshaling the data. It's not perfect, but I think it's a pretty good start to consuming data URIs in an AngularJS application.
Want to use code from this post? Check out the license.
Reader Comments
@All,
While not entirely related, I refactored this demo to use a patched version of Plupload and pre-signed URLs instead of a form upload policy:
www.bennadel.com/blog/2790-using-plupload-to-upload-files-directly-to-amazon-s3-using-put-and-pre-signed-query-string-authenticated-urls.htm
From a workflow point of view, the change is fairly transparent - you still need to pass around an "upload settings" object; but, pre-signed URLs are a bit easier to produce, that's for sure!
I apologize in advance for my "noviceness" when it comes to jquery and passing variables around, but I need some assistance way back to your original Plupload example, where you are provided a basic example of how to drag files to the server using Plupload.
The comments section has been closed for some time on the original post, so maybe I can get a much needed push from this one.
I am trying to do what Jay Hersh was trying to accomplish (mainly pass a dynamic ID down through the layers to where I can insert it into a table in the upload_files.cfm along with the file name, etc.). However, I can't seem to figure out how to do that (so retarded, I know).
If there is any help out there possible, I would be so greatly in debt. I've tried quite a few ways to pass down the variable in an "id" value, but it hasn't worked yet. My goal of course is to call the drag module on a specific record's URL or page and then record the file name, record's ID and date, etc. to a database upon upload. Pretty simple in theory--out of reach of this simpleton.
Ben, if there is ever a soul that needed CF Superman's help it would be me and certainly be now...