Using Pusher WebSockets To Show Progress During Page Unload / Redirect In Lucee CFML 5.3.7.47
At InVision, we use - and freaking love - Pusher to manage our WebSocket-based communication between the ColdFusion servers and the browser. And while WebSocket communication is awesome, I've always treated it as a "nice to have" feature - one that shouldn't make or break the user experience (UX). In that vein, I wanted to see if I could use Pusher WebSockets to show a progress indicator during a page redirect. This would be especially nice during report generation where the user may have to sit there for a while as the "numbers get crunched." To explore this idea, I'm going to use the sleep() function to simulate some latency in Lucee CFML 5.3.7.47.
As I discussed in my previous post about tracer Cookies and download prompts in ColdFusion, I'll often have an "action page" that performs an operation and then returns some sort of result to the user. And, while that "action page" is processing, the user just sits there waiting. The post on tracer cookies was an exploration of how to make that "just sits there waiting" experience a bit more enjoyable.
This exploration, of using Pusher WebSockets during the page unload / page redirect operation is very much the same. Meaning, the user has to sit there waiting for something, such a report to be generated, and it would be great to be able to use Pusher to emit events that inform the user as to how the report generation is proceeding.
CAUTION: In order for this to work, the browser needs to be able to update the DOM (Document Object Model) after the page redirect has been initiated. Unfortunately, it looks like Safari (at least my version) does not want to do this. <add snarky comment about Safari being the "new IE" here>
To explore this, I created a mock report generation page. It accepts a Pusher channel-name and iterates over a fake set of "steps". During each step, it will emit an event using Pusher (and CFThread) before it finally renders the "results" page to the user:
<cfscript>
// This is the Pusher WebSocket channel over which we will emit progress events.
param name="url.progressChannel" type="string";
mockSteps = [
{ offset: 0, name: "Initializing..." },
{ offset: 10, name: "Gathering data..." },
{ offset: 15, name: "Crunching numbers..." },
{ offset: 33, name: "Applying business intelligence..." },
{ offset: 42, name: "Adding secret sauce..." },
{ offset: 55, name: "Generating folder structures..." },
{ offset: 60, name: "Downloading files..." },
{ offset: 75, name: "Compiling archive..." },
{ offset: 90, name: "Compressing report..." },
{ offset: 98, name: "Cleaning up..." }
];
for ( mockStep in mockSteps ) {
// Since the Pusher Event is sent over an HTTP request, it will be a BLOCKING
// call. Which means, there's a chance it will hang. Which means, there's a
// chance it will hurt th performance of the overall page. As such, we want to
// emit the Pusher Event asynchronously.
thread
name = "generate.#mockStep.offset#"
channel = url.progressChannel
message = mockStep
{
application.pusher.pushToAllSubscribers(
channel = channel,
eventType = "progress",
message = message
);
}
// Simulate some random processing overhead for the given step.
sleep( randRange( 100, 1000 ) );
}
</cfscript>
<!doctype>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>
Your Report Has Been Generated
</title>
<link rel="stylesheet" type="text/css" href="./styles.css"></link>
</head>
<body>
<h1>
Your Report Has Been Generated
</h1>
<p>
<a href="./index.cfm">Generate another report</a>.
</p>
</body>
</html>
As you can see, during each step of the mock report generation we're emitting an event that contains a numeric offset value between 0 and 100. This offset value is going to be used to render a progress bar on the previous page - the one that is redirecting the user to this report generation page.
Note that we are emitting the Pusher event inside a CFThread tag. This is because the communication between the ColdFusion server and the Pusher WebSocket server takes place over an HTTP request. Which means, it has the opportunity to hang and block the report generation. But, since Pusher is a "nice to have" feature, we definitely don't want it to interfere with the report generation itself. By moving the event emission to an asynchronous thread, we allow the report generation to proceed without having to worry about the Pusher communication.
Of course, by moving the Pusher event emission to an asynchronous thread, we increase the chances that our WebSocket events will arrive out of order on the browser. To cope with this, we just need to keep track of the current "offset" and make sure that we never render a smaller offset than the one that was last received.
With that said, let's look at the ColdFusion page that is directing the user to the report generation page. This page does several things:
- Generates a unique Pusher channel name.
- Subscribes to the Pusher channel (for "progress" events).
- Redirects the user to the report-generation page above.
- Updates the progress-bar when Pusher events arrive.
The last step - updating the progress-bar - happens after the redirect is initiated. While this works swimmingly in Chrome and decently in Firefox, it appears not to work at all in Safari. The good news is, nothing breaks. Meaning, since we built the Pusher integration as a "nice to have", Safari users still get the report - they just have to sit there and wait without having any pretty UI to look at.
That said, in order to remove confusion about the progress-bar, I'm going to keep it hidden until the first Pusher WebSocket event is received. This way, the progress-bar only becomes visible to those users that have a browser capable of updating the DOM post-redirect.
<cfscript>
// Normally when we use Pusher, we want to use PRIVATE channels so that no one can
// maliciously try to view the communication that is taking place. However, in this
// context, all we are doing is reporting PROGRESS UPDATES - not sensitive data. As
// such, let's just use a randomly-named channel so that we can skip the overhead of
// the authentication step that takes place with a private channel.
progressChannel = "pusher-multi-#createUniqueId()#-#createUUID()#";
</cfscript>
<cfoutput>
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>
Using Pusher WebSockets To Show Progress During Page Unload / Redirect In Lucee CFML 5.3.7.47
</title>
<link rel="stylesheet" type="text/css" href="./styles.css"></link>
</head>
<body>
<h1>
Using Pusher WebSockets To Show Progress During Page Unload / Redirect In Lucee CFML 5.3.7.47
</h1>
<p>
We are processing your request. <strong>Please be patient</strong>.
</p>
<!---
CAUTION: Hidden by default - this progress indicator will be shown to the
user when Pusher events start to arrive.
--->
<div class="progress">
<h4 class="progress__title">
<!--- To be populated by Pusher event. --->
</h4>
<div class="progress__bar">
<div class="progress__indicator"></div>
</div>
</div>
<script type="text/javascript" src="https://js.pusher.com/7.0/pusher.min.js"></script>
<script type="text/javascript">
// NOTE: We are neither using a PRIVATE channel nor an "Auth" end-point here.
// Security is NOT a concern since we're using a randomly-generated, one-off
// channel to do nothing more than report progress of an event. There's no
// sensitive information here worth hiding.
var pusher = new Pusher(
"3c697ad963151001c11f",
{
cluster: "mt1"
}
);
// Connect to our randomly-generated progress channel name, and pipe update
// events into our handler so we can update the DOM state.
// --
// NOTE: We don't have to worry about unsubscribing / unbinding from this
// channel because our page is about to be unloaded (see window.location
// assignment at the bottom). The Pusher connection will get implicitly
// cleaned-up by the unloading of the current page.
pusher
.subscribe( "#encodeForJavaScript( progressChannel )#" )
.bind( "progress", handleProgress )
;
// As progress events come through the Pusher WebSocket connection, we're
// going to update the display of our "progress" widget. Let's grab the DOM
// element references to be updated.
var progress = document.querySelector( ".progress" );
var progressTitle = document.querySelector( ".progress__title" );
var progressIndicator = document.querySelector( ".progress__indicator" );
// Keep track of the current progress offset so that we never show a
// "backwards step" in the progress bar.
var currentOffset = -1;
// I handle the progress events coming down over the Pusher WebSocket.
function handleProgress( data ) {
// Even though the events are coming down over a WebSocket, we should
// assume that there's a chance that the events can arrive out-of-order
// (especially since we don't know how all of this is being implemented
// in the WebSocket server). As such, let's only consume the given
// progress event if the event offset is GREATER THAN the last reported
// event offset.
if ( data.offset < currentOffset ) {
return;
}
currentOffset = data.offset;
progressTitle.innerText = data.name;
progressIndicator.style.width = ( currentOffset + "%" );
// CAUTION: By default, we're hiding the progress indicator (display
// none). This is because some browsers (Safari) don't appear to want to
// update the DOM after the page begins to unload. As such, we're going
// to keep the progress indicator hidden until we receive our first
// Pusher event.
progress.style.display = "block";
}
// ----------------------------------------------------------------------- //
// ----------------------------------------------------------------------- //
// The page that we are currently rendering ISN'T THE REPORT PAGE - it's the
// progress page. It's really just a starting point that is used to provide
// updates on how the SUBSEQUENT REQUEST is going. Once this page renders,
// we're using the LOCATION API to forward the user to the actual report-
// generation page; and, we're passing along the Pusher Channel that the
// current UI will use to listen for updates.
window.location.href = "./generate.cfm?progressChannel=#encodeForUrl( progressChannel )#";
</script>
</body>
</html>
</cfoutput>
As you can see, the very last thing that we do on this page is redirect the user to the report generation page using a window.location.href assignment. And, since the report generation takes a few seconds, the user will be left to stare at the above page until the report generation is complete. Luckily, in Chrome and Firefox and Edge, the Pusher WebSocket events make that fairly enjoyable:
As you can see, once the user clicks the Start Generating link, we take them to the "start" page, which is really nothing more than a jumping-off point. There, we start to render Pusher events which indicate where within the report-generation process we are. And, eventually, the window.location.href redirect completes and the user is presented with the report (theoretically - we have no actual report in this simulation).
Now, I said that this works in Chrome, Firefox, and Microsoft Edge; but, not so much in Safari. However, since the whole Pusher integration was constructed as a "nice to have", the report generation still works - it's just not as enjoyable. Here's what a Safari user sees:
As you can see, it's a much less enjoyable user experience (UX). But, it's not a broken experience. Meaning, we still tell the user that the report is being generated and we're asking them to "please be patient". And, the report does get generated. It's just, the progress bar never shows up because Safari won't update the DOM (Document Object Model) after the redirect has been initiated.
Essentially, we're treating the Pusher WebSocket integration as a progressive web enhancement. For modern browsers like Chrome, Firefox, and Edge we are able to create a better experience during the Unload / Redirect portion of the user's navigation. But, for older browsers like Safari, it still works, we just don't offer the progressively-enhanced experience.
Want to use code from this post? Check out the license.
Reader Comments
Good stuff! I wonder how many of us developers end up looking like superheros because you've been so generous with your knowledge sharing. I'd definitely consider myself one of those beneficiaries!
Also, I always (ALWAYS) love it when you include a video. Thanks for going the extra mile, especially for this specific topic!
@Chris,
Ha ha, you are very kind, good sir! I'm glad this stuff is interesting. I do enjoy making the videos, I'm just not always sure they make much of a difference for the ColdFusion stuff (I almost always make them for JavaScript posts). But, it's good to know they help even in a context like this. Noice!
Definitely very interesting stuff! The videos (at least for me) make the information feel more like a conversation, which is a really great experience in my opinion. Always appreciated!
Ben. Do you have to pay for Pusher? I had a quick look at their website, but I couldn't really find anything about pricing, which is either very good news [free] or very bad news [very expensive!!!!]
Lucee 5+, has its own websocket, baked in. I haven't actually used it, because I tend to use NodeJs with socket.io.
Anyway, nice exploration, as always.
What kind of settings do you need to add in your Application.cfc, to integrate with pusher?
One question:
Why doesn't the:
Fire immediately? How are we allowed to see the progress bar complete, before the page is redirected?
@Charles,
Pusher has a "freemium" model - there is a small free tier, but then after that you pay across two dimensions: Concurrent connections and Messages per day:
https://pusher.com/channels/pricing
At work, we find that the Concurrent connections dimension is the limiting factor. Meaning, it's the ceiling that we keep hitting where we have to keep raising our plan (we're on a custom Enterprise plan with Pusher).
ColdFusion does have a built-in WebSocket implementation, which is great; but, I think it is most effective when you have a single ColdFusion server where the browser / client is always connecting to the same server. In our case, we have dozens of ColdFusion servers being load-balanced. In that case, I think it would become very challenging to get ColdFusion's native WebSockets to work since every request from a single browser may arrive at any number of CF servers.
And, to be honest, I'm not that knowledgeable about how WebSockets work :D
@Charles,
Re:
window.location.href, it is firing right away -- that's the beauty of this exploration. The user hits the "start page", and is immediately redirected to the "generate" page. However, the "generate" page's content isn't available right away (simulated withsleep()calls). In that "in between" time where the "start" page is unloading and the "generate" page is still being processed, that's where the magic happens - that's the period where we start to update the DOM with Pusher events.Of course, as you can see in the second GIF, this doesn't actually work in Safari. Once the page starts to unload in Safari, it simply stops updating the user-interface, even though it is receiving Pusher events.
As a follow-up post, I'll try to take a look at how something like this could be wired-up without having to rely on this "in between" period. Meaning, how can we show a progress indicator using WebSockets, but also using polling as a fall-back mechanism.
I will try the Fremium version.
It looks like a really clean websocket implementation.
And the free tier 200K messages a day sounds pretty generous!
@Charles,
Word up -- plenty to get your feet wet and try it out.
Just signed up! Exciting stuff.
Do I need to add anything in the Application.cfc
I see you made a reference to an application variable:
Ben. You are a legend.
Thank you for building the pusher.cfc:
https://github.com/bennadel/Pusher.cfc
Awesome stuff!
Now, I think I understand:
Ben. I have downloaded your pusher GitHub repo:
https://github.com/bennadel/Pusher.cfc
I am using:
I have substituted the key, secret and app ID for my pusher app settings.
I get:
But, nothing else happens.
I thought I would see a message from the CFTHREAD, at the bottom of the:
By the way, I am allowing non secure connections in my Pusher dashboard. Because I am testing this from:
@All,
In this post, I talked about WebSockets as a "nice to have" - a way to progressively enhance the user experience (UX). And, in this case, the "fall back", as we saw with Safari, was to just not update the UI. This got me thinking about other ways in which we may create a fallback mechanism, which got me thinking about Redis and its blocking list operations. I started to wonder if I could use
BLPOPto power a long-polling operation that could act as a fall-back:www.bennadel.com/blog/3921-using-redis-blocking-list-operations-to-power-long-polling-in-lucee-cfml-5-3-7-47.htm
I've only used long-polling once or twice in my life, so take this with a grain-of-satl; but, if nothing else, it was a fun thought experiment.
@Charles,
Hmmm, that example code is like 8-years old. I wonder if the version of the Pusher library that is included in it is even still functionality anymore. That example is using
1.12, and in this post I'm using7.0. I'm actually shocked that the ColdFusion component even still works :DI should really give that repo some love - I don't think I've committed to it in the last 7-years. Time for a little make-over :D
OK. Cheers Ben. I didn't even look at the date on the repo:)
It's pretty good that there was no CF Error! It seems like Coldfusion is pretty robust and backward compatible.
I will wait for your updates.
I am not really in any rush to use Pusher, but it does look like a really clean WebSocket implementation. And I really like the fact that we can use Pusher's servers for the data exchange. It means we can scale our own web server/application server set up, without having to worry about how adapt the WebSocket side of things.
It would be great if you could do the same using socket.io......😀