Creating An Infinite Scroll Effect With jQuery And ColdFusion
A couple of days ago, I was on a site where additional content was being dynamically added to the bottom of the page as I scrolled down. This is not a new effect - DZone and Bing (formerly Live) have been doing this for as long as I can remember (just to name two sites). But, this "infinite scrolling" effect is definitely interesting and something that I've never played around with. As such, I figured I would take some time to experiment with it.
While at first this might seem like a very complicated concept, it's turns out to be surprisingly straightforward. The most complicated thing about it is the calculation of element offsets; but, jQuery makes this as easy as calling a few intelligent methods. Assuming the calculations are readily available, we just need to start thinking in terms of a view frame (the browser window) and the content over which we are scrolling. At the most abstract level, all we want to do is add more content to the content area when the bottom of the content area approaches (or passes) the bottom of the view frame.
As you can see in the above graphic, we think of our content area in the context of the browser's view frame which "clips" the document content. To make the infinite scrolling effect more seamless, however, we don't deal with the content bottom directly - we deal with an offset, "buffered" bottom. In this way, the user doesn't have to hit the bottom before new content is loaded; rather, new content is loaded when the user simply gets close to the bottom of the content. This increases the likelyhood that our AJAX requests will have returned (and updated the DOM) before the user has to stop scrolling.
The concept is rather easy - it's the calculations of the element heights and offsets that is complicated; but, as you will see in the following demo, jQuery actually takes care of most of the heavy mathematical lifting.
<!DOCTYPE HTML>
<html>
<head>
<title>Infinite Scroll With jQuery And AJAX</title>
<style type="text/css">
#list {
list-style-type: none ;
margin: 0px 0px 0px 0px ;
padding: 0px 0px 0px 0px ;
}
#list li {
border: 2px solid #D0D0D0 ;
cursor: pointer ;
float: left ;
height: 50px ;
line-height: 49px ;
margin: 0px 10px 10px 0px ;
text-align: center ;
white-space: no-wrap ;
width: 150px ;
}
#list li.on {
background-color: #F0F0F0 ;
border-color: #FFCC00 ;
font-weight: bold ;
}
#loader {
clear: both ;
}
</style>
<script type="text/javascript" src="../jquery-1.4a2.js"></script>
<script type="text/javascript">
// I get more list items and append them to the list.
function getMoreListItems( list, onComplete ){
// Get the next offset from the list data. If the next
// offset doesn't exist, default to one (1).
var nextOffset = (list.data( "nextOffset" ) || 1);
// Check to see if there is any existing AJAX call
// from the list data. If there is, we want to return
// out of this method - no reason to overload the
// server with extraneous requests.
if (list.data( "xhr" )){
// Let the active AJAX request complete.
return;
}
// Get a reference to the loader.
var loader = $( "#loader" );
// Update the text of the loader to denote AJAX
// activity as the list items are retreived.
loader.text( "Loading New Items" );
// Launch AJAX request for next set of results and
// store the resultant XHR request with the list.
list.data(
"xhr",
$.ajax({
type: "get",
url: "./infinite_scroll.cfm",
data: {
offset: nextOffset,
count: 60
},
dataType: "json",
success: function( response ){
// Append the response.
appendListItems( list, response );
// Update the next offset.
list.data(
"nextOffset",
(nextOffset + 60 + 1)
);
},
complete: function(){
// Update the loader text to denote no
// AJAX activity.
loader.text( "Page Loaded" );
// Remove the stored AJAX request. This
// will allow subsequent AJAX requests
// to execute.
list.removeData( "xhr" );
// Call the onComplete callback.
onComplete();
}
})
);
}
// I append the given list items to the given list.
function appendListItems( list, items ){
// Create an array to hold our HTML buffer - this will
// be faster than creating individual DOM elements and
// appending them piece-wise.
var htmlBuffer = [];
// Loop over the array to create each LI element.
$.each(
items,
function( index, value ){
// Append the LI markup to the buffer.
htmlBuffer.push( "<li>" + value + "</li>" );
}
);
// Append the html buffer to the list.
list.append( htmlBuffer.join( "" ) );
}
// I check to see if more list items are needed based on
// the scroll offset of the window and the position of
// the container.
function isMoreListItemsNeeded( container, list ){
// Get the view frame for the window - this is the
// top and bottom coordinates of the visible slice of
// the document.
var viewTop = $( window ).scrollTop();
var viewBottom = (viewTop + $( window ).height());
// Get the offset of the bottom of the list container.
//
// NOTE: I am using the container rather than the list
// itself since the list has FLOATING elements, which
// might cause the UL to report an inacturate height.
var containerBottom = Math.floor(
container.offset().top +
container.height()
);
// I am the scroll buffer; this is the amount of
// pre-bottom space we want to take into account
// before we start loading the next items.
var scrollBuffer = 150;
// Check to see if the container bottom is close
// enought (with buffer) to the scroll of the
// window to trigger the loading of new items.
if ((containerBottom - scrollBuffer) <= viewBottom){
// The bottom of the container is close enough
// to the bottom of thew view frame window to
// imply more item loading.
return( true );
} else {
// The container bottom is too far below the view
// frame bottom - no new items needed yet.
return( false );
}
}
// I check to see if more list items are needed, and, if
// they are, I load them.
function checkListItemContents( container, list ){
// Check to see if more items need to be loaded.
if (isMoreListItemsNeeded( container, list )){
// Load new items.
getMoreListItems(
list,
function(){
// Once the list items have been loaded
// re-trigger this method to make sure
// that enough were loaded. This will make
// sure that there are always enough
// default items loaded to allow the
// window to scroll.
checkListItemContents( container, list );
}
);
}
}
// -------------------------------------------------- //
// -------------------------------------------------- //
// When the DOM is ready, initialize document.
jQuery(function( $ ){
// Get a reference to the list container.
var container = $( "#container" );
// Get a reference to the list.
var list = $( "#list" );
// Bind a click handler to the list so we can toggle
// the ON class of the list items upon click. This is
// simply to demonstrate "live" binding to the list
// elements as they are loaded.
list.click(
function( event ){
var target = $( event.target );
// Check to make sure the list item is the
// event target.
if (target.is( "li" )){
// Toggle the ON class.
target.toggleClass( "on" );
}
}
);
// Bind the scroll and resize events to the window.
// Whenever the user scrolls or resizes the window,
// we will need to check to see if more list items
// need to be loaded.
$( window ).bind(
"scroll resize",
function( event ){
// Hand the control flow off to the method that
// worries about the list content.
checkListItemContents( container, list );
}
);
// Now that the page is loaded, trigger the "Get"
// method to populate the list with data.
checkListItemContents( container, list );
});
</script>
</head>
<body>
<h1>
Infinite Scroll With jQuery And AJAX
</h1>
<div id="container">
<ul id="list">
<!--- Content loaded dynamically. --->
</ul>
<!---
NOTE: While this element does have some "feedback"
purposes, I am mostly using it to make sure the
"CONTINER" element reports back a valid height -
it will cause proper rendering.
--->
<div id="loader">
Page Loaded.
</div>
</div>
</body>
</html>
In the above code, when the page is first loaded, or whenever the window is scrolled or resized, I call the checkListItemContents() method. This method checks the layout of the content against the view frame of the browser (as in the graphic above) and determines if more content needs to be loaded. When this method executes an AJAX request for more content, it defines a "complete" callback that, again, calls the checkListItemContents() method. This circular execution ensures that more and more content will be loaded until the content container's bottom is successfully pushed below the page fold.
For demonstration purposes, I created a very small ColdFusion page that would return an array of simple string values, to be used to generate the content:
infinite_scroll.cfm
<!--- Param the offset variable. --->
<cfparam name="url.offset" type="numeric" default="1" />
<cfparam name="url.count" type="numeric" default="30" />
<!--- Create the return array. --->
<cfset items = [] />
<!---
Loop over the count to populate the return array with
test data.
--->
<cfloop
index="index"
from="#url.offset#"
to="#(url.offset + url.count)#"
step="1">
<cfset arrayAppend( items, "Item: #index#" ) />
</cfloop>
<!--- Serialize the array. --->
<cfset serializedItems = SerializeJSON( items ) />
<!--- Convert it to binary for streaming to client. --->
<cfset binaryItems = toBinary( toBase64( serializedItems ) ) />
<!--- Set the content length. --->
<cfheader
name="content-length"
value="#arrayLen( binaryItems )#"
/>
<!--- Stream binary content back as JSON. --->
<cfcontent
type="application/x-json"
variable="#binaryItems#"
/>
As you can see, this just returns an array of string data. Each of these index values is then transformed into an LI element and appended to the content.
In my demo, I included a Div (#loader) at the bottom of the content, but within the content container. While this div does provide some feedback for AJAX activity, mostly I included it to make sure the content container rendered with a proper height. Because the LI elements within my content list float left without any "float-clear magic", I thought it might prevent the height() of the content container from being calculated accurately. Since the loader div clears floats and does itself have a rendered height, I knew it would force the browser to render the height of the content container effectively.
When I first thought about creating an infinite scrolling effect, I have to admit, it made me nervous; while I understood the concept, I thought it would be far too complicated coming up with appropriate calculations. As I hope I have demonstrated above, however, using jQuery makes this something of a non-issue. With methods like height() and offset(), jQuery takes away all the complications surrounding these type of calculations and makes creating this effect much easier than first anticipated.
Want to use code from this post? Check out the license.
Reader Comments
Awesome, that's a really comprehensive explanation! I like how you dig into a certain topic and not just explain how to use a plugin but how things actually work in real life ;)
Also gotta say: the illustration rocks :)
May I ask how you do that, principally the snappy arrows?
@Martin,
Thanks a lot my man - digging is sometimes the only way I can wrap my head around a topic (and make it stick).
For the graphics, I use Fireworks - the arrows are created with the Pen tool.
We've had this on our site for a while now as well. You'll find that when you start testing with other browsers and more complicated content, the problem isn't as simple as you'd imagine. Once you get variable height list items, dynamic content (images, etc) and suchlike, height calculations become a nightmare!
It's an interesting programming exercise but please, for the love of all that is holy, keep it an educational one only. Using this type of content loader slows down the user experience, puts a huge strain on your servers as the number of HTTP requests goes through the roof, and is a disaster on mobile devices as their JavaScript performance is generally less than ideal. Users are used to pagination and if it aint broke, don't fix it!
George.
@George,
The good thing with the variable-height content, at least in my example, is that it is using the height of the content container, not the content itself. As such, the way the content is rendered shouldn't much make a difference.
I agree that this is something that would *never* work on a mobile device! I am not sure the bandwidth would like that at all.
As far as standard users, I am not sure that it would really slow down their experience? I can only see this making the experience better. I think the trick is to no make any additional event bindings so that the only toll is on the size of the content that has to be rendered.
Of course, the more content you have in the window, the more the browser will start to lag. I wonder if we can come up with a "Best of both worlds" approach that uses this AND pagination (to some degree)?
As for the server, yeah, there's gonna be more demand on it.... but only as much as the user wants to request. Meaning, the new content is only requested when the user scrolls. And, I think we can assume the user only scrolls when they think there is more content they want to see. So, really, I think it provides a nice user experience.
... I'll do some more thinking on this though, thanks.
@Ben,
I know it's written in Objective C, but the eBay app on the iPhone has this similar functionality. Works nicely through that. Once you scroll to the bottom of your watch/sell/bid list, it shows "Loading", then loads the next items.
You mention paging, perhaps you could do something at the bottom *and* top of your page. Limit it to perhaps 100/page. You could remove the top, say, 25 items, then load the next 25 items at the bottom. Scroll up, it removes the bottom 25 then reloads the top 25. More hits on the server, but less likely to crash a user's browser.
@Gareth,
That's an interesting approach! The toughest thing would be to remove content from the top without *jumping* the scroll around.
great article. greetings from germany
Ben, nice job...
Is your code meant to serve as an example or are you thinking of your JavaScript code as more of a JQUERY Plug-in?
I ask because I'm interested in implementing your technique but concerned with future support/maintenance.
Thanks
Brett
@Brett,
Really, it's just meant as an experiment; it's not architected as a plugin currently. I am actually in the middle of writing a new blog post on the topic to be released tonight / tomorrow, so you might want to check that out as well.
I'll post a comment here when that is done.
Hey guys, I took this concept and I went "bidirectional" with it:
www.bennadel.com/blog/1803-Creating-A-Bidirectional-Infinite-Scroll-Page-With-jQuery-And-ColdFusion.htm
I don't use coldfusion at all, but it's an interesting process anyway.
I'll adapt it for Drupal, using the Views module.
Less clics, mean a better user experience.
@Guillem,
I don't know anything about Drupal... but I'll agree with you - less clicks *does* mean a better user experience.
How would you prevent this from contacting the server after you have reached the end of the results? Would you just unbind the scroll and resize if no data was returned?
Thanks!
Randy
What I did to stop this was check to see that there was no data returned, and then I did a list.data("done",true);
Then right underneath where we check for the list.data("xhr")
I added this:
if (list.data( "done" )){
return;
}
That did the trick.
Randy
@Randy,
That sounds like a good plan. You could either set a flag, or you could unbind the scroll/resize event handler as you first suggested. I think both approaches are fine. The only benefit, that I can see, of unbind the event handlers:
1. The logic of the scroller feels like it has less to think about.
2. The event handlers don't need to be call after they are no longer needed, which will free up a minimal amount of processing (for the client).
Of course, with the method called, isMoreListItemsNeeded(), it seems like a semantically perfect place to add the request logic.
Can someone please post a quick update on how you would implement this with a "More Button" click rather than plain scrolling? For my application I would like users to click a button to dynamically load more content at the bottom of the transcript. Thanks.
Nevermind, I eventually figured it out, I was just being lazy at first. :)
The incredible simple solution is to change the $( window ).bind handler from window to "input" (the quotes are required around input) then change the action from "scroll resize" to just "click" then create a simple button input type (I did it inside the div "Container" not sure if that matters) and when you click the button it will execute the same as if you scrolled
Hi, thank you for this post.
I'm trying to do an infinite scrolling, i want to load tweets from different twitter users and different hashtag.
Do someone as a solution ?
i sucedeed to do this : http://geoffrey-menissier.fr/test/ but its only load tweets from one user.
(based on this tutorial : http://www.marcofolio.net/webdesign/jquery_quickie_unlimited_scroll_using_the_twitter_api.html)
Thank you, and sorry for my bad english :)
It works for me only when I put all functions in the jQuery(function( $ ) bracket..