How Client-Side Validation Is Changing The Shape Of Server-Side Validation
It used to be (and often times, still is) that when a user submitted a form in a web application, the entire page would be submitted to the server where it would be validated. Then, the form would either be re-rendered with user-friendly error messages; or, the user would be forwarded on to a "success" page. With the explosive growth of JavaScript and DOM (Document Object Model) manipulation libraries, however, we now see many web applications using both client-side validation for an improved user experience and server-side validation for security concerns. While this shift towards the client-side leads to a richer, more responsive user interface (UI), it is also changing the shape of server-side validation. This latter change, while invisible to the end-user, has significant benefits waiting to be embraced by the developer.
NOTE: I am not talking about sites that are using "progressive enhancement" to add client-side validation. I am referring to web application "platforms" that require JavaScript in order to work properly.
Before there was JavaScript and AJAX (Asynchronous JavaScript and XML), developers had to rely on the server-side code to do all of the form processing, data validation, and page rendering. As such, the server-side code was tightly bound to the user experience. This limitation required the server-side code to be written in a way that resulted in the best user experience.
To see what I'm talking about, take a look at the following code that demonstrates a server-side-only approach to validation:
<!--- Param the form values. --->
<cfparam name="form.name" type="string" default="" />
<cfparam name="form.email" type="string" default="" />
<cfparam name="form.birthday" type="string" default="" />
<!--- Create a collection of form validation errors. --->
<cfset errors = [] />
<!--- Validate the name. --->
<cfif !len( form.name )>
<cfset arrayAppend( errors, "Please enter your name." ) />
</cfif>
<!--- Validate the email. --->
<cfif !isValid( "email", form.email )>
<cfset arrayAppend( errors, "Please enter a valid email." ) />
</cfif>
<!--- Validate the birthday. --->
<cfif !isNumericDate( form.birthday )>
<cfset arrayAppend( errors, "Please enter a valid birthday." ) />
</cfif>
<!---
Check to see if any errors were found during form
validation - if so, we'll have to pass them back to
the user.
--->
<cfif !arrayLen( errors )>
<!--- Form data is valid, continue to process the request. --->
</cfif>
<!---
If we made it this far, re-render the input form and display
the errors for the user.
--->
NOTE: This code is incomplete as in only intended for discussion purposes.
In the above code, we are processing a form submission that has three fields (name, email, birthday). If any one of the fields is invalid, the form cannot be processed properly; however, since the server-side code is solely responsible for the user experience, it is paramount that all fields be validated at the same time (to reduce the number of request-response life-cycles). It is also necessary for the server-side code to know how to best phrase the error messages as they relate to the user interface (UI).
The structure of the server-side validation code is byproduct of the server-centric architecture and its tight coupling to the user interface.
As JavaScript and AJAX have caught on like wildfire, our browsers have transformed from thin-clients into thick-client, robust, web application platforms. While our servers still house our data, business constraints, and security measures, much of the business logic and user-interaction model has been offloaded to the client. Server-side validation, which we saw above, is now handled instantly on the client-side using JavaScript:
<!DOCTYPE html>
<html>
<head>
<title>Client-Side Validation</title>
</head>
<body>
<form>
<ul id="errors" style="display: none ;">
<!-- Errors go here. --->
</ul>
Name: <input type="text" name="name" /><br />
Email: <input type="text" name="email" /><br />
Birthday: <input type="text" name="birthday" /><br />
<input type="submit" value="Submit" />
</form>
<script type="text/javascript" src="../jquery-1.7.js"></script>
<script type="text/javascript">
// Bind to the form submission.
$( "form" ).submit(
function( event ){
// Get the form being submitted.
var form = $( this );
// Get the errors collection.
var errorsList = $( "#errors" )
.empty()
;
// Validate the name.
if (isEmpty( form, "name" )){
errorsList.append(
"<li>Please enter your name.</li>"
);
}
// Validate the email.
if (isEmpty( form, "email" )){
errorsList.append(
"<li>Please enter a valid email.</li>"
);
}
// Validate the birthday.
if (isEmpty( form, "birthday" )){
errorsList.append(
"<li>Please enter a valid birthday.</li>"
);
}
// Check to see if we have any errors.
if (errorsList.children().length){
// Show the errors.
errorsList.show();
// Cancel the form submission.
event.preventDefault();
}
}
);
// I check to see if the given field is empty.
function isEmpty( form, fieldName ){
// Return true if the field is empty.
return( form.find( "input[ name = '" + fieldName + "' ]" ).val() === "" );
}
</script>
</body>
</html>
NOTE: This code is incomplete as in only intended for discussion purposes.
As you can see here, the name, email, and birthday validation are handled in realtime using client-side JavaScript as soon as the user submits the form. It is only after the client-side code has validated the data that the form is actually submitted to the server.
For security concerns, and for data that simply cannot be validated on the client, it is still critically important to have server-side validation. However, with the application of client-side validation, we can start to significantly simplify the way our server-side code works.
If you look at the code in our server-side-only validation model (the first example), there are several aspects that become irrelevant in a world with client-side validation:
- We need to validate all form fields in order to limit the number of request-response life-cycles.
- Our error messages have to be user-friendly and have to relate directly to the user interface (UI) in which they will be displayed.
- Our server-side processing code needs to re-render the page in order to display the error messages.
When data validation is offloaded to the client in a thick-client web application, handling the above concerns on the server-side is sub-optimal and redundant. Instead, we can keep the concerns of our server-side code narrowly focused and decoupled form the user interface / user experience:
<!--- Param the form values. --->
<cfparam name="form.name" type="string" default="" />
<cfparam name="form.email" type="string" default="" />
<cfparam name="form.birthday" type="string" default="" />
<!--- Create a default API response. --->
<cfset response = {} />
<cfset response[ "success" ] = true />
<cfset response[ "code" ] = "200" />
<cfset response[ "data" ] = "" />
<cfset response[ "error" ] = "" />
<!--- Try to process the form data. --->
<cftry>
<!--- Validate the name. --->
<cfif !len( form.name )>
<cfthrow
type="BadRequest"
message="Name is required."
/>
</cfif>
<!--- Validate the email. --->
<cfif !isValid( "email", form.email )>
<cfthrow
type="BadRequest"
message="Email is required (and must be valid)."
/>
</cfif>
<!--- Validate the birthday. --->
<cfif !isNumericDate( form.birthday )>
<cfthrow
type="BadRequest"
message="Birthday is required (and must be valid)."
/>
</cfif>
<!--- ..... --->
<!--- Form data is valid - continue to process request. --->
<!--- ..... --->
<!--- Catch bad request exceptions. --->
<cfcatch type="BadRequest">
<!--- Flag the response as a failure. --->
<cfset response.success = false />
<cfset response.code = "400" />
<cfset response.error = cfcatch.message />
</cfcatch>
</cftry>
<!--- Return the API response back to the client. --->
<cfcontent
type="text/x-application-json"
variable="#toBinary( toBase64( serializeJSON( response ) ) )#"
/>
NOTE: This code is incomplete as in only intended for discussion purposes.
Here, our server-side validation code is taking on an API-oriented approach to application architecture. Gone are the user-friendly, tightly coupled error messages; gone is the need to validate unnecessary form fields; gone is the need to re-render an HTML page. Our server-side code still validates the form submission; but, with the burden of user-experience hoisted upon the shoulders of the client, our server-side code can execute efficiently and cohesively.
As we start building client-side validation into our thick-client web applications, it is tempting to make the client-side logic a direct mirroring of the server-side logic; I know because I still do this (more often than I'd like to admit). This mentality is a hold-over from an era when server-side code was all that we had available. I keep wanting to make my server-side error messages user-friendly because somewhere, in the back of my mind, there's still a little voice that says, "But what if the user doesn't have JavaScript turned on?" Of course, I completely ignore the fact that without JavaScript, a user would have never made it this far into my application. The reality is, as our client-side code becomes more robust and more powerful, our server-side code can and should change to reflect its new role in the interaction model.
Want to use code from this post? Check out the license.
Reader Comments
Hi Ben,
Interesting stuff as always. One question:
At the end of your last example you return a JSON response to the browser like this:
toBinary( toBase64( serializeJSON( ....
Any particular reason for the back-and-forth conversion between base 64 and binary? I think I 've seen you do it before, but I never really understood why.
@Martijn,
The Variable="" attribute of the CFContent tag has to take a byte array (binary value). In order to get pass the JSON through, I first convert it to base64 and then to binary.
You could probably do something like this:
... in order to get the bytes for the serialized response value. But, the getBytes() method is not really documented - it's a byproduct of the underlying Java String class. I go back and forth on whether its worth using the underlying Java methods.
Since ColdFusion sees strings and Java String instances as the *same* thing, one would be led to believe that using the String class methods is completely fine:
www.bennadel.com/blog/1023-ColdFusion-Wants-You-To-Access-The-Underlying-Java-Methods.htm
But, there's definitely still an emotional constraint (which is not really rational at all).
I was so fed up having to code form validation twice, once in JavaScript for the client and again in CFML for the server that I too changed to doing it just once on the server via XHR.
And you know what, submitting a form for validation via XHR doesn't appear any slower than doing it in JS on the client (at least if you have a broadband connection).
My CFML response sends back the #id of each field that failed validation with apprioriate messages and some JavaScript on the form page ensures the correct message is shown next to each failed field. Neat.
e.g. ["id":"firstname","msg":"Your first name is required"], ["id":"email","msg":"Your email address is not valid"]
Hi Ben,
Good stuff.
Of late, I've been looking into features introduced in HTML5. Sometime back I had posted on form validation in HTML5 -http://www.sagarganatra.com/2011/07/custom-validation-messages-for-html5.html.
The idea is to let the browser validate the form fields. Though this is not supported on all browsers as of now, but I think once all browsers implement it, we can make use of it.
@Sagar, HTML5 form validation should still be used in addition to server-side validation, so it won't reduce the work required by developers. Unless it's possible to tell the browser to trigger its built-in validation prompts through JavaScript when the XHR response from CF is returned. e.g. "Tell the email field that it's failed and to flag it up to the user." (It might save you 20 lines of JS and CSS.)
Most of my clients are enterprises and are stuck on IE6/7/8 so I won't be implimenting any HTML5 stuff for years! On the plus side I appreciate their repeat business!
Ben,
Nice post. I layed out some similar concepts in a recent post on how I use CF with Ajax:
http://www.cutterscrossing.com/index.cfm/2011/9/26/How-I-Do-Things-ColdFusion-and-Ajax-Requests
@Gary,
I understand you on that one. Mostly, I create some sort of in-between situation like that as well. I still have JavaScript assemble the data and submit it to the server's API. However, really, the error checking is being handled, like you, completely on the server side.
I think this happens for two reasons:
1. I've been slow to actually create well structure API architecture on the server-side.
2. I'm lazy about creating a nice validation experience on the client-side (I'll be honest, I'm not that great at it, as you can see by my use in UL/LI error-message elements I use in pretty much every demo I've ever created).
Now that I am starting to concentrate more on creating good server-side architecture, I'm *hoping* that I'll also become better at creating a better client-side experience.
NOTE: I am not saying that your/mine approaches cannot be used to create an elegant client-side approach. I'm only saying that *I* am personally lazy about the client-side experience... which I guess is really a different conversation altogether.
@Sagar,
The HTML5 validation seems pretty cool. So far, I've only seen presentations and demos - I've never actually tried anything out myself. I think the only thing I've actually tried with it is to use the "email" input to bring up a special keyboard on the iOS devices :D
@Cutter,
Oh cool, I'll check it out, thanks!
@Gary, The form validation part on the client side can be taken care by the browser itself. It can validate the field based on the input type. But, if one has to customize the error message then some additional JS code would be required.
And yes, as I mentioned earlier, this feature is not available in all browsers yet. But, I do see people using modern browsers like Chrome, Firefox etc, nowadays. IE users can upgrade to IE 10 which IMO is fast and supports several features.
@Sangar, sorry I don't think I explained myself well. I know that HTML5 has some advanced client-side validation but you must also do server-side validation for security. For consistency it would be useful for the server-side validation to tell the browser what error message to display, especially if your server-side validation is more thorough or complex.
@Ben, I'm lazy too but the idea of just having to do form validation code once made me think that spending an hour or two on the presentation of the validation messages was worth it. I made it fairly transplantable to other projects.
I'm sure I've mentioned it before, but I get really fed up having to do it twice. I wish there was some way for the browser to not allow Javascript to be disabled. >:-} MUAHAHAHA...Or that some new scripting language or other type of language/technology could come that would then be required by all browsers and not be able to be turned off, and all later versions of browsers would not be able to turn it off, and eventually all people would be using later versions of those browsers and it wouldn't have to be done twice anymore. My main complaint is the time it takes to do this and unreasonable demands on one's time, along with unreasonable time constraints and deadlines. If the pm's said...sure, take all day to do validation on this form...or whatever...take all the time you need...I doubt I would be as aggravated with it as I am, but when you are being rushed through project after project, and time is of the essence, and you are getting in trouble when certain projects aren't done, and then you have to take the time to do double validation, it can get kind of annoying. Anyway. Sorry for the rant. I currently work in a controlled environment where our users aren't allowed to turn off javascript, or if they do, it is turned back on for them. So many of our projects only do the javascript validation...and that is all that is required of my current projects. So I am not complaining about my current job or work I am doing now. Just how much of a hassle it has been at times in the past.
@Anna. I think Ben said that using this server-side + XHR approach to post & validate the form you're forcing the users to enable Javascript. Otherwise if it's disabled they can't post the form at all! Therefore invalid data won't end up in your database.
What % of users turn off JS? And who are they? I think I've identified them by group:
1) The mega paranoid and overly-cautious security squirrels.
2) The few remaining purists who believe JS should not be required to use a website.
3) Family & friends advised by the above folk to disable JS otherwise their computer will get a virus, their bank accounts will be emptied and their car stolen while they sleep.
We really should haven't to worry about them. I'm sure most of today's websites that offer any kind of interactivity would fail if you disable JS on your browser, so you're in good company. Imagine trying to code a modern site without using JS. With that mindset people might as well remove Bluetooth connectivity from new cars or WiFi from buildings!
You could always detect if the browser is running JS and send them to a special page to either apologise that they can't use your site, or to a static version of the site with minimal content. But it's not worth the effort.
Statistics I've found around the web for clients disabling JS vary from 0.4% to 1.5%
@Anna, @Gary, et al,
I think you need to maintain both client and server-side validation. Client side validation for user experience, and server-side for security. I also think that you have to craft this carefully, to maintain graceful degradation for those few out there who do operate with JS turned off. Aside from the groups Gary mentioned, there are those disabled who are working with screen readers and such.
@Gary,
What about people who are, for instance, visually handicapped and use screen readers, or people who need some other special technolgy to be able to take part in online interaction? Will they be able to use your Javascript-based thick-client application?
Sometimes this is not an issue of course, but if you do projects for the government for instance, you are required to accomodate the needs these people als well as "regular" users (at least, that's how it is in the Netherlands).
@Cutter,
Sorry, missed your comment, no need to make the same point twice of course. :)
@Gary, good point! I should have read the post more carefully before I commented. I guess I was just kind of skimming over it while at work, trying to meet one of my unreasonable deadlines. :-D Ha ha, just joking, of course...where I work now, I don't run into that much, but if the demand on the programmer is going to be to have to use all of this validation...both sides validation -- client and server, then more time will HAVE to be made for development. I'm sorry, but I have had project managers pushing for way over-the-top ridiculous and unreasonable deadlines, fueled by users pushing them who believe that a hyperlink is just "text on a page that you click to go to another page", demanding you develop a fully-functional application on the page that you click through to using that hyperlink, and don't take into consideration the development time that is reasonably required to develop that app. They think all you have to do is throw some text on a page, make it magically link to the app, and voila!!! There it is!!! And that should take less than an hour. Anyway. Off my soapbox. Back to my projects. Thanks!
@Gary,
@Cutter and @Martijn are right. But it's not all screen readers. The group that used to turn off JavaScript more than any other was users of the JAWS screen reader. At one point in time, JAWS just couldn't handle JavaScript.
That's changed a lot. Have a look at
http://webaim.org/projects/screenreadersurvey3/#javascript
I guess you could still say that blind folks turn off JavaScript more than any other group. That is, their 1.6% is still above the high end of your perceived range of 0.4% - 1.5%, ... but not by much.
Another interesting trend is that free screen readers are eating into the marketshare of commercial screen readers. In particular, VoiceOver is now up to 3rd place (barely):
http://webaim.org/projects/screenreadersurvey3/#used
But back to the topic at hand: Requiring JavaScript to be on is not the forbidden, horrifically discriminatory outrage it once was. JavaScript is now enhancing the user experiences of blind folks just as much as sighted folks. You could now make the argument that avoiding JavaScript (and not having the conveniences that JavaScript makes possible) is more discriminatory against the blind than requiring it to be on.
I like the approach that Gary outlines via XHR.
That said the other approaches are valid based on the application. ValidateThis makes it very easy to set up both server and client validation.
@All,
I don't know too much about handicap compliance and 508 stuff. I can say that of a good deal of what I build, I can't imagine any of it being used by someone who is visually impaired. There's simply too much data on the screen and too much dynamic HTML for it to be usable.
That said, I'm also referring to rich web application platforms in this post - where there is a certain understanding that JavaScript is required as a technology.
@Sam,
I've heard a lot of people say great things about ValidateThis. I think I even went to a Bob presentation on it (or perhaps it was someone else - I can't remember). I'll have to take another look.
@Ben,
Yeah, that was my point. Used properly, JavaScript is the magic ingredient that can fix everything that's still royally messed up about HTML and CSS. It now helps accessibility much more than it hurts it.
You're well justified in your numerous experiments and explorations to expand JS and its usefulness. The JavaScript haters have lost.
Nice post Ben.
I spend a lot of time with our back-end Java programmers discussing how the validation/form data "handoff" should occur. I think the modern front-end layer should make it easy for users to complete form inputs, while making sure data is valid. This ensures a richer user experience with modern web applications. Our Java programmers like to handle a % of input error/success responses, but I think the UI should handle that in most cases. I do understand their position, as I'm also a PHP/PERL programmer. Either way, I'm glad the old days of pure server-side validation are over (mid-to-late 1990s).
Like @Gary, I've been using 100% XHR server-side validation for years with great results. And there is one huge bonus: when you submit your form with XHR, and there is a validation error, you do not have to repopulate the entire form: if success, redirect with java script, if errors, show / highlight errors -- that's it.
The experience is just as good (typically) as with client-side validation, and you don't have to repeat yourself.