Exercise List: Thinking About Data Validation - Who, What, And Where?

What about client-side validation too? Maybe with Ajax on blur of a field?

Some interesting plugins for the client-side validation:
http://digitalbush.com/projects/masked-input-plugin
http://bassistance.de/jquery-plugins/jquery-plugin-validation/

Glen

Ben,
You really are diving in with both feet! The conclusion you came to is what I utilize (service layer validation) for server side validation. If there is an error on the server side, I return an error collection containing the messages to return to the user. Each of these messages uses a resource bundle key to get the actual text (exercise.nameInUse = "That exercise name is already in use"). I place the message directly into the error collection prior to returning it to the view under the "key" of the property. I push the responsibility of message display onto the view (this gives the ability to provide contextually relevant error messages).

I'm surprised Exception is not mentioned throughtout the article. Mr. Sean A Corfield himself stated that "...conceptually CF has less overhead with exceptions that even Java has." - http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:120487

I would add a .validate() method in the Bean/Business Object (BO). The validate() method returns void but throw exceptions. The UI is responsible of catching the exception and display the cfthrow.message or a more friendly message. You may choose to add a validateBean() method or something of that nature in the service layer if u want your service layer to be acted as a facade.

What do you think?

@ben:

That could work, the only problem I have is the feel that you have now tightly bound your view to the implementation of the constants in your service layer. Here is a code snippet of what i was talking about, which may give you a clearer insight into what I am talking about:

<cfif NOT len( Exercise.getName() )>
<cfset Errors.add("nameLength", "A Name must be provided for the exercise" ) />
</cfif>

Then in the view:
<input name="exerciseName" value="#Exercise.getName()#">
<cfif Errors.hasError("nameLength")>* #Errors.getError("nameLength")#</cfif>

This can be done as i've shown with an error object, or can be implemented via structures and structKeyExists for the view check. This decouples the view from the service. To take it a step further, you can swap out:

<cfset Errors.add("nameLength", "A Name must be provided for the exercise" ) />

with:

<cfset Errors.add("nameLength", getResourceBundle().getResource("exercise.nameLength") ) />

this will allow you to utilize multiple text strings (localization usually) within the view tier just by adding additional resource bundles such as:

exercise_US -
exercise.nameLength=A Name must be provided for the exercise.

exercise_GB -
exercise.nameLength=A Name must be provided, you bloody fool!

I'm with you Ben. I like to keep my validation outside my domain objects AND my controllers. I've taken to the approach that you metioned where there' s a validateXxx method in the service layer that accepts an instance of the CFC you want to validate. I also perform a validation check on the CFC instance before I send it off to my DAO to be saved. This works pretty well for making sure only valid CFC instances get persisted, but this isn't the most helpful validation at the cilent level.

Since you're a pretty creative guy who likes to try things out I'd like to share an idea I've been kicking around with you. I've been thinking about developing a validation and form-processing "framework/library/whatever" that can perform validations at all levels of the request lifecycle. What I mean is I want to declare my validations for my domain objects in one place and be able to have those validations translated to work in all layers of an application. For instance, translating the validations to ActionScript so if I'm using a Flex client the Flex client doesn't need to do a server request to validate the data; or to translate them to JavaScript so if an HTML client is being used client side validation can be performed automatically. I'd also like the framework to perform server-side validation by validating FORM submissions and CFC instances. I'll explain the difference between these validation types more in a little.

Before I go any farther, SOME of this can already be accomplished by using the Validat framework developed by Jeff Chastain of alagad. See web site here: http://trac.alagad.com/Validat

And I'd also like to mention that some of the concepts mentioned above have already been implemented in other validation engines like the Apache Commons Validator framework.

What I'd like to do is expand on the implentations and concepts provided by these frameworks. First I'd like to diffentiate between two types of validations. FORM validations and DOMAIN object validations.

1.) Domain object validations are pretty straightforward. You're validating stuff like "first name can't be blank, and can't be more than 30 characters in length", and that an email address is really an validly formatted email address." You can also validate much more complex stuff that can only be done on the server.

2.) Form validation is usually a pretty custom thing. Sure you can re-use routines for checking data types, lengths, etc... but the fields on a form may not map directly to the properties of a domain object, causing you to not be able to re-use your domain object validation for your form fields. Or in the case of very complex and/or long forms, a single form may only represent a segment of a domain object and not the entire thing.

I'll get to some examples in a minute, but what I'm getting at here is it would be beneficial to have a validation framework/library/whatever that:

a.) Allows you to declaritively define what needs to be validated
b.) Allows you to easily re-used existing/common validation rules and add new custom rules
c.) Allows you to specify if a validation can only be done on the server side
d.) Allows you to specify rule implementations/translations in various languages (ActionScript, JavaScript, heck any language)
e.) Allows you to create validation/rule inheritence hierarchies so you can over-ride and add additional validations to a validation/rule set definition.
f.) Allows you to apply only the validations for a domain object that you need to apply (for instance only run the validations for the first, middle, and last name fields for a Contact object; ignore the address, address2, city, state, zip fields for now.)

So lets take the example of a Customer CFC. The CFC has the following properties:

FirstName: String
LastName: String
MiddleInitial: String
Address: String
City: String
State: String
Zip: String
Phone: String
Email: String
FavoriteNumber: Numeric

The declarative validation for the Customer.cfc could be stored in an XML file like so:

<validationSet component="com.myapp.Customer" >
<property name="firstName" >
<rule name="required" />
<rule name="maxLength" length="30" />
</property>
<property name="middleInitial" >
<rule name="maxLength" length="1" />
</property>
<property name="lastName" >
<rule name="required" />
<rule name="maxLength" length="30" />
</property>
<property name="address" >
<rule name="required" />
<rule name="maxLength" length="30" />
</property>
<property name="city" >
<rule name="required" />
<rule name="maxLength" length="30" />
</property>
<property name="state" >
<rule name="required" />
<rule name="exactLength" length="2" />
<rule name="UnitedStatesState" />
</property>
<property name="zip" >
<rule name="required" />
<rule name="UnitedStatesZipCode" />
</property>
<property name="phone" >
<rule name="required" />
<rule name="maxLength" length="30" />
</property>
<property name="email" >
<rule name="required" />
<rule name="maxLength" length="60" />
<rule name="email" />
</property>
<property name="favoriteNumber" type="integer" >
<rule name="maxLength" length="30" />
</property>
<!-- Complex rules -->
<rules>
<rule name="atLeastOnePopulated"
properties="phone,email"
hint="one or more properties must be populated"
/>
<rule name="AddressVerification"
runAt="server"
hint="Makes a web service call to the United States Postal Service to check if the address is valid"
/>

</rules>

</validationSet>

Assuming we had a validation engine that could parse the above XML and run the validations on an instance of the Customer.cfc we could simply call: validator.validate(myCustomerInstance) to perform validations.

Now for the client side validation. There needs to be a way to bind the client-side form to the various parts of the domain object so we can determine which validtions can be performed on the client side. To define the FORM validation for the client, we'd also do that declaratively.

<formValidations>
<form name="customerForm" validates="com.myapp.Customer" />
</formValidations>

By specifying the markup above you would basically be saying, "my form has all the fields included in the domain object for Customer and they are named the exact same, and all possible client-side validations for the Customer object should be applied to the form "someForm" "

Your HTML form would look something like:

<form action="..." onSubmit="validationEngine.validate(customerForm', this);" >
<input type="text" name="firstName" >
.....
<input type="text" name="favoriteNumer" >
</form>

And the coldfusion script processing the form submission could make a call like so to validate the form contents on the server-side as well:

validator.validate("customerForm", form) where "form" is the form scope or some other structure.

Now lets say you have "Enter Customer Record Wizard", you could potentially have something like:

<formValidations>
<form name="customerForm1" validates="com.myapp.Customer" properties="firstName,middleInitial,lastName" />
<form name="customerForm2" validates="com.myapp.Customer" properties="address,city,state,zip" />
</formValidations>

Your HTML for customerForm1 would look something like:

<form action="..." onSubmit="validationEngine.validate(customerForm1', this);" >
<input type="text" name="firstName" >
<input type="text" name="middleInitialName" >
<input type="text" name="lastName" >
</form>

When you submit the form, only the validations for the first, middle, and last names would get applied because the FORM validation has limited the validation rules to just the rules for those 3 properties.

Your HTML for customerForm2 would look something like:

<form action="..." onSubmit="validationEngine.validate(customerForm2', this);" >
<input type="text" name="address" >
<input type="text" name="city" >
<input type="text" name="state" >
<input type="text" name="zip" >
</form>

Now lets say you create a new CFC (SpecialCustomer.cfc) that extends your Cutomer.cfc. You wouldn't want to re-define all the validations for Customer.cfc for SpecialCustomer.cfc so the validation engine should be able to look at the validation definition below and perform introspection on the SpecialCustomer CFC to see that it extends the Customer.cfc and have it automatically merge the two validation sets together so that the SpecialCustomer CFC has all the same validations as the Customer.cfc and it's "somethingSpecial" validation as well.

<validationSet component="com.myapp.SpecialCustomer" >
<property name="somethingSpecial" >
<rule name="required" />
</property>
</validationSet>

There are TONS of ways that this could be enhanced and implemented. For instance, you could add mappings in the formValidations section to provide html field aliases to the business properties in case the form field names don't match up with the business object properties. Plus there could be ways to consolidate the validation definitions so they're not as bulky and are included in the actual metadata for a CFC so that it could be extracted via introspection. There could also be ways to create the XML via code generation of some sort, etc.... You could adapt this to allow for internationalization/customization of validation error messages....

This is all pretty rough, and I hope it helps get across the ideas that I have. Does anybody else have opinions on the above?

My approach is similar to what Kurt suggested, except instead of an XML file I configure the error checking rules directly in my database.

For my home baked Content Management System I've been slowly working on, I have a table called "t_entity_attributes" that lists all possible attributes that can be associated with the entities described within the CMS.

Example entities are "user", "image", "movie", "entry", "message", "item" etc.
Example attributes are "title", "description", "width", "firstname", "price" etc.

The main fields in the t_entity_attributes table are:
attribute_name, attribute_name_hashed, attribute_type, attribute_pattern, attribute_error_id

When a form is submitted to add/update an entity I basically do the following:

Get the instance of my form validation cfc (exists as a singleton in my application scope) and pass it the entity name and the FORM structure.

e.g. o_result = getInstance("validator").validateForm("resource",FORM)

Inside the validator I check each field to see if it represents a valid attribute:

query SELECT * t_entity_attributes WHERE attribute_name_hashed = '#hash(FORM.fieldname)#'
(I SELECT using the hashed value to prevent SQL inject)

if I get a match, I then do the validation checks
- attribute_name isValid for the entity
- fieldvalue isValid attribute_type (e.g. string, credit card number etc.)
- fieldvalue doesMatch attribute_pattern (e.g. a regular expression)

If there is an error/no match:
- I create an error object and pass it the attribute_error_id, or default error_id + fieldname, to get the correct error message from my messages table.
- The collection of error objects are returned as part of the result

e.g. o_result.errors

If there is no error:
- the attribute's value is added to an instance of the object that represents the entity:
e.g. o_entiy.addAttribute(attribute_name,attribute_value);

- if all the attributes validate correctly, I can pass the entity object to the next step in the process, whatever that may be (e.g. write to database, file, log screen, email etc)

Why did I do all of this?

This allows me to have an admin interface that lets me add new attributes (and in some cases entities) to the system without modifying my coldfusion code.

Though I still typically have to tweak the CSS and FORM HTML for the new fields.

@Ben

I've used the same datamodel for an inventory system, think of the entities as store items. with attributes such as colour, price, model, manufacturer, etc.

This approach has worked for me, BUT has a heavy reliance on components (entity, iterator, DAO) to actually manipulate the data, so you performance may vary.

I forgot to mention before, in association with the entity_attributes table, I have an entity_attributes_allowed_values table that is used to limit attributes to a predefined list of values e.g. allowed colours.

For the hash.

I'm just very wary of taking any string sent by a user and using it directly in an SQL statement (even when using CFQUERYPARAM), especially when I'm accepting the string through a page available to the general public, and when I plan to use the string as part of a lookup statement.

So my quick method in the past for protecting my database is hashing incoming strings used for lookups , e.g. user name (for log in), or in this case a field name, and having a hashed version of the string I'm looking for in my database to search against.

Since I'm now comparing two hashed strings in my SQL statement, and a hashed string can only be numbers and letters, the ability of hackers to exploit my code using SQL injects or some Coldfusion Bug I'm not aware of "should" be greatly decreased.

Again there are other ways to protect against these problems (cfqueryparam etc.) So I may drop this approach in the future.