Thinking About Data Gateways, Collections, And Raising Exceptions
Since my talks with Steven Neiland at cf.Objective(), I've been trying to think very deeply about Model-View-Controller (MVC) architecture. To move me along in my understanding, I've been slowly trying to put together a small MVC example in ColdFusion. It's been a tough and humbling process as it is really forcing me to think differently about the way I approach problems. My latest hurdle, and possible unnecessary tangent, has been thinking about Data Gateways and raising exceptions. I've lived in the world of Query objects and RecordCount properties for so long that I feel ill-equipped to think about how and when to raise "data access exceptions".
Because my current approach to application architecture doesn't create a separation between business logic and data access, I've never really needed to raise exceptions or throw errors around data manipulation. With my currently-evolving approach to MVC (Model-View-Controller), however, I am trying to draw a hard line between my business logic and my data access layer. This separation of concerns forces me to ask questions about what to do when data access doesn't go according to plan:
- What happens if I go to retrieve a single record, but none exists?
- What happens if I go to update a single record, but none exists?
- What happens if I go to delete a single record, but none exists?
In the world of Database Queries, I would simply let the SQL execute; and, if no rows were affected, it didn't matter - I would let the page process (typically using CFLocation) as if nothing unexpected had occurred. With a data access layer, however, I should think that more caution need be taken - if I go to affect a row and it doesn't exist, something should be done about it.
For the moment, here are the rules that I have constructed in my head:
- Any data access request that doesn't explicitly limit records based on an ID (primary key) should return a collection. This collection may be empty if no matching records are found.
- Any data access request that explicitly limits records based on an ID (primary key) should return a single object.
- Any data access request for a single object based on ID (ex. Read, Update, Delete) should raise a "DataAccessException" if the target record cannot be found.
In my research, I found a number of approaches that return True/False or {ID}/-1 upon unsuccessful data access requests. This doesn't sit well with me, however, because it doesn't seem to allow for good transactional behavior. With an exception-based approach to data access, you can easily halt a workflow and rollback a transaction that cuts across multiple data gateways.
As I was thinking about this, the kind of method that kept tripping me up was the following:
- getAccountWithCredentials( email, password )
From a business-logic standpoint, this query should return a single record. That is, my business rules dictate that no two accounts should have the same credentials; however, from a query standpoint, there's nothing in my data cache that enforces uniqueness on the Email and Password properties. As such, according to my rules above (no limit by ID), this method should return a collection, not an entity; and, if no records were found, the returned collection should be empty.
Thinking about this stuff is rather stressful; but, it's super fun at the same time! I'm really trying to think about my application architecture in a totally different way. I don't yet have a clear understanding of what scope a Service object or a Gateway object should have (in terms of its behavior); but, as I put together my demo, hopefully this information will come to light. If anyone has any thoughts or feedback, I'd love to hear them.
Reader Comments
"Any data access request for a single object based on ID (ex. Read, Update, Delete) should raise a "DataAccessException" if the target record cannot be found."
I really have to disagree. To me, a DataAccessException would indicate that there's a problem with the query itself- incorrect syntax, database connection error, something like that. As far as the data layer is concerned, if it successfully ran the query, it's job was successful.
I could possibly see doing this on an Update, but only if the operation *had* to be an update, i.e. You couldn't set up the data layer to do an Insert of there was no record to Update.
As far as Reads that are supposed to return a single object, I still wouldn't throw an exception. In C#, I'd return a null. Since CF still (I assume) doesn't have the concept of a null, I'd probably return an empty object.
I get what you're saying about transactions that span data objects, but I think you're putting your "control mechanism" in the wrong layer. If you have a transaction defined in a layer that sits just above the data layer, then that layer should be responsible for determining whether to continue with the transaction after an operation, or whether to roll back.
@Matt,
Definitely the fact that ColdFusion doesn't have a great mechanism for "Null" weighed into this process. My first thought was just to return the object or NULL if it couldn't be found. But, handling NULL objects in ColdFusion feels like a bit of a burden.
As far as Transactions are concerned, I am not saying that returning False, or Empty objects, cannot help the transactional layer; I'm just saying that an exception could make it easier since the transactional layer doesn't have to check *each* data access result - rather, it simply has to "try" all of them and if any of them fail, they all fail.
As far as returning empty objects, that is basically what I do in my current code; except, instead of returning objects, I return empty query objects (ie. recordCount is zero). Then, I check to see if any records were returned before I figure out how to process the rest of the page. So, in that regard, an empty object return is definitely something I am "used to."
I tried to look at the way ColdFusion ORM handles this kind of stuff. In ORM, at least in the ColdFusion world, if you execute a Load request that expects a single object:
EntityLoadByPK( "entityName", id )
... and none is found, I *believe* that ColdFusion will raise an error. Furthermore, if you tell ORM that it should only expect one return value:
EntityLoad( "entityName", id, isUnique=true )
... and more than one object is returned, ColdFusion will raise an error.
Based on that, I figured that any "unexpected" result for single objects should raise an error. But, as I said, I'm just thinking this all through for the first time. And it hurts my brain :D
"But, handling NULL objects in ColdFusion feels like a bit of a burden."
Yeah- it did to me, too. To the point where I started building null.cfc and then realizing that I was overreacting. :)
"I'm just saying that an exception could make it easier since the transactional layer doesn't have to check *each* data access result - rather, it simply has to "try" all of them and if any of them fail, they all fail."
True. However, when you start building your data layer with the idea of making known behavior in your service layer easier, you start limiting your data layer. In this case, if you wanted to check to see if a certain record is present, you'd have to use a Try/Catch and either have an empty Catch or have your Try/Catch handle branching your code. Something like (and this is the first time I've tried the code tag):
Whereas, if your data layer just retrieves whatever data meets the criteria its given, you have the much more manageable
All of your business logic is in the proper layer. Your data layer is like your dog. Just fetch the stick.
@Ben,
Another thing to think about is to break down your service layer methods into methods that are used to serve data to the controller/views and those that are used to facilitate business logic.
Taking your delete blog example for a moment. In your service layer you would have deleteBlog service method and deleteAllComments service methods.
Each one would handle security access before delegating to the gateway. This is where I personally would put a cfthrow call.
Typo, the last function should return the deleteSuccess variable.
@Matt,
I can definitely see the benefits of using Null return values when you want to GET an object. Where I start to get cautious is if I go to UPDATE an a record. What happens when I go to update a record that doesn't exist? Do I return a new, empty object?
There's something about raising an error across all single-object actions that feels nicely uniform.
@Steven,
The security in the Service layer makes sense. But, just so we're on the same page, would you consider "logged-in" aspects of security a Controller responsibility? Right now, I'm trying to keep session references out of the Service layer (at least on the first pass). Therefore, I would consider the session-based security part of the controller.
For example:
This way, "web access" security is part of the controller; but, "business security" is part of the Service layer.
Really there are two aspects to security for mvc.
#1 access which is concerned with displaying controllers etc
#2 function which is concerned with running service methods.
I put the access aspect in the controller layer (which is what you describe).
How I build my app is something like this.
My first step in the controller request logic is to check if the user is authorized to get at a view. I do this globally using the setupRequest controller method defined by FW/1 in application.cfc (which is basically a version of onRequest).
Then if the user can use a controller/view I selectively turn on/off elements to display from the controller.
Finally when a service layer method is called I confirm function permission.
That way I have double protection.
#1 At the controller I am confirming if a user can see a particular view and what parts of it eg "Delete Comment" button
#2 Then if the delete comment function is called I am confirming they can run the Function from within the service layer.
Both ways though I am running through the same securityService component which handles the session object.
For simplicity sake I actually proxy the service layer functions I need out at the controller layer as udfs.
@Ben,
Just re-read your comment and yes that's exactly how I deal with it. The only modification is again I call the security service from the controller to do the check of session variable as opposed to directly referencing it in the controller code.
@Steven,
Right, cause I remember you saying you abstract your security check in some authorization method (which references Session).
I totally forgot that some features of an actual View wouldn't be available (ex. Delete button). To be honest, it's been a while since I had a View that was seen by more than one "type" of user (just hasn't been necessary in the software I've built recently).
I suppose to deal with that you could either determine the security at the Controller level (example):
showAdminFeatures = security.isAdmin( user )
... and then pass the Boolean flag into the view/view data.
Or, you could have the View perform the security check. The former feels "better" because it keeps the View dumber. But, the ladder feels much less repetitive (ie. action doesn't have to be performed by every Controller).
@Ben,
Yes it is a bit more repetitive to do the call in the controller than the view. But remember like you said, the individual controller component methods are only part of the overall controller system.
FW/1 for example defines a before method which you can define repetitive actions in. So you could end up with a controller file that looks like this.
Taking this further if you want to just do this check at every page request you can leverage onRequestStart (FW/1 defines a custom onRequestStart type method called "setupRequest" in the application.cfc file).
Finally its important to remember that one of the benefits of having a controller is that you can switch out the default view with a different one for different user types. This is particularly useful if the layout is significantly different.
In FW/1 for example you could have two views for an article. One for normal users which would be defined in article.cfm and an administrator specific view defined as articleAdmin.cfm. Both would pull the same data and have the same url, but the controller would switch between the two like this.
<Strong>General Thoughts
Thanks Ben that is an interesting idea to throw an exception when the result set is not what the method intended. It make me think that it is a good thing on the update or delete functions...
But the retrieveByKey I have mixed feelings on. On the one hand when I'm doing a save in my service layer, retrieveByKey throwing an error would be good for the transaction, but when my service is just trying to look up if it exists, then it would be a unneeded exception.
*debating in head* - I'm really starting to think this may be a good idea, especially since we also throw exceptions when "validation" occurs.
<Strong>Another Gateway idea
And Not to throw another idea into the mix, but myself and my-coworkers just adopted a slight change to what seems to be normal in CF community.
Our DAO (data gateways) functions only ever return a query(retrieve,etc) or void (insert/update/delete).
And then within our Service layer we'll convert it to an DTO or an array/structure of DTOs depending on how many rows are expected.
I know it doesn't remove the dependency on the DTO from the DAO, but it does remove the need for the DAO to know how to create a DTO.
Security Layer
I haven't used FW/1 yet, so I don't know if this exists. But I am a heavy user of ColdBox and it has introduced AOP into the framework, which allows you to remove the Security out of your services based on design, of course this takes even more design upfront before just coding. I will say I'm an expert, so I'll refer anyone reading to the ColdBox site http://wiki.coldbox.org/wiki/WireBox-AOP.cfm
I understand your head hurts. I've been doing some sort of MVC for almost 10 years now (within CF), and until lately with the frameworks, ColdBox and FW/1, I struggled with why I was doing it that way. Now I see the power and I also see my applications getting more "complicated" or complete compared to in the past. I truly believe that is the case because of the frameworks adding value to my applications.
Great blog entry, Ben. It's nice to know that I'm not the only one whose head hurts after thinking so much.
@Steven,
That makes a lot of sense. Since I still put all of my "Controller" logic into CFM pages with CFSwitch statements, my before/after is basically just top/bottom of sub-index file :) The method-approach makes a lot of sense to me.
@Craig,
Dealing with Queries just makes life really easy, doesn't it?! Especially the fact that you can return a zero-row query and calls to columns doesn't raise an exception. As I've been noodling on all this OO/MVC stuff, it definitely makes me appreciate how sweet we've had it in ColdFusion with the query object for all these years!
@Evik,
Slowly, I'm having moments where its starting to make some sense... if only for a brief moment :)
Ben,
Dealing with Queries in my mind is just like an Array of DTOs! There are just some extra advantages with actually using a DTO rather than a Query. That is the reason I use both, just a different layers of the OO/MVC framework.