Software Application Layers And Responsibilities
I know that I haven't done much blogging lately (due to work); but, I've been doing a ton of thinking about software application architecture. Much of this thought has been influenced by previos projects as well as the pain and subsequent refactoring of my current project. On top of that, the people who comment on this blog have been extremely influential in shaping the way that I am coming to understand application architecture. That said, as we close out 2012, I wanted to take a few minutes to outline my current feelings about how I see the application layers and what their respective responsibilities are.
Right now, I am thinking in terms of building software with the following layers:
- Controller Layer
- Application Layer
- Domain Layer
- Infrastructure Layer
Before I get into the details of each layer, let me make some broad sweeping philosophical statements:
- Each of the above layers depends exclusively on the layer immediately below it.
- Outside of entities (ex. Linked Lists), circular references are to be avoided. The more I think about it, the more I tend to feel that circular references are a sign that I am missing orchestration at a higher level of the application.
That said, let's look at each layer. Because I am still forming all of these thoughts in my head, I'm going to write mostly in bullet-points. I don't think my understanding is refined enough to write technical prose.
Controller Layer
- It is the public face of the application.
- It routes incoming requests and returns responses.
- Both incoming and outgoing data is restricted to simple data types (ex. structs, arrays, numbers, strings, etc.). This could be JSON, HTML, XML, binary, AMF, etc..
- It performs high-level security and authentication. Since different requests may be using different forms of identification (ex. cookies, HTTP Basic Authentication, oAuth, SAML), it is the Controller's responsibility to orchestrate the transformation of request data into authentication data.
- The Controller layer may invoke the application layer to help with authentication. However, if requirements are simple enough (ex. Basic Authentication for 3rd-party Web Hooks), the controller layer may perform its own authentication.
- It is the only part of the entire application that knows about session management.
- It is very thin, using the Application layer to perform most of the work.
Application Layer
- It is the programmatic boundary of the application.
- It accepts and returns simple data structures. However, unlike the Controller layer, it probably shouldn't deal with data like JSON or HTML. Rather, it deals with non-serialized data structures and relies on the Controller to serialize and deserialize data representations.
- It performs low-level security and checks access permissions. In the case of a request made by a user, the application would be responsible for checking whether or not the requesting user has the right to perform the given action (ex. can this User delete that Publication).
- All methods in the application layer that can be invoked by a user's request should require the authenticated user ID as the first argument.
- It orchestrates Transactions management.
- It orchestrates workflow, but defers to the Domain Layer to carry out each step of the workflow.
- It is moderately thin and defers to the domain layer for the bulk of its logic.
Domain Layer
- It contains the "business logic" of the application.
- It maintains data integrity.
NOTE: Since I don't currently deal with "domain entities," my thoughts on the domain layer are, by far, the least evolved. I typically deal with domain services and raw data structures.
Infrastructure Layer
- It provides adaptors for technologies such as persistence, file systems, email, Twitter, Twilio, Pusher, etc..
- It deals with simple data structures (ex. structs, arrays, numbers, strings, binary, etc.).
So, right now, as of Dec 21, 2012 at 5:05 PM, this is how I see application architecture. I am hyper-specific about the time since my understanding and thoughts about software application architecture are constantly evolving. I've come a long way in 2012 (I think); I hope that 2013 brings new levels of enlightenment and understanding!
Reader Comments
This reminds me of our senior CSI project. We had a REST API that was a thin layer on top of a single facade. That way we could have implemented a SOAP API as well. External things like web services and database calls were abstracted out into Java interfaces. This was nice when we had to replace one of the 3rd-party web services with a different one and when we needed to mock up a web service for load testing. Our project was fairly simple compared to some real-world applications, but it was nice to have everything organized, and all layers making calls only "downward" in the stack.
Ben, great work! This is an excellent summary, and I think this architecture should work well for many applications.
As you noted, security can be a bit of a cross-cutting concern, depending in part on the granularity of permissions that are required.
You might not need to make it a universal rule to require each user-invoked method to specify the user ID as the first argument. Depending on the particular use case, there could possibly be some operations that wouldn't actually make use of it. But if that's what you end up with on your app, that's cool.
Again, great summary!
I always enjoy reading your thoughts, Ben. You're eager to challenge your current thinking, which I find a bit rarer than I'd like. I really am with you on having a thin Controller layer, and view it as more of a traffic cop role than anything else.
Regarding dependencies, I have found that to be a longer learning curve for me than in other areas. I started seeing some light at the end of the tunnel when I began using dependency injection (DI). Perhaps you've already explored all this stuf, but... In ColdFusion, DI/1, the sister to FW/1, is a great example of how this can be done cleanly and without much fuss. The ColdBox project also has a very full featured dependency injection framework. You'll find that the two projects come from different mindsets. I like them both, depending on the project. One thing you will need to consider with ColdBox, though, is a relatively steep learning curve.
Another thing that accelerated my view of (web) application architecture was exploring various frameworks in various languages. Building non-trivial applications in a variety of frameworks gets you well informed and you'll see the same patterns approached with different points of view.
When dealing with frameworks in general, I've encountered two main groups of people: Adherents to micro frameworks that provide a bare implementation of some design pattern(s), and full-stack fans who prefer the kitchen sink approach. I kind of sit in the middle, since I've come across both styles that I either enjoyed or didn't, for myriad reasons. If you're working in the ColdFusion space, I think one is well served exploring either / both FW/1 and ColdBox.
Anyway, back on point. Again, I really encourage a look at DI and how it can help you solve those cross cutting concerns. For instance, your security layer could be something that is injected where needed, and depending on environment tweaked so that the context sensitive security checks you mentioned are done, or perhaps user emulation is possible on a certain environment but not another. Logging is another classic cross cutting concern that DI solves with aplomb. For example, log verbosely in development, but not production, depending on log level, etc.
I could be wasting my fingers though... you may be coming at this from a JavaScript oriented view!
Oh, the other thing I think that bears mention from a dependency perspective is this: Strongly consider how coupled your service / data persistence layer is to anything else. Ideally, it should be able to be propped up, tested, and survive on its own accord without anything else touching it. Certain ORM implementations will lead you down a dark, shadowy path of inbuilt framework dependency.
@Xdhmoore,
I recently had to rewrite an API that used to live as a parallel site that connected to a database. When I built the site originally, it took me like 3 weeks to put the API site together, duplicating all of the business logic from the main site.
Then, going back and gutting the API and connecting in the domain layer that was used by the main site, I got the API site up and running in a single day. That was a pretty amazing moment for me - it really drove home how powerful it is to have a well bounded application core that multiple sites can hook into.
I'm sure, even with that instance, I'm not dong a great job of it (I have some odd sharing of components); but, even there, it's clear to see how powerful a good, layered architecture can be.
@Dave,
My current app makes heavy use of user-based permissions for all kinds of actions within the application logic. Some users can perform XYZ, but only the owner of a particular entity can perform action ABC (and things like that). Then, it depends on the type of subscription a given user has (and if that user is an "owner" or simply a "member").
That said, in hindsight (given what you said), I can see that that is very tightly coupled to my application's requirements. If I had an app that was limited to logged-in/logged-out requirements, I wouldn't see any need to pass the userID around.
That said - I love that turn of phrase too much :) - if you *did* pass in the userID to user-based methods, it would allow the application logic to evolve (ex. adding user logging or request limits) without the Controller having to change too much. Of course, you don't necessarily want to plan for problems you don't have yet.
So, all to say, I agree with what you're saying.
Interesting read Ben,
I can draw similarities to the structure we have in TeamworkPM; we have:
Main app = controller
Api = controller
Twcore = all business logic = application/domain
Util = infrastructure
We also have TwData = shared memcached query lookups
Thanks for the food for thought as always. Have a great Xmas.
Peter
@Kerr,
I've recently started using DI/1 as my first "official" dependency injection framework. Up until now, I've bootstrapped my application my explicitly instantiating (and caching) my ColdFusion component with constructor arguments. So, basically doing this:
And, since I never had many components, this was actually fairly easy and let to a great deal of flexibility in how I created and cached components. Furthermore, I never really felt the "pain" that so many people talk about when it comes to organizing their components (I think people exaggerate pain in coding).
That said, I have really come to enjoy the way I can simply add "property" tags to the top of my components and they just work. Of course, this enjoyment is disrupted when I have a component that requires some constructor arg like "username" or "apiKey" and suddenly I have to jump through a bunch of hoops to get that working in a DI framework.
So, for me, it's nice, but it seems like 6 of one, half a dozen of the other.
As far as the dependency of the Model on the Database, it currently goes through a Data Gateway [set of] components. I don't really deal with "domain entities" currently; my model layer is really more of a collection of domain services than anything else. I'm hoping to learn more about entities and how they behave in a good architecture.
@Topper,
Definitely sound very similar. The shared caching layer sound very interesting also!
Have a wonderful Christmas and New Yar!
@Ben,
That's great you're using DI/1. I imagine you've come across the injectProperties factory method for dealing with bean arguments. It can be interesting making sure that argument names don't conflict when you do lots of setter injection though. Still, not being married to com.x.y.z or some other pre-ordained directory structure is a big payoff for me. One could do dynamic mappings, but that IMO is a very slightly different solution.
I guess I'd rather deal with the argument side effects than physical directory issues. Plus, if you need to alter contextual behavior (pass in an entirely different object), that is a snap with DI. My experience is that it's been cumbersome without it.
As far as model dependencies go, I was writing from a framework<->model dependency point of view. There are full stack frameworks that require you to fire up the framework itself just to get at your services. If you're not there, and it sounds like you're not, it's all gravy!
@Kerr,
This is my first project with DI/1, so I'm not fully familiar with the various methods. For things that are more 3rd-party components (like a Twilio / Twitter gateway), I'll typically just instantiate it with the various API keys that it needs and then just stuff it into the beanFactory as a constant or named-bean.
3rd-party stuff, however, is the minority of our uses-cases in the app, so mostly, we get to just wire stuff up simply with "property" tags.
Thanks Ben. Your architecture is very similar to what I ended up with. I ended up building my own framework in order to learn about these issues. However, I do use Taffy for my REST controller (just to parse and return the web request).
user web request --> Taffy(REST request parsing) --> Security (injection detection, user authentication) --> Controller --> ClientFormatting (for return only) --> Application --> Domain.
The reason for the ClientFormatting layer is that I typically use 100% JS for the client side (typically ExtJS / Sench Touch) and CF for the server side. Now, ExtJS has components like forms, grids, trees etc and can consume json, XML, etc. However, since a form requires a single "object" while a grid is an array of objects and a tree is nested data the construction of the (say) json will differ depending upon what component is asking for the data. And, metadata required by the component (eg recordcounts for grids, validation data from the server side for forms, etc) also has a required format / layout that depends upon the component asking for the data.
Now, given that a server side "entity" may potentially have it's data represented on the client side as a form, grid or tree, but it should not know that, my ClientFormatting layer takes the CF data structure returned from the Application layer and rearranges it to produce the kind of json layout and metadata that the specific component needs. The client web request contains a parameter like (forForm, forGrid, forTree) and the ClientFormatting layer is the only layer that uses those parameters.
I set it up so that the ClientFormatting can also handle other server side languages as "plugins". So the server side then effectively "speaks" Ext json for a form, or Ext XML for a tree, or {your server side library} for {your choice of data format} for {your choice of client side component}. So, the ClientFormatting layer is all that needs to change if I start using a new client side library, or if I want the same server app to provide data as a published "API" with different data types (json, xml, etc).
While I havent done it yet, I intend to add RPC capability at some stage too.
So, to sum up, the server can then "speak" multiple languages and "dialects" without touching the Controller or Application layers. The REST request determines what action to take and what data is requested (via the Controller), while the ClientFormatting layer controls the format of the data being passed back to the Controller depending upon "who" the client side library is.
Thanks for the discussion,
Murray
@Murray,
Formatting the response for the client is definitely something that I am "hacking" right now. We jump through a number of hoops to strip out unsafe data (ex. passwords, secret keys) and making sure the case of keys translates well from ColdFusion (non-case sensitive) to JavaScript (case sensitive) and that data types make sense (ex. converting CF date/time stamps to client-side UTC milliseconds).
All to say, that's one of the weakest parts of my app. But, part of that is based on the shoe-horning of a single model into both the Command AND Query aspects of the application (as you know from our other discussion).
Uggg, so much research to do... no enough time!
Often my controller layer manages transaction boundaries, so I think transaction management is the responsibility of both the controller and application layers.
I tend to keep my domain layer fairly light. My domain classes have methods relevant to those, but functionality that requires coordinating multiple entities usually ends up in my application layer.
Many of my choices are influenced by how course-grained or fine-grained I want my application layer API to be. I often avoid adding course-grained methods to the application layer by allowing the controller layer to manage transactions across multiple calls to the application layer.
@Eric,
To be completely philosophical for a second, one thing I don't like about putting my transaction in the Controller layer is that it requires the controller to know about the type of storage that is being used. If you keep the transaction control in the Application layer, then all the Controller has to know about is the data that can go into and come out of the application - it doesn't know anything else.
That said, to get off my philosophical soap box :) in 12 years of web development, I've only ever changed my data storage layer twice.
I tried to continue this thought experiment with a specific use-case in mind. I've found myself getting a bit lost and concerned with "necessary" duplication of effort.
www.bennadel.com/blog/2454-Exploring-Sample-Software-Application-Layers-And-Responsibilities.htm
It's just some pseudo code. I'd love any feedback that people might have. Thanks!!!