Writing My First Unit Tests With MXUnit And ColdFusion
Last week, I wrote my first ever unit tests [of any kind] with Jasmine and RequireJS. It was pretty exciting! And, now that I've played around a bit on the client-side, I thought it was time to do some experimentation on the server-side. This morning, I ran my first server-side unit tests with MXUnit and ColdFusion.
The setup for MXUnit was a bit more complicated than it was for Jasmine and RequireJS. Things on the client-side are easier because everything has to be web-accessible and addressable by relative file-paths (since the client is loading everything remotely). On the server, however, only the test runner has to be web-accessible. This allows for greater variation in how files can be, or are intended to be organized.
Since I am very new to all of this unit testing and TDD (Test-Driven Development) stuff, I had to come up with a strong rule about putting this code together:
I want my testing library to be in the same code repository as the app it is testing. This way, if anything needs to be adjusted (ex. monkey-patching, framework updates) for one app, there are no unwanted side-effects showing up in unrelated applications.
Luckily, with the introduction of ColdFusion 8, we can now set up per-application mappings. This means that each application can have its own definition of the "mxunit" class-path. This allows me to keep the MXUnit framework inside the "Tests" directory of the application I am testing:
- / app /
- / app / model /
- / app / model / RomanNumeralEncoder.cfc
- / app / tests /
- / app / tests / mxunit / [mxunit framework]
- / app / tests / server /
- / app / tests / server / spec /
- / app / tests / server / spec / RomanNumeralEncoderTest.cfc
- / app / tests / server / Application.cfc
- / app / tests / server / runner.cfm
- / app / Application.cfc
In the above directory structure, notice that the "tests/server" directory has its own Application.cfc. This is because our unit tests do not want to be part of the root application. By isolating the tests within their own, light-weight application, we are able to load only the code that is being tested. As Robert Martin (Uncle Bob) would say, your application should be something that is "plugged-in" to your Domain Model.
The test-level Application.cfc serves to both isolate and configure the testing environment. In addition to mapping the "mxunit" class-path, it also has to map the root-level Model directory and the test-level "spec" directory so that the tests can be loaded and executed from within the MXUnit framework.
Application.cfc - Our Testing Silo
<cfcomponent
output="false"
hint="I define the unit-testing application settings.">
<!--- Define the application settings. --->
<cfset this.name = hash( getCurrentTemplatePath() ) />
<cfset this.applicationTimeout = createTimeSpan( 0, 0, 5, 0 ) />
<!---
I am explicitly turning off session management for the MXUnit
testing since I don't believe I want this level of testing
for my "units." Sessions are managed in the Controller, not
in my Model.
--->
<cfset this.sessionManagement = false />
<!---
Get the tests directory - we'll be setting up mappings for
directories relative to our testing directory.
--->
<cfset this.directory = getDirectoryFromPath( getCurrentTemplatePath() ) />
<!---
Get our MXUnit directory - we'll need this in order to run
the test suties from our tests directory.
--->
<cfset this.mxunitDirectory = (this.directory & "../mxunit/") />
<!---
Get the application-root directory (where our non-MXUnit,
application code exists).
--->
<cfset this.appDirectory = (this.directory & "../../") />
<!---
***** MX UNIT FRAMEWORK *****
Set up a mapping to the MXUnit framework; this is requied for
the framework installation to run without a global mapping.
--->
<cfset this.mappings[ "/mxunit" ] = this.mxunitDirectory />
<!---
***** TEST SPECIFICATIONS *****
Map the spec folder so that our local tests can be located
from within the MXUnit framework components.
--->
<cfset this.mappings[ "/spec" ] = (this.directory & "spec/") />
<!---
***** APPLICATION COMPONENTS *****
Map the Model directory so we can include our application's
components for unit testing.
--->
<cfset this.mappings[ "/model" ] = (this.appDirectory & "model/") />
</cfcomponent>
Notice that the application sets up app-specific mappings for:
- /mxunit
- /model
- /spec
These mappings allow our test specifications (ie. Specs) and the MXUnit framework to load components that are not in directly-accessible folders.
Also notice that our testing application has no session management enabled. While our root application may use sessions, nothing in our domain model will need to know about it. Enforcing this at the test level will allow for more isolated testing; and, it will force us to keep the responsibilities of our application separate.
In the Spec folder, I created one test specification for our RomanNumeralEncoder.cfc component (not shown). When pointed at a test specification, MXUnit will automatically execute all methods that either start with or end with the term, "Test".
RomanNumeralEncoderTest.cfc - Our MXUnit Test Specification
<cfcomponent
extends="mxunit.framework.TestCase"
output="false"
hint="I test the Roman Numeral encoder.">
<cffunction
name="testDecode"
hint="I test the decode() method.">
<!--- Create an instance of our roman numeral encoder. --->
<cfset var encoder = createObject( "component", "model.RomanNumeralEncoder" ).init() />
<!--- Test encoding. --->
<cfset assertEquals( "I", encoder.encode( 1 ), "Encoding 1 Failed" ) />
<cfset assertEquals( "II", encoder.encode( 2 ), "Encoding 2 Failed" ) />
<cfset assertEquals( "III", encoder.encode( 3 ), "Encoding 3 Failed" ) />
<cfset assertEquals( "IV", encoder.encode( 4 ), "Encoding 4 Failed" ) />
<cfset assertEquals( "V", encoder.encode( 5 ), "Encoding 5 Failed" ) />
<cfset assertEquals( "VI", encoder.encode( 6 ), "Encoding 6 Failed" ) />
<cfset assertEquals( "VII", encoder.encode( 7 ), "Encoding 7 Failed" ) />
<cfset assertEquals( "VIII", encoder.encode( 8 ), "Encoding 8 Failed" ) />
<cfset assertEquals( "IX", encoder.encode( 9 ), "Encoding 9 Failed" ) />
<cfset assertEquals( "X", encoder.encode( 10 ), "Encoding 10 Failed" ) />
</cffunction>
<cffunction
name="testEncode"
hint="I test the encode() method.">
<!--- Create an instance of our roman numeral encoder. --->
<cfset var encoder = createObject( "component", "model.RomanNumeralEncoder" ).init() />
<!--- Test encoding. --->
<cfset assertEquals( "1", encoder.decode( "I" ), "Decoding I Failed" ) />
<cfset assertEquals( "2", encoder.decode( "II" ), "Decoding II Failed" ) />
<cfset assertEquals( "3", encoder.decode( "III" ), "Decoding III Failed" ) />
<cfset assertEquals( "4", encoder.decode( "IV" ), "Decoding IV Failed" ) />
<cfset assertEquals( "5", encoder.decode( "V" ), "Decoding V Failed" ) />
<cfset assertEquals( "6", encoder.decode( "VI" ), "Decoding VI Failed" ) />
<cfset assertEquals( "7", encoder.decode( "VII" ), "Decoding VII Failed" ) />
<cfset assertEquals( "8", encoder.decode( "VIII" ), "Decoding VIII Failed" ) />
<cfset assertEquals( "9", encoder.decode( "IX" ), "Decoding IX Failed" ) />
<cfset assertEquals( "10", encoder.decode( "X" ), "Decoding X Failed" ) />
</cffunction>
</cfcomponent>
Here, I am testing the encode() and decode() methods of the RomanNumeralEncoder.cfc component (not shown).
Once I had my application and my test specification in place, I created a test runner to load and execute the given spec:
runner.cfm - Our MXUnit Test Runner
<!--- Set up our test suite. --->
<cfset testSuite = createObject( "component", "mxunit.framework.TestSuite" ).TestSuite() />
<!--- Add the roman numeral spec (ie. test specification). --->
<cfset testSuite.addAll( "spec.RomanNumeralEncoderTest" ) />
<!---
Run the tests that have been added. This will include all the
methods of the all the components that we added above (in this
case, it will be the encode() and decode() methods of the
RomanNumeralEncoderTest.cfc).
--->
<cfset results = testSuite.run() />
<!---
Output the results. Pass in the web-root of the MXUnit
folder so that the rendering can properly set up the CSS
and JavaScript URLs.
--->
<cfoutput>
#results.getHtmlResults( "../mxunit/" )#
</cfoutput>
When we create the test suite, we simply give it the class-paths to our test specifications. MXUnit will automatically instantiate the test specifications and run the appropriate test methods. The HTML result for the above test looks like this:
As you can see, both of my test methods passed! For some reason, though, none of the links on this page work. They all try to use "/spec" as their base URL; and since "spec" is not a top-level sub-directory of my web-root, these links all break.
I am sure that some of the decisions that I made are not that great; but, at least I built and executed a working unit test on the server! This is very exciting! I need to figure out how application directories are organized to include tests. I know that they should be part of the same code repository (and part of source-control); but, I also know that they probably shouldn't be pushed to production. So, I just need to see how this all fits together for the project life-cycle.
Want to use code from this post? Check out the license.
Reader Comments
This is a great posting. I'm trying to get myself more knowledgable about MXUnit and TDD for Coldfusion. It's been too long in coming. I was originally exposed to TDD on a Rails course last year, and I'm slowly getting more comfortable with MXUnit. There are still some things that befuddle me, but I'll work them out. Thanks for the example of having MXUnit under the tests folder in the project - that answered one of my questions.
@Jason,
Granted, I'm very new to this stuff; but, my gut is telling me that I want all test-related material in the "Tests" folder at the top level. Of course, that is, in part, influenced by the fact that I don't currently have a "build" process. If my source code and my "app code" was in two different places due to a build process, the organization of files might be different.
But for now, this is what I'm working with.
I hate to nitpick here as someone who themselves are just getting into TDD and unit testing, but I was told that each function should have one and only one test in it. Meaning your testDecode function would really be something like testDecodeNoNumber, testDecodeOneNumber, testDecodeMultipleNumbers or something like that, not 10 asserts inside of one function. Again, that's the way it was explained to me in developing tests.
@Phil,
That could make sense; I really don't know. I assumed that as long as the tests indicate a failure, it wouldn't / shouldn't matter how many are grouped together.
I would think that the main difference would be whether or not state is important; and, if the target component(s) need to be reinitialize in the "before" method calls or something?
But really, I have no idea :)
@Ben,
That's my understanding. Let me try and pull up an example and code from the training. i was using the before function for all initializations of the component I was testing.
@Phil:
I started originally doing one method for each individual test, but I found it ended up just taking way too much of my time and just ended up making for really bloated Test.cfcs.
I found as long as you make sure to include a unique description for each error (so it's easy to find your error) it made my tests much easier to manage and navigate, if I essentially did one test for each method or test behavior I was doing.
In a nutshell, this means I'm structuring my tests in a similar fashion to the way Ben did.
I'm sure someone can give argument to why you should isolate each test, but that's just my experience.
Ben-
Why haven't you written any unit tests before? I'm sure you knew they existed...?
The next step is for you to integrate unit tests with your build process. I haven't done this yet with CFML, but I plan on doing so soon using ANT:
http://wiki.mxunit.org/display/default/Ant+Task+Doc
Good luck with your TDD!
@Dan,
Oh, I questioned that during my training and am definitely taking a wait and see approach. The instructors did seem to follow a pattern of "nothing passed in" which you check an error type to which the code should respond, pass in a value but blank, then pass in whatever you really were expecting.
@Phil:
There's are definitely tests that only do one thing. For example, you have to write your exception tests like that.
I just started finding I wrote a lot less code if I just create an array of tests, w/expected results. Because I'm often doing multiple assertions test as well. I might be checking the length of results, then actually checking a result set, etc. I find my Gateway/Service tests end up testing multiple conditions and splitting everything into a bunch of small methods then just becomes even more troublesome.
@Dan, @Phil,
I think I saw that MXUnit also had a way to test private methods. Jasmine, similarly (I think) had a way to create function "spies" to make sure that subsequent methods get called. Do you think this is necessary? At least, not generally? To me, it seems that I should be testing two things:
1) Public API
2) Change of state (as a result of API calls)
I'm not saying that there's no benefit to testing private methods; but, it seems like it may be redundant (at first) - if you can check state and change it, then it seems that only calling a public API would be necessary.
Of course, if you run into edge cases in which it becomes necessary, then you can add them yet. But at least **initially**, I think the (2) items above make the most sense.
@Russ,
Excellent question. There are two factors at play. The first, most obvious is just a lack of appreciation for them. Either I don't have time; which is really another way of saying I won't *make* time to learn them - they just weren't at the top of my priority list.
The second, sadder factor was simply that so much of my code is procedural; and, the bits that were in CFCs mostly just read and wrote to the database - there's wasn't much business logic actually stored in "units" that I could easily test.
I've been using ColdFusion components for years; but, mostly in "gateway" capacities with the bulk of my "work" being done in the Controller, which has always been in CFM files for me. Now that I am reinvigorated in my learning of MVC and OOP, I'm just trying to bone up on the relevant stuff.
ANT is a whole other beast for me. Right now, I don't have much to automate yet. Maybe once I get more LESS and SASS and what not, then I'll start to have source vs build directories. But, I don't really know much about that stuff yet.
Ben,
If you want to learn more about other tools and languages, I highly recommend finding open source projects that use them and trying to contribute. I've spent the past year diversifying my skill set by making contributions (however small) to lots of different github projects in languages that I don't already know. It is challenging, but very rewarding. (I have to credit the book Seven Languages in Seven weeks for the inspiration.)
Hey Ben,
don't forget about he MXUnit google group...the developers that monitor the group are awesome and have helped me a lot with the tests I have tried to create...the code I have to test is crazy so I have needed quite a lot of help, and so far I don't think there has been anything they could not help.
@Russ,
If you are interested in automating your unit tests I highly recommend Mike Henke's "Cloudy With A Chance Of Tests" build file.
It covers virtually everything you could want including
* java compilation check
* mxunit testing
* selenium testing
* js listing
* css linting
* queryparam checker
* var scope checker
the list keeps going.
https://github.com/mhenke/Cloudy-With-A-Chance-Of-Tests/
Great to see you testing, Ben!
One quick note re: the whole "many assertions vs. many tests" subject. DataProviders can really help out here. They're documented at http://wiki.mxunit.org/display/default/Data+driven+testing+with+MXUnit+dataproviders
For your scenario, you might have something like this:
component extends="mxunit.framework.TestCase"{
romans = [['I',1],['II',2]]; //etc
/**
@mxunit:dataprovider romans
*/
function testEncode( romanSet ){
var roman = romanSet[1];
var digit = romanSet[2];
assertEquals( roman, encoder.encode(digit), "Encoding digit failed" );
}
/**
@mxunit:dataprovider romans
*/
function testDecode( romanSet ){
var roman = romanSet[1];
var digit = romanSet[2];
assertEquals( digit, encoder.decode(roman), "Decoding digit failed" );
}
}
I'd echo the comments about posting stuff to the google group. Though I'd say that you're in a particularly good position to get help, what with working with Jamie Krug and all ;-)
As you continue on your testing journey, I'll be interested to see how "test infected" you get. Some people (me) get bitten hard and couldn't imagine writing apps without tests.
Also, if you're interested, the "mightyspec" branch of mxunit (https://github.com/mxunit/mxunit/tree/mightyspec) has a BDD implementation modeled after Jasmine, for CF10. Ray describes it well here: http://www.raymondcamden.com/index.cfm/2012/6/27/Best-of-ColdFusion-10-MightySpec
Again, great to see you using MXUnit!
What I have read with respect to one-test-per-method is that the idea is to give you as much information as possible each time you run the test suite. If you have ten tests within a method, and the first test fails, then you don't know whether or not any of the other nine tests passed, because they won't be run.
On the other hand, yes, it doesn't take long for that approach to give you either monster test components or multiplying test components (from splitting a gargantuan component into smaller ones, thus breaking the one-to-one relationship between live object and test object).
If it doesn't take long to run the test suite, then it seems reasonable to me to put similar tests into a single method to keep things neat, because you can quickly update the code and rerun. (Anyway, if you're doing TDD, then you won't have that issue in the first place, because there should be only two scenarios for a test failing: the last test you wrote fails, because you just wrote it, or other tests fail based on the code you wrote to get the most recent test to pass, in which case you back out that code and try again.)
Once you get basic unit tests down, you'll have to make sure not to let your unit tests sneak into territory marked for integration or system testing ... you'll start using mocks and stubs to make sure your unit tests are testing only the current object. That to me is one of the biggest challenges: separating the tests and doing them in the right places, especially when it means refactoring the code so that it can be testable in the first place.
@Russ,
That's a really cool idea!
@Marc,
I'm definitely interesting in seeing how all this testing stuff works out :) Recently, I've completely the "view" portion of a learning project:
https://github.com/bennadel/MVC-Learning
... and I'm now ready to start working on the Model and fleshing out the Controllers more effectively. Perhaps I'll really try to start with the Tests!!
@Dave,
I am not sure that I understand what you mean here:
Once you get basic unit tests down, you'll have to make sure not to let your unit tests sneak into territory marked for integration or system testing.
I thought all the testing can be done as part of the Unit Testing (except for UI-testing which probably needs to be done with a different set of tools).
Are you saying that if I need to mock out stuff, then I'm doing it wrong?
Even a year after this post, I still have not really adopted any unit testing in my code :( This is sad and shameful; and, to help combat this slothfulness, I tried learning about unit testing more from the inside-out. I ended up creating a super tiny, super simple, zero config framework that fits very nicely with my own coding style:
www.bennadel.com/blog/2469-Tiny-Test-An-Exploration-Of-Unit-Testing-In-ColdFusion.htm
Hopefully NOW I'll actually start to think about unit testing :D
Ben, I like you approach (at least in theory). I too am trying to make myself adopt TDD - I have seen how it can help with Javascript (Jasmine) and it's got to be a good thing for your Cf code too, I am implementing something like wot u have done (sorry I'm a Londoner) for Mura. I noticed your comment about the links not working in the test results html. I found the same and have a cludgy kinda fix for it. It seems when MXUnit writes out the html it 'doubles up' the context folder when it constructs the href's. Look round line 184 in HtmlTestResults - Something to do with CGI.CONTEXT_PATH I think.