ColdFusion CFSwitch Tag, CFCase Order, And Performance
Now, I am sure I will catch some flack for this performance test, but until someone actually gives me CONSTRUCTIVE criticism on performance testing, this is the only way I know how to do it.
That said, I was chilling in my bed last night watching a Family Guy rerun when I started to think about ColdFusion frameworks. Nothing in particular, just frameworks in general. This got me thinking about CFSwitch statements. I started to wonder if the order of CFCase statements mattered for performance. I know that in a CFIF/CFELSEIF/CFELSE statement, you want to put the more commonly used statements first (so that less conditions need to be evaluated), but I wondered if anything like that existed for CFSwitch / CFCase statements?
To test, I programmatically built a very large CFSwitch statement into its own file, cases.cfm:
<cfoutput>
<cfsavecontent variable="strCode">
[[cfswitch expression="##strValue##"><cfloop index="i" from="#Asc( 'A' )#" to="#Asc( 'Z' )#">
<cfloop index="j" from="1" to="70"
>[[cfcase value="#Chr( i )##j#">
[[cfset WriteOutput( "#Chr( i )##j#" ) />
[[/cfcase>
</cfloop></cfloop>[[/cfswitch>
</cfsavecontent>
</cfoutput>
<cffile
action="WRITE"
file="#ExpandPath( './cases.cfm' )#"
output="#strCode.ReplaceAll( '\[\[', '<' ).Trim()#"
/>
Sorry for the crap readability, but I was trying to make the file as small as possible. This makes case statements that start with a letter (A-Z) and for each letter makes cases 1 to 70. This comes out to be 1820 case statements in the switch in alphabetical ASC order.
Then, I basically just loop over that a few times picking random cases. In my first loop I make sure to pick only cases that start with "A". These should be the first ones in the CFSwitch statement. In the second loop I make sure to pick only cases that start with "Z". These should be the last ones in the CFSwitch statement:
<!--- Only pick early cases. --->
<cftimer label="A - Cases" type="outline">
<cfloop index="intI" from="1" to="50" step="1">
<!--- Get a random case to test. --->
<cfset strValue = (
"A" &
RandRange( 1, 50 )
) />
<!--- Include the cases. --->
<cfinclude template="cases.cfm" />
</cfloop>
</cftimer>
<!--- Only pick later cases. --->
<cftimer label="Z - Cases" type="outline">
<cfloop index="intI" from="1" to="50" step="1">
<!--- Get a random case to test. --->
<cfset strValue = (
"Z" &
RandRange( 1, 50 )
) />
<!--- Include the cases. --->
<cfinclude template="cases.cfm" />
</cfloop>
</cftimer>
What I found was that there was absolutely no noticeable trend. Sometimes one was faster, sometimes the other was faster. That's pretty cool to know. And, this was for a HUGE number of cases. On a small case set, it must be even faster. Man, I love ColdFusion, it's so freakin' sweet! So, in conclusion, CFCase statement order does not seem to have any affect on look-up times. I guess this all comes down to how the language eventually compiles down into machine code. I did take a class in that, or something similar (SPARC Architecture), but that was an eon ago and all I remember is understanding that Programming Languages were built so we didn't have to deal with it :)
Want to use code from this post? Check out the license.
Reader Comments
Just a note and reminder that WebApper found some performance issues with CFSWITCH.
http://www.webapper.net/index.cfm/2006/7/27/20060727042244
@Ben:
Loop testing isn't efficient "performance" testing when it's a single serialized test. In a single serialized test environment, results can be skewed by anamolies--such as garbage collection or other background tasks--which can result in misleading results (which is why often in serialized tests you see seemingly random results.)
Also, when you're only testing on a single thread, you may come up w/some code that performs efficiently as a single thread, but as soon as the code comes under real load, it begins to show performance issues.
This is why it's recommended to actually do performance testing by throwing some actual load at the server. You can use a free load testing tool (like jMeter) to actually simulate a heavy load on the code.
This will give your performance testing better real world simulation.
@Michael
Thanks for the link.
@Dan
I just downloaded JMeter and it looks SUPER complicated and has a ton of documentation. I know zero about load testing. Is there a really simple way to do this? Basically I am assuming that I just want to hit the same URL with a bunch of simultaneous requests. Or is that too simplistic of a view?
Would it be possible to use Java's asyncHttp and just run a page that launches like 50 http requests to the same URL and check to see how long it takes all of them to complete? Is that like retarded simple? Am I not realizing what Load Testing actually does?
Thanks!
@Ben:
jMeter really isn't that complicated, once you use it a couple of times. See if this guide doesn't help you out:
http://www.informit.com/guides/content.asp?g=java&seqNum=268&rl=1
I'd also recommend Microsoft's Web Application Stress Tool (WAST). It looks pretty old school, but it works rather nicely for simple to medium complexity stress testing. It also saves fairly decent reports which are easy to get the numbers out of for comparing different test runs. I found JMeter to be good at generating load, but the reporting seemed pretty low-tech.
Barny
www.paessler.com also have a nice load testing tool. It's not free but easy to use. They have a limited trial version. Opensta is also another good - free tool which can be used for loadtesting.
Regarding load testing - you essentially want to simulate load in a way that will mirror real world usage patterns. For example, users will not sit there loading the same page 200 times a minute . These loadtesting tools allow you to capture a browsing session - say a user performing a search or registering with a site. You can then play this session back multiple times. You can have a set or random delay between when each session starts and can also run different types of browsing sessions at the same time. So you can simulate the effect it has on the server when someone is trying to register meanwhile 300 over simultaneous users are performing searches, 40 users are pulling off reports etc.
KOla
Kola,
Thanks for the link. I will check it out. I tried using the one from Jakarata and I just couldn't get it to do anything. I will have to really sit down and figure this stuff out.
Thanks.