Getting ColdFusion, Helicon Ape, X-SendFile, URL Rewriting, And IIS To Work Together
For the last couple of weeks, I've been working on a ColdFusion server migration. And, unfortunately, the last 80% of that effort has been put into getting Helicon Ape and, specifically, X-SendFile (mod_xsendfile), to serve up files that are outside of the ColdFusion web-root. Finally, this morning, after much back-and-forth with both my server team as well as the Helicon Ape tech support team, we've got it working! It wasn't easy, so I thought I would share my journey.
Using Helicon Ape For Both URL-Rewriting (mod_rewrite) And X-SendFile (mod_xsendfile)
On my old server, I was using the Ionic ISAPI Rewrite filter to perform URL-rewriting and Helicon Ape / X-SendFile to serve files. As I migrated to the new server, I thought it would be a great opportunity to ditch the old ISAPI filter approach and move my URL rewriting over to Helicon Ape. This way, all my "Apache" functionality would be consolidated in the Helicon Ape pipeline.
Unfortunately, this plan was a no-go. Helicon Ape's URL Rewriting doesn't work so well when you are rewriting to a ".cfm" page. Of course, you can get the URL rewriting to work by making sure that your ".cfm" mappings point to the wildcard connector:
jrun_iis6_wildcard.dll
However, if you do this, Helicon Ape's X-SendFile functionality stops working, according to Helicon Ape Tech support:
It's a known issue that after jrun_iis6_wildcard.dll the request processing in IIS stops, which means that Helicon Ape doesn't receive the events that are executed after the handler. Mod_xsendfile, in its turn, takes place after the handler ... This means that mod_xsendfile cannot work together with ColdFusion using jrun_iis6_wildcard.dll.
As such, we had to revert back to Ionic ISAPI rewrite filter so that we can could rebind the standard jrun_iis6.dll connector. This way, IIS would continue to process the page after it was sent to ColdFusion, allowing us to set the "x-sendfile" CFHeader value.
NOTE: I could have started using IIS's URL Rewriting module; I'm just more comfortable with the Apache-compatible syntax.
Using Helicon Ape With Only X-SendFile (mod_xsendfile)
After we reverted back to Ionic ISAPI Rewrite for the URL rewriting, I thought we were in the clear. But, unfortunately, X-SendFile continued to fail. This was super frustrating because there were no errors; it just didn't work. In fact, even the Helicon Ape error log seemed to be reporting an OK success:
X-Sendfile: C:\Websites\example.com\wwwroot\test\image.png
X-Sendfile: C:\Websites\example.com\wwwroot\test\image.png - OK
After inspecting the HTTP response (200 OK) of the test page, I saw that the "x-sendfile" response header was still in-tact. This is a telltale sign that X-SendFile (mod_xsendfile) is not serving up the file as it [mod_xsendfile] will strip out the "x-sendfile" header.
At this point, I had no idea what to do. So, I purchased some premium support for Helicon Tech; and, after a few days of back-and-forth, they were able to narrow down the problem. Apparently, my wildcard URL handler mapping in IIS had an incompatible buffer limit:
<add
name="AboMapperCustom-51168"
path="*"
verb="*"
type=""
modules="IsapiModule"
scriptProcessor="C:\ColdFusion9\runtime\lib\wsconfig\1\jrun_iis6_wildcard.dll"
resourceType="Unspecified"
requireAccess="None"
allowPathInfo="false"
preCondition="bitness64"
responseBufferLimit="0"
/>
Notice that the mapping defines a "responseBufferLimit" of zero. This is an optional setting that, according to the IIS documentation, "Specifies the maximum size, in bytes, of the response buffer for a request handler." By removing this attribute, it allowed X-SendFile to successfully stream the binary file content!
It was quite a journey; but, we finally made it! I hope this helps anyone else who might be having trouble with getting ColdFusion, Helicon Ape, URL rewriting, and X-SendFile to work with IIS.
Reader Comments
We use CF10, IIS7.x and Windows 2008 R2 Datacenter here, and there have been many times where I contemplated why certain tasks take as much effort as they do.
Originally, ColdFusion's beauty was that it took those convoluted tasks and allowed us to simplify the process, coining the "Rapid Application Development" it was renowned for. I sometimes wish people would continue this tradition with the plugin architecture of servers.
Recently we wanted to remove all server-side code support at the site level and below except CF. (Remove any PHP/.NET handlers to ensure faster performance), and you're left with a very manual process that isn't well documented.
I feel your pain, Ben. But like you, we persevere until we get the job done.
We're trying to get sendfile rolling over here too. I was getting 200's but a blank file pretty much. I've been swamped with stuff, and haven't had time to dink around with it.
If the solution ends up being that easy, I definitely owe you a drink or something.
@Aaron,
At least the internal ColdFusion stuff "just works" ... most of the time. Any time I complain about anything not working properly on Windows, everyone is like, "Just use Linux and Apache." If only I knew more about server management, I might. What I do, I do out of necessity and use a lot of Googling. I need that Windows GUI.
@Dana,
I hope it works out; would love to hear if this helps!
@Ben,
Oh man, you got that right. I have deep respect for Linux, but some people (myself included) at not cut out for command lining every command. A GUI serves a purpose (however, as much as I love IIS' GUI, I have to say it is not the most intuitive; then again, Microsoft has never been known to be anywhere NEAR cutting edge when it comes to UI).
I like your attitude, though. Being told "Just use Linux/Apache" is not a solution. Heck, it's not even a workaround. It's a brush off. The problem still stands! And that's why (as you already know), Google comes to the rescue!
We've taught ourselves that searching for others who have already been through the same problem is the preferred solution than to expect us to come across that nugget of info in some random book on the subject. Much of what we do is improvisation. :)
Ben - seriously dude, checkout NGINX. You'll stop losing sleep.
Ben,
Just out of curiosity, why are you going with Ape instead of native IIS URL Rewrite and CFCONTENT? Is there a limitation that you are working around?
@Sami,
nginx is all kinds of awesome but has the Windows support improved? Last time I looked at it, it still didn't have support to run as a service and can't scale as well as it does on linux (http://nginx.org/en/docs/windows.html). I still avoid IIS when I had the choice, but go with Apache as I'm stuck with Windows :P
@Ben,
Useful post, thanks for sharing :) Where I work we've been talking about using Helicon Ape for this sort of task. Now I know whose name is going to appear when we search for information on it. ;)
@Aaron,
The worst part is, after being on a Mac for 2 years, and using the command-line to run things like Git and NodeJS, I almost feel like maybe I could do it. Though, I know nothing about how servers work or about permissions and security and all that jazz. So, I am sure my Mac-oriented knowledge is just enough to be dangerous :D
@Sami,
I've heard of nginx, but know nothing about it. For some reason, I thought it was something that related to NodeJS; but, a quick Google search and it looks like it's just an HTTP server like Apache (though, with apparently better performance under high load).
@David,
I'd also consider using Apache on Windows (which I used to do on desktop machine); but, again, I just feel like I don't know enough about servers and security to really manage all that stuff.
For example, I know that in IIS, you need to have "CF" be a user that has certain permissions so that it can run stuff; but, at the same time, I guess, not too much permissions to be very dangerous?? I don't really know.
@Roland,
I've been going with X-SendFile for a while simply so that I can decrease some load on the ColdFusion thread-pool. Right now, I'm working with some highish-traffic stuff; and I just try to free-up ColdFusion threads as quickly as possible.
All,
NGINX on Windows rocks. We run it as a service, and have done so for almost 3 years. It does all the caching, gzipping, header manipulation, SES, and yes, X-SendFile stuff. All the trick you could ever want.
Basically we run NGINX -> Tomcat -> CF. Its super light, super fast, and super flexible.
Joseph Lamoree has done some presos on this (he is a member of my team). Maybe this has finally prompted me to write up our windows setup, and how we do some brilliantly super stuff with CF.
Trust me, NGINX is a life saver.
I share your pain with deploying CF on IIS.
It's all a headache, when you get right down to it, whether it's Apache/Tomcat or IIS 7.
CF is magical because for 90% of what we do, it's adequate and it's fast. It's that 10% where we run into issues, and I find leaning on host offerings, be it .NET or Java, to be the redemptive factor for ColdFusion. If you have a need that exceeds CF's capabilities, it opens the door for you to find a solution elsewhere.
That "elsewhere" is why I ultimately prefer IIS to Apache/Tomcat. The .NET integration is great, provided you install the .NET frameworks and the integration service in the right order. Getting the right combination of Java dependencies for some functional equivalents really is a headache. I've found some success in developing classes in both .NET and Java, with the latter especially necessary for making a prepackaged solution I'm pushing run in both CF and Railo. Perhaps it's a factor of my comfort level with programming in either, but I prefer C# in .NET to Java. Usually, all of my dependencies are right there in the .NET framework, so there's no arm wrestling with CF and what version of axis2 goes with jaxws and such.
I've found a plus for ColdFusion 10 on IIS in recent months...the rapidity of developing REST services in CF10 vs. WCF REST via .NET is matched by ease of deployment, even with permission nuisances. There are some hurdles with IIS you just won't have to revisit once you leap past them.
@Sami,
It sounds groovy. Someone actually brought nginx up to me in a meeting on Friday - funny timing. I'll have to put some time into checking it out.
@Brian,
100% agreed. I can't tell you how grateful that I am the ColdFusion is built on top of Java and that I can dip down into the Java layer. I find myself doing that a LOT. It's part of what makes CF so awesome! (and at times, so frustrating!) :)
Hi Ben,
it is possible to get CF to work together with APE, I actually informed Helicon of this issue originally and informed them of this work around, so i'm surprised they didn't tell you about it.
http://www.michaels.me.uk/post.cfm/getting-coldfusion-working-with-helicon-ape
@Russ,
I think our issue was more about trying to get both URL rewriting AND X-SendFile to work in conjunction with ColdFusion. The problem is that while we could get the URL Rewriting to work with the wildcard mapping, the subsequent X-SendFile requests would break (due to a limitation of IIS and the wildcard mapping). That's why we need to revert back to the non-wildcard mapping.
@Ben,
yes it is the wildcard mapping which is the issue, check the article for the workaround.