Understanding The Basic Authentication Request-Response Life Cycle
For the last couple of days, I have been very frustrated with trying to get Basic Authentication to work with a piece of code being integrated with Twilio. I have used Basic Authentication a few times before; but, I guess I never really understood exactly what was required during the request/response authorization life cycle. And, as it turns out, login credentials only get passed to the target server if and when they are required for authorization as determined by the client making the request.
That's kind of a mouth-full so, what does it actually mean? From what I can gather (and I'm absolutely not a security expert), it basically means that the existence of the basic authentication "Authorization" header is dependent on the client making the login request. That is to say, the way in which authorization headers are handled is not entirely consistent.
To explore the basic authentication life cycle in more depth, I set up a simple page that would echo the incoming HTTP request headers. But, more than just echo headers, it could also return a "401 Unauthorized" header on demand:
Authorize.cfm
<!---
We will be able to request this page without returning
a request for authorization.
--->
<cfparam name="url.use401Response" type="boolean" default="false" />
<!--- Get the incoming request headers. --->
<cfset httpHeaders = getHttpRequestData().headers />
<!---
Check to see if we need to check for authorization and possibly
return a 401 status code response (Unauthorized).
--->
<cfif url.use401Response>
<!---
Check to see if the authorization header has been passed
with the incoming request.
--->
<cfif !structKeyExists( httpHeaders, "Authorization" )>
<!---
Since the user did not submit any authorization, return
a request for Basic Authentication along with a 401
Unauthorized response.
--->
<cfheader
statuscode="401"
statustext="Unauthorized"
/>
<!--- Ask the user to provide basic authentication. --->
<cfheader
name="WWW-Authenticate"
value="basic realm=""website"""
/>
</cfif>
</cfif>
<!--- Return the incoming HTTP headers. --->
<cfdump
var="#httpHeaders#"
label="HTTP Headers"
/>
As you can see in this code, the request outputs a CFDump of the incoming HTTP request headers collection. However, if you ask for the page to use a 401 status code response, it will check for the existence of the Authorization HTTP header; and, if it doesn't exist, it will respond with a 401 status code and abort the request.
Now that we have this authorization page set up, let's look at various approaches to basic authentication so that we can see how it is handled across clients. The first client that we'll look at is the ColdFusion HTTP client:
<!---
Set up the login URL. No username or password is
passed in using the URL since it will be defined in
the CFHTTP tag.
--->
<cfset loginUrl = (
"http://" &
cgi.server_name &
getDirectoryFromPath( cgi.script_name ) &
"authorize.cfm"
) />
<!---
Log into the target page, passing in a username and password.
Notice that we are NOT requesting a 401 Unauthorized status
code to be presented by the target page.
--->
<cfhttp
result="get"
method="get"
url="#loginUrl#"
username="username"
password="password"
/>
<!--- Output the echoed headers. --->
<cfoutput>
#get.fileContent#
</cfoutput>
Notice that in this CFHTTP request, we are passing the username and password but we are not asking the target page to worry about authorization. That is, we are not asking the target page to announce that it needs credentials to be submitted.
When we run the above ColdFusion code, we get the following CFDump output:
As you can see in the returned HTTP request header collection, a CFHTTP request will always submit the basic authentication Authorization header even if the target page doesn't require it. If you submit a username and password, ColdFusion will pass it along, no questions asked.
Now, let's jump into a web browser to see how a more manually-driven client-interaction takes place. In this case, I'll be using Firefox and setting up a page with two links. Both of the links will submit a username and password to the "authorize.cfm" target page; however, only one of them will ask the target page to require authorization.
NOTE: This is much more clearly illustrated in the above video.
<!---
Define our test URL with credentials being passed in both cases.
It is only the "401" url parameter that will determine how those
credentials get used in the test page.
--->
<cfset loginUrl = (
"http://username:password@" &
cgi.server_name &
getDirectoryFromPath( cgi.script_name ) &
"authorize.cfm"
) />
<cfoutput>
<!DOCTYPE html>
<html>
<head>
<title>Basic Authentication</title>
<script type="text/javascript" src="./jquery-1.5.1.js"></script>
</head>
<body>
<h1>
Basic Authentication
</h1>
<!---
Just send username and password, but don't ask the
target server to *request* authorization.
--->
<p>
<a href="#loginUrl#">Get Headers</a>
- This request does NOT require authorization.
</p>
<!---
Send the usernamee and password AND ask the target
server to request authorization.
--->
<p>
<a href="#loginUrl#?use401Response=1">Get Headers</a>
- This request DOES require authorization.
</p>
<div class="output">
<!--- This is where our response (CFDump) will go. --->
</div>
<script type="text/javascript">
// Hook up the links to perform AJAX requests rather
// than native anchor requests.
$( "a" ).click(
function( event ){
// Prevent the default link behavior.
event.preventDefault();
// Load the request into the outpu using AJAX.
$( "div.output" ).load(
$( this ).attr( "href" )
);
}
);
</script>
</body>
</html>
</cfoutput>
As you can see, both links pass the username and password for basic authentication:
http://username:password@www......
However, it is only the latter link that passes the 401 query string flag:
use401Response=1
When I click on the first link, here is the response I get:
As you can see, even though this link presented a username and password, the client (browser) did not pass the credentials along to the target page.
Now, let's click on the second link - the one that requests that the target page use a 401 status code response if the Authorization header is not supplied:
Here, you can see that the client made an initial request which, due to the use401Response query string parameter, resulted in a 401 status code response. The client then automatically made a subsequent request, this time passing the given credentials through to the target page using the basic authentication Authorization header.
What you can't see in this screenshot (check the video) is that the browser initially prompted me for a username and password. This prompt was for the NTLM credentials. I don't know much about these; but they are supposedly more secure than using Basic Authentication. And, as far as the browser is concerned, it wants you to use the most secure login possible. As such, it is only after we escape out of the NTLM credentials that the browser proceeds with submitting the already-supplied basic authentication credentials.
When it comes to using Twilio, their web proxy client acts very much like a standard web client; that is, it doesn't supply the basic authentication credentials unless your web-based end-point actually requires them. This is where I kept getting tripped up - I was not returning a 401 status code - I was just assuming that the Authorization header would always exist (as it does with a CFHTTP request).
I am sure that my understanding of the Basic Authentication life cycle still has some serious gaps in it; but, from what I have seen so far, I am going to consider it a best practice to always include "401 Unauthorized" status code logic if I want the requesting client to pass along its credentials. This isn't always required for clients like the ColdFusion CFHTTP agent; but, using a 401 response seems to be the lowest common denominator.
Want to use code from this post? Check out the license.
Reader Comments
Great post. Haven't finished reading it, but like the explanation and test code walkthrough. This is what is missing from so many IIS posts on the net.
Kevin
@Kevin,
Thanks! The video is from my live box. On my local machine, I am using Mac / Apache and therefore don't get that initial prompt. Apache doesn't present an NTLM option and therefore the browser simply defaults to the Basic Authentication option. In both cases, however, the browser only presents the credentials IF and ONLY IF a 401 status code is sent back.
Hey Ben,
If you need to password protect areas of your sites then .htaccess is by far the best way.
From what you have said I am guessing your are running IIS on your live server, in which case try IISPASSWORD, which works the same way as .htaccess.
http://www.iistools.com/en/iispassword.html
@Russ, good point.
The CGI defines a "REMOTE_USER" when authentication has been done through HTTP. Apache or IIS should pass (CGI.)REMOTE_USER to ColdFusion (i.e. the "interface" part of the common gateway interface) allowing you to move authentication to a layer beneath your CF application.
In fact the life cycle is the same whether it's basic, digest, NTLM, Kerberos, etc. It might be better called the "HTTP authentication...life cycle."
The nice thing (or really bad thing, depending on your perspective) about 'basic', is you can get at the password from within Coldfusion and handle auth at that layer. (http://coldfusion.sys-con.com/node/230503?page=0,1 comes to mind.)
A bit off-topic; the cfinvoke for webservices or cfhttp have no support for NTLM Authentication.
And Java libraries that do have them (Apache HTTP Client for example) have other bugs, so if you want to connect CF to a Microsoft Webservice (like Sharepoint or Dynamics) protected with NTLM you are fresh out of luck.
Or has someone found a way to do that ?
@Snake, @Jon,
I admittedly don't know all that much about the non-CF layer of the stack. I mean sure, I know my way about SQL and HTML and what not; but as far as how it all gets pulled together, I'm still a bit of a novice. The nice thing about handling security at the ColdFusion level is that I get to do on-demand security.
In this demo, for example, I could give Twilio a unique username/password for every interaction; or I could make it unique to each particular user/session. I can't think of a great use-case for that; but, since it's all at the ColdFusion level, it's doable. Plus, I would assume that logging bad requests is easier at the ColdFusion level?
@Nelle,
I know that CFHTTP only has Basic Authentication support; but, I can't say I know anything about the NTLM protocol. Perhaps dropping down into a (shutter) SOAP API, you can use SOAP-based security or even something like a digitally-signed XML request or header value? As far as security is concerned, I've only just gotten my feet wet.
The problem with ColdFusion authentication is that it only protects ColdFusion pages. Any non CF pages are still open to the world.
.htaccess style auth protects everything in the folder, and you can also progmatically manage users/groups via CFML so you actually have MORE control than you do with a plain old CF authentication system as you have built in group based access for no extra work.
If you then want to add an extra level of roles based security or access to specific resources within your CFML app you can still do that once they are logged in.