Jason Dean Tells Me To Use AES (Advanced Encryption Standard) Encryption

Thanks for yet another wonderful post Ben! Long-time reader, first time commenter :)

You might want to check out The Code Book by Simon Singh. It was a great read on the history of encryption.

http://books.google.com/books?id=fbp9V9dkaNkC

Awesome post Ben. Good work, and I am glad I am helping.

A couple minor corrections.

"... you will also have to persist the secret key used to encrypt it. This is typically done in a database"

Careful with that advice. All too often people make the mistake of storing the encryption key in the database WITH the data they are trying to protect. Key management is the hardest part of crypto and should be thought through carefully and with the guidance of experts. THose interested should check out the NIST Key Management Guidelines here: http://csrc.nist.gov/groups/ST/toolkit/key_management.html

"And, the PKCS5Padding part tells the encryption algorithm how to break the plain text data into blocks of equal byte-length."

More specifically, padding is used only when the last block of the message is not equal to the bit length of the blocks being used. AES is a 128-bit block cipher. If you are processing a message that is 284 bits in length you will end up with two 128-bit blocks and a 28-bit block. The algorithm needs 128-bit blocks, so if you tried to encrypt that message without padding you would get a block size exception. When we opt for padding, however, it will add characters to the end of that block until it gets to 128-bit and then process it. PKCS5(Public Key Cryptography Standard #5 http://tools.ietf.org/html/rfc2898) is the method of padding it uses. Of course, the method is important because they algorithm needs to know how to remove that padding on decryption.

Anyway, thanks for the plugs and the great post. THis is the kind of information that developers need.

Ben,

You don't HAVE to use generateSecretKey(), it's just the easiest way to do it if your application is the one determining the key and will be doing both the encrypting/decrypting of the data. However, you may have situations where an external application or service requires a specific key be generated.

I had this happen when working with the Single Sign On API for UserVoice, they required generating a token by encrypting a JSON serialized representation of a user using AES/CBC/PCKS5Padding, but had a strict algorithm for creating the key.

Obviously, this is required when one application is encrypting and another is decrypting.

I created a project for generating these tokens on GitHub, but the function for generating a key appropriate for AES using the encryptBinary() function is below:

<cffunction name="generateKey" access="private" returntype="string" hint="Generates the Base64 encoded encryption key" output="false">
<cfargument name="accountKey" type="string" required="true" />
<cfargument name="apiKey" type="string" required="true" />

<cfscript>
var salted = arguments.apiKey & arguments.accountKey;
var hashed = binaryDecode(hash(salted, "sha"), "hex");
var trunc = arrayNew(1);
var i = 1;
for (i = 1; i <= 16; i ++) {
trunc[i] = hashed[i];
}

return binaryEncode(javaCast("byte[]", trunc), "Base64");
</cfscript>
</cffunction>

@Jason,

Thanks for posting the NIST link. Key management is often overlooking when implementing an encryption system.

I copied and pasted your code exactly, and I get an error for both blocks of code.

First block, I get:
"The key specified is not a valid key for this encryption: Illegal key size or default parameters."

Second block, I get:
"The key specified is not a valid key for this encryption: Illegal key size."

Is there something I'm missing?

@CD,

In your generateSecretKey() call, try changing it to an uppercase AES

generateSecretKey("AES")

I saw it reported a few weeks ago that, at least on some systems, that if you use lower case AES that you get a 256-bit key instead of a 128-bit key, and if you do not have the unlimited strength JCE policy file installed on your system then a 256-bit key is not an allowed key length.

I'm using CF 8.0.0.176276.

Don't break your back. I'll try it again when I get home tonight. I was just curious since this was something I was actually playing with last week. It's still good information to know. :-)

That's it, Jason! Thanks! Changing it to uppercase "AES" worked perfectly. :-)

@Ben, you may want to modify your code sample to use the uppercase. Just so others don't have similar issues.

It's also possible to derive the secret key from a string such as a password, to avoid storing the encryption key at all. Of course, if the user forgets the password used to generate the key access to the encrypted data is lost. As Jason mentioned, key management is one of the hardest parts. I had a situation where I needed to use a password-derived AES key in ColdFusion and the following function and sample code was the result for those interested.

<!---
genAESKeyFromPW function by Justin Scott
Generates an AES encryption key based on a provided password and salt.
Based on some code from:
http://stackoverflow.com/questions/992019/java-256bit-aes-encryption
http://shariffdotnet.blogspot.com/2009/04/encrypt-and-decrypt-image-with.html
Download the "unlimited strength jurusdiction policy files" to enable 192 or 256-bit keys.
https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jce_policy-6-oth-JPR@CDS-CDS_Developer
--->
<cffunction name="genAESKeyFromPW" access="public" returnType="string">
<cfargument name="password" type="string" required="true" />
<cfargument name="salt64" type="string" required="true" />
<cfargument name="length" type="numeric" required="true" default="128" />
<!--- Decode the salt value that was provided. --->
<cfset var salt = toString(toBinary(arguments.salt64)) />
<!--- Go fetch Java's secret key factory so we can generate a key. --->
<cfset var keyFactory = createObject("java", "javax.crypto.SecretKeyFactory").getInstance("PBKDF2WithHmacSHA1") />
<!--- Define a key specification based on the password and salt that were provided. --->
<cfset var keySpec = createObject("java", "javax.crypto.spec.PBEKeySpec").init(
arguments.password.toCharArray(), <!--- Convert the password to a character array (char[])--->
salt.getBytes(), <!--- Convert the salt to a byte array (byte[]) --->
javaCast("int", 1024), <!--- Iterations as Java int --->
javaCast("int", arguments.length) <!--- Key length as Java int (192 or 256 may be available depending on your JVM) --->
) />
<!--- Initialize the secret key based on the password/salt specification. --->
<cfset var tempSecret = keyFactory.generateSecret(keySpec) />
<!--- Generate the AES key based on our secret key. --->
<cfset var secretKey = createObject("java", "javax.crypto.spec.SecretKeySpec").init(tempSecret.getEncoded(), "AES") />
<!--- Return the generated key as a Base64-encoded string. --->
<cfreturn toBase64(secretKey.getEncoded()) />
</cffunction>

<!--- Password method, generates an AES key based on a password and salt. --->

<!--- User would provide their encryption password to the system. --->
<!--- Password is case-sensitive and could be passed through LCase() or UCase() to negate case. --->
<cfset myPassword = "this is the password" />

<!--- The system would generate a salt in Base64 for their account. --->
<!--- The salt is used by the key generator to help mitigate against dictionary attacks. --->
<!--- It's recommended to use a salt generated based on Java's SecureRandom object. Bill Shelton wrote a genSalt() function in CF to do just that, available at http://blog.mxunit.org/2009/06/look-ma-no-password-secure-hashing-in.html --->
<cfset mySalt64 = toBase64("some salt string") />

<!--- Call the password-based key generator with the password and salt. --->
<cfset generatedKey = genAESKeyFromPW(myPassword, mySalt64) />

<!--- The generated key drops right in where a key from generateSecretKey("AES") would normally go. --->
<cfset encrypted = encrypt("Hello World!", generatedKey, "AES", "Base64") />

<p><cfoutput>encrypted: #encrypted#</cfoutput></p>

<!--- Decryption works the same way. --->
<cfset decrypted = decrypt(encrypted, generatedKey, "AES", "Base64") />

<p><cfoutput>decrypted: #decrypted#</cfoutput></p>

@Jason - that code is awesome. I've been wracking my brain to accomodate a very similar set of functionality in a new site I'm doing.

@All - is this useful beyond passwords? I had a client a while back that was looking for some basic encryption on a proprietary messaging system they were using, and they didn't want to deal with third party plugins. I lacked the background in key management so was at a loss where to go with it, and lost that particular piece of the contract. I.E. what's the speed on this in CF 9 for longer strings?

@Brian - For passwords I would normally use a hashing algorithm rather than one that can be decrypted (there's usually no need to be able to decrypt passwords, so a hash is appropriate in all but a few edge cases). AES, especially with larger key sizes, is pretty strong encryption and would be acceptable for storing credit cards and other sensitive information assuming proper key management is utilized.

AES is actually rated by the U.S. Dept of Defense for classified material rated up to TOP SECRET (128-bit key for CLASSIFIED, 192-bit and 256-bit keys for SECRET AND TOP SECRET). So it is certainly useful beyond passwords.

As Justin said, passwords are best stored as hashes unless you have a really good reason for decrypting them (note: sending them to your users in email when they forget is not a good reason).

AES is very fast for such a strong algorithm. I am not sure what a "longer string" is. If you are trying to encrypt/decrypt a few hundred bytes once-in-a-while, you'll probably be fine. But if you are trying to encrypt/decrypt Stephen King's Dark Tower series every 18ms, then you might be in for some slow downs.

Maybe my information is out of date, but once upon a time I thought there was a limitation in using AES that it had to be in the US or US export friendly countries. I could be wrong.

There are import/export laws pertaining to AES Encryption with key sizes higher than 128-bit. I don't pretend to understand the specifics. But if you are using 128-bit keys or less then I don't think you have anything to worry about.

I believe this is why, if you want to use AES keys larger than 128-bit you must download the Unlimited Strength Jurisdiction files from Oracle.

http://www.oracle.com/technetwork/java/javase/downloads/index.html (Bottom of the page)

Ben great article. I am working with encryption for the first time in years. The AES is new to me so this is a great read.

I am having some issues. I used your 2nd example to encrypt some addresses, phones numbers, emails into a database. I was having display issues so I pulled them farther down on the submission page in a query and decrypted them. everything displayed like I entered it.

when I tried decrypting them on an administrative side page to view them, using the same code I used to decrypt them on the submission page, I keep getting the same response:
"An error occurred while trying to encrypt or decrypt your input string: Given final block not properly padded." I fooled around with it for a bit, But I am stuck. Thought it might have something to do with the secret key, but I am not sure.

@Adam,

If I had to guess I would say that the problem is with how you are getting the encrypted data to the admin page. Possibly a problem with how you are staring or passing the data or maybe with how it is being encoded.

It is hard to tell without some sample code/data. Would it be possible for you to put something up on Pastebin to show how you are doing it? Perhaps with some sample data?

Jason

@Jason

no problem. Here is the insert and the query I am using to call it with. and then the output. I am using the same output on the admin page.

<cfquery name="Insert" datasource="#DSN#">
INSERT INTO Applications
	(Address, Address2, Phone, Email)
VALUES
	(<cfqueryparam value="#Encrypt(attributes.ADDRESS,SecretKey,'AES/CBC/PKCS5Padding','hex')#" cfsqltype="CF_SQL_VARCHAR">,
	<cfqueryparam value="#Encrypt(attributes.ADDRESS2,SecretKey,'AES/CBC/PKCS5Padding','hex')#" cfsqltype="CF_SQL_VARCHAR">,
	<cfqueryparam value="#Encrypt(attributes.PHONE,SecretKey,'AES/CBC/PKCS5Padding','hex')#" cfsqltype="CF_SQL_VARCHAR">,
	<cfqueryparam value="#Encrypt(attributes.EMAIL,SecretKey,'AES/CBC/PKCS5Padding','hex')#" cfsqltype="CF_SQL_VARCHAR">)
</cfquery>
 
<cfquery name="getApp" datasource="#DSN#">
SELECT	*
FROM	Applications
WHERE	ID=<cfqueryparam value="#ID#" cfsqltype="CF_SQL_INTEGER">
</cfquery>
 
<br />
<br />
 
Address: #Decrypt(Address,SecretKey,"AES/CBC/PKCS5Padding","hex")#
<br />
#Decrypt(Address2,SecretKey,"AES/CBC/PKCS5Padding","hex")#
<br />
Phone: #Decrypt(Phone,SecretKey,"AES/CBC/PKCS5Padding","hex")#
<br />
Email: #Decrypt(Email,SecretKey,"AES/CBC/PKCS5Padding","hex")#
<br />

In my applications that use encryption, I generally define a global encryption/decryption function for the application that all encrypted data passes through. This custom function acts as a wrapper for the CF encrypt/decrypt functions and acts as a clearinghouse so that I don't have to make the key globally accessible or specify the other parameters each and every time that I need to encrypt/decrypt information. Makes things a lot easier to manage and less exposure for the key.

@Adam,

I dunno, that all seems reasonable. How are you storing and retrieving the the key? You might also want to throw some trims around your data as you are putting it into the DB. Whitespace can show up in unusual places and cause you issues.

@Justin

I saw your post earlier with the cffunction and I am going through it now and reading it all. Thanks.

@Jason
That is my problem. I wasnt sure how to define a global aes secretkey. I am going through Jason's post and going to try that. I will put some trim's on the variables. Any idea's?

thanks ben for great article! currently working with encryption. all run smoothly but i always generateSecretKey. it is ok?

@Ben @Jason @Justin Scott

I just wanted to say thank you for the information, code samples and for putting me on the right path for my encryption needs. Clarifying AES is used for Top Secret data confirmed it can cover our HIPPA requirements. I appreciate all of you taking the time to put this together.

Have a great day!
Jenny

Can someone explain how it is that the AES algorithm is considered stronger than DESEDE, but generateSecretKey("AES") gives a 24 character key, the last two of which are padding ("=="), while generateSecretKey("DESEDE") gives a 32 character key without padding? Obviously, more goes into the security of an algorithm than its key length, but I still find this somewhat surprising.

@Dave,

There is a lot more to consider in a crypto algorithm than the key length. However, key length is important.

The length lengths you are seeing are for AES with its weakest key length (128-bit) and TripleDES with its strongest key length (168-bit). Even still AES-128 is still probably better than TripleDES at 168-bit.

The next thing to consider is the algorithm itself. The name TripleDES is a bit of a misnomer. It is not three times better than DES, it is actually DES three times. It uses the DES algorithm, nothing more, it just does it thrice with 3 keys. In its strongest form it does it with three keys. Here is pseudocode:

DESEncrypt(k3, DESDecrypt(k2, DESEncrypt(k1, string)))

k1, k2, and k3 are each 56-bit keys. In TripleDES weakest form, it uses the same key all three times, which is essentially the same as using DES.

AES in stronger algorithm that can use key lengths up to 256-bit. It was actually selected to replace DES as the standard.

In my own experience AES is also faster than TripleDES. I do not have any numbers here to back that up, I suggest you try it yourself.

This is a very simplistic overview and there are certainly aspects that I have not covered nor probably could cover. I am not a cryptanalyst.

Some other things to look at:

http://www.icommcorp.com/downloads/Comparison%20AES%20vs%203DES.pdf

http://stackoverflow.com/questions/5554526/comparison-of-des-triple-des-aes-blowfish-encryption-for-data

http://www.differencebetween.net/technology/difference-between-aes-and-3des/

Thanks Jason, good explanation and links.

I'm not actually using encryption in any apps at this point, only strong hashes, but it's something I'm generally interested in.

My priority is to be able to encrypt a string on ColdFusion and decrypt on non-ColdFusion. Correct me if I'm wrong, but I understand that in order to do this reliably I must ensure the following:

1. The same character encoding must be used for encryption as is used for decryption (such as UTF-8)
2. Don't encrypt the string itself; rather, encrypt the byte array of the string.
3. When you take the encrypted byte array of a string and try to decrypt it, this decrypts it straight into a string again rather into a byte array.

Here's some code that functions on both ColdFusion and Railo:

<!--- String --->
<cfset variables.mystring = "Today is the first day of the rest of your life!">
<cfdump var="#variables.mystring#" label="String">
 
<br>
 
<!--- Convert string to UTF-8 byte array --->
<cfset variables.mystringEncoded = CharsetDecode(variables.mystring, "utf-8")>
<cfdump var="#variables.mystringEncoded#" label="Convert string to UTF-8 byte array">
 
<br>
 
<!--- Encryption algorythm and strength --->
<cfset variables.encAlgorythm = "AES">
<cfset variables.encStrength = "128">
 
 
<!--- Encryption key --->
<cfset variables.key = generateSecretKey(variables.encAlgorythm, variables.encStrength)>
<cfdump var="#variables.key#" label="Encryption key (#variables.encAlgorythm# #variables.encStrength#)" />
 
<br>
 
<!--- Encrypted byte array --->
<cfset variables.encryptedByteArray = encrypt(variables.mystringEncoded, variables.key, variables.encAlgorythm)>
<cfdump var="#variables.encryptedByteArray#" label="Encrypted byte array" />
 
<br>
 
<!--- Decrypted byte array --->
<cfset variables.decryptedByteArray = decrypt(variables.encryptedByteArray, variables.key, variables.encAlgorythm)>
<cfdump var="#variables.decryptedByteArray#" label="Decrypted byte array" />
 
<br>

I have one question though. If I want to use AES for my encryption algorithm, and exchange encrypted data between two systems (encrypting in ColdFusion, and decrypting in VB.NET), I have to use Cypher Block Chaining (CBC) as my feedback mode. The default feedback mode in ColdFusion is known as Electronic Code Book (ECB).
Source: http://helpx.adobe.com/coldfusion/kb/strong-encryption-coldfusion-mx-7.html

In my code above I didn't specify a feedback mode, and so the default is used which is ECB. As far as I know, VB.NET requires CBC rather than ECB.
Source: http://www.danielansari.com/wordpress/2010/08/decrypting-aes-encrypted-values-from-coldfusion-in-net/

So if I want to improve interoperability between CF and VB.NET, I have to use a compatible feedback mode, which in this case is CBC. The trouble is, I can't seem to get Railo to accept an argument to the Encrypt function that looks like this: "AES/CBC/PKCS5Padding" In my case, it only works if I use "AES"

Has anyone here had a similar experience, and if so, how were you able to adapt your code so that CBC would work on Railo?

This post has been really helpful in understanding and using AES encryption, so thanks! My question is related to key management.

What is the best way to store a key?

Right now I store the key in a OS text file (not in a web root) that is locked down to the ColdFusion service user account only. It works, but...is there a better/more secure/preferred way? I know it would be bad practice to put the key in the DB where the encrypted data is. But would it be better off in a separate db or even a different server?

A followup question is what do others do in terms of key rotation? change once a year, month, etc...? Has anyone come up with a process or best practice for changing a key and re-encrypting the existing encrypted data in the DB with the new key (like a custom tag/cfc)?

@Jason McNeill,

I am downloading Railo now to see if I can make it work. I also sent a Twitter message to the Railo team to ask them to look at your comment.

@Patrick G,

That is a BIG question. The question everyone wants an answer to. I touch on it briefly in one of my presentations (which I will be giving at NCDevCon in September, http://www.ncdevcon.com/).

Key management is a BIG job. HUGE. So big that NIST has published hundreds, maybe thousands, of pages on it. http://csrc.nist.gov/groups/ST/toolkit/key_management.html

The way you handle keys depends on the sensitivity of your data and your resources. You can spend thousands of dollars on a key management server, but if you are only protecting passwords on your local knitting clubs website, then that may not be prudent or reasonable.

But if you are a multi-bazillion dollar banking enterprise protecting the assets of your world-wide client base, then dropping coin on super-mega crypto key-management servers is a wise investment.

If, like most of us, you are somewhere in between those two extremes, then you have to find a solution that works for you. Your method is better than putting it in the source, or having the file in the web accessible area, but is not as good as other solutions.

In all honesty, this is not something I have a great deal of experience with. I have done reading and research, but have never implemented a key management server, nor dealt with credit card data. I have dealt with HIPPA data.

In my experience the best solution we have found is to not deal with crypto at the application level, but in the DB server itself. Store the data in an encrypted database and connect to that database over TLS/SSL or SSH. The data is then protected in transit and at rest. It is never stored in session variables or anything in the application. If needed, and ONLY if needed, it is requested from the database.

@Jason McNeill

I just took a look at Railo and at the Railo source. The encrypt() and decrypt() functions are not implemented to the CFMX7+ standard. They are at pre=CFMX7 levels.

I am afraid that if you want to use AES/CBC you will need to dig down into Java.

Here is a blog post I found on how to do it.

http://www.markomedia.com.au/aes-128-padded-encryptiondecryption-with-railo-java-and-as3/