Converting IP Addresses To And From Integer Values With ColdFusion
I've been playing with my bits a lot lately, parsing RGB colors and embedding secret messages inside image data; but, bit manipulation is still something that feels somewhat awkward to me. I guess I live in a base10 (decimal) world and it's hard for me to think in terms of base2 (binary). As such, when a conversation about IP-to-Integer conversion popped up on my InputBaseN() / FormatBaseN() post, I figured it would be a great opportunity for me to strengthen my bit-manipulation skills. I know converting an IP address to an integer (and back) is something that's been done a million times over; but for me, personally, it's new and ripe for practice.
In an IP to Integer conversion, we want to take an IP address string value, such as "70.112.108.147", and convert it into an integer value, such as 1181772947. Typically, this is done for storage and comparison purposes - I'm told it's both easier and faster to store and compare two integer values than it is two string values. Database philosophy aside, though, this value conversion is done in a bit-wise manner.
Before we get into the code, let's think about what needs to be done. Each part (octet) of an IP address consists of a single number whose value ranges between 0-255. Furthermore, each decimal number (in general) can be represented in binary as a string of bits. When we combine the individual parts of an IP address, we have to form a single integer value; but, we have to do this in such a way that none of the underlying bits overlap.
As you can see, our final integer value is not much more than a glorified bit-mask in which each "flag" is represented, not by a single bit, but rather by 8 bits. To make sure that the "flags" (IP octet values) remain intact during the conversion, we have to shift them over by multiples of 8 as we add them together.
Now that we see what we're trying to do, let's take a look at the ColdFusion code. In this first demo, we're going to convert an IP address into an integer:
<!--- Set the IP address. --->
<cfset ipAddress = "70.112.108.147" />
<!--- Break the IP address into numeric units. --->
<cfset ipParts = listToArray( ipAddress, "." ) />
<!--- Create a running total for our numeric IP equivalent. --->
<cfset ipNumber = 0 />
<!---
Loop over the parts of the array. For each part, we are going
to let the 8 bits be continually shifted over to add exclusive
bits to the running total.
--->
<cfloop
index="offset"
from="1"
to="#arrayLen( ipParts )#"
step="1">
<!---
Since each IP unit is a max of 255, we need to shift it
over (bit-wise) for multiples of 8.
--->
<cfset ipNumber += bitSHLN(
ipParts[ offset ],
((arrayLen( ipParts ) - offset) * 8)
) />
</cfloop>
<!--- Output the resultant IP number equivalent. --->
IP Address: #ipAddress#<br />
IP Number: #ipNumber#
As you can see, we have a running total to which we are adding each IP octet value. As we add the octet value, however, we are using ColdFusion's bitSHLN() method to perform a left-shift of the bits as explained in the diagram above. When we run the above code, we get the following output:
IP Address: 70.112.108.147
IP Number: 1181772947
To go back the other way - convert an Integer to an IP Address - we basically do the above, but in reverse. Rather than shifting left, we'll shift right; rather than adding, we'll bitAnd().
<!--- Set the IP numeric equivalent. --->
<cfset ipNumber = 1181772947 />
<!---
Create an array to hold the parts of the IP address as we
parse them out of the interger equivalent.
--->
<cfset ipParts = [] />
<!---
Now, let's keep shifting the IP integer right by 8 bits
(the number of bits required for 255) until we have nothing
left to shift over.
--->
<cfloop condition="val( ipNumber )">
<!---
At this point, the next value we want to get is in the
last 8 bits of the number. To get at it, we can bitAnd()
with 255, which is the bit configuration, 11111111.
NOTE: Since we are getting the right-most IP units first,
we are going to PREpend the values to our IP array.
--->
<cfset arrayPrepend(
ipParts,
bitAnd( ipNumber, 255 )
) />
<!---
Now that we have gotten the right-most bits, let's shift
the number right by 8 bits. This will put the next IP
unit we want to get in the last 8 bits of the number.
--->
<cfset ipNumber = bitSHRN( ipNumber, 8 ) />
</cfloop>
<!--- Output the parsed IP address. --->
IP Address: #arrayToList( ipParts, "." )#<br />
As you can see, with each right-shift of the bits using ColdFusion's bitSHRN() function, the next octet becomes the right-most 8 bits of the running "unTotal". We then extract those right-most bits by bitAnd()'ing the value with 255. When we run the above code, we get the following output:
IP Address: 70.112.108.147
As you can see, we were able to convert the IP address to an integer and then back to an IP address.
While this worked well, it really took ColdFusion to the limits of its integer capabilities. If we needed to left-shift just one or two more times, we would have required the creation of an integer too large for ColdFusion's (and Java's) basic data types. I don't know much of anything about the next generation of IP addresses - IPv6 - but I do know that it will require more bits than an Integer value will comfortably work with. As such, I thought it would be a fun experiment to perform this task again, this time with a few additional octets.
Because Java's int data type only allows for 32 bits, it means that ColdFusion, which is built on top of Java, also only allows for 32-bit integers. When dealing with extra octets, and subsequently extra bits, we'll need to go beyond basic data types. Luckily, Java provides for such cases with classes like BigInteger. In the following demos, all bit manipulation and addition will need to be done through BigInteger instances.
First, let's convert our IP address to an integer:
<!--- Set the LARGER address. --->
<cfset ipAddress = "70.112.108.147.123.123" />
<!--- Break the IP address into numeric units. --->
<cfset ipParts = listToArray( ipAddress, "." ) />
<!---
Create a running total for our numeric IP equivalent.
Because our running total must hold the VERY large value,
we need to use Java's BigInteger.
--->
<cfset ipNumber = createObject( "java", "java.math.BigInteger" )
.init( javaCast( "string", "0" ) )
/>
<!---
Loop over the parts of the array. For each part, we are going
to let the 8 bits be continually shifted over to add exclusive
bits to the running total.
--->
<cfloop
index="offset"
from="1"
to="#arrayLen( ipParts )#"
step="1">
<!---
Since each IP unit is a max of 255, we need to shift it
over (bit-wise) for multiples of 8. However, since the
amount we are shifting by can be very large, we need to
create an intermediary BigInteger to hold this value.
--->
<cfset ipShift = createObject( "java", "java.math.BigInteger" )
.init( javaCast( "string", ipParts[ offset ] ) )
/>
<!---
Shift the current value by multiples of 8 bits and add
the shifted value to the running total.
--->
<cfset ipNumber = ipNumber.add(
ipShift.shiftLeft(
javaCast(
"int",
((arrayLen( ipParts ) - offset) * 8)
)
)
) />
</cfloop>
<!--- Output the resultant IP number equivalent. --->
IP Address: #ipAddress#<br />
IP Number: #ipNumber.toString()#
As you can see this time, our IP address has two more octets: "123.123". Because of this, we need to play with more bits than a standard int value can hold. Java's BigInteger class allows for very large integer manipulation with a fairly easy API. Notice that when we create a BigInteger class instance, we have to initialize it with a string value rather than a numeric value; this is due to the fact that the initial value might be larger than an int data type can hold. Likewise, when we retreive the value from a BigInteger instance, we have to retreive it as a string. When we run the above code, we get the following output:
IP Address: 70.112.108.147.123.123
IP Number: 77448671886203
Going back the other way - Integer to IP Address - we need to do the above in reverse:
<!---
Set the IP numeric equivalent. Because this is such a huge
number, we are going to have to use Java's BigInteger to
model and mutate it.
--->
<cfset ipNumber = createObject( "java", "java.math.BigInteger" )
.init( javaCast( "string", "77448671886203" ) )
/>
<!---
Since big integers require other big intergers for
mathematical forumuls, let's create one to represent 255. This
will be ANDed with the IP number as we parse it.
--->
<cfset bigInt255 = createObject( "java", "java.math.BigInteger" )
.init( javaCast( "string", "255" ) )
/>
<!---
Create an array to hold the parts of the IP address as we
parse them out of the interger equivalent.
--->
<cfset ipParts = [] />
<!---
Now, let's keep shifting the IP integer right by 8 bits
(the number of bits required for 255) until we have nothing
left to shift over.
--->
<cfloop condition="val( ipNumber.toString() )">
<!---
At this point, the next value we want to get is in the
last 8 bits of the number. To get at it, we can bitAnd()
with 255, which is the bit configuration, 11111111.
--->
<cfset arrayPrepend(
ipParts,
ipNumber.and( bigInt255 ).toString()
) />
<!---
Now that we have gotten the right-most bits, let's shift
the number right by 8 bits. This will put the next IP
unit we want to get in the last 8 bits of the number.
--->
<cfset ipNumber = ipNumber.shiftRight(
javaCast( "int", 8 )
) />
</cfloop>
<!--- Output the parsed IP address. --->
IP Address: #arrayToList( ipParts, "." )#<br />
To get the right-most 8 bits of the BigInteger value, we need to bit-AND it with 255. While 255 could easily fit into an int data type, since all math performed with a BigInteger needs to be done with BigIntegers, we needed to create a BigInteger representation of 255. Other than this new data type, the overall algorithm is practically unchanged.
Now that I've had to do this and explain it, I'm starting to feel much more comfortable with bit-wise manipulation. I know that there are easier ways to perform IP-to-Integer conversions (such as multiplying by powers of 255); but, I think that doing it this way, with explicit bit-shifting, really gets you to think about the underlying mechanics of what is taking place.
Want to use code from this post? Check out the license.
Reader Comments
Hi,
The problem with your first solution is that for any IP adresse in the upper half of the IPv4 pool (> 128.0.0.1) coldfusion will answer with a negative number, ex:
IP Address: 170.112.108.147
IP Number: -1435472749
Coldfusion bit operations are working on 32 bits signed integer, while it's possible to build bigger integer (up to 40 bits) using standard multiply operation
As for IPv6, it a 128 bits integer, usually coded in the hexadecimal form, of height 16bits blocks:
1fff:0000:0a88:85a3:0000:0000:ac1f:8001
It's easier to convert to decimal form: just remove the ':' and convert from hex to decimal (well, with a system supporting 128 integers obviously).
Thumbs up for creativity ;)
Always did this in more conventional way, with reg. expressions and listGetAt.
@Silmaril,
I am not sure what you are saying about the negative number. Remember, we don't really *care* what the integer is - it's just a collection of bits. From an INT standpoint, the left-most bit might be for signing; but, from a bit-mask standpoint, it's just another bit that can be shifted.
I tried plugging in 170 as the first octet and got the following results:
IP Address: 170.112.108.147
IP Number: -1435472749
Then, going from -1435472749 back to an IP address, I got the following:
IP Address: 170.112.108.147
As you can see, the bit-wise manipulation was not affected by the use of the signed bit.
As for the IPv6, I tried looking them up last night, but I didn't read too much on them.
@Ben
Yes indeed it's reversible, but the main use of decimal based IP addresse is for comparaison for systems that cannot directly compare IP addresses, and while 128.0.0.1 is between 127.0.0.1 and 129.0.0.1, it will be harder to verify using a signed integer
@Silmaril,
Ahh, I see what you're saying. Sorry - I'm not too familiar with the database concepts behind converting IP to integers; I was just using this as an experiment in bit manipulation. I'll have to learn up on that a bit - thanks.
That should be: (such as multiplying by powers of 256)
@Gary,
Ah good catch - I wasn't sure about that. I'll have to do some learning on that.
Hi Ben. My programming knowledge a very low but i would try to understand why this is better than a string. my questen is, is this only for better performance? is this only for coldfusion or else php, c++ and so on?
@Carsten,
Bit-wise manipulation, and this general approach, should be available in just about any language. In this post, I used ColdFusion, but then, I dipped down into Java to use the BigInteger class. I would assume that C++ has something similar. As for PHP, it definitely has bit manipulation - whether or not it can handle HUGE integers, I don't know.
I think the use of numbers is for performance and, @Silmaril mentioned, the numbers allow you to compare ranges of IP address (something that you cannot easily do with string representations).
Just as a footnote, Ben, MySQL has two handy functions: INET_ATON() to convert in IP string to int, and INET_NTOA() to go the other way.
Saving IPs as ints rather than varchars can significantly reduce the storage size of a table with a lot of rows. Access logging tables where you're capturing visitor IPs are a prime candidate.
@Julian,
That is awesome! Thanks for letting me know about that.
@Julian,
Thanks again, I did a little experimentation with these functions - they work great.
www.bennadel.com/blog/1833-Converting-An-IP-Address-To-An-Integer-Using-MySQL-Thanks-Julian-Halliwell-.htm
Hi Ben, thank you for your article which really helped me a lot. I implemented a similar solution using T-SQL so, for reference, and if you don't mind, I'll add a link to it here. It may be useful to people in the future.
https://gist.github.com/simonbingham/5000258
@Simon,
Very cool :)
Here's a much easier and shorter way (one line) to convert an IPv4 address to a numeric value...
If you want to convert this (assume it's store in variable "ip"): "12.48.40.0"
Just do this:
(ListGetAt(ip, 4, ".") + ListGetAt(ip, 3, ".") * 256 + ListGetAt(ip, 2, ".") * 256 * 256 + ListGetAt(ip, 1, ".") * 256 * 256 * 256)
Hi Ben, may I know how to deal with IPv6 in ColdFusion 8 in term of the checking for which version of IP that the client using, how to convert it to the IP number and so on.
Thanks