Annotating An Image Point Using GraphicsMagick And Lucee CFML 5.2.9.31
Continuing my exploration of GraphicsMagick in Lucee CFML, this morning, I wanted to see how I might go about annotating an image. That is, labeling a point on the image with a marker and an annotation index. I suspect that this can be achieved in several ways; however, the first approach that jumps to mind involves drawing Circle and Text primitives using GraphicsMagick and Lucee CFML 5.2.9.31.
CAUTION: I am very much a novice when it comes to GraphicsMagick. As such, please take the following as an exploration, not an explanation.
In a previous demo, I looked at how to get the RGB value of a given pixel using GraphicsMagick. In that exploration, the user would click on the image in the browser; and then, I would send the {X,Y} coordinates of the click to the ColdFusion server, which would extract the RGB color value at the given location of the target image. For this demo, I want to use a similar approach; only, instead of reading a pixel, I'm going to annotate the image at the given offset.
And, to make things a bit more interesting, I want the user to be able to apply more than one annotation to the image. To do this, I'm going to use Lucee's special URL syntax that allows a collection of values to be spread over multiple query-string parameters:
?points[]=x,y&points[]=x,y&points[]=x,y
By suffixing the points query-string entry with array notation, [], it indicates to the ColdFusion server that all of the points[] query-string parameters should be collected into a single Array within the url scope. As such, making a request to the URL:
?points[]=1,1&points[]=99,99
... results in the following url.points value on the server:
[ "1,1", "99,99" ]
To leverage this behavior, every time the user clicks on the image within the browser, I'm just going to refresh the page with an additional points[] value. Then, on the server, I'm going to start with a base image and apply an individual annotation for each of the collected point coordinates.
Each annotation will be implemented using two -draw options: one for the circle and one for the label. Example:
-draw 'circle 50,50 50,60'-draw 'text 50,50 "Hello"'
Here's what I came up with:
<cfscript>
param name="url.points" type="array" default=[];
startedAt = getTickCount();
inputFilename = "demo.jpg";
inputFilepath = expandPath( "./#inputFilename#" );
// Copy the fresh image into the current directory - we always start with the fresh
// image even if we have multiple annotations to add. This way, we're not constantly
// adding additional compression artifacts to intermediary images.
fileCopy( "../images/face-square.jpg", "./demo.jpg" );
// If the user has provided a collection of points, annotate the image.
if ( url.points.len() ) {
// All of the image utilities are provided through the GraphicsMagick binary.
command = "gm";
// We're going to use the Convert utility to draw primitive shapes.
utility = "convert";
// We want to be EXPLICIT about which input reader GraphicsMagick should use.
// If we leave it up to "automatic detection", a malicious actor could fake
// file-type and potentially exploit a weakness in a given reader.
// --
// READ MORE: http://www.graphicsmagick.org/security.html
utilityReader = "jpg";
// Setup the initial options for the Convert utility.
commandOptions = [
utility,
// Provide the source image to be read with the an explicit reader.
( utilityReader & ":" & inputFilepath ),
];
// Add each point annotation to the Convert command.
// --
// NOTE: Some of configurations in the following Draw commands could be factored
// out to reduce repetition. However, in order to make the code a bit more
// simple, I'm just repeating configurations as needed such that all of the code
// is collocated.
url.points.each(
( point, i ) => {
var x = point.listGetAt( 1 );
var y = point.listGetAt( 2 );
var radius = 19;
commandOptions.append( "-fill '##ff3366'" );
commandOptions.append( "-stroke '##ffffff'" );
commandOptions.append( "-strokewidth 2.5" );
commandOptions.append( "-draw 'circle #x#,#y# #( x + radius )#,#y#'" );
// We have to manually center the text within the circle.
var textX = ( x - 4 );
var textY = ( y + 6 );
// If the stringified annotation index has more than one digit, nudge the
// text a little further to the left to try and keep it centered within
// the circle.
if ( len( i ) > 1 ) {
textX -= 5;
}
commandOptions.append( "-font 'Helvetica-Bold'" );
commandOptions.append( "-pointsize 17" );
commandOptions.append( "-fill '##ffffff'" );
commandOptions.append( "-stroke 'transparent'" );
commandOptions.append( "-draw 'text #textX#,#textY# ""#i#""'" );
}
);
commandOptions.append( "-quality 90" );
commandOptions.append( inputFilepath );
// Execute GraphicsMagick on the command-line.
execute
name = command
arguments = commandOptions.toList( " " )
variable = "successResult"
errorVariable = "errorResult"
timeout = 5
;
// If the error variable has been populated, it means the CFExecute tag ran into
// an error - let's dump-it-out and halt processing.
if ( variables.keyExists( "errorVariable" ) && errorVariable.len() ) {
dump( errorVariable );
abort;
}
}
duration = ( getTickCount() - startedAt );
</cfscript>
<cfoutput>
<p>
<!--
This is our annotated image - as the user clicks on it, we will reload the
page to add additional annotations.
-->
<img
id="image"
src="./#inputFilename#"
width="600"
height="584"
style="cursor: pointer ;"
/>
</p>
<p>
<a href="./index.cfm">Reset image</a>
</p>
<p>
Duration: #numberFormat( duration )# ms
</p>
<script type="text/javascript">
// When the user clicks anywhere on the demo image, we want to translate the
// viewport coordinates to image-local coordinates and then refresh the page
// with the additional offset.
document.getElementById( "image" ).addEventListener(
"click",
( event ) => {
// Get viewport coordinate data.
var viewportX = event.clientX;
var viewportY = event.clientY;
var imageRect = event.target.getBoundingClientRect();
// Translate viewport coordinates to image-local coordinates.
var localX = ( viewportX - imageRect.left );
var localY = ( viewportY - imageRect.top );
// We're going to be passing each point as an additional entry in the
// points collection on the query-string.
var nextHref = ( window.location.search )
? ( window.location.href + "&" )
: ( window.location.href + "?" )
;
// NOTE: Using special array-notation for a single query-string parameter
// that is spread across multiple URL entries.
nextHref += `points[]=${ Math.round( localX ) },${ Math.round( localY ) }`;
window.location.href = nextHref;
}
);
</script>
</cfoutput>
In order to keep the demo as simple as possible, I'm repeating all of the configuration options in each iteration of my points.each() loop. In reality, I could have factored-out some of the options, like -font and -pointsize; however, for the sake of readability, I liked having all of the configuration objects collocated within the code.
As you can see from the annotation logic, I'm manually centering the text within the annotation circle. This is unfortunate; however, I believe it's the only way to do this with my selected approach. In a subsequent exploration, I might try writing each annotation to an intermediary image using the -gravity center option; and then, overlay that annotation image onto the base image. But, I haven't actually tried that yet - one exploration at a time.
That said, if we run this Lucee CFML code and click around on the image, we get the following output:
Getting the right configurations for the various -draw options required a lot of trial-and-error. But, in the end, I think I came up with a workable solution. I think there are other ways to annotate images using GraphicsMagick. But, this it the first approach I wanted to try.
Want to use code from this post? Check out the license.
Reader Comments
@All,
I should add that in order to find a usable font,
Helvetica-Bold, I had to use the following command:gm convert -list fontThis gave me the following output:
I assume it's possible to install other fonts. I don't know how to do that; so, I just tried a few of the fonts in the given list until I found one that worked well-enough.
where/how do you find these hidden (to me) gems such as the special "points[]" notation? Very cool.
Also, my instinct would have been to create a new image with the same dimensions as the original (transparent png more than likely) and add all the points to that, then absolutely position it over the top of the original. This way, you could easily toggle the points on/off.
@Chris,
Ha ha, they hid it away in the "hidden gems" portion of the docs:
https://docs.lucee.org/guides/cookbooks/Hidden_gems.html
I think the secondary image makes sense. Tomorrow morning, I'm going to try writing the markers themselves to tiny individual files, and then gluing them altogether. Similar idea, but ultimately I end up with a single file. It will be interesting cause I'll need the markers to be in PNG (so they have alpha-transparency).
@Ben
They're super sneaky, hiding them in plain sight like that!
Looking forward to the next evolution :)
@All,
As I think I mentioned somewhere in here, I had planned to do a follow-up exploration where I generate the annotations as separate files in an attempt to use the "gravity" to center the text better. This ended up having two issues:
One -- The annotation images were going to have a transparent canvas. But, it seems that drawing primitives on a transparent canvas causes issues wherein the partially-transparent pixels end-up generating a "halo" effect. This is touched on here:
There is some discussion on getting around this; but, most of the technical details around alpha-channels and matte options went way over my head.
Two -- it turns out that using "gravity center" really didn't center the text all that much better than me just eye-balling it and adjusting based on the number of digits in the annotation.
Three -- it's way slower creating intermediary images.
Altogether, I'm going to abandon that thought-experiment. It didn't seem like it was going to be a value-add in the long run.
@All,
I just posted a follow-up exploration where, instead of plainly annotating the image, I actually create a fixed-size "clipping" of the image that attempts to center the annotation:
www.bennadel.com/blog/3783-centering-an-image-annotating-using-graphicsmagick-and-lucee-cfml-5-2-9-31.htm
In that post, I'm trying to recreate the comment-notification header that we use at InVision.