Using ColdFusion Custom Tags To Create An HTML Email DSL In Lucee CFML 5.3.7.47, Part XV
Rolling my ColdFusion custom tag DSL (Domain Specific Language) for HTML emails out at InVision for transactional emails has been fairly straightforward. The content of those emails are mostly static; and, the dynamic bits are highly predictable (just names and URLs). However, now that I want to try applying this DSL approach to my blog, I'm suddenly faced with User-Generated Content (UGC). When interacting with my blog, readers can use Markdown to format their comments, which then get published - via email - to subscribers. As such, if I use my DSL for those comment emails, I now need to figure out how to merge that user-generated content alongside my static transactional content.
View this code in my ColdFusion Custom Tag Emails project on GitHub.
Option One: Do Nothing
The easy option is to just do nothing. After all, I've been sending out comment emails for the last 15-years with no special handling of user-generated content (UGC) and life has been fine. The comment content is coming through as valid HTML; and, every email client will do an "OK" job of rendering that HTML using some default stylesheet.
So, even if I start rendering my comment emails using the ColdFusion custom tag DSL, I could easily just include the comment content - as is - and call it a day.
Option Two: Wrap it in a Container and Use a Style Tag
Another option which would get me a bit more mileage is to wrap the user-generated content (UGC) in a <div> tag with a class name; and then, use a <style> tag to override all of the styling within that container. So, imagine that part of my transactional email pulled-in the UGC like this:
<div class="user-generated-content">
<cfoutput>#commentHtml#</cfoutput>
</div>
Then, in my Email <head> - using the <core:HeaderStyles> DSL tag - I could target that .user-generated-content class and attempt to override all of the styling within that container:
<core:HeaderStyles>
.user-generated-content h1,
.user-generated-content h2,
.user-generated-content h3,
.user-generated-content h4,
.user-generated-content h5,
.user-generated-content p,
.user-generated-content ol,
.user-generated-content ul,
.user-generated-content blockquote {
margin-bottom: 16px ;
margin-top: 0px ;
}
.user-generated-content blockquote *:last-child {
margin-bottom: 0px ;
}
.user-generated-content h1,
.user-generated-content h2,
.user-generated-content h3,
.user-generated-content h4,
.user-generated-content h5 {
color: #333333 ;
font-family: garamond, georgia, serif ;
font-size: 18px ;
font-weight: 700 ;
line-height: 25px ;
}
.user-generated-content p,
.user-generated-content li,
.user-generated-content blockquote {
color: #333333 ;
font-family: helvetica, arial, sans-serif ;
font-size: 18px ;
font-weight: 400 ;
line-height: 25px ;
}
.user-generated-content a {
color: #ff3366 ;
}
.user-generated-content strong,
.user-generated-content b {
font-weight: 600 ;
}
.user-generated-content em,
.user-generated-content i {
font-style: italic ;
}
</core:HeaderStyles>
This approach actually gets me pretty far. According to the Campaign Monitory CSS support for <style> tags, most email clients support the <style> tag as long as it's within the <head> tag. In fact, all the email clients that I "care about" support this. As such, this is definitely a viable option.
Option Three: Transpile the HTML into the ColdFusion Custom Tag DSL
The final option (that I can think of) is to actually take the user-generated content (UGC) and transpile it into the ColdFusion custom tag DSL. Meaning, take HTML tags like:
<p> Hello world </p>
... and transpile them into:
<html:p> Hello world </html:p>
... wherein they would automatically pick up all the magic and styling that the DSL has to offer.
It took me about a week to get this approach working in my DSL code repository; but, I think I have something that is finally working well. Unfortunately, it is a Lucee CFML only approach at this time because it uses very Lucee-specific features:
I parse the user-generated content into an XML DOM using
htmlParse().I use Tag Islands to invoke my DSL tags inside CFScript so as to exert fine-tuned control over whitespace.
To this in action, here's Example 15 from my code repository - note that in the middle of this Email there is a custom tag that wraps the non-DSL content:
<!--- Import custom tag libraries. --->
<cfimport prefix="core" taglib="./core/" />
<cfimport prefix="html" taglib="./core/html/" />
<cfimport prefix="ex15" taglib="./ex15/" />
<!--- // ------------------------------------------------------------------------- // --->
<!--- // ------------------------------------------------------------------------- // --->
<core:Email
subject="Integrating user-generated content"
teaser="Yikes!">
<core:Body>
<html:h1>
Integrating user-generated content
</html:h1>
<html:p>
This ColdFusion custom tag DSL depends on the use of <html:strong>custom tags</html:strong>
instead of <html:strong>raw HTML tags</html:strong> in order to generate email-friendly
markup and styling. As such, integrating user-generated content alongside our
DSL is a bit challenging.
</html:p>
<!---
Note that the following block of HTML markup is just that - HTML - not the
custom tags DSL. But, the "UserGeneratedContent" tag will transpile the HTML
content into DSL content and output the results.
--->
<ex15:UserGeneratedContent>
<p>
Hey, I <em>really like</em> this article. But, have you seen this other
article on the same topic: <a href="https://www.bennadel.com/">Some cool things</a>.
I particularly liked this quote:
</p>
<blockquote>
<p>
Guess what?
</p>
<p>
Chicken butt!
</p>
</blockquote>
<h3>
Some other interesting points:
</h3>
<ul>
<li>This thing</li>
<li>That thing</li>
<li>The other thing</li>
</ul>
<h4>
A cool <code>code</code> snippet:
</h4>
<pre><code language="text">This is some code!
Code is fun!</code></pre>
</ex15:UserGeneratedContent>
<html:p>
This is going to be interesting.
</html:p>
</core:Body>
</core:Email>
As you can see, the HTML markup inside of <ex15:UserGeneratedContent> is just that: HTML markup. But the UserGeneratedContent.cfm tag will take that HTML and transpile it into ColdFusion custom tag DSL markup.
As I mentioned above, I'm doing this by parsing the HTML into an XML DOM (Document Object Model). I then recursively traverse the XML DOM, invoking various DSL tags for each HTML element node that I visit:
<!--- Import custom tag libraries. --->
<cfimport prefix="core" taglib="../core/" />
<cfimport prefix="html" taglib="../core/html/" />
<!--- // ------------------------------------------------------------------------- // --->
<!--- // ------------------------------------------------------------------------- // --->
<cfswitch expression="#thistag.executionMode#">
<cfcase value="end">
<cfoutput>
#translateHtmlContent( thistag.generatedContent )#
<!--- Reset the generated content since we're overriding the output. --->
<cfset thistag.generatedContent = "" />
</cfoutput>
</cfcase>
</cfswitch>
<!--- // ------------------------------------------------------------------------- // --->
<!--- // ------------------------------------------------------------------------- // --->
<cfscript>
/**
* I translate the given USER GENERATED CONTENT (UGC) HTML into the output that would
* have been generated had the user been using the equivalent DSL tags.
*
* CAUTION: This translation DOES NOT PROVIDE any additional SANITIZATION. All
* security measures are assumed to have already been taken; and, that the given HTML
* can be displayed as-is in the case of a parsing error fallback.
*
* @htmlContent I am the user generated content (UGC) to translate.
*/
public string function translateHtmlContent( required string htmlContent ) {
try {
// To perform the translation, we're going to parse the HTML content into a
// Document Object Model (DOM) - really just an XML document in this case -
// and then traverse the DOM in order to REBUILD it using the most relevant
// DSL tags.
var bodyNode = htmlParse( htmlContent )
.search( "//*[ local-name() = 'body' ]" )
;
savecontent variable = "local.translatedContent" {
translateNode( bodyNode.first(), true );
}
return( translatedContent );
} catch ( any error ) {
// If anything goes wrong in the parsing / translation, just fallback to
// using the raw content. It may not be formatted as nicely, but it should
// still render in an HTML email.
return( htmlContent );
}
}
/**
* I translate the given DOM (XML) node into a properly formatted HTML string by
* re-creating it with the most appropriate DSL tags.
*
* CAUTION: Since we are assuming that the user generated content (UGC) has already
* been run through some sort of sanitization routine (like AntiSammy), we're going to
* further assume that we are only dealing with a subset of HTML tags which have been
* allow-listed during processing.
*
* @node I am the DOM node being translated and serialized.
*/
public void function translateNode(
required xml node,
required boolean allowBottomMargin
) {
var tagAttributes = {
class: "user-generated-content",
style: ( node.xmlAttributes.style ?: "" )
};
if ( node.xmlAttributes.keyExists( "class" ) ) {
tagAttributes.class &= " #node.xmlAttributes.class#";
}
// All default margins on block elements are normalized to use bottom margins
// only. These defaults are pulled from the email-wide configurations. As such,
// if we need to suppress a bottom-margin, all we have to do is set the margins
// to "none" and this will automatically remove the bottom margin.
if ( ! allowBottomMargin ) {
tagAttributes.margins = "none";
}
// CAUTION: We're using TAG ISLANDS to exert fine-tunes control over whitespace
// output within our content translation. Notice that some back-ticks are on the
// same line (for inline elements) and some are new lines (for block elements).
switch ( node.xmlName ) {
case "a":
tagAttributes.href = ( node.xmlAttributes.href ?: "" );
```<html:a attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:a>```
break;
case "blockquote":
```
<html:blockquote attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes, false )#</cfoutput>
</html:blockquote>
```
break;
case "body":
// This is the ROOT container for the user-generated content.
```
<core:HtmlEntityTheme entity="td" class="user-generated-content-wrapper">
border: 1px solid #262626 ;
padding: 16px 20px 16px 20px ;
</core:HtmlEntityTheme>
<html:table width="100%" cellpadding="10">
<html:tr>
<html:td class="user-generated-content-wrapper">
<cfoutput>#translateChildNodes( node.xmlNodes, false )#</cfoutput>
</html:td>
</html:tr>
</html:table>
```
break;
case "code":
```<html:code attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:code>```
break;
case "div":
```
<html:div attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:div>
```
break;
case "em":
case "i":
```<html:em attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:em>```
break;
case "h1":
```
<html:h1 attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:h1>
```
break;
case "h2":
```
<html:h2 attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:h2>
```
break;
case "h3":
```
<html:h3 attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:h3>
```
break;
case "h4":
```
<html:h4 attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:h4>
```
break;
case "h5":
```
<html:h5 attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:h5>
```
break;
case "hr":
```
<html:hr attributeCollection="#tagAttributes#" />
```
break;
case "li":
```
<html:li attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes, false )#</cfoutput>
</html:li>
```
break;
case "ol":
```
<html:ol attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:ol>
```
break;
case "p":
```
<html:p attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:p>
```
break;
case "pre":
```<html:pre attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:pre>```
break;
case "span":
```<html:span attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:span>```
break;
case "strike":
```<html:strike attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:strike>```
break;
case "strong":
case "b":
```<html:strong attributeCollection="#tagAttributes#"><cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput></html:strong>```
break;
case "ul":
```
<html:ul attributeCollection="#tagAttributes#">
<cfoutput>#translateChildNodes( node.xmlNodes )#</cfoutput>
</html:ul>
```
break;
// If we don't have an equivalent ColdFusion custom tag, just pass-through
// the given tag as-is. We're going to rely on the email client to be able
// to render it "good enough".
default:
```<cfoutput><#node.xmlName#>#translateChildNodes( node.xmlNodes )#</#node.xmlName#></cfoutput>```
break;
}
}
/**
* I translate the given DOM (XML) nodes into a properly formatted HTML string by
* re-creating them with the most appropriate DSL tags.
*
* @nodes I am the DOM nodes being translated and serialized.
* @allowLastElementMargin I determine if the last Element node can have a bottom margin.
*/
public void function translateChildNodes(
required array nodes,
boolean allowLastElementMargin = true
) {
var elementCount = getElementCount( nodes );
var elementIndex = 0;
for ( var node in nodes ) {
switch ( node.getNodeType() ) {
case "TEXT_NODE":
echo( translateTextNode( node ) );
break;
case "ELEMENT_NODE":
// For element nodes, we need to determine if the given node is the
// last element in its parent container; and, if so, whether or not
// it's allowed to have a bottom margin.
var isLastElement = ( ++elementIndex == elementCount );
var allowBottomMargin = ( allowLastElementMargin || ! isLastElement );
echo( translateNode( node, allowBottomMargin ) );
break;
}
}
}
/**
* I translate the given XML text node into plain text.
*
* @node I am the XML node being serialized.
*/
public void function translateTextNode( required xml node ) {
echo( toString( node ).listRest( ">" ) );
}
/**
* I return the count of element-nodes within the given nodes collection.
*
* @nodes I am the collection of XML nodes being inspected.
*/
public numeric function getElementCount( required array nodes ) {
var total = 0;
for ( var node in nodes ) {
if ( node.getNodeType() == "ELEMENT_NODE" ) {
total++;
}
}
return( total );
}
</cfscript>
As you can see, the meat of this algorithm is a large switch statement that performs the HTML-to-DSL translation of element nodes. The case values essentially act as an allow-list of elements that the DSL knows how to handle. And, of course, if there's an element that I don't handle, the default case will just transcribe that element as-is while continuing to recursively traverse deeper into the XML DOM tree.
If we run this ColdFusion custom through Lucee CFML, we get the following output:
As you can see from the page-source, the user-generated HTML has all been rendered by the DSL, complete with the automatic style-cascading and the inlining of CSS properties.
I don't think that this is an approach that I could generalize for the core DSL implementation. Not only does it depend heavily on Lucee CFML specific features (ones that are not so easy to translate over to Adobe ColdFusion); but, there's also opportunity here to transcribe HTML elements as non-core DSL tags - and, I would never know what those should be.
ASIDE: Re: performing this transpilation in Adobe ColdFusion, I could perform the DOM parsing and traversal with something like jSoup; but, the whitespace management would be much more challenging. This is one reason by Tag Islands are so absurdly cool in Lucee CFML.
The great irony here is that while it took me a week to get this working, I probably won't even use it on my blog, which runs on Adobe ColdFusion 2018. In reality, I'll likely go with something like Option Two, which just wraps the user-generated content (UGC) in a container and then uses a <style> tag to override the visual aspects.
That said, I've been loving using the ColdFusion custom tag DSl to generate HTML emails at InVision. It's really made life a whole lot easier.
Epilogue on Security
Anytime you integrate user-generated content of any kind, you have to be security-minded. Bad actors are always looking for ways to encode, double-encode, and otherwise embed persisted XSS (Cross-Site Scripting) attacks in the content that they post. As such, you should never execute untrusted content.
Part of my assumption here (in this post) is that the user-generated content already passed-through something like the OWASP AntiSamy project, which allow-lists the content that can be rendered. Even so, you'll see that I am never actually "executing" any of the user-content (as if it were ColdFusion code) - I'm only ever treating it as plain-text.
Never trust your users!
Want to use code from this post? Check out the license.
Reader Comments
Love the tag islands. Great use case! And a very cleaver approach to solving the problem in general 👏
@Chris,
I really love the concept of Tag Islands in Lucee. It's a killer feature. I would have loved to have combined some of those cases into something more dynamic with the
CFModuletag, as in:But, in Lucee CFML, the
CFModuletag doesn't quite work the same way as the other custom tag invocation (which is a bug). It doesn't present the tag as one that can contain "data", so it messes up the way the DSL tags communicate. Which may or may not have actually caused any issues - it's unclear. The tag island approach felt like the safe approach.