Using Module Augmentation To Safely Inject Runtime Methods Using TypeScript And Node.js
JavaScript's prototypal inheritance mechanism was designed to be modified at runtime. But, as a community, we've grown to look down upon this concept, condemning it as unsafe and leading to unpredictable behavior. But, it doesn't have to be unsafe or unpredictable. If we take a cue from the RxJS library, I think we can safely alter existing modules - both native and custom - by using explicit, side-effect-driven "module augmentation" in the calling contexts for which the new runtime behavior is desired.
When you import a module for its side-effects, it means that you're not importing it to access an exported value. Instead, you're importing the module because it will, in turn, alter the state of the application. This is how the RxJS library progressively loads Observable and Operator functionality as your application needs it. For example, if your application needs to use the .fromEvent() RxJS Observable, you import the Observable in order to expose the desired behavior:
import "rxjs/add/observable/fromEvent";
Notice that we're not tracking the imported value. This is because we only care about the import side-effects, which, in this case, adds the .fromEvent() static method to the Observable constructor. By putting this import statement in each module that needs the .fromEvent() method, not only does it clearly document the source of the method, it also creates a more cohesive module that can be easily tested in isolation (since it doesn't depend on some unseen "bootstrap" file to ensure the expected behavior).
This approach works very well for RxJS; so, I wanted to see how well it would work in Node.js for other modules, both native and custom. And, what's more, seeing how it would work with TypeScript. It's one thing to alter behavior at runtime; it's an entirely different matter to do it an way that facilitates type-safety and static code analysis.
To explore this topic, I wanted to try the following:
- Add an instance method to the native Array prototype.
- Add a static method to the native Array constructor.
- Add an instance method to a custom Logger prototype.
- Add a static method to a custom Logger constructor.
Each one of these module augmentations will be driven by a side-effect import statement. Each import is clearly defined and adds a single new behavior to one of the existing modules. First, let's look at the calling context to see how it all fits together:
// Import our application modules.
import { Logger } from "./logger";
// Import these modules for their side-effects. These modules are going to augment
// other modules, which would normally be unpredictable and mysterious. However, by
// using local imports to identify the expected augmentations, it makes them
// predictable - it removes the Fear, Uncertainty, Doubt (FUD).
import "./add/array/isEmpty";
import "./add/array/prototype/filterMap";
import "./add/array/prototype/flatMap";
import "./add/logger/isLogger";
import "./add/logger/prototype/info";
// ----------------------------------------------------------------------------------- //
// ----------------------------------------------------------------------------------- //
var names = [ "Sarah", "Joanna", "Kit", "Gina" ];
// Use the "module augmentation" method .filterMap() which will only include defined
// values in the resultant collection.
names = names.filterMap(
( name: string ) : string => {
if ( name !== "Gina" ) {
return( name.toUpperCase() );
}
// NOTE: If we made it this far, the method will implicitly return "undefined"
// which will cause the input to be excluded from the final results.
}
);
// Use the "module augmentation" method .flatMap() which will merge arrays into the
// resultant collection.
names = names.flatMap(
( name: string ) : string[] | string => {
if ( name !== "JOANNA" ) {
return( name );
} else {
return( [] );
}
}
);
// ----------------------------------------------------------------------------------- //
// ----------------------------------------------------------------------------------- //
var logger = new Logger();
logger.log( `Names: ${ names.join( ", " ) }` );
// Use the "module augmentation" method .isEmpty() to see if the collection is empty.
logger.log( `Empty: ${ Array.isEmpty( names ) }` );
// Use the "module augmentation" methods .info() and .isLogger().
logger.info( "Hello there" );
logger.info( `Is logger: ${ Logger.isLogger( logger ) }` );
As you can see, at the top of this module, I have several side-effect-only import statements:
- import "./add/array/isEmpty";
- import "./add/array/prototype/filterMap";
- import "./add/array/prototype/flatMap";
- import "./add/logger/isLogger";
- import "./add/logger/prototype/info";
The first three imports add runtime methods to the native Array object; and, the last two imports add methods to the custom Logger class. These methods are then consumed later-down in the control-flow of this module. Now, as a developer, if I'm looking at this code and wondering where in the heck a .flatMap() method on the Array instance came from, it becomes [more] clear that this was a runtime modification that our module explicitly asked for.
Now, if we run this code through ts-node, we can see that it worked beautifully:
As you can see, it worked perfectly. The side-effect driven import statements explicitly altered the runtime behavior of the Array and Logger classes, which our module then consumed in a safe and predictable fashion. No fear, no uncertainty, no doubt - no FUD.
In this experiment, I used nodemon to re-run my TypeScript code on every change because this demo took me over 6-hours to complete. It turns out, "module augmentation" in TypeScript is not exactly straightforward. It's easy to import code and monkey-patch it; it's hard to tell TypeScript what you're doing and how the compiler should analyze the rest of your application after the import has been performed.
The approach to module augmentation requires a slight chicken-and-egg workflow. First, we have to tell TypeScript that the target modules have the methods we're about to inject. Then, once the "declaration" of those modules has been updated, we can go about injecting the new methods that the modules should have according to their definition.
And, to make things more interesting, augmenting a global module (Array) is different than augmenting an imported module (Logger). And, injecting a static method, on the Constructor, requires a different approach than injecting an instance method on the prototype. This is why it took me so many hours to get this working. It was basically six hours of this:
- Tweak code.
- Does this compile?
- No.
- Tweak code.
- Does this compile?
- No.
- Tweak code.
- Does this compile?
- No.
- (repeat for three days)
The foundation of each approach is the same. We use "declaration merging" to merge new items into an existing definition. The tricky part was figuring out how to reference each module and what kind of declaration merging needed to be performed (ex, interface into interface, interface into Class, namespace into Class). And, on top of that, figuring out how to alter the definition and perform the method injection in the same file in such a way that would adhere to TypeScript's rules about "external modules."
I've tried to include a lot of comments in the following code; so, I'm just going to offer of the implementation without much additional information:
Injecting the static method, Array.isEmpty:
// If we want to augment one of the global / native modules, like Array, we have to use
// the special "global" module reference.
declare global {
// If we want to add STATIC METHODS to one of the native Classes, we have to
// "declaration merge" an interface into the existing class interface.
// --
// NOTE: Unlike application class augmentation, which uses a "namespace" merge to
// create static methods, native classes are modeled as Interfaces. As such, we
// have to use interface merging using one of the special Constructor interfaces.
interface ArrayConstructor {
isEmpty( collection: any[] ) : boolean;
}
}
// I determine if the given collection is empty (a silly example for the demo).
// --
// CAUTION: Augmentations for the global scope can only be directly nested in external
// modules or ambient module declarations. As such, we are EXPORTING this function to
// force this file to become an "external module" (one that imports or exports other
// modules).
export function isEmpty( collection: any[] ) : boolean {
return( collection.length === 0 );
}
// Protect against conflicting definitions. Since each module in Node.js is evaluated
// only once (and then cached in memory), we should never hit this line more than once
// (no matter how many times we include it). As such, the only way this method would
// already be defined is if another module has injected it without knowing that this
// module would follow-suit.
if ( Array.isEmpty ) {
throw( new Error( "Array.isEmpty is already defined - overriding it will be dangerous." ) );
}
// Augment the global Array constructor (ie, add a static method).
Array.isEmpty = isEmpty;
Injecting the instance method, Array.prototype.filterMap:
interface Operator<T, U> {
( value: T, index: number, values: T[] ) : U;
}
// If we want to augment one of the global / native modules, like Array, we have to use
// the special "global" module reference.
declare global {
// If we want to add INSTANCE METHODS to one of the native classes, we have
// to "declaration merge" an interface into the existing class.
interface Array<T> {
filterMap<T, U>( operator: Operator<T, U>, context?: any ) : U[];
}
}
// I map the contextual collection onto another collection by appending defined results
// of the operation onto the mapped collection. This means that "undefined" results will
// be omitted from the mapped collection.
// --
// CAUTION: Augmentations for the global scope can only be directly nested in external
// modules or ambient module declarations. As such, we are EXPORTING this function to
// force this file to become an "external module" (one that imports or exports other
// modules).
export function filterMap<T, U>( operator: Operator<T, U>, context: any = null ) : U[] {
var results = this.reduce(
( reduction: U[], value: T, index: number, values: T[] ) : U[] => {
var mappedValue = operator.call( context, value, index, values );
if ( mappedValue !== undefined ) {
reduction.push( mappedValue );
}
return( reduction );
},
[]
);
return( results );
};
// Protect against conflicting definitions. Since each module in Node.js is evaluated
// only once (and then cached in memory), we should never hit this line more than once
// (no matter how many times we include it). As such, the only way this method would
// already be defined is if another module has injected it without knowing that this
// module would follow-suit.
if ( Array.prototype.filterMap ) {
throw( new Error( "Array.prototype.filterMap is already defined - overriding it will be dangerous." ) );
}
// Augment the global Array prototype (ie, add an instance method).
Array.prototype.filterMap = filterMap;
Injecting the instance method, Array.prototype.flatMap:
interface Operator<T, U> {
( value: T, index: number, values: T[] ) : U[] | U;
}
// If we want to augment one of the global / native modules, like Array, we have to use
// the special "global" module reference.
declare global {
// If we want to add INSTANCE METHODS to one of the native classes, we have
// to "declaration merge" an interface into the existing class.
interface Array<T> {
flatMap<T, U>( operator: Operator<T, U>, context?: any ) : U[];
}
}
// I map the contextual collection onto another collection by spreading / merging the
// operation results into the mapped collection.
// --
// CAUTION: Augmentations for the global scope can only be directly nested in external
// modules or ambient module declarations. As such, we are EXPORTING this function to
// force this file to become an "external module" (one that imports or exports other
// modules).
export function flatMap<T, U>( operator: Operator<T, U>, context: any = null ) : U[] {
var results = this.reduce(
( reduction: U[], value: T, index: number, values: T[] ) : U[] => {
var mappedValue = operator.call( context, value, index, values );
return( reduction.concat( mappedValue ) );
},
[]
);
return( results );
};
// Protect against conflicting definitions. Since each module in Node.js is evaluated
// only once (and then cached in memory), we should never hit this line more than once
// (no matter how many times we include it). As such, the only way this method would
// already be defined is if another module has injected it without knowing that this
// module would follow-suit.
if ( Array.prototype.flatMap ) {
throw( new Error( "Array.prototype.flatMap is already defined - overriding it will be dangerous." ) );
}
// Augment the global Array prototype (ie, add an instance method).
Array.prototype.flatMap = flatMap;
Injecting the static method, Logger.isLogger:
// Import our application modules.
import { Logger } from "../../logger";
// ----------------------------------------------------------------------------------- //
// ----------------------------------------------------------------------------------- //
// If we want to augment one of our application modules, we have to tell TypeScript
// which module to augment by using the same path-resolution that will be used by the
// rest of the application modules.
declare module "../../logger" {
// If we want to add STATIC METHODS to one of the application Classes, we have
// to "declaration merge" a namespace into the existing class.
namespace Logger {
export function isLogger( target: any ) : boolean;
}
}
// I check to see if the given value is an instance of Logger.
// --
// NOTE: Since we already have an "import" statement at the top of this module, we don't
// technically need to export this function (in order to implicitly create an "external"
// module). However, I am exporting it to maintain symmetry with the other files.
export function isLogger( target: any ) : boolean {
return( target instanceof Logger );
}
// Protect against conflicting definitions. Since each module in Node.js is evaluated
// only once (and then cached in memory), we should never hit this line more than once
// (no matter how many times we include it). As such, the only way this method would
// already be defined is if another module has injected it without knowing that this
// module would follow-suit.
if ( Logger.isLogger ) {
throw( new Error( "Logger.isLogger is already defined - overriding it will be dangerous." ) );
}
// Augment the Logger constructor (ie, add a static method).
Logger.isLogger = isLogger;
Injecting the instance method, Logger.prototype.info:
// Import the core node modules.
import chalk = require( "chalk" );
// Import our application modules.
import { Logger } from "../../../logger";
// ----------------------------------------------------------------------------------- //
// ----------------------------------------------------------------------------------- //
// If we want to augment one of our application modules, we have to tell TypeScript
// which module to augment by using the same path-resolution that will be used by the
// rest of the application modules.
declare module "../../../logger" {
// If we want to add INSTANCE METHODS to one of the application Classes, we have
// to "declaration merge" an interface into the existing class.
interface Logger {
info( value: any ) : void;
}
}
// I log the given value to the console with log-level "info".
// --
// NOTE: Since we already have "import" statements at the top of this module, we don't
// technically need to export this function (in order to implicitly create an "external"
// module). However, I am exporting it to maintain symmetry with the other files.
export function info( value: any ) : void {
console.info( chalk.bold.cyan( "[INFO]" ), value );
}
// Protect against conflicting definitions. Since each module in Node.js is evaluated
// only once (and then cached in memory), we should never hit this line more than once
// (no matter how many times we include it). As such, the only way this method would
// already be defined is if another module has injected it without knowing that this
// module would follow-suit.
if ( Logger.prototype.info ) {
throw( new Error( "Logger.prototype.info is already defined - overriding it will be dangerous." ) );
}
// Augment the Logger prototype (ie, add an instance method).
Logger.prototype.info = info;
... and, so you can see what class .info() was being injected into, here's the base Logger class:
// Import the core node modules.
import chalk = require( "chalk" );
// ----------------------------------------------------------------------------------- //
// ----------------------------------------------------------------------------------- //
export class Logger {
// I log the given value to the console with log-level "log".
public log( value: any ) : void {
console.log( chalk.red.bold( "[LOG]" ), value );
}
}
As you can see, each side-effect-driven module alters the declaration for the target module (using either the special term "global" or the path at which the module would be resolved); then, goes about injecting the new method.
In this approach, I'm throwing an error if the desired method already exists on the target module. Since Node.js will resolve a module only once, and then cache it in memory, we know that our method injection will only execute once. Therefore, if the injected method already exists, it's likely due to another module that's attempting to perform similar runtime changes. Rather than allowing one change to overwrite another, I'm throwing an error since the entire purpose of the import-based approach is to create predictable and well-defined behavior. If we're in a situation that's not predictable, we should not proceed.
There's ample reason to perform runtime modifications to existing code. Shimming future versions of JavaScript; progressively loading functionality (as with RxJS); and, adding custom functionality to core classes (as with Array). The issue with runtime modification is never the modification itself; but, rather, the predictability and maintainability of the code. By using side-effect driven import statements, we can clearly document where the modifications come from and which modules depend on the exposed behavior. This should make monkey-patching code much more palatable.
Want to use code from this post? Check out the license.
Reader Comments
@All,
When I was banging my head against the wall trying to get this to compile, I came across this post by Oleg Varaksin:
https://medium.com/@OlegVaraksin/modern-way-of-adding-new-functionality-to-typescript-libraries-by-patching-existing-modules-6dcde608de56
Oleg is exploring the same topic and has some good examples. I actually learned the term "module augmentation" from him - I didn't realize it had a name.