Follow-Up: Creating Leaky Abstractions With RxJS In Angular 2.1.1
Over the weekend, I did some noodling on my own use of RxJS in Angular 2 and came to realization that I was creating a "leaky abstraction" by returning Http streams directly from my service layer. Ben Lesh - project lead for RxJS - read my post and characterized it as "promoting incorrect usage out of what seems to be a misunderstanding about observables." I'll be the first to tell you that my understanding of RxJS is incomplete, at best. But, I'm still not sold on the idea that my perspective is incorrect. As such, I wanted to try and noodle on the topic a bit more.
Run this demo in my JavaScript Demos project on GitHub.
When I talk about a "leaky abstraction" in the context of a Service class, I am heavily considering the "intent" of the author of that service class. If the author of the service class intends for an Http stream to be returned - and intends for all possible consequences of what that entails - then there is no leak. However, if the calling context can consume the Http stream in a way that the service author did not intend, then some aspect of the implementation has leaked out into the calling context.
So, to be clear, when I talk about creating a "leaky abstraction" with RxJS, I am talking specifically about services layers and how their usage aligns, or rather does not align, with how the author of the service layer intended their return values to be consumed. I am in no way making a blanket statement about RxJS in general.
To more clearly articulate the problem as I see it, I have created a Service method that does two things:
- Logs the method call for some sort of statsD / analytics / user-tracking analysis.
- Configures and returns an Http stream.
For the purposes of this demo, the "tracking" portion just logs to the console - my goal here is to showcase intent:
// Import the core angular services.
import { Http } from "@angular/http";
import { Injectable } from "@angular/core";
import { Observable } from "rxjs/Observable";
import { Response } from "@angular/http";
// Load modules for side-effects.
import "rxjs/add/observable/fromPromise";
import "rxjs/add/operator/map";
import "rxjs/add/operator/toPromise";
export interface IFriend {
id: number;
name: string;
}
@Injectable()
export class FriendService {
private http: Http;
// I initialize the service.
constructor( http: Http ) {
this.http = http;
}
// ---
// PUBLIC METHODS.
// ---
// I return an observable collection of Friends.
public getFriends() : Observable<IFriend[]> {
this.trackMethod( "getFriends" );
var stream = this.http
.get( "./app/friend.service.NOT_FOUND.json" ) // Go to the server for the data.
.map(
( response: Response ) : IFriend[] => {
return( response.json() );
}
)
;
// By returning the actual HTTP stream, our underlying implementation of this
// service-layer method leaks out into the calling context. By doing this, we
// allow the HTTP request to be made (or NOT MADE AT ALL) outside the context
// of the service layer, bypassing analytics and other tracking methods.
// --
return( stream );
// By disconnecting the "implementation stream" from the "results stream", we
// keep clean boundaries around our encapsulated logic.
// --
// return( Observable.fromPromise( stream.toPromise() ) );
}
// ---
// PRIVATE METHODS.
// ---
// I track method calls for subsequent analytics and analysis.
private trackMethod( methodName: string ) : void {
console.warn( "Tracking method:", methodName );
}
}
As you can see, the getFriends() call starts with an invocation of .trackMethod(). It then configures and returns the Http stream. In this version of the code, notice that the GET request is going to return a 404 Not Found (hence the NOT_FOUND portion of the URL).
Now, in the root component, when we consume this service method, we're going to use the Async Pipe to bind the resultant stream to the root template. We're also going to tack on a .retry(2) operator to the stream so that it will retry the request if it should fail (which we know it will - remember the NOT_FOUND URL).
// Import the core angular services.
import { Component } from "@angular/core";
import { Observable } from "rxjs/Observable";
import { OnInit } from "@angular/core";
// Load modules for side-effects.
import "rxjs/add/operator/retry";
// Import the application components and services.
import { FriendService } from "./friend.service";
import { IFriend } from "./friend.service";
@Component({
selector: "my-app",
// CAUTION: Notice that we are using the ASYNC pipe THREE times in this template.
template:
`
<p *ngIf="( ( friends | async ) === null )">
<em>Loading....</em>
</p>
<ul *ngIf="( ( friends | async ) !== null )">
<li *ngFor="let friend of friends | async">
{{ friend.name }}
</li>
</ul>
`
})
export class AppComponent implements OnInit {
public friends: Observable<IFriend[]>;
private friendService: FriendService;
// I initialize the component.
constructor( friendService: FriendService ) {
this.friendService = friendService;
this.friends = null;
}
// ---
// PUBLIC METHODS.
// ---
// I get called once after the component has been instantiated and the inputs have
// been bound for the first time.
public ngOnInit() : void {
// Get the friend stream.
// --
// NOTE: Instead of extracting the "friends" collection from the stream, we are
// going to be using the Async Pipe in the template to automatically bind to the
// stream response.
this.friends = this.friendService
.getFriends()
.retry( 2 ) // If request fails, try again, 2 more times.
;
}
}
Notice that we are using three different Async Pipe bindings in the UI (User Interface) in order to provide the necessary user experience.
When we run this code, we get the following application output:
Notice that we only track one call to the .getFriends() method; and yet, we end up making 6 HTTP requests to the server. So, the questions I have to ask myself, as the service author, are:
- Would I only want that workflow to track one method call?
- Should the controller have been able to "retry" the Http call without going back through the service layer?
- Did I want separate Http requests to be initiated due to Async Pipe usage?
If the answer to all of those is a confident "Yes", then I would say that there is no leaky abstraction. In that case, the implementation returned from the service layer is being used exactly how it was intended to be used.
If, however, I answer "No" to any of these questions, then I would argue that we have a leaky abstraction - that the internal implementation of my service layer leaked out into the calling context where it was consumed in a way that was not intended by the service author.
Now, going back to my concept from the weekend, if we disconnect the underlying "implementation stream" from the "response stream", we get a very different outcome. Here's a version of the service method that uses an intermediary promise:
// I return an observable collection of Friends.
public getFriends() : Observable<IFriend[]> {
this.trackMethod( "getFriends" );
var stream = this.http
.get( "./app/friend.service.NOT_FOUND.json" ) // Go to the server for the data.
.map(
( response: Response ) : IFriend[] => {
return( response.json() );
}
)
;
// By returning the actual HTTP stream, our underlying implementation of this
// service-layer method leaks out into the calling context. By doing this, we
// allow the HTTP request to be made (or NOT MADE AT ALL) outside the context
// of the service layer, bypassing analytics and other tracking methods.
// --
// return( stream );
// By disconnecting the "implementation stream" from the "results stream", we
// keep clean boundaries around our encapsulated logic.
// --
return( Observable.fromPromise( stream.toPromise() ) );
}
Notice that in this case we are decoupling the implementation from the response through an intermediary Promise. And, this time, when we run the code, we get the following application output:
In this outcome, we are essentially tracking every Http request because we don't allow the Http request to be retried by an out-of-scope consumer. As such, it would force the calling context to go back through the service layer if it wanted to retry Http request (presumably with a .catch() operator). A byproduct of this decoupling is that we still only make one request to the server despite several uses of the Async Pipe.
To me, as a service layer author, this latter outcome is the outcome that I would have expected. Meaning, this outcome aligns with the intended usage of the service method. As such, for me, the former implementation, in which the Http stream is returned directly to the consumer, is creating a leaky abstraction.
Again, to be clear, I am not making a blanket statement for RxJS usage. I am specifically talking about service methods in which the out-of-scope consumption of a stream does not align with the intended usage of the Service method's return value. If you intend to return an Http stream; and, intend for that Http stream to be initiated and retried outside of the context of the service method; then, there is no leaky abstraction.
Want to use code from this post? Check out the license.
Reader Comments
Your suggestion is interesting for sure. I agree with you, you could debate both sides of the fence here. I do think your examples could benefit from using smart/dumb components where you create another component solely for the purpose of the presentation and pass it the Observable<>[] once into an Input() on the dumb child. This would at least reduce the number of async pipes you had in play. This doesn't directly answer your question, but thought I would share as its the pattern that is emanating from the good folks working on ngrx.
@Stu,
The Smart/Dumb component pattern is interesting. I've read about it a bit in the ReactJS world, which may have been where the pattern found its maturity? Not sure. The part of that pattern that I struggle with is the potential complexity of the data structure. If you have an object that has an object that has an object, and you sort of "unwrap" that data as you pass it down through each layer of dumb components, I feel like you inadvertently end up coupling all the layers because any change at a higher layer may have an impact on the data consumption below.
Of course, I've never really dealt with it, so this may be wrong; or it may just be FUD (fear, uncertainty, doubt).
Aside from this, however, I'd love for Ben Lesh to drop by and comment on this perspective (I reached out on Twitter). He seemed to state that I am *way off base* here; but, I would like to understand more about why he thinks that.
@Ben
Interesting read indeed - specifically as the angular2 approach to observables bothers me as well.
The biggest problem is that `async` pipe does not work well with cold observables. A very similar problem hit (and I would say almost kill) cyclejs some time ago - the solution Andre found was to create their own hot-only custom stream library (http://staltz.com/why-we-built-xstream.html).
Still: cold observables are way cooler than hot :). The latter should be used only if really needed as they are kind of global scope and therefore much harder to reason about and keep control of.
Another problem is `http` service interface. Using observable just for making - by definition - single value ajax calls does not make much sense. It brings confusion to those unfamiliar with Rx and offers very little in return.
With promise based approach you could always upgrade to observable with `.fromPromise` or `.defer` and some operators are promise compatible for example: `.mergeMap`. In case of {value|error} contract promises are more precise which clearly is a better thing.
Excellent bluebird promise library solved cancellation problem long ago (http://bluebirdjs.com/docs/features.html#cancellation-and-timeouts) and even offers some elementary transform operators like `.map`, `.filter` etc.
Ben Lesh is - imho - right that returning `.fromPromise( .toPromise )` is an anti-pattern. It just for the sake of cacheability - and imho unnecessarily - makes `getFriends` impure, as calling it causes some side effects. Ideally the consumer of a (cold) stream activates/fires any side effects by the act of subscription. From that perspective cold observables are more like a always safe plan of execution - not execution itself. Just sticking to this simple principle increases composability and transparency dramatically. Actually - imho - observables are the best mechanism of hiding any implementation details. Having reference to `aStream$` I should really do not care where it came from and where it goes (Erik Meijer on mindful programming: https://youtu.be/WKore-AkisY?t=15m59s 1min).
Back to your problem.
According to: https://angular.io/docs/ts/latest/guide/pipes.html async pipe is promise compatible. In such a case you could return lazy observable from `srv.getFriends`, use it like `this.friends = srv.getFriends().toPromise()` and in the view multiple `friends | async`s will not misbehave anymore. This approach, too, leaks abstraction unfortunately - as the implementation of the underling stream may start to deliver multiple values one day. So to prevent this future scenario you should stick to the very first value: `this.friends = srv.getFriends().first().toPromise()`.
Alternatively, you can turn observable into a hot one by `.publish()`, but still I can foresee scenarios, where multiple async pipes will not work as expected - basically they may subscribe after the underling stream completed or published a value / values of interest, so some nasty race conditions.
The solution I would implement requires some shift in mindset. Instead of `getFriends` I would expose just a stream `srv.friends$`. `srv.friends$` exposes a cached value and initiates http request upon the first subscription and/or when cache expired and/or periodically (periodically only if >0 subscriptions via `.refCount`).
If really needed you could add mechanism for getting newest friends eg. `srv.refreshFriends`, which force a refresh but still allows max. 1 pending call at a time. But ideally this mechanism would not be needed, as more elegantly would be to switch to real-time change propagation via websocket for example. The fact the you can change underling stream without touching consumers' code is kind of a proof that no abstractions leaks in this case.
PS:
A separate problem in case of async pipes is error handling. The above guide ignores this aspect entirely (sic!).
@All,
Another quick follow-up post to this one. The outcome is exactly the same - decoupling the "implementation" stream from the "results" stream; only, I tried to come up with a more "canonical" RxJS approach:
www.bennadel.com/blog/3186-using-hot-rxjs-observables-in-your-service-layer-in-angular-2-1-1.htm
This new post uses the .publishLast() operator to created a shared stream proxied by an Observable that caches the last Http response. Again, no experiential difference between that and an intermediary Promise; but, I think this way might be more RxJS(ish) :D
@Artur,
I really appreciate the feedback, especially since I am so new to RxJS. As a quick aside, I don't really have much interest in using Async Pipe in my actual code. I think it leads to overly complex templates (as you can see in my demo) in all by the most simple use-cases. I really used Async Pipe - a feature of Angular 2 - as a means to add perspective to the use of cold streams as a "service layer return value".
In you comment you mention, .publish(), which I am actually starting to use in my most recent post. Specifically, I am using .publishLast() to create a hot stream that caches the most recent value. But, to your point, such an approach then opens up the door for the calling context to use .defer() in order to incorporate my service layer's "hot" stream in the calling context's "cold" stream so that it can use it however it wants.
It could just be that I don't have enough experience with streams; but, to me, the service layer of an application doesn't return "configuration" information, so to speak, it performs actions. To return a cold stream from a service layer makes the service layer dependent upon the calling context to fulfill its responsibilities ... of course we now have a chicken-and-egg conversation. Meaning, "failure to fulfill responsibilities" is only meaningful if you already agree to what the responsibilities of the service layer are. Should we disagree (and it sounds like we do), then my argument no longer makes sense.
Once recent thought experiment that I came up with, however, as I outline in the most recent post, is that of a pending HTTP request that has not yet completed before a user navigates away from a routable component. In such a case, unsubscribing from the "cold" observable may abort a pending Http request, which just feels like it would be an unintended action.
Anyway, clearly I have very much to learn. It seems like on either side of the argument there are ways to take the resultant stream and make it do whatever you want, either by publishing it or deferring it; so, at the end of the day, I think it comes down to how the author of the service layer intends for the service to be used.... and just go from there in either direction.
@Artur,
Upon further reflection, I think there's also something to be said about the "Principled of Least Surprise." Given the fact that RxJS is new to so many people; and, that so many people have experience with things like jQuery.ajax() and $http.get(); I wonder who will be *most surprised* by stream implementation?
I would posit that given the majority of people's experience, a "hot" stream is more likely to coincide with their expectation. And, that a "cold" stream, returned from a service layer, would be more surprising to most people.
That's no to say that we should always code *down* to people's mental model; but, I think it should be taken into account.
Doing a little bit more noodling on the "principle of least surprise":
www.bennadel.com/blog/3187-partial-stream-execution-a-case-for-hot-rxjs-observables-in-angular-2-1-1.htm
Cold streams allow for situations in which the "intent" of an action can remain partially unfulfilled. While this is exactly how Cold streams work, I believe it speaks to what is "expected" of a service layer and how Cold streams can lead to surprising "service behavior," regardless of how Cold streams are documented to work.
@Ben,
Totally agree with you that "Principle of Least Surprise" was violated in case of Http service.
Additionally, will repeat myself here, but returning an observable (value*|error) for an action that may result in a single value or an error (value|error) increases uncertainty of the codebase. The question is if benefits are greater than the cost. Imho: they cannot be *that big* as you can always upgrade to an observable from a promise anyway.
About responsibilities of the service layer and the leaking abstractions.
I think I understand your point better now. Indeed returning a cold / not-fired observable from a service layer method is problematic and has to increase complexity:
1) service layer is in some state when the method is called and returns an observable which potentially depends on that state
2) subscribing to a cold observable is *at least as powerful* as calling another parameterless method - but the moment of the call is totally unknown and so is the state of the service layer
3) indeed: it is a recipe for a disaster
[Please mind that exposing an observable (`.friends$`) vs a parameter-less method returning promise aka fired-side effect (`.getFriends()`) are more or less equivalent.]
The open question is what is problematic here the service layer or the observable. Or to rephrase it in more general terms OOP or FP.
OO makes code understandable by encapsulating moving parts. FP makes code understandable by minimizing moving parts.
https://twitter.com/mfeathers/status/29581296216
About navigating away. I think it depends on the nature of the request:
if it is some read-only resource - it can be cancelled safely
if it is some tracking code / analytics - it should continue (though its cancellation probably should not depend on the router)
if it is some state-changing action of minor importance - it probably should continue
if it is some state-changing action of major importance - imho the navigation should not be allowed in first place (cheaper solution) or you should implement a mechanism of failure recovery (more expensive solution)
@Artur,
To be honest, I am not well versed enough to talk about the pros and cons of OOP vs. Functional Programming - I'll have to defer to people who have better mental models for it - you can see how much I struggle with just one small aspect of application design :D
That said, I think you are totally spot-on when it comes to navigating away from a UI that has a pending action response. Things get a lot more complicated if you start to deal with "optimistic updating" of local cache data. For example, how do you handle something that you optimistically record as a success but then eventually comes back as a failure? How do you let the user know?
Obviously each situation is going to be different. And, to be completely honest, I've definitely made the *wrong choices* in my apps when thinking about some of these use cases. But, to that point, consumers of a service are going to make the wrong choices. They are going to call things at weird times or unsubscribe when they're not supposed to or navigate away from UIs before critical actions have completed. As a service Provider, you can't stop that; and, as such, I think you shouldn't couple yourself to the *consumption* of the response (hence the Hot Observable).
@All,
Another quick follow-up -- the more I noodle on this, the more I am beginning to think about my web-app as a "layered" architecture. And in the "core" of that architecture, I am coalescing around Promises once again:
www.bennadel.com/blog/3202-my-evolving-angular-2-mental-model-promises-and-rxjs-observables.htm
In my current mental model, the "core" is primary Promise based; but the "query" portion of the Core will likely expose Streams for specialized edge-cases (like type-ahead and anything where canceling the underlying HTTP request is actually meaningful so as not to starve the HTTP request pool).
I'll still use RxJS in the Controller layer, or what I'm referring to as the "Web App" layer, when they make sense. But, the core of the app will be return to a Promise-base workflow.