Saturday Hackathon: Integrating Twitter, oAuth, Twilio, And ColdFusion
In the last couple of months, I've taken to running my own, solo hackathons - a solid day of coding with almost no interruptions. So far, I have found this process to be tremendously awesome. This latest "hackathon," however, took this concept and pushed it a little bit further than expected: I couldn't get it done in one day, so I ended up distributing the code over about 3 days of effort (about 20 hours in total). The final project - Twitio - was a learning project that integrated Twitter, oAuth, Twilio, and ColdFusion.
| |
|
|
||
| |
 |
|
||
| |
|
|
To try this out for yourself, visit http://twitio.tw.
To download the code for the demo site, click here.
The concept behind Twitio is simple: you log into the website using your Twitter account (via oAuth authentication). Once logged-in, you activate your phone number. Then, you can call the Twitio phone number, leave a voice message, and Twitio will automatically post a status update to your Twitter account (on your behalf) with a link back to the voice message. In essence, Twitio allows you to leave "audio tweets" in your timeline.
| |
|
|
||
| |
 |
|
||
| |
|
|
Typically, in a learning project like this, I start out with the design - I'm a firm believer that a project needs to look good inside and outside in order to be the most effective learning tool. However, because I knew that oAuth would be such a bear to wrap my head around, I decided to start immediately with the ColdFusion code, trying to get the oAuth "dance" (as it is often referred to euphemistically) to work.
I know there are a number of ColdFusion and Java Twitter libraries that abstract oAuth interaction; however, when I'm learning about something, I tend to like to build things from scratch in order to figure out how they actually operate. All in all, getting oAuth to work took about 12 hours - more than 50% of the overall project effort. In the end, though, I was able to distill oAuth down to just two ColdFusion components - OAuthService.cfc and OAuthRequest.cfc. These components deal very narrowly with oAuth and know nothing about Twitter itself.
Luckily, I've used HMAC-SHA1 hashing with Twilio and HMAC-SHA256 hashing with Pusher App in the past; so, the hashing required in the oAuth requests was not entirely new. I do have to say, however, that I definitely reviewed Matt Gifford's incredibly robust Twitter library - MonkehTweets - whenever I hit a wall. His library helped me get over some hurdles regarding Base64 and request-parameter encoding. Matt, you rock!
Because I like to distribute the code that I write during my hackathons, I like to make the applications that I create as portable as possible. In order to do that in this case, I needed to create database-like functionality without an actual database. Luckily, ColdFusion provides in-memory SQL functionality that rocks the party that rocks the body. The database tables that I use are actually in-memory ColdFusion query objects. Of course, these contain sensitive information (oAuth access tokens and phone numbers); as such, when I serialize these queries as WDDX packets, the data gets encrypted using a context-sensitive encryption key. In this approach, not even the encryption key is hard-coded in the application.
While ColdFusion provides amazing in-memory SQL functionality through its query-of-queries, I do find myself getting very frustrated with the "guessing" that ColdFusion makes about the query column data types. Even when I explicitly type my query objects (using QueryNew() and cf_sql_XYZ types), I find that string values (such as phone numbers) are often viewed as numeric values. I would absolutely love to see future releases of ColdFusion be more strict about the implicit (and often times unwanted) runtime type-casting of column data values.
The Twilio aspect of the application was the easiest to implement. Between the online Twilio debugger and my own local input/output logging, there's really no mystery as to how the Twilio client is interacting with the ColdFusion application. Twilio continues to be a joy to work with. I am constantly blown away by how much mobile-integration functionality can be provided with such a minimal amount of effort. Of course, that's the same reason I love ColdFusion.
As part of the demo application, I am having Twilio transcribe the phone calls in order to provide an in-page textual preview of the voice tweet. Ultimately, however, when I take this beyond a demo app, I'll get rid of this functionality. Not only does transcription limit the phone call to 2 minutes, it also costs more money and is not all that accurate.
Once again, I have found my solo hackathons to be incredibly productive. This one took way longer that I had wanted; but, when you sit down and plan to code without interruption, it awesome how much you can actually get done. This approach has also completely removed any fear that I have about tackling projects that might end up being total failures. As long as I know I'll have enough time to learn, I almost don't care what ends up happening. In Any case, I hope this code might help some people get more comfortable with oAuth, Twitter, and Twilio integration.
Reader Comments
Hey Ben!
The application is fantastic. I knew you were doing something Twitter / OAuth based over the weekend, and couldnt wait to see the results.
It was a sad day when Twitter closed their Basic Authentication protocol allowance ( it made everything so much easier :) ), but it's made everyone stand up and take a look at OAuth. I tried to tackle it last year for a few other projects I have yet to release, and it is a mean S.O.B at times.
Fantastic work on the hackathon, and I can't wait to check out the rest of your code.
@Matt,
Thanks my man - this was a load of fun to create. Frustrating, but very fun. The oAuth stuff was the beast of the project. Your MonkehTweet project definitely made me feel more comfortable in my exploration, so thanks so much for that.
@All,
I should also mention that I got a great overview of the oAuth "dance" from this website:
http://shiflett.org/blog/2010/sep/twitter-oauth
Here, Chris Shiflett provides a nice overview of how the oAuth life cycle works without getting bogged down in the details; this was great just to get a bearing on how this was supposed to work.
Also, Twitter's Dev site has a decent walk through:
http://dev.twitter.com/pages/auth
What's nice about this page is that it gives you testable input / output parameters that you can use to help debug your own oAuth logic.
As always, thanks for sharing, Ben!
I'm downloading the code now to see if I can integrate any of it into my soon-to-be-in-Alpha http://refynr.com web app.
Cheers!
@Aaron,
Awesome my man. Of course, you'll want to swap out the Query-of-queries with an *actual* database :) But, I hope it helps you move in the right direction.
Great post Ben, and another excellent app. Thanks for demoing at the NYC Twilio Meetup.
Cheers,
John Britton
@John,
Thanks my man - it's been a real pleasure seeing how Twilio can be leveraged. I look forward to pulling more ideas together.
Ben, I want you to know that you are like the Jordan of the CF/jQuery/Programming community. We are all like Scottie Pippen, and you are Jordan. Thanks for putting the code out there for the community.
Awesome Ben nice work, thanks for sharing this.
@Jyoseph,
Ha ha ha, thanks :) And thank you for using names that I recognize (the last time I knew anything about basketball was playing NBA Jam on Nintendo!). I'm happy to put code out there and get a good conversation going about it.
@Glyn,
My pleasure my friend. Hope you find it useful.
This is good stuff it got me interested in Speech Recognition again!
Must...sign...up... Must...play...with...Ben's...cool...toy.
You have quite the following -- people were testing it within minutes of you posting this.
@Randall when a guy who's blog has got your coding but out of the sling so many times ask for help, you can't help but answer!
@Randall, @Tim,
Ha ha ha, thanks guys, I appreciate the enthusiasm.
@Tim,
The voice encoding stuff you are doing is very cool (and seemingly more accurate than the Twilio text encoding). I definitely need to look at the library you're using.
@Aaron,
That was a typo and has been removed. Thanks for the catch.
Ben,
4 years later, your post is still helping someone out! Thank you for spending the time on your blog. If I could pick your brain for one second, it would be greatly appreciated.
I bought a fitbit and trying to play with the API. Runs oauth1. I downloaded your code and stripped out the extras.
I'm at the AccessToken Request portion of the authorize.cfm sending a request that is getting bounced for hot being a valid signature.
The fitbit team came back and said:
"Check that you are signing the access_token request with the consumer_secret&temporary_token_secret (temporary_token_secret = oauth_token_secret from the request_token api call)."
I'm assuming that I have to create a new signature using the token issued during the callback rather than the one that was issued during the registarion of my app and stored in the application.twitter.consumersecret during startup.
If that is the case, does your application already handle that scenario and I am just not catching on, or would I have to add another variable to the application.twitter. xxx settings and remake an alternate signature for that call... and how is that call made?
Sorry almost 2 days of pulling my hair out for a self-project I wanted to do for fun!
@Steve,
I don't remember the specifics of this project. But, I can say that "signing stuff" can always be a headache when it comes to talking to APIs. In my experience, the biggest problems with signing is either in:
1. Generating the normalized message to be signed (which is what I think they are telling you to use)
2. Actually signing of the message using the appropriate hashing algorithm (ex. HmacSHA512).
3. Encoding the hashed message authentication code for the new request (ex, encoding the bytes in Hex or Base64).
I would start with making sure the code is using the right Hmac (Hashed message authentication code) algorithm. If that's not right (ex, they want SHA256 and you're using SHA1) it won't matter what the input is as the output will never match anyway.