The User Experience (UX) Of Conversion-Driven Development
Yesterday morning, I had an epiphany. Or perhaps more accurately, I was finally able to identify and articulate something that I've implicitly understood for years: a high-conversion workflow does not necessarily imply an enjoyable user experience (UX). This creates an interesting dilemma in which the short-term goals of a business may not align well with its company culture. More interestingly, it brings into constrast the abstraction of customers as either "people" or as "statistics."
A while back, NPR (National Public Radio) ran an interesting story about a machine nozzle whose design was evolved by a random algorithm. For each generation of nozzle, random changes were applied to individual units and then tested (looking at spray patterns). The top performing nozzles were then kept and the bottom ones were discarded. The next series of random changes was then applied to the top performing nozzles of the previous generation. And so on, for many short-lived generations.
Eventually, the company came up with a spray nozzle that demonstrated superior performance; but, for reasons that weren't necessarily well understood by the engineers. After all, it was just a series of random experiments - an industrial manifestation of Darwin's "Survival of the fittest."
I remember hearing this story and immediately thinking about A/B testing in a piece of software. The thought of applying random changes to a user interface was, at the same time, both fascinating and terrifying. The terrifying part, for me, was not understanding why a change may lead to better performance. At the time, I attributed this emotion to personal insecurity: as a product designer, if I can't explain why a change works, then how can I be trusted to make changes in the future?
What I realize now is that the angst I was feeling at the time was not an "insecurity" but, rather, an "empathy." If a high-conversion workflow doesn't necessarily imply an enjoyable user experience (UX), then how can I be sure that randomly applying changes to a user interface (UI) continues to provide a positive experience? If I don't understand how a change works, I can't know if it's the right change for my users, regardless of what the numbers dictate.
Ultimately, there's a fundamental difference between spray nozzles and users. One is an inert, 3D-printed piece of plastic; the other is a thinking, feeling, sentient being, with an evolved pattern-matching brain and a weakness for familiarity bias. I can't have empathy with a piece of plastic; but, I can with a user. Until, of course, I start letting conversion-numbers rationalize decisions that defy reasonable explanation. At which point, I have to actively suppress empathetic tendencies and put blind faith in the math. I have to stop seeing users as people and start seeing them as statistics.
Now, I'm not trying to paint a picture in which all A/B testing is bad - A/B testing is great. Experiments are great. Evolving a product over time is great. Formulating a hypothesis and then collecting data is great. Until, that is, the numbers outpace the product designer's ability to understand and explain them. At that point, things become dubious and slopes become slippery.
Take, for example, a common sign-up form:
| |
|
|
||
| |
 |
|
||
| |
|
|
There's nothing special about this sign-up form - it's the same form you've seen a million times before. Now, imagine that we try to randomly change the order of the fields:
| |
|
|
||
| |
 |
|
||
| |
|
|
Here, we're asking for the user's Email address before their Name. To me, as a product designer, this is awkward. It breaks with a decade of established patterns and social moraes. And, as a user myself of many sign-up forms, seeing this atypical ordering leaves me with an ever-so-slight feeling of discomfort (one that I may keep with me as I continue to evaluate a product).
But, the analytics say that this version of the sign-up form - email first - converts at a rate 1.3% higher than the previous version.
At this point, what do I do? As a product designer that cultivates a sense of empathy with the user, the latter version of the form is worse and I should discard it. But, if I think about users as a "statistic" in a mathematical model, this version of the form is better for the business (at least in the immediate sense) and should be kept. And, if I keep it, I either have to evolve my understanding of the users such that I can maintain an empathetic model; or, I have to decide that the numbers trump empathy whenever there's a disconnect.
It's interesting that I started this post thinking about the experience of the user and have quickly discovered that conversion-driven development affects the experience of the product designer just as much (maybe more?). I think it's a fascinating topic. There's no surprise that a high-conversion workflow is likely to be an easy-to-use, enjoyable workflow; but, there's no promise of it either. And, I just think we, as product designers, need to be cautious about suppressing our empathetic powers when they can't otherwise be cultivated through reason and understanding.
Reader Comments
Just when you think you have a solid grip on site design and layout...and you have to go and post this. :D
Since we're about to do a wholesale site redesign launch (new site is built...just waiting for the owners to give the thumbs up) I kind of wish I'd run across these thoughts earlier in that rebuild process. As it is, we do have some areas that are suitable for an A/B conversion driven layout almost exactly as your example here shows. Once we compile enough conversion data, I'll try and remember to return here and share our results.
Great post and something I'll be keeping in mind as we move forward.
@Maryann,
To be clear, I don't have anything against A/B testing or anything like that. We use it heavily at work, in conjunction with feature-flags (like we'll roll out some change to X-percentage of users) and then we'll see how things perform. My only point of hesitation is when we see higher performance coming from a change that I *don't like*. It's only in this case -- where my "spidey sense" disagrees with the numbers. In those cases, it's just an uncomfortable feeling.
I hope your redesign and launch go well, though! Good luck :D