Tim's Musings

NoMoreClipboard

2010-10-20T11:24:00.000-07:00

I'm at the Mobile Health Expo in the Ceasars Palace Convention Center in Las Vegas. Having succussfully maneuvered past Barry Manillow, Cher, Donny & Marie Osmond, and 20,000 slot machines, I encountered NoMoreClipboard and their interesting use of PHRs with low income diabetes patients.

Functionality
Looks pretty straightforward—view portions of the medical record, enter blood glucose, send prompts to patients and to physicians.
Uses desktop web, mobile web, and SMS

Implementation
They’re conducting a pilot w/ Howard University Hospital Diabetes Treatment Center in Washington, DC, in neighborhoods that have a high incidence of diabetes. Patients are typically low income and either Medicaid or uninsured. The program begins with community-based screening and initial treatment in a tricked-out RV (aka mobile health clinic). In the RV, their information gets entered in Howard’s EMR.

When they get off the RV, the patient is greeted by a “PHR Educator” (first time I’ve heard that term). The PHR Educator gets them set up with an account, which includes downloading their data from the Howard EMR system into the PHR, as well as filling in gaps in the data. Patients are encouraged to enter their glucose readings several times/day. About every three months, they collect HEDIS data from patients via online surveys.

They have 232 patients using the system so far. They've only recently launched the mobile component, and it's growing fast.

Incentives
For the patients: The program provides “medical minutes,” essentially subsidizing part of the patient’s data plan. But they’re very clear that they don’t want to pay for everything. The patient needs to pay for at least part of the phone and part of the data plan, to ensure they have skin in the game. The program will provide new phones, or patients can use their existing phones.

For the physicians: Cupcakes. They’ve succeeded in getting clinical buy-in by providing cupcakes. Every time a physician gets another 20 patients signed up, they get a hand-delivered box of Georgetown Cupcakes, which are evidently delicious.

Preliminary findings & observations

Not all the patients use the system, but those who use it tend to use it a lot. This is consistent with findings from the California Health Care Foundation study that showed low PHR adoption by people with low socioeconomic status, but high usage by those who do adopt.
Age 60 appears to be dividing line—over 60 they tend not to use it. This is different from other data I’ve seen, which shows 70 or 75 as a clearer dividing line
1/3 use once/week or more; 1/3 use PHR at least once a month; 1/3 rarely use it
MDs report enhanced dialog between patients and providers. Communication is more frequent, complete, and accurate
They’re claiming reduced HA1C, BP, cholesterol, ER visits, and hospital readmissions, but didn’t provide any specifics (to be published, but not yet)

Data Visualization, Information Overload, and Compression

2010-09-17T09:36:00.000-07:00

In his TED Presentation on data visualization, David McCandless touches on information overload (starting ~16:38), suggesting that data visualization is one tool in our battle with information overload--that good data visualizations enable us to take in data through our eyes and process it in our brains much faster than similar amounts of data communicated through text and numbers.

This reminded me of Mark Hurst's Bit Literacy work:

Bits are heavy. Though they have no physical weight, bits--the electronic data that flows in and out of our e-mail inboxes, cell phones, Web browsers, and so on--place a weight on anyone who uses them. A laptop computer weighs the same few pounds whether it holds one e-mail or a thousand, but to the person who has to deal with all those e-mails, there is a big difference. Appearing in large numbers as they often do, bits weight people down, mentally and emotionally, with incessant calls for attention and engagement....

The problem can be solved by learning bit literacy, a new set of skills for managing bits. Those who attain these skills will surmount the obstacles of overload and rise to the top of their professions, even as they enjoy a life with less stress, greater health, and more time for family and friends. Bit literacy makes people more effective today, even as it equips them for the future.

Mark points out that you can read every day about the information overload problem, but it's very difficult to find practical help dealing with information overload. So his book, Bit Literacy, provides elegant, practical techniques for just that, most of which involve filtering, prioritizing, and organizing incoming data.

I see an intriguing connection between data visualization and bit literacy--an underlying suggestion of a powerful technique that I'll call "compression." Think of it this way:

When a program like WinZip or iTunes compresses a file, it creates a new file that contains most or all of the source information, but using fewer bits to represent that information.

And data visualization does the same thing. A good data visualization takes a large amount of data, either qualitative or quantitative, and displays it in form that conveys most or all of the source information, but using fewer bits to represent that information. This suggests the notion of "compression" as one technique for dealing with information overload.

A few compression examples come to mind.

In the last few years, management "dashboards" have started proliferating. These dashboards essentially take a large amount of information about how a product or company is performing, and compress it into one or two pages of charts, key performance indicators, and short explanatory text. This compressed version of the information enables a manager to quickly take in a tremendous number of bits very rapidly.

Design personas fulfill a similar function. We start with mountains of data from many sources to understand our customers and their needs, and we compress that data into a small number of composite characters called personas. Then we use those personas to communicate with the project team and stakeholders. Essentially, we create compressed versions of the data.

Both of these are examples of "lossy" compression. In the world of compression, "lossless" compression means the compressed file contains all of the information from the original--it's just stored more efficiently. When you download a software application, that software is typically stored in a lossless format, so that when you decompress it, you get all the information of the original. Contrast this with "lossy" compression, in which the compressed file is both smaller, and takes up less space, than the original. This is what you get with an mp3 audio file--you can still enjoy the song, but some of the audio fidelity has been removed so you can fit more songs on your iPod. The trick with lossy compression is to systematically determine a) how much fidelity is required, and b) which data can be removed while still retaining the key information.

Back in our information overload space, this becomes the key question--how can we systematically reduce the bits coming at us so that we can send and receive the essence of a large data set while retaining the key information we need to make informed decisions.

One more example highlights the potential power, and the risk, of using data visualization to combat information overload:

A stock ticker widget essentially compresses all of the data about stock trading into a handful of numbers. After millions of trades today, the Dow Jones was up 1.2%, ending at 10,603.54. This is an attempt to compress not only the stock market, but the economy as a whole. If the Dow is at 10,603.54, the economy is probably better than it was last year, but still struggling.

So the stock ticker saves me the trouble of having to look at all of the data about today's trading. This is good. On the other hand, when there's a TV screen in my elevator barraging me with data about how the Dow, NASDAQ, and S&P 500 are changing from one minute to the next, that's way more information than I need or want. Some further compression would help. As in software compression, it's not only a question of which data to keep and which to remove--it's primarily a question of how small I need the compressed version to be. In the case of a typical consumer, we could add information and compress it even more by presenting a weekly updated graph of performance over the past 10 years.

So I'm having fun playing around with this metaphor, and I have three main questions:

1) Who else has written about compression and/or data visualization as a means to combat information overload?
2) What are some more examples of compression being used effectively to combat information overload?
3) How might we apply this concept in fresh ways to make ourselves more productive and happier each day?

eHealth Disparities - strategies continued

2010-08-11T14:41:00.000-07:00

A few more thoughts on potential strategies based on meaningful access...

For those who do not currently have meaningful access, but who could get meaningful access as a result of our efforts, we might think of two complimentary paths:

Bring the people to the technology
Bring the technology to the people

In the first case, we're changing the people. In the second case, we're changing the technology.

By "changing the people," I simply mean finding ways to help these folks take advantage of tools others already have. For example:

Public access computers in libraries, medical centers, etc.
Subsidized access (e.g., some health plans give away cell phones with unlimited minutes for interactions with the health plan)
Training on how to purchase, use, maintain, and troubleshoot

In the second case, "bring the technology to the people," we're changing the technology, content, and functionality to make it more accessible, appropriate, and useful to people. For example,

Change our push messages from phone and email to SMS
Optimize existing web sites for access on pocketable devices
Convert key Web interactions to work on IVR (touch-tone telephone trees)

For folks who don't have meaningful access and who won't have meaningful access regardless of what we do, when I blogged a couple of days ago I left off what could be a key strategy:

Use higher end technologies with other people so as to free up more traditional resources to attend to the needs of those who don't use those technologies. Here's a way to think about it: If we can use the web to save phone calls to a call center, that should free up call center resources. We would then need to deploy those call center resources to better serve the needs of the people who don't use the web.

I'm liking this basic approach of organizing our strategies based on meaningful access. But I also have a suspicion that we might do better to simply look at age and socioeconomic status (income & education). There's a ton of data out there, and the trip remains finding ways to simplify our approach while respecting the integrity of all that data.

eHealth Disparities Segmentation by Meaningful Access

2010-08-09T21:05:00.000-07:00

Many of the same populations that suffer from health disparities also have lower Internet usage. How can we use the Web and mobile technologies to close the gap between the haves and have-nots, rather than increasing the gap?

As always, we need to start by understanding the people involved.

There are many ways to describe the people likely to get the short end of the stick in terms of health, healthcare, and technology. The most obvious are:

Demographics (e.g., ethnicity, age, language, socioeconomic status)
Psychographics (e.g., those deeply engaged in their health vs. those who don’t pay much attention to their health, or those who love the latest gadget vs. those who fear computers)
Access to Technology (e.g., those with desktop broadband vs. those with dial-up, vs. those with smart phones, vs. those with cell phones & SMS, vs. those with none of these)

The problem I keep bumping into is that these factors overlap in very complex ways, and all simple approaches to segmentation seem to oversimplify way too much. For example, Hispanics are more likely to have adverse health outcomes than whites, and they’re also less likely to have broadband, but they’re more likely to access the Internet on their phones. Does this mean we can use smartphones to decrease health disparities for Hispanics? Not necessarily—I'd guess that the Hispanics suffering most from health disparities are those least likely to have smartphones.

Four years ago, in the report Expanding the Reach and Impact of Consumer eHealth Tools, Cynthia Bauer and colleagues at the Dept. of Health and Human Services did an impress job of researching, analyzing, and organizing the field of eHealth Disparities. One of their main conclusions was that we needed more data at the subpopulation level. That gap in our understanding has closed a little in the last four years, but we're still struggling to understand the individuals most at risk of being caught between health disparities and digital disparities.

That said, I think we're close to having a practical starting point.

As we think about strategies to address eHealth Disparities, we might find it helpful to start segmenting in terms of meaningful access to technology. “Meaningful Access” refers to the need to have more than just a computer. Meaningful access requires:

hardware
Internet connection
skills to use them
ongoing technical support
relevant useful content and functionality

If we take the people most vulnerable to health disparities and subsegment them by meaningful access, then some high-level strategies start to emerge:

The “haves”

Those who already have meaningful access, or those who will gain meaningful access in the next few years with or without our efforts.

Strategies:

Promote existing content and functionality to them

Enhance current content and functionality to be more useful to them

Create new content and functionality for them

The “could haves”

Those who don’t have meaningful access, but who could gain meaningful access as a result of our efforts.

Strategies:

Use the same strategies as for the “haves” above, and also…

Support public access points (libraries, medical centers, shopping malls, etc.)

Support simple and inexpensive access on devices they already own, e.g., SMS texting, including paying the per-message fee

Support public policies and funding that increase access for the underserved (e.g., community-wide wi-fi, extend universal access programs to cover not just phone but also Internet)

The “won’t haves”

Those who don’t have meaningful access, and who still won’t have meaningful access 3 years from now regardless of our efforts.

Strategies:

Support “infomediaries” such as family members who use the Web and mobile devices on behalf of those who don’t

Maintain and enhance non-technology-based services

This might be a starting point. The next step would be to gather more information about each of these groups to understand whether these groups are homogeneous enough to have similar needs that can be addressed with similar efforts.

Mobile Health - the 2 big deals

2010-07-14T16:48:00.000-07:00

I've been doing online consumer health for over 15 years, most of it with Kaiser Permanente. As I think back on some of the key capabilities that were originally visions on the far horizon and are now simply part of the landscape around us, I remember when each of these was "the next big thing."

health information previously available only to professionals, made widely available to consumers
health risk assessments with personalized feedback
online appointment requests
online prescription refills
online appointments booked in real time
select a physician online
apply online for coverage
email my doctor
secure messaging with my doctor
view my medical record

Each of these is now everyday reality to millions of Kaiser members. We've reached these horizons and moved on to the next. So what's next? When someone asks me, "What's the next big thing," I usually end up talking about two areas:

A better user experience
Broader reach

Despite lots of powerful and valuable possibilities for new functionality, from personalization to portable medical records to home monitoring, I think the biggest value to individuals and society will come from improving the user experience of the current functionality, and making that experience available to a broader audience.

1. Better user experience
We need to take all the capabilities we've already implemented, and make them...

easier to use
more integrated
better adapted to real-life scenarios and tasks of our users

2. Broader reach
Over 3 million Kaiser members use the powerful tools we've provided. That's not nearly enough. In addition to increasing the number of web-using Kaiser members who use this stuff, we need to expand these tools to...

people who are traditionally underserved by the healthcare system
people who don't have easy access to PCs with broadband connections
people whose physicians aren't currently part of an integrated group practice

The Mobile Factor
Cell phones won't take us all the way to these horizons. But they can certainly help us get there. In terms of user experience, mobile devices can make simple transactions ridiculously easy, and they can fill in the gaps between in-person, telephone, and desktop web interactions. If we do it right, mobile interactions will become a lynch pin of ubiquitous, integrated, cross-channel experiences.

Not only can mobile devices support much better experiences, it's getting clearer all the time that they can help us extend these services to people who are traditionally left behind by the latest technology. If we do it wrong, our mobile efforts will just exacerbate the already shameful chasm between the haves and have-nots. But if we do it right, and I think we can, we can use mobile technologies as a powerful tool in shrinking that gap.

Empathy, Humility, and Moxie

2010-05-25T19:41:00.000-07:00

It starts with empathy.

Every journey to a great user experience starts when someone somewhere empathizes with a user in a way that is authentic, human, and compelling. It starts when someone realizes that the most important person to pay attention to is not the person paying for the project or implementing the software, though these folks are certainly important. It starts when someone really, truly, deeply pays attention to the person who will end up using what's being created. User experience starts when someone starts caring about the user.

Next comes humility.

Anyone who has watched usability testing or ethnography or a focus group knows the feeling. I remember my first usability test - I had spent weeks with a talented team designing a great user interface. All we needed to do was validate the design with some end users and possibly make some tweaks. An evening of usability testing provided a huge heap of humility. Labels that were intuitive to me were enigmas to the users. Buttons that were totally obvious to me were invisible to users. Most of the people I showed it to couldn't even figure out what it was supposed to be.

Sometimes we argue with the first few people who have trouble with a design:

"This first test participant is an outlier."
"The second test participant isn't a good representative of our customer base."
"The third test participant must just be stupid."

But by the time we get to the 4th user who can't find their way through our design, we start to realize, maybe the problem isn't with the users. Do this enough times with enough designs and we learn a very healthy sense of humility. Two quotations that help keep me humble:

Several years ago we were testing a new home page design. We brought in several users and gave them what we thought were a handful of simple tasks to complete. By the time we got to the third user I was already starting to hear the humility music playing, and it came to a crescendo when the moderator asked, "so where would you click?" and the user responded, "I'd click over here on 'careers' so a could get a job with these bozos and fix their damn web site!"

And I remember Haim Hirsch, my first user experience mentor, who had conducted hundreds or thousands of usability tests, tell me, "I've never conducted a test where I wasn't surprised by something." So that's my goal - to be humble enough to assume that my design has major problems, and to be grateful to the users who will help me find those flaws.

Empathy, humility, and...

...moxie.

From the dictionary:

moxie
–noun Slang.
1. vigor; verve; pep.

2. courage and aggressiveness; nerve.

3. skill; know-how.

Empathy is about caring. Humlity is about knowing that I don't know everything. And moxie is about knowing that I do know some very important things, and in some cases I know more than my users do. Moxie means that I sometimes contradict the users. It means I have the expertise and the larger vision to create something they hadn't thought of or to do something in a way they would not suggest. We need to conjure up some moxie, because users are great at identifying problems, but not necessarily always great at identifying solutions. User needs should drive all key decisions; user design suggestions should get added to the list of possibilities.

Moxie in the absence of empathy and humility is a recipe for disaster. But in combination with them, moxie enables us to break paradigms, innovate, and create solutions that meet our users' needs and desires better than anything they could have designed themselves.

Empathy - I care
Humility - I don't know everything
Moxie - I know something

How to demo software

2010-05-19T20:51:00.001-07:00

The demo starts out nicely enough...

In a dark conference room, the vendor lays out a simple user scenario and starts walking me through the experience. But before he gets to the second click, he needs to mention the cool feature that he isn't clicking on, which leads to a short discussion of the underlying technology, which evolves into a few key points about their design process. When he finally finishes telling me about their unique value proposition and returns to the screen, I can't remember anything about the scenario, which is irrelevant anyway, because now he's running through the menu bar telling me about each of the features.

The thing that's screwed up about this story is that the story gets lost.

Stories are perhaps our most powerful tool when demo'ing software. As human beings, stories grab our attention--we can't help it. Stories are sticky. They stick in our minds much more than explanations, descriptions, or data, and once they get into our brains, all kinds of powerful things start to happen. We intuitively fill in gaps in the narrative; we make connections with other parts of our lives, and--very importantly--we remember. If you want your audience to remember what you said, tell a story.

A good story tends to have some typical components that can all be included quite easily in a software demo, even in a short demo that lasts only 60 seconds. Be sure to include:

The main character, your user--give him or her a name (let's use "Bored Bob")
The setting/context--when and where is Bob using the software (Bob's at home, sitting on his couch with his laptop on the coffee table and the tv on across the room)
The central conflict--every story needs a conflict, and for demos, the conflict is usually a problem the user is having that the software will help solve (Bob wants to go to a movie, but he doesn't know which movie to see)
The plot--what happens (Bob googles "movies near me" and clicks on my web site. He starts by browsing through a list of new releases, gets drawn into a couple of previews, purchases a ticket for the one he wants, and buys the soundtrack on a whim.)
The resolution--in a demo, this is the "so what" part--what are the benefits to Bob and to the owners of the software? (Bob loves his movie and finds a new favorite musician on the soundtrack; my movie site grabs a customer that would have otherwise gone elsewhere, and we make an extra $5 profit from the soundtrack sale)

Five elements sounds like a lot, but it doesn't have to be. Here's what we ended up with in my simple example:

Bored Bob is at home, sitting on his couch with his laptop on the coffee table and the tv on across the room. He wants to go to a movie, but he doesn't know which movie to see, so he googles "movies near me" and clicks on my web site. He starts by browsing through a list of new releases, gets drawn into a couple of previews, purchases a ticket for the one he wants, and buys the soundtrack on a whim. Bob loves his movie and finds a new favorite musician on the soundtrack. And my movie site grabs a customer that would have otherwise gone elsewhere, and we make an extra $5 profit from the soundtrack sale.

A good demo of my web site will tell this story very early on and, very importantly, without interruptions. If I stop the story to explain a feature or why we did something the way we did, the story gets lost--much better to tell the story beginning to end, simulating Bob's experience on the screen, and then go back for explanations and details. When I'm done with the story, I can talk about the incredibly cool matching algorithm we developed to push the right movies to Bob, and the subtle up-sell techniques that prompted him to buy the soundtrack, and the projected increase in revenue due to these enhancements. Those are important things to tell my audience about, but they should never detract from the star of the show.

The star of any great demo is the end user experiencing the product.

Stories and Data

2010-05-13T09:10:00.000-07:00

Last week I spent 3 days in storyland. I was in Chicago at a meeting of the Innovation Learning Network, focused on the use of stories in business and innovation. I listened to lectures on the psychology of stories; played games with a Hollywood script writer, a radio show editor, and an acting troupe; and spent a morning redesigning airport security procedures.

It got me thinking about the interaction between stories and data.

Ideally, stories and data have a symbiotic relationship. Stories add richness, context, and meaning to data while data adds richness, context, and proof points to stories. Used together well, they are a powerful combination for telling the truth about the past, present, and future. But if we get sloppy, they can undermine each other.

I think there are two ways that data supports stories:

Data helps us figure out which story to tell
Data helps us tell the story

These are two very different uses. Confusing one for the other can have tragic consequences.

Data helps figure out which story to tell
First, data helps us analyze reality to determine what's actually happening in the organic stories of real life. By collecting and analyzing data, we begin to piece together a narrative that represents what's going on--who is doing what? how often? how many? how consistently? etc. That's what data analysis is all about--good data helps me understand reality.

For example, if I want to tell the story of how someone uses my web site, I can look at data about user demographics, site traffic, user reactions, and user behaviors. This data enables me to construct a story that accurately represents something real. It may also enable me to create a vision for the future that is grounded in today's reality, or to describe a future that is demonstrably different from today's reality.

With a foundation of good data analysis, I can then create narrative stories that describe reality to my audience in ways that are accessible, compelling, and memorable.

Data helps us tell the story
Once I know which story to tell, I can use data to improve the impact of the story I'm telling. Data enhances the narrative, adding proof points to make it believable, adding context to make it understandable, and adding details to make it compelling.

The pitfall
Problems arise when I mix up the two uses of data. If I'm telling a story, and I grab some data to make my story more compelling, but I have not yet used data to make sure I'm telling the right story, then I'm in trouble. For example, let's look at data and stories about global climate change.

If I want to tell a story about the relationship between human activity and the climate, I need to start by examining the data in order to figure out what's real. If I'm paying any attention at all to science, I will conclude that the story to tell is one in which humans are driving the earth to the brink of climate catastrophe. I can then use data, both qualitative and quantitative, in the telling of that story. E.g., I can cite average temperatures rising, thinning ice sheets, oceanic acidification, and displacement of species. All of these will support my story. These data make my story more accessible, compelling, and memorable.

However, if I skip the first step (using data to determine what is real), then I may tell a story in which global warming is a crazy liberal plot, and I can still use data to support my story--it snowed in April this year; global temperatures historically rise and fall; etc.

I think the critical step is for a storyteller to be self-aware of which of these two roles data is playing. Am I using data to figure out which story represents reality, or am I using data to support the story I'm telling? Both are important, and both are powerful. But if I do one without the other, I either end up with a less compelling story, or I'm distorting the truth.

In-person or virtual?

2010-04-27T19:43:00.000-07:00

As it becomes easier to interact digitally, there will be plenty of opportunities to see your doctor without the two of you being in the same room at the same time. In many instances this will be more convenient, with equal quality, and with potentially lower costs to the health care system.

On the one hand, a virtual visit can be a fabulous thing all around, but there are certainly times when an in-person visit is more appropriate. So the question I'm asking myself lately is,

How do we decide between an in-person visit and a virtual visit?

The easiest, and possibly best, answer is, "Let the patient choose." This initially looks like the patient-centered approach. And we essentially let the patient choose today in most situations, as patients choose whether to call on the phone, email, or schedule an in-person appointment. But I'm guessing that approach oversimplifies the situation.

There will certainly be situations in which the health care provider should strongly recommend an in-person visit, even when the patient initially prefers a virtual visit. Possible criteria for an in-person visit include:

a physical examination is needed
an emotionally intense decision needs to be addressed
the patient and their primary physician don't yet have a solid relationship, and an in-person visit could help establish rapport that could then be carried over into future virtual visits

Conversely, what are the indications that a virtual visit is more appropriate? When should the health care provider strongly recommend a virtual visit? Here are a few possibilities:

the ordeal of traveling to the clinic would be unhealthy
biometrics are needed that can be more accurately measured when the patient hasn't just spent two hours walking, riding buses, and being quizzed by people in white coats
time is of the essence and a virtual visit could take place sooner than an in-person visit (this criteria could be applied, e.g., in rural areas or anywhere that has long distances between patients and providers)

These are some starting points, hopefully raising interesting and useful questions. Here are a few:

Who has done work on this problem? I'm particularly interested in any efforts to quantify the analysis.
What metrics can be used to assess the relative value of an in-person or virtual visit? The obvious starting places are quality, cost, and satisfaction, but we would need some way to measure these across a wide variety of contexts, and I fear the measurement would quickly get too complex.
How can we apply the tools of human-centered design to these questions? E.g., are there opportunities for rapid prototyping of a decision tool to choose between in-person and virtual?
How can we ensure patient safety while experimenting in this space?
What are the implications for pricing of care? Should all virtual visits and all in-person visits be covered, or only those deemed "appropriate"?
Has anyone attempted to bake this decision into a clinical guideline?
Telephone nurse triage systems currently have similar decisions built into their protocols--can we take care of this person over the phone, or do we need to see them? Care models that use telephone MD visits are also relevant. How can we expand what we've learned from these experiences to address a new type of interaction that is typically a less rich than an in-person visit, and more rich than a phone call?

I can't help but come back to the original, simplest answer: an in-person visit is appropriate when the patient wants an in-person visit. The challenge for health care providers is the same challenge as usual: how can we apply our knowledge, expertise, and compassion to help people make sound judgments in the face of uncertainty? Despite the complexities of reimbursement, diagnostics, relationships, and technology, the patient should get to decide. How can we best help them with that decision?

uHealth

2010-03-29T21:29:00.000-07:00

Everybody’s talking about mHealth, the use of mobile devices to improve health and healthcare.

“mHealth will replace the web”

“What’s your mHealth strategy?”

“How does SMS map to your mHealth initiative?”

But I think the focus on mHealth may be misplaced. Mobile devices will certainly enable us to increase the reach of our current efforts, and they will help us do valuable things we couldn’t do before. But focusing on the mobile devices themselves creates a big, all-too-familiar risk.

We risk basing our strategy and actions on the technology, rather than focusing on our users and our mission, and then looking for ways that technology can help.

So I propose a new buzz word to replace mHealth:

Let’s talk about uHealth.

I saw the phrase knocked around a bit a few years ago, but it didn’t stick and there doesn’t seem to be a commonly accepted definition, so I’ll try to bend the jargon to my message. Here’s what I mean by uHealth:

uHealth means health and healthcare that is…

ubiquitous
universal
user-centered

Mobile technologies can help with all three.

Ubiquitous Health

The most obvious benefit of mobile technologies are that they can help promote health anytime, anywhere. We can connect people with their data, their care team, their peer support, and their information regardless of where or when they are needed.

This isn’t ultimately about having access when you’re on the road. It’s about having access on the road, in the clinic, at home, at work. It’s about access via my iPhone, my home phone, my PC, my television, my physician in person, my self-care book, and any other mechanisms that I choose. Mobile health shouldn’t stand alone—it should be an integral part of a larger ubiquitous network of support.

Universal Health

How can we use mobile technologies to extend the reach of our health support systems to those who are currently left out? In the near future, more people worldwide will use mobile devices than tethered PCs to access the Internet. Key underserved groups currently use mobile phones more than desktop PCs, and they use them to stay connected with the people, culture, and information they value.

So as we plan what to do with mobile, we need to be very clear from the beginning that this is so much more than an opportunity to give rich people even more—it’s an opportunity to level the playing field, cross the digital divide, and reach the people who most need help. This sensibility needs to impact everything from feature choices to technology platforms (e.g., texting vs. iPhone apps) to payment structures (how much does that data plan cost? Who pays for the SMS?).

User-Centered Health

It’s not mobile-centered health or technology-centered health—it’s user-centered health. As we look for strategic and practical ways to take advantage of what mobile offers, we need to see mobile in the context of a person’s everyday life. What’s important to each individual? How can this fit into their daily workflow? What big problems to they have that mobile can help us solve?

The principles of user-centered design will ultimately be our most powerful tools for helping us leverage technologies to improve health. If it doesn’t work from the perspective of the consumer/patient/end-user, then it doesn’t work.

And that’s my soapbox proposing the increased use of a new buzzword. mHealth can only become valuable in the context of uHealth. Mobile initiatives need to support health that is ubiquitous, universal, and user-centered.

The User Always Wins, or The False Tradeoff Between UX and ROI

2010-02-22T20:55:00.000-08:00

A few days ago I said to a discouraged colleague, "the user always wins." She said, "not this time--when they made the decision to prioritize business needs over user needs, the users lost."

But I stick to my assertion. When all is said and done, the user wins all arguments, because the user makes the ultimate decisions that drive business value.

Users decide whether or not to buy
Users decide whether or not to recommend you
Users decide whether or not to return
Users determine your brand

So ignoring user needs for the sake of ROI is the ultimate false tradeoff. It's sometimes possible to gain some short-term ROI this way, but in the long run, it's the users who will determine your success or failure. If you only look as far as next week or next month, it may look like ROI can be achieved at the expense of the user experience. But if you look out to the long term viability of your business, it's a fool's bargain. ROI and user experience are forever intertwined.

Business value derives from, and is usually dependent on, a great user experience. When all is said and done, the user always wins.

False Tradeoffs - Time to Market

2010-02-02T18:55:00.000-08:00

We were scheduled to go live in just 4 more weeks. When we finally did serious usability testing, we found out the application was barely usable. Users didn't understand the paradigm we were using, couldn't find their way around, and were generally frustrated with the whole experience. When we described what we were trying to give them, the loved it--they just couldn't use the product we had designed.

So naturally we delayed the go-live date several weeks to give us time to fix the usability issues.

Hah! That'll be the day!

We went live, celebrated the launch, and then spent the next couple of years training our customer service reps to respond to complaints while we built the next round of products. Last I checked we were hoping to fix the problem "in an upcoming release tbd."

It will be a watershed day when we do actually delay a go-live date in order to fix a usability problem. It will be a sign that we've finally figured out how to get the fastest return on investment.

I contend that the faster we rush to market with poor UX, the longer it will take us to get our money's worth out of the product. The easiest time to fix a user experience and fix it fast is before going live. Certainly it's important, critical even, to optimize the experience once the product is live, but basic usability problems in a production environment take a whole lot longer to fix than the same problems before going live. Before going live, there's a magic combination of:

a focused team, deeply familiar with the product
an intensity in the drive to completion that this product will most likely will never see again
few or no encumbrances related to an existing user base already familiar with a flawed product

Use this combination to drive solutions prior to going live, even at the "cost" of delaying. I put "cost" in quotations, because of the fundamental premise of user-centered design:

Principle #1: Business value derives from, and is usually dependent on, a great user experience.

This all leads to...

Principle #2: Rushing to go live with a deeply flawed product delays the release of a user experience that will produce the business value we're funded to produce.

That's what makes the UX vs. time-to-market argument a false tradeoff. One sign of a maturing organization is a willingness to delay go-live in order to fix a bad user experience.

Of course, a sign of an even more mature organization is to never have to act on that willingness, because we never get close to our go-live date before discovering and fixing the user experience. So that brings us to...

Principle #3: The way to make "Principle #2" moot is to deeply engage with users from the very beginning.

False Tradeoffs - Technical Complexity

2010-01-21T19:24:00.000-08:00

Technical complexity hurts the user experience. Technical complexity makes it more difficult to change, optimize, enhance, and scale your design. In my experience, when the technical design gets too complex, only big changes are worth the effort--small changes are just not worth the amount of time it takes to make and QA the change. This leads to myriad small enhancements that get left by the wayside, and a UI that grows stale and stodgy.

So when your developer says "I can can implement your design, but I'll have to hack something together to make it work," you should hear, "This had better be a darned good design, because you may never get a chance to change it."

UX and global sustainability

2009-11-12T20:08:00.000-08:00

I just won a contest! I think I last won a contest of any kind in about 1982.

For this year's World Usability Day, Human Factors International sponsored an essay contest on the theme, "How can the User Experience Community support the future of sustainability?"

It turns out I won the contest, and I will soon be the proud owner of a new Kindle.

Check out some of the other entries as well. Here's a cross-post of my entry:

The user experience community can make a powerful contribution to future sustainability by demonstrating the fundamentals of user-centered design to a world in need of radical new solutions. We can do this because we understand how to solve fundamental problems of human behavior.

First, we can help identify the underlying needs, tasks, and motivations that drive non-sustainable behaviors, e.g.,
Why do humans drive cars?
Why do we spend billions on bottled water each year?
Why do people in poor nations repeat the self-destructive mistakes of rich nations?
Second, we can propose creative alternatives that are perceived as enhancements rather than sacrifices.

And third, we can lower barriers to adoption of sustainable alternatives by designing highly usable solutions.

Widespread success will require our efforts to be tightly focused initially, then highly visible, and then broadly dispersed.

Focused: By Q2, 2010, convene an international panel of opinion leaders from the user experience, sustainability, and business communities to produce a short list of focus areas for demonstration projects.

Visible: From Q3, 2010 through 2011, organizations across disciplines issue calls for papers, proposals, and solutions that demonstrate the power and relevance of user experience to sustainability in each focus area. Using common themes and cross-industry publicity, establish in the public mind the concept that the most practical and powerful solutions to sustainability problems result from user-centered design.

Dispersed: By 2012, the seeds planted by the demonstration projects begin to take root and spread as organizations, on their own initiative, increasingly direct their efforts toward user-centered sustainability.

The global community can only achieve sustainability by collaborating to design and adopt new solutions—solutions that solve our most basic problems, address our underlying motivations, and are easy to use. The user experience community has the theory, the science, and the passion to lead the way forward.

False tradeoffs

2009-11-09T18:51:00.000-08:00

I know I'm in trouble when someone says to me, "Sure, user experience is important, but...." This usually means that the speaker doesn't think user experience is important, and they're concerned that something they care about will be sacrificed for the sake of fluffy user experience. The conversation then quickly turns into assertions that the fluffy user experience people just don't respect engineering, and retorts about how user experience is more important than technical solutions, and when everyone is tired out by the confrontation it ends up in an ineffectual agreement about how important it is to "make good trade-off decisions."

I contend that these trade-off decisions should not be framed as win-lose propositions, where user experience needs to win or lose, nor should they be framed as compromises, where both sides give up what they really need. Obviously, the best outcome is a creative solution that meets user needs with minimal impact to the technical infrastructure. But what if we can't find that solution? It's a difficult conversation in part because we're comparing apples to oranges. And sometimes we really do need to choose between bruised apples or rotten oranges.

I suggest that we try framing the supposedly competing interests as contributors to the same goals. We can treat them either as subsets of the overarching user experience, or as contributors to business results. By using common goals, and even common metrics as the basis of the conversation, we create the possibility of grounding the discussion with a rational and sometimes quantifiable goal--a user experience that drives results. How is this possible in the real world?

Let's take a look at some of the specific needs that can get positioned in opposition to user experience:

performance
technical complexity
time-to-market
sales (or other business results)

Performance
We'll start with user experience vs. performance. The argument goes that we can't implement that JavaScript, or display that image, or place that call to the server, because it would cause too great a performance hit. So we have to make a trade-off decision--will user experience lose, or will performance lose?

This is an apples to oranges trade-off, virtually impossible to resolve rationally, and with no metrics we can use to compare the impact of the decision to both the user experience discipline and the engineering discipline.

We need to reframe this decision as all about user experience, and we need to do that while vigorously defending the importance of performance.

Why is performance important? Why does anyone care about page load times and infrastructure scalability? First and foremost, performance is important because it is a significant contributor to the user experience.

We created a key performance indicator that measures the overall user experience, and then we analyzed the impact of several variables on the user experience scores. Sure enough, performance (response time, site availability, etc.) significantly correlated with user experience. As user's impressions of performance went up, so did their satisfaction with the site, likelihood to recommend, likelihood to return, etc.

And there's one more frame that we can use to turn this trade-off discussion into a more rational dialog: Why else might we care about the impact of a change on performance? Money. Performance can cost big bucks, and if we cavalierly design products that take big bites out of the servers and networks, that costs money. And for most of us, the reason anyone pays us to pay attention to the user experience is because this has been repeatedly shown to drive positive results, which typically translates into either money saved or money earned.

With either of these new frames, we can now compare apples to apples. Which solution will have the greatest overall impact on user experience? Which will best contribute to our financial bottom line? Move the conversation away from a religious war between the UX believers and the high priests of engineering. Make it a rational apples-to-apples equation grounded in common goals.

Coming soon, the next false trade-off: user experience vs. technical complexity

User Experience Metrics

2008-03-13T09:25:00.000-07:00

Metrics-driven organizations help staff focus on what's important by focusing on metrics.

I want to use the power of metrics to drive our organization toward a strategic goal of providing a fabulous user experience. Toward that end, I'm drafting an approach that I'll lay out here, including the implications for both management and for product/development teams.

Step One--Incent management

I've worked with senior management in my group to include user experience (UX) metrics in the annual goals of managers in each product line. They, in turn, will put these metrics into the goals of their product owners. Here's the language for those goals:

Prior to going live, [product/functionality] is ranked by users on the Standard Usability Scale (SUS). Threshold = 70/100. Target = 90/100.

On the live web, the User Experience Indicator for [Audience_segment] will be xx or higher by [date].

To be clear, salaries and bonuses are riding (in part) on these goals. If I'm a manager or product owner, I get more money if I provide a great, usable, experience for my end users.

The remainder of this document introduces these goals in more detail.

Standard Usability Scale
Language for Objectives

Prior to going live, [product/functionality] is ranked by users on the Standard
Usability Scale (SUS). Threshold = 70/100. Target = 90/100.

Background
The Standard Usability Scale (SUS) is a widely used instrument for measuring usability. The SUS focuses on just one aspect of the user experience: usability. It asks users the degree to which they agree or disagree with each of 10 questions:

I found the web site unnecessarily complex.
I thought the web site was easy to use
I think that I would need the support of a technical person to be able to use this web site.
I found the various functions in this web site were well integrated.
I thought there was too much inconsistency in this web site.
I would imagine that most people would learn to use this web site very quickly.
I found the web site very cumbersome to use.
I felt very confident using the web site.
I need to learn a lot about this web site before I could effectively use it.
I found this site was easy to use.

This score will normally require testing with 5-12 users. Our User Experience Team can help a product team determine the optimal number of users. We also plan to build the SUS into virtually all usability tests, and we plan to make usability tests easily accessible to product teams. We want it to be super-easy for a team to know the current SUS score of its product.

Appropriate Use
Use SUS in 2 places:

on an increment of functionality, prior to going live.
on an entire web experience that is already live, as part of the User Experience Indicator for a given audience segment (see “User Experience Indicator” below)

SUS should be administered under controlled circumstances, typically as part of a usability test

The User Experience Team also recommends using SUS as exit criteria for a sprint or release; essentially, an SUS score of 90 is part of the definition of “done.”

User Experience Indicator
Since the SUS measures only usability, we need at least one more metric to measure overall user experience. We have not finished defining a standard metric for this, and we do not have sufficient baseline data, so our 2008 objectives will be to create baselines that will enable us to set hard targets in 2009 and beyond.

Language for Objectives
2008: For [audience_segment], establish a baseline measurement of the User Experience Indicator.
2009 and beyond: The User Experience Indicator for [Audience_segment] will be xx or higher by [date].

The Metric Being Created
The User Experience Indicator will most likely be a combination of 3-5 questions, most likely addressing the following dimensions:

Would you recommend this web site to a friend?
How satisfied were you with this site?
Compared with other web sites, how well did this site meet your expectations?
System Usability Scale

A key piece of our organization's strategy is to provide an unexpectedly enjoyable experience for our users—we want them to say, “wow!” Our assertion is that by combining answers to these questions, we will be able to elicit a reliable measure of the overall user experience, including the “wow factor.” If we have hit the mark with the wow factor, responses to all four of these dimensions will be very favorable.

Appropriate Use
These metrics are best associated with the overall experience of a segment of users (e.g., member, brokers, etc.), with a site already live, rather than for an individual feature or a site in development.

The User Experience Team will create a simple formula for combining these 4 measurements into a single User Experience Indicator and will make it easy for product teams to produce a User Experience Indicator for their product. We hope to have this methodology ready in Q2 2008.

Background & Rationale
The User Experience Indicator is a very high-level metric. It measures our success at providing a great user experience, but it is not intended to tell us why a user’s experience was good or bad. We have a whole collection of tools available to dig deeper into the “why” questions. This metric gives us one simple indicator of how an audience’s experience is improving over time and in relation to other audience segments. Here’s the significance of each dimension of the Indicator:

Would you recommend this web site to a friend?
This is based on the work of Frederick F. Reichheld, as originally published in the Harvard Business Review piece entitled The One Number You Need to Grow. It has since been adopted by a wide range of industries and is gathering steam as a standard indicator of business success. This is the basis of the “Net Promoter” discipline. Here’s a brief synopsis of the original article:

Companies spend lots of time and money on complex tools to assess customer
satisfaction. But they're measuring the wrong thing. The best predictor of
top-line growth can usually be captured in a single survey question: Would you
recommend this company to a friend? This finding is based on two years of
research in which a variety of survey questions were tested by linking the
responses with actual customer behavior--purchasing patterns and referrals--and
ultimately with company growth. Surprisingly, the most effective question wasn't
about customer satisfaction or even loyalty per se. In most of the industries
studied, the percentage of customers enthusiastic enough about a company to
refer it to a friend or colleague directly correlated with growth rates among
competitors. Willingness to talk up a company or product to friends, family, and
colleagues is one of the best indicators of loyalty because of the customer's
sacrifice in making the recommendation. When customers act as references, they
do more than indicate they've received good economic value from a company; they
put their own reputations on the line. The findings point to a new, simpler
approach to customer research, one directly linked to a company's results.

We hypothesize that a user with a “wow” experience is more likely to say they would recommend the site to a friend.

How satisfied were you with this site?
While the “would you recommend” question has many proponents, there are also those who argue it does not adequately address overall satisfaction. E.g., maybe I would recommend this site because it’s the only place in the world I can buy a particular product, even though my experience of the web site itself is horrible. Or maybe I love the site, but I wouldn't recommend it to a friend because it's not relevant to any of my friends (or colleagues). Because of these limitations, we add a basic satisfaction question.

A key limitation to this kind of generic satisfaction question is that it doesn't help us understand why they're satisfied or dissatisfied, so I've heard concerns that a satisfaction score isn't useful. However, in this instance, we're using the score as part of a measure of success, rather than as formative research. If my satisfaction scores are not high enough, I am then incented to figure out why my users are dissatisfied and what I can do about it. We have lots of other tools available to help with those tasks.

The User Experience Indicator will be particularly sensitive to the extremes on this scale. Since our goal is to produce an unexpectedly enjoyable experience, we don’t want people to be only somewhat satisfied, we want them to be thrilled (wow!). So we will initially aim for a threshold of “satisfied or extremely satisfied,” but we will quickly move to a target of changing users from “satisfied” to “extremely satisfied.” For example, in a November 2007 survey of registered members of kp.org, 86% said that they were either satisfied or very satisfied with kp.org. This looks very good and would be a very difficult number to improve. But a closer look at the data shows that this 86% is a combination of 49% satisfied and 37% very satisfied. A meaningful User Experience Indicator would incent teams to increase the percentage reporting “very satisfied.”

Compared with other web sites, how well did this site meet your expectations?
This dimension adds two elements not covered by the previous two:

Comparison to other web sites (across industries). These are a critical to the context in which users access our web sites. It’s important to go across multiple industries, because their perceptions are ultimately based on their experiences with their favorite sites (shopping, banking, blogging, etc.), rather than only with our direct competitors.
Experience in relation to expectations. Our strategy is to provide an “unexpectedly enjoyable” experience, so we want to find out how well we did relative to what they expected. As users’ expectations increase, we will need to continue innovating to stay ahead of their expectations. This dimension helps us understand the “unexpected” part of the “wow” factor.

System Usability Scale (SUS)
See above for an introduction to this metric. When included in the User Experience Indicator, the SUS score applies to an entire web presence for a given audience, rather than to an increment of functionality. It is measured in a production environment.

The User Experience Team intends to provide a framework that makes it easy to regularly measure the SUS for each major audience segment.

Next Steps

To use metrics effectively, and organization needs to do 4 things:

Define metrics that measure what's important
Make it easy to measure these metrics
Make these metrics widely visible
Formally incent staff to meet targets for these metrics

This post is an initial crack at #1 and #4. The User Experience Team will create the User Experience Indicator methodology. As we all collect and compare the resulting data, we will analyze it for validity and will refine the methodology over time.

How does this sound to you? Please comment.

Stages of Acceptance of User-centered Design

2008-03-06T12:54:00.000-08:00

Here's a draft framework for thinking about how people move from not appreciating the importance of user experience to a place where they build it into everything they do.

The idea here is that if you want to create a user-centered product, you need to create a user-centered culture. And in order to create a user-centered culture, you need to move individuals through these stages of understanding. The type of training and influence required for an individual depends on which stage they're in.

Here's a hypothesis for the stages of "getting it" that people need to go through.

You are not your user
Understanding users requires direct contact with them
Knowledge about users must be grounded in real data and must be actionable
Business results depend on satisfying users
Every decision should be influenced by its implications for the user experience
Specialists can really help, but everyone is responsible for the user experience

What's your experience in helping people through stages? What part of this rings true, and what parts need to be removed or revised?

Training

2008-02-12T20:45:00.000-08:00

One of the most powerful roles the small core UX team can play is that of trainer. The team will need to, over time, implement a comprehensive training program that's repeatable enough to be efficient and flexible enough to meet the needs of each team. Here are some early thoughts about the training program:

Run through everybody in the following rough sequence:

Surrogate evangelists (initial development team, key sponsors, product owners, )
People who need specific skills (scrummasters and business analysts)
Everybody else, one team at a time

Early on, pick one or two development teams and do a deep UX initiation with the whole team. The best team for this would be one that is highly respected by other teams that that's already very open to UX values. This team becomes a key evangelist and thought-partner in creating future trainings and in evolving the model.

Next, bring key sponsors on board. These are people in our web organization who manage the overall business portfolios--they manage the overall demand coming in from customers, set the high level priorities, oversee product management, and they usually supervise the product owners and SMEs. Getting these folks on board with the overall UX movement will grease a lot of wheels, help me get a good budget, and prime the pump with everyone else. (After a year of informal evangelism targeted at these folks, along with a great directive from our director, these folks basically understand the importance of user experience.)

I want to do one formal session with key sponsors to give them three things:

Effective language and approaches to use with their stakeholders and their teams. We've learned a lot about how to evangelize UX, and I want these key managers to be as effective as possible.
Introduce them to the base tool set, so they have some background when their staff start talking to them about personas, card sorts, IA, etc.
Give them specific assignments. I haven't worked this out yet, but I want to help them channel their support for UX. Some possibilities are...
Each manager should incorporate some standard language into the formal objectives of themselves and their staff, to give everyone financial incentives to address UX issues. We can give them some boilerplate language as a starting point.
Each manager should ask some specific questions during weekly demos to help the team stay focused on UX, and to help the managers maintain a good sense of where each team is at in this regard. We can help them formulate these questions.
Each manager should let their staff know that they are encouraged, and in some cases required, to attend UX-related training. This gives staff permission to take an hour or two (or a day or two) away from their deadlines to do UX-related training.

Next, Product Owners:

Evangelize them on user experience
Introduce them to the core toolset and the support available
Think together about how to build UX into the product backlog and sprint exit criteria
Work with them on an ongoing shared product backlog for things like creating and implementing design patterns or templates across the site

Next (or at the same time), Scrummasters:

Start with an introduction to the kinds of help we have to offer and how to recognize when they need that help.
Then transition the "training session" to a collaborative working session to come up with some shared best practices and places to innovate, using as a starting point the "UX-in-the-product lifecycle" model (I'll blog on this soon).

After Product Owners & Scrummasters, hit the Business Analysts:

Evangelize--give them something to live for to replace the requirements documents that have dominated their work lives for the last 3 years
Introduce them to the core toolset and the support available
Give them the hands-on skills to do things like user-centered stories, effective use of personas, and card sorts.

By this time we'll have a pretty decent skeleton of a support system in place, and three people on each team who can tangibly make use of general excitement about the user experience, we then go from team to team showing them:

how valuable UX is
how easy we're making it for them to be successful
how to know when they need help

Sound like a lot of work just on training? Yes, but less work than it would be to embed a UX specialist in every team, and in the long run everything will be easier if the development teams are convinced they need user-centered design in order to produce optimal results.

I'm guessing that the core UX team could create the bulk of the training curriculae in one or two short (2-week) sprints. Using 3 or 4 people from the core UX team as trainers/evangelists, we could blow through this program pretty quickly, simultaneously creating demand for our services; enabling self-service; and establishing some shared agreements for how we'll all work together.

What do you think? Is this crazy? Way too much overhead for agile? Or is it a sensible way to leverage a small UX team across many development teams? We'll give it a try and find out. In the mean time, what's your advice?

Core User Experience Team (part 2 of 7)

2008-02-07T20:23:00.000-08:00

When trying to leverage a handful of UX specialists across a large number of agile develoment teams, the first question is, should we just divvy up the user experience (UX) specialists across all the teams? I say no. That would give each UX specialist 5 teams to support, and would more or less require each UX specialist to be good at the whole range of UX tools, from IA to interaction design to user research. I think a 1:5 ratio is too small for this to work. So I'm starting on the premise of a core team that provides support as a team.

Who's on the core UX team?
This team will include...

UX specialists
Creative specialist
Page developer
Product Owner
Scrummaster
Business Analyst
SME who understands the web sites built so far

We start with 2 people with formal training and significant experience doing things like IA, usability testing, interaction design, etc. We've gotten approval to hire two more.

We'll start with one creative specialist (graphic design of UI). He's formally part of a small creative group, so he can call in help as needed, and he'll have help from the creative group in terms of staying on brand, etc.

We'll start with one page developer who will do HTML, javascript, CSS, Ajax, etc. In our case, the page developer(s) will need to really understand how to work with WebSphere Portal (themes, skins, page layouts, etc.), since that will be the presentation layer for much of what we do.

This core UX team will operate as a scrum. Just like any other scrum, they'll have a product backlog (more on that below), sprints, releases, daily scrums, etc. The key difference between this core UX team and the more traditional scrums (are we allowed to call scrums "traditional" yet?) is the nature of the product and the backlog.

A new spin on the product backlog

In most development teams, the product is a functioning collection of software that meets specific non-functional requirements and produces business value. A roadmap might typically include a list of major batches of features, with some infrastructure along the way. And the typical product backlog would be specific bits of functionality that can be coded.

The core UX team I'm proposing will still have a roadmap and a backlog, but the product might be described as "user centered design across the enterprise." An initial product roadmap might include things like...

Training program
Repeatable process for usability testing
High-level IA and wireframes
Design pattern library
Model for financing work requests from development teams
Reusable navigation widgets

The product backlog would include relatively tangible things, like page templates or reusable navigation widgets, but it will also include less tangible "services," like training programs and user research. I could imagine the team focusing a release on an initial training program. In the first sprint of the release, the team might produce all the materials required to train one development team in the art and science of personas and scenarios. Just like "potentially shippable software" of a traditional scrum, the Product Owner could decide whether to go ahead and conduct that training, or to wait for the next sprint to produce training materials that enable a business analyst to conduct card sorts.

Sprints that don't produce code

The team would figure out how to maximize the use of all team members during the sprint--if the creative person isn't needed for the card sort activities, he or she could spend that sprint preparing reusable CSS and background graphics, for which a training could be quickly developed in the following sprint.

We'll have a challenge figuring out how to maintain a unifying theme for each sprint, when not all activities involve graphic design or page development, but this sounds similar to a typical agile team when they're in a sprint focusing on something like upgrading the hardware infrastructure. Team members will go in seemingly unrelated directions, but they'll understand amongst themselves how it will all come together.

Ideally, the team would prioritize repeatable processes that could be used immediately by specific development teams, and would create just enough organizational infrastructure to support the immediate needs (e.g., initially just a sign-up sheet for training; a later release might include a recharge mechanism and a schedule for giving the core training to 200 people.)

These are just some quick examples. The point is that the product backlog contains both software and services, both of which are "potentially shippable" at the end of each sprint. A release could consist not only of software, but of a repeatable process with the infrastructure in place to support it.

What do you think? Does this sound like an agile team? Will it work? What am I leaving out?

Next post will be a draft approach to UX training throughout the enterprise.

Leveraging a small user experience team (1 of 7)

2008-02-06T17:18:00.001-08:00

We have a small handful of user experience (UX) specialists, and we're ramping up our agile development teams. How can 4 UX specialists support 20 teams when the 20 teams are dedicated, co-located, moving on their own schedules, constantly changing direction, and not necessarily on board with the importance of user experience?

We had a good all-day session today mapping out how we want to start approaching this. The central theme is that the small team of UX specialists will operate as an agile product team. But unlike a typical agile development team dedicated to producing executable code, our product will be user-centered design. Each sprint will produce "shippable user-centered design." Sometimes this will look like software, sometimes it will look more like a service. The result will be a large number of teams producing software designed to produce a great user experience.

Here are the key ideas so far...

We are an agile team, and user-centered design is our product.

We form a small core team that uses standard agile methodologies in order to provide user-centered design services and products to all the development teams.

We empower the development teams to do user-centered design.

We put a lot of energy into training and UX evangelism.
We make it super-easy for the develpment teams to get face-time with end-users throughout their work.
We inject the core UX team into development teams at key leverage points in the product lifecyle.

We do hands-on user-centered design.

We create reusable artifacts that have good UX principles built in.
We provide ad hoc consulting and design services to the application teams.

I'll flesh out each of these bullets in subsequent posts. How does this look as a starting point?

User Research as a Commodity (part 3 of 7)

2008-02-05T11:10:00.000-08:00

The Problem
I’d like to describe an approach we’ve using at Kaiser to make it easier for development teams to incorporate user insights into their work. Just about every development team on the planet could benefit from more user research—usability testing, card sorts, label tests, brand reactions, cognitive interviews, etc. The more exposure teams get to their end users, the more user-centered their work will be, and the better the user experience (UX).

In my experience, there are two main reasons why teams don’t do more user research:

They don’t understand its importance
It’s too hard. You have to schedule it into the project plan, find a place to do it, prepare stimuli, recruit participants, deal with incentives, figure out what to test, and then spend time actually testing.

There’s a cool relationship between these two barriers to research: If we can make it super-easy for teams to do the research, they’re more likely to actually do it, and once a team does a little user research, they usually understand its importance and want more. The key is to prime this cycle.

So how do we get this cycle going? How do we make it incredibly easy? How do we change user research from a hassle that interrupts the work to a commodity that can be easily acquired on-demand?

The Challenge
Last year we commissioned an agile-like team to build a new web site for brokers--the professionals who help employers select and purchase health plans for their employees. The team needed to produce a beta in two months and a fully operational site in four months, and they needed to do it on an entirely new technology platform.

The pressure was on. At that time, our typical waterfall timeline for even simple projects was over a year. We typically did a week or two of usability testing in conjunction with the requirements phase, before development, or even technical design, started.

We had already done some ethnography with brokers, and the Product Manager was totally convinced of the need to incorporate usability testing, etc. into the design & development work. But many others, including developers, the project manager, and sponsors, thought of user research as a nice-to-have that was likely to blow the tight schedule.

Everybody was excited about moving to agile, but we didn’t have a clue how we were going to fit two weeks of usability testing into the work. Should we do it mid-way through when we’d have good comps to show users? But we couldn’t afford that kind of a break in the project plan. Or maybe front-load the testing, because the developers needed to take several weeks early on to get their environments in order? But we didn’t yet have any idea what the new technology would allow in terms of the UI. Complicating matters, our usability specialists were in Pasadena, CA, while the agile-like team was in Pleasanton, CA—350 miles away. Not only that, but brokers are difficult participants to recruit—they work in offices all over the country and they have very busy schedules, so the team couldn’t just take a paper prototype down to the nearest Starbucks and ask them what they think (as we’ve done with some other audiences).

When asked, the sponsors and most of the team thought the best thing to do would be to crank out an initial site on a tight timeline, doing the best we could based on intuition and a few heuristics, and then test it with users after we went live. We were going agile, they figured, so we can change it easily after we’re live. After all—it’s only UI. ;-)

Clearly, the UX people didn’t understand agile, and the software people didn’t understand UX.

The Solution: Testing as a commodity
The Product Manager and I were convinced that we needed to expose our work to end users prior to going live, and we could see there was no way we’d get even a week out of the schedule. So we decided to try something new—prescheduled testing. Here’s how it worked:

Every other Thursday morning, four or five brokers (our target audience) would show up at our offices in Pasadena. Our user research specialist would work with each participant for about an hour. While this testing took place in Pasadena, we piped audio and video of the testing up north to Pleasanton so the team could watch in real-time and IM questions and comments to the moderator in Pasadena.

Since we knew the testing schedule several weeks, and even months, in advance, we were able to easily schedule a room for the testing. One of our admin staff, who’s particularly good on the phone, took on recruitment. With multiple dates prescheduled, recruitment was easier—“OK, you can’t make it next Thursday; how about two weeks later? How about the Thursday of the following month?” We also had an admin person manage all the incentive checks, greet the participants in the lobby, and help with set-up and tear-down, all of which was essentially the same each week.

In the early stages testing consisted mostly of card sorts and cognitive interviews. As the project progressed, we moved to various stages of UI, focusing on whatever the team had just built or was about to build—one week working on a page layout; the next week focusing on a particular widget.

Results
The key breakthrough for us was that we made it super-easy for the agile team to get the benefit of user research. They didn’t have to stop what they were doing, they didn’t have to deal with logistics, and the only planning they needed to do was to be sure they had something to show users by Thursday morning, along with some good questions to ask.

It worked great. The team made constant course corrections based on the research. We would typically end sessions by asking the participants, “on a scale of 1-5, how easy was this site to use? (5 is easiest)” In two months of iteration we moved this from a 2.5 to a 4.5. The team could come up with ideas and never have to wait more than two weeks to test the ideas with users.
Sponsors could tune in to view the testing whenever they wanted, and they were delighted to see their customers, the brokers, delighted. The personas became real people. We virtually eliminated arguments within the team about what would work best for users. Instead of pressing the point, the Product Manager could just say, “I’ll ask them this Thursday.”

Why it worked
I can’t stress enough the importance of two key elements:

A regularly scheduled research time, scheduled up to months in advance.
Logistical support from outside of the project team

Scaling

So the pilot was successful. Everybody loves the initial site. Everybody wants to go agile. That means we’ll soon be looking at up to 20 agile teams operating simultaneously, working on multiple sites that support several different audience segments (brokers, Kaiser members, employers, etc.). Can we scale this approach to work in that environment? Here are my thoughts so far…

Brokers are now actively involved in a beta program, and we still have every-other-Thursday prescheduled research available to them as a tool. As we start up an agile team for employer groups, I think we can use a very similar model. The challenge will be with Kaiser members, because we’re likely to have many agile team running simultaneously, focusing on different content and functionality, and attending to different subsegments of the member audience. It sound pretty unwieldy to give each team its own half-day slot every other week.

What if we instead treat the user research environment and participants as a commodity available to all the teams? Every Monday and Wednesday we bring in members all day long. Every Tuesday we bring in employers. Every Thursday we bring in brokers. We post a schedule, and individual agile teams can sign up for time.

Not every agile team will have an hour’s worth of user research tasks for participants each week, but the commodity approach helps out again: We can bring in 8 participants in one day and spend an hour with each. During that hour, we may do a simple 5 minute “find this content” task for Team A, a longer “complete this transaction” task for Team B, and a broad “see if we just broke the UI” task for Team C.

What if teams don’t generate enough tasks to fill up the time this week? No problem--our “Platform UI Team” will maintain a backlog of non-urgent areas to test with each audience, so they can fill in as needed. What if Team D has some questions, but next Monday’s schedule is already filled by other teams? No problem--we’ve got open time on Wednesday’s schedule.
With the recruitment and logistics down to a routine managed by admin staff, and the bulk of the testing budget managed as a shared service (and thus “free” to the teams), this leaves the user research specialists and Product Owners free to formulate good tasks and questions and to apply the results of the research.

Back to metrics
Here’s one more possible extension—we haven’t tried it yet, but we’re toying around with the idea.

Kaiser Permanente as a whole is struggling to become a more metrics-driven organization. Our physicians are internationally recognized for how well they practice evidence-based care, but our business practices haven’t yet caught up. Over the next months and years, our performance will be increasingly judged on metrics, and our agile teams will be measured and incented based on metrics like:

Health outcomes
Sales
Operational efficiencies
Time-to-market
Cost
Backlog burndown

Along with these “bottom line” metrics, we’ll also pay a lot of attention to leading indicators—the metrics that show early on whether we’re moving toward improvements in the bottom line metrics. As we become a more metrics-driven organization, how can we ensure that people pay attention to user experience?

Those who already “get it” will know that the best way to achieve great business results is to provide a fabulous user experience designed around the needs and perspectives of the end-users. But for those who don’t yet get it, time-to-market, cost efficiencies, and even sales can seem to be more important than user experience. How can we use the metrics system to incent people to both perform good user research and to use that research to improve the user experience?

SUS Tracking as a Commodity
A while back we had some significant availability issues with one of our sites—too many unplanned outages. A one-page dashboard of metrics was arguably the most powerful influence for fixing the problems. Suddenly, people at every level of the organization could easily see, on a weekly basis, how many times it experienced slowness, how many minutes it was down for planned outages and how many minutes it was down for unplanned outages. The numbers were bad. The numbers were visible. The numbers got executives and team members to come together to make those numbers improve dramatically.

What gets measured gets managed.

What if we published a similar metric for user experience? What if teams were incented to “bring your user experience metric up a point?” What if teams could see that their recent release moved their score up (or down), and what if all of our peers could see our user experience scores? How could we do that without wasting a whole bunch of time and getting in everybody’s way?

What if each site (or product) had a regularly scheduled UX Checkup? I’m thinking maybe every 2-3 months. This would be part of the user research schedule. The user research specialists could work with Product Owners to create a script . Then every couple of months, without the product team needing to lift a finger, we would take participants to the site in the production environment and run through the script.

Finally fixed that annoying bug? Score goes up.
Haven’t yet implemented the feature the users most want? Score goes stays the same.
Rushed to get a new feature in on time and screwed up the IA in the process? Score goes down.

My incentive pay in 2007 was based in part on an objective of brokers scoring the site at least ‘4’ on a 1-5 usability scale. Could we possible scale this so that everyone is incented to provide a measurably excellent user experience? There are two fairly obvious candidates for this metric:

The System Usability Scale (SUS)
A generic satisfaction question

SUS deals with usability quite nicely, but leaves out other critical aspects of the user experience, whereas satisfaction questions are notoriously unfocused.

The two might work well In combination, with the SUS score assessed in-person and the satisfaction question routinely asked via survey.

Inspirational Conclusion

Creating a fabulous user experience is all about a user-centered culture with access to the tools of user-centered design, combined with the ability to deliver software and operations support. The biggest lever we have for creating a user-centered culture is exposing everyone involved to their end-users in valuable ways. If we can do that, the users will thrive, and so will the business.

One way to promote exposure to end-users is to remove the main barriers to basic user research by offering pre-scheduled user research that is easy, scalable, and measurable.

Please comment!