With that said, here's who & what I've been reading recently on the Web...
]]> The IA Institute]]> Attendance was up (over 500!), and the quality of the programs was as high as I can recall.
Plus, my panel session went well...very well indeed. I think we might've drawn about 100 people. Considering that some of the sessions I chaired as a volunteer had fewer than 15 people, that's nothing to sniff at.
My presenters did great. On first was my grad school bud Wendy Castleman. She presented on how she and Kaaren Hanson, both of Intuit, have managed to integrate and grow the user-centered design function within development teams at Intuit.
Mary Theofanos of NIST presented a case study about the redesign of the HHS web site.
I presented my (somewhat long-in-the-tooth) case study describing my redesign of an IVR system for my former company. I know I've gotten more comfortable presenting in front of large crowds, but I was pleasantly shocked at how well I did. The session evaluation comment sheets said I was "engaging", which is doubly shocking if you've ever met me in person. And somehow I managed to get a couple of big laughs from some throwaway lines...I can't even remember exactly what I said.
Finally, Dr. Susan L. Hura gave a very good talk about how she established usability engineering in the design of voice-enabled applications at Intervoice.
Then there were the extra-curricular activities. I usually hide out in my room watching movies or surfing the 'net when I go to conferences. This one was different. I found myself hanging out with a large and diverse group of UPA'ers, and really enjoying it. Particularly that (Belgian? French?) brewpub on Wednesday night. This place had four-foot tall, test tube-like pitchers with spigots on the bottom. Good call, Mr. Soggy Pants...
]]>I claimed that this happens because enterprise software vendors don't set goals for the learnability and usability of their systems, and because the enterprises themselves don’t hold vendors to high enough standards of application learnability, usability, and efficiency.
In this entry I'll relate some case studies where negative outcomes could have been prevented. I'll also discuss why the factors that contribute to these poor outcomes seem to be persistent.
In the next post, I'll provide examples of how to justify usability for enterprise software, and discuss a model for creating and deploying enterprise software that will result in more positive outcomes.
]]> In the last entry I argued that enterprise software often falls short of the mark when enterprise vendors don’t pay sufficient attention to the specific wants needs of intended users.I claimed that this happens because enterprise software vendors don't set goals for the learnability and usability of their systems, and because the enterprises themselves don’t hold vendors to high enough standards of application learnability, usability, and efficiency.
I this post I relate case studies where negative outcomes could have been prevented. I also discuss why the factors that contribute to these poor outcomes seem to be persistent. Finally, I relate a model for creating and deploying enterprise software that will result in more positive outcomes.
Case 1: The “Business Intelligence” System
The technical support group for a major financial software application was responsible for generating weekly and monthly reports on calls to the help desk. Statistics on call burden, call reasons, call resolution time, hold time, post-call work time, and other statistics tracking cost and rep productivity were summarized for management. The process of producing the reports was mostly manual – the data resided in three separate systems, data was extracted via complex (though mostly repetitive) queries, and reports were generated and formatted in a spreadsheet application.
Other groups within the organization – product management, development, user-centered design, and quality assurance - frequently requested custom reports and raw data from the amalgam of systems. Individuals within technical support and the IT group were sometimes overwhelmed trying to deliver their regular reports and comply with “outside” requests.
The business decided to deploy an enterprise-level application enabling support to centralize the data, automate data retrieval and report production, and provide outside groups with the ability to self-service their data and reporting needs.
Upon deploying the application the business discovered that, even after support and IT staff trained to proficiency, it took 5 to 10 times as long for staff to extract data and produce reports. What these groups discovered was that the application’s user interface, which was presented via a Web browser, only allowed users to “drag and drop” date ranges, field names, and other delimiters into a “reporting wizard” form. Complex queries that previously took 5 – 10 seconds to type now took 2 or 3 minutes to drag, drop, delimit, and run.
These users, with considerable technical abilities and expert-level knowledge, were essentially forced to interact with the system as if they were neophytes).
Furthermore, the reports were formatted into static HTML. Once produced, users were unable to reformat, rotate, or otherwise adjust the output. Although the simplistic process of query-building proved easier for occasional users from outside groups, the inability to manipulate the output proved frustrating. As a result, support and IT staff were again forced into a bottleneck role, laboriously creating reports to comply with outside requests for reports.
Within six months, the technical support group had brought their old systems back on-line, and reverted to their previous process.
Case 2: The Expense Reporting System
A large telecommunications equipment manufacturer decided to move from spreadsheet-based expense reporting to a system that enabled users to input expense information directly into the company’s accounting system.
The application promised to eliminate manual steps, including double entry of data, remove data entry bottlenecks, and streamline the accounting process.
Employees at this company had an inkling that the new system might pose difficulties when, two weeks prior to the system rollout date, HR disseminated a 50-slide training presentation to all employees. This was followed up by a mandatory, all-employee desktop video training session in the use of the new system. The training session, developed jointly by HR and IT, was 1.5 hours long.
The productivity lost to training for the new system was a significant expense, as was the projects related to producing training. These expenses were dwarfed by the time and expense lost when the system went live.
Everything about the application’s user experience was a mess. The process employees followed to enter, describe and categorize expenses was confusing, long, and ill-thought out. The screens to capture the data were poorly designed. The terminology used throughout the application, while familiar to finance and accounting professionals, was opaque and unclear to most other employees. Information was presented in illogical formats; users were forced to scroll through 200-item drop down lists with nonsensical ordering.
Successfully submitting an expense report, which had previously taken only a few minutes, was now a half-hour undertaking fraught with error and frustration. As a result, productivity and morale suffered. Worse, compliance waned and systemic errors were propagated through the accounting system: some employees simply stopped expensing small purchases, or assigned expenses to accounts that appeared near the top of long account lists.
The Vendor’s Lament: If You Build It, They Will Complain
With their page-spanning feature matrixes, long lists of supported platforms and databases, ROI calculators and downloadable case studies, the enterprise application provider makes fantastic promises. However, many enterprise software development environments don’t adequately incorporate users’ specific wants and needs until too late in the engineering process (if at all).
So why weren’t the user’s wants and needs represented earlier in the development lifecycle? There are many reasons. They can be boiled down to this short list:
1. The application is built to satisfy the vendor’s perception of users’ needs, not users’ actual needs.
2. Engineering groups own too much responsibility for user interface design.
3. “Featuritis”
The vendor is not the user: Often, applications are built without incorporating the perspective of actual user groups. Product managers, requirements analysts and engineers make assumptions about users, instead of observing them and asking the users themselves.
Gathering users’ wants and needs is not difficult, but it’s important to do it right. It’s remarkably easy to gather user needs poorly or incompletely, resulting in a biased or incomplete perspective of users’ wants and needs.
The key to developing an accurate picture of user needs is to distinguish the main user groups (and how the groups differ), then identify the users’ skills, tasks, and needs in the role they assume while using the application. It’s also helpful to usability test conceptual prototypes with actual users from the target customer groups. In this way, early concepts and designs can be tested and iterated very inexpensively.
Engineers are not the users: In many development organizations, engineers are given responsibility for transforming requirements into user interactions, process flows, and screen designs. What results is a user interface that reflects engineers’ mental models. Their models for how things work differ drastically from users. Consider this example:
The engineer: “It’s a state-persistent container for database objects...it requires authentication and setting of cookies, etc etc...”
The user: “It’s a shopping cart.”
Indeed. To the engineer, it is in fact a "state-persistent yadda yadda." But not to the user, it ain't...
Featuritis: Featuritis is a pernicious malady. Both vendors and purchasers contribute to this disorder.
Here's what happens on the vendor side of the equation:
“Competitor A has these 5 features..competitor B has those 10… we’d better put them all in our next release.”
This kitchen sink approach leads to a mishmash of features, with no organizing principle or overarching information architecture.
Purchasers applications with undocumented usability in the door because they don’t know any better.
The evaluation and decision-making process for enterprise applications usually looks like this:
The need for a better, more scalable, faster, etc. process is identified.
The business case is established
The IS organization sets technical and feature requirements (often informed, in somewhat circular fashion, by vendor’s application feature lists)
Vendors are solicited, and sometimes asked to respond to a Request for Proposal, or RFP
Vendors are evaluated on the basis of their responses, and a short list of vendors is generated
Vendors’ systems are often brought into the enterprise’s test labs for performance and technical trials
A vendor is selected, and the deployment project is undertaken.
This process typically does not provide methods for evaluating the goodness of fit between the enterprise users’ processes, wants, and needs and the vendor’s solution. Many a rollout disaster could have been avoided simply by usability testing vendors’ solutions with employees during the trial phase.
Solution:
I propose that user-centered design methods and usability testing can aid both applications producers and application purchasers.
Application vendors can utilize user centered design methods as a competitive advantage, to produce solutions that meet the enterprise users specific wants and needs.
Enterprise customers can utilize usability testing to ensure that the IT investments they make deliver fully on their value propositions.
Next post: some examples of how to justify usability for enterprise software, and a model for creating and deploying enterprise software that will result in more positive outcomes.
For enterprise employees who must use the enterprise application, their complexity poses a considerable challenge. When an application is deployed, users are expected to learn the new system, integrate it into their existing work processes, and become proficient enough to allow the organization to realize the system’s full benefits. Far too often, however, enterprise employees find these new systems hard to learn, hard to master, and difficult to integrate into existing processes.
]]> Enterprise software, which broadly encompasses functions such as enterprise resource planning and management, customer relationship management, supply chain management, project portfolio management, and business intelligence, is a multi-billion per year industry Well-known vendors include BMC, Business Objects, Hyperion, i2 Technologies, PeopleSoft, Oracle, SAP, and Siebel, to name a few.Most Fortune 500 companies have multiple enterprise software products installed and many mid-sized business are either actively considering or have implemented solutions. As the market has matured and vendors have searched for new growth opportunities, even small businesses with fewer than 100 people have options available to them from enterprise software makers.
To the growing company, enterprise software promises to convey benefits in a variety of areas, for example:
Centralizing customer information from sales, marketing, customer service, and support to improve customer service and enable better prospect identification.
Identifying and managing enterprise-wide resources needed to receive, process, and account for orders.
Gathering, storing, analyzing, and providing access to data to enable enterprise users make better business decisions.
Tracking and organizing information related to project planning, tracking, and resource management for multiple projects, enabling an enterprise-wide view of project scope, resource allocation, risk, cost, and performance.
Complex stuff.
From the CTO's and IT Director's perspective, these promises assume that the internal user groups can and will learn the new systems and incorporate them into their work processes. But these outcomes are far from assured. Some of the problems and pitfalls:
Some businesses find that their employees’ productivity decreases because common or critical processes actually take longer using the new application. Others fail to realize an application’s benefits because users “vote with their fingers” and don’t adopt the new system.
Businesses can also experience reduced employee morale and increased turnover related to the imposition of new systems and processes. There will always be employees who resist change in any form. However, if the business mandates process changes and deploys systems that users perceive as difficult to learn, use and remember, the user population will see it as a change for the worse. In this situation morale will decline, and the sufficiently disgruntled will leave.
IT organizations responsible for supporting an enterprise application can find themselves overwhelmed as they struggle under unexpectedly high numbers of support requests that often accompany an application rollout. As anyone who’s worked the help desk knows, rollout day for a complex application often seems like “the perfect storm” for Level 1 support staff.
Why do these scenarios play out in organization after organization? I argue that two factors are driving these outcomes:
1. Enterprise software developers don’t pay sufficient attention to the specific wants needs of the internal user groups.
2. Enterprises don’t hold their vendors to high enough standards of application learnability, usability, and efficiency.
Next entry:
Some case studies where poor outcomes could have been prevented.
Why the factors that contribute to these poor outcomes seem to be persistent.
A model for creating and deploying enterprise software that results in more positive outcomes that these products can deliver.
]]>Once the test session begins, the facilitator makes every effort to ensure that the participant is free from distraction. Other than asking for occasional clarification, the effective facilitator is trained to interact in a neutral, non-judgmental, clinical manner. This allows the participant to concentrate on the tasks at hand.
However, the typical office worker deals with a range of distractions and interruptions throughout the day - some self-imposed, some fromexternal sources.
Given the ubiquitous nature of distractions and interruptions, it might make sense to replicate some of them in the test environment. Doing so in a controlled, deliberate manner would help illuminate how the product might fare under real-world conditions.
As a hypothetical example, consider an intranet site within a large corporation that allows employees to enroll in and make changes to their benefits. Imagine also that the benefits enrollment process was user tested in the usability lab with a variety of user types ranging from administrative assistants to software engineers. For our purposes, let’s assume that it earned high marks with users during this round of testing.
However, when it’s rolled out the organization finds that a significant number of employees are committing errors when enrolling or making benefits changes. Further investigation reveals that errors are most prevalent among tech support employees and mid-level managers. Why wasn’t this revealed during user testing?
One reason could be because tech support and managers’ work time is characterized by frequent interruptions and a multitude of distractions competing for their attention. While the benefits enrollment process might “test well” in the serene confines of the lab, in this case efficient use of the web site’s interface is hindered by frequent phone calls, emails, and the vagaries of working in a team environment.
It turns out to be difficult for people to ascertain the status of the application and of the operation being performed when a constant stream of distractions and interruptions characterizes their work environment. Had this been discovered before rollout, the design could have been adapted accordingly.
Studying how people use a product under conditions that replicate the distractions and interruptions of ordinary life would reveal additional information about the product that would not necessarily be revealed by traditional laboratory testing,
According to the UPA, the goal of World UDay is "to promote the fields of usability engineering and user-centered design. We aim to do this by encouraging, organizing, and sponsoring activities at the local level around the globe, all occurring on November 3, 2005."
More here.
]]>