June,
2006
Summary
There was a lot of chapter business this month. Various members of the Fidelity usability team gave condensed versions of their UPA conference presentations. Those interested also got a tour of the Fidelity usability lab.
Chapter Business
Chris advised us that August is when submissions Open for Presentations, Papers, Panels, Tutorials, Advanced Topics and Workshops for the 2007 UPA conference in Austin, TX. He also told us how much other chapters are in awe of the Boston UPA and want to learn from us. Jay, a member of both the Boston and New Jersey chapters, gave us a brief overview of his chapter, and Kayta told us about her talk in Moscow. Our next meeting is July 19th at Museum of Science. It is now possible to register online for the Deborah Mayhew workshop in August. Our August meeting will be on the 23rd, and it will feature Jared Spool at Molecular. He will talk about optimal design teams. On Sept 20th at Staples we will have 10 minute talks; talk to Susie Robson if you are interested in speaking. The Oct meeting will be at MIT. PJ Gardner talked about her upcoming accessibility event, where teams of web developers have one day to build an accessible web site for a non profit. Talk to her if you're interested. As usual, folks looking for jobs and those hiring gave brief overviews of what they're looking for.
World Usability day is 11/14, and we'll need volunteers, teams, and activities! We will be having a brainstorming meeting about this on a weeknight in early July. We received a "volunteer group of the year" award for having the most complicated World Usability Day production of anyone in the world! This year Karen is doing national and international PR for WUD, and we're seeking larger sponsors to make the event larger, better, and stronger, plus more engaging to the general public. The WUD web site is up, and you can volunteer there. They are also looking for information on accessibility, and want content editors for their web site. WUD events will be web cast, and we are trying to get WUD proclaimed by the UN!
Presentation Detail
First, Laura talked about an innovative and potentially controversial facial tracking technology, designed to detect and score participants' reactions when performing a task as positive or negative (i.e. "struggle" reactions). We saw recorded demos of the FaceIt software, and Laura asked us many thought provoking questions to illicit our reactions. Is this a replacement for usability specialists? How do we expect it would be perceived/received by clients? We discussed how it might deal with a 50/50 split (relative score + time on task), how accurate it was (would it require calibration for individuals), whether would it distinguish expressions from involuntary movement, and whether it could be used with heuristic evaluations with many experts. We talked about its cost and simplicity being dangerous because clients would love it, how any talking would have to be done post task, and whether it could distinguish laughing "at it" from being happy. : We thought it might be good as a preliminary indicator of problematic areas in which further testing might be required. We were concerned that some folks might react by smiling when they're really nervous or unhappy, and talked about cultural and gender differences might throw things off. A lively discussion then ensued about the following situation: "You're having trouble with a web site, and a built in camera detects this in your facial expression and then automatically notifies an agent to help you." Privacy issues were discussed. Chris thought it might be helpful for monitoring patients for pain in hospitals, etc. Some thought it would be helpful for remote testing.
Next, Ellen and Michelle talked about the Fidelity IVR system for older users. Age affects web usage, but would it have similar affects on IVR? In 2004 they replaced their touchtone system with a completely voice driven system, and complaints from those in higher age brackets soared. 51% of callers were 60+, and preferred the phone to the web. No older users were tested prior to the rollout. Physical and cognitive changes in hearing, motor dexterity, working memory capacity all affected the ease of learning and adoption of this new technology. The task success rates decreased as a function of age, because older users had unrealistic expectations of working with the system (added too many conversational words), because the pace was too fast, because too many menu choices resulted in information overload, because there was no context for jargon/terminology, and because older users were too polite to the system. They would not interrupt and wanted to be prompted to hang up. However, they were willing to try it over speaking with a live representative, and it was deemed as good for those with arthritis. Recommendations for improvement included: offering both speech and touch tone options, changing the tone of the system, transitioning users better to the new system, being consistent in the mechanism (all touch tone or all speech), use lower voices, provide better spacing etc. Perhaps even provide a modified system for older users when they identify themselves to the system.
Next, Tom and Donna talked about whether any one method for gathering post task ratings was best. They asked internal users to perform various tasks on their Intranet, and used various methods for gathering post task ratings: 5 pt Likert scale with standard question; same with rewording of question; Lewis' ASQ (satisfaction with time on task), Albert and Dixen before/after expectation ratings (quadrant), and McGee's usability magnitude estimation (ratio, in relation to other experiences; supposedly better accuracy due to lack of boundaries). They had a hard time comparing the before/after ratings so just used the after, and normalized the data for magnitudes. For all post-task methods, there was a clear pattern of results that indicated all methods worked very well. However, they had a large sample of users and wanted to know whether smaller samples would still hold up. Turns out the standard post task question is best for smaller samples.
Next, Marguerite and Ann talked about testing 1) a section 508 complaint web site and 2) a section 508 compliant web site WITH usability added with JAWS users. JAWS users do not have a good sense of hierarchy or relationships. The second site had some extras like 1x1 pixel images with alt tags providing an overview of what was on the page and what they could do there. There was also embedded information about various access keys and clear headings. Success rates for tasks were slightly higher with site 2, but time spent per task went up slightly, and subjective ratings were only slightly higher. They might do some things differently next time (i.e. make overviews shorter), but believe that a site that's technically accessible doesn't ensure that it's usable. There's a local users group in Boston (Vibug) that has members who are interested in helping make web site better for visually impaired users. When testing with this population, one MUST pilot, double the time it takes for the task, provide tasks in Braille, and often provide transportation. JAWS does not play nice with Morae. Universal design is possible; what benefits visually impaired users benefits all.