The full report is available here:
http://mzl.la/User_Testing_Report_2013

Executive Summary

The Webmaker.org user experience team conducted user testing at Mozilla Toronto between August 9 and October 1 with 11 users.  The goal: examine the navigation and site architecture, and whether users understood  the concept of remix.  Sessions lasted approximately 1 hour.

Key issues

• The navigation between WebMaker.org and the individual tools is unclear (e.g., no consistent way to return to the homepage)

• The meaning of X-Ray Goggles, Thimble and Popcorn is not intuitive to users

• Publishing and saving is an unclear process for users

• Help is not easily available in Thimble or Popcorn

• Search is confusing for some users.  The filter option on the homepage is sometimes confused for search, and the magnifying glass icon is missed

• Deleting a user account does not result in data being erased from the site (this is being fixed)

Bugs to review / triage / file as a result of this testing:

Users had problems navigating between the tools and the Webmaker.org homepage.

• Current fix: a logo to link back to Webmaker.org homepage was added to the right side of the interface on Popcorn and Thimble.
• Next step options: Creating a breadcrumb link below the Popcorn/Thimble logos could be more intuitive than having a logo on the far right of the interface.
  •   e.g., “Webmaker.org  |  Popcorn “
• Could consider making the Webmaker link dynamic like the Mozilla one it replaces, so that the Webmaker site functions more consistently.

Thimble: Confusion with save/publish  
In Thimble the “publish” option appears only after sign in. On Popcorn there is a greyed out save. (Center interface)

• Short term options: Have both Thimble and Popcorn include the “Publish” option (after ‘saving’ in Popcorn, a link is generated and it is effectively published).
• Move the “Publish” options to the same location (center) with consistent greyed out buttons until the user is signed in.
• Long term options: Add a “Save Draft” option to allow users to login to edit their work over multiple sessions before they publish.

Thimble: Lack of help for novice users or those new to the Webmaker.org interface

Options:

• Provide a “tour” option for new users
• Do research on other interface options, such as arrows between code and the rendered website
• Better labelling of makes to direct users to appropriate projects.

Thimble: “Show hints” doesn’t do anything for most users

• Make this more specific (i.e. show HTML tag meanings to produce captions pop-ups on rollover of tags)

Thimble: Lack of colour picker

• Add a colour picker to give users the hex values to type into their code.

Thimble: users expected “content upload ” (e.g., upload image)

• An image uploader in Thimble (similar to in Popcorn) could be useful.
• Creative Commons/Public domain content could be emphasized as an available resource.

Thimble: Lorum ipsum text in starter makes confuses some users

• Avoid using lorum ipsum text in starter makes and edit it out of existing examples

Popcorn: Lack of help for novice users and those new to the interface 

• Link to a tutorial or intro video for new users

Popcorn: the “event” tab is not noticed by users (i.e. they can’t easily add text, images or maps)

• Have a layer type selector when users add a layer (i.e., blank, video or text) and dynamically change the tabs on the right of the screen to events if they choose text

Popcorn: Zoom issue.  Sometimes users zoom out too much and can’t find their project anymore. 

• Provide a “Fit to timeline” option to automatically scale multimedia for easy editing.

Popcorn: Audio loop issue

• Fix so that audio files can loop not just video

Popcorn: Copyright concerned users

• Allow for a search of Creative Commons licensed materials where available (i.e., Flickr, YouTube)