Building a community-powered user testing process

TLDR version:

• We want to work with you on user testing for Webmaker. Building a more regular, agile and community-powered process to gather and act on feedback from users.
• The goal: make Webmaker.org work better for the people we most want to serve. Especially lead users: teachers, informal educators / Hive members, and techies interested in teaching.
• It’s more than just “user testing.“ It’s about bringing our community closer into the design and build process — to co-build, share ownership, and take Webmaker’s user experience to the next level.

How to get involved

1. Attend our prototype user testing event in Toronto. We’re hosting an experimental user testing event on Jan 24 in the Mozilla Toronto community space. If you’re a teacher, informal educator or Hive member, please come and test, hack and play with us!
   • More events will follow. This one’s  just a prototype we can learn from, document, and hopefully spread everywhere. 
     
2. Help build a new Webmaker User Testing Kit. So that other community members can host their own user testing sessions.
3. Sign up for updates and discussion on the Webmaker newsgroup. To discuss next steps, share thoughts, help edit and localize the new kit, etc.

Prototyping a user testing Science Fair

We’ve done user testing sessions for Webmaker in the past, and we’ve learned a lot from them. The goal now is to make them more regular, actionable, and community-powered.

“Watching real users can give you that eureka moment. It’s like travel: a broadening experience. You realize that the rest of the world doesn’t live and think the same way you do. This profoundly changes your relationship to users, making you a better developer, designer or manager.” —Steve Krug

To that end, Karen Smith and I have been working on a lightweight event model. Here’s the  recipe we’re using for our prototype user testing event on Jan 24:

• Invite community. Ideally, a representative mix of our lead users: teachers, informal educators / Hive-members and techies interested in teaching.
• Set up user-testing “stations.” Like a Science Fair. Participants can then wander casually from station to station. Anyone can propose something they’d like to test; it’s open!  Could be a new feature, new tools like Appmaker, observing users performing a task on webmaker.org, etc. Participants should feel free to bring and test their stuff, too.
  • The draft list of user testing stations for the Jan 24 event is here.
• Offer a mix of testing, training and fun. To ensure participants get as much value from the process as possible. (We know some of the Jan 24 participants are curious about using Popcorn, for example, so we’re offering a “Popcorn Crash Course” station as well.)
• Document the testing results. Ensure our testers have tools, methodology and best practices for documenting the results and key learnings they come away with. The challenge is to make this data actionable — easily filed and acted on.
• Give swag. Make sure we reward our user testing participants with some nice swag and rewards for their hard work.
• Make it a social mixer. Once we’re done testing, we’ll share some drinks, snacks and conversation. To bring Webmaker designers, developers, and users closer together in casual conversation and idea sharing.
• Document the event. And then turn it into a Webmaker User Testing Kit others can use to host their own events.

Help build a new Webmaker User Testing Kit

I just saw this bug and I am damn interested to help you out. Can I be told how to join you and help testing webmaker? (p.s. I am a peer in the webmaker task force of Bangladesh) –Tanha Islam, Mozilla Reps

The ultimate goal, of course, is to build off this prototype towards a community-driven system that scales. Since we first started working on this ticket, we’ve received requests from a number of community asking how they can get involved. There’s already big interest in hosting user testing events for Appmaker in Brazil, Webmaker tools in Bangladesh, and beyond.

We think the best approach is to write and localize a Webmaker User Testing Kit together. Like our current Webmaker event guides, but specifically geared towards user testing.

We haven’t started writing it, yet.  So let’s get started on it now! If you’d like to get involved in building a Webmaker User Testing Kit, here’s how:

1. Add yourself to this ticket. That’s where we’ll track and manage the process of writing, designing, and shipping it.
2. Add ideas and links to the draft. What are you already using to do user testing? What links or resources would be good to share with others? What steps should people follow?  We’ll write the kit in the open together in this etherpad.
3. Got more questions or comments? Join the discussion on the Webmaker newsgroup. Or get in touch with Karen Smith or OpenMatt.