Last month we asked for your help in shaping the future of the Drumbeat.org web site. As Mark Surman explained, we’re working with Mozilla Labs and Mozilla Research to tie Mozilla’s innovation efforts together — including an overhaul of drumbeat.org and other Mozilla sites aimed at getting people involved.
More than 180 members of the Drumbeat community participated in our survey, sharing their thoughts on:
- what’s working with the current drumbeat.org site,
- what’s not working, and
- where we should focus next.
This post summarizes the survey results. The complete raw survey data is available here (PDF).
How can we amplify Drumbeat’s success online?
Mozilla Drumbeat has been a success. We’ve gathered a great community, demonstrated the value of reaching out to new kinds of audiences, and instilled a participatory, maker-builder spirit through inspiring projects. Drumbeat’s central premise has been proven: we’ve successfully brought together innovators in open web tech with innovators in other spaces like learning and media.
The question going forward is: how can we best support and amplify those successes online, specifically through Mozilla Drumbeat’s web presence?
What you told us about Drumbeat.org
Here are the high level take-aways:
1) You want easy ways to participate and stay informed.
- When asked *why* they joined Drumbeat.org, the overwhelming response from respondents was: to stay informed and get involved in Mozilla projects.
2) The existing Drumbeat.org site isn’t making that easy enough.
- Drumbeat.org isn’t yet fully delivering on that need. 56% of users said they’re “just lurking, waiting for something exciting to happen.” And over half of all users haven’t logged in to Drumbeat.org in their recent memory.
3) We need to provide smaller, easier ways to pitch in.
- The #1 suggestion on how to enable greater community participation was: “Provide small, easy ways for me to pitch in.”
Here’s what survey respondents said Drumbeat.org is already doing well:
- Our focus on education & learning. 73% said Mozilla’s community innovation efforts should continue to focus here. And a majority cited education as the most important area for Mozilla to focus on.
- Reaching out to new kinds of people and projects.
- Involving the community to make Mozilla grow.
Other direct quotes from respondents on what Drumbeat.org is doing well:
- “Having all the projects, people and events in one place to browse.”
- “Providing a way for web builders to connect.”
- “Collecting open source projects with a real-world impact.”
- “Cross-promoting the work of colleagues.”
- “Reaching out to people *outside* of the current community.”
- “Offering several ways people can be involved…whether working on projects or just commenting on them or just watching and learning…”
- “Advancing Mozilla’s mission (beyond software) by providing a place/platform for those that share Mozilla’s ideals and overall goals.”
- “Letting people from different fields and disciplines meet and share their ideas on what the Internet would and should be.”
What’s not working?
- Helping a million different projects. Only a small portion of respondents are using the site to support their own project (17%). And those that have posted projects there are not finding the site significantly helpful.
- We haven’t been able to give all projects the attention and support they need. Many respondents cited the need for greater focus. Narrowing the site to deliver on a single core competency, rather than trying to do too much.
- Other responses on what Drumbeat.org could do better:
- “Add a search functionality to the site!” (+1,000,000)
- “There’s too many projects.”
- “Filtering/selecting/presenting new projects better.”
- “Many projects are not set up to the point where someone can step in and help.”
- “Make it clear how contributors can help.”
- “Figuring out how to keep people active and in the loop.”
- “There needs to be more of an ability to actually connect with projects and people.”
- “Don’t take on too many diverse projects (as currently). Stay focused on a single mission and walking path per session.”
What should we definitely *not* do with Drumbeat.org going forward?
A sampling of your responses on what Drumbeat.org should avoid:
- “Don’t go too wide.”
- “Don’t try to do everything.”
- “Dont loose focus. It happens all too often in web projects.”
- “Don’t accept ideas from everyone without any form of moderation.”
- “Don’t make everything go through a moderation process.”
- “Don’t think that you can be completely hands off about new projects and that the good ones will bubble up to the surface by themselves.”
- “Don’t be too nice about it. Grow a backbone, get tough.”
- “Don’t try to be a social network. Really. Don’t do that.”
- “Don’t get too far down the crowdsourcing path. The site has the right balance right now, and it may make sense to pull back a little bit at this point.”
- “Don’t ABANDON Drumbeat. It´s an excellent work.”
- “Don’t give up :p”
Where does that leave us?
Given this feedback, what are we committed to going forward?
- Expanding participation into Mozilla projects. Making it clearer, easier to get involved and make a difference.
- Continuing to collaborate with people and projects in areas like education and media. That’s clearly working, and we can scale up community participation here.
- Shining a brighter light on successful projects. Highlighting a smaller number of projects that are succeeding in attracting participation and momentum, rather than flooding users with a deluge of different projects.
- Clearer, more focused ways to get support for your project. Mozilla’s new WebFWD initiative, for example, is a way for a small number of promising project ideas to get more direct, hands-on support from Mozilla. Like Y Combinator for the open web.
Things we want to hear from you:
- How do we strike the right balance between being focused versus being open to all? Many respondents cited the need for greater focus. How do we do that while maintaining the right balance of being as inclusive as possible?
- How should we pick the projects we passionately support? If we do narrow our focus, what criteria should we use for choosing those projects?
- What’s the best way to provide smaller, easier ways to pitch in? Given that this is the #1 suggestion for improvement, we’ll need to dig into specifics here together. (We had some useful early conversation about this in today’s Drumbeat call — more on this in subsequent posts.)
Is higher education broken?
Good Magazine cites a growing gap between what learners want and what they’re currently getting. Take stats like this from the U.S.:
- 57 percent of respondents in a recent Pew Research Center poll said they believed that college “fails to provide students with good value for the money they and their families spend.”
- From 1982 to 2007, median family income rose 147 percent — while college tuition and fees grew a massive 439 percent.
- For 67 percent of students, that means getting a four-year college degree requires going into debt.
The result is a tough choice:
Buy into an expensive higher education system you believe to be hugely problematic, or suffer the consequences of trying to earn a living without a college degree, which studies consistently show increase a person’s earning power?
Accreditation is the key to making alternatives work
How do we do better and create a “new ecology of learning?” Good’s article lists a range of emerging alternatives, from Mozilla’s collaboration with Peer 2 Peer University to MIT’s free OpenCourseWare initiative, iTunes University and Knext.
But what’s interesting about the piece is the way it seizes on the value of new approaches to assessment and accreditation, like Mozilla’s Open Badges project, as the key to realizing the full potential of efforts like P2PU:
What sets P2PU apart from things like iTunes U is accreditation. Because unless alternative-education users have an official way to present their merit to potential employers, people with traditional degrees are going to continue dominating the labor force.
The Mozilla Foundation and P2PU are developing “badges,” notifiers to potential employers that a person has completed coursework and is capable of doing what they say they can. “Imagine people could earn badges for their learning, skills and achievements regardless of where those occur or how they are achieved,” Erin Knight, a badge and assessment specialist for P2PU, writes me in an email. “And the collection of badges could serve as a living transcript for each learner, telling a much more complete story about that person than traditional degrees or transcripts.” She later adds, “I think this is the future.”
Mind the gap.
As part of this week’s Hackasaurus game sprint, eight rambunctious kids aged 8 to 11 joined us in the new (and still under construction) Mozilla Toronto office. Our mission:
- Test early game ideas and prototypes. Like X-Ray Goggle training missions, hack mazes, and hackable comic books.
- Brainstorm totally new ideas. Have them surprise us with cool stuff we’ve never thought of.
- Stage a mini dress rehearsal for what we want to do on a larger scale at the Mozilla Festival. Bringing kids, game designers and developers together in London for Hackasaurus design jams and hack sprints this November.
1) Icebreaker: Hacking board games
To get warmed up and into the spirit of the thing, Jess had the kids start by hacking some traditional board games.
Kids hacked and reinvented some traditional board games -- like Snakes and Ladders, Monopoly and chess -- changing the rules and game mechanics to make them more fun.
Ways to hack traditional games. The kids came up with a fifth: adding narrative and back story.
Turning Monopoly into "Hackopoly."
In this version, each game piece has unique powers.
2) Testing Hackasaurus game prototypes
Next we tested some early game prototypes for the X-Ray Goggles. Like Atul’s “Parable of the Hackasaurus,”and an early bare-bones “hack maze” prototype Dan Mosedale and David Humphrey put together.
In both cases, players are given a “broken” page and then use the X-Ray Goggles to fix it using simple HTML.
Tristan uses the X-Ray Goggles to fix broken elements on the page. The page responds in real time, counting down the number of elements left to fix.
3) Kids designing their own game prototypes
Next we divided the kids into three groups:
- A “hackable comic book” group. Working on illustrations and early narrative for an interactive comic-book style experience using the X-Ray Goggles.
- A “build your own game using Scratch” group. Thinking about how you might incorporate the X-Ray Goggles into the gameplay.
- A “hack maze” group. Working on how to create “Portal”-style games where users create and share their own hack mazes.
This group began paper-prototyping a hackable online comic book experience. Players would use the X-Ray Goggles to change the characters and narrative, with remixing elements in HTML and CSS built right into the story.
This group designed their own game using Scratch. In less than 90 minutes, they were able to create their own "zombies vs. machine gun tower" game. They imagined using the X-Ray Goggles to hack leveled-up super zombies back into normal zombies.
Most games have a "training mission" or two to help you figure out how to play. The X-Ray Goggles need the same. Using the "Parable of the Hackasaurus" as a prototype, Atul hacked together this quick training mission (using artwork from the kids) to help users understand how to remix images. Individual training puzzles like these could be linked together into a larger "X-Ray Goggles obstacle course" or "hacker parcours."
What did we learn?
- Testing and co-designing with kids is *always* a good idea. I was worried that we weren’t ready for real kid feedback. Jess helped me see that was wrong — and I’m totally sold. Testing with users — especially kids — is always going to yield interesting results.
- It’s important to really listen and ask twice. Some of the kids would give one response in a group, but then give different answers when asked individually afterward. Like anyone, they don’t want to hurt your feelings. They won’t necessarily tell you “I think your prototype is lame” right to your face. So it’s crucial to ask and re-ask in different contexts.
- Most kids instinctively get and speak game language. They think like game designers. To an even greater extent than anticipated. With complex concepts for character, narrative and game mechanics.
- Some of our stuff is skewing too young. Two 12-year-olds privately described one of our prototypes as “baby-ish” and “cheesy.” Given that we’re targeting a 10 to 15-year-old age group, this is feedback we need to take seriously, and incorporate into our re-branding efforts in Q4 / Q1. Over-earnestness or babyish-ness is the kiss of death.
Portal gives you tests and progressive levels that require you to learn new skills as you go. An X-Ray Goggles "obstacle course" or set of training missions can do the same.
What ideas should we incorporate going forward?
- Prioritize the X-Ray Goggle training missions. We’ve added this to the Hackasaurus roadmap for beta in September. The current X-Ray Goggles page makes it easy to install the Goggles — but doesn’t help you figure out how to use them in a scaffolded, self-guided way. We need to add that next.
The basic skills you need to start messing around with the X-Ray Goggles
- The hackable comic book concept shows promise. Kids immediately got it, and it gave them a chance to get their hands dirty with illustrations and paper prototypes quickly.
- So do the Tilt 3D and “Portal for the web” concepts. Kids liked the video demo of Tilt 3D, and liked the idea of busting hacks as a way to “unlock” pages and do cool stuff to them.
- Unlocking super-gadgets. One of the best ideas was that busting a hack on a page would unlock a super-gadget — like a flamethrower — that you can then use to “set fire to the web page” you’re on. Or unlocking a Katamari ball you can roll around, or jackhammer you could use to bust up the page. This gives regular incentives to progress through various hack puzzles and pages — unlocking cool new web superpowers as you go.
- Consider avatars and characters. This was the other idea we’d never really considered — web characters or avatars that run around the page, jump through portals, etc.
More detail on training missions, comic books and hack mazes coming soon