Filed under: Measurement

Who's really listening, and what are you doing with what you hear?

So, the mantra is "Listen to the users. They'll tell you what you need to know". Are you listening? What are you hearing? Are you hearing?
Media_httpmeganmurray_xzpjk
One of the toughest and most culturally revealing aspects of deploying E2.0 is discovering where the egalitarian line will be drawn, and make no mistake, it will be drawn. Let's face it, deployment of an E2.0 solution won't turn an organization into a democracy. The decision to begin a project of the E2.0 ilk doesn't come solely from the passion of ideology or the desire for a more effective and respectful working world. Organizations are taking this path because there is a hope that the bottom line will improve. In a large organizational world user advocacy comes with challenges. For example, in many large corporate environs, few will ever challenge a CIO with "hey wait, the users said they didn't want that". In others you may be able to openly advocate and might over ridden anyway. If you've done your work (and the planets align) you can sell your point. Regardless of culture, it's important as a practitioner to build a case for advocacy. Being careful to speak the language of your audience is a foot in the door. The ability to deliver hard numbers and analysis will greatly improve your chances of selling user perspective. Here are a few ideas on hearing, capturing, analyzing feedback, and delivering it to potentially closed ears.
  • Listen with a spreadsheet as well as your gut. Capture and measure every bit of feedback your users have to offer.
  • Ask for more. If you only hear "I don't like this" ask  "what would you expect to see, and how would it support your work".
  • Take it offline. Talk, in person, to a variety of users. Arrange focus groups, community town halls or social meet ups where you can engage and get more.
  • Use a matrix to capture the variety of messages users can deliver with one comment.
  • Understand the demographics. Not just geography and position, but their location in the adoption cycle as well.
  • Use your collected site stats, help reporting, and user feedback to draw the full spectrum of user experience.
  • When a trend reveals itself watch closely. Use your data to hone in on why.
  • Raise your trends with clear analysis and a succinct message. Have the data ready to back it up, but don't throw it all out at once. A tough message and hard numbers can be jarring if ears aren't ready.
  • Don't stop if you don't get the answer you hoped for. Keep measuring, analyzing and messaging consistently. Adoption works both ways. E2.0 is just as new to the teams supporting it as it is to your users.
Let me know what's worked for you. What do you do with what you hear? Thanks for reading.
Image credit: The Museum of Retro Tech

Adoption and Metrics - Where to start

I began pulling my ideas for this post together by seeking a simple definition of the word adoption. There wasn't a single definition that conveyed the scope I was looking for. It's a common issue with English. A single word with many meanings reflecting a variety of professional or personal scenarios. To some adoption means accepting a new software, processes or ideas. To others it's simply general acceptance with 'approval'. To others still it's gobbling up a whole religion or accepting an unrelated child into a family. In the E2.0 trenches? I'm guessing any of those definitions might sound familiar (metaphorically of course). So with a word that can evoke such varied responses, what measuring stick reflects the truest of realities? What does it mean to say you have xx% adoption?

 

Media_httpmeganmurray_pxgrc

 

 

I believe there is no single metric which reveals absolute adoption, moreover that only your organization can define adoption in a way that's meaningful and further, that the acceptance of those adoption measures will vary across an organizations internal cultural spectrum. Stakeholders want numbers. They want those numbers to climb or fall over time to reflect a wise choice; that the horse they opted to back is yielding the purse they expect. It makes perfect sense. Despite the continued din of conversation for or against traditional ROI applied to E2.0, those of us on the ground will have to offer indicators and analysis. The simplest answer is to focus on the goals stated when you sold the business plan in the first place. Measure the problems you were trying to solve. Your job is to make sure the selected measures reflect reality, are viable in the long term, and not so narrow that they hobble your ability to be agile as you evolve. Much of the work that's being done in E2.0 is the stewarding of new ideas and evolving norms to the organizational world. It's important as a practitioner to know what's working, and what's not. That last bit is key. Knowing which aspects of your project need promotion, clarification, modification, or education will keep you from blindly throwing darts at a target. In a nutshell, analyze often and resist the urge to believe your own press releases. Adoption can be a fuzzy thing. It's important that you build a common definition of adoption within your organization. Does it mean logins, contribution (posts, updates, edits, comments), does it mean 'wins' that can be attributed to your project, and your project alone, or is it some other established success indicator? You have to define these factors explicitly. Like any number of rethinks that you'll encounter, you need to think new when it comes to the metrics you need. Page hits and logins might not reflect the true story. Think back to the problem you're solving and ask "Why are we doing this? How can we make it better, and how can we measure 'better'?". There are a number of creative ideas around RO_ (return on collaboration, adoption, networking... etc.) These ideas are a important to consider as you gain a sense of scope. They are perfect examples of the kind of flexibility you'll need maintain agility. Though it seems the text books are still being written, there are a handful of folks who've got very solid ideas about ROI as it relates to E2.0, though much of the conversation is happening well above the tactical level, or speaks to specific case studies that may not resonate with your project.  Dion Hinchcliffe (as always) does a great job of creating clarity around the discussion. I count his posts/presentations as the right place to start. You can, and really must, create an RO_ which is supported with adoption metrics that reflect the realities of your space. With research and a solid understanding of your goals and culture, you'll be off to a great start. Have you tackled it adoption measures for your organization already? Are you tackling it right now? I'd love to hear how others have approached it. Thanks for reading.

Image credit: http://www.flickr.com/photos/cambodia4kidsorg/3085209738/ - cambodia4kidsorg