November 11th, 2015

Week 5 – Idea Testing

By mary

Week #5 for the Education Shield's online learning innovation group (code-named MCOLi) was all about zooming in on a particular idea and starting the process of testing it.

To recap, our research showed that cases are a compelling and efficient way for healthcare practitioners to adopt new working knowledge (this will likely not be a surprise for anyone). We believe that the discussion around cases can deepen learning on the content presented in cases, and our hope is that such discussion can be made asynchronous and online.

When we begin testing an idea, the first thing we do is run an assumptions and risks exercise.  We cannot test everything, so we need to figure out what is most important.

First, we lay out our current belief system for the idea -- what it is, who it is for, how it makes money, etc.  Then we ask a critical question: "What assumptions do we have that, if proven wrong, would cause this to fail?"

The next step is to loosely rank these risks by the level of uncertainty (if there is little evidence to work from, it is highly uncertain) and the impact on the business (high impact means it could seriously help or damage us).  The following graph might be too small to read, but it gives you a taste of the output of this exercise:


Next, we map our experiment plan against the most important risks, like so:


And lastly, we try to keep a running check on how well our risks are faring given our experiments.  Things that are positive are flagged green, and things that are at risk of being invalidated get marked red. As you can see, we have just begun:


During the week, we did a fairly thorough competitive review, looking at everything from Figure 1 to QuantiaMD and various virtual patient applications. We also kicked off another wave of customer development -- this time broadening our range beyond NP/PAs to consultants, fellows and residents, RNs and physical therapists.

The noisy competitive landscape proved that there was interest and activity in the space, but made us want to attempt to leapfrog the crowd a bit. While we are hoping that access to fascinating Mayo Clinic cases will be a differentiator, that cannot be all we rely on.  We are thus testing out a product design risk and focusing not just on mobile but on mobile video for both the cases and the discussion.

Next on our plate is a working prototype that will allow us to share a case and see what kind of discussion does or does not unfold.  You can see a quick sketch below of the simple interaction:


We have set three success metrics for this prototype:

  • More than 50% of invited participants will try the prototype
  • More than 30% of users who view cases will respond with a question or comment
  • More than 40% of users will state that they have learned something from the discussion of the case

Next up (underway as we write): first, recruit our first sponsors who will contribute a case and help us gather together an invite-only "team" for discussion of the case; second, put the prototype into the wild and see if it sinks or swims!

Tags: Testing, Weekly Updates


Please login or register to post a comment.