SCV Workshop:
Clinical Innovation Games

Clinical Innovation Games: A Qualitative Assessment of Innovation Skills in Victorian Clinicians

Posted by Khoa Cao on March 14th, 2024

Collaborators: Leah Heiss, Daphne Flynn, Monika Engelhard, Andrew Carey, Kim Griffiths, Sasha Clements, Felicity Height, Carol Jewell & many others from MNHS, MADA, MIME and SCV

Executive Summary

  • In July 2023, we invited 60+ clinicians to a series of "Clinical Innovation Games" to assess their innovation-related decision-making skills using a qualitative Biodesign framework at a Safer Care Victoria workshop.
  • In the problem/solution stage, most teams missed key considerations, such as clinical observation, solution brainstorming and prototyping. Few teams had a structured & objective process.
  • In the innovation roadmap stage, few teams had a coherent clinical, technical or operational strategy, particularly after reaching their first prototype. Teams significantly underestimated the time required to reach key milestones.
  • Victorian clinicians urgently need higher-quality innovation education. We will launch the first Biodesign course for Australian clinicians in 2024, which will be fully-funded by SCV/Monash. We will also launch a series of advanced Biodesign masterclasses at Monash which will cover regulatory, reimbursement, IP and commercial strategy.
  • This assessment framework provides a more concrete method to understand the reasons behind the failure of a medical technolgy project than the oft-cited "valley of death".
  • Please contact us if you would like to run your own Clinical Innovation Games.



    • Innovation Games
    In July 2023, we invited 60+ Victorian clinicians to collaborate in small teams on a set of "clinical innovation games" designed to reflect real-world entrepreneurial challenges. The workshop enabled us to directly observe and assess key clinical innovation skills - namely the ability to prioritise problems and solutions in a clinical environment, and the ability to plan and sequence the innovation roadmap from idea to bedside. We constructed three activities to achieve this:

    Activity 1: Problem Prioritisation
    Teams were provided a real-world, problem-rich clinical vignette and asked to prioritise their selected problem. We examined the team's decision-making process and considerations.

    Activity 2: Solution Prioritisation
    Teams were provided credible solution concepts for their selected problem and asked to prioritise. We examined the team's decision-making process and considerations.

    Activity 3: Innovation Roadmap
    Teams were asked to construct a roadmap comprising key milestones and activities to turn their idea into a real product. Roadmaps were constructed using the Tactile Tools kit developed by A/Prof. Leah Heiss' team. We examined roadmap content, sequencing and quality.

    Let's make this more concrete with some visuals! One of our vignettes for Activities 1 and 2:


    An innovation roadmap from Activity 3 for a different clinical scenario:



    Following the workshop, we completed a qualitative analysis of the following skills:

    Activity 1: Problem Prioritisation

  • Needs Characterisation: Did teams consider the problem from the user's perspective, the clinical population of interest and a compelling outcome for a potential purchaser?
  • Needs Research: Did teams consider the competitive landscape, market size, recent developments in the disease and potential stakeholders?
  • Needs Finding: Did teams discuss the need for clinical observation and stakeholder interviews?
  • Needs Screening: Did teams follow a structured and objective process to prioritise problems?

    • Activity 2: Solution Prioritisation

  • Concept Generation: Did teams discuss the need for further ideation or brainstorming?
  • Concept Exploration: Did teams discuss the need for prototyping?
  • Concept Screening Did teams follow a structured and objective process to prioritise solutions?

    • Activity 3: Implementation Roadmap

  • Clinical Strategy: Did teams have a plan for clinical testing, positioning and regulatory approval?
  • Technical Strategy: Did teams have a plan for R&D milestones and intellectual property?
  • Operational Strategy: Did teams have a plan for building their team, funding their R&D, commercialisation and reimbursement?


    • We graded these skills using a traffic light system, where red corresponded to no/minimal knowledge and green corresponded to a high degree of knowledge of the abilities in question:



    Results
    Before proceeding, it is worth briefly discussing what a "successful project" looks like. Because of the inherently fragile nature of medtech projects, all bases need to be "covered".


    And this is what a "failed project" looks like:


    It isn't hard to understand why. A medtech project which fails to complete any one of these tasks (e.g. observe the problem in the real-world, brainstorm alternative solutions, consider regulatory strategy) usually has an intractable blind spot that doesn't disappear with additional funding, be it from research grants or venture capital. In fact, many successful ventures are built with the clinical, technical and operational strategy firmly in place before a single dollar leaves the company. As an aside, this structure provides a much more concrete way to discuss why projects fail rather than the "valley of death". These were our results:




    A sea of red and yellows! Some further insights across the 10 teams:

    Activity 1: Problem Prioritisation

  • Only one team stated they would observe the problem in the clinical environment
  • No team considered the purchaser (i.e. the stakeholder who would be responsible for the purchasing decision)
  • Only three teams had a structured process to prioritise the problem - some were based on "whatever they liked best"

    • Activity 2: Solution Prioritisation

  • No teams considered ideation or brainstorming
  • Only one team considered prototyping
  • Only two teams had a structured process to prioritise the solution

    • Activity 3: Innovation Roadmap

  • Most roadmaps had relatively high resolution up until prototyping, but little to no detail after the first prototype.
  • Most roadmaps significantly underestimated the time to achieve certain milestones (e.g. device approval in 2 years).
  • Most teams identified the need for regulatory approval. Few teams discussed having more than 1 clinical trial.
  • Most teams identified prototyping. Few teams referred to actual product development.
  • Four teams identified that intellectual property was required, although no team had a coherent IP strategy.
  • Most teams recognised the need for clinical & technical experts, but only two recognised operational expertise.
  • Four teams identified business models. No teams identified funding milestones or a reimbursement strategy.


    • Limitations
    Innovation games represented an imperfect simulation of real-world challenges and may not have captured all potential options (e.g. the ability to receive advice to consider all these factors). To partially mitigate this, we did encourage teams to write down experts who they would contact, although we found these to be relatively vague.

    Teams were heterogeneous in their constitution and representation. Some teams (such as Team 4 - Veinforge) had several experienced clinical innovators, while others did not have anyone with prior innovation experience. We did not quantitatively assess whether team performance correlated to task performance. It is also likely there is some selection bias as clinicians needed to go out of their way to attend an "innovation workshop" (i.e. this analysis may have selected for clinicians interested in innovation and therefore represent a conservative estimate of innovation skill).


    Conclusion and Next Steps
    It is clear that Victorian clinicians urgently need higher-quality innovation education. In response to these findings, we have launched the first Biodesign course for Australian clinicians in 2024, which will be fully-funded by SCV/Monash and be delivered online. We have also launched a series of advanced Biodesign masterclasses at Monash which will cover regulatory, reimbursement, IP and commercial strategy.