VN IPP - Week 1 - Session 04 - Experiment Design Basics & The Real Book
Experiment Design Basics & The Real Book Learning goal: Which value prop is better? Faster download time or obfuscated IP address Hypothesis: Using VP of Faster download time instead of obfuscated IP address will increase conversion rate on home page above 15% Plan: Use optimizely to change VP on home page for 2000 visits Time box: days Learning goal: Which value prop is better? Faster download time or obfuscated IP address Hypothesis: Using VP of Faster download time instead of obfuscated IP address will increase conversion rate on home page above 15% Plan: Use optimizely to change VP on home page for 2000 visits Time box: days Result: 16%, Win! Click Me + = • How can we solve this problem? • What form should this take? • How important is the design? • What’s the quickest hack? • What is the minimum feature set? • How should we prioritize? • Solution interview • Contextual inquiry / ethnography • Demo pitch • Concierge test / Consulting • Competitor Usability • Picnic in the Graveyard Our First Version (The Concierge Model) + = • Is this solution working? • Are people using it? • Which solution is better? • How should we optimize this? • What people like / dislike? • Why they that? • Paper prototypes • Clickable prototypes • Usability • Hallway • Live • Remote • Wizard of Oz • Takeaway • Functioning products • Analytics / Dashboards * • Surveys (NPS) References & Contributors ● ● ● GrasshopperHerder Innovation Partnership Programme Dan Toma, Kenny Nguyen, Nick Norena, Ryan MacCarrigan Tristan Kromer ... find them? How can we solve this problem? What form should this take? How important is the design? What’s the quickest hack? What’s the minimum feature set? How should we prioritize? Test Hypotheses... is there drop off? What they think? OPEN-ENDED QUESTIONS Solution Product Problem Market • • • • • • • • Who is our customer? What are their pains? What job needs to be done? How we find them?... box: days Result: 16 %, Win! Experiment Templates* Learning Goal Hypothesis Metric Fail Condition Plan Time Box Results Next Step wtf,s! Experiment Templates* Learning Goal Hypothesis Metric Fail