Metrics driven development – Proposed workflow and implementation – Measuring

View Github Repository Open presentation in a new window

pacog

See all presentation from pacog

Metrics driven development – Proposed workflow and implementation – Measuring

0 0

metrics-presentation

Presentation using reveal-js talking about general workflow of metrics tracking, and suggested implementation

On Github pacog / metrics-presentation

Metrics driven development

Proposed workflow and implementation

paco@hyves.nl

Why?

  • We are (supposed to be) lean
  • Validated and shared knowledge
  • Reality beats intuition

General idea

Measure everything the user does Decide the important thing to improve Propose improvement See if it works Share the knowledge

Measuring

Track every action the user does

How? We'll talk about implementation later

Using kissmetrics, GA, you can visualize them

Also we can check browsers, visits per day, etc

Fine grain

Details are important

If you don't add them now, you lose them

Focusing

Decide what is the important thing to improve

Create statistics that can show how well are we doing

Viewmeister(c) example

Most important thing was making it viral

In order to go viral this had to happen:

Watch video Like (share) video Friends see scrap They watch video They share it

We decided to optimize the conversion:

Watch video -> Like video

Watch out for collateral damage!

People like the video after watching it, 10% more But... We have 50% less people watching because UI is annoying them

Optimizing

How do we improve this conversion?

AB tests to gather useful stats

Formulate some hypothesis

Making the like button prettier will make people click it more Showing that button in an overlay will annoy people, they will leave

Choose the ones to test based on intuiton and previous (validated) learnings

Disrupt once in a while

Viewmeister examples:

  • Like buttons
  • Facebook spillover

Like button optimization

Option A: inline buttons

Option B: overlay after play

Result:

Option A: 14% conversion (watch->play)
Option B: 18% conversion (watch->play)
And it didn't change the other key metrics
(didn't annoy users)

Metrics to choose the best option

Kissmetrics allows to create:

  • Total number events
  • Average per user events
  • Conversion rates
  • Time between events
  • Funnel reports
  • Cohort reports

So, choose the solution that makes your metrics improve

Share the knowledge!

Once the experiment is complete, please share your findings

If they are important or shocking enough, please send them around or host a tech lunch

AB tests

Client side vs server side (setProperty)

Proposed implementation

We are doing different code in every project

The one I was reusing doesn't work well

Some things the implementation should provide:

A facade to have only one interface for many metrics services:

        identify
        trackEvent
        addBrowserStats
        ab
        logDemographic
        setProperty

Independent of js FWK

Easy to disable in staging and dev

Independent of the actions

    <a data-metric="play" class="action-play button">Play</a>

Testable, how?

Maybe sinonjs

Slides: pacog.github.com/metrics-presentation