Tuesday, December 17, 2013

When audiences lie to you

Summary of 'The Mom Test' (v2 2013-11-05) from Max Völkel

I've been filling in audience panel forms for the last couple of decades, often keeping track of the questions being posed. I really wonder what conclusions were ever drawn from them. Problem is that they are often templates and put together by researchers who are not regular listeners/viewers of the programme. I'm interested because I have always valued audience input - I believe it's essential to audience engagement. And the factual programmes that had the maximum involvement of the audience were the ones they remembered, often decades later.

In my more recent work with start-ups, I've been using the Lean Startup Methodology which encourages you to build a business model canvas and then test it on lead customers. Few business models survive that first encounter. If you've been involved in leading film, radio or TV production you already know about making a pilot production and testing it on a representative group before commissioning. So the method is nothing new. But I think many of us in the media could learn from books like the Mom Test in formulating much more useful questions in audience surveys. Just replace the word customer with listener/viewer/site user and you'll see what I am ranting on about.

The answers could be so much more use to programme makers in understanding why certain programmes or sites are more appreciated than others. If you look at slide 9 above, it shows you what someone is thinking during the test.

Found a great presentation by Rob Fitzpatrick from the Pioneers festival.

and in Berlin

Rob Fitzpatrick - How To Do (And What To Expect From) Early-Stage Customer Development & Sales from HackFwd on Vimeo.

I've done something similar with a recent survey that came in from an international broadcaster. It was clear that they simply wanted me to confirm a producer's hunch that they were making a better programme now than 6 months ago. They weren't interested in alternatives. They were making programmes for me, not with me. There was no open question at the end where I could share my thoughts. And they said my answers would be anonymized to protect my identity. In other words, if I did have something to say, the producer would never find out who suggested it. So, because I knew I had no influence on the future, I stopped wasting my time and got on with something else. In fact that was designing a much more useful questionnaire for a client. Learned a lot from others mistakes. Anyone have a similar experience? 

No comments: