By Steve Safran
Ze Frank calls it the Nerd Fight. He is disputing Rocketboomâ€™s assertions that they have 350,000 viewers a day. At the center of this dispute is what we decide a â€œviewâ€ really is. Actually, we can all agree on what a â€œviewâ€ is â€” someone watched something. Itâ€™s how you measure it thatâ€™s up in the air. One thingâ€™s for sure: just because you download a podcast doesnâ€™t mean youâ€™ve watched it. And just because youâ€™ve started watching it doesnâ€™t mean youâ€™re going to finish it. Thereâ€™s simply no way for us to measure viewership of podcasts. But we keep reporting numbers from the networks, big sites and podcasters without questioning them (guilty as charged) and we need to step back for a moment and ask: â€œHow do we figure out who is really watching or listening to our podcasts?â€ Then we have to admit â€œWe donâ€™t know.â€
What does a download number mean?
Last week, ABC News announced it had reached five million podcast downloads in a month. An impressive number. And thereâ€™s no reason to doubt it. But what does that translate to in terms of viewership? And by extension, how do you sell that to an advertiser?
I donâ€™t know.
If youâ€™re a podcast subscriber, you know the process: you subscribe to a bunch of shows, they download to your iPod (or whatever) and thenâ€¦ what? Maybe you watch. Maybe you donâ€™t. Maybe you start it but thatâ€™s it. Come on â€” my iPod is jammed with the things. I get to a fraction of the ones actually on my iPod. Itâ€™s a bit like the TiNo syndrome: you gather all those episodes of “NOVA” and “Frontline,” you just donâ€™t watch them.
Look at it another way: having the podcasts on your device is a victory for the producers, no doubt. You canâ€™t watch the show if you didnâ€™t download it. But all that means is that the show is available to you. Itâ€™s really no different than having a cable channel â€œmake itâ€ to your cable box. You have access to it â€” but you donâ€™t always watch PBS. (Admit it.)
All the â€œdownloadsâ€ metric does is outline your potential universe. It does not tell the advertiser the most important metric: who frickinâ€™ watched us?
I donâ€™t know.
What I do know is that paid downloads are another matter. It doesnâ€™t mean a thing to ABC whether you watched that download of â€œLost,â€ â€” as long as you paid for the download. So if there were 35,000 downloads of a show at $1.99 apiece, thatâ€™s real money with a real number. In that case viewership doesnâ€™t matter any more than the fact that you bought the extended DVD of â€œLawrence of Arabiaâ€ but you and I both know youâ€™re never going to watch it. At least take it out of the plastic wrapper, will you?
Over at Zeâ€™s Nerd Fight, (complete, as any war coverage should be, with a theme song) he fires a salvo at Andrew Barron, Rocketboomâ€™s producer: â€œAndrew’s caught in the web of these fuzzy numbers. To him, “complete download” not only means a “completely watched show” but also a “unique audience member.” He also touts his large subscriber base and brags how a number of software products download his show by default after they’re installed — without even asking the user if he’s interested or wants that to happen.
â€œThose aren’t viewers, they’re nickels. Grab a thousand; swipe $50 from an advertiser.â€
Still, Ze allows that everyone engages in the practice of reporting the download stat and not an actual view stat: â€œMSN, YouTube and other sites are also Rocketbooming. They’re all desperate to come across as popular, so they all present aggressive background statistics. On some gallery sites just browsing a page full of thumbnails increases the view count for all the items on that page. Other times you click a blurry video, immediately leave, but it still counts as a view.â€
Barron fires back with his own post, Ze Errors, in which he points out that TechCrunch audited his numbers successfully. He also implies a certain jealously on Ze’s part – Ze gets 30,000 downloads to Rocketboom’s 300,000:
â€œThe core of the matter is not hits or page views or uniques or subscribers or Alexa #’s it’s how many completed videos were served. The video carries the ad. So that is what the advertiser wants to know, that is the ultimate value number and that is what Mr. Frank stated he was most concerned about: ad revenue potential. For any website that uses Apache and keeps a log file (i.e. the majority of the websites on the internet), this is a no brainer to see. Again, its simply a video file and files are easily counted.â€
But notice that Barronâ€™s not really disputing Zeâ€™s point: we just donâ€™t know if people are watching.
A new metric: â€œEngagementâ€
What metric could we possibly figure out that would let us know if the podcast we sent out was actually effective? Web legend and current VP of Media Development, at the PodTech Network, Robert Scoble, has an interesting suggestion: a metric he calls â€œengagement.â€ Itâ€™s less definable than the actual number of downloads, but far more interesting.
At Streaming Media West 2006 this past week, Scoble gave an interesting anecdote about a form of engagement. (Iâ€™m going to quote my own transcription from the panel session now, which was more of a â€œquotes-meets-my-summaryâ€ liveblog.)
Scoble: My friend Buzz at ActiveWords (released new software), had a five star review in USA Today and had all of 32 downloadsâ€¦ he then went on a 50 minute podcast and late in it he mentioned heâ€™d give a free download to people who were still listening. He had 450 downloads as a result of that mention, so that gave him a metric.
Itâ€™s an interesting idea and proof that traditional PR efforts like getting your client in a major newspaper are less valuable than targeting niche PR. Itâ€™s really not all that different from the notion of â€œdirect responseâ€ advertising. You can measure effectiveness, at least in this example. And you can then go to an advertiser after you have a proven track record of that effectiveness.
Always question that PR number
So maybe we can know, after all, how effective our podcasts are. The networks should look to this example and try to steer clear of the Nerd Fights. Most important, as journalists, we should always question numbers handed to us on a silver email. There’s a reason PR offices exist. They are simply a pipeline of the good news that companies want you to know. Period. I can tell you I’ve never been able to get what I suspect to be a bad number out of them. (And you know who you are and to what service I refer.) We need to keep trying to be creative with new metrics, and not simply parrot the old ones.