Small Numbers, Huge Changes

In a recent inter­view, Sun­dar Pichai of Google dis­cuss­es improve­ments in the accu­ra­cy of their voice recog­ni­tion:

Just in the last three years, we have tak­en things like error in word recog­ni­tion down from about 23 per­cent to 8 per­cent.

That’s the dif­fer­ence between mis­un­der­stand­ing one word in four, to one word in twelve; the dif­fer­ence between com­plete­ly unus­able, and annoy­ing.

Andew Ng, for­mer­ly of Google and now of Baidu, expands on this:

Most peo­ple don’t under­stand the dif­fer­ence between 95 and 99 per­cent accu­rate. Nine­ty-five per­cent means you get one-in-20 words wrong. That’s just annoy­ing, it’s painful to go back and cor­rect it on your cell phone.

Nine­ty-nine per­cent is game chang­ing. If there’s 99 per­cent, it becomes reli­able. It just works and you use it all the time. So this is not just a four per­cent incre­men­tal improve­ment, this is the dif­fer­ence between peo­ple rarely using it and peo­ple using it all the time.

It’s fas­ci­nat­ing to see how these small num­bers make a huge dif­fer­ence; you might think Google’s 92% accu­rate is only a lit­tle less than Baidu’s 95% accu­rate, but in prac­ti­cal terms there’s a big gulf. And it gives me pause to think about the mon­ey, human resource and com­put­ing pow­er spent on try­ing to make those small huge increas­es in accu­ra­cy.