Rating, Reputation, Conformance and Compliance

Chi­na has made obe­di­ence to the State a game. A new tool called Sesame Cred­it gives users a rat­ing based on their com­pli­ance to ‘good behav­iour’, using data from social net­works and con­nect­ed ser­vices:

If you post pic­tures of Tianan­men Square or share a link about the recent stock mar­ket col­lapse, your Sesame Cred­it goes down. Share a link from the state-spon­sored news agency about how good the econ­o­my is doing and your score goes up.

Com­menters in the West pre­dictably see this as an exam­ple of oppres­sive gov­ern­ment, seem­ing­ly with­out the self-aware­ness to know that we are already prac­tis­ing obedience—the dif­fer­ence being that we con­form in order to fit pri­vate ser­vices, not the state.

From eBay to Uber to AirBNB, rat­ing sys­tems are used to cre­ate com­pli­ance to be a ‘good fit’ to the ser­vice. As a user, if your score isn’t high enough, you’re more like­ly to be turned down for a rental or a car jour­ney. As Matt Webb tweet­ed:

The blessed relief of rid­ing in a black cab and know­ing you don’t need to work for that 5* rat­ing from the dri­ver.

But it’s worse for an Uber dri­ver, where if you don’t keep a suf­fi­cient­ly high rat­ing you can lose your job entire­ly. Josh Dzieza’s arti­cle, The Rat­ing Game, describes the prob­lems this cre­ates:

When Judge Edward Chen denied Uber’s motion for sum­ma­ry judge­ment on the Cal­i­for­nia dri­vers’ class action suit, he seized on the idea that rat­ings aren’t just a cus­tomer feed­back tool — they rep­re­sent a new lev­el of mon­i­tor­ing, far more per­va­sive than any watch­ful boss. Cus­tomer rat­ings, Chen wrote, give Uber an “arguably tremen­dous amount of con­trol over the ‘man­ner and means’ of its dri­vers’ per­for­mance.”

This has a huge chill­ing effect: as well as the poten­tial loss of ser­vice or liveli­hood, com­pli­ance forces con­for­mi­ty of per­son­al­i­ty. You have to keep your head down to get on:

Sev­er­al dri­vers said the best way to behave is like a ser­vant. “The ser­vant antic­i­pates needs, does them effort­less­ly, speaks when spo­ken to, and you don’t even notice they’re there,” said a dri­ver in Sacra­men­to.

In the rep­u­ta­tion econ­o­my, the best way to get by is to be bland­ly pleas­ant. As Bret Eas­t­on Ellis put it in Liv­ing in the Cult of Lik­a­bil­i­ty:

The rep­u­ta­tion econ­o­my depends on every­one main­tain­ing a rev­er­en­tial­ly con­ser­v­a­tive, immi­nent­ly prac­ti­cal atti­tude: Keep your mouth shut and your skirt long, be mod­est and don’t have an opin­ion.

In Chi­na, keep­ing a good Sesame Cred­it score will give real world advan­tages — and con­se­quences:

High scores will grant users ben­e­fits “like mak­ing it eas­i­er to get the paper­work you need to trav­el or mak­ing it eas­i­er to get a loan”. There have even been rumours about imple­ment­ing penal­ties for low scores “like slow­er inter­net speeds, or restrict­ing jobs a low scor­ing per­son is allowed to hold.”

At the height of the pop­u­lar­i­ty of the rep­u­ta­tion rank­ing tool Klout, some employ­ers stat­ed a min­i­mum Klout score as a require­ment for apply­ing for a role with their com­pa­ny. That didn’t last, thank­ful­ly, but the Chi­na state are show­ing that it’s an idea that may well make a come­back.

If we don’t imme­di­ate­ly start ques­tion­ing the desir­abil­i­ty of the rep­u­ta­tion econ­o­my, we could end up with a very neg­a­tive sce­nario: at best, a cul­ture of bland con­for­mi­ty; and at worst a sys­tem of mass com­pli­ance.

Robots and Homeopaths

How robots mess with our minds is a great arti­cle on emo­tion­al respons­es to robots by artist and engi­neer, Alexan­der Reben. In it he explores how peo­ple are able to speak inti­mate­ly to machines, and the impli­ca­tions of that as robots gain ‘per­son­al­i­ty’. I was par­tic­u­lar­ly inter­est­ed in a quote from the philoso­pher John Camp­bell:

One of the pos­si­bil­i­ties this opens up is automat­ing aspects of our emo­tion­al lives where we usu­al­ly depend on oth­er peo­ple for sym­pa­thy and sup­port. Rather than rely­ing on your part­ner to lis­ten to the prob­lems you’ve had at work all day, why not explain them to a sym­pa­thet­ic robot that makes eye con­tact with you, lis­tens with appar­ent inter­est, mak­ing all the right nois­es, remem­bers and cross-index­es every­thing you say?

This made me think of home­opa­thy. Although there’s no evi­dence at all that the treat­ment itself is effec­tive, many peo­ple who use it claim that after treat­ment they do feel bet­ter, or even ful­ly cured. One expla­na­tion for this is that a con­sul­ta­tion with a prac­ti­tion­er can have a pos­i­tive effect on well-being; just hav­ing some­one lis­ten com­pas­sion­ate­ly can make you feel bet­ter.

So in the future we could be pre­scribed a ses­sion with a sym­pa­thet­ic robot to make us feel bet­ter.