An interview with Andrew Ng

The Huff­in­g­ton Post series, Sophia, brings ‘life lessons from fas­ci­nat­ing peo­ple’. Their lat­est inter­view is with Andrew Ng, a Stan­ford Uni­ver­si­ty pro­fes­sor, co-founder of Cours­era, and key mem­ber of the deep learn­ing teams at first Google and now Baidu. I real­ly like some of the insights in his inter­view, the prac­ti­cal­i­ty with which he treats inno­va­tion and the easy way he explains hard con­cepts.

For exam­ple, here he talks about career advice:

I think that “fol­low your pas­sion” is not good career advice. It’s actu­al­ly one of the most ter­ri­ble pieces of career advice we give peo­ple. Often, you first become good at some­thing, and then you become pas­sion­ate about it. And I think most peo­ple can become good at almost any­thing.

On retrain­ing the work­force for a more heav­i­ly auto­mat­ed future:

The chal­lenge that faces us is to find a way to scal­ably teach peo­ple to do non-rou­tine non-repet­i­tive work. Our edu­ca­tion sys­tem, his­tor­i­cal­ly, has not been good at doing that at scale. The top uni­ver­si­ties are good at doing that for a rel­a­tive­ly mod­est frac­tion of the pop­u­la­tion. But a lot of our pop­u­la­tion ends up doing work that is impor­tant but also rou­tine and repet­i­tive. That’s a chal­lenge that faces our edu­ca­tion­al sys­tem.

On why machine learn­ing is sud­den­ly more pop­u­lar, despite the tech­nol­o­gy being around for decades in some cas­es:

I often make an anal­o­gy to build­ing a rock­et ship. A rock­et ship is a giant engine togeth­er with a ton of fuel. Both need to be real­ly big. If you have a lot of fuel and a tiny engine, you won’t get off the ground. If you have a huge engine and a tiny amount of fuel, you can lift up, but you prob­a­bly won’t make it to orbit. So you need a big engine and a lot of fuel. We final­ly have the tools to build the big rock­et engine – that is giant com­put­ers, that’s our rock­et engine. And the fuel is the data. We final­ly are get­ting the data that we need.

And the chal­lenges of talk­ing to com­put­ers:

Most peo­ple don’t under­stand the dif­fer­ence between 95 and 99 per­cent accu­rate. Nine­ty-five per­cent means you get one-in-20 words wrong. That’s just annoy­ing, it’s painful to go back and cor­rect it on your cell phone. Nine­ty-nine per­cent is game chang­ing. If there’s 99 per­cent, it becomes reli­able. It just works and you use it all the time. So this is not just a four per­cent incre­men­tal improve­ment, this is the dif­fer­ence between peo­ple rarely using it and peo­ple using it all the time.

It’s a real­ly inter­est­ing inter­view, I encour­age you to read it in full.


On filters and feelings

An inter­est­ing arti­cle from Wired on a recent study into how peo­ple use fil­ters on their dig­i­tal pho­tographs.

Seri­ous hob­by­ists” use fil­ters only to cor­rect a problem—say, cor­rect the expo­sure. “More casu­al pho­tog­ra­phers” are more like­ly to manip­u­late their images with fil­ters or adjust­ments that make them appear more “arti­fi­cial,” accord­ing to the study.

I used to be a bit of a fil­ter snob, but now I tend to think that, cer­tain­ly on Insta­gram, using a fil­ter is less about try­ing to cap­ture a moment than it is about cap­tur­ing the way that moment felt. It’s like, when you take a pic­ture on a hot day but that hot­ness doesn’t come across in the pho­to, Sier­ra or Valen­cia are tools to help com­mu­ni­cate that feel­ing to every­one else who views it.


Sarah Lynham, 1979–2015.

Book dedication: “For Sarah, my sister. Your courage is inspiring.”

Just before 9am on 1st May 2015, my sis­ter, Sarah Lyn­ham, died.

In 2013 Sarah was diag­nosed with can­cer of the cervix. In ear­ly 2014, after long chemother­a­py, she was pro­nounced all clear — with the pro­vi­so that the can­cer could return. In Octo­ber the same year, a few days before her 35th birth­day, she received the news that the can­cer had returned, had spread into her lungs, and was treat­able but not cur­able. She had fur­ther chemother­a­py to try to arrest the spread and give her a lit­tle more time. In April 2015 the can­cer had spread to her brain and she fell uncon­scious and was tak­en to hos­pi­tal. After a few days she was moved to a hos­pice where she was kept sedat­ed to avoid pain. And there she spent her final days.

I now under­stand the dif­fer­ence between grief and mourn­ing. I griev­ed when I heard the news of Sarah’s can­cer, again when it returned and the fore­cast was ter­mi­nal, again when she went into hos­pi­tal, and once more when she died; grief occurs in phas­es. You mourn only once, when some­one is gone. Grief is sharp, it rolls over you in waves, it hurts, it racks your body with sobs. Mourn­ing is blunt, flat, an absence.

Grief is unfair on those who can’t process it. Sarah has three chil­dren, who each dealt with death as their under­stand­ing of it allowed. On find­ing out that Sarah had gone to hos­pi­tal the youngest child (aged sev­en) screamed that she couldn’t live with­out her mum­my. But after that, through the weeks of decline in the hos­pice and the death, she bare­ly revealed any sad­ness at all. Grief will find her in a dif­fer­ent way, and it won’t be nice to be around.

For the first few days of her hos­pi­tal stay, Sarah would wake up occa­sion­al­ly and you could exchange a few sen­tences with her. Some­times what she said would make lit­tle sense, which was large­ly because she was on a high dose of mor­phine and hav­ing rich dreams. One time I asked her if she was okay, and she said yes, she was just redis­cov­er­ing the island. But as the tumours in her brain became more aggres­sive, her hap­py hal­lu­ci­na­tions turned into non­sense, before she stopped speak­ing entire­ly.

She died in Spring, with skies that were blue and large­ly free of clouds. This weath­er felt inde­cent. The trees were in rich blos­som and the fields around the hos­pi­tal in Dorset full of new-born lambs. A rude and most­ly wel­come reminder that life goes on.

Because her dying was extend­ed over weeks, and because life goes on, the mun­dane rou­tines of the day-to-day had to hap­pen. You still have to take the bins out when your sis­ter is dying, you still have to make phone calls to British Gas. I took my nephews and nieces to Nando’s for lunch, but felt too ashamed to check in on Swarm in case peo­ple thought I was enjoy­ing myself and not being suf­fi­cient­ly solemn. I was enjoy­ing myself, I was with my fam­i­ly, but felt I had to self-cen­sor to give the appro­pri­ate appear­ance. Also, there are a lot of hours involved in wait­ing for some­one to die; you have to pass the time between the peaks of grief with banal and triv­ial activ­i­ties: read­ing, social media, tele­vi­sion.

Ques­tions of pro­pri­ety… when I heard she was dying and went to vis­it I need­ed to pack a case, but found myself com­plete­ly paral­ysed by inde­ci­sion: what clothes do you wear to your sister’s death? And what is appro­pri­ate behav­iour at the side of the death bed? One time all my imme­di­ate fam­i­ly were around the bed and some­one made a rude joke, and we all fell around laugh­ing. I don’t know if I’m meant to feel guilty about that.

I’m not angry about any per­ceived unfair­ness at her death; I know that life seems cru­el because it’s impar­tial. But I get angry at the mys­ti­cism around death. I know peo­ple have dif­fer­ent beliefs to me, and if it’s com­fort­ing for them to believe that Sarah’s gone to a bet­ter place, then I wel­come the sen­ti­ment. But it hurts me to hear peo­ple say that Sarah has become an angel, she was flown to heav­en on the wings of a dove, she’s smil­ing down on us… It’s not true. She’s dead, and she’s gone, and she’s nev­er com­ing back. Per­haps that’s unrea­son­able, I don’t want to tell peo­ple how to deal with loss; but for me, the end of her con­sid­er­able pain should be suf­fi­cient solace to take from her death.

I dread hav­ing to talk to peo­ple about her death. I don’t know what to say. There is noth­ing nice or orig­i­nal to be said about death. On the night she was admit­ted to hos­pi­tal I took a taxi home, and the dri­ver tried to make con­ver­sa­tion. Was I vis­it­ing some­one? Yes, I replied, my sis­ter. Well I hope she gets bet­ter soon, he said. How could I respond to that? I didn’t want to make him feel bad for ask­ing a ques­tion with a hor­ri­ble answer. All I could say was, unfor­tu­nate­ly not, then we sat in silence for ten min­utes.

This post doesn’t have a con­clu­sion, it was about me try­ing to make sense of the past few weeks, putting my thoughts into some kind of order. And real­ly I just want­ed to write that I loved her, and I’ll miss her.

To the world you may be one per­son, but to one per­son you may be the world.

Post­script: Sarah’s end was made eas­i­er for every­one by the gen­tle and humane staff of the Weld­mar Hos­pice­care Trust. They made Sarah’s last days more com­fort­able, and helped her fam­i­ly — espe­cial­ly the chil­dren — come to terms with the loss. I’ve made a pri­vate dona­tion to their trust, but want­ed to do more to help oth­er fam­i­lies deal­ing with can­cer.

Because of this, I took part in the Nightrid­er 2015 event, a 100km overnight cycle around Lon­don, and I chose to raise funds for Marie Curie. Although the event is over now, dona­tions are still open. If you could spare a few pounds,  you can donate through my Just Giv­ing page, or if you’re in the UK you can text your dona­tion to 70070 using the code PGNR72. For exam­ple, to donate £5, you would send: PGNR72 £5

Im very grate­ful for every con­tri­bu­tion, which all go to help thou­sands of fam­i­lies going through the pain of los­ing some­one they love.


Blogging the Highlights: Alex Through the Looking Glass

Anoth­er in the occa­sion­al series of blog posts about the high­lights I make in books I read. This time it’s Alex Bel­los’ Alex Through The Look­ing Glass (called The Grapes of Math in the US), a look at the hid­den pat­terns in math­e­mat­ics, and math­e­mat­i­cal pat­terns in life. There’s some quite com­plex maths in it and I found it quite tough going at times, but there are also many fas­ci­nat­ing facts and sto­ries.

Anoth­er clever menu strat­e­gy is to show the prices imme­di­ate­ly after the descrip­tion of each dish, rather than list­ing them in a col­umn, since list­ing prices facil­i­tates price com­par­i­son.

I find these lit­tle psy­cho­log­i­cal manip­u­la­tions end­less­ly inter­est­ing. It’s about the con­cept of bound­ed ratio­nal­i­ty, how we think we make ratio­nal deci­sions but are in fact con­stant­ly manip­u­lat­ed.

Ben­ford argued that the phe­nom­e­non must be evi­dence of a uni­ver­sal law, which  he called the Law of Anom­alous Num­bers. The coinage didn’t catch on. His name, how­ev­er, did. The phe­nom­e­non is known as Benford’s law.

Benford’s Law is a law of fre­quen­cy of dig­its in many data sets, and has been used reli­ably to detect fal­si­fied data in account­ing, sci­ence, eco­nom­ics and more. It’s quite fas­ci­nat­ing.

In oth­er words, well-con­nect­ed nodes become even bet­ter con­nect­ed. The rich get rich­er. The famous get more famous. The node with the most links has the high­est chance of get­ting new links, and the more links it gets the more attrac­tive it becomes.

Net­works — whether that’s hyper­linked pages on the Web or fol­low­ers on Twit­ter — tend to fol­low pow­er laws and grow in very pre­dictable ways.

More recent­ly it has been argued that 360 was cho­sen because six equi­lat­er­al tri­an­gles fit snug­ly with­in a cir­cle, as shown below, and that each of these angles was divid­ed into 60  as demand­ed by sex­a­ges­i­mal frac­tions.

Why do cir­cles have 360 degrees? It could be because the Baby­lo­ni­ans count­ed in six­ties rather than tens. Ancient ori­gins of every­day con­cepts fas­ci­nate me.

In the sec­ond cen­tu­ry BCE the Greeks appro­pri­at­ed Baby­lon­ian frac­tions, which have been in use ever since. The degree was tra­di­tion­al­ly divid­ed into six­ty small­er units,  each a pars min­u­ta pri­ma, or first minute part, which were then divid­ed into six­ty small­er units, each a pars min­u­ta secun­da, or sec­ond minute part. From the trans­la­tion of these  Latin phras­es we get the words minute and sec­ond, our units of time, which are the most promi­nent mod­ern relics of the ancient prac­tice of count­ing in groups of six­ty.


[Tycho] Bra­he was a flam­boy­ant aris­to­crat. He wore a pros­thet­ic gold and sil­ver nose, after a cousin sliced the orig­i­nal one off in a duel about a math­e­mat­i­cal for­mu­la.

Actu­al­ly they exhumed the astronomer in 2010 and found out that his nose was prob­a­bly brass.

… Christo­pher Wren, a young Eng­lish astron­o­my pro­fes­sor…

I had no idea Wren was an astronomer before an archi­tect. Seems to have been a very com­mon pro­fes­sion.

To keep his posi­tion as pro­fes­sor of math­e­mat­ics at the Col­lège de France, the  country’s most pres­ti­gious seat of learn­ing, [Gilles Per­son­ne de Rober­val] had to pro­vide the best answer to a prob­lem announced pub­licly every three years.

He set the prob­lems him­self, but still this is a great test of suit­abil­i­ty for a role.

[John] Whit­ney could adjust the speed and size of the sinu­soids elec­tron­i­cal­ly, giv­ing him much more con­trol and elim­i­nat­ing the effects of damp­ing. The pat­terns he pro­duced were daz­zling and  became some of the most icon­ic images in the his­to­ry of math­e­mat­i­cal art. They were famous­ly used in the title sequence and posters for Alfred Hitchcock’s 1958 movie Ver­ti­go.

The first com­put­er-gen­er­at­ed art used in a Hol­ly­wood fea­ture film, in 1958.

It was not long, how­ev­er, before engi­neers were using cate­nar­ies. Before the com­put­er age the quick­est way to make one was to hang a chain, trace out the curve, build a mod­el  using a rigid mate­r­i­al and stand it upside down.

Cate­nar­ies are a curve where the ten­sion is so per­fect­ly dis­trib­uted that it makes an arch which needs no brace or but­tress­es. Gaudí used them exten­sive­ly in design­ing the Colò­nia Güell.

Under­ly­ing the whim­sy, how­ev­er, is a whole field of incred­i­bly use­ful the­o­ry, called ‘opti­mal stop­ping’, or the maths of when is the best time  to stop.

There real­ly is sci­en­tif­ic the­o­ry around the best time to take a par­tic­u­lar action.

Since [1976] about 200 loop­ing roller-coast­ers have been built around the world, all of them using Stengel’s prin­ci­ple.

Wern­er Sten­gel invent­ed the first loop­ing roller-coast­er, when he used a clothoid instead of a cir­cle for the loops.

The term for a word that only appears once in a text is hapax legomenon.

There’s a word for every­thing (and a Wikipedia page).