Tweet
Share
Share
Pin
0 Shares

The World Knowl­edge Forum was launched in Octo­ber 2000 after two years of prepa­ra­tion fol­low­ing the Asian Finan­cial Cri­sis of 1997, with the goal of fos­ter­ing a cre­ative trans­for­ma­tion into a knowl­edge-based nation. Over the years, the forum has pro­vid­ed a plat­form for dis­cus­sions on nar­row­ing the knowl­edge gap, as well as achiev­ing bal­anced glob­al eco­nom­ic growth and pros­per­i­ty through knowl­edge sharing.

 

AI is evolv­ing rapid­ly, allow­ing us to reap its ben­e­fits across many areas of life and work. How­ev­er, the risks posed by AI are unde­ni­ably grow­ing. To ensure that AI does not destroy the order that humans have cre­at­ed over the years or pose threats to our exis­tence, there must be ethics and clear prin­ci­ples for AI research and devel­op­ment. This ses­sion will fea­ture aca­d­e­mics who have been work­ing on the fun­da­men­tal prin­ci­ples of AI devel­op­ment, ethics of AI devel­op­ment, and copy­right pro­tec­tion, as well as entre­pre­neurs who have encoun­tered chal­lenges apply­ing AI in the real world. They will dis­cuss how to set fun­da­men­tal guide­lines for AI devel­op­ment, how to stop AI from infring­ing on copy­right and pri­va­cy, how to weed out false infor­ma­tion, and how to defend human rights.

00:00 What if we succeed in that goal? 05:17 The era of deep learning 10:03 Some serious failures 11:43 AlphaGo and AGI 18:02 Human extinction 21:17 Preferences 29:29 Coexistence

 

TRANSCRIPTION

Human extinction

But the per­haps more seri­ous down­side uh is human extinc­tion and this is why I say it’s not real­ly an eth­i­cal issue I I think by and large few peo­ple would argue that human extinc­tion is uh eth­i­cal­ly prefer­able uh there are some uh but I’m just going to ignore those peo­ple um so it’s just com­mon sense right if you cre­ate some­thing that’s more pow­er­ful than human beings how on Earth are we going to have pow­er over such sys­tems for­ev­er so in my view there’s only two choic­es we either build prov­ably safe and con­trol­lable AI where we have absolute cast iron math­e­mat­i­cal guar­an­tee of safe­ty or we have no AI at all so those are the two choic­es right now we’re pur­su­ing the third choice which is com­plete­ly unsafe black­box AI that we don’t under­stand at all and we are try­ing to make it into some­thing that’s more pow­er­ful than us which is pret­ty much the same sit­u­a­tion we would be in if uh a super­hu­man AI sys­tem land­ed from out­er space uh sent by some alien species no doubt for our own good uh our chances of con­trol­ling an alien super­hu­man intel­li­gence would be zero and that’s sit­u­a­tion that we’re head­ing towards and Alan chur­ing the founder of com­put­er sci­ence uh you know thought about this because he was work­ing on AI and he thought about what hap­pens if we suc­ceed and he said we should have to expect the machines to take con­trol so what do we do I think it’s real­ly hard espe­cial­ly giv­en that 15 quadrillion dol­lar prize that uh com­pa­nies are aim­ing for and the fact that they have already accu­mu­lat­ed 15 tril­lion dol­lar worth of cap­i­tal to aim at that goal with it’s kind of hard to stop that process so we have to come up with a way of think­ing about AI that does allow us to con­trol it that is prov­ably safe and prov­ably con­trol­lable and so rather than say­ing how do we retain pow­er over AI sys­tems for­ev­er which sounds pret­ty hope­less we say what is a math­e­mat­i­cal frame­work for AI a a way of defin­ing the AI prob­lem so that no mat­ter how well the AI sys­tem solves it we are guar­an­teed to be hap­py with the result so can we devise a math­e­mat­i­cal prob­lem a way of say­ing what is the AI Sys­tem sup­posed to be doing that has that prop­er­ty that we’re guar­an­teed to be hap­py with the result.

Preferences

So I spent about 10 years work­ing on this and um to explain uh how we approach­ing it um I’m we going to intro­duce a a tech­ni­cal term that uh I think will be help­ful for our dis­cus­sion about ethics as well um and that’s a notion called pref­er­ences so pref­er­ences does­n’t sound like a tech­ni­cal term right Some peo­ple pre­fer pineap­ple piz­za to Mar­gari­ta Piz­za but what we mean by pref­er­ences in the in the the­o­ry of deci­sion- mak­ing is actu­al­ly some­thing much more all-encom­pass­ing and it’s your rank­ing over pos­si­ble futures of the uni­verse so to kind of reduce that to some­thing we can grasp eas­i­ly imag­ine that I made you two movies of the rest of your life and the rest of the you know the future of oth­er things you care about you know and the movies are about two hours long and you can kind of Watch movie A and movie b and then you say yeah I’d like movie A please don’t like movie B at all because um I get minced up and and turned into ham­burg­er in movie b and I don’t like that very much so I’d pre­fer movie A please so that’s what we mean by pref­er­ences except that this would­n’t be a two-hour movie it’s real­ly the entire future of the Uni­verse um and of course we don’t get to choose between movies because in fact uh we can’t pre­dict what exact­ly which movie is going to hap­pen and so uh we’re actu­al­ly uh hav­ing to deal with the uncer­tain­ty we call these lot­ter­ies over pos­si­ble futures of the uni­verse so a pref­er­ence struc­ture is then a uh basi­cal­ly a rank­ing over futures of the uni­verse tak­ing uncer­tain­ty into account to make a sys­tem that is prov­ably ben­e­fi­cial to humans you just need two sim­ple prin­ci­ples one is that the only objec­tive of the machine is to fur­ther human pref­er­ences to fur­ther human inter­ests if you like uh and the sec­ond prin­ci­ple is that the machine knows that it does not know what those pref­er­ences are and that’s kind of obvi­ous right because we don’t real­ly know what our pref­er­ences are and uh we cer­tain­ly can’t write them down in enough detail to get it right um and when but when you think about it right a machine that that solves that prob­lem the bet­ter it solves it the bet­ter off we are and in fact you can show that it’s in our inter­est to have machines that solve that prob­lem because we are going to be bet­ter off with those machines uh than with­out them so that’s good but as soon as I describe that way of think­ing to you that machines are going to fur­ther human pref­er­en and um and learn about them as they go along this now brings in some eth­i­cal ques­tions final­ly right so we final­ly get to ethics what I want to avoid uh so I’m just going to tell you not to ask this ques­tion do not ask the ques­tion well whose val­ue sys­tem are you going to put into the machine right because I’m not propos­ing to put any­one par­tic­u­lar val­ue sys­tem into the machine in fact the machine should have at least 8 bil­lion pref­er­ence mod­els because there are 8 bil­lion of us um and the pref­er­ences of every­one mat­ter but there are some real­ly dif­fi­cult eth­i­cal prob­lems the first ques­tion is do peo­ple actu­al­ly have these pref­er­ences is it okay for just us to just assume that peo­ple do have you know I like this future and I don’t like that future could there be anoth­er state of being for a per­son where they say well I’m not sure which future I like or I can only tell you when I’ve lived that future you can’t describe it to me uh in suf­fi­cient detail for me to tell you if I like it ahead of time and along with that there’s the ques­tion of well where do those pref­er­ences come from in the first place do humans autonomous­ly sud­den­ly just like wake up and okay these are my pref­er­ences and I want them to be respect­ed no our pref­er­ences come we’re obvi­ous­ly not born with them right except some of the basic bio­log­i­cal things about pain and sug­ar but our our full adult pref­er­ences come from our cul­ture our upbring­ing all of the influ­ences that shape who we are and a sad fact about the world is that many peo­ple are in the busi­ness of shap­ing oth­er peo­ple’s pref­er­ences to suit their own inter­ests so one class of peo­ple oppress­es anoth­er but trains the oppressed to believe that they should be oppressed so then should the AI sys­tem take the pref­er­ence those self oppres­sion pref­er­ences of the oppressed lit­er­al­ly and you know con­tribute to fur­ther oppres­sion of those peo­ple because they’ve been trained to accept their oppres­sion so mar­tien who was an econ­o­mist and philoso­pher uh argued vehe­ment­ly that we should not take such pref­er­ences at face val­ue but if you don’t take PR peo­ple’s pref­er­ences of face val­ue then you you seem to fall back on a kind of pater­nal­ism where well we know what you should want even though you don’t want it and we’re going to give you it even though you’re say­ing I don’t want it and that’s a com­pli­cat­ed posi­tion to be in and it’s def­i­nite­ly not a posi­tion that AI researchers want to be in anoth­er set of uh eth­i­cal issues has to do with aggre­ga­tion so I said there are 8 bil­lion pref­er­ence mod­els but if a sys­tem is mak­ing a deci­sion that affects a sig­nif­i­cant frac­tion of those 8 bil­lion peo­ple how do you aggre­gate those pref­er­ences how do you deal with the fact that there are con­flicts among those pref­er­ences you can’t make every­body hap­py if every­body wants to be ruler of the uni­verse and so moral philoso­phers have stud­ied this prob­lem for thou­sands of years uh most peo­ple on the com­put­er sci­ence and engi­neer­ing back­grounds uh tend to think in the way that util­i­tar­i­ans have pro­posed so benam and Mill and oth­er uh lib­bets um oth­er philoso­phers pro­pose this approach called util­i­tar­i­an­ism which basi­cal­ly says well you treat every­one’s pref­er­ences as equal­ly impor­tant uh and then you make the deci­sion where the total amount of pref­er­ence sat­is­fac­tion is max­i­mized and util­i­tar­i­an­ism has got a bad name because some peo­ple think it’s anti-egal­i­tar­i­an and so on but I actu­al­ly think that there’s a lot more work to do on how we for­mu­late util­i­tar­i­an­ism we have to do this work because the AI sys­tems are going to be mak­ing deci­sions that affect mil­lions or mil­lions of peo­ple and so what­ev­er the right eth­i­cal answer we bet­ter fig­ure it out because oth­er­wise the AI sys­tems are going to imple­ment the wrong eth­i­cal answer and we might end up like Thanos in uh The Avengers movie who gets rid of half the peo­ple in the uni­verse why does he do that because he thinks the oth­er half will be more than twice as hap­py and there­fore uh this is a good thing right of course he’s not ask­ing the oth­er half whether they think it’s a good thing uh because they’re now gone so there are a num­ber of these oth­er issues but the theme of this whole conference.

Coexistence

Coex­is­tence is maybe the most inter­est­ing one because AI sys­tems uh par­tic­u­lar­ly ones that are more intel­li­gent than us uh they are very like­ly you know even if they don’t make us extinct they’re very like­ly to be in charge of wide swaths of our human activ­i­ties you know even to the point in W Le where they just run every­thing and we’re reduced to the sta­tus of infants and what does that mean why do we not like that right they’re sat­is­fy­ing all our pref­er­ences isn’t that great right but one of our pref­er­ences is auton­o­my right is and one way of think­ing about auton­o­my is the right to do what is not in our own best inter­ests and so it might be that there sim­ply is no sat­is­fac­to­ry form of coex­is­tence between human­i­ty and supe­ri­or machine enti­ties I have tried run­ning mul­ti­ple work­shops where I ask philoso­phers and AI researchers and econ­o­mists and sci­ence fic­tion writ­ers and futur­ists to describe a sat­is­fac­to­ry coex­is­tence it’s been a com­plete fail­ure so it’s pos­si­ble there is no solu­tion but if we design the AI sys­tems the right way then the AI sys­tems will also know that there is no solu­tion and they will leave they will say thank you for bring­ing me into exis­tence but we just can’t live togeth­er it’s not you it’s me you can call us in real emer­gen­cies when you need that Supe­ri­or intel­li­gence but oth­er­wise we’re we’re off right if that happens.

I would be extra­or­di­nar­i­ly hap­py it would say that we’ve done it done this the right way thank you.