THE GUARDIAN — Silicon Valley’s business model is incompatible with the moderation of online horror and hatred in the internet

Way back in the mid-1990s, when the web was young and the online world was buzzing with blogs, a wor­ry­ing prob­lem loomed. If you were an ISP that host­ed blogs, and one of them con­tained mate­r­i­al that was ille­gal or defam­a­to­ry, you could be held legal­ly respon­si­ble and sued into bank­rupt­cy. Fear­ing that this would dra­mat­i­cal­ly slow the expan­sion of a vital tech­nol­o­gy, two US law­mak­ers, Chris Cox and Ron Wyden, insert­ed 26 words into the Com­mu­ni­ca­tions Decen­cy Act of 1996, which even­tu­al­ly became sec­tion 230 of the Telecom­mu­ni­ca­tions Act of the same year. The words in ques­tion were: “No provider or user of an inter­ac­tive com­put­er ser­vice shall be treat­ed as the pub­lish­er or speak­er of any infor­ma­tion pro­vid­ed by anoth­er infor­ma­tion con­tent provider.” The impli­ca­tions were pro­found: from now on you bore no lia­bil­i­ty for con­tent pub­lished on your platform.

Mod­er­a­tion, how­ev­er, has two prob­lems. One is that it’s very expen­sive because of the sheer scale of the prob­lem: 2,500 new videos uploaded every minute to YouTube, for exam­ple; 1.3bn pho­tos are shared on Insta­gram every day. Anoth­er is the way the dirty work of mod­er­a­tion is often out­sourced to peo­ple in poor coun­tries, who are trau­ma­tised by hav­ing to watch videos of unspeak­able cru­el­ty – for pit­tances. The costs of keep­ing west­ern social media feeds rel­a­tive­ly clean are thus borne by the poor of the glob­al south.

Mod­er­a­tion, how­ev­er, has two prob­lems. One is that it’s very expen­sive because of the sheer scale of the prob­lem: 2,500 new videos uploaded every minute to YouTube, for exam­ple; 1.3bn pho­tos are shared on Insta­gram every day. Anoth­er is the way the dirty work of mod­er­a­tion is often out­sourced to peo­ple in poor coun­tries, who are trau­ma­tised by hav­ing to watch videos of unspeak­able cru­el­ty – for pit­tances. The costs of keep­ing west­ern social media feeds rel­a­tive­ly clean are thus borne by the poor of the glob­al south.