AI Now-人工智能（AI）多元化危机（英文）-2019.4-33页.pdf

《AI Now-人工智能（AI）多元化危机（英文）-2019.4-33页.pdf》由会员分享，可在线阅读，更多相关《AI Now-人工智能（AI）多元化危机（英文）-2019.4-33页.pdf（35页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、DISCRIMINATING SYSTEMSGender,Race,and Power in AISarah Myers West,AI Now Institute,New York University Meredith Whittaker,AI Now Institute,New York University,Google Open ResearchKate Crawford,AI Now Institute,New York University,Microsoft ResearchAPRIL 2019Cite as:West,S.M.,Whittaker,M.and Crawford

2、,K.(2019).Discriminating Systems:Gender,Race and Power in AI.AI Now Institute.Retrieved from https:/ainowinstitute.org/discriminatingsystems.html.This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International LicenseRESEARCH FINDINGSRECOMMENDATIONSINTRODUCTIONWHICH HUMANS

3、 ARE IN THE LOOP?HOW WORKFORCES AND AI SYSTEMS INTERACTWHO MAKES AI?Diversity Statistics in the AI Industry:Knowns and UnknownsFROM WORKFORCES TO AI SYSTEMS:THE DISCRIMINATION FEEDBACK LOOPCORPORATE DIVERSITY:BEYOND THE PIPELINE PROBLEMCore Themes in Pipeline ResearchLimitations of Pipeline Research

4、Pipeline Dreams:After Years of Research,The Picture WorsensWORKER-LED INITIATIVESTHE PUSHBACK AGAINST DIVERSITYCONCLUSION3458 1012 15 19 212325 262832CONTENTSRESEARCH FINDINGS There is a diversity crisis in the AI sector across gender and race.Recent studies found only 18%of authors at leading AI co

5、nferences are women,i and more than 80%of AI professors are men.ii This disparity is extreme in the AI industry:iii women comprise only 15%of AI research staff at Facebook and 10%at Google.There is no public data on trans workers or other gender minorities.For black workers,the picture is even worse

6、.For example,only 2.5%of Googles workforce is black,while Facebook and Microsoft are each at 4%.Given decades of concern and investment to redress this imbalance,the current state of the field is alarming.The AI sector needs a profound shift in how it addresses the current diversity crisis.The AI in

7、dustry needs to acknowledge the gravity of its diversity problem,and admit that existing methods have failed to contend with the uneven distribution of power,and the means by which AI can reinforce such inequality.Further,many researchers have shown that bias in AI systems reflects historical patter

8、ns of discrimination.These are two manifestations of the same problem,and they must be addressed together.The overwhelming focus on women in tech is too narrow and likely to privilege white women over others.We need to acknowledge how the intersections of race,gender,and other identities and attribu

9、tes shape peoples experiences with AI.The vast majority of AI studies assume gender is binary,and commonly assign people as male or female based on physical appearance and stereotypical assumptions,erasing all other forms of gender identity.Fixing the pipeline wont fix AIs diversity problems.Despite

10、 many decades of pipeline studies that assess the flow of diverse job candidates from school to industry,there has been no substantial progress in diversity in the AI industry.The focus on the pipeline has not addressed deeper issues with workplace cultures,power asymmetries,harassment,exclusionary

11、hiring practices,unfair compensation,and tokenization that are causing people to leave or avoid working in the AI sector altogether.The use of AI systems for the classification,detection,and prediction of race and gender is in urgent need of re-evaluation.The histories of race science are a grim rem

12、inder that race and gender classification based on appearance is scientifically flawed and easily abused.Systems that use physical appearance as a proxy for character or interior states are deeply suspect,including AI tools that claim to detect sexuality from headshots,iv predict criminality based o

13、n facial features,v or assess worker competence via micro-expressions.vi Such systems are replicating patterns of racial and gender bias in ways that can deepen and justify historical inequality.The commercial deployment of these tools is cause for deep concern.i.Element AI.(2019).Global AI Talent R

14、eport 2019.Retrieved from https:/jfgagne.ai/talent-2019/.ii.AI Index 2018.(2018).Artificial Intelligence Index 2018.Retrieved from http:/cdn.aiindex.org/2018/AI%20Index%202018%20Annual%20Report.pdf.iii.Simonite,T.(2018).AI is the future-but where are the women?WIRED.Retrieved from https:/ neural net

15、works are more accurate than humans at detecting sexual orientation from facial images.Journal of Personality and Social Psychology.v.Wu,X.and Zhang,X.(2016).Automated Inference on Criminality using Face Images.Retrieved from https:/arxiv.org/pdf/1611.04135v2.pdf.vi.Rhue,L.(2018).Racial Influence on

16、 Automated Perceptions of Emotions.Retrieved from https:/ Systems:Gender,Race,and Power in AI|Research Findings|3RECOMMENDATIONS 1.Publish compensation levels,including bonuses and equity,across all roles and job categories,broken down by race and gender.2.End pay and opportunity inequality,and set

17、pay and benefit equity goals that include contract workers,temps,and vendors.3.Publish harassment and discrimination transparency reports,including the number of claims over time,the types of claims submitted,and actions taken.4.Change hiring practices to maximize diversity:include targeted recruitm

18、ent beyond elite universities,ensure more equitable focus on under-represented groups,and create more pathways for contractors,temps,and vendors to become full-time employees.5.Commit to transparency around hiring practices,especially regarding how candidates are leveled,compensated,and promoted.6.I

19、ncrease the number of people of color,women and other under-represented groups at senior leadership levels of AI companies across all departments.7.Ensure executive incentive structures are tied to increases in hiring and retention of under-represented groups.8.For academic workplaces,ensure greater

20、 diversity in all spaces where AI research is conducted,including AI-related departments and conference committees.Recommendations for Addressing Bias and Discrimination in AI Systems9.Remedying bias in AI systems is almost impossible when these systems are opaque.Transparency is essential,and begin

21、s with tracking and publicizing where AI systems are used,and for what purpose.10.Rigorous testing should be required across the lifecycle of AI systems in sensitive domains.Pre-release trials,independent auditing,and ongoing monitoring are necessary to test for bias,discrimination,and other harms.1

22、1.The field of research on bias and fairness needs to go beyond technical debiasing to include a wider social analysis of how AI is used in context.This necessitates including a wider range of disciplinary expertise.12.The methods for addressing bias and discrimination in AI need to expand to includ

23、e assessments of whether certain systems should be designed at all,based on a thorough risk assessment.Recommendations for Improving Workplace DiversityDiscriminating Systems:Gender,Race,and Power in AI|Recommendations|4INTRODUCTIONThere is a diversity crisis in the AI industry,and a moment of recko

24、ning is underway.Over the past few months,employees have been protesting across the tech industry where AI products are created.In April 2019,Microsoft employees met with CEO Satya Nadella to discuss issues of harassment,discrimination,unfair compensation,and lack of promotion for women at the compa

25、ny.1 There are claims that sexual harassment complaints have not been taken seriously enough by HR across the industry.2 And at Google,there was an historic global walkout in November 2018 of 20,000 employees over a culture of inequity and sexual harassment inside the company,triggered by revelation

26、s that Google had paid$90m to a male executive accused of serious misconduct.3 This is just one face of the diversity disaster that now reaches across the entire AI sector.The statistics for both gender and racial diversity are alarmingly low.For example,women comprise 15%of AI research staff at Fac

27、ebook and just 10%at Google.4 Its not much better in academia,with recent studies showing only 18%of authors at leading AI conferences are women,5 and more than 80%of AI professors are male.6 For black workers,the picture is worse.For example,only 2.5%of Googles workforce is black,7 while Facebook a

28、nd Microsoft are each at 4%.8,9 We have no data on trans workers or other gender minorities.Given decades of concern and investment to redress the imbalances,the current state of the field is alarming.The diversity problem is not just about women.Its about gender,race,and most fundamentally,about po

29、wer.10 It affects how AI companies work,what products get built,who they are designed to serve,and who benefits from their development.This report is the culmination of a year-long pilot study examining the scale of AIs current diversity crisis and possible paths forward.This report draws on a thoro

30、ugh review of existing literature and current research working on issues of gender,race,class,and artificial intelligence.The review was purposefully scoped to encompass a variety of disciplinary and methodological perspectives,incorporating literature from computer science,the social sciences,and h

31、umanities.It represents the first stage of a multi-year project examining the intersection of gender,race,and power in AI,and will be followed by further studies and research articles on related issues.1 Tiku,N.(2019,Apr.4).Microsoft Employees Protest Treatment of Women to CEO Nadella.WIRED.Retrieve

32、d from https:/ Gershgorn,D.(2019,Apr.4).Amid employee uproar,Microsoft is investigating sexual harassment claims overlooked by HR.Quartz.Retrieved from https:/ Statt,N.(2018,Nov.2).Over 20,000 Google employees participated in yesterdays mass walkout.The Verge.Retrieved from https:/ Simonite,T.(2018)

33、.AI is the future-but where are the women?WIRED.Retrieved from https:/ Element AI.(2019).Global AI Talent Report 2019.Retrieved from https:/jfgagne.ai/talent-2019/.6 AI Index 2018.(2018).Artificial Intelligence Index 2018.Retrieved from http:/cdn.aiindex.org/2018/AI%20Index%202018%20Annual%20Report.

34、pdf.7 Google.(2018).Google Diversity Annual Report 2018.Retrieved from https:/ Williams,M.(2018,July 12).Facebook 2018 Diversity Report:Reflecting on Our Journey.Retrieved from https:/ Microsoft.(2019).Diversity&Inclusion.Retrieved from https:/ As authors of this report,we feel its important to ackn

35、owledge that,as white women,we dont experience the intersections of oppression in the same way that people of color and gender minorities,among others,do.But the silence of those who experience privilege in this space is the problem:this is in part why progress on diversity issues moves so slowly.It

36、 is important that those of us who do work in this space address these issues openly,and act to center the communities most affected.Discriminating Systems:Gender,Race,and Power in AI|Introduction|5To date,the diversity problems of the AI industry and the issues of bias in the systems it builds have

37、 tended to be considered separately.But we suggest that these are two versions of the same problem:issues of discrimination in the workforce and in system building are deeply intertwined.Moreover,tackling the challenges of bias within technical systems requires addressing workforce diversity,and vic

38、e versa.Our research suggests new ways of understanding the relationships between these complex problems,which can open up new pathways to redressing the current imbalances and harms.From a high-level view,AI systems function as systems of discrimination:they are classification technologies that dif

39、ferentiate,rank,and categorize.But discrimination is not evenly distributed.A steady stream of examples in recent years have demonstrated a persistent problem of gender and race-based discrimination(among other attributes and forms of identity).Image recognition technologies miscategorize black face

40、s,11 sentencing algorithms discriminate against black defendants,12 chatbots easily adopt racist and misogynistic language when trained on online discourse,13 and Ubers facial recognition doesnt work for trans drivers.14 In most cases,such bias mirrors and replicates existing structures of inequalit

41、y in society.In the face of growing evidence,the AI research community,and the industry producing AI products,has begun addressing the problem of bias by building on a body of work on fairness,accountability,and transparency.This work has commonly focused on adjusting AI systems in ways that produce

42、 a result deemed“fair”by one of various mathematical definitions.15 Alongside this,we see growing calls for ethics in AI,corporate ethics boards,and a push for more ethical AI development practices.16But as the focus on AI bias and ethics grows,the scope of inquiry should expand to consider not only

43、 how AI tools can be biased technically,but how they are shaped by the environments in which they are built and the people that build them.By integrating these concerns,we can develop a more accurate understanding of how AI can be developed and employed in ways that are fair and just,and how we migh

44、t be able to ensure both.Currently,large scale AI systems are developed almost exclusively in a handful of technology companies and a small set of elite university laboratories,spaces that in the West tend to be extremely white,affluent,technically oriented,and male.17 These are also spaces that hav

45、e a history of problems of discrimination,exclusion,and sexual harassment.As Melinda Gates describes,“men who demean,degrade or disrespect women have been able to operate with such impunitynot just in Hollywood,but in tech,venture capital,and other spaces where 11 Alcine,J.(2015).Twitter.Retrieved f

46、rom https:/ Angwin,J.,Larson,J.,Mattu,S.and Kirchner,L.(2016,May 3).Machine Bias.ProPublica,https:/www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.13 Vincent,J.(2016,Mar 24).Twitter taught Microsofts AI chatbot to be a racist asshole in less than a day.The Verge.Retri

47、eved from https:/ Melendez,S.(2018,Aug.9).Uber driver troubles raise concerns about transgender face recognition.Fast Company,Retrieved from https:/ Narayanan,A.(2018).21 fairness definitions and their politics.ACM Conference on Fairness,Accountability and Transparency.Retrieved from https:/ Vincent

48、,J.(2019,Apr.3).The Problem with AI Ethics.The Verge.Retrieved from https:/ Crawford,K.(2016,June 25).Artificial Intelligences White Guy Problem.The New York Times.Retrieved from https:/ Systems:Gender,Race,and Power in AI|Introduction|6their influence and investment can make or break a career.The a

49、symmetry of power is ripe for abuse.”18 Or as machine learning researcher Stephen Merity noted at the end of 2017,“Bias is not just in our datasets,its in our conferences and community.”19 Both within the spaces where AI is being created,and in the logic of how AI systems are designed,the costs of b

50、ias,harassment,and discrimination are borne by the same people:gender minorities,people of color,and other under-represented groups.Similarly,the benefits of such systems,from profit to efficiency,accrue primarily to those already in positions of power,who again tend to be white,educated,and male.Th