In the home stretch.
I'm wrapping up my semester-long research project on machine learning classification systems. I've been investigating how to operationalize the many suggestions for governance and regulation of this space.
Here's the current draft of the intro (minus the citations). A partial works referenced list can be found here. If you have feedback or relevant recent research on these topics, drop me a line at rnw13@georgetown.edu. I'm eager to learn more!
What has come to be known as artificial intelligence can provide substantial benefit to society. While in the public consciousness, the term may stir up visions of superintelligent machines or job-stealing robots, these technologies are about much more than doom and gloom. They can help humans tackle real-world issues, from cancer research to wildlife protection to child welfare decisions.
Yet, as decisionmaking is increasingly automated, driven by massive datasets that are fed into powerful pattern-recognizing algorithms that make statistical predictions, problems with these machine learning models have emerged. Some of them have made incorrect predictions about health risks. Others have been shown to discriminate based on gender, sexual orientation, or race. They make what could be described as questionable decisions about people’s employability, who should to be sent to jail and who can go home following arrest, and teacher performance. The technology can amplify values and patterns that are offensive to some and harmful to others.
At heart, these automated decisionmaking tools are about putting humans into categories, then drawing conclusions and making predictions about them based on that characterization. This isn’t a new phenomenon. The capacity to weave narratives about the world—to recognize patterns and categorize things—has helped humans become the thinking beings they are today. And more specifically, the credit industry, among others, has made these kinds of predictions for decades.
But the current moment is different. Because of advances in computing power and data availability, these machine learning tools can make predictions in a wider range of areas than ever before. They are touching lives in unprecedented ways. And they are on track to continue to creep into uncharted territory. These models can reveal information about a person without a clear reason why—the prediction made may not be directly related, as far as humans can tell, to any data points that the machine was fed. That’s a disconcerting feature that breeds distrust and fear, particularly when the predictions are wrong.
Both the popular press and the academic literature have called for something to be done—for automated systems that make consequential decisions about humans to be regulated or held accountable in some way.
An alternative framing may be more useful. Though these tools can be used for ill, they also offer a chance to understand humanity better, and to improve societies in the process. There is nothing inherently good or bad about the technology itself. But it puts a fine point on deeply embedded societal biases and problems, challenging us to think deeply and precisely about the kind of society we have and the kind of society we want.
Many of the proposals to build accountability into these machine learning classification systems—including the European Union’s General Data Protection Regulation that enters into force in 2018—focus on the end of the stream. The spotlight is on the need to make predictive models understandable and more transparent to those affected by the decisions the machines make.
Yet, the challenge is much broader than building external regulatory frameworks that protect humans from harms or offer recourse when they occur. It is broader too than technical approaches to solve internal problems with system design, or efforts that aim to inject foundational values into developer education. It is about making all parts of this broad, complex network of actors and ideas work together to embed norms and protections for all stakeholders at all levels. The goal should be building robust systems that make the best possible use of the technology while protecting values and embedding accountability mechanisms every step of the way.
The question is, how do you do that? The answer in practice will vary depending on the context. But it is clear that a focus from the outside looking in risks missing a significant part of the discussion. It also risks forcing technical tradeoffs that can unnecessarily curtail the benefits of technological advances.
To flesh out an answer, I took machine learning classification systems as a starting point given the robust debate in that space. Based on literature and discussions, I built out a sociotechnical map of the life cycle of these systems, from foundational principles to development to recourse options, paired with an overview of the processes, actors, and interventions. There are other ways to go about tackling this problem. I could have, for instance, chosen to dive deep into an industry or harm. There is indeed good work being done in those spaces. But stepping back, taking a broad view, and seeing how all the puzzle pieces fit together is necessary for building comprehensive and flexible solutions.
This is meant to be a road map for those seeking to orient themselves in this space, be they new entrants or seasoned subject-area experts seeking a broader view. It is a guide for those interested in governance of emerging technologies that rely on big data and have consequential impacts on human life. The aim is not to restate the substantial and ever-growing literature in this field, but to instead map out a common framework for further development of best practices and other rules of the road.
More generally, this is a call to see all sides. The challenges in this space are sociotechnical ones, as many have pointed out. That means they must be tackled from both technical and social perspectives in a holistic way that excavates and balances the needs of all stakeholders. No one solution will work on its own. And because it is early days for work in this area, that means keeping as many options on the table as possible until paths forward become clearer.