posted May 18, 2015, 6:17 PM by Kevin Esvelt
updated Oct 25, 2016, 9:30 AM
The question of whether it is ethical to insist that decision-making be empirically grounded remains difficult. To reprise, is it fair for us to initiate the terms of the debate, making sure that “no” is a legitimate answer and inviting suggestions for which experiments should be done to better analyze the risks and improve our models, but specifically attempting to ground the decision-making process in empirical reality?
This approach may be just like democracy - it's a terrible system, except for all the others. That said, there's a clear catch if we're going to insist on empirical grounding: we must recognize the legitimacy of other types of concerns. The concern itself may not be empirically grounded, but the person expressing it is quite real. This person may be factually mistaken. Their own model of reality may be woefully inaccurate. They may even have a value system that would make many recoil. Yet none of that matters, because everyone should have a voice when it comes to shaping the future.
Not only is it the right thing to do from a moral perspective, ensuring that discussions are broadly inclusive has practical benefits. First, ignoring concerns is a recipe for mistrust, resentment, and determined opposition, even in those who might otherwise be neutral or supportive. If future advances will be built upon a foundation of trust – and most will – actively inclusive listening is a cornerstone.
Second, every so often an unexpected voice will catch something the rest of us missed. The more complex, ambitious, and failure-prone the technological working, the more important it is to solicit a wide variety of opinions. For the most important technologies, the ones that will likely affect nearly everyone, we should make a point of inviting every critic, Luddite, and fanatic to speak, and do our best to distill their objections down to something empirically actionable. There's simply no better way to simultaneously identify failure modes and update our priors to reflect the beliefs of others.
All of this represents a dramatic departure from conventional approaches to technology development and public engagement. Typically, troubleshooting was limited to small panels of the technically qualified, while popular opinion and the public interest were seldom consulted when it came to innovation. But we now live in a more connected and less trusting world. The cost of communication has decreased dramatically, making it far easier to collaborate and solicit feedback from people with diverse viewpoints and expertise. At the same time, trust is more valuable – and easily lost – than ever before. And the single best way to promote trust is to be radically transparent. For some powerful technologies, the relevant stakeholders may include all the citizens of the globe. As has been demonstrated again and again, the world does not readily change without popular support. In this age of greater connectedness and skepticism, that support cannot be taken for granted - for trust must be earned.
Tags: transparency, legitimacy, responsibility, innovation, philosophy, connectedness, trust