Think further into the future: An approach to better RAI programs
As artificial intelligence (AI) continues to permeate various aspects of society, the urgency to implement robust Responsible AI (RAI) programs has never been greater. Traditional approaches often focus on immediate risks and foreseeable consequences, but the rapidly evolving nature of AI demands a more forward-thinking strategy. To truly safeguard against ethical pitfalls and unintended consequences, organizations must adopt a mindset that looks further into the future, predicting the far-reaching impacts of current trends.
Let’s look at a framework I use in my advisory work grounded in (1) proactive risk assessment, (2) scenario planning, and (3) ethical foresight. By extrapolating further than conventional methods and anticipating potential scenarios that might initially seem improbable, organizations can build AI systems that are responsible today and resilient to future challenges. The goal is to create AI governance structures that are adaptable, inclusive, and prepared for a wide range of outcomes.
Key takeaways
Anticipate Future Challenges: By looking beyond the obvious, you prepare for a wider range of outcomes.
Integrate Diverse Perspectives: Combining insights from various fields helps in understanding complex trends.
Adapt and Evolve: Flexible policies and continuous learning ensure the Responsible AI program remains relevant and effective.
While some may write this off as a futurist or forecasting approach, others might also evoke the pre-mortem, the framework that we discuss here is a systematized way of thinking about the entire lifecycle of the RAI program, i.e., we want to ensure that robustness is a baked-in characteristic rather than an afterthought when the RAI program inevitably faces challenges of efficacy over the duration of its lifetime.
Proactive Risk Assessment
Go beyond immediate consequences and anticipate long-term impacts, including unlikely but plausible scenarios. In particular, think about second-order effects and adopt a systems-thinking mindset that can help propagate the blast radius of the harmful impacts from unintended outcomes from the AI system as far as possible (and if you’re up for it, as severely as possible).
For example, when designing a machine translation system, don’t just think about the immediate set of users who might experience disparate outcomes due to degraded performance on low-resourced languages, e.g., non-English languages. Instead, go beyond and also think about if you have this system exposed as an API, and there are downstream developers who integrate your machine translation service to provide critical support for translating government documents (e.g., a potential use case for a country like India that has >200 languages), it might lead to people being unable to access benefits that they are entitled to (and depend on!).
Scenario Planning
Develop a wide range of scenarios, including extreme cases that might initially seem unreasonable. This helps prepare for a broad spectrum of outcomes and ensures the program is resilient. In particular, don’t limit yourself to a simple extrapolation based on historical trends that you’ve experienced. This is a common oversight, e.g., we saw risk models fail catastrophically in the 2008 financial crisis because there was no accounting for outcomes that lay more than a couple of standard deviations away from the norm. Yet, black swan events are what really test whether a contingency system works and whether the system is resilient or not, so as much as possible, plan for them!
For example, would your AI system survive a coordinated attack from motivated malicious actors who seek to poison the interactions that the system uses to learn continually to the point that the system becomes unusable? If so, will the system be able to recognize that it is operating out of bounds and should cease operations? Are there fallback mechanisms that can take their place when they need to be pulled offline? In fact, in some cases, there isn’t even the possibility of being able to pull a system offline because it is the only option or it is too deeply intertwined with the rest of the software infrastructure, such that pulling it offline would mean shutting down the entire service. If so, is your offering so critical that it would endanger the safety, health, and well-being of human users who depend on it?
Ethical Foresight
Predict future ethical dilemmas that might arise as AI capabilities and the surrounding societal ecosystems evolve. This includes considering potential misuse or unintended consequences far ahead of current discussions. In particular, we want to be resilient to changing societal norms and what constitutes acceptable behavior. It also means thinking about how the supplementing legal landscape might change and any other regulatory shifts that might occur that might fundamentally alter how people interact with AI systems and with each other, potentially mediated by these systems.
For example, use of language is constantly evolving such that terms that were previously pejorative might be retaken by the target community as a term of empowerment in which case flagging those terms in content moderation would actually impede valid expression rather than protecting users as might have been the case originally when the phrase/term might have been on a banned list. Multimodality is another place where clashes can happen. For example, perfectly innocuous text and images might separately be ok but if combined into a meme such as overlaying “You smell nice today” over the picture of a skunk can become problematic.
So, now that we know the core components of this framework to think further into the future, let’s take a look at a simple implementation plan that can help get started with putting these ideas into practice:
Establish a Foresight Team: Create a dedicated team to continuously monitor trends and project future impacts. They can do so by looking at GitHub repositories, research conferences, preprint servers, stalking X (I know, I know!), and frequenting relevant Discord servers where some of the more active minds gather to discuss what they are working on and what they anticipate to be coming up as significant developments. As always, having diversity both in the sources that the team consults and the composition of the team itself will help with the efficacy of this strategy.
Develop Long-term Metrics: Implement metrics that track both short-term and long-term impacts of AI deployments. There are several places like Metaculus, MLPerf, and various LLM eval benchmarks that look at the objective metrics that rank AI systems. In addition, paying attention to subjective metrics such as those captured in the Stanford AI Index report’s chapter on AI ethics and the AI Incident Database are good places to keep an eye out for the long-term impacts.
Conduct Regular Futures Scenario Workshops: Host workshops with internal and external experts to explore future scenarios. No idea is too crazy to consider. In fact, following the principles of ugly babies from Ed Catmull and kill your darlings from (recently popularized) Steven Pressfield serves as a good balance of following the diamond of divergent and then convergent thinking to ensure that there is full-spectrum coverage of possible futures.
Build Flexible Policies: Design policies that are not rigid but can adapt to new information and changing circumstances. Given the pace of development in AI, with new, more powerful, and capable models being released almost every month, we never know how the capabilities frontier is going to change. Having to go through a long-winded process to revise a policy unnecessarily introduces wear-and-tear on the organizational processes (and people!) who are involved in setting, implementing, and revising policies.
While there are no perfect solutions to an inherently (very) complex beast that we’re trying to tackle, i.e., implementing a program within an organization (a complex system) to address sociotechnical issues (a complex domain), we can certainly try to reduce our blindspots by widening the aperture a bit and peering into a variety of futures that are made possible with AI capabilities. And most importantly, using the insights from those exercises to build better policies and processes today to steer ourselves and our organizations toward success.
