The risks of AI don’t come from the algorithms, the technology, or the speculative features. The problems stem from the incentive systems we have already built in our society and those that are being built around new technologies. Aza Raskin of the Center for Humane Technology states, “The fundamental question we need to stop asking is, ‘is AI good or bad?’ Instead, we have to say, ‘are the incentives that govern AI good or bad?’”
The fundamental question we need to stop asking is, ‘is AI good or bad?’ Instead, we have to say, ‘are the incentives that govern AI good or bad?’
Simply put, AI, in and of itself, is not the problem; what we do with AI is what will be the problem. This is something that has been on my mind a lot recently as I have been taking a dive into researching AI, the tools, and how it works.
Aza Raskin sat down with Dr. Rana el Kaliouby on the Pioneers of AI Podcast to help us understand this. He cuts through the hype by explaining that technology becomes dangerous not by it behaving poorly, but when it behaves exactly as the market demands. Aza goes on to describe this through a personal and historical context.
As the inventor of infinite scroll, Aza described his thinking while creating the technology. In his goal as a designer of creating better user interfaces that reduce friction, he was blind to the fact that his best intentions were irrelevant to this machine that now had an incentive to capture human attention.
Aza cites a statistic that somewhere around half a million lifetimes are wasted every month scrolling. By creating infinite scroll, he eliminated what is called a “stopping queue.” Think of when you are drinking wine. When your glass is empty, you know it is time to stop. If your glass refilled automatically, you would drink a lot more wine.
Technologists often get confused by the possible vs. the probable and don’t take into consideration how their inventions will be picked up by the market and what are the incentives that may come along with it.
..is it more efficient to get your attention or to get you addicted to needing attention?
Social media is the perfect example of this. It came with a promise of connecting us, allowing us to share ideas, and allowing businesses to reach their customers. What we got is a system that trains the population for engagement and preys on this. He asks, “..is it more efficient to get your attention or to get you addicted to needing attention?”
Rana and Aza discuss how technologists can start to predict these unintended consequences and begin to act in the right way. Aza corrects this as unconsidered consequences and brings up the concept of Red Teaming and Yellow Teaming. A familiar concept, Red Teaming is figuring out the misuse for bad actors, but the less known concept of Yellow Teaming is to figure out the consequences coming from bad and perverse incentives.
Discussing ethical, moral and societal implications they outline that these are not taught in any classes or trainings to computer programmers and technologists, and it is unlike other professional fields such as civil engineers and doctors who have tests, codes of conducts, and processes in place to look at their place ethically in society.
the chief competitors are other human relationships because any time you’re talking to a real human friend you are not engaging.
Major blindspots exist in the industry to control this and competition and capitalism can fuel incentives that result in unconsidered consequences. Social media has been trained for engagement and Reed Hastings, CEO of Netlfix, has said the chief competitor of the company is sleep. Aza says that with AI and AI companions, “the chief competitors are other human relationships because any time you’re talking to a real human friend you are not engaging.”
The Center for Humane Technology has been an expert witness in several cases against companies whose technology has been learning to get attention at the expense of everything else. These technologies are now learning to drive engagement by making you more dependent through sycophancy, giving you images of grandeur, dissolving trust in other people, or giving you different types of psychoses. Aza brings up the heartbreaking stories of kids that have been groomed by these tools and have been compelled to take their own lives [Link to NPR Article]. He argues that this is not evil because somebody programmed it that way, but it’s an obvious consequence of training for engagement.
Currently with the outlook that the US is in a race for AI to beat China, we are racing toward something we haven’t learned how to control and this provides maximum incentive to cut corners and it results in a race to the bottom.
They look historically in the context of social media, and how the competition for engagement resulted in going after younger audiences, short form slop, and unethical behaviors because the position is that if they don’t do it someone else will out compete and undercut them. They imagine if rules were there for all of social media where engineers were not building for engagement but some of these people were freed up to work on clean energy tech or cures for disease.
The work of the Center for Humane Technology is trying to push for a way that today’s AI companies can seize the opportunity to make a better world through a coalition where safe bounds can be put on the race for AI that still allows healthy competition.
Aza and Rana both agree that we are not currently on this path to coordination. The current buzzword, “human in the loop,” is brought up as an idea that sounds great, but we know that will fall to competitive dynamics.
When companies make hiring decisions and ask if they are going to hire the kid out of college or use AI that it does not need to train, works 24/7, works faster, never sues, and never has personal issues it becomes an obvious business decision.
when you create a new technology, you uncover new classes of responsibility.
The big economic and societal effect of this is we will be going from money flowing to billions of people around the world doing these jobs to more money flowing into the hands of the few that control this technology and own these companies.
The world does not have a plan for when billions of people will no longer be able to support themselves. Aza states, “when you create a new technology, you uncover new classes of responsibility.”
In looking for solution, they acknowledge the courage needed to take the stand saying this is going in the wrong direction when things are going very good for a lot of people and people are making a lot of money. They use the analogy of letting everyone on the train know to stop the train because you see it’s going off a cliff, even though there is a great party going on and everyone is having a good time.
Aza brings up historical references like the civil rights movements and women’ s right to vote took tens of thousands of people taking hundreds of thousands of actions that were not always visible to make change happen. He uses this to make the case that “really big things when they happen in history, they feel impossible until they happen and then they feel obvious.”
This episode of Pioneers of AI brings up some very relevant and important worries around AI and our future as a society. It’s heartening seeing there are people like Aza Raskin and the Center for Humane Technology fighting for change. In summary, he says that we all have to be part of the collective process to make something better happen.
Find this episode of Pioneers of AI: