A common analogy for the way that AI systems learn is the way a child learns: Not by being told rules for how to recognize an image or walk, but by taking in information from many examples and drawing connections that we can’t necessarily articulate.
You wouldn’t leave a child without supervision. And we shouldn’t leave AI systems without supervision, either.
The simple fact is that, despite the learning potential of AI, it is in its infancy and cannot be left unattended. From over-optimized GPS sending unwitting tourists into barely existent dirt paths inside Death Valley to Microsoft’s AI Twitter bot going rogue after Twitter pranksters filled its brain with hate speech; whether unintentional or by malicious design, algorithms behaving unexpectedly are now a fact of life.
Unlike humans, algorithms, don’t have experiences of their own. They are not able to reflect upon past experiences to course correct. While humans may use common-sense skills to wrap around their job-skills, algorithms don’t have that option. They rely entirely on the data presented to them during ‘training.’
AI algorithms ‘learn on the job’ by using human feedback. They are under the influence of millions, if not billons, of people interacting with them. If companies aren’t careful, the tendencies of these people influence algorithms toward unintended outcomes.
That’s why organizations need to ‘parent’ algorithms that operate in the most critical components of their business. They must closely track an algorithm’s behavior as it changes over time (and in different contexts); and when needed, mitigate malicious behavior. Just as a parent shapes a child’s development by managing influential forces like friends, teachers, TV and more, algorithms require similar support as they mature by consuming new data.
The simple act of gathering the business data used in training an algorithm is very subjective; two people picking data for the training could bias the system unexpectedly. Bias can also be introduced by anyone who alters the data along the way, like business analysts, data engineers, or data scientists.
The most frequent intervention performed by a “parent” is correcting an algorithm’s response to some unforeseen pattern that has emerged in the data. “Parenting” comes in the form of putting safeguards in place. The most important variation of this is to combine an ‘expert-in-the-loop’ with a “golden data set.” Golden data refers to specific data which has been reviewed by a human (with multiple experts) to ensure that, for a certain input into the machine learning system, there is a single expert-agreed correct output. For example, all the big search engine companies use humans to train their search engines. They heavily rely on golden data sets to validate that these humans are not teaching the “wrong things” to the search engine, but also to ensure that the search engine has not drifted from producing the correct inputs.
A classic example of algorithms needing “parenting” is the now popular effect of algorithms promoting fake news articles to unsuspecting readers. The proliferation of these articles is a product of bad actors figuring out what parameter the News Recommender Algorithm is using and then creating fake news articles that leverage those parameters. For example, Facebook and Google recently announced that they would spend billions of dollars fixing the fake news problem—this “fix” will involve treading down the path of setting up processes that involve parenting.
Eventually, the proliferation of intricate human common knowledge into everyday use might help algorithms grow up and start to rely on broader experiences outside of their training data. But today, companies need to take responsibility for the quality, effectiveness, integrity and resilience of these algorithms–and one way to do that is by acknowledging and addressing the need to parent algorithms over their entire life cycle.
Sreekar Krishna is the managing director of data science & innovation at KPMG.