Addressing the NeurIPS annual conference on machine learning, Facebook AI researcher Joelle Pineau told those studying reinforcement learning they need to branch out beyond the confines of their simulated environments and incorporate more of the natural world into their work.
Facebook AI researcher urges peers to step out into the real world
The video game Breakout from Atari first burst onto the scene in 1976. The game, a simple matter of moving a paddle horizontally to bounce a ball into walls and chip away at them, has become a mainstay of training artificial intelligence.
But perhaps it is time to break out of Breakout.
Want to publish your own articles on DistilINFO Publications?
Send us an email, we will get in touch with you.
That was one notion that emerged Wednesday morning in a keynote talk by Joelle Pineau, a scientist with Facebook’s AI research unit, at the prestigious NeurIPS conference on machine learning in Montreal.
Pineau’s immediate concern is that too much of today’s work in machine learning, despite many virtues, leaves out aspects of rigor. For example, a lot of the research is done with simulated worlds, things constructed in a pristine fashion within the computer. Those simulations may help make results reproducible, which is good, but they miss a lot of the complexity of the natural world, which may make the work less meaningful, less rigorous, Pineau suggested.
“I love that the simulators help reproducibility,” said Pineau. “We want that convenience, but we also want to include some of the real world.
“We have to break out of these simulators and tackle the real world,” she urged.
Pineau focused on a flavor of machine learning she’s passionate about, called reinforcement learning, which makes extensive use of simulators. Reinforcement learning technology is about enabling a computer “agent” to take sequences of actions in an environment, and to figure out a rule for making good choices between the possible actions based on maximizing a reward that the agent receives.
Atari’s Breakout is one of many video games that are used to simulate the environment in which the agent makes choices. Such simulated training has led to computers increasingly surpassing human performance in the games. And many researchers are exploring promising avenues for use of reinforcement learning, such as for control systems for “neuro-stimulation” devices embedded in the body that could help prevent epileptic seizures.
But Pineau implied that such critical uses of reinforcement learning need to eventually grapple with real-world complexity.
Reflecting on the matter, Pineau and her team wondered, “Could we make this a lot more interesting with some natural-world signals?” They inserted videos of street scenes into the background of Breakout, and re-trained the reinforcement learning systems with the new, more “noisy” environment.
“Videos are an endless source of natural noise,” observed Pineau. As she pointed out, when humans are asked to play the game with the video background, “it’s a little harder, but they can still play.” That implies there are challenges that emerge when adding noise that can result in more impressive results if the computer can succeed against such adversity.
There is “a lot we can do to incorporate natural noise in the environment,” said Pineau. “We can set the bar much higher in the realism of our simulators.
“From static images to video backgrounds, there is a lot further we can go.”
That includes the “creation of simulation environments that are photo realistic,” she said. She showed some images of simulated house interiors, with real-world imagery grafted into the space, right down to strikingly reflective mirrors.
What is to be gained with all this, said Pineau, is doing “better science” by bringing more rigor to machine learning and AI.
Pineau ended her talk with the exhortation, “I encourage you to step out into the real world!”
Date: December 12, 2018
Source: ZDNet