On the 10th of May this year, DeepMind published a paper that again seized the attention of the scientific community as a whole: they implemented self-navigating AI agents using the latest neural findings.
But what on earth does that mean, and why does it matter?
In short, it’s a navigation algorithm that works in the partially observable environment, working out the route incrementally without human guidance.
It’s completely unique from the kinds of algorithms that a computer science student may encounter in their AI courses. Algorithms like BFS, DFS, A* Search, and Dijkstra are all heuristics heavily influenced by human intelligence. They only work in the fully observable environment, meaning that a roadblock left out by a map update will completely mess with the navigation.
These algorithms can’t re-adjust their route until the destination is reached again, and a complete rerun with an up-to-date map is necessary to circumvent the obstacle. But the algorithm that DeepMind introduced can update its navigation strategy on the fly with only velocity, direction, and sight as input.
Yep, exactly like a creature.
The key ingredient is a series of algorithmic creations: recurrent neural nets, convolutional neural nets, and reinforcement learning (RL). In 2013, DeepMind successfully created RL agents which learned to play Atari games and achieve superhuman performance. Two years later, built on that core technology, Alphago came into being.
Reinforcement learning works as the mathematical engine for the agents’ self-development, and DeepMind is convinced that RL will be the stepping stone to achieving strong AI. This time, RL is implemented with a kind of recurrent neural network called long short-term memory, which essentially stores previous inputs and builds itself upon them.
The fact that this algorithm exceeds the navigational ability of animals proves that it’s a solid step towards stronger AI. It also shows great potential for new applications, like in-house path-finding robots which don’t rely on satellites.
Interestingly, DeepMind’s network is showing grid-like representations that resemble the 2005 Nobel Prize-winning discovery of grid cells in the brain.
For the past 13 years, neural science had been yet to find hard evidence to prove grid cells’ contribution to vector-based navigation, but the insights revealed by DeepMind’s paper has strengthened scientists’ beliefs.
This could redefine the role of computer science as a branch of scientific discipline. Computer science has long been the field of experimentation for the upper-stream disciplines like physics and neuroscience. It takes inspiration from how nature and the human mind work to create effective algorithms, and in return validates or falsifies the theories. But with DeepMind’s new work, computer science may have a new role in shaping the way theories are developed in natural science.
VET moves fast. Stay informed, with blogs straight to your inbox.