Real World Applications of Reinforcement Learning

Reinforcement learning is a type of machine learning that determines the action within a specific environment in order to maximize a reward. One of the characteristics of reinforcement learning is that the agent can only receive a reward after performing the action, and thus must continue to interact with the environment in order to determine the optimal policy through trial and error.

Reinforcement learning is used in a wide variety of applications.

Self-driving cars

Self-driving cars are not science fiction anymore. Companies such as Toyota and Ford have invested millions of dollars for R&D in this technology. Taxi services such as Uber and Lyft, currently paying human drivers, may soon deploy entire fleets of self-driving cars. In the next two to three years, hundreds of thousands of self-driving cars may be sold to regular consumers.

Google is also taking a lead in this. The Google self-driving car project is called Waymo; it stands for a new way forward in mobility.

Thirty-three corporations are working on autonomous vehicles and over $450 M is invested across 36 deals to date; auto tech start-ups are on track for yearly highs in both deals and dollars.

Many influential personnel from automobile and technology industries predict that this will happen. But the big question behind this is, when will this actually happen? The timing is the key here; by 2020 many relevant companies are planning to launch autonomous cars. Refer to the following predictions by motor companies:

Autonomous cars are the core and long-term strategy. IEEE predicts that 75 percent of vehicles will be fully autonomous by 2040.

Planning for a self-driving car is done via reinforcement learning. The car learns to continuously correct its driving capability over time through trial and error when training.

Drone autonomous aerial taxi

While people are still debating about the safety of self-driving cars, the United Arab Emirates is actually preparing to launch an autonomous aerial taxi or drone taxi. It is one of the finest examples of applying reinforcement learning.

The Road and Transport Authority (RTA), Dubai employs the Chinese firm Ehang's 184, which is the world's first passenger drone. It's capable of a range of about 30 miles on a single charge and can carry one person weighing up to 220 lbs, as well as a small suitcase. The entire flight is managed by a command center; all you need to do is hop in and choose from the list of predetermined destinations where you want to land.

<Image taken from Dubai, RTA website>

Riders can use a smartphone app to book their flights to pick them up from the designated zones. The drone taxi arrives at the designated place and the rider will go inside and get into a seat and select the pre-programmed designation using a touchscreen. They will just sit and enjoy the flight. All the flights are monitored in the control room remotely for passenger safety.

This drone autonomous taxi can carry a weight of 110 kg, and it uses eight motors to fly at a speed of up to 70 kilometers/hour.

Aerobatics autonomous helicopter

Computer scientists at Stanford have successfully created an AI system that can enable robotic helicopters to learn and perform difficult stunts watching other helicopters performing the same maneuvers. This has resulted in autonomous helicopters which can perform a complete airshow of tricks on its own. Controlling the autonomous helicopter flight is the most challenging problem.

Autonomous helicopter flight is widely regarded to be a highly challenging control problem. Despite this fact, human experts can reliably fly helicopters through a wide range of maneuvers, including aerobatic maneuvers.

How does it work? By using reinforcement learning for the optimal control problem, it optimizes the model and reward functions.

All these reinforcement learning algorithms discussed in detail with practical implementation in the book title “Practical Reinforcement Learning”.

TD-Gammon – computer game

TD-Gammon is a widely played computer backgammon program developed in 1992. TDGammon is a neural network which teaches itself to play backgammon and improves its strategies by playing the game with itself and learns from the results. It is a good example of reinforcement learning algorithm. It begins with random initial weights (and hence a random initial strategy), TD-Gammon eventually develops a strong level of play. While raw description of the board state is given, but with zero information built-in, the system teaches itself and develops strong ability to play at intermediate level. Moreover, with additional hand-crafted features the systems performs stunningly well.

A backgammon game

The current version of TD-Gammon is very close to the level of the best human player of all time. It explored a lot of strategies that humans had never used, and that is the reason for the advancement in current TD-backgammon play.

AlphaGo

The game of Go originated in China more than 3,000 years ago. The rules of the game are simple. Players take turns to place white or black stones on a board, trying to capture the opponent's stones or surround empty space to make points out of territory. As simple as the rules are, Go is a game of profound complexity. There are more possible positions in Gothan there are atoms in the universe. That makes Go more complex than chess.

The game of Go is a classic and very challenging game. Computer scientists have been trying for decades to at least achieve a beginner level of performance with a computer as compared to a human. Now, with advancements in deep reinforcement learning, the computer learns a network policy (which selects actions) and also a network value (which predicts the winner) through self-play.

AlphaGo uses a state-of-the-art tree search and deep neural network techniques. It is the first program that beat a professional human player in Oct 2016. Later on, AlphaGo also defeated Lee Sedol, one of the strongest players with 17 world titles. The final score of the game was 4 to 1; this match was seen by 200 million viewers.

Summary

I mentioned only few applications of reinforcement learning. Actually reinforcement learning is now applying in almost all the industries and there are lots of use cases of it.