Catalogue of Artificial Intelligence Techniques
Keywords: reinforcement learning
Author(s): James Gresham
Reinforcement learning is a sub-area of machine learning, concerned with maximizing long term rewards (and in some cases reduce punishments). Reinforcement learning algorithms are designed to find rules to map states to the correct action that ought to be taken from that state. It should not be necessary to specify how the programming agents work to achieve the task.
Unlike many forms of machine learning, the learner is not told which actions to take. Instead, the learner uses a trial and error system to find which action provides the greatest long term reward. Reinforcement learning is thought of as a class of problems rather than a set of techniques. Also, in many of the more complex examples of reinforcement learning, the actions of the learner can affect the result of future actions as well. Therefore the concept of future reward is an important one in reinforcement learning.
Obviously, in order to use trial and error results, a reinforcement learning agent must be able to sense features of the environment on which it bases it's decisions, and be able to take actions that affect the environment. Also, the agent must have some goal, generally a variable that it must maximize.
Key features of an effective reinforcement learning agent include a good balance between exploitation (taking advantage of previous actions that have proved to be successful) and exploration (searching untested actions to see if they provide a greater reward). Without a good balance between the two, the agent will either spend all it's time trying every pointless action, or get stuck in a single set of actions which lead to a smaller reward.
Reinforcement learning is more useful than supervised learning in some interactive problems. Since supervised learning is dependent on an external supervisor to provide examples upon which to make it's judgment, it fails in areas of uncharted territory. Reinforcement learning, being able to learn from it's own experience, is useful in these areas.
In addition, reinforcement learning explicitly studies a complete problem, unlike many other machine learning approaches, which look at subproblems without thought as to how they affect the entire problem to be solved. Reinforcement learning does require subproblems to be isolated and solved, but these subproblems must be clearly motivated to further the overall goal to be achieved by the agent involved, even if the agent is incomplete.
Reinforcement learning is particularly useful in dealing with situations where there are choices between long term and short term rewards. Major uses of reinforcement learning include robotics and control, and intelligent game playing.
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning:, MIT Press, Cambridge, MA, 1998.
- Leslie Pack Kaelbling and Michael L. Littman, Reinforcement Learning: A Survey.