Model Based Reinforcement Learning

Reinforcement Learning (RL) [1] is a class of machine learning algorithms in which an agent interacts by trial-and-error in an environment.

RL together with Deep Learning has obtained excellent results in a great number of simulated environments, like video-games [2] or board-games [3]. This algorithms are very promising for industrial applications which involve optimal control.

The most Deep Reinforcement Learning (DRL) algorithms developed so far are model-free that is, they do not use a model of the environment during the training phase. Starting from the Dyna-Q tabular algorithm [1], the research community has tried to speed up RL training introducing a model of the environment. The model allows to reuse multiple times the experience acquired during the agent-environment interaction and it improves the agent’s actions with respect to the control objective. Recently proposed solutions merge the ideas of DRL with model-based RL [4, 5].

The objective of the thesis is to study the possibility to develop existing model-free DRL algorithms by incorporating a model into the training phase.

The model introduce a computational overhead that has to be quantified with respect to the training speed-up. Therefore, an extended comparison between model-based and model-free has to inquire the cost-benefit impact of the model itself.