Model Based Reinforcement Learning - Addfor S.p.A.

Introduction

Reinforcement Learning (RL) [1] is a class of machine learning algorithms in which an agent interacts by trial-and-error in an environment.

RL together with Deep Learning has obtained excellent results in a great number of simulated environments, like video-games [2] or board-games [3]. This algorithms are very promising for industrial applications which involve optimal control.

The most Deep Reinforcement Learning (DRL) algorithms developed so far are model-free that is, they do not use a model of the environment during the training phase. Starting from the Dyna-Q tabular algorithm [1], the research community has tried to speed up RL training introducing a model of the environment. The model allows to reuse multiple times the experience acquired during the agent-environment interaction and it improves the agent’s actions with respect to the control objective. Recently proposed solutions merge the ideas of DRL with model-based RL [4, 5].

The objective of the thesis is to study the possibility to develop existing model-free DRL algorithms by incorporating a model into the training phase.

The model introduce a computational overhead that has to be quantified with respect to the training speed-up. Therefore, an extended comparison between model-based and model-free has to inquire the cost-benefit impact of the model itself.

Planned Activities

Acquire strong theoretical basis on Deep Reinforcement Learning;
Investigate the different possibilities to integrate a model into an existing model-free DRL algorithm;
Compare different pairs model-free and model-based algorithms finding the break-even value from the points of view of computational overhead and training speed-up.

Competencies to be acquired

The candidate will acquire:

Expertise on recent Deep Reinforcement Learning algorithms;
Experience in algorithm design, analysis and comparison with respect to a real application;
Experience of data-driven modelling in the field of physical systems.

Duration of this Project: 5-6 months.

Who we’re looking for

Students that are about to get their Master Degree in: computer science, computer engineering, mechatronic engineering, electronic engineering, aerospace engineering, mathematical engineering, mathematics, physics, physics of complex systems.

Required Skills:

Proficiency in at least one programming language (Python, Lua, Matlab, C++, Java);
Basic knowledge of machine learning, in particular supervised learning;
Good knowledge of linear algebra.
Basic knowledge of dynamical system modelling