Introduction
Reinforcement Learning (RL) [1] is a class of machine learning algorithms in which an agent interacts by trial-and-error in an environment.
RL together with Deep Learning has obtained excellent results in a great number of simulated environments, like video-games [2] or board-games [3]. This algorithms are very promising for industrial applications which involve optimal control.
The most Deep Reinforcement Learning (DRL) algorithms developed so far are model-free that is, they do not use a model of the environment during the training phase. Starting from the Dyna-Q tabular algorithm [1], the research community has tried to speed up RL training introducing a model of the environment. The model allows to reuse multiple times the experience acquired during the agent-environment interaction and it improves the agent’s actions with respect to the control objective. Recently proposed solutions merge the ideas of DRL with model-based RL [4, 5].
The objective of the thesis is to study the possibility to develop existing model-free DRL algorithms by incorporating a model into the training phase.
The model introduce a computational overhead that has to be quantified with respect to the training speed-up. Therefore, an extended comparison between model-based and model-free has to inquire the cost-benefit impact of the model itself.

Planned Activities
- Acquire strong theoretical basis on Deep Reinforcement Learning;
- Investigate the different possibilities to integrate a model into an existing model-free DRL algorithm;
- Compare different pairs model-free and model-based algorithms finding the break-even value from the points of view of computational overhead and training speed-up.
Competencies to be acquired
The candidate will acquire:
- Expertise on recent Deep Reinforcement Learning algorithms;
- Experience in algorithm design, analysis and comparison with respect to a real application;
- Experience of data-driven modelling in the field of physical systems.
Duration of this Project: 5-6 months.
Who we’re looking for
Students that are about to get their Master Degree in: computer science, computer engineering, mechatronic engineering, electronic engineering, aerospace engineering, mathematical engineering, mathematics, physics, physics of complex systems.
Required Skills:
- Proficiency in at least one programming language (Python, Lua, Matlab, C++, Java);
- Basic knowledge of machine learning, in particular supervised learning;
- Good knowledge of linear algebra.
- Basic knowledge of dynamical system modelling
Check these links before moving on
[1]Reinforcement Learning: An Introduction
http://incompleteideas.net/book/the-book.html
[2]Deep-Q-Network
https://www.nature.com/articles/nature14236
[3] AlphaGo
https://deepmind.com/research/alphago/
[4] Imagination-Augmented Agents for Deep Reinforcement Learning
https://deepmind.com/research/publications/imagination-augmented-agents-deep-reinforcement-learning/
[5] Simulated Policy Learning in Video Models
https://ai.googleblog.com/2019/03/simulated-policy-learning-in-video.html?m=1
Contact Us
Directly by email to: [email protected]
By LinkedIn: linkedin.com/in/cannavò-sonia-66a95467