Modelling uncertainty in reinforcement learning

This paper discusses the model problem presented in “A model for system uncertainty in reinforcement learning”, Systems and Control Letters, 2018, for certain tasks in reinforcement learning. The model provides a framework to deal with situations in which the system dynamics is not known and encodes the available information about the state dynamics as a measure on the space of functions. Such a measure is updated in time, taking into account all the previous measurements of the state variable and extracting new information from them. Here we will mainly focus on the differences between the present model and central algorithms used in reinforcement learning (i.e. value iteration and Thompson sampling).