Umberto, Antti O., Antti K., Zeinab, Aki, Kashyap, Katya, Mert, Homayun, Tomi, Sami, Aurelién
- note: first longer updates and then short ones this time, because Summer School
Modeling Made Easy
Sub-optimal reinforcement learning
Pedram/Kashyap: Modeling Made Easy focuses on tradeoff between Interactive Modeling and Optimal Teaching
The assumption is that the user is an expert about the phenomena (data) but not an expert in statistical models (e.g. he knows about linear regression but he does not know how to selecting variables, relevances, ...).
Interactive Modeling the user is providing is knowledge as prior in order to build up the best model. On the other hand, in Optimal Teaching, the user is supposed to learn and to understand the problem and the modeling. In the latter case the user is learning through testes provided by the "teacher", that wants to make user the user understands.
suggestions: (Sami) display in an intuitive way (plots/p.values) the solutions proposed by the user and by the expert, in order to make for the user easier to understand the phenomena.
(Aki) for the future, including causal models?
open questions: how to define properly the model-likelihoods?
how to go forward?
Mert: Sub-optimal human-in-the-loop reinforcement learning
There are basically 2 types of situations: in the first one the AI is right and the user is wrong, in the second one the AI is wrong and the user is right. Here we focus on the first one, where the user thinks having an optimal policy while it is sub-optimal.
The analysis takes place by considering the optimal gap between the AI optimal solution and the user's sub optimal solution. Once a threshold epsilon is defined, the user might decide to intervene in order to change the AI's suggested path. Since the AI's solution is white and the user's is black, in order to understand the user's model a trigger is required. In this way it is possible to learn the user's sub-optimal solution.
suggestions: (Sami) refer to the "Horse metaphor" when discussing the intervene/non intervene policy by the user.
- Kashyap, Pedram and Katja will be giving a walkthrough on the planned UI next week