Abstract: Interactive interfaces are a common feature of many systems ranging from field robotics to video games. In most applications, these interfaces must be used by a heterogeneous set of users, with substantial variety in effectiveness with the same interface when configured differently. We address the issue of personalizing such an interface, adapting parameters to present the user with an environment that is optimal with respect to their individual traits – enabling that particular user to achieve their personal optimum. We introduce anew class of problem in interface personalization where the task of the adaptive interface is to choose the subset of actions of the full interface to present to the user. In formalizing this problem, we model the user as a Markov decision process (MDP), wherein the transition dynamics within a task depends on the type (e.g., skill or dexterity) of the user, where the type parametrizes the MDP. The action set of the MDP is divided into disjoint set of actions, with different action-sets optimal for different type (transition dynamics). The task of the adaptive interface is then to choose the right action-set.Given this formalization, we present experiments with simulated and human users in a video game domain to show that (a) action set selection is an interesting class of problems(b) adaptively choosing the right action set improves performance over sticking to a fixed action set and (c) immediately applicable approaches such as bandits can be improved upon.

Citation: M.M.H. Mahmud, B. Rosman, S. Ramamoorthy, P. Kohli. Adapting interaction environments to diverse users through online action set selection. In Proc. AAAI Workshop on Machine Learning for Interactive Systems (AAAI-MLIS), 2014.

Download: http://bit.ly/2iKKXZb

Abstract: This paper considers the problem of providing advice to an autonomous agent when neither the behavioural policy nor the goals of that agent are known to the advisor. We present an approach based on building a model of “commonsense” behaviour in the domain, from an aggregation of different users performing various tasks, modeled as MDPs, in the same domain. From this model, we estimate the normalcy of the trajectory given by a new agent in the domain, and provide behavioural advice based on an approximation of the trade-off in utility between potential benefits to the exploring agent and the costs incurred in giving this advice. This model is evaluated on a maze world domain by providing advice to different typesof agents, and we show that this leads to a considerable and unanimous improvement in the completion rate of their tasks.

Citation: B. Rosman, S. Ramamoorthy (2014). Giving advice to agents with hidden goals. In Proc. IEEE International Conference on Robotics and Automation (ICRA), 2014.

Download: http://bit.ly/2iKvWqc