Invited Speakers

  • Lucian Busoniu, Technical University of Cluj-Napoca
  • Danica Kragic, Royal Institute of Technology, KTH
  • Matteo Leonetti, University of Texas, Austin
  • Jan Peters, Technischen Universität Darmstadt


Lucian Busoniu

Lucian Busoniu

Reinforcement learning and planning algorithms

Lucian Busoniu and Levente Tamas

Abstract: Many learning and planning methods for robot motion control are built on a foundation of optimal control in Markov decision processes. In this talk, we describe this basic problem and some fundamental reinforcement learning and planning methods to solve it. We start with dynamic programming algorithms, and then move on to model-free approaches in reinforcement learning. Special attention is paid to function approximation, which is essential in robotics. In the second part of the talk, we describe an online planning framework, identifying its relation to reinforcement learning and detailing a few recent optimistic planning algorithms. We connect to several robotics applications along the way.

Bio: Lucian Busoniu received his Ph.D. degree (cum laude) from the Delft University of Technology, the Netherlands, in 2009. He is an associate professor with the Department of Automation at the Technical University of Cluj-Napoca, and has previously held research positions in the Netherlands and France. His fundamental interests include planning-based methods for nonlinear optimal control, reinforcement learning and dynamic programming with function approximation, and multiagent systems; while his practical focus is applying these techniques to robotics. He coauthored an introductory book on approximate reinforcement learning and planning, and was the recipient of the 2009 Andrew P. Sage Award for the best paper in the IEEE Transactions on Systems, Man, and Cybernetics.

Danica Kragic

Danica Kragic

Learning Interpretable Representations for Planning

Johannes Stork, Carl Henrik Ek, Danica Kragic

Abstract: Planning is the process that facilitates connecting task,  the environment and the robot’s abilities. Central to this task is a abstraction allowing the robot to reason about its actions. As such planning is an integral part of most robotic system in order to yield meaningful behavior. The history of planning is tightly connected with the advent of autonomous robots and many different approaches have been designed in the past. With limited perception and computational resources planning is confined to reasoning in symbolic representation. The advent of significant mobile computational resources and a large data-sets statistical representations now allow us to plan in continuous and stochastic environments. However, in this context it is challenging to design representations for the reasoning mechanism. In this talk we will outline recent work within the robotics community of learning state space representations directly grounded in observable quantities of the environment. However, this creates new challenges in terms of planning as a representation in terms of observables does not naturally reflect the semantics of most tasks. We will describe recent work which allows for the task to be incorporated in the representation learning facilitating generalization and semantical interpretation making planning easier.

Bio: Danica Kragic is a Professor at the School of Computer Science and Communication at the Royal Institute of Technology, KTH. She received MSc in Mechanical Engineering from the Technical University of Rijeka, Croatia in 1995 and PhD in Computer Science from KTH in 2001. She has been a visiting researcher at Columbia University, Johns Hopkins University and INRIA Rennes. She is the Director of the Centre for Autonomous Systems. Danica received the 2007 IEEE Robotics and Automation Society Early Academic Career Award. She is a member of the Royal Swedish Academy of Sciences and Young Academy of Sweden. She holds a Honorary Doctorate from the Lappeenranta University of Technology. She chaired IEEE RAS Technical Committee on Computer and Robot Vision and served as an IEEE RAS AdCom member. Her research is in the area of robotics, computer vision and machine learning. In 2012, she received an ERC Starting Grant. Her research is supported by the EU, Knut and Alice Wallenberg Foundation, Swedish Foundation for Strategic Research and Swedish Research Council.


Matteo Leonetti

Reinforcement Learning, Layered Learning and Adaptive Planning for Autonomous Robots

Matteo Leonetti , Peter Stone

Abstract: A severe challenge for autonomous robots is that the physical world, and the people they interact with, can be very difficult to capture in their knowledge representation. Automated planning is based on assumptions that may be violated, and some of which inevitably are, regardless of the extent to which models are learned or revised. This may be due to the environment changing over time, or aspects inherently difficult to both model and learn, like people’s behavior. I will describe a framework for adapting plans online to the environment through learning, while at the same time constraining the exploration through planning. I will then draw from other research threads in the Learning Agents Research Group at UT Austin, and discuss some recent work in layered learning, a hierarchical machine learning paradigm that enables learning of complex behaviors by incrementally learning a series of sub-behaviors, and its use in the 3D RoboCup Soccer Simulation League.

Jan Peters

Jan Peters