The Department of Advanced Robotics at the Italian Institute of Technology (an English-language research institute) is seeking to appoint a well-motivated full-time postdoctoral researcher [continue reading...]]]> I have an open postdoctoral position in my team, in the field of Machine Learning for Robotics. The details are listed below. For further information please contact me by e-mail.

The Department of Advanced Robotics at the Italian Institute of Technology (an English-language research institute) is seeking to appoint a well-motivated full-time postdoctoral researcher in the area of machine learning applied to robotics in general, and in particular to Autonomous Underwater Vehicles (AUV).

The successful candidate will join an ongoing research project funded by the European Commission under FP7 in the category Cognitive Systems and Robotics called “PANDORA” (Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning) which started in January 2012. The project is a collaboration of five leading universities and institutes in Europe: Heriot Watt University (UK), Italian Institute of Technology (Italy), University of Girona (Spain), King’s College London (UK), and National Technical University of Athens (Greece). Details about the project can be found at: http://persistentautonomy.com/

The accepted candidate will contribute to the development and experimental validation of novel reinforcement learning and imitation learning algorithms for robot control, as well as their specific application to autonomous underwater vehicles. The research will be conducted at the Department of Advanced Robotics within the “Learning and Interaction Group” with project leader Dr. Petar Kormushev.

The research work will include conducting experiments with two different AUVs (Girona 500 and Nessie V) in water tanks in Spain and UK in collaboration with the other project partners. The developed machine learning algorithms can also be applied to other robots available at IIT, such as the compliant humanoid robot COMAN, the hydraulic quadruped robot HyQ, the humanoid robot iCub, two Barrett WAM manipulator arms, and a KUKA LWR arm robot.

Application Requirements:

• PhD degree in Computer Science, Mathematics or Engineering
• Excellent publication record
• Strong competencies in: machine learning, reinforcement learning, imitation learning
• Good programming skills, preferably in MATLAB and C/C++
• Experience in robot control and ROS is a plus

International applications are encouraged.  The successful candidate will be offered a fixed-term project collaboration contract for the remaining duration of the project due to end in December 2014 with a highly-competitive salary which will be commensurate with qualifications and experience. Expected starting date is as soon as possible, preferably before September 1^st, 2013.

Application Procedure:

To apply please send a detailed CV, a list of publications, a statement of research interests and plans, degree certificates, grade of transcripts, the names of at least two referees, and other supporting materials such as reference letters to: Dr. Petar Kormushev (petar.kormusheviit.it), quoting [PANDORA-PostDoc] in the email subject. For consideration, please apply by June 21^th, 2013.

Doctoral Course on “Robotics, Cognition and Interaction Technologies” Call for PhD students for 2013

PhD positions with scholarships are available at the Italian Institute of Technology (IIT) in Genoa, Italy. Doctoral course starting in January 2013 Application deadline: September 21, 2012 Online application here Please note that IIT is an English-language research [continue reading...]]]>  

Doctoral Course on “Robotics, Cognition and Interaction Technologies”
Call for PhD students for 2013

PhD positions with scholarships are available at the Italian Institute of Technology (IIT) in Genoa, Italy.
Doctoral course starting in January 2013
Application deadline: September 21, 2012
Online application here

Please note that IIT is an English-language research institute, so it is not required to speak Italian.

I have one PhD opening in my team, in the field of Reinforcement Learning with application to Robot Control. The details can be found in Annex A4 – Doctoral course on “Robotics, Cognition and Interaction Technologies”, and are as follows.

[Section 3. Department of ADVANCED ROBOTICS – PROF. DARWIN CALDWELL]

STREAM 1: Machine Learning, Robot Control and Human-Robot Interaction

Theme 3.1: Developmental robotics and robot learning for agile locomotion of compliant humanoid robots
Tutor: Dr. Petar Kormushev, Dr Nikos Tsagarakis
 
Developmental robotics offers a completely different approach for controlling humanoid robots than the currently predominant approach based on manually engineered controllers. For example, currently, the majority of bipedal walking robots use variants of ZMP-based walking with largely simplified models of the robot dynamics. As a result, despite the significant mechatronic advances in humanoid robot legs, the locomotion repertoire of current bipedal robots merely includes slow walking on flat ground or inclined slopes, and primitive forms of disturbance rejection. This is far behind from even a two-year old child.

The creation of novel, high-performance, passively-compliant humanoid robots (such as the robot COMAN developed at IIT) offers a significant potential for achieving more agile locomotion. However, the bottleneck is not the hardware anymore, but the software that controls the robot. It is no longer reasonable to use over-simplified models of robot dynamics, because the novel compliant robots possess much richer and more complex dynamics than the previous generation of stiff robots. Therefore, a new solution should be sought to address the challenge of compliant humanoid robot control.

In this PhD theme, the use of developmental robotics and robot learning methods will be explored, in order to achieve novel ways for whole-body compliant humanoid robot control. In particular, the focus will be on achieving agile locomotion, based on robot self-learned dynamics, rather than on pre-engineered dynamics model. The PhD candidates will be expected to develop new algorithms for robot learning and to advance the state-of-the-art in developmental robotics.

The expected outcome of these efforts includes the realization of highly dynamic bipedal locomotion such as omni-directional walking on uneven surfaces, jumping and running robustly on uneven terrain and in presence of high uncertainties, demonstrating robustness and tolerance to external disturbances, etc. The ultimate goal will be achieving locomotion skills comparable to a 1.5 – 2 year-old child.

Requirements: This is a multidisciplinary theme where the successful candidates should have strong competencies in machine learning and artificial intelligence, and good knowledge of robot kinematics and dynamics. The candidates should have top-class degree and a background in Computer Science, Engineering, or Mathematics. Required technical skills: C/C++ and/or MATLAB. Knowledge of computer vision is a plus.

For further details about this particular PhD position, please contact me by e-mail.

In February of 2012 the first Global Future 2045 Congress was held in Moscow. There, over 50 world leading scientists from multiple disciplines met to develop a strategy for the future development of humankind. One of the main goals of the Congress was to [continue reading...]]]> Global Future 2045 – A New Era for Humanity

In February of 2012 the first Global Future 2045 Congress was held in Moscow. There, over 50 world leading scientists from multiple disciplines met to develop a strategy for the future development of humankind. One of the main goals of the Congress was to construct a global network of scientists to further research on the development of cybernetic technology, with the ultimate goal of transferring a human’s individual consciousness to an artificial carrier.

The successful candidate will participate in a 3-year research project funded by the European Commission [continue reading...]]]> The Department of Advanced Robotics at the Italian Institute of Technology (an English-language research institute), has a Post Doc opening in the research areas of Reinforcement learning and Imitation learning applied to robot control of Autonomous Underwater Vehicles (AUV).

The successful candidate will participate in a 3-year research project funded by the European Commission under the Seventh Framework Programme (FP7-ICT, STREP, Cognitive Systems and Robotics) called “PANDORA” (Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning) which started in January 2012 (http://persistentautonomy.com/).

The project is a collaboration of five universities and institutes in Europe: Heriot Watt University (UK), Italian Institute of Technology (Italy), University of Girona (Spain), King’s College London (UK), and National Technical University of Athens (Greece).

The accepted candidate will contribute to the development and experimental validation of novel reinforcement learning and imitation learning algorithms for specific application to robot control of autonomous underwater vehicles.

The research work includes conducting experiment with AUVs in water tanks in collaboration with the other project partners. The developed machine learning algorithms will also be applied for other robots available at IIT, such as the compliant humanoid robot COMAN, the humanoid robot iCub, Barrett WAM manipulator arm, and KUKA LWR arm robot.

The research will be conducted within the “Learning and Interaction Group” (http://www.iit.it/en/advr-labs/learning-and-interaction.html) with team leaders: Dr. Petar Kormushev and Dr. Sylvain Calinon.

The salary will depend on the candidate’s experience. Policies provide additional pension and health benefits. Applicants may also qualify for reduced taxes benefits. Contracts will be for the duration of the project. Expected starting date is as soon as possible.

International applications are encouraged and will receive logistic support with visa issues. For further information please contact: Dr. Petar Kormushev (petar.kormushev AT iit.it).

APPLICATION DEADLINE: June 29, 2012

The official job posting on the IIT website can be found here:
http://www.iit.it/en/openings/research-departments/advanced-robotics-openings/1406-postdoctoral-position-in-machine-learning-for-robot-control-of-autonomous-underwater-vehicles.html

Currently, the academics are set to boycott the Elsevier’s business practices, as explained in this Statement of Purpose.

You can take a stand by signing up [continue reading...]]]> The Cost of Knowledge is a movement started by mathematicians and other academics who are protesting against the business model of the big publishers like Elsevier, Springer, Wiley, and etc.

Currently, the academics are set to boycott the Elsevier’s business practices, as explained in this Statement of Purpose.

You can take a stand by signing up here: http://thecostofknowledge.com/

Workshop Topic: Advances in Robot Learning and Human-Robot Interaction

Location: Varna, Bulgaria

Date: September 12, 2012

Venue: AIMSA 2012 conference on Artificial Intelligence

URL: http://kormushev.com/AIMSA-2012/

The invited speaker will be Prof. Alexander [continue reading...]]]> I am co-organizing a workshop at AIMSA 2012 on behalf of the IEEE RAS Technical Committee on Robot Learning:

Workshop Topic: Advances in Robot Learning and Human-Robot Interaction

Location: Varna, Bulgaria

Date: September 12, 2012

Venue: AIMSA 2012 conference on Artificial Intelligence

URL: http://kormushev.com/AIMSA-2012/

The invited speaker will be Prof. Alexander Stoytchev, from Iowa State University, USA. All accepted papers will be published in a special issue of a journal (for details see the website of the workshop). The workshop location is really nice, it is the biggest and best sea resort in Bulgaria, with magnificent sand and pleasant weather.

http://www.thezeitgeistmovement.com/

http://www.thezeitgeistmovement.com/

…

Of course, there are a few jerks out there, as Marc Raibert puts it, who can write perfect manuscripts on the first try, but if you’re reading this, I assume you are not one of these disgusting individuals.

So, for the rest of [continue reading...]]]> Good papers are like good wine: they need time to mature.

Of course, there are a few jerks out there, as Marc Raibert puts it, who can write perfect manuscripts on the first try, but if you’re reading this, I assume you are not one of these disgusting individuals.

So, for the rest of us mortals, I have tried to collect advice from various sources about how to write good scientific papers. Also, I contribute some of my own humble personal experience.

One of my most favourite papers on this topic is, without doubt, Marc Raibert’s paper about “Spilling the beans”. If you haven’t read it yet, please do so!
I totally agree with Raibert, and always try to “spill the beans” in my own papers as much and as early as possible.

Another classic in the genre is Jim Kajiya’s article “How to Get Your SIGGRAPH Paper Rejected”.

The “Cargo Cult Science”, as named by Richard Feynman, is a must-see for all researchers, in my opinion. If you don’t know what I’m talking about, I recommend watching Feynman’s commencement address at Caltech at my Inspiration page.

The Department of Advanced Robotics (http://www.iit.it/en/advanced-robotics) at the Italian Institute of Technology (IIT, an English-language research institute, located in Genoa, Italy) has 2 Post-Doc openings (starting from January March 2012) in the research areas of Reinforcement learning and Imitation learning applied [continue reading...]]]> Postdoctoral positions in Machine Learning for Robot Control of Autonomous Underwater Vehicles (AUV)

The Department of Advanced Robotics (http://www.iit.it/en/advanced-robotics) at the Italian Institute of Technology (IIT, an English-language research institute, located in Genoa, Italy) has 2 Post-Doc openings (starting from January March 2012) in the research areas of Reinforcement learning and Imitation learning applied to robot control of Autonomous Underwater Vehicles (AUV).

The successful candidates will participate in a 3-year research project funded by the European Commission under the Seventh Framework Programme (FP7-ICT, STREP, Cognitive Systems and Robotics) called “PANDORA” (Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning) which will start in January 2012.

The project is a collaboration of five universities and institutes in Europe: Heriot Watt University (UK), Italian Institute of Technology (Italy), University of Girona (Spain), University of Strathclyde (UK), and National Technical University of Athens (Greece).

The accepted candidates will contribute to the development and experimental validation of novel reinforcement learning and imitation learning algorithms for specific application to robot control of autonomous underwater vehicles. The research work includes conducting experiment with AUVs in water tanks in collaboration with the other project partners. The developed machine learning algorithms will also be applied for other robots available at IIT, such as the compliant humanoid robot COMAN, the humanoid robot iCub, Barrett WAM manipulator arm, and KUKA LWR arm robot.

The research will be conducted within the “Learning and Interaction Group” (http://www.iit.it/en/advanced-robotics/research-areas/learning-and-interaction)
with team leaders: Dr. Petar Kormushev and Dr. Sylvain Calinon.

The salary will depend on the candidate’s experience and also includes additional pension and health benefits. Applicants may also qualify for reduced taxes benefits. Contracts are for up to 3 years with a possible renewal and future career options upon successful completion. Expected start date is February March 2012.

International applications are encouraged and will receive logistic support with visa issues. For further information please contact: Dr. Petar Kormushev (by e-mail).

Application Requirements:

- PhD degree in Computer Science, Mathematics or Engineering
- High-quality publication record
- Strong interest in machine learning algorithms
- Strong competencies in some of these areas: machine learning, reinforcement learning, imitation learning, MATLAB and C/C++ programming
- Experience in robot control is a plus
- Fluency in both spoken and written English

Application Procedure:

To apply please send a detailed CV, a statement of motivation, degree certificates, grade of transcripts, contact information of at least two references, and other support materials such as reference letters to: Dr. Petar Kormushev (by e-mail).

For consideration, please apply by:  December 4, 2011
DEADLINE EXTENDED TO: January 29, 2012

–

Petar Kormushev, PhD
Team Leader – Advanced Robotics Dept.
Italian Institute of Technology (IIT)
Via Morego 30, 16163 Genova

http://www.iit.it/en/advanced-robotics/people/advanced-robotics/team-leader/petar-kormushev

“I hear, I forget; I see, I may remember; I do, I will never forget.” ― Confucius

“Be prapared: ‘luck’ is where preparation meets opportunity” ― Randy Pausch

“The only way of discovering the limits of the possible is to venture a little way past – into [continue reading...]]]> “Stay hungry, stay foolish” ― Steve Jobs

“I hear, I forget; I see, I may remember; I do, I will never forget.” ― Confucius

“Be prapared: ‘luck’ is where preparation meets opportunity” ― Randy Pausch

“The only way of discovering the limits of the possible is to venture a little way past – into the impossible”.

“Life is not measured by the number of breaths we take, but by the number of moments that take our breath away”

“The best way to predict the future is to invent it.” ― Alan Kay

“Don’t only practice your art, but force your way into its secrets, for it and knowledge can raise men to the divine.” ― Ludwig van Beethoven

“We must become the change we want to see in the world.” ― Mahatma Gandhi

…to be continued

The application deadline for the positions below has passed already!   But IIT regularly announces other open positions in several fields of science and has permanent ongoing calls here: http://www.iit.it/en/resources/calls.html

Doctoral Course on Robotics, Cognition and Interaction Technologies Call for PhD students for 2012

30 open PhD positions with scholarship [continue reading...]]]>  

The application deadline for the positions below has passed already!
 
But IIT regularly announces other open positions in several fields of science and has permanent ongoing calls here:
http://www.iit.it/en/resources/calls.html

Doctoral Course on Robotics, Cognition and Interaction Technologies
Call for PhD students for 2012

30 open PhD positions with scholarship are available at the Italian Institute of Technology (IIT) in Genoa, Italy.
Doctoral course starting in January 2012
Application deadline: September 23, 2011
Online application here

I have 2 PhD openings in my group, in the field of Reinforcement Learning with application to Robot Control. You can find details in this PDF document.
The 2 PhD positions under my supervision are in:

STREAM 3: Machine Learning, Robot Control and Human- Robot Interaction

• Theme 2.7: Machine learning for robot control of autonomous underwater vehicles
  Tutor: Dr. Petar Kormushev, Dr. Sylvain Calinon, Prof. Darwin G. Caldwell
  Number of available positions: 1

• Theme 2.8: Machine learning for a soft robotic arm assisting in minimally invasive surgery
  Tutor: Dr. Petar Kormushev, Dr. Sylvain Calinon, Prof. Darwin G. Caldwell
  Number of available positions: 1

For further details about these particular PhD positions, please contact me by e-mail.

The event was the so-called “Robotic Challenge @ AAAI”, and this year the topic was “Food preparation”.

Our robot is a modified Barrett WAM 7-dof robot arm manipulator, with a wooden rolling pin at [continue reading...]]]>
At the international conference AAAI 2011 in San Francisco, my colleague Sylvain and I presented our pizza-making robot.

The event was the so-called “Robotic Challenge @ AAAI”, and this year the topic was “Food preparation”.

Our robot is a modified Barrett WAM 7-dof robot arm manipulator, with a wooden rolling pin at the end-effector.

The robot learns from demonstrations how to roll out the pizza dough, in order to make the most perfect circular pizzas! Below you can see a video of the Robotic Challenge event, and here are a few photos of our setup:

Learning from Demonstration Robotics Challenge @ AAAI 2011
Video credit: Brandon Rohrer

Steve Jobs’s Commencement Speech in 2005 at Stanford University

Randy Pausch’s Last Lecture: Achieving Your Childhood Dreams

Jeff Hawkins’s TED Talk: Brain science is about to fundamentally change computing

Andrew Ng (Director of [continue reading...]]]> A collection of highly-motivational and inspirational materials (at least for me):

Steve Jobs’s Commencement Speech in 2005 at Stanford University

Randy Pausch’s Last Lecture: Achieving Your Childhood Dreams

Jeff Hawkins’s TED Talk: Brain science is about to fundamentally change computing

Andrew Ng (Director of Stanford Artificial Intelligence Lab) – The Future of Robotics and Artificial Intelligence

Richard Feynman’s commencement address given in 1974 at Caltech – “Cargo Cult Science”

Bruno Bozzetto – Freedom must always be conquered (Celebration of 60 years of freedom, by Council of Bergamo)

If you would like me to give a presentation at your institution, please invite me! You can download a tentative abstract of my invited talk here.

Endowing robots [continue reading...]]]> I give invited talks occasionally, to present my latest research in machine learning and its application for robot control.

If you would like me to give a presentation at your institution, please invite me!
You can download a tentative abstract of my invited talk here.

Machine Learning Algorithms
for Robot Control

Abstract

Endowing robots with human-like abilities to perform motor skills in a smooth and natural way will greatly promote the use of robots in everyday life. However, acquiring new motor skills is not simple and involves various forms of learning. The efficiency of the process lies in the interconnections between imitation and self-improvement strategies.
Similarly to humans, a robot should ideally be able to learn new skills by employing such mechanisms. Some tasks can be successfully transferred to the robot using only imitation strategies. Other tasks can be learned very efficiently by the robot alone using reinforcement learning.
The recent development of compliant robots progressively moves their operational domain from industrial applications to home and office uses, where the role and tasks can not be determined in advance. While some tasks allow the user to interact with the robot to teach it new skills, it is generally preferable to provide a mechanism that permits the robot to learn to improve and extend its skills to new
contexts under its own guidance.
I will present my efforts in developing machine learning algorithms which allow robots to learn faster and more successfully real-world skills.

Biography

Petar Kormushev is a Team Leader (equivalent to Assistant Professor) at the Advanced Robotics department of the Italian Institute of Technology (IIT). His research interests include robotics and machine learning,
especially reinforcement learning for intelligent robot behavior. In 2009 he obtained a PhD degree in Computational Intelligence from Tokyo Institute of Technology. He also holds a MSc degree in
Artificial Intelligence, a MSc degree in Bio- and Medical Informatics, and a BSc degree in Computer Science. He has participated in the INFRAWEBS project for designing the future Semantic Web, as well as the Japanese NEDO project for developing next-generation robots. He received the first “John Atanasoff” scholarship by the Eureka foundation, and a 4-year Japanese research fellowship.

Reinforcement Learning is different than supervised learning and unsupervised learning. It can be considered as “semi-supervised” learning, because of the information given by the reward function.

The [continue reading...]]]> Reinforcement Learning is a type of Machine Learning approach in which the learning algorithm discovers by itself how to reach a given goal by a trial-and-error process.

Reinforcement Learning is different than supervised learning and unsupervised learning. It can be considered as “semi-supervised” learning, because of the information given by the reward function.

The reward function is the way in which the goal is specified…  to be continued.

COMAN stands for “COmpliant huMANoid”, because this robot is designed with passive compliance (via springs) in his joints. This allows it to be more robust to [continue reading...]]]>
The COMAN robot is a compliant humanoid robot which is currently under development by the Advanced Robotics dept. of the Italian Institute of Technology in Genoa, Italy.

COMAN stands for “COmpliant huMANoid”, because this robot is designed with passive compliance (via springs) in his joints. This allows it to be more robust to environment perturbations (e.g. walking on uneven ground), to be safer for human-robot interaction (soft to touch), to be more energy-efficient, and to perform more dynamic motions (e.g. jumping, running).

COMAN can also be interpreted as Co-Man, meaning a co-worker, a robot which is a partner to humans, designed for safe physical human-robot interaction. The robot’s design is derived from the compliant joint design of the cCub bipedal robot.

This is a close-up of the passively-compliant legs of the robot:

Below is a video of the COMAN walking experiment I did together with Barkan Ugurlu and Nikos Tsagarakis. The goal was to learn to minimize the energy consumption used for walking by COMAN. This video accompanies my IROS 2011 paper presented in San Francisco, in September 2011.

We present a learning-based approach for minimizing the electric energy consumption during walking of a passively-compliant bipedal robot. The energy consumption is reduced by learning a varying-height center-of-mass trajectory which uses efficiently the robot’s passive compliance. To do this, we propose a reinforcement learning method which evolves the policy parameterization dynamically during the learning process and thus manages to find better policies faster than by using fixed parameterization. The method is first tested on a function approximation task, and then applied to the humanoid robot COMAN where it achieves significant energy reduction.

Link to publication:
Kormushev, P., Ugurlu, B., Calinon, S., Tsagarakis, N., and Caldwell, D.G., “Bipedal Walking Energy Minimization by Reinforcement Learning with Evolving Policy Parameterization“, IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS-2011), San Francisco, 2011. [pdf] [bibtex]

This is the latest prototype of COMAN, as of July 2012:

This is work done by my former colleague Stephen Hart: Dexter robot [continue reading...]]]> In recent years, there have been some amazing demonstrations of successful learning robots, which master some difficult motor skills.
Here I have collected some of the most impressive ones, which I consider being major milestones at the time they were done:

This is work done by my former colleague Stephen Hart: Dexter robot learning to reach

Work by James Kuffner in CMU:

This is work done by my friend and colleague Sylvain Calinon:

Work by Pieter Abbeel: