Human-robot contactless collaboration with mixed reality interface

Maram Khatib; Khaled Al Khudir; Alessandro  De Luca

doi:10.1016/j.rcim.2020.102030

Human-robot contactless collaboration with mixed reality interface

Maram Khatib, Khaled Al Khudir, Alessandro De Luca

Sapienza University of Rome

Research output: Contribution to journal › Article › peer-review

35 Citations (Scopus)

170 Downloads (Pure)

Abstract

A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.

Original language	English
Article number	102030
Number of pages	12
Journal	Robotics and Computer-Integrated Manufacturing
Volume	67
Early online date	5 Aug 2020
DOIs	https://doi.org/10.1016/j.rcim.2020.102030
Publication status	Published - Feb 2021
Externally published	Yes

Bibliographical note

NOTICE: this is the author’s version of a work that was accepted for publication in Robotics and Computer-Integrated Manufacturing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Robotics and Computer-Integrated Manufacturing, 67, (2021)
DOI: 10.1016/j.rcim.2020.102030

© 2021, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

Keywords

Collision avoidance
Control system
Human-robot collaboration
Mixed reality
Redundancy control
Robotics

ASJC Scopus subject areas

Control and Systems Engineering
Software
General Mathematics
Computer Science Applications
Industrial and Manufacturing Engineering

Access to Document

10.1016/j.rcim.2020.102030

Post-PrintAccepted author manuscript, 5.5 MBLicence: CC BY-NC-ND

Cite this

@article{ceb0e0cc3c7344559a92ff28a1471f5f,

title = "Human-robot contactless collaboration with mixed reality interface",

abstract = "A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.",

keywords = "Collision avoidance, Control system, Human-robot collaboration, Mixed reality, Redundancy control, Robotics",

author = "Maram Khatib and {Al Khudir}, Khaled and {De Luca}, Alessandro",

note = "NOTICE: this is the author{\textquoteright}s version of a work that was accepted for publication in Robotics and Computer-Integrated Manufacturing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Robotics and Computer-Integrated Manufacturing, 67, (2021) DOI: 10.1016/j.rcim.2020.102030 {\textcopyright} 2021, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ ",

year = "2021",

month = feb,

doi = "10.1016/j.rcim.2020.102030",

language = "English",

volume = "67",

journal = "Robotics and Computer-Integrated Manufacturing",

issn = "0736-5845",

publisher = "Elsevier",

}

TY - JOUR

T1 - Human-robot contactless collaboration with mixed reality interface

AU - Khatib, Maram

AU - Al Khudir, Khaled

AU - De Luca, Alessandro

N1 - NOTICE: this is the author’s version of a work that was accepted for publication in Robotics and Computer-Integrated Manufacturing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Robotics and Computer-Integrated Manufacturing, 67, (2021) DOI: 10.1016/j.rcim.2020.102030 © 2021, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

PY - 2021/2

Y1 - 2021/2

N2 - A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.

AB - A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.

KW - Collision avoidance

KW - Control system

KW - Human-robot collaboration

KW - Mixed reality

KW - Redundancy control

KW - Robotics

UR - http://www.scopus.com/inward/record.url?scp=85088913156&partnerID=8YFLogxK

U2 - 10.1016/j.rcim.2020.102030

DO - 10.1016/j.rcim.2020.102030

M3 - Article

SN - 0736-5845

VL - 67

JO - Robotics and Computer-Integrated Manufacturing

JF - Robotics and Computer-Integrated Manufacturing

M1 - 102030

ER -

Human-robot contactless collaboration with mixed reality interface

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Flexible Automation and Intelligent Manufacturing International Conference

Task Priority Matrix at the acceleration level: Collision avoidance under relaxed constraints

Stable torque optimization for redundant robots using a short preview

Faster motion on Cartesian paths exploiting robot redundancy at the acceleration level

Cite this

Human-robot contactless collaboration with mixed reality interface

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Activities

Flexible Automation and Intelligent Manufacturing International Conference

Research output

Task Priority Matrix at the acceleration level: Collision avoidance under relaxed constraints

Stable torque optimization for redundant robots using a short preview

Faster motion on Cartesian paths exploiting robot redundancy at the acceleration level

Cite this