Towards a deep feature-action architecture for robot homing

Abdulrahman Altahhan

Research output: Chapter in Book/Report/Conference proceedingChapter

11 Downloads (Pure)

Abstract

This paper describes a model for robot navigation that uses an architecture similar to an actor-critic reinforcement learning architecture. Contrary to the abundance of models that use two neural networks one for the actor and one for the critic, this model sets up the actor as a layer seconded by another layer which deduce the value function. Therefore, the effect is to have similar to a critic outcome combined with the actor in one network. Hence, the model paves the way for a deep reinforcement learning architecture for future work The reward signal is back propagated through the critic then the actor. At the same time, the features layer have been deeply trained by applying a simple PCA on the whole set of images histograms acquired during the first running episode. The model is then able to shrink the whole architecture to fit a new reduced features dimension. Initial experimental result on real robot shows that the agent accomplished good level of accuracy and efficacy in reaching the goal.
Original languageEnglish
Title of host publication2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM)
PublisherIEEE
Pages205 - 209
ISBN (Print)978-1-4673-7337-1
DOIs
Publication statusPublished - 2015
Event2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) - Siem Reap, Cambodia
Duration: 15 Jul 201517 Jul 2015

Conference

Conference2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM)
CountryCambodia
CitySiem Reap
Period15/07/1517/07/15

Fingerprint

Robots
Reinforcement learning
Navigation
Neural networks

Bibliographical note

© 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Keywords

  • backpropagation
  • mobile robots
  • navigation
  • neurocontrollers
  • principal component analysis
  • PCA
  • actor-critic reinforcement learning architecture
  • deep feature-action architecture
  • image histogram
  • neural network
  • reward signal
  • robot homing
  • robot navigation
  • value function
  • Conferences
  • Decision support systems
  • Random access memory
  • World Wide Web

Cite this

Altahhan, A. (2015). Towards a deep feature-action architecture for robot homing. In 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (pp. 205 - 209). IEEE. https://doi.org/10.1109/ICCIS.2015.7274621

Towards a deep feature-action architecture for robot homing. / Altahhan, Abdulrahman.

2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM). IEEE, 2015. p. 205 - 209.

Research output: Chapter in Book/Report/Conference proceedingChapter

Altahhan, A 2015, Towards a deep feature-action architecture for robot homing. in 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM). IEEE, pp. 205 - 209, 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), Siem Reap, Cambodia, 15/07/15. https://doi.org/10.1109/ICCIS.2015.7274621
Altahhan A. Towards a deep feature-action architecture for robot homing. In 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM). IEEE. 2015. p. 205 - 209 https://doi.org/10.1109/ICCIS.2015.7274621
Altahhan, Abdulrahman. / Towards a deep feature-action architecture for robot homing. 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM). IEEE, 2015. pp. 205 - 209
@inbook{3ace5ec382dc4acba9ed3318b951a898,
title = "Towards a deep feature-action architecture for robot homing",
abstract = "This paper describes a model for robot navigation that uses an architecture similar to an actor-critic reinforcement learning architecture. Contrary to the abundance of models that use two neural networks one for the actor and one for the critic, this model sets up the actor as a layer seconded by another layer which deduce the value function. Therefore, the effect is to have similar to a critic outcome combined with the actor in one network. Hence, the model paves the way for a deep reinforcement learning architecture for future work The reward signal is back propagated through the critic then the actor. At the same time, the features layer have been deeply trained by applying a simple PCA on the whole set of images histograms acquired during the first running episode. The model is then able to shrink the whole architecture to fit a new reduced features dimension. Initial experimental result on real robot shows that the agent accomplished good level of accuracy and efficacy in reaching the goal.",
keywords = "backpropagation, mobile robots, navigation, neurocontrollers, principal component analysis, PCA, actor-critic reinforcement learning architecture, deep feature-action architecture, image histogram, neural network, reward signal, robot homing, robot navigation, value function, Conferences, Decision support systems, Random access memory, World Wide Web",
author = "Abdulrahman Altahhan",
note = "{\circledC} 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",
year = "2015",
doi = "10.1109/ICCIS.2015.7274621",
language = "English",
isbn = "978-1-4673-7337-1",
pages = "205 -- 209",
booktitle = "2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM)",
publisher = "IEEE",

}

TY - CHAP

T1 - Towards a deep feature-action architecture for robot homing

AU - Altahhan, Abdulrahman

N1 - © 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2015

Y1 - 2015

N2 - This paper describes a model for robot navigation that uses an architecture similar to an actor-critic reinforcement learning architecture. Contrary to the abundance of models that use two neural networks one for the actor and one for the critic, this model sets up the actor as a layer seconded by another layer which deduce the value function. Therefore, the effect is to have similar to a critic outcome combined with the actor in one network. Hence, the model paves the way for a deep reinforcement learning architecture for future work The reward signal is back propagated through the critic then the actor. At the same time, the features layer have been deeply trained by applying a simple PCA on the whole set of images histograms acquired during the first running episode. The model is then able to shrink the whole architecture to fit a new reduced features dimension. Initial experimental result on real robot shows that the agent accomplished good level of accuracy and efficacy in reaching the goal.

AB - This paper describes a model for robot navigation that uses an architecture similar to an actor-critic reinforcement learning architecture. Contrary to the abundance of models that use two neural networks one for the actor and one for the critic, this model sets up the actor as a layer seconded by another layer which deduce the value function. Therefore, the effect is to have similar to a critic outcome combined with the actor in one network. Hence, the model paves the way for a deep reinforcement learning architecture for future work The reward signal is back propagated through the critic then the actor. At the same time, the features layer have been deeply trained by applying a simple PCA on the whole set of images histograms acquired during the first running episode. The model is then able to shrink the whole architecture to fit a new reduced features dimension. Initial experimental result on real robot shows that the agent accomplished good level of accuracy and efficacy in reaching the goal.

KW - backpropagation

KW - mobile robots

KW - navigation

KW - neurocontrollers

KW - principal component analysis

KW - PCA

KW - actor-critic reinforcement learning architecture

KW - deep feature-action architecture

KW - image histogram

KW - neural network

KW - reward signal

KW - robot homing

KW - robot navigation

KW - value function

KW - Conferences

KW - Decision support systems

KW - Random access memory

KW - World Wide Web

U2 - 10.1109/ICCIS.2015.7274621

DO - 10.1109/ICCIS.2015.7274621

M3 - Chapter

SN - 978-1-4673-7337-1

SP - 205

EP - 209

BT - 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM)

PB - IEEE

ER -