An Efficient and Rapidly Adaptable Lightweight Multi-Destination Urban Path Planning Approach for UAVs using Q-Learning

Michael R. Jones; Soufiene Djahel; Kristopher Welsh

doi:10.1109/TIV.2024.3387018

An Efficient and Rapidly Adaptable Lightweight Multi-Destination Urban Path Planning Approach for UAVs using Q-Learning

Michael R. Jones, Soufiene Djahel, Kristopher Welsh

Centre for Future Transport and Cities

Manchester Metropolitan University

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

28 Downloads (Pure)

Abstract

Advancement in UAV technologies have facilitated the development of lightweight airborne platforms capable of fulfilling a diverse range of tasks due to a varied array of mountable sensing and interaction modules available. To further advance UAVs and widen their application spectrum, providing them with fully autonomous operations capability is necessary. To address this challenge, we present Multiple Q-table Path Planning (MQTPP), a novel method specifically tailored for UAV path planning in urban environments. Unlike a conventional Qlearning approach that necessitates relearning in response to dynamic changes in urban landscapes or targets, MQTPP is designed to adaptively re-plan UAV paths with notable efficiency, utilising a singular learning phase executed prior to take-off. Results obtained through simulation demonstrate the exceptional capability of MQTPP to swiftly generate new paths or modify existing ones during flight. This performance significantly surpasses existing state-of-the-art methods in terms of computational efficiency, while still achieving near-optimal path planning results. Thus, demonstrating MQTPP’s potential as a robust solution for real-time, adaptive in-flight UAV navigation in complex urban settings.

Original language	English
Pages (from-to)	(In-Press)
Number of pages	12
Journal	IEEE Transactions on Intelligent Vehicles
Volume	(In-Press)
Early online date	10 Apr 2024
DOIs	https://doi.org/10.1109/TIV.2024.3387018
Publication status	E-pub ahead of print - 10 Apr 2024

Bibliographical note

© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders.

This document is the author’s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it.

Keywords

Autonomous aerial vehicles
Heuristic algorithms
Intelligent vehicles
Path planning
Planning
Q-learning
Vehicle dynamics
multiple Q-table
path planning
reinforcement learning
unmanned aerial vehicle (UAV)
urban environment

ASJC Scopus subject areas

Control and Optimization
Artificial Intelligence
Automotive Engineering

Access to Document

10.1109/TIV.2024.3387018Licence: CC BY

Djahel2024AAMAccepted author manuscript, 2.3 MBLicence: CC BY

Cite this

@article{681e214c7f304c0fb3647ad99751096c,

title = "An Efficient and Rapidly Adaptable Lightweight Multi-Destination Urban Path Planning Approach for UAVs using Q-Learning",

abstract = "Advancement in UAV technologies have facilitated the development of lightweight airborne platforms capable of fulfilling a diverse range of tasks due to a varied array of mountable sensing and interaction modules available. To further advance UAVs and widen their application spectrum, providing them with fully autonomous operations capability is necessary. To address this challenge, we present Multiple Q-table Path Planning (MQTPP), a novel method specifically tailored for UAV path planning in urban environments. Unlike a conventional Qlearning approach that necessitates relearning in response to dynamic changes in urban landscapes or targets, MQTPP is designed to adaptively re-plan UAV paths with notable efficiency, utilising a singular learning phase executed prior to take-off. Results obtained through simulation demonstrate the exceptional capability of MQTPP to swiftly generate new paths or modify existing ones during flight. This performance significantly surpasses existing state-of-the-art methods in terms of computational efficiency, while still achieving near-optimal path planning results. Thus, demonstrating MQTPP{\textquoteright}s potential as a robust solution for real-time, adaptive in-flight UAV navigation in complex urban settings.",

keywords = "Autonomous aerial vehicles, Heuristic algorithms, Intelligent vehicles, Path planning, Planning, Q-learning, Vehicle dynamics, multiple Q-table, path planning, reinforcement learning, unmanned aerial vehicle (UAV), urban environment",

author = "Jones, {Michael R.} and Soufiene Djahel and Kristopher Welsh",

note = "{\textcopyright} 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Copyright {\textcopyright} and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. This document is the author{\textquoteright}s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it. ",

year = "2024",

month = apr,

day = "10",

doi = "10.1109/TIV.2024.3387018",

language = "English",

volume = "(In-Press)",

pages = "(In--Press)",

journal = "IEEE Transactions on Intelligent Vehicles",

issn = "2379-8858",

publisher = "IEEE",

}

TY - JOUR

T1 - An Efficient and Rapidly Adaptable Lightweight Multi-Destination Urban Path Planning Approach for UAVs using Q-Learning

AU - Jones, Michael R.

AU - Djahel, Soufiene

AU - Welsh, Kristopher

N1 - © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. This document is the author’s post-print version, incorporating any revisions agreed during the peer-review process. Some differences between the published version and this version may remain and you are advised to consult the published version if you wish to cite from it.

PY - 2024/4/10

Y1 - 2024/4/10

N2 - Advancement in UAV technologies have facilitated the development of lightweight airborne platforms capable of fulfilling a diverse range of tasks due to a varied array of mountable sensing and interaction modules available. To further advance UAVs and widen their application spectrum, providing them with fully autonomous operations capability is necessary. To address this challenge, we present Multiple Q-table Path Planning (MQTPP), a novel method specifically tailored for UAV path planning in urban environments. Unlike a conventional Qlearning approach that necessitates relearning in response to dynamic changes in urban landscapes or targets, MQTPP is designed to adaptively re-plan UAV paths with notable efficiency, utilising a singular learning phase executed prior to take-off. Results obtained through simulation demonstrate the exceptional capability of MQTPP to swiftly generate new paths or modify existing ones during flight. This performance significantly surpasses existing state-of-the-art methods in terms of computational efficiency, while still achieving near-optimal path planning results. Thus, demonstrating MQTPP’s potential as a robust solution for real-time, adaptive in-flight UAV navigation in complex urban settings.

AB - Advancement in UAV technologies have facilitated the development of lightweight airborne platforms capable of fulfilling a diverse range of tasks due to a varied array of mountable sensing and interaction modules available. To further advance UAVs and widen their application spectrum, providing them with fully autonomous operations capability is necessary. To address this challenge, we present Multiple Q-table Path Planning (MQTPP), a novel method specifically tailored for UAV path planning in urban environments. Unlike a conventional Qlearning approach that necessitates relearning in response to dynamic changes in urban landscapes or targets, MQTPP is designed to adaptively re-plan UAV paths with notable efficiency, utilising a singular learning phase executed prior to take-off. Results obtained through simulation demonstrate the exceptional capability of MQTPP to swiftly generate new paths or modify existing ones during flight. This performance significantly surpasses existing state-of-the-art methods in terms of computational efficiency, while still achieving near-optimal path planning results. Thus, demonstrating MQTPP’s potential as a robust solution for real-time, adaptive in-flight UAV navigation in complex urban settings.

KW - Autonomous aerial vehicles

KW - Heuristic algorithms

KW - Intelligent vehicles

KW - Path planning

KW - Planning

KW - Q-learning

KW - Vehicle dynamics

KW - multiple Q-table

KW - path planning

KW - reinforcement learning

KW - unmanned aerial vehicle (UAV)

KW - urban environment

UR - https://ieeexplore.ieee.org/document/10496242

UR - http://www.scopus.com/inward/record.url?scp=85190169324&partnerID=8YFLogxK

U2 - 10.1109/TIV.2024.3387018

DO - 10.1109/TIV.2024.3387018

M3 - Article

SN - 2379-8858

VL - (In-Press)

SP - (In-Press)

JO - IEEE Transactions on Intelligent Vehicles

JF - IEEE Transactions on Intelligent Vehicles

ER -

An Efficient and Rapidly Adaptable Lightweight Multi-Destination Urban Path Planning Approach for UAVs using Q-Learning

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this