Abstract

Motion planning framed as optimisation in structured latent spaces has recently emerged as competitive with traditional methods in terms of planning success while significantly outperforming them in terms of computational speed. However, the real-world applicability of recent work in this domain remains limited by the need to express obstacle information directly in state-space, involving simple geometric primitives. In this work we address this challenge by leveraging learned scene embeddings together with a generative model of the robot manipulator to drive the optimisation process. In addition we introduce an approach for efficient collision checking which directly regularises the optimisation undertaken for planning. Using simulated as well as real-world experiments, we demonstrate that our approach, AMP-LS, is able to successfully plan in novel, complex scenes while outperforming competitive traditional baselines in terms of computation speed by an order of magnitude. We show that the resulting system is fast enough to enable closed-loop planning in real-world dynamic scenes.

Overview

We introduce a method remarkably extending Latent Space Path Planning (LSPP) and name it Activation Maximisation Planning in Latent Space (AMP-LS). Similar to LSPP, AMP-LS leverages a variational autoencoder (VAE) to learn a structured latent space to generate kinematically feasible joint trajectories. While a collision predictor in the prior work relies on state-based obstacle representations, our collision predictor leverages scene embeddings obtained from SceneCollisionNet to readily achieve zero-shot transfer to unseen environments. Further, we present an approach for explicit collision checking to directly regularise the optimisation to plan collision-free trajectories.

Experimental results

Open-loop planning for reaching static targets in simulation

AMP-LS (ours)

RRTConnect — Collision

LazyPRM*

CHOMP — Fails to plan

AMP-LS (ours)

RRTConnect

LazyPRM* — Collision

CHOMP — Fails to plan

AMP-LS (ours)

RRTConnect

LazyPRM* — Collision

CHOMP — Fails to plan

AMP-LS (ours)

RRTConnect

LazyPRM* — Collision

CHOMP — Fails to plan

Open-loop planning in a complex scene in real

Closed-loop planning for moving obstacles and target in simulation

Closed-loop planning for moving obstacles and target in real

Track the QR code

Maintain the same pose

Citation

@inproceedings{yamada2023leveraging,
  title={Leveraging scene embeddings for gradient-based motion Planning in latent space},
  author={Yamada, Jun and Hung, Chia-Man and Collins, Jack and Havoutis, Ioannis and Posner, Ingmar},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={},
  year={2023},
  organization={IEEE}
}

Acknowledgement

This work was supported by a UKRI/EPSRC Programme Grant [EP/V000748/1], we would also like to thank the University of Oxford for providing Advanced Research Computing (ARC) facility in carrying out this work (http://dx.doi.org/10.5281/zenodo.22558).