-
HuMoT: Human Motion Representation using Topology-Agnostic Transformers for Character Animation Retargeting
Authors:
Lucas Mourot,
Ludovic Hoyet,
François Le Clerc,
Pierre Hellier
Abstract:
Motion retargeting is the long-standing problem in character animation that consists in transferring and adapting the motion of a source character to another target character. A typical application is the creation of motion sequences from off-the-shelf motions by transferring them onto new characters. Motion retargeting is also promising to increase interoperability of existing animation systems a…
▽ More
Motion retargeting is the long-standing problem in character animation that consists in transferring and adapting the motion of a source character to another target character. A typical application is the creation of motion sequences from off-the-shelf motions by transferring them onto new characters. Motion retargeting is also promising to increase interoperability of existing animation systems and motion databases, as they often differ in the structure of the skeleton(s) considered. Moreover, since the goal of motion retargeting is to abstract and transfer motion dynamics, effective solutions might provide expressive and powerful human motion models in which operations such as cleaning or editing are easier. In this article, we present a novel neural network architecture for retargeting that extracts an abstract representation of human motion agnostic to skeleton topology and morphology. Based on transformers, our model is able to encode and decode motion sequences with variable morphology and topology -- extending the current scope of retargeting -- while supporting skeleton topologies not seen during the training phase. More specifically, our model is structured as an autoencoder, and encoding and decoding are separately conditioned on skeleton templates to extract and control morphology and topology. Beyond motion retargeting, our model has many applications since our abstract representation is a convenient space to embed motion data from different sources. It may potentially be benefical to a number of data-driven methods, allowing them to combine scarce specialised motion datasets (e.g. with style or contact annotations) and larger general motion datasets, for improved performance and generalisation ability. Moreover, we show that our model can be useful for other applications beyond retargeting, including motion denoising and joint upsampling.
△ Less
Submitted 15 June, 2023; v1 submitted 30 May, 2023;
originally announced May 2023.
-
UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup
Authors:
Lucas Mourot,
Ludovic Hoyet,
François Le Clerc,
Pierre Hellier
Abstract:
Human motion synthesis and editing are essential to many applications like film post-production. However, they often introduce artefacts in motions, which can be detrimental to the perceived realism. In particular, footskating is a frequent and disturbing artefact requiring foot contacts knowledge to be cleaned up. Current approaches to obtain foot contact labels rely either on unreliable threshol…
▽ More
Human motion synthesis and editing are essential to many applications like film post-production. However, they often introduce artefacts in motions, which can be detrimental to the perceived realism. In particular, footskating is a frequent and disturbing artefact requiring foot contacts knowledge to be cleaned up. Current approaches to obtain foot contact labels rely either on unreliable threshold-based heuristics or on tedious manual annotation. In this article, we address foot contact label detection from motion with a deep learning. To this end, we first publicly release UnderPressure, a novel motion capture database labelled with pressure insoles data serving as reliable knowledge of foot contact with the ground. Then, we design and train a deep neural network to estimate ground reaction forces exerted on the feet from motion data and then derive accurate foot contact labels. The evaluation of our model shows that we significantly outperform heuristic approaches based on height and velocity thresholds and that our approach is much more robust on motion sequences suffering from perturbations like noise or footskate. We further propose a fully automatic workflow for footskate cleanup: foot contact labels are first derived from estimated ground reaction forces. Then, footskate is removed by solving foot constraints through an optimisation-based inverse kinematics (IK) approach that ensures consistency with the estimated ground reaction forces. Beyond footskate cleanup, both the database and the method we propose could help to improve many approaches based on foot contact labels or ground reaction forces, including inverse dynamics problems like motion reconstruction and learning of deep motion models in motion synthesis or character animation. Our implementation, pre-trained model as well as links to database can be found at https://github.com/InterDigitalInc/UnderPressure.
△ Less
Submitted 9 August, 2022;
originally announced August 2022.
-
A Survey on Deep Learning for Skeleton-Based Human Animation
Authors:
L. Mourot,
L. Hoyet,
F. Le Clerc,
François Schnitzler,
Pierre Hellier
Abstract:
Human character animation is often critical in entertainment content production, including video games, virtual reality or fiction films. To this end, deep neural networks drive most recent advances through deep learning and deep reinforcement learning. In this article, we propose a comprehensive survey on the state-of-the-art approaches based on either deep learning or deep reinforcement learning…
▽ More
Human character animation is often critical in entertainment content production, including video games, virtual reality or fiction films. To this end, deep neural networks drive most recent advances through deep learning and deep reinforcement learning. In this article, we propose a comprehensive survey on the state-of-the-art approaches based on either deep learning or deep reinforcement learning in skeleton-based human character animation. First, we introduce motion data representations, most common human motion datasets and how basic deep models can be enhanced to foster learning of spatial and temporal patterns in motion data. Second, we cover state-of-the-art approaches divided into three large families of applications in human animation pipelines: motion synthesis, character control and motion editing. Finally, we discuss the limitations of the current state-of-the-art methods based on deep learning and/or deep reinforcement learning in skeletal human character animation and possible directions of future research to alleviate current limitations and meet animators' needs.
△ Less
Submitted 23 November, 2021; v1 submitted 13 October, 2021;
originally announced October 2021.
-
A Perceptually-Validated Metric for Crowd Trajectory Quality Evaluation
Authors:
Beatriz Cabrero Daniel,
Ricardo Marques,
Ludovic Hoyet,
Julien Pettré,
Josep Blat
Abstract:
Simulating crowds requires controlling a very large number of trajectories and is usually performed using crowd motion algorithms for which appropriate parameter values need to be found. The study of the relation between parametric values for simulation techniques and the quality of the resulting trajectories has been studied either through perceptual experiments or by comparison with real crowd t…
▽ More
Simulating crowds requires controlling a very large number of trajectories and is usually performed using crowd motion algorithms for which appropriate parameter values need to be found. The study of the relation between parametric values for simulation techniques and the quality of the resulting trajectories has been studied either through perceptual experiments or by comparison with real crowd trajectories. In this paper, we integrate both strategies. A quality metric, QF, is proposed to abstract from reference data while capturing the most salient features that affect the perception of trajectory realism. QF weights and combines cost functions that are based on several individual, local and global properties of trajectories. These trajectory features are selected from the literature and from interviews with experts. To validate the capacity of QF to capture perceived trajectory quality, we conduct an online experiment that demonstrates the high agreement between the automatic quality score and non-expert users. To further demonstrate the usefulness of QF, we use it in a data-free parameter tuning application able to tune any parametric microscopic crowd simulation model that outputs independent trajectories for characters. The learnt parameters for the tuned crowd motion model maintain the influence of the reference data which was used to weight the terms of QF.
△ Less
Submitted 16 September, 2021; v1 submitted 27 August, 2021;
originally announced August 2021.
-
Virtual Co-Embodiment: Evaluation of the Sense of Agency while Sharing the Control of a Virtual Body among Two Individuals
Authors:
Rebecca Fribourg,
Nami Ogawa,
Ludovic Hoyet,
Ferran Argelaguet,
Takuji Narumi,
Michitaka Hirose,
Anatole Lécuyer
Abstract:
In this paper, we introduce a concept called ''virtual co-embodiment'', which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared…
▽ More
In this paper, we introduce a concept called ''virtual co-embodiment'', which enables a user to share their virtual avatar with another entity (e.g., another user, robot, or autonomous agent). We describe a proof-of-concept in which two users can be immersed from a first-person perspective in a virtual environment and can have complementary levels of control (total, partial, or none) over a shared avatar. In addition, we conducted an experiment to investigate the influence of users' level of control over the shared avatar and prior knowledge of their actions on the users' sense of agency and motor actions. The results showed that participants are good at estimating their real level of control but significantly overestimate their sense of agency when they can anticipate the motion of the avatar. Moreover, participants performed similar body motions regardless of their real control over the avatar. The results also revealed that the internal dimension of the locus of control, which is a personality trait, is negatively correlated with the user's perceived level of control. The combined results unfold a new range of applications in the fields of virtual-reality-based training and collaborative teleoperation, where users would be able to share their virtual body.
△ Less
Submitted 9 June, 2020; v1 submitted 8 November, 2019;
originally announced November 2019.
