research-article

A Perceptually-Validated Metric for Crowd Trajectory Quality Evaluation

Authors:

Beatríz Cabrero Daniel,

Ricardo Marques,

Julien Pettré, and

Josep BlatAuthors Info & Claims

Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 4, Issue 3

Article No.: 42, Pages 1 - 18

https://doi.org/10.1145/3480136

Published: 27 September 2021 Publication History

Abstract

Simulating crowds requires controlling a very large number of trajectories and is usually performed using crowd motion algorithms for which appropriate parameter values need to be found. The study of the relation between parametric values for simulation techniques and the quality of the resulting trajectories has been studied either through perceptual experiments or by comparison with real crowd trajectories. In this paper, we integrate both strategies. A quality metric, QF, is proposed to abstract from reference data while capturing the most salient features that affect the perception of trajectory realism. QF weights and combines cost functions that are based on several individual, local and global properties of trajectories. These trajectory features are selected from the literature and from interviews with experts. To validate the capacity of QF to capture perceived trajectory quality, we conduct an online experiment that demonstrates the high agreement between the automatic quality score and non-expert users. To further demonstrate the usefulness of QF, we use it in a data-free parameter tuning application able to tune any parametric microscopic crowd simulation model that outputs independent trajectories for characters. The learnt parameters for the tuned crowd motion model maintain the influence of the reference data which was used to weight the terms of QF.

Supplementary Material

daniel (daniel.zip)

Supplemental movie, appendix, image and software files for, A Perceptually-Validated Metric for Crowd Trajectory Quality Evaluation

Download
20.41 MB

References

[1]

A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese. 2016. Social LSTM: Human Trajectory Prediction in Crowded Spaces. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Las Vegas, NV, USA, 961--971. https://doi.org/10.1109/CVPR.2016.110

[2]

Javad Amirian, Wouter van Toll, Jean-Bernard Hayet, and Julien Pettré. 2019. Data-Driven Crowd Simulation with Generative Adversarial Networks. In Proceedings of the 32nd International Conference on Computer Animation and Social Agents (Paris, France) (CASA '19). Association for Computing Machinery, New York, NY, USA, 7--10. https://doi.org/10.1145/3328756.3328769

Digital Library

[3]

Glen Berseth, Mubbasir Kapadia, Brandon Haworth, and Petros Faloutsos. 2014. SteerFit: Automated Parameter Fitting for Steering Algorithms. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Copenhagen, Denmark) (SCA '14). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 113--122. http://dl.acm.org/citation.cfm?id=2849517.2849536

[4]

Panayiotis Charalambous and Yiorgos Chrysanthou. 2014. The PAG crowd: A graph based approach for efficient data-driven crowd simulation. Computer Graphics Forum 33, 8 (2014), 95--108.

Digital Library

[5]

Panayiotis Charalambous, Ioannis Karamouzas, Stephen Guy, and Yiorgos Chrysanthou. 2014. A Data-Driven Framework for Visual Crowd Analysis. Computer Graphics Forum 33 (10 2014). https://doi.org/10.1111/cgf.12472

[6]

Ujjal Chattaraj, Armin Seyfried, and Partha Chakroborty. 2009. Comparison of Pedestrian Fundamental Diagram Across Cultures. Advances in Complex Systems (ACS) 12 (06 2009), 393--405. https://doi.org/10.1142/S0219525909002209

[7]

Teófilo Dutra, Ricardo Marques, Joaquim Cavalcante-Neto, Creto Vidal, and Julien Pettre. 2017. Gradient-based steering for vision-based crowd simulation algorithms. Computer Graphics Forum 36 (05 2017). https://doi.org/10.1111/cgf.13130

[8]

A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi. 2018. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Las Vegas, NV, USA, 2255--2264.

[9]

Stephen J Guy, Jatin Chhugani, Sean Curtis, Pradeep Dubey, Ming C Lin, and Dinesh Manocha. 2010. PLEdestrians: A Least-Effort Approach to Crowd Simulation, In Eurographics/ ACM SIGGRAPH Symposium on Computer Animation. Proc. of Eurographics/ACM SIGGRAPH Symposium on Computer Animation 2010, 1, 119--128. https://doi.org/10.2312/SCA/SCA10/119-128

[10]

Stephen J Guy, Jur Van Den Berg, Wenxi Liu, Rynson Lau, Ming C Lin, and Dinesh Manocha. 2012. A statistical similarity measure for aggregate crowd dynamics. ACM Transactions on Graphics 31, 6 (2012), 1--11.

Digital Library

[11]

Dirk Helbing and Péter Molnár. 1995. Social force model for pedestrian dynamics. Physical Review E 51, 5 (may 1995), 4282--4286. https://doi.org/10.1103/PhysRevE.51.4282

[12]

Ludovic Hoyet, Anne-Hélène Olivier, Richard Kulpa, and Julien Pettre. 2016. Perceptual Effect of Shoulder Motions on Crowd Animations. ACM Transactions on Graphics 35 (07 2016), 1--10. https://doi.org/10.1145/2897824.2925931

[13]

Roger L Hughes. 2003. The flow of human crowds. Annual review of fluid mechanics 35, 1 (2003), 169--182. Asja Jelić, Cecile Appert-Rolland, Samuel Lemercier, and Julien Pettre. 2012. Properties of pedestrians walking in line: Fundamental diagrams. Physical review. E, Statistical, nonlinear, and soft matter physics 85 (03 2012), 036111. https://doi.org/10.1103/PhysRevE.85.036111

[14]

Mubbasir Kapadia, Shawn Singh, Brian Allen, Glenn Reinman, and Petros Faloutsos. 2009. SteerBug: An Interactive Framework for Specifying and Detecting Steering Behaviors. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (New Orleans, Louisiana) (SCA '09). Association for Computing Machinery, New York, NY, USA, 209--216. https://doi.org/10.1145/1599470.1599497

Digital Library

[15]

Mubbasir Kapadia, Matt Wang, Shawn Singh, Glenn Reinman, and Petros Faloutsos. 2011. Scenario Space: Characterizing Coverage, Quality, and Failure of Steering Algorithms. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Vancouver, British Columbia, Canada) (SCA '11). Association for Computing Machinery, New York, NY, USA, 53--62. https://doi.org/10.1145/2019406.2019414

Digital Library

[16]

Ioannis Karamouzas, Peter Heil, Pascal Beek, and Mark H. Overmars. 2009. A Predictive Collision Avoidance Model for Pedestrian Simulation. In Proceedings of the 2nd International Workshop on Motion in Games (Zeist, The Netherlands) (MIG '09). Springer-Verlag, Berlin, Heidelberg, 41--52. https://doi.org/10.1007/978-3-642-10347-6_4

[17]

Ioannis Karamouzas, Brian Skinner, and Stephen J. Guy. 2014. Universal Power Law Governing Pedestrian Interactions. Physical Review Letters 113, 23 (dec 2014), 238701. https://doi.org/10.1103/PhysRevLett.113.238701

[18]

Richard Kulpa, Anne-Hélène Olivier, Jan Ondřej, and Julien Pettré. 2011. Imperceptible Relaxation of Collision Avoidance Constraints in Virtual Crowds. ACM Trans. Graph. 30, 6 (Dec. 2011), 1--10. https://doi.org/10.1145/2070781.2024172

Digital Library

[19]

Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski. 2007. Crowds by Example. Computer Graphics Forum 26, 3 (2007), 655--664. https://doi.org/10.1111/j.1467-8659.2007.01089.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-8659.2007.01089.x

[20]

Rachel McDonnell, Michéal Larkin, Simon Dobbyn, Steven Collins, and Carol O'Sullivan. 2008. Clone attack! Perception of crowd variety. ACM Transactions on Graphics 27 (08 2008). https://doi.org/10.1145/1360612.1360625

[21]

Rachel McDonnell, Michéal Larkin, Benjamin Hernandez, Isaac Rudomin, and Carol O'Sullivan. 2009. Eye-catching Crowds: Saliency based Selective Variation. ACM Transactions on Graphics 28 (08 2009). https://doi.org/10.1145/1531326.1531361

Digital Library

[22]

Jan Ondřej, Julien Pettré, Anne-Hélène Olivier, and Stéphane Donikian. 2010. A synthetic-vision based steering approach for crowd simulation. ACM Transactions on Graphics 29, 4 (2010), 123.

Digital Library

[23]

Sébastien Paris, Julien Pettre, and Stéphane Donikian. 2007. Pedestrian Reactive Navigation for Crowd Simulation: a Predictive Approach Abstract. Comput. Graph. Forum 26 (09 2007), 665--674. https://doi.org/10.1111/j.1467-8659.2007.01090.x

[24]

Craig W. Reynolds. 1987. Flocks, Herds and Schools: A Distributed Behavioral Model. SIGGRAPH Comput. Graph. 21, 4 (Aug. 1987), 25--34. https://doi.org/10.1145/37402.37406

Digital Library

[25]

Adrien Treuille, Seth Cooper, and Zoran Popović. 2006. Continuum Crowds. ACM Trans. Graph. 25, 3 (July 2006), 1160--1168. https://doi.org/10.1145/1141911.1142008

Digital Library

[26]

A. Turnwald, S. Eger, and D. Wollherr. 2015. Investigating similarity measures for locomotor trajectories based on the human perception of differences in motions. In 2015 IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO). 2015 IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO), Lyon, France, 1--6. https://doi.org/10.1109/ARSO.2015.7428196

[27]

J. van den Berg, Ming Lin, and D. Manocha. 2008. Reciprocal Velocity Obstacles for real-time multi-agent navigation. In 2008 IEEE International Conference on Robotics and Automation. IEEE, New York, NY, 1928--1935. https://doi.org/10.1109/ROBOT.2008.4543489

[28]

D. Wolinski, S. J. Guy, A.-H. Olivier, M. Lin, D. Manocha, and J. Pettré. 2014. Parameter Estimation and Comparative Evaluation of Crowd Simulations. Comput. Graph. Forum 33, 2 (May 2014), 303--312. https://doi.org/10.1111/cgf.12328

Cited By

Silva GKnob PMontanha RMusse S(2024)Evaluating and comparing crowd simulationsGraphical Models10.1016/j.gmod.2023.101212131:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.gmod.2023.101212
Shi HYao QLi Y(2023)Learning to Simulate Crowd Trajectories with Graph NetworksProceedings of the ACM Web Conference 202310.1145/3543507.3583858(4200-4209)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583858
Hu KHaworth BBerseth GPavlovic VFaloutsos PKapadia M(2023)Heterogeneous Crowd Simulation Using Parametric Reinforcement LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.313903129:4(2036-2052)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TVCG.2021.3139031
Show More Cited By

Index Terms

A Perceptually-Validated Metric for Crowd Trajectory Quality Evaluation
1. Computing methodologies
  1. Artificial intelligence
    1. Control methods
      1. Motion path planning
    2. Distributed artificial intelligence
      1. Multi-agent systems
  2. Modeling and simulation
    1. Simulation evaluation
    2. Simulation types and techniques
      1. Agent / discrete models
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Dimensionality reduction

Recommendations

Trajectory Query Based on Trajectory Segments with Activities
UrbanGIS'17: Proceedings of the 3rd ACM SIGSPATIAL Workshop on Smart Cities and Urban Analytics

Searching trajectories with activities has attracted much attention in the last decade. Existing studies tend to find trajectories with activities matched to the required keywords. However, returned trajectories may have a satisfying textual matching ...
Read More
Slice-based Motion Trajectory Privacy Protection Method in Crowd Sensing
2021 IEEE Wireless Communications and Networking Conference (WCNC)
In the crowd sensing application, a slice-based trajectory privacy protection method (STPP) is proposed for the privacy of mobile object trajectory data with rich spatiotemporal information. The trajectory data is first sliced with the position (GPS) as ...
Read More
Private Trajectory Data Publication for Trajectory Classification
Web Information Systems and Applications
Abstract
Trajectory classification (TC), i.e., predicting the class labels of moving objects based on their trajectories and other features, has many important real-world applications. Private trajectory data publication is to anonymize trajectory data, ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques

Proceedings of the ACM on Computer Graphics and Interactive Techniques Volume 4, Issue 3

September 2021

268 pages

EISSN:2577-6193

DOI:10.1145/3488568

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 September 2021

Published in PACMCGIT Volume 4, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
140
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)2

Other Metrics

View Author Metrics

Citations

Cited By

Silva GKnob PMontanha RMusse S(2024)Evaluating and comparing crowd simulationsGraphical Models10.1016/j.gmod.2023.101212131:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.gmod.2023.101212
Shi HYao QLi Y(2023)Learning to Simulate Crowd Trajectories with Graph NetworksProceedings of the ACM Web Conference 202310.1145/3543507.3583858(4200-4209)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583858
Hu KHaworth BBerseth GPavlovic VFaloutsos PKapadia M(2023)Heterogeneous Crowd Simulation Using Parametric Reinforcement LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.313903129:4(2036-2052)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TVCG.2021.3139031
Kwiatkowski AKalogeiton VPettré JCani M(2023)Understanding reinforcement learned crowdsComputers and Graphics10.1016/j.cag.2022.11.007110:C(28-37)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.cag.2022.11.007

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents