SwimXYZ: A large-scale dataset of synthetic swimming motions and videos

Guénolé Fiche1, Vincent Sevestre2, Camila Gonzalez-Barral3, Simon Leglaive1, Renaud Séguier1
1CentraleSupélec, IETR UMR CNRS 6164, France - 2Centrale Nantes, France 3Université technologique de Compiègne, France

ACM MIG 2023
Paper Dataset
Teaser

Abstract

Technologies play an increasingly important role in sports and become a real competitive advantage for the athletes who benefit from it. Among them, the use of motion capture is developing in various sports to optimize sporting gestures. Unfortunately, traditional motion capture systems are expensive and constraining. Recently developed computer vision-based approaches also struggle in certain sports, like swimming, due to the aquatic environment. One of the reasons for the gap in performance is the lack of labeled datasets with swimming videos. In an attempt to address this issue, we introduce SwimXYZ, a synthetic dataset of swimming motions and videos. SwimXYZ contains 3.4 million frames annotated with ground truth 2D and 3D joints, as well as 240 sequences of swimming motions in the SMPL parameters format. In addition to making this dataset publicly available, we present use cases for SwimXYZ in swimming stroke clustering and 2D pose estimation.

Pipeline for generating videos


Pipeline

We first generate a unique motion using GANimator trained on a clean swimming motion , and re-target it on a human body model. We then create an environment by choosing parameters such as camera view, water effects, and lighting. We obtain the final animation by putting the swimmer in the virtual swimming pool.



Sample videos


SwimXYZ is a synthetic dataset specialized in swimming, with synthetic monocular videos annotated with ground truth 2D and 3D joints. SwimXYZ consists of 11520 videos for a total of 3.4 million frames with variations in camera angle, subject and water appearances, lighting, and motion.



2D pose estimation

We use SwimXYZ for finetuning ViTPose, the state-of-the- art model for 2D human pose estimation. Qualitative evaluation is performed on images collected on the web.

Pipeline

BibTeX

@inproceedings{fiche2023swimxyz,
        title={SwimXYZ: A large-scale dataset of synthetic swimming motions and videos},
        author={Fiche, Gu{\'e}nol{\'e} and Sevestre, Vincent and Gonzalez-Barral, Camila and Leglaive, Simon and S{\'e}guier, Renaud},
        booktitle={Proceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games},
        pages={1--7},
        year={2023}
      }