Study of human pose
Description
We study pose in videos and derive 2D and 3D information on the anatomical joints, the estimated head direction, and prototypical poses.
Themes
We study methods for gaze or head direction estimation in video sequences containing multiple individuals. Our goal is to rely on multiple information gathered from the scene under analysis, starting from the outputs of 2D pose estimation methods, and estimating the 2D apparent gaze as well as the 3D orientation of the head. We address the challenges of occlusions and partial information with a methodology able to provide an estimate associated with an uncertainty prediction.
References
G Cantarini, FF Tomenotti, N Noceti, F Odone “HHP-Net: A light heteroscedastic neural network for head pose estimation with uncertainty” IEEE WACV 2022
P A Dias, D Malafronte, H Medeiros and F Odone “Gaze Estimation for Assisted Living Environments” WACV 2020Collaboration with
Henry Medeiros Marquette University
We investigate the use of graphs, borrowing methods from network analysis and Natural Language Processing, for the characterisation of human motion patterns with interpretable models. We start from videos and exploit the output of pose estimators and semantic features extractors to collect the configuration of important joints in a body performing a movement. Each joint plays the role of a node in a (possibly complex) network of which we study the dynamic evolution and the presence of prototypical sub-networks and configurations.
References
Garbarino, D., Moro, M., Tacchino, C., Moretti, P., Casadio, M., Odone, F. and Barla, A., 2021, November. Attributed Graphettes-Based Preterm Infants Motion Analysis. In International Conference on Complex Networks and Their Applications (pp. 82-93). Springer, Cham.
Collaboration with
We tackle marker-less motion analysis to describe the motion evolution in time and to provide a quantitative analysis of human behavior in supervised or unsupervised way. Our goal is to understand the quality of motion, derive information on functional impairments, possibly also assessing the benefits of a rehabilitation procedure. Long term objectives of our research are ecological, non invasive, unbiased motion analysis methods to be adopted in the clinical practice. We address different application: motion analysis in Multiple Sclerosis patients (with FISM) and Stroke survivors, General Movements analysis in premature infants (with Gaslini Hospital, Genova), analysis of motor learning in instruments players (with Conservatorio Nicolò Paganini, Marquette University, Music Institute of Chicago), epilepsy and sleep disorders assessment (with Gaslini Hospital, Genova).
Funded by FISM and DIBRIS.References
Moro, M., Marchesi, G., Hesse, F., Odone, F. and Casadio, M., Markerless vs. Marker-Based Gait Analysis: A Proof of Concept Study. Sensors, 22(5), p.2011.
Moro, M., Marchesi, G., Odone, F. and Casadio, M., 2020, March. Markerless gait analysis in stroke survivors based on computer vision and deep learning: A pilot study. In Proceedings of the 35th Annual ACM Symposium on Applied Computing (pp. 2097-2104).
Garello, L., Moro, M., Tacchino, C., Campone, F., Durand, P., Blanchi, I., Moretti, P., Casadio, M. and Odone, F., 2021, August. A Study of At-term and Preterm Infants' Motion Based on Markerless Video Analysis. In 2021 29th European Signal Processing Conference (EUSIPCO) (pp. 1196-1200). IEEE.Collaboration with
Collaboration with Maura Casadio Neurolab UniGe
Regaining functional independence plays a crucial role to improve the quality of life of individuals with motor disabilities. We address this problem within the framework of Body-Machine Interfaces (BoMIs). BoMIs enable individuals with restricted mobility to extend their capabilities by mapping their residual body movements into commands to control an external device. We develop video-based marker-less BoMIs that allows the user to control the computer cursor with the motion of the head and the shoulders
References
Moro, M., Rizzoglio, F., Odone, F. and Casadio, M., 2021, January. A Video-Based MarkerLess Body Machine Interface: A Pilot Study. In International Conference on Pattern Recognition (pp. 233-240). Springer, Cham.
Collaboration with
Neurolab UNIGE, Northwestern University, Shirley Ryan Ability Lab in Chicago.
Pose estimation enables the assessment of physical well-being: by analysing patients over medium to long time periods, evaluating in particular their motility and the quality of Activity of Daily Living (how much they move, how active they are), we derive qualitative and quantitative measures of their overall physical well-being. In the analysis of older population we can estimate a frailty parameter which is coherent with the outcomes of medical practices.
References
Collaboration with
MoDiPro facility (Modello di Dimissione Protetta, Protected Discharge Facility), Ospedale Galliera (Genova IT)
Emotional well-being is related to the sense of fulfillment; it includes satisfaction, optimism, having a purpose in life as well as being able to make the most of your abilities to cope with the normal challenges of life. An increasing body of research suggests that initiatives promoting physical wellbeing disregarding mental and social wellbeing may lead to failure. In this general framework we consider in particular social interaction analysis. Starting from pose and gaze estimation, we compute the quantity and quality of social interactions, with a focus on dyadic interaction, considering both in presence gatherings and virtual meetings.
Funded by CARIPLOReferences
Stairway to Elders: Bridging Space, Time and Emotions in Their Social Environment for Wellbeing, ICPRAM, 2020
Positive Technologies for Elderly well-being: a review, Pattern Recognition Letters, 2020Collaboration with
PHuSe Lab UNIMI
Andrea Gaggioli (UniCatt)
Claudio DeSperati (HSR)
Team
Francesca Odone DIBRIS, Università di Genova
Nicoletta Noceti DIBRIS, Università di Genova
Matteo Moro DIBRIS, Università di Genova