2024

1. CVPR

   PREGO: online mistake detection in PRocedural EGOcentric Videos

   Alessandro Flaborea, Guido Maria D’Amely Di Melendugno, Leonardo Plini, and 5 more authors

   Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition (CVPR) 2024

   Abs Bib HTML

   Promptly identifying procedural errors from egocentric videos in an online setting is highly challenging and valuable for detecting mistakes as soon as they happen. This capability has a wide range of applications across various fields, such as manufacturing and healthcare. The nature of procedural mistakes is open-set since novel types of failures might occur, which calls for one-class classifiers trained on correctly executed procedures. However, no technique can currently detect open-set procedural mistakes online. We propose PREGO, the first online one-class classification model for mistake detection in PRocedural EGOcentric videos. PREGO is based on an online action recognition component to model the current action, and a symbolic reasoning module to predict the next actions. Mistake detection is performed by comparing the recognized current action with the expected future one. We evaluate PREGO on two procedural egocentric video datasets, Assembly101 and Epic-tent, which we adapt for online benchmarking of procedural mistake detection to establish suitable benchmarks, thus defining the Assembly101-O and Epic-tent-O datasets, respectively.

   @article{flaborea2024prego,
     title = {PREGO: online mistake detection in PRocedural EGOcentric Videos},
     author = {Flaborea, Alessandro and D’Amely Di Melendugno, Guido Maria and Plini, Leonardo and Scofano, Luca and De Matteis, Edoardo and Furnari, Antonino and Farinella, Giovanni Maria and Galasso, Fabio},
     journal = {Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition (CVPR)},
     year = {2024},
   }

2023

1. ICCV

   Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection

   Alessandro Flaborea, Luca Collorone, Guido Maria D’Amely Di Melendugno, and 3 more authors

   Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2023

   Abs Bib HTML

   Anomalies are rare and anomaly detection is often therefore framed as One-Class Classification (OCC), i.e. trained solely on normalcy. Leading OCC techniques constrain the latent representations of normal motions to limited volumes and detect as abnormal anything outside, which accounts satisfactorily for the openset’ness of anomalies. But normalcy shares the same openset’ness property, since humans can perform the same action in several ways, which the leading techniques neglect. We propose a novel generative model for video anomaly detection (VAD), which assumes that both normality and abnormality are multimodal. We consider skeletal representations and leverage state-of-the-art diffusion probabilistic models to generate multimodal future human poses. We contribute a novel conditioning on the past motion of people and exploit the improved mode coverage capabilities of diffusion processes to generate different-but-plausible future motions. Upon the statistical aggregation of future modes, an anomaly is detected when the generated set of motions is not pertinent to the actual future. We validate our model on 4 established benchmarks: UBnormal, HR-UBnormal, HR-STC, and HR-Avenue, with extensive experiments surpassing state-of-the-art results.

   @article{flaborea2023mocodad,
     title = {Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection},
     author = {Flaborea, Alessandro and Collorone, Luca and D’Amely Di Melendugno, Guido Maria and D'Arrigo, Stefano and Prenkaj, Bardh and Galasso, Fabio},
     journal = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
     year = {2023},
   }

2. CVPR WS

   Are We Certain It’s Anomalous?

   Alessandro Flaborea, Bardh Prenkaj, Bharti Munjal, and 4 more authors

   Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2023

   Abs Bib HTML

   The progress in modelling time series and, more generally, sequences of structured-data has recently revamped research in anomaly detection. The task stands for identifying abnormal behaviours in financial series, IT systems, aerospace measurements, and the medical domain, where anomaly detection may aid in isolating cases of depression and attend the elderly. Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations and since the definition of anomalous is sometimes subjective. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD). HypAD learns self-supervisedly to reconstruct the input signal. We adopt best practices from the state-of-the-art to encode the sequence by an LSTM, jointly learnt with a decoder to reconstruct the signal, with the aid of GAN critics. Uncertainty is estimated end-to-end by means of a hyperbolic neural network. By using uncertainty, HypAD may assess whether it is certain about the input signal but it fails to reconstruct it because this is anomalous; or whether the reconstruction error does not necessarily imply anomaly, as the model is uncertain, e.g. a complex but regular input signal. The novel key idea is that a detectable anomaly is one where the model is certain but it predicts wrongly. HypAD outperforms the current state-of-the-art for univariate anomaly detection on established benchmarks based on data from NASA, Yahoo, Numenta, Amazon, Twitter. It also yields state-of-the-art performance on a multivariate dataset of anomaly activities in elderly home residences, and it outperforms the baseline on SWaT. Overall, HypAD yields the lowest false alarms at the best performance rate, thanks to successfully identifying detectable anomalies.

   @article{flaborea2022we,
     author = {Flaborea, Alessandro and Prenkaj, Bardh and Munjal, Bharti and Sterpa, Marco Aurelio and Aragona, Dario and Podo, Luca and Galasso, Fabio},
     title = {Are We Certain It's Anomalous?},
     journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
     year = {2023},
     pages = {2896-2906},
     doi = {https://doi.org/10.48550/arXiv.2211.09224},
   }

3. CVPR WS

   Best Practices for 2-Body Pose Forecasting

   Muhammad Rameez Ur Rahman*, Luca Scofano*, Edoardo De Matteis, and 3 more authors

   Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2023

   Abs Bib HTML

   The task of collaborative human pose forecasting stands for predicting the future poses of multiple interacting people, given those in previous frames. Predicting two people in interaction, instead of each separately, promises better performance, due to their body-body motion correlations. But the task has remained so far primarily unexplored. In this paper, we review the progress in human pose forecasting and provide an in-depth assessment of the singleperson practices that perform best for 2-body collaborative motion forecasting. Our study confirms the positive impact of frequency input representations, space-time separable and fully-learnable interaction adjacencies for the encoding GCN and FC decoding. Other single-person practices do not transfer to 2-body, so the proposed best ones do not include hierarchical body modeling or attention-based interaction encoding. We further contribute a novel initialization procedure for the 2-body spatial interaction parameters of the encoder, which benefits performance and stability. Altogether, our proposed 2-body pose forecasting best practices yield a performance improvement of 21.9% over the state-of-theart on the most recent ExPI dataset, whereby the novel initialization accounts for 3.5%

   @article{Rahman_2023_CVPR,
     author = {Rahman*, Muhammad Rameez Ur and Scofano*, Luca and De Matteis, Edoardo and Flaborea, Alessandro and Sampieri, Alessio and Galasso, Fabio},
     title = {Best Practices for 2-Body Pose Forecasting},
     journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
     year = {2023},
     pages = {3613-3623},
   }

4. preprint

   Contracting Skeletal Kinematic Embeddings for Anomaly Detection

   Alessandro Flaborea, Guido Maria D’Amely Di Melendugno, Stefano D’arrigo, and 3 more authors

   arXiv preprint arXiv:2301.09489 2023

   Abs arXiv Bib

   Detecting the anomaly of human behavior is paramount to timely recognizing endangering situations, such as street fights or elderly falls. However, anomaly detection is complex, since anomalous events are rare and because it is an open set recognition task, i.e., what is anomalous at inference has not been observed at training. We propose COSKAD, a novel model which encodes skeletal human motion by an efficient graph convolutional network and learns to COntract SKeletal kinematic embeddings onto a latent hypersphere of minimum volume for Anomaly Detection. We propose and analyze three latent space designs for COSKAD: the commonly-adopted Euclidean, and the new spherical-radial and hyperbolic volumes. All three variants outperform the state-of-the-art, including video-based techniques, on the ShangaiTechCampus, the Avenue, and on the most recent UBnormal dataset, for which we contribute novel skeleton annotations and the selection of human-related videos. The source code and dataset will be released upon acceptance.

   @article{flaborea23,
     doi = {10.48550/ARXIV.2301.09489},
     author = {Flaborea, Alessandro and Di Melendugno, Guido Maria D'Amely and D'arrigo, Stefano and Sterpa, Marco Aurelio and Sampieri, Alessio and Galasso, Fabio},
     keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
     title = {Contracting Skeletal Kinematic Embeddings for Anomaly Detection},
     journal = {arXiv preprint arXiv:2301.09489},
     year = {2023},
   }

5. PR journal

   Query-guided networks for few-shot fine-grained classification and person search

   Bharti Munjal, Alessandro Flaborea, Sikandar Amin, and 2 more authors

   Pattern Recognition 2023

   Abs arXiv Bib

   Few-shot fine-grained classification and person search appear as distinct tasks and literature has treated them separately. But a closer look unveils important similarities: both tasks target categories that can only be discriminated by specific object details; and the relevant models should generalize to new categories, not seen during training. We propose a novel unified Query-Guided Network (QGN) applicable to both tasks. QGN consists of a Query-guided Siamese-Squeeze-and-Excitation subnetwork which re-weights both the query and gallery features across all network layers, a Query-guided Region Proposal subnetwork for query-specific localisation, and a Query-guided Similarity subnetwork for metric learning. QGN improves on a few recent few-shot fine-grained datasets, outperforming other techniques on CUB by a large margin. QGN also performs competitively on the person search CUHK-SYSU and PRW datasets, where we perform in-depth analysis.

   @article{MUNJAL2023109049,
     title = {Query-guided networks for few-shot fine-grained classification and person search},
     journal = {Pattern Recognition},
     volume = {133},
     pages = {109049},
     year = {2023},
     issn = {0031-3203},
     doi = {https://doi.org/10.1016/j.patcog.2022.109049},
     url = {https://www.sciencedirect.com/science/article/pii/S0031320322005295},
     author = {Munjal, Bharti and Flaborea, Alessandro and Amin, Sikandar and Tombari, Federico and Galasso, Fabio},
     keywords = {Meta-learning, Few-shot learning, Fine-grained classification, Person search, Person re-identification},
   }

6. AIIM journal

   A self-supervised algorithm to detect signs of social isolation in the elderly from daily activity sequences

   Bardh Prenkaj, Dario Aragona, Alessandro Flaborea, and 5 more authors

   Artificial Intelligence in Medicine 2023

   Abs Bib

   Considering the increasing aging of the population, multi-device monitoring of the activities of daily living (ADL) of older people becomes crucial to support independent living and early detection of symptoms of mental illnesses, such as depression and Alzheimer’s disease. Anomalies can anticipate the diagnosis of these pathologies in the patient’s normal behavior, such as reduced hygiene, changes in sleep habits, and fewer social interactions. These abnormalities are often subtle and hard to detect. Especially using non-intrusive monitoring devices might cause anomaly detectors to generate false alarms or ignore relevant clues. This limitation may hinder their usage by caregivers. Furthermore, the notion of abnormality here is context and patient-dependent, thus requiring untrained approaches. To reduce these problems, we propose a self-supervised model for multi-sensor time series signals based on Hyperbolic uncertainty for Anomaly Detection, which we dub HypAD. HypAD estimates uncertainty end-to-end, thanks to hyperbolic neural networks, and integrates it into the ”classic” notion of reconstruction loss in anomaly detection. Based on hyperbolic uncertainty, HypAD introduces the principle of a detectable anomaly. HypAD assesses whether it is sure about the input signal and fails to reconstruct it because it is anomalous or whether the high reconstruction loss is due to the model uncertainty, e.g., a complex but regular signal (cf. this parallels the residual model error upon training). The proposed solution has been incorporated into an end-to-end ADL monitoring system for elderly patients in retirement homes, developed within a funded project leveraging an interdisciplinary consortium of computer scientists, engineers, and geriatricians. Healthcare professionals were involved in the design and verification process to foster trust in the system. In addition, the system has been equipped with explainability features.

   @article{PRENKAJ2023102454,
     title = {A self-supervised algorithm to detect signs of social isolation in the elderly from daily activity sequences},
     journal = {Artificial Intelligence in Medicine},
     volume = {135},
     pages = {102454},
     year = {2023},
     issn = {0933-3657},
     publisher = {Elsevier},
     doi = {https://doi.org/10.1016/j.artmed.2022.102454},
     url = {https://www.sciencedirect.com/science/article/pii/S0933365722002068},
     author = {Prenkaj, Bardh and Aragona, Dario and Flaborea, Alessandro and Galasso, Fabio and Gravina, Saverio and Podo, Luca and Reda, Emilia and Velardi, Paola},
     keywords = {Anomaly detection, ADL, Elderly social isolation, HyperNN, Hyperbolic uncertainty},
   }