Project overview
MATRIX-HAR is designed for event-based human action recognition under variable environmental conditions and limited computing resources. Its Adaptive Temporal Resolution module changes temporal granularity according to the input, while the Motion Guided Feature Module exploits motion information already encoded in the event stream.
Core contributions
- Adaptive Temporal Resolution for input-dependent temporal processing.
- A Motion Guided Feature Module that emphasizes intrinsic motion patterns in event streams.
- Four hierarchical transfer-learning strategies from simpler to more challenging conditions.
- Post-training quantisation for compact edge deployment with limited performance loss.
Reported evidence
The paper reports 97.23% on PAF, 95.73% on THU-E-ACT-50, and 65.8% on THU-E-ACT-50 CHL. The reported gains over prior methods are 0.71, 0.74, 3.03, and 7.3 percentage points on the four datasets, respectively.
Datasets and evaluation
- HARDVS for large-scale event-based human action recognition.
- PAF for pedestrian, action, and fall-related event sequences.
- THU-E-ACT-50 for controlled event-based action recognition.
- THU-E-ACT-50 CHL for evaluation under more challenging environmental conditions.
Relevance to human-in-the-loop robotics
Human-in-the-loop robots need action understanding that is fast enough for interaction and compact enough for edge hardware. MATRIX-HAR directly targets both requirements while also studying how knowledge transfers from easier laboratory conditions to harder operating environments.