NeuroVisionHRT
Home / Projects / Accepted paper

Trigger-Free Synchronization of RGB and Event Cameras

A comparison of global clock mapping and per-packet anchoring for synchronized RGB-event perception without dedicated trigger hardware.

Accepted · ECCV 2026 Workshop

Accepted at the Workshop on Neuromorphic Vision: Advantages and Applications of Event Cameras, ECCV 2026.

Project overview

This project develops a practical RGB-event acquisition pipeline for camera combinations that cannot share a hardware trigger. A Prophesee EVK4 and a Basler acA1300-60gmNIR observe the same optical path through a 50/50 beam splitter, while host-clock mapping and packet-level timestamp anchoring align their streams on an NVIDIA Jetson AGX Orin mounted to a Unitree Go2.

Core contributions

  • A coaxial beam-splitter rig that removes parallax by construction and uses pitch-ratio normalization plus one-time residual-offset calibration.
  • A robust global affine mapping from each camera clock to the host timeline, explicitly accounting for clock offset and drift.
  • A local per-packet anchoring method implemented in a ROS 2 acquisition pipeline for drift-independent event timing.
  • Frame-centered event windows, event rendering, YOLO detections, tracking identities, and annotation transfer for multimodal dataset generation.

Current evidence

0.079 msBasler-to-host temporal mapping RMSE
0.091 msevent-to-host temporal mapping RMSE
0.858 pxspatial RMS reprojection error

The reported calibration used 90 accepted image pairs and 4,860 correspondences, with a 100% RANSAC inlier ratio. The results support sub-millisecond temporal alignment and sub-pixel spatial registration for frame-to-event label transfer.

Multimodal RGB-event camera setup showing the Jetson host, EVK4 event camera, Basler camera, beam splitter, PoE injector, power bank, and Unitree Go2 robot.
Multimodal acquisition platform: system architecture and the RGB–event sensor rig deployed on the Unitree Go2.

System and deployment

  • Prophesee EVK4 event camera and Basler acA1300-60gmNIR monochrome frame camera.
  • Matched optics behind a 50/50 non-polarizing beam splitter.
  • NVIDIA Jetson AGX Orin and ROS 2 on a Unitree Go2 mobile robotic platform.

Why it matters for robotics

Accurate cross-modal alignment lets mature RGB annotation and tracking tools supervise event data. That makes it possible to build mobile-robot datasets for human detection, tracking, segmentation, and interaction without manually labeling sparse event streams.