1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis

Authors: Raphaela Kreiser*, Asude Aydin*, Yin Bi, Claudio Fanconi, Peter Dürr, Naoya Takahashi

* denotes equal authorship.

Affiliations: Sony AI, Zurich

Venue: arXiv preprint, 2026

Note: Preprint from research completed during my 2023 Sony AI internship.

Links: Paper PDF

Figure overview for 1000 Rallies: event-camera perception for robotic table tennis — Figure 1: Event-camera perception pipeline for real-time robotic table tennis ball-state estimation.

Abstract

Robotic table tennis is a demanding testbed for real-time perception because accurate ball-state estimation must happen at high frequency and low latency. This preprint introduces a large-scale event-camera dataset for table tennis, with over 1000 rallies recorded from players ranging from amateurs to elite-level athletes, synchronized with high-speed frame-based cameras to produce pseudo ground-truth labels for ball position, velocity, and spin. The paper trains a convolutional neural network to estimate ball position and image-plane velocity directly from event streams, then uses those predictions inside a Kalman-filter trajectory pipeline. Incorporating event-based velocity reduces bounce-point prediction error relative to a position-only baseline, and the system is integrated with a robotic arm to enable real-time human-robot table tennis rallies driven by event-based perception.