Tracking objects that change in appearance with phase synchrony

Healthy humans depend on their ability to track objects while they move through the world even as they change in appearance. Here, we introduce the FeatureTracker challenge to systematically evaluate and compare the abilities of humans and state-of-the-art deep neural networks (DNNs) to track objects that change in appearance over time. While humans can effortlessly solve this task, DNNs cannot. Drawing inspiration from cognitive science and neuroscience, we describe a novel recurrent neural circuit that can induce this tracking capability in DNNs by leveraging the oscillatory activity of its neurons to follow objects even as their appearances change. The resulting complex-valued recurrent neural network (CV-RNN) outperformed all other DNNs and approached human accuracy on the FeatureTracker challenge. The success of this novel neural circuit provides computational evidence for a long-hypothesized role of phase synchronization for visual attention and reasoning.

Recommended citation: Muzellec, S.*, Linsley, D.*, Ashok, A. K., VanRullen, R., Serre, T. (2024). Tracking objects that change in appearance with phase synchrony. Conference on Cognitive Computational Neuroscience (CCN).