Visual Features and Their Own Optical Flow - Betti, Alessandro; Boccignone, Giuseppe; Faggi, Lapo; Gori, Marco; Melacci, Stefano (Frontiers in Artificial Intelligence, 2021)
Symmetries, invariances and conservation equations have always been an invaluable guide in Science to model natural phenomena through simple yet effective relations. For instance, in computer vision, translation equivariance is typically a built-in property of neural architectures that are used to solve visual tasks; networks with computational layers implementing such a property are known as Convolutional Neural Networks (CNNs). This kind of mathematical symmetry, as well as many others that has been recently studied, are typically generated by some underlying group of transformations (translations in the case of CNNs, rotations, etc..) and are particularly suitable to process highly structured data such as molecules or chemical compounds which are known to possess those specific symmetries. When dealing with video streams, common built-in equivariances are able to handle only a small fraction of the broad spectrum of transformations encoded in the visual stimulus and, therefore, the corresponding neural architectures have to resort to a huge number of supervisions in order to achieve good generalization capabilities. In the paper we formulate a theory on the development of visual features that is based on the idea that movement itself provides trajectories on which to impose consistency. We introduce the principle of Material Point Invariance which states that each visual feature is conjugated with its own characteristic optical flow, so as features and correspondent velocities are an indissoluble pair. Then we discuss the interaction of features and velocities and show that certain motion invariance traits could be regarded as a generalization of the classical concept of affordance. These analyses of feature-velocity interactions and their invariance properties leads to a visual field theory which expresses the dynamical constraints of motion coherence and we might led to discover the joint evolution of the visual features along with the associated optical flows.