The role of locality and weight sharing in convolutional neural networks

TL;DR

This paper analyzes the role of locality and weight sharing in convolutional neural networks, finding that locality affects sample complexity and weight sharing's advantage decreases with spatial correlations. The authors derive ordinary differential equations to describe the dynamics of convolutional and locally connected students. They compare the learning trajectories of these students under different conditions.

Why It Matters

Understanding the effects of locality and weight sharing is crucial for designing efficient convolutional neural networks. This research has practical implications for computer vision and image processing applications. The findings can inform the development of more effective neural network architectures.

Key Findings

• Locality affects sample complexity
• Weight sharing's advantage decreases with spatial correlations
• Convolutional students learn faster than locally connected students for uncorrelated positions