An Analytical Theory of Auxiliary Learning

TL;DR

This paper provides an analytical theory of auxiliary learning, explaining how training a neural network on additional tasks improves its performance on a target task. The theory derives a closed system of differential equations and obtains a closed-form expression for the generalization error. Auxiliary learning balances forcing dynamics with gradient noise to improve generalization.

Why It Matters

Understanding auxiliary learning is crucial for optimizing neural network performance and improving generalization. This theory has practical significance in multi-task learning and transfer learning applications. It can help researchers design more effective auxiliary tasks to enhance target task performance.

Key Findings

• closed-form expression for generalization error
• fluctuation-dissipation analytical theory
• auxiliary tasks balance forcing dynamics with gradient noise