Continual Learning

Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs
This paper presents a comprehensive analysis of the performance-forgetting trade-offs inherent in low-rank adaptation using principal components as initialization.
Alessio Quercia, Arya Bangun, Ira Assent, Hanno Scharr
PDF Cite
Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs