EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

TL;DR

EngramEdit enables decoupled knowledge updates in large language models through conditional memory, allowing for independent factual knowledge updates without changing the Transformer backbone. This approach achieves near-perfect editing success and preserves unrelated knowledge. EngramEdit extends conditional memory's role beyond model scaling.

Why It Matters

EngramEdit's ability to update factual knowledge independently has significant practical implications for maintaining and improving large language models. This approach can help reduce the need for frequent full-model retraining, saving computational resources and reducing environmental impact. EngramEdit also enables more efficient knowledge graph updates and multi-hop reasoning.

Key Findings

• near-perfect editing success
• preservation of unrelated knowledge
• improved multi-hop reasoning accuracy