Back to Home
arXiv AI··Papers & Tech

LoRAWeaver: Directional Adapter Routing in LoRA Representation Space

中文摘要

LoRAWeaver 提出 LoRA 定向路由,实现动态环境下无需重训即可高效选择新适配器。

English Summary

LoRAWeaver introduces directional routing in LoRA space, enabling efficient adapter selection in dynamic settings without retraining when adding new adapters.

Original Excerpt

arXiv:2611.17340v1 Announce Type: new Abstract: Large language models are expensive to update through full retraining. Parameter-efficient fine-tuning reduces the cost of adaptation, but repeated adaptation creates a growing pool of task-specific LoRA adapters. This pool complicates adapter selection during inference. Existing routing and encoder-based methods often require retraining when new adapters are added, which limits their use in dynamic settings. This paper treats adapter routing as a representation alignment problem in the adapter-output space. The proposed training-free method selects adapters by matching adapter-conditioned query representations against task prototypes using cosine similarity. Experiments with LLaMA-2-7B and LLaMA-2-13B on a 48-task FLAN-v2 benchmark and on out-of-distribution tasks show that the method outperforms training-free baselines in the cross-task setting and approaches a trained retrieval encoder without learning a router. An ablation confirms the benefit of directional alignment in the adapter-output space.