NumLLM: Numeric-Sensitive Large Language Model for Chinese Finance
arxiv(2024)
摘要
Recently, many works have proposed various financial large language models
(FinLLMs) by pre-training from scratch or fine-tuning open-sourced LLMs on
financial corpora. However, existing FinLLMs exhibit unsatisfactory performance
in understanding financial text when numeric variables are involved in
questions. In this paper, we propose a novel LLM, called numeric-sensitive
large language model (NumLLM), for Chinese finance. We first construct a
financial corpus from financial textbooks which is essential for improving
numeric capability of LLMs during fine-tuning. After that, we train two
individual low-rank adaptation (LoRA) modules by fine-tuning on our constructed
financial corpus. One module is for adapting general-purpose LLMs to financial
domain, and the other module is for enhancing the ability of NumLLM to
understand financial text with numeric variables. Lastly, we merge the two LoRA
modules into the foundation model to obtain NumLLM for inference. Experiments
on financial question-answering benchmark show that NumLLM can boost the
performance of the foundation model and can achieve the best overall
performance compared to all baselines, on both numeric and non-numeric
questions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要