Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks
arxiv(2024)
摘要
We investigate the use of Natural Language Inference (NLI) in automating
requirements engineering tasks. In particular, we focus on three tasks:
requirements classification, identification of requirements specification
defects, and detection of conflicts in stakeholders' requirements. While
previous research has demonstrated significant benefit in using NLI as a
universal method for a broad spectrum of natural language processing tasks,
these advantages have not been investigated within the context of software
requirements engineering. Therefore, we design experiments to evaluate the use
of NLI in requirements analysis. We compare the performance of NLI with a
spectrum of approaches, including prompt-based models, conventional transfer
learning, Large Language Models (LLMs)-powered chatbot models, and
probabilistic models. Through experiments conducted under various learning
settings including conventional learning and zero-shot, we demonstrate
conclusively that our NLI method surpasses classical NLP methods as well as
other LLMs-based and chatbot models in the analysis of requirements
specifications. Additionally, we share lessons learned characterizing the
learning settings that make NLI a suitable approach for automating requirements
engineering tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要