conference paper yes Iris Gräßler author 475650000-0001-5765-971X Jan Niklas Pfeifer author 62841 IrisGraessler editor International Symposium on Hybrid Intelligence in Product and Production Engineering 1. 2026 Paderborn Manufacturers operate under dynamic regulatory conditions, leading to frequently changing requirements that must be managed with high effort. AI-based requirements extraction using Large Language Models (LLMs) has the potential to support engineers in this task by analyzing large documents, for example. Existing evaluation approaches in AI-supported requirements engineering mainly assess individual requirements and their consistency with source documents, while quality criteria of requirements sets are largely neglected. This paper presents a five-step approach to evaluate the performance of current LLMs in generating requirements sets: a systematic literature review to identify quality criteria of requirements sets (1), exemplary generation of requirements sets from the Machinery Directive 2006/42/EC using LLMs (2), examination (3), evaluation (4) and comparison (5) of the generated requirements sets. The results support engineers in selecting LLMs for creating requirement sets and in evaluating future LLMs. Universitätsbibliothek2026Paderborn eng Requirements EngineeringArtificial IntelligenceLarge Language ModelsGenerative AI 10.17619/UNIPB/1-2646 Gräßler I, Pfeifer JN. Performance evaluation of large language models in extracting requirements sets from technical documents. In: Graessler I, ed. <i>1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University</i>. Universitätsbibliothek; 2026. doi:<a href="https://doi.org/10.17619/UNIPB/1-2646">10.17619/UNIPB/1-2646</a> @inproceedings{Gräßler_Pfeifer_2026, title={Performance evaluation of large language models in extracting requirements sets from technical documents}, DOI={<a href="https://doi.org/10.17619/UNIPB/1-2646">10.17619/UNIPB/1-2646</a>}, booktitle={1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University}, publisher={Universitätsbibliothek}, author={Gräßler, Iris and Pfeifer, Jan Niklas}, editor={Graessler, Iris}, year={2026} } Gräßler, Iris, and Jan Niklas Pfeifer. “Performance Evaluation of Large Language Models in Extracting Requirements Sets from Technical Documents.” <i>1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University</i>, edited by Iris Graessler, Universitätsbibliothek, 2026, doi:<a href="https://doi.org/10.17619/UNIPB/1-2646">10.17619/UNIPB/1-2646</a>. I. Gräßler, J.N. Pfeifer, in: I. Graessler (Ed.), 1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University, Universitätsbibliothek, 2026. Gräßler, Iris, and Jan Niklas Pfeifer. “Performance Evaluation of Large Language Models in Extracting Requirements Sets from Technical Documents.” In <i>1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University</i>, edited by Iris Graessler. Universitätsbibliothek, 2026. <a href="https://doi.org/10.17619/UNIPB/1-2646">https://doi.org/10.17619/UNIPB/1-2646</a>. Gräßler, I., &#38; Pfeifer, J. N. (2026). Performance evaluation of large language models in extracting requirements sets from technical documents. In I. Graessler (Ed.), <i>1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University</i>. Universitätsbibliothek. <a href="https://doi.org/10.17619/UNIPB/1-2646">https://doi.org/10.17619/UNIPB/1-2646</a> I. Gräßler and J. N. Pfeifer, “Performance evaluation of large language models in extracting requirements sets from technical documents,” in <i>1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University</i>, Paderborn, 2026, doi: <a href="https://doi.org/10.17619/UNIPB/1-2646">10.17619/UNIPB/1-2646</a>. 668372026-08-24T14:31:30Z2026-08-24T14:41:41Z