{"publication":"Semantic Systems. The Power of AI and Knowledge Graphs","ddc":["004"],"publisher":"Springer International Publishing","date_created":"2024-10-10T14:25:00Z","doi":"10.1007/978-3-030-33220-4_25","citation":{"ama":"Gusmita RH, Jalota R, Vollmers D, Reineke J, Ngonga Ngomo A-C, Usbeck R. QUANT - Question Answering Benchmark Curator. In: Acosta M, Cudr{\\’e}-Mauroux P, Maleshkova M, Pellegrini T, Sack H, Sure-Vetter Y, eds. <i>Semantic Systems. The Power of AI and Knowledge Graphs</i>. Springer International Publishing; 2019:343--358. doi:<a href=\"https://doi.org/10.1007/978-3-030-33220-4_25\">10.1007/978-3-030-33220-4_25</a>","apa":"Gusmita, R. H., Jalota, R., Vollmers, D., Reineke, J., Ngonga Ngomo, A.-C., &#38; Usbeck, R. (2019). QUANT - Question Answering Benchmark Curator. In M. Acosta, P. Cudr{\\’e}-Mauroux, M. Maleshkova, T. Pellegrini, H. Sack, &#38; Y. Sure-Vetter (Eds.), <i>Semantic Systems. The Power of AI and Knowledge Graphs</i> (pp. 343--358). Springer International Publishing. <a href=\"https://doi.org/10.1007/978-3-030-33220-4_25\">https://doi.org/10.1007/978-3-030-33220-4_25</a>","ieee":"R. H. Gusmita, R. Jalota, D. Vollmers, J. Reineke, A.-C. Ngonga Ngomo, and R. Usbeck, “QUANT - Question Answering Benchmark Curator,” in <i>Semantic Systems. The Power of AI and Knowledge Graphs</i>, M. Acosta, P. Cudr{\\’e}-Mauroux, M. Maleshkova, T. Pellegrini, H. Sack, and Y. Sure-Vetter, Eds. Cham: Springer International Publishing, 2019, pp. 343--358.","short":"R.H. Gusmita, R. Jalota, D. Vollmers, J. Reineke, A.-C. Ngonga Ngomo, R. Usbeck, in: M. Acosta, P. Cudr{\\’e}-Mauroux, M. Maleshkova, T. Pellegrini, H. Sack, Y. Sure-Vetter (Eds.), Semantic Systems. The Power of AI and Knowledge Graphs, Springer International Publishing, Cham, 2019, pp. 343--358.","mla":"Gusmita, Ria Hari, et al. “QUANT - Question Answering Benchmark Curator.” <i>Semantic Systems. The Power of AI and Knowledge Graphs</i>, edited by Maribel Acosta et al., Springer International Publishing, 2019, pp. 343--358, doi:<a href=\"https://doi.org/10.1007/978-3-030-33220-4_25\">10.1007/978-3-030-33220-4_25</a>.","bibtex":"@inbook{Gusmita_Jalota_Vollmers_Reineke_Ngonga Ngomo_Usbeck_2019, place={Cham}, title={QUANT - Question Answering Benchmark Curator}, DOI={<a href=\"https://doi.org/10.1007/978-3-030-33220-4_25\">10.1007/978-3-030-33220-4_25</a>}, booktitle={Semantic Systems. The Power of AI and Knowledge Graphs}, publisher={Springer International Publishing}, author={Gusmita, Ria Hari and Jalota, Rricha and Vollmers, Daniel and Reineke, Jan and Ngonga Ngomo, Axel-Cyrille and Usbeck, Ricardo}, editor={Acosta, Maribel and Cudr{\\’e}-Mauroux, Philippe and Maleshkova, Maria and Pellegrini, Tassilo and Sack, Harald and Sure-Vetter, York}, year={2019}, pages={343--358} }","chicago":"Gusmita, Ria Hari, Rricha Jalota, Daniel Vollmers, Jan Reineke, Axel-Cyrille Ngonga Ngomo, and Ricardo Usbeck. “QUANT - Question Answering Benchmark Curator.” In <i>Semantic Systems. The Power of AI and Knowledge Graphs</i>, edited by Maribel Acosta, Philippe Cudr{\\’e}-Mauroux, Maria Maleshkova, Tassilo Pellegrini, Harald Sack, and York Sure-Vetter, 343--358. Cham: Springer International Publishing, 2019. <a href=\"https://doi.org/10.1007/978-3-030-33220-4_25\">https://doi.org/10.1007/978-3-030-33220-4_25</a>."},"language":[{"iso":"eng"}],"place":"Cham","date_updated":"2024-11-19T15:34:24Z","author":[{"full_name":"Gusmita, Ria Hari","first_name":"Ria Hari","id":"71039","last_name":"Gusmita"},{"first_name":"Rricha","last_name":"Jalota","full_name":"Jalota, Rricha"},{"last_name":"Vollmers","first_name":"Daniel","full_name":"Vollmers, Daniel"},{"first_name":"Jan","last_name":"Reineke","full_name":"Reineke, Jan"},{"full_name":"Ngonga Ngomo, Axel-Cyrille","first_name":"Axel-Cyrille","id":"65716","last_name":"Ngonga Ngomo"},{"full_name":"Usbeck, Ricardo","first_name":"Ricardo","last_name":"Usbeck"}],"has_accepted_license":"1","main_file_link":[{"open_access":"1","url":"https://rdcu.be/d0KJc"}],"file":[{"date_updated":"2024-11-19T15:28:45Z","file_id":"57244","access_level":"closed","file_name":"Gusmita_et_al-2019-Semantic_Systems._The_Power_of_AI_and_Knowledge_Graphs,_15th_International....pdf","date_created":"2024-11-19T15:28:45Z","success":1,"creator":"gusmita","file_size":1179910,"content_type":"application/pdf","relation":"main_file"}],"type":"book_chapter","year":"2019","keyword":["Benchmark","Question answering","Knowledge base"],"file_date_updated":"2024-11-19T15:28:45Z","title":"QUANT - Question Answering Benchmark Curator","oa":"1","user_id":"71039","conference":{"name":"SEMANTiCS 2019","end_date":"2019-09-12","location":"Karlsruhe, Germany","start_date":"2019-09-09"},"publication_identifier":{"isbn":["978-3-030-33219-8"],"issn":["0302-9743","1611-3349"],"eisbn":["978-3-030-33220-4"]},"publication_status":"published","editor":[{"last_name":"Acosta","first_name":"Maribel","full_name":"Acosta, Maribel"},{"last_name":"Cudr{\\'e}-Mauroux","first_name":"Philippe","full_name":"Cudr{\\'e}-Mauroux, Philippe"},{"first_name":"Maria","last_name":"Maleshkova","full_name":"Maleshkova, Maria"},{"full_name":"Pellegrini, Tassilo","first_name":"Tassilo","last_name":"Pellegrini"},{"first_name":"Harald","last_name":"Sack","full_name":"Sack, Harald"},{"last_name":"Sure-Vetter","first_name":"York","full_name":"Sure-Vetter, York"}],"_id":"56579","abstract":[{"lang":"eng","text":"Question answering engines have become one of the most popular type of applications driven by Semantic Web technologies. Consequently, the provision of means to quantify the performance of current question answering approaches on current datasets has become ever more important. However, a large percentage of the queries found in popular question answering benchmarks cannot be executed on current versions of their reference dataset. There is a consequently a clear need to curate question answering benchmarks periodically. However, the manual alteration of question answering benchmarks is often error-prone. We alleviate this problem by presenting QUANT, a novel framework for the creation and curation of question answering benchmarks. QUANT sup-ports the curation of benchmarks by generating smart edit suggestions for question-query pair and for the corresponding metadata. In addition, our framework supports the creation of new benchmark entries by pro-viding predefined quality checks for queries. We evaluate QUANT on 653questions obtained from QALD-1 to QALD-8 with 10 users. Our results show that our framework generates reliable suggestions and can reduce the curation effort for QA benchmarks by up to 91%."}],"status":"public","page":"343--358"}