---
_id: '60990'
abstract:
- lang: eng
text: 'Large Language Models (LLMs) have demonstrated remarkable performance across
a wide range of natural language processing tasks. However, their effectiveness
in low-resource languages remains underexplored, particularly in complex tasks
such as end-to-end Entity Linking (EL), which requires both mention detection
and disambiguation against a knowledge base (KB). In earlier work, we introduced
IndEL — the first end-to-end EL benchmark dataset for the Indonesian language
— covering both a general domain (news) and a specific domain (religious text
from the Indonesian translation of the Quran), and evaluated four traditional
end-to-end EL systems on this dataset. In this study, we propose ELEVATE-ID, a
comprehensive evaluation framework for assessing LLM performance on end-to-end
EL in Indonesian. The framework evaluates LLMs under both zero-shot and fine-tuned
conditions, using multilingual and Indonesian monolingual models, with Wikidata
as the target KB. Our experiments include performance benchmarking, generalization
analysis across domains, and systematic error analysis. Results show that GPT-4
and GPT-3.5 achieve the highest accuracy in zero-shot and fine-tuned settings,
respectively. However, even fine-tuned GPT-3.5 underperforms compared to DBpedia
Spotlight — the weakest of the traditional model baselines — in the general domain.
Interestingly, GPT-3.5 outperforms Babelfy in the specific domain. Generalization
analysis indicates that fine-tuned GPT-3.5 adapts more effectively to cross-domain
and mixed-domain scenarios. Error analysis uncovers persistent challenges that
hinder LLM performance: difficulties with non-complete mentions, acronym disambiguation,
and full-name recognition in formal contexts. These issues point to limitations
in mention boundary detection and contextual grounding. Indonesian-pretrained
LLMs, Komodo and Merak, reveal core weaknesses: template leakage and entity hallucination,
respectively—underscoring architectural and training limitations in low-resource
end-to-end EL.11Code and dataset are available at https://github.com/dice-group/ELEVATE-ID.'
article_type: original
author:
- first_name: Ria Hari
full_name: Gusmita, Ria Hari
id: '71039'
last_name: Gusmita
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Hamada Mohamed Abdelsamee
full_name: Zahera, Hamada Mohamed Abdelsamee
id: '72768'
last_name: Zahera
orcid: 0000-0003-0215-1278
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Gusmita RH, Firmansyah AF, Zahera HMA, Ngonga Ngomo A-C. ELEVATE-ID: Extending
Large Language Models for End-to-End Entity Linking Evaluation in Indonesian.
Data & Knowledge Engineering. 2026;161:102504. doi:https://doi.org/10.1016/j.datak.2025.102504'
apa: 'Gusmita, R. H., Firmansyah, A. F., Zahera, H. M. A., & Ngonga Ngomo, A.-C.
(2026). ELEVATE-ID: Extending Large Language Models for End-to-End Entity Linking
Evaluation in Indonesian. Data & Knowledge Engineering, 161,
102504. https://doi.org/10.1016/j.datak.2025.102504'
bibtex: '@article{Gusmita_Firmansyah_Zahera_Ngonga Ngomo_2026, title={ELEVATE-ID:
Extending Large Language Models for End-to-End Entity Linking Evaluation in Indonesian},
volume={161}, DOI={https://doi.org/10.1016/j.datak.2025.102504},
journal={Data & Knowledge Engineering}, author={Gusmita, Ria Hari and Firmansyah,
Asep Fajar and Zahera, Hamada Mohamed Abdelsamee and Ngonga Ngomo, Axel-Cyrille},
year={2026}, pages={102504} }'
chicago: 'Gusmita, Ria Hari, Asep Fajar Firmansyah, Hamada Mohamed Abdelsamee Zahera,
and Axel-Cyrille Ngonga Ngomo. “ELEVATE-ID: Extending Large Language Models for
End-to-End Entity Linking Evaluation in Indonesian.” Data & Knowledge Engineering
161 (2026): 102504. https://doi.org/10.1016/j.datak.2025.102504.'
ieee: 'R. H. Gusmita, A. F. Firmansyah, H. M. A. Zahera, and A.-C. Ngonga Ngomo,
“ELEVATE-ID: Extending Large Language Models for End-to-End Entity Linking Evaluation
in Indonesian,” Data & Knowledge Engineering, vol. 161, p. 102504,
2026, doi: https://doi.org/10.1016/j.datak.2025.102504.'
mla: 'Gusmita, Ria Hari, et al. “ELEVATE-ID: Extending Large Language Models for
End-to-End Entity Linking Evaluation in Indonesian.” Data & Knowledge Engineering,
vol. 161, 2026, p. 102504, doi:https://doi.org/10.1016/j.datak.2025.102504.'
short: R.H. Gusmita, A.F. Firmansyah, H.M.A. Zahera, A.-C. Ngonga Ngomo, Data &
Knowledge Engineering 161 (2026) 102504.
date_created: 2025-08-24T11:38:51Z
date_updated: 2025-08-25T09:40:13Z
department:
- _id: '574'
doi: https://doi.org/10.1016/j.datak.2025.102504
intvolume: ' 161'
keyword:
- LLMs
- Evaluation
- End-to-end EL
- Indonesian
language:
- iso: eng
main_file_link:
- url: https://www.sciencedirect.com/science/article/pii/S0169023X25000990?utm_campaign=STMJ_220042_AUTH_SERV_PA&utm_medium=email&utm_acid=78351008&SIS_ID=&dgcid=STMJ_220042_AUTH_SERV_PA&CMX_ID=&utm_in=DM591673&utm_source=AC_
page: '102504'
publication: Data & Knowledge Engineering
publication_identifier:
issn:
- 0169-023X
status: public
title: 'ELEVATE-ID: Extending Large Language Models for End-to-End Entity Linking
Evaluation in Indonesian'
type: journal_article
user_id: '71039'
volume: 161
year: '2026'
...
---
_id: '63027'
author:
- first_name: Reda
full_name: Ihtassine, Reda
id: '103911'
last_name: Ihtassine
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Nikit
full_name: Srivastava, Nikit
id: '70066'
last_name: Srivastava
orcid: 0009-0004-5164-4911
- first_name: Ali
full_name: Manzoor, Ali
id: '77309'
last_name: Manzoor
orcid: https://orcid.org/0000-0001-8403-5160
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
- first_name: Mohamed Ahmed
full_name: Sherif, Mohamed Ahmed
last_name: Sherif
citation:
ama: 'Ihtassine R, Firmansyah AF, Srivastava N, Manzoor A, Ngonga Ngomo A-C, Sherif
MA. NL2LS: LLM-based Automatic Linking of Knowledge Graphs. In: Proceedings
of the Knowledge Capture Conference 2025. ACM; 2025. doi:10.1145/3731443.3771374'
apa: 'Ihtassine, R., Firmansyah, A. F., Srivastava, N., Manzoor, A., Ngonga Ngomo,
A.-C., & Sherif, M. A. (2025). NL2LS: LLM-based Automatic Linking of Knowledge
Graphs. Proceedings of the Knowledge Capture Conference 2025. https://doi.org/10.1145/3731443.3771374'
bibtex: '@inproceedings{Ihtassine_Firmansyah_Srivastava_Manzoor_Ngonga Ngomo_Sherif_2025,
title={NL2LS: LLM-based Automatic Linking of Knowledge Graphs}, DOI={10.1145/3731443.3771374},
booktitle={Proceedings of the Knowledge Capture Conference 2025}, publisher={ACM},
author={Ihtassine, Reda and Firmansyah, Asep Fajar and Srivastava, Nikit and Manzoor,
Ali and Ngonga Ngomo, Axel-Cyrille and Sherif, Mohamed Ahmed}, year={2025} }'
chicago: 'Ihtassine, Reda, Asep Fajar Firmansyah, Nikit Srivastava, Ali Manzoor,
Axel-Cyrille Ngonga Ngomo, and Mohamed Ahmed Sherif. “NL2LS: LLM-Based Automatic
Linking of Knowledge Graphs.” In Proceedings of the Knowledge Capture Conference
2025. ACM, 2025. https://doi.org/10.1145/3731443.3771374.'
ieee: 'R. Ihtassine, A. F. Firmansyah, N. Srivastava, A. Manzoor, A.-C. Ngonga Ngomo,
and M. A. Sherif, “NL2LS: LLM-based Automatic Linking of Knowledge Graphs,” 2025,
doi: 10.1145/3731443.3771374.'
mla: 'Ihtassine, Reda, et al. “NL2LS: LLM-Based Automatic Linking of Knowledge Graphs.”
Proceedings of the Knowledge Capture Conference 2025, ACM, 2025, doi:10.1145/3731443.3771374.'
short: 'R. Ihtassine, A.F. Firmansyah, N. Srivastava, A. Manzoor, A.-C. Ngonga Ngomo,
M.A. Sherif, in: Proceedings of the Knowledge Capture Conference 2025, ACM, 2025.'
date_created: 2025-12-10T17:19:20Z
date_updated: 2026-03-09T12:42:07Z
doi: 10.1145/3731443.3771374
language:
- iso: eng
publication: Proceedings of the Knowledge Capture Conference 2025
publication_status: published
publisher: ACM
status: public
title: 'NL2LS: LLM-based Automatic Linking of Knowledge Graphs'
type: conference
user_id: '77309'
year: '2025'
...
---
_id: '59054'
author:
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Hamada Mohamed Abdelsamee
full_name: Zahera, Hamada Mohamed Abdelsamee
id: '72768'
last_name: Zahera
orcid: 0000-0003-0215-1278
- first_name: Mohamed
full_name: Sherif, Mohamed
id: '67234'
last_name: Sherif
orcid: https://orcid.org/0000-0002-9927-2203
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Firmansyah AF, Zahera HMA, Sherif M, Moussallem D, Ngonga Ngomo A-C. ANTS:
Abstractive Entity Summarization in Knowledge Graphs. In: ESWC2025. The
Semantic Web. pringer Nature Switzerland; 2025:133--151. doi:10.1007/978-3-031-94575-5_8'
apa: 'Firmansyah, A. F., Zahera, H. M. A., Sherif, M., Moussallem, D., & Ngonga
Ngomo, A.-C. (2025). ANTS: Abstractive Entity Summarization in Knowledge Graphs.
ESWC2025, 133--151. https://doi.org/10.1007/978-3-031-94575-5_8'
bibtex: '@inproceedings{Firmansyah_Zahera_Sherif_Moussallem_Ngonga Ngomo_2025, series={The
Semantic Web}, title={ANTS: Abstractive Entity Summarization in Knowledge Graphs},
DOI={10.1007/978-3-031-94575-5_8},
booktitle={ESWC2025}, publisher={pringer Nature Switzerland}, author={Firmansyah,
Asep Fajar and Zahera, Hamada Mohamed Abdelsamee and Sherif, Mohamed and Moussallem,
Diego and Ngonga Ngomo, Axel-Cyrille}, year={2025}, pages={133--151}, collection={The
Semantic Web} }'
chicago: 'Firmansyah, Asep Fajar, Hamada Mohamed Abdelsamee Zahera, Mohamed Sherif,
Diego Moussallem, and Axel-Cyrille Ngonga Ngomo. “ANTS: Abstractive Entity Summarization
in Knowledge Graphs.” In ESWC2025, 133--151. The Semantic Web. pringer
Nature Switzerland, 2025. https://doi.org/10.1007/978-3-031-94575-5_8.'
ieee: 'A. F. Firmansyah, H. M. A. Zahera, M. Sherif, D. Moussallem, and A.-C. Ngonga
Ngomo, “ANTS: Abstractive Entity Summarization in Knowledge Graphs,” in ESWC2025,
2025, pp. 133--151, doi: 10.1007/978-3-031-94575-5_8.'
mla: 'Firmansyah, Asep Fajar, et al. “ANTS: Abstractive Entity Summarization in
Knowledge Graphs.” ESWC2025, pringer Nature Switzerland, 2025, pp. 133--151,
doi:10.1007/978-3-031-94575-5_8.'
short: 'A.F. Firmansyah, H.M.A. Zahera, M. Sherif, D. Moussallem, A.-C. Ngonga Ngomo,
in: ESWC2025, pringer Nature Switzerland, 2025, pp. 133--151.'
conference:
name: ESWC 2025
date_created: 2025-03-17T13:41:48Z
date_updated: 2025-09-11T10:00:44Z
department:
- _id: '574'
doi: 10.1007/978-3-031-94575-5_8
keyword:
- firmansyah mousallem ngonga sherif zahera
language:
- iso: eng
page: 133--151
publication: ESWC2025
publication_identifier:
isbn:
- 978-3-031-94575-5
publication_status: published
publisher: pringer Nature Switzerland
series_title: The Semantic Web
status: public
title: 'ANTS: Abstractive Entity Summarization in Knowledge Graphs'
type: conference
user_id: '67234'
year: '2025'
...
---
_id: '62119'
author:
- first_name: Reda
full_name: Ihtassine, Reda
id: '103911'
last_name: Ihtassine
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Nikit
full_name: Srivastava, Nikit
id: '70066'
last_name: Srivastava
orcid: 0009-0004-5164-4911
- first_name: Manzoor
full_name: Ali, Manzoor
last_name: Ali
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
- first_name: Mohamed
full_name: Sherif, Mohamed
id: '67234'
last_name: Sherif
orcid: https://orcid.org/0000-0002-9927-2203
citation:
ama: 'Ihtassine R, Firmansyah AF, Srivastava N, Ali M, Ngonga Ngomo A-C, Sherif
M. NL2LS: LLM-based Automatic Linking of Knowledge Graphs. In: Proceedings
of the 12th Knowledge Capture Conference 2025, {K-CAP} 2025, The Thirteenth International
Conference on Knowledge Capture, December 10 - 12, 2025, Dayton, Ohio, USA.
ACM; 2025.'
apa: 'Ihtassine, R., Firmansyah, A. F., Srivastava, N., Ali, M., Ngonga Ngomo, A.-C.,
& Sherif, M. (2025). NL2LS: LLM-based Automatic Linking of Knowledge Graphs.
Proceedings of the 12th Knowledge Capture Conference 2025, {K-CAP} 2025, The
Thirteenth International Conference on Knowledge Capture, December 10 - 12, 2025,
Dayton, Ohio, USA.'
bibtex: '@inproceedings{Ihtassine_Firmansyah_Srivastava_Ali_Ngonga Ngomo_Sherif_2025,
title={NL2LS: LLM-based Automatic Linking of Knowledge Graphs}, booktitle={Proceedings
of the 12th Knowledge Capture Conference 2025, {K-CAP} 2025, The Thirteenth International
Conference on Knowledge Capture, December 10 - 12, 2025, Dayton, Ohio, USA}, publisher={ACM},
author={Ihtassine, Reda and Firmansyah, Asep Fajar and Srivastava, Nikit and Ali,
Manzoor and Ngonga Ngomo, Axel-Cyrille and Sherif, Mohamed}, year={2025} }'
chicago: 'Ihtassine, Reda, Asep Fajar Firmansyah, Nikit Srivastava, Manzoor Ali,
Axel-Cyrille Ngonga Ngomo, and Mohamed Sherif. “NL2LS: LLM-Based Automatic Linking
of Knowledge Graphs.” In Proceedings of the 12th Knowledge Capture Conference
2025, {K-CAP} 2025, The Thirteenth International Conference on Knowledge Capture,
December 10 - 12, 2025, Dayton, Ohio, USA. ACM, 2025.'
ieee: 'R. Ihtassine, A. F. Firmansyah, N. Srivastava, M. Ali, A.-C. Ngonga Ngomo,
and M. Sherif, “NL2LS: LLM-based Automatic Linking of Knowledge Graphs,” 2025.'
mla: 'Ihtassine, Reda, et al. “NL2LS: LLM-Based Automatic Linking of Knowledge Graphs.”
Proceedings of the 12th Knowledge Capture Conference 2025, {K-CAP} 2025, The
Thirteenth International Conference on Knowledge Capture, December 10 - 12, 2025,
Dayton, Ohio, USA, ACM, 2025.'
short: 'R. Ihtassine, A.F. Firmansyah, N. Srivastava, M. Ali, A.-C. Ngonga Ngomo,
M. Sherif, in: Proceedings of the 12th Knowledge Capture Conference 2025, {K-CAP}
2025, The Thirteenth International Conference on Knowledge Capture, December 10
- 12, 2025, Dayton, Ohio, USA, ACM, 2025.'
date_created: 2025-11-06T15:13:39Z
date_updated: 2025-11-06T15:19:28Z
department:
- _id: '574'
keyword:
- Srivastava ali dice enexa firmansyah ihtassine ngonga sailproject sherif whale
language:
- iso: eng
publication: Proceedings of the 12th Knowledge Capture Conference 2025, {K-CAP} 2025,
The Thirteenth International Conference on Knowledge Capture, December 10 - 12,
2025, Dayton, Ohio, USA
publisher: ACM
status: public
title: 'NL2LS: LLM-based Automatic Linking of Knowledge Graphs'
type: conference
user_id: '76787'
year: '2025'
...
---
_id: '54412'
author:
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Firmansyah AF, Moussallem D, Ngonga Ngomo A-C. ESLM: Improving Entity Summarization
by Leveraging Language Models. In: The Semantic Web. Springer Nature Switzerland;
2024. doi:10.1007/978-3-031-60626-7_9'
apa: 'Firmansyah, A. F., Moussallem, D., & Ngonga Ngomo, A.-C. (2024). ESLM:
Improving Entity Summarization by Leveraging Language Models. In The Semantic
Web. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-60626-7_9'
bibtex: '@inbook{Firmansyah_Moussallem_Ngonga Ngomo_2024, place={Cham}, title={ESLM:
Improving Entity Summarization by Leveraging Language Models}, DOI={10.1007/978-3-031-60626-7_9},
booktitle={The Semantic Web}, publisher={Springer Nature Switzerland}, author={Firmansyah,
Asep Fajar and Moussallem, Diego and Ngonga Ngomo, Axel-Cyrille}, year={2024}
}'
chicago: 'Firmansyah, Asep Fajar, Diego Moussallem, and Axel-Cyrille Ngonga Ngomo.
“ESLM: Improving Entity Summarization by Leveraging Language Models.” In The
Semantic Web. Cham: Springer Nature Switzerland, 2024. https://doi.org/10.1007/978-3-031-60626-7_9.'
ieee: 'A. F. Firmansyah, D. Moussallem, and A.-C. Ngonga Ngomo, “ESLM: Improving
Entity Summarization by Leveraging Language Models,” in The Semantic Web,
Cham: Springer Nature Switzerland, 2024.'
mla: 'Firmansyah, Asep Fajar, et al. “ESLM: Improving Entity Summarization by Leveraging
Language Models.” The Semantic Web, Springer Nature Switzerland, 2024,
doi:10.1007/978-3-031-60626-7_9.'
short: 'A.F. Firmansyah, D. Moussallem, A.-C. Ngonga Ngomo, in: The Semantic Web,
Springer Nature Switzerland, Cham, 2024.'
date_created: 2024-05-22T09:00:03Z
date_updated: 2024-06-04T11:09:23Z
department:
- _id: '574'
- _id: '34'
doi: 10.1007/978-3-031-60626-7_9
language:
- iso: eng
place: Cham
publication: The Semantic Web
publication_identifier:
isbn:
- '9783031606250'
- '9783031606267'
issn:
- 0302-9743
- 1611-3349
publication_status: published
publisher: Springer Nature Switzerland
status: public
title: 'ESLM: Improving Entity Summarization by Leveraging Language Models'
type: book_chapter
user_id: '76787'
year: '2024'
...
---
_id: '46516'
abstract:
- lang: eng
text: Linked knowledge graphs build the backbone of many data-driven applications
such as search engines, conversational agents and e-commerce solutions. Declarative
link discovery frameworks use complex link specifications to express the conditions
under which a link between two resources can be deemed to exist. However, understanding
such complex link specifications is a challenging task for non-expert users of
link discovery frameworks. In this paper, we address this drawback by devising
NMV-LS, a language model-based verbalization approach for translating complex
link specifications into natural language. NMV-LS relies on the results of rule-based
link specification verbalization to apply continuous training on T5, a large language
model based on the Transformerarchitecture. We evaluated NMV-LS on English and
German datasets using well-known machine translation metrics such as BLUE, METEOR,
ChrF++ and TER. Our results suggest that our approach achieves a verbalization
performance close to that of humans and outperforms state of the art approaches.
Our source code and datasets are publicly available at https://github.com/dice-group/NMV-LS.
author:
- first_name: Abdullah Fathi Ahmed
full_name: Ahmed, Abdullah Fathi Ahmed
id: '29670'
last_name: Ahmed
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Mohamed
full_name: Sherif, Mohamed
id: '67234'
last_name: Sherif
orcid: https://orcid.org/0000-0002-9927-2203
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Ahmed AFA, Firmansyah AF, Sherif M, Moussallem D, Ngonga Ngomo A-C. Explainable
Integration of Knowledge Graphs Using Large Language Models. In: Natural Language
Processing and Information Systems. Springer Nature Switzerland; 2023. doi:10.1007/978-3-031-35320-8_9'
apa: Ahmed, A. F. A., Firmansyah, A. F., Sherif, M., Moussallem, D., & Ngonga
Ngomo, A.-C. (2023). Explainable Integration of Knowledge Graphs Using Large Language
Models. In Natural Language Processing and Information Systems. Springer
Nature Switzerland. https://doi.org/10.1007/978-3-031-35320-8_9
bibtex: '@inbook{Ahmed_Firmansyah_Sherif_Moussallem_Ngonga Ngomo_2023, place={Cham},
title={Explainable Integration of Knowledge Graphs Using Large Language Models},
DOI={10.1007/978-3-031-35320-8_9},
booktitle={Natural Language Processing and Information Systems}, publisher={Springer
Nature Switzerland}, author={Ahmed, Abdullah Fathi Ahmed and Firmansyah, Asep
Fajar and Sherif, Mohamed and Moussallem, Diego and Ngonga Ngomo, Axel-Cyrille},
year={2023} }'
chicago: 'Ahmed, Abdullah Fathi Ahmed, Asep Fajar Firmansyah, Mohamed Sherif, Diego
Moussallem, and Axel-Cyrille Ngonga Ngomo. “Explainable Integration of Knowledge
Graphs Using Large Language Models.” In Natural Language Processing and Information
Systems. Cham: Springer Nature Switzerland, 2023. https://doi.org/10.1007/978-3-031-35320-8_9.'
ieee: 'A. F. A. Ahmed, A. F. Firmansyah, M. Sherif, D. Moussallem, and A.-C. Ngonga
Ngomo, “Explainable Integration of Knowledge Graphs Using Large Language Models,”
in Natural Language Processing and Information Systems, Cham: Springer
Nature Switzerland, 2023.'
mla: Ahmed, Abdullah Fathi Ahmed, et al. “Explainable Integration of Knowledge Graphs
Using Large Language Models.” Natural Language Processing and Information Systems,
Springer Nature Switzerland, 2023, doi:10.1007/978-3-031-35320-8_9.
short: 'A.F.A. Ahmed, A.F. Firmansyah, M. Sherif, D. Moussallem, A.-C. Ngonga Ngomo,
in: Natural Language Processing and Information Systems, Springer Nature Switzerland,
Cham, 2023.'
date_created: 2023-08-16T08:57:11Z
date_updated: 2024-06-04T12:23:45Z
department:
- _id: '34'
- _id: '574'
doi: 10.1007/978-3-031-35320-8_9
language:
- iso: eng
place: Cham
publication: Natural Language Processing and Information Systems
publication_identifier:
isbn:
- '9783031353192'
- '9783031353208'
issn:
- 0302-9743
- 1611-3349
publication_status: published
publisher: Springer Nature Switzerland
status: public
title: Explainable Integration of Knowledge Graphs Using Large Language Models
type: book_chapter
user_id: '76787'
year: '2023'
...
---
_id: '46572'
abstract:
- lang: eng
text: Indonesian is classified as underrepresented in the Natural Language Processing
(NLP) field, despite being the tenth most spoken language in the world with 198
million speakers. The paucity of datasets is recognized as the main reason for
the slow advancements in NLP research for underrepresented languages. Significant
attempts were made in 2020 to address this drawback for Indonesian. The Indonesian
Natural Language Understanding (IndoNLU) benchmark was introduced alongside IndoBERT
pre-trained language model. The second benchmark, Indonesian Language Evaluation
Montage (IndoLEM), was presented in the same year. These benchmarks support several
tasks, including Named Entity Recognition (NER). However, all NER datasets are
in the public domain and do not contain domain-specific datasets. To alleviate
this drawback, we introduce IndQNER, a manually annotated NER benchmark dataset
in the religious domain that adheres to a meticulously designed annotation guideline.
Since Indonesia has the world’s largest Muslim population, we build the dataset
from the Indonesian translation of the Quran. The dataset includes 2475 named
entities representing 18 different classes. To assess the annotation quality of
IndQNER, we perform experiments with BiLSTM and CRF-based NER, as well as IndoBERT
fine-tuning. The results reveal that the first model outperforms the second model
achieving 0.98 F1 points. This outcome indicates that IndQNER may be an acceptable
evaluation metric for Indonesian NER tasks in the aforementioned domain, widening
the research’s domain range.
author:
- first_name: Ria Hari
full_name: Gusmita, Ria Hari
id: '71039'
last_name: Gusmita
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Gusmita RH, Firmansyah AF, Moussallem D, Ngonga Ngomo A-C. IndQNER: Named
Entity Recognition Benchmark Dataset from the Indonesian Translation of the Quran.
In: Natural Language Processing and Information Systems. Springer Nature
Switzerland; 2023. doi:10.1007/978-3-031-35320-8_12'
apa: 'Gusmita, R. H., Firmansyah, A. F., Moussallem, D., & Ngonga Ngomo, A.-C.
(2023). IndQNER: Named Entity Recognition Benchmark Dataset from the Indonesian
Translation of the Quran. In Natural Language Processing and Information Systems.
International Conference on Applications of Natural Language to Information Systems
(NLDB) 2023, Derby, UK. Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-35320-8_12'
bibtex: '@inbook{Gusmita_Firmansyah_Moussallem_Ngonga Ngomo_2023, place={Cham},
title={IndQNER: Named Entity Recognition Benchmark Dataset from the Indonesian
Translation of the Quran}, DOI={10.1007/978-3-031-35320-8_12},
booktitle={Natural Language Processing and Information Systems}, publisher={Springer
Nature Switzerland}, author={Gusmita, Ria Hari and Firmansyah, Asep Fajar and
Moussallem, Diego and Ngonga Ngomo, Axel-Cyrille}, year={2023} }'
chicago: 'Gusmita, Ria Hari, Asep Fajar Firmansyah, Diego Moussallem, and Axel-Cyrille
Ngonga Ngomo. “IndQNER: Named Entity Recognition Benchmark Dataset from the Indonesian
Translation of the Quran.” In Natural Language Processing and Information Systems.
Cham: Springer Nature Switzerland, 2023. https://doi.org/10.1007/978-3-031-35320-8_12.'
ieee: 'R. H. Gusmita, A. F. Firmansyah, D. Moussallem, and A.-C. Ngonga Ngomo, “IndQNER:
Named Entity Recognition Benchmark Dataset from the Indonesian Translation of the Quran,”
in Natural Language Processing and Information Systems, Cham: Springer
Nature Switzerland, 2023.'
mla: 'Gusmita, Ria Hari, et al. “IndQNER: Named Entity Recognition Benchmark Dataset
from the Indonesian Translation of the Quran.” Natural Language Processing
and Information Systems, Springer Nature Switzerland, 2023, doi:10.1007/978-3-031-35320-8_12.'
short: 'R.H. Gusmita, A.F. Firmansyah, D. Moussallem, A.-C. Ngonga Ngomo, in: Natural
Language Processing and Information Systems, Springer Nature Switzerland, Cham,
2023.'
conference:
end_date: 2023-06-23
location: Derby, UK
name: International Conference on Applications of Natural Language to Information
Systems (NLDB) 2023
start_date: 2023-06-21
date_created: 2023-08-17T12:41:45Z
date_updated: 2024-11-19T15:41:34Z
department:
- _id: '34'
- _id: '574'
doi: 10.1007/978-3-031-35320-8_12
keyword:
- NER benchmark dataset
- Indonesian
- specific domain
language:
- iso: eng
place: Cham
publication: Natural Language Processing and Information Systems
publication_identifier:
isbn:
- '9783031353192'
- '9783031353208'
issn:
- 0302-9743
- 1611-3349
publication_status: published
publisher: Springer Nature Switzerland
related_material:
link:
- relation: confirmation
url: https://link.springer.com/chapter/10.1007/978-3-031-35320-8_12
status: public
title: 'IndQNER: Named Entity Recognition Benchmark Dataset from the Indonesian Translation
of the Quran'
type: book_chapter
user_id: '71039'
year: '2023'
...
---
_id: '29486'
author:
- first_name: Asep Fajar
full_name: Firmansyah, Asep Fajar
id: '76787'
last_name: Firmansyah
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Firmansyah AF, Moussallem D, Ngonga Ngomo A-C. GATES: Using Graph Attention
Networks for Entity Summarization. In: Proceedings of the 11th on Knowledge
Capture Conference. ACM; 2021:73–80. doi:10.1145/3460210.3493574'
apa: 'Firmansyah, A. F., Moussallem, D., & Ngonga Ngomo, A.-C. (2021). GATES:
Using Graph Attention Networks for Entity Summarization. Proceedings of the
11th on Knowledge Capture Conference, 73–80. https://doi.org/10.1145/3460210.3493574'
bibtex: '@inproceedings{Firmansyah_Moussallem_Ngonga Ngomo_2021, place={Virtual
Event, USA}, title={GATES: Using Graph Attention Networks for Entity Summarization},
DOI={10.1145/3460210.3493574},
booktitle={Proceedings of the 11th on Knowledge Capture Conference}, publisher={ACM},
author={Firmansyah, Asep Fajar and Moussallem, Diego and Ngonga Ngomo, Axel-Cyrille},
year={2021}, pages={73–80} }'
chicago: 'Firmansyah, Asep Fajar, Diego Moussallem, and Axel-Cyrille Ngonga Ngomo.
“GATES: Using Graph Attention Networks for Entity Summarization.” In Proceedings
of the 11th on Knowledge Capture Conference, 73–80. Virtual Event, USA: ACM,
2021. https://doi.org/10.1145/3460210.3493574.'
ieee: 'A. F. Firmansyah, D. Moussallem, and A.-C. Ngonga Ngomo, “GATES: Using Graph
Attention Networks for Entity Summarization,” in Proceedings of the 11th on
Knowledge Capture Conference, 2021, pp. 73–80, doi: 10.1145/3460210.3493574.'
mla: 'Firmansyah, Asep Fajar, et al. “GATES: Using Graph Attention Networks for
Entity Summarization.” Proceedings of the 11th on Knowledge Capture Conference,
ACM, 2021, pp. 73–80, doi:10.1145/3460210.3493574.'
short: 'A.F. Firmansyah, D. Moussallem, A.-C. Ngonga Ngomo, in: Proceedings of the
11th on Knowledge Capture Conference, ACM, Virtual Event, USA, 2021, pp. 73–80.'
date_created: 2022-01-20T08:37:12Z
date_updated: 2022-01-20T09:03:38Z
department:
- _id: '574'
doi: 10.1145/3460210.3493574
language:
- iso: eng
page: 73–80
place: Virtual Event, USA
publication: Proceedings of the 11th on Knowledge Capture Conference
publication_identifier:
isbn:
- 978-1-4503-8457-5
publication_status: published
publisher: ACM
status: public
title: 'GATES: Using Graph Attention Networks for Entity Summarization'
type: conference
user_id: '76787'
year: '2021'
...