---
_id: '61753'
abstract:
- lang: eng
text: This paper presents LOLA, a massively multilingual large language model trained
on more than 160 languages using a sparse Mixture-of-Experts Transformer architecture.
Our architectural and implementation choices address the challenge of harnessing
linguistic diversity while maintaining efficiency and avoiding the common pitfalls
of multilinguality. Our analysis of the evaluation results shows competitive performance
in natural language generation and understanding tasks. Additionally, we demonstrate
how the learned expert-routing mechanism exploits implicit phylogenetic linguistic
patterns to potentially alleviate the curse of multilinguality. We provide an
in-depth look at the training process, an analysis of the datasets, and a balanced
exploration of the model{’}s strengths and limitations. As an open-source model,
LOLA promotes reproducibility and serves as a robust foundation for future research.
Our findings enable the development of compute-efficient multilingual models with
strong, scalable performance across languages.
author:
- first_name: Nikit
full_name: Srivastava, Nikit
id: '70066'
last_name: Srivastava
orcid: 0009-0004-5164-4911
- first_name: Denis
full_name: Kuchelev, Denis
id: '70842'
last_name: Kuchelev
- first_name: Tatiana
full_name: Moteu Ngoli, Tatiana
id: '99174'
last_name: Moteu Ngoli
- first_name: Kshitij
full_name: Shetty, Kshitij
last_name: Shetty
- first_name: Michael
full_name: Röder, Michael
id: '67199'
last_name: Röder
orcid: https://orcid.org/0000-0002-8609-8277
- first_name: Hamada Mohamed Abdelsamee
full_name: Zahera, Hamada Mohamed Abdelsamee
id: '72768'
last_name: Zahera
orcid: 0000-0003-0215-1278
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Srivastava N, Kuchelev D, Moteu Ngoli T, et al. LOLA – An Open-Source Massively
Multilingual Large Language Model. In: Rambow O, Wanner L, Apidianaki M, Al-Khalifa
H, Eugenio BD, Schockaert S, eds. Proceedings of the 31st International Conference
on Computational Linguistics. Association for Computational Linguistics; 2025:6420–6446.'
apa: Srivastava, N., Kuchelev, D., Moteu Ngoli, T., Shetty, K., Röder, M., Zahera,
H. M. A., Moussallem, D., & Ngonga Ngomo, A.-C. (2025). LOLA – An Open-Source
Massively Multilingual Large Language Model. In O. Rambow, L. Wanner, M. Apidianaki,
H. Al-Khalifa, B. D. Eugenio, & S. Schockaert (Eds.), Proceedings of the
31st International Conference on Computational Linguistics (pp. 6420–6446).
Association for Computational Linguistics.
bibtex: '@inproceedings{Srivastava_Kuchelev_Moteu Ngoli_Shetty_Röder_Zahera_Moussallem_Ngonga
Ngomo_2025, place={Abu Dhabi, UAE}, title={LOLA – An Open-Source Massively Multilingual
Large Language Model}, booktitle={Proceedings of the 31st International Conference
on Computational Linguistics}, publisher={Association for Computational Linguistics},
author={Srivastava, Nikit and Kuchelev, Denis and Moteu Ngoli, Tatiana and Shetty,
Kshitij and Röder, Michael and Zahera, Hamada Mohamed Abdelsamee and Moussallem,
Diego and Ngonga Ngomo, Axel-Cyrille}, editor={Rambow, Owen and Wanner, Leo and
Apidianaki, Marianna and Al-Khalifa, Hend and Eugenio, Barbara Di and Schockaert,
Steven}, year={2025}, pages={6420–6446} }'
chicago: 'Srivastava, Nikit, Denis Kuchelev, Tatiana Moteu Ngoli, Kshitij Shetty,
Michael Röder, Hamada Mohamed Abdelsamee Zahera, Diego Moussallem, and Axel-Cyrille
Ngonga Ngomo. “LOLA – An Open-Source Massively Multilingual Large Language Model.”
In Proceedings of the 31st International Conference on Computational Linguistics,
edited by Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara
Di Eugenio, and Steven Schockaert, 6420–6446. Abu Dhabi, UAE: Association for
Computational Linguistics, 2025.'
ieee: N. Srivastava et al., “LOLA – An Open-Source Massively Multilingual
Large Language Model,” in Proceedings of the 31st International Conference
on Computational Linguistics, 2025, pp. 6420–6446.
mla: Srivastava, Nikit, et al. “LOLA – An Open-Source Massively Multilingual Large
Language Model.” Proceedings of the 31st International Conference on Computational
Linguistics, edited by Owen Rambow et al., Association for Computational Linguistics,
2025, pp. 6420–6446.
short: 'N. Srivastava, D. Kuchelev, T. Moteu Ngoli, K. Shetty, M. Röder, H.M.A.
Zahera, D. Moussallem, A.-C. Ngonga Ngomo, in: O. Rambow, L. Wanner, M. Apidianaki,
H. Al-Khalifa, B.D. Eugenio, S. Schockaert (Eds.), Proceedings of the 31st International
Conference on Computational Linguistics, Association for Computational Linguistics,
Abu Dhabi, UAE, 2025, pp. 6420–6446.'
date_created: 2025-10-08T11:02:30Z
date_updated: 2026-01-06T10:11:37Z
editor:
- first_name: Owen
full_name: Rambow, Owen
last_name: Rambow
- first_name: Leo
full_name: Wanner, Leo
last_name: Wanner
- first_name: Marianna
full_name: Apidianaki, Marianna
last_name: Apidianaki
- first_name: Hend
full_name: Al-Khalifa, Hend
last_name: Al-Khalifa
- first_name: Barbara Di
full_name: Eugenio, Barbara Di
last_name: Eugenio
- first_name: Steven
full_name: Schockaert, Steven
last_name: Schockaert
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://aclanthology.org/2025.coling-main.428.pdf
oa: '1'
page: 6420–6446
place: Abu Dhabi, UAE
publication: Proceedings of the 31st International Conference on Computational Linguistics
publisher: Association for Computational Linguistics
status: public
title: LOLA – An Open-Source Massively Multilingual Large Language Model
type: conference
user_id: '70066'
year: '2025'
...
---
_id: '50797'
author:
- first_name: Michael
full_name: Röder, Michael
id: '67199'
last_name: Röder
orcid: https://orcid.org/0000-0002-8609-8277
- first_name: Denis
full_name: Kuchelev, Denis
id: '70842'
last_name: Kuchelev
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
citation:
ama: 'Röder M, Kuchelev D, Ngonga Ngomo A-C. A Topic Model for the Data Web. In:
Ortiz-Rodriguez F, Villazón-Terrazas B, Tiwari S, Bobed C, eds. Knowledge Graphs
and Semantic Web. Springer Nature Switzerland; 2023:183–198. doi:10.1007/978-3-031-47745-4_14'
apa: Röder, M., Kuchelev, D., & Ngonga Ngomo, A.-C. (2023). A Topic Model for
the Data Web. In F. Ortiz-Rodriguez, B. Villazón-Terrazas, S. Tiwari, & C.
Bobed (Eds.), Knowledge Graphs and Semantic Web (pp. 183–198). Springer
Nature Switzerland. https://doi.org/10.1007/978-3-031-47745-4_14
bibtex: '@inproceedings{Röder_Kuchelev_Ngonga Ngomo_2023, place={Cham}, title={A
Topic Model for the Data Web}, DOI={10.1007/978-3-031-47745-4_14},
booktitle={Knowledge Graphs and Semantic Web}, publisher={Springer Nature Switzerland},
author={Röder, Michael and Kuchelev, Denis and Ngonga Ngomo, Axel-Cyrille}, editor={Ortiz-Rodriguez,
Fernando and Villazón-Terrazas, Boris and Tiwari, Sanju and Bobed, Carlos}, year={2023},
pages={183–198} }'
chicago: 'Röder, Michael, Denis Kuchelev, and Axel-Cyrille Ngonga Ngomo. “A Topic
Model for the Data Web.” In Knowledge Graphs and Semantic Web, edited by
Fernando Ortiz-Rodriguez, Boris Villazón-Terrazas, Sanju Tiwari, and Carlos Bobed,
183–198. Cham: Springer Nature Switzerland, 2023. https://doi.org/10.1007/978-3-031-47745-4_14.'
ieee: 'M. Röder, D. Kuchelev, and A.-C. Ngonga Ngomo, “A Topic Model for the Data
Web,” in Knowledge Graphs and Semantic Web, 2023, pp. 183–198, doi: 10.1007/978-3-031-47745-4_14.'
mla: Röder, Michael, et al. “A Topic Model for the Data Web.” Knowledge Graphs
and Semantic Web, edited by Fernando Ortiz-Rodriguez et al., Springer Nature
Switzerland, 2023, pp. 183–198, doi:10.1007/978-3-031-47745-4_14.
short: 'M. Röder, D. Kuchelev, A.-C. Ngonga Ngomo, in: F. Ortiz-Rodriguez, B. Villazón-Terrazas,
S. Tiwari, C. Bobed (Eds.), Knowledge Graphs and Semantic Web, Springer Nature
Switzerland, Cham, 2023, pp. 183–198.'
date_created: 2024-01-23T11:39:58Z
date_updated: 2024-06-04T09:59:50Z
department:
- _id: '574'
- _id: '923'
doi: 10.1007/978-3-031-47745-4_14
editor:
- first_name: Fernando
full_name: Ortiz-Rodriguez, Fernando
last_name: Ortiz-Rodriguez
- first_name: Boris
full_name: Villazón-Terrazas, Boris
last_name: Villazón-Terrazas
- first_name: Sanju
full_name: Tiwari, Sanju
last_name: Tiwari
- first_name: Carlos
full_name: Bobed, Carlos
last_name: Bobed
keyword:
- sail dice roeder kuchelev ngonga
language:
- iso: eng
page: 183–198
place: Cham
publication: Knowledge Graphs and Semantic Web
publication_identifier:
isbn:
- 978-3-031-47745-4
publisher: Springer Nature Switzerland
status: public
title: A Topic Model for the Data Web
type: conference
user_id: '67199'
year: '2023'
...
---
_id: '54614'
author:
- first_name: Nikit
full_name: Srivastava, Nikit
last_name: Srivastava
- first_name: Aleksandr
full_name: Perevalov, Aleksandr
id: '94275'
last_name: Perevalov
- first_name: Denis
full_name: Kuchelev, Denis
id: '70842'
last_name: Kuchelev
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
- first_name: Andreas
full_name: Both, Andreas
last_name: Both
citation:
ama: 'Srivastava N, Perevalov A, Kuchelev D, Moussallem D, Ngonga Ngomo A-C, Both
A. Lingua Franca - Entity-Aware Machine Translation Approach for Question Answering
over Knowledge Graphs. In: Venable KB, Garijo D, Jalaian B, eds. Proceedings
of the 12th Knowledge Capture Conference 2023, {K-CAP} 2023, Pensacola, FL, USA,
December 5-7, 2023. ACM; 2023:122–130. doi:10.1145/3587259.3627567'
apa: Srivastava, N., Perevalov, A., Kuchelev, D., Moussallem, D., Ngonga Ngomo,
A.-C., & Both, A. (2023). Lingua Franca - Entity-Aware Machine Translation
Approach for Question Answering over Knowledge Graphs. In K. B. Venable, D. Garijo,
& B. Jalaian (Eds.), Proceedings of the 12th Knowledge Capture Conference
2023, {K-CAP} 2023, Pensacola, FL, USA, December 5-7, 2023 (pp. 122–130).
ACM. https://doi.org/10.1145/3587259.3627567
bibtex: '@inproceedings{Srivastava_Perevalov_Kuchelev_Moussallem_Ngonga Ngomo_Both_2023,
title={Lingua Franca - Entity-Aware Machine Translation Approach for Question
Answering over Knowledge Graphs}, DOI={10.1145/3587259.3627567},
booktitle={Proceedings of the 12th Knowledge Capture Conference 2023, {K-CAP}
2023, Pensacola, FL, USA, December 5-7, 2023}, publisher={ACM}, author={Srivastava,
Nikit and Perevalov, Aleksandr and Kuchelev, Denis and Moussallem, Diego and Ngonga
Ngomo, Axel-Cyrille and Both, Andreas}, editor={Venable, Kristen Brent and Garijo,
Daniel and Jalaian, Brian}, year={2023}, pages={122–130} }'
chicago: Srivastava, Nikit, Aleksandr Perevalov, Denis Kuchelev, Diego Moussallem,
Axel-Cyrille Ngonga Ngomo, and Andreas Both. “Lingua Franca - Entity-Aware Machine
Translation Approach for Question Answering over Knowledge Graphs.” In Proceedings
of the 12th Knowledge Capture Conference 2023, {K-CAP} 2023, Pensacola, FL, USA,
December 5-7, 2023, edited by Kristen Brent Venable, Daniel Garijo, and Brian
Jalaian, 122–130. ACM, 2023. https://doi.org/10.1145/3587259.3627567.
ieee: 'N. Srivastava, A. Perevalov, D. Kuchelev, D. Moussallem, A.-C. Ngonga Ngomo,
and A. Both, “Lingua Franca - Entity-Aware Machine Translation Approach for Question
Answering over Knowledge Graphs,” in Proceedings of the 12th Knowledge Capture
Conference 2023, {K-CAP} 2023, Pensacola, FL, USA, December 5-7, 2023, 2023,
pp. 122–130, doi: 10.1145/3587259.3627567.'
mla: Srivastava, Nikit, et al. “Lingua Franca - Entity-Aware Machine Translation
Approach for Question Answering over Knowledge Graphs.” Proceedings of the
12th Knowledge Capture Conference 2023, {K-CAP} 2023, Pensacola, FL, USA, December
5-7, 2023, edited by Kristen Brent Venable et al., ACM, 2023, pp. 122–130,
doi:10.1145/3587259.3627567.
short: 'N. Srivastava, A. Perevalov, D. Kuchelev, D. Moussallem, A.-C. Ngonga Ngomo,
A. Both, in: K.B. Venable, D. Garijo, B. Jalaian (Eds.), Proceedings of the 12th
Knowledge Capture Conference 2023, {K-CAP} 2023, Pensacola, FL, USA, December
5-7, 2023, ACM, 2023, pp. 122–130.'
date_created: 2024-06-04T15:57:41Z
date_updated: 2024-06-04T15:58:16Z
department:
- _id: '574'
doi: 10.1145/3587259.3627567
editor:
- first_name: Kristen Brent
full_name: Venable, Kristen Brent
last_name: Venable
- first_name: Daniel
full_name: Garijo, Daniel
last_name: Garijo
- first_name: Brian
full_name: Jalaian, Brian
last_name: Jalaian
keyword:
- dice kuchelev moussallem ngonga srivastava
language:
- iso: eng
page: 122–130
publication: Proceedings of the 12th Knowledge Capture Conference 2023, {K-CAP} 2023,
Pensacola, FL, USA, December 5-7, 2023
publisher: ACM
status: public
title: Lingua Franca - Entity-Aware Machine Translation Approach for Question Answering
over Knowledge Graphs
type: conference
user_id: '67199'
year: '2023'
...
---
_id: '57274'
author:
- first_name: Nikit
full_name: Srivastava, Nikit
id: '70066'
last_name: Srivastava
orcid: 0009-0004-5164-4911
- first_name: Aleksandr
full_name: Perevalov, Aleksandr
id: '94275'
last_name: Perevalov
- first_name: Denis
full_name: Kuchelev, Denis
id: '70842'
last_name: Kuchelev
- first_name: Diego
full_name: Moussallem, Diego
id: '71635'
last_name: Moussallem
- first_name: Axel-Cyrille
full_name: Ngonga Ngomo, Axel-Cyrille
id: '65716'
last_name: Ngonga Ngomo
- first_name: Andreas
full_name: Both, Andreas
last_name: Both
citation:
ama: 'Srivastava N, Perevalov A, Kuchelev D, Moussallem D, Ngonga Ngomo A-C, Both
A. Lingua Franca – Entity-Aware Machine Translation Approach for Question Answering
over Knowledge Graphs. In: Proceedings of the 12th Knowledge Capture Conference
2023. ACM; 2023. doi:10.1145/3587259.3627567'
apa: Srivastava, N., Perevalov, A., Kuchelev, D., Moussallem, D., Ngonga Ngomo,
A.-C., & Both, A. (2023). Lingua Franca – Entity-Aware Machine Translation
Approach for Question Answering over Knowledge Graphs. Proceedings of the 12th
Knowledge Capture Conference 2023. https://doi.org/10.1145/3587259.3627567
bibtex: '@inproceedings{Srivastava_Perevalov_Kuchelev_Moussallem_Ngonga Ngomo_Both_2023,
title={Lingua Franca – Entity-Aware Machine Translation Approach for Question
Answering over Knowledge Graphs}, DOI={10.1145/3587259.3627567},
booktitle={Proceedings of the 12th Knowledge Capture Conference 2023}, publisher={ACM},
author={Srivastava, Nikit and Perevalov, Aleksandr and Kuchelev, Denis and Moussallem,
Diego and Ngonga Ngomo, Axel-Cyrille and Both, Andreas}, year={2023} }'
chicago: Srivastava, Nikit, Aleksandr Perevalov, Denis Kuchelev, Diego Moussallem,
Axel-Cyrille Ngonga Ngomo, and Andreas Both. “Lingua Franca – Entity-Aware Machine
Translation Approach for Question Answering over Knowledge Graphs.” In Proceedings
of the 12th Knowledge Capture Conference 2023. ACM, 2023. https://doi.org/10.1145/3587259.3627567.
ieee: 'N. Srivastava, A. Perevalov, D. Kuchelev, D. Moussallem, A.-C. Ngonga Ngomo,
and A. Both, “Lingua Franca – Entity-Aware Machine Translation Approach for Question
Answering over Knowledge Graphs,” 2023, doi: 10.1145/3587259.3627567.'
mla: Srivastava, Nikit, et al. “Lingua Franca – Entity-Aware Machine Translation
Approach for Question Answering over Knowledge Graphs.” Proceedings of the
12th Knowledge Capture Conference 2023, ACM, 2023, doi:10.1145/3587259.3627567.
short: 'N. Srivastava, A. Perevalov, D. Kuchelev, D. Moussallem, A.-C. Ngonga Ngomo,
A. Both, in: Proceedings of the 12th Knowledge Capture Conference 2023, ACM, 2023.'
date_created: 2024-11-20T10:39:39Z
date_updated: 2024-11-20T10:59:04Z
doi: 10.1145/3587259.3627567
language:
- iso: eng
publication: Proceedings of the 12th Knowledge Capture Conference 2023
publication_status: published
publisher: ACM
status: public
title: Lingua Franca – Entity-Aware Machine Translation Approach for Question Answering
over Knowledge Graphs
type: conference
user_id: '70066'
year: '2023'
...