D 2025

Tuning of Language Models in Eastern European Languages on Twitter/X

FILIP, Tomáš; Martin PAVLÍČEK and Petr SOSÍK

Basic information

Original name

Tuning of Language Models in Eastern European Languages on Twitter/X

Authors

FILIP, Tomáš; Martin PAVLÍČEK and Petr SOSÍK

Edition

4092. vyd. Neuveden, Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025), p. 242-252, 11 pp. 2025

Publisher

CEUR-WS

Other information

Language

English

Type of outcome

Proceedings paper

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Slovakia

Confidentiality degree

is not subject to a state or trade secret

Publication form

electronic version available online

Marked to be transferred to RIV

Yes

RIV identification code

RIV/47813059:19240/25:A0001541

Organization unit

Faculty of Philosophy and Science in Opava

ISSN

Keywords in English

Large language model; Sentiment analysis; Twitter; Eastern-european language; Russa-Ukraine conflict; Llama; Mistral; BERTweet; BERT; XLM-T; GPT-4

Tags

International impact, Reviewed
Changed: 18/12/2025 10:00, Mgr. Kamil Matula, Ph.D.

Abstract

In the original language

We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.

Files attached

Filip-Pavlicek-Sosik_Tuning-of-language-models-in-Eastern-European-languages-on-Twitter-X.pdf
Request the author's version of the file