2025
Tuning of Language Models in Eastern European Languages on Twitter/X
FILIP, Tomáš; Martin PAVLÍČEK and Petr SOSÍKBasic information
Original name
Tuning of Language Models in Eastern European Languages on Twitter/X
Authors
FILIP, Tomáš; Martin PAVLÍČEK and Petr SOSÍK
Edition
4092. vyd. Neuveden, Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025), p. 242-252, 11 pp. 2025
Publisher
CEUR-WS
Other information
Language
English
Type of outcome
Proceedings paper
Field of Study
10201 Computer sciences, information science, bioinformatics
Country of publisher
Slovakia
Confidentiality degree
is not subject to a state or trade secret
Publication form
electronic version available online
References:
Marked to be transferred to RIV
Yes
RIV identification code
RIV/47813059:19240/25:A0001541
Organization unit
Faculty of Philosophy and Science in Opava
ISSN
Keywords in English
Large language model; Sentiment analysis; Twitter; Eastern-european language; Russa-Ukraine conflict; Llama; Mistral; BERTweet; BERT; XLM-T; GPT-4
Tags
International impact, Reviewed
Changed: 18/12/2025 10:00, Mgr. Kamil Matula, Ph.D.
Abstract
In the original language
We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.