Zsófia Rakovics, statistician and sociologist. During her university studies she conducted a research on precarity, globalization and its effect on society and work. She also took part in other sociological investigations (e.g. about gender inequality), and studied the remembrance of the Holocaust, by qualitatively analysing testimonies of survivors. Her methodological interest includes both qualitative and quantitative methods. After completing her master studies, she worked as a data scientist at an international company using computer vision and machine learning technologies to capture emotional reactions. Recently, her interest turned back to memory research, and towards studying populism and the political public discourse with natural language processing methods, through which she is linked to the research group. Her doctoral research focuses on the study of language polarisation, analysing online political public discourse using deep learning language models.

 

Doctoral research related to the project titled The layers of political public sphere in Hungary (2001-2020) (supported by NKFIH-K-134428)

Sociological study of language polarization

Doctoral student: Zsófia Rakovics

Advisors: Renáta Németh, PhD and Domonkos Sik, PhD

Description

The doctoral research empirically investigates language change and polarization tendencies in online political discourse (Gentzkow et al. 2016; Prior 2013) using deep learning-based language models (Vaswani et al. 2017; Devlin et al. 2018).

Deep learning-based language models are artificial neural networks with many layers and parameters that are capable of generating meaningful texts by learning syntactic and semantic features of the natural language (Vaswani et al. 2017; Devlin et al. 2018). According to the internal, abstract representation of the input texts, these language models are able to generate text that best matches the input (Brown et al. 2020; Mikolov et al. 2013; Radford et al. 2019).

The textual traces of information diffusion patterns can be used to reconstruct the discursive structure of the online political space, identifying network centers and peripheries (Bryden et al. 2013) and ‘contagion patterns’ of information diffusion (Alshaabi et al. 2021; Hamilton and Hamilton 2010). By empirically scanning online political communication for language change and language polarization, and describing their characteristics and dynamics in detail, we can gain deeper insights into the ways in which public discourse operates and thus their impact on society.

The goal of the research is to make the models trained on the Hungarian data available together with an easy-to-use graphical user interface through which new tasks (e.g., classification and abstraction of texts) can be defined and answers gained without deep technical knowledge. The methodology developed in this research can help to pave the way for a wider sociological application of language models.

References

Alshaabi, T., Dewhurst, D. R., Minot, J. R., Arnold, M. V., Adams, J. L., Danforth, C. M., & Dodds, P. S. (2021). The growing amplification of social media: measuring temporal and social contagion dynamics for over 150 languages on Twitter for 2009–2020. EPJ data science, 10(1), 1-28.

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., … & Amodei, D. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165.

Bryden, J., Funk, S., & Jansen, V. A. (2013). Word usage mirrors community structure in the online social network Twitter. EPJ Data Science, 2(1), 1-9.

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

Gentzkow, M., Shapiro, J., & Taddy, M. (2016). Measuring polarization in high-dimensional data: Method and application to congressional speech (No. id: 11114).

Hamilton, J. D., & Hamilton, L. C. (2010 [1981]). Models of social contagion. Journal of Mathematical Sociology, 8(1), 133-160.

Mikolov, T., Yih, W. T., & Zweig, G. (2013). Linguistic regularities in continuous space word representations. In Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: Human language technologies (pp. 746-751).

Prior, M. (2013). Media and political polarization. Annual Review of Political Science, 16, 101-127.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762.

 

ÚNKP Research 2022/23

Title: Examining the linguistic and political polarization of parliamentary speeches

PhD student: Zsófia Rakovics

Supervisor: Domonkos Sik, PhD

 

Politicians have an influential role in the construction of social reality and the thematization of topical issues of a country’s public sphere. They simultaneously influence and are influenced by the political polarization of society. Therefore, studying the political speeches of the members of a parliament provides us with unique insights into this process and about society. Parliament is an important forum for political communication, and it has undergone significant changes in recent decades.

I have analyzed parliamentary speeches from 1998 to 2020 using natural language processing, machine learning and deep learning methods to provide a picture of the linguistic and political polarization of politicians’ speeches in parliament over the studied period. To better understand the changes in political communication, I have also examined the available metadata in the analysis, including the date, type, names, and party affiliations of speakers.

At the level of individual words, I used keyness analysis to identify the keywords that most distinguish the parliamentary parties from each other. By measuring the similarities of these keywords, I mapped patterns of change in speeches, revealing the political communication strategy of each party. Then, I performed quantitative modelling of the texts at the level of word distributions using structural topic modelling (Robert et al. 2019). The topic model can be used to identify dominant themes in an exploratory way based on the word choice of the texts, so that a thematic map of speakers can be constructed even for a manually unprocessable set of texts. The dynamic topic model variant can also be used to track the temporal prevalence of topics in a more sophisticated way than pattern analysis focusing on individual words. Next, I used a word embedding vector space model (Mikolov et al. 2013) to investigate the change in meaning of previously identified keywords.

Using the Word Mover’s Embedding (Wu et al. 2018a, 2018b) document embedding, I embedded the speeches in an abstract vector space in which the relationships between speeches, politicians and parties can be mapped and networks constructed. Examining the transitivity of the graph generated by the similarity of speeches by parliamentary term and party (Wasserman & Faust 1994), I observed an increase over time, indicating a homogenization of speeches within a given party. This tendency was also indicated by the classification of document vectors by party; I built classification models – XGBoost (Hastie et al. 2009) and BERT (Devlin et al. 2018, Nemeskey 2021) – to predict the party of speakers. Evaluating the models by party and year allowed me to examine changes in the amplitude and characteristics of linguistic polarization over time.

The patterns of intra-party homogenization and inter-party polarization evident in the speeches were in line with the characteristics of neopopulism-theory (Csigó 2016), according to which a speculative mediatized political space emerged where politicians observe the strategies employed by other politicians to appear popular and distinguish themselves from others by creating self-referential communication bubbles.

 

References

• Csigó, P. (2016). The Neopopular Bubble. Speculating on” the People” in Late Modern Democracy. Central European University Press.
• Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
• Hastie, T., Tibshirani, R., Friedman, J. H., & Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction. (Vol. 2, pp. 1-758). New York: Springer.
• Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
• Nemeskey, D. M. (2021). Introducing huBERT. In Berend Gábor. Magyar Számítógépes Nyelvészeti Konferencia: MSZNY 2021: Szeged. pp. 3-14.
• Rakovics, Z. (2022). A Temporal Positive Pointwise Mutual Information (TPPMI) időbeli szóbeágyazási modell alkalmazásában rejlő lehetőségek demonstrálása: A miniszterelnöki beszédek szavainak jelentésváltozása. In: Feledy, A.; Egle, B. (szerk.) Van új a nap alatt: Az ELTE Angelusz Róbert Társadalomtudományi Szakkollégium konferenciájának tanulmánykötete. Budapest, Magyarország: ELTE ARTSZ pp. 31-48.
• Roberts, M. E., Stewart, B. M., & Tingley, D. (2019). STM: An R package for structural topic models. Journal of Statistical Software, 91, 1-40.
• Wasserman, S., & Faust, K. (1994). Social Network Analysis: Methods and Applications. Cambridge: Cambridge University Press.
• Wu, L., Yen, I. E., Xu, K., Xu, F., Balakrishnan, A., Chen, P. Y., … & Witbrock, M. J. (2018a). Word mover’s embedding: From word2vec to document embedding. arXiv preprint arXiv:1811.01713.
• Wu, L., Yen, I. E. H., Xu, F., Ravikumar, P., & Witbrock, M. (2018b). D2ke: From distance to kernel and embedding. arXiv preprint arXiv:1802.04956.