Institute of Formal and Applied Linguistics

Since the release of ChatGPT Large Language Models (LLMs) are constantly mentioned whenever someone tries to speculate about potential impacts of machine learning. The size of these models makes it very hard to run reproducible experiments with them from scratch. However, LLMs are not omnipotent and there are several bottlenecks that could be vital for further progress in this field. What is particularly interesting is that one can experiment with those bottlenecks on a relatively low budget. In my talk I will try to highlight the areas of research that I personally find particularly interesting. We will start with some fundamentals of LLMs such as more efficient tokenisation procedures and evaluation methodologies and then try to spread out into several application areas such as personalisation, for example, methods to assess how an LLM-based system can emulate empathetic behaviour and domain adaptation from ancient dead languages.