Evaluating the Summarization Comprehension of Pre-Trained Language Modelsстатья
Статья опубликована в журнале из списка RSCI Web of Science
Информация о цитировании статьи получена из
Scopus
Статья опубликована в журнале из перечня ВАК
Статья опубликована в журнале из списка Web of Science и/или Scopus
Дата последнего поиска статьи во внешних источниках: 15 февраля 2024 г.
Аннотация:Recent advances in abstractive summarization demonstrate the importance of pre-training tasks, however, general purpose language models manage to outperform summarization-specialized pre-training approaches. While several works addressed the question of pseudo-summarization pre-training efficiency in abstractive summarization fine-tuning, none has explored the properties of pre-trained models in a low-resource setting. This work attempts to fill this gap. We benchmark 5 state-of-the-art pre-trained language models on 5 single-document abstractive summarization datasets of different domains. To probe the models, we propose 4 novel task comprehension tests that evaluate the main components of summarization models. Our experiments reveal that pseudo-summarization pre-training biases the models towards more extractive behavior and inhibits their ability to properly filter the salient content, leading to worse generalization.