We find that BERT was significantly undertrained and propose an im-proved recipe for training BERT models, which we call RoBERTa, that can match or exceed the performance of all of …

This study aims to develop a classification model using RoBERTa, a pre-trained language model to predict levels of depression, anxiety, and stress. The dataset comprises 39,776 responses …

Our single RoBERTa model outperforms all but one of the single model submissions, and is the top scoring system among those that do not rely on data augmentation.

In Table 8 we present the full set of development set results for RoBERTa on all 9 GLUE datasets.11 We present results for a LARGE configuration with 355M parameters that follows …

RoBERTa is a more flexible and all-purpose NLP model than BERT since it is assessed on a larger range of tasks and benchmarks than BERT, including activities like question answering …

Transformer models such as BERT, RoBERTa, and DeBERTa have revolutionized the field of Natural Language Processing in recent years with substantial improvements in the contex-tual …

The RoBERTa study (Liu et al., 2019) aimed to carry out a methodical approach to better understand the parameters most useful in developing BERT, with the goal of improving it.