Hardening Soft Information: A Transformer-Based Approach to Forecasting Stock Return Volatility

Historically, the field of financial forecasting almost exclusively relied on so-called hard information – i.e., numerical data with well-defined and unambiguous meaning. Over the last few decades, however, researchers and practitioners alike have, following the advances in natural language understanding, started recognizing the benefits of integrating soft information into financial modelling. In line with the above, this paper examines whether contemporary attention-based sequence-to-sequence models, known as Transformers, can help improve stock return volatility prediction when applied to corporate annual reports. Using a publicly available benchmark dataset, we show, in an empirical analysis, that out-of-the-box Transformer models have the ability to outmatch current state-of-the-art results and, more importantly, that our proposed feature-based Transformer approach can outperform a robust numerical baseline. To the best of our knowledge, this is the first empirical study focusing on stock return volatility prediction (1) to ever experiment with state-of-the-art Transformer architectures and (2) to demonstrate that a model based solely on soft information can surpass its numerical counterpart. Furthermore, we show that by including an additional numerical feature into our best text-only model, we can push the performance of our model even further, suggesting that soft and hard information contain different predictive signals.