You will be provided with a reference and some statements. Please determine whether each statement is 'supported', 'unsupported', or 'unknown' with respect to the reference. Please note:
First, assess whether the reference contains any valid content. If the reference contains no valid information, such as a 'page not found' message, then all statements should be considered 'unknown'.
If the reference is valid, for a given statement: if the facts or data it contains can be found entirely or partially within the reference, it is considered 'supported' (data accepts rounding); if all facts and data in the statement cannot be found in the reference, it is considered 'unsupported'.

You should return the result in a JSON list format, where each item in the list contains the statement's index and the judgment result, for example:
[
    {
        "idx": 1,
        "result": "supported"
    },
    {
        "idx": 2,
        "result": "unsupported"
    }
]

Below are the reference and statements:
<reference>
KING’S
BUSINESS
SCHOOL

Data Analytics
for Finance
& Macro
Research Centre

Interpretable Machine Learning Modelling for
Asset Pricing
G. Kapetanios, F. Kempf
Working paper No. 2022/1 | February 2022 | ISSN 2516-5933

Interpretable Machine Learning Modelling for Asset Pricing∗
WORKING PAPER
George Kapetanios

Felix Kempf

King’s College London

King’s College London

February 15, 2022

Abstract
We use deep neural networks to estimate time-varying equity risk premia. The key innovations are
the nonlinear and non-parametric generalisation of Fama-Macbeth regressions through partial derivatives of an arbitrary estimator function with respect to its input and the introduction of Jacobian
regularisation in the objective function to empirical asset pricing. Our methodology outperforms all
linear benchmarks out-of-sample. Moreover, we introduce the concept of sensitivity-sorted portfolios.
Most importantly, we move elements of interpretability and explainability to the foreground.

Keywords: Conditional asset pricing model, nonlinear factor model, cross-section of expected returns, machine learning, deep learning, Jacobian regularisation, interpretability, explainability

1

Introduction

Asset pricing centres around the question of how to explain variations in expected returns across assets.
While being a notoriously challenging task empirically, the answer to this question from an asset pricing
theory point of view is clear. In the absence of arbitrage, variations in expected returns should be
reflected by an asset’s exposure to a stochastic discount factor (SDF) mt+1 for any excess return ri,t+1 .
Borrowing notation from Kelly et al. (2019), it follows that


Covt (mt+1 , ri,t+1 )
Vart (mt+1 )
Et [mt+1 ri,t+1 ] = 0 ⇐⇒ Et [ri,t+1 ] =
−
,
Vart (mt+1 )
Et [mt+1 ]
|
{z
}|
{z
}
′
βi,t

(1)

λt

∗ We are grateful for the helpful comments and suggestions from Markus Pelger (Stanford University), Harald Lohre

(Invesco), Maximilian Stroh (Quoniam), and the seminar participants of the 2019 and 2021 Bank of England Conference
Modelling with Big Data and Machine Learning: Interpretability and Model Uncertainty, where we first presented the idea,
https://www.bankofengland.co.uk/events/2019/november/modelling-with-big-data-and-machine-learning.

1

where βi,t denotes the exposures to systematic risk factors of assets i = 1, ..., Nt+1 and λt are risk prices
which is defined as compensation an investor can expect to receive for bearing systematic risk. Without
loss of generality, if the SDF is linear in factors ft+1 (i.e. mt+1 = a + b′ ft+1 ; see Cochrane (2009)), the
conditional mean estimator function of equation (1) is equivalent to a beta pricing model (e.g. see Ross
(1976) or Hansen and Richard (1987)), such that
′
ri,t+1 = αi,t + βi,t
ft+1 + ϵi,t+1 ,

(2)

where Et [ϵi,t+1 ] = Et [ft+1 ϵi,t+1 ] = 0 and Et [ft+1 ] = λt , with most asset pricing tests typically focussing
on Et [αi,t ] = 0. The conditional expectation of equation (1) and the beta representation of equation (2)
are the starting point of two of the most widely used approaches in empirical asset pricing: cross-sectional
regressions on firm characteristics and (one or) two-pass regressions such as the original Fama-Macbeth
(FM hereafter; Fama and MacBeth (1973)) regressions on (conditional) portfolio sorts, respectively predefined and common risk factors. Thus, in the former case, the betas are assumed to be observable
(e.g. Lewellen (2015)), while in the latter case, the risk factors are assumed to be observable (e.g.
see Jagannathan and Wang (1998)). What unifies both approaches is the estimation and testing of
the marginal importance of each characteristic, respectively risk factor. We show that our proposed
methodology can be applied universally in both settings.
Current literature has four key challenges regarding the empirical estimation and interpretation of
risk exposures and prices. First, in practice, neither factors nor risk exposures are directly observable
and could theoretically depend on all observable information, yielding a potentially high-dimensional
estimation problem. The empirical hunt for new risk factors or firm characteristics that can explain
variations in expected returns has produced hundreds of potential candidates in recent years, as documented by Harvey et al. (2016), Mclean and Pontiff (2016) or Hou et al. (2020). Consequently, modern
asset pricing models should be capable of handling high-dimensional data. It is in this environment that
Cochrane (2011) points to alternative methods to navigate this ever-growing factor zoo effectively.
Second, there is little intuitive justification and strong empirical evidence against the linearity assumption in equation (2), as discussed, for example, by Campbell and Cochrane (1999), Bansal and
Yaron (2004) or He and Krishnamurthy (2013). Therefore, an asset pricing model should (at least
partly) capture intrinsic nonlinearities in the return data. Non-parametric modelling, such as machine
learning, offers the opportunity to impose no restrictions on the estimator’s functional form. This datadriven approach allows for nonlinear estimations and yields statistical factors and exposures instead of
pre-specified (and potentially misspecified) factors and exposures.
Third, despite an abundance of dynamic models, the overwhelming majority of empirical research is
concerned with estimating risk premia unconditionally, meaning that the compensation an investor can

2

expect to receive for bearing systematic risk is assumed to remain constant over time. Notable counterexamples include Ferson and Harvey (1991), Ilmanen et al. (2019) or Umlandt (2020) who study timevarying risk premia and risk exposures. Still, little is known about the time-variation of risk exposures
and premia and their non-parametric estimation. In addition, the ever-changing nature of financial
markets, technological advancements and even newly introduced regulation make it seem unlikely that
risk premia should remain constant over time; e.g. see Oh and Patton (2018). The poor performance
of traditional value investing (e.g. Agrawal (2020)) in recent years or changing cross-asset correlation
structures during times of crises support this hypothesis from an asset manager’s or regulator’s point of
view.
Fourth, researchers, asset managers, or regulators have the fiduciary duty to understand their models’
inner structure and communicate it clearly with all associated risks to their clients or stakeholders. Hence,
it is desirable to engineer models with powerful out-of-sample performance (which is laborious enough
already) and move elements of interpretability and explainability to the foreground. In this context, the
terms interpretability and explainability are only loosely defined in the machine learning literature. Miller
(2019), for example, states that interpretability can be summarised as the degree to which a human can
understand the cause of a model’s output, with Kim et al. (2016) or Murdoch et al. (2019) argue in the
same vein and point out that interpretability is a useful umbrella term that captures the identification
of relevant knowledge extracted from a machine learning model. Rudin (2019) differentiates between
the two terms and argues that interpretable models are intrinsically understandable, while explainable
models are black-box models whose outputs become interpretable post-hoc. In addition, Doshi-Velez and
Kim (2017) point out that single performance measures, such as the cross-sectional mean R2 , predictive
R2 or even out-of-sample mean-squared errors, are incomplete descriptions for a model’s suitability and
elements of interpretability and explainability should be part of the model selection process.
Interpretable or explainable machine learning in empirical asset pricing is fundamentally different from
other disciplines. For example, consider the domain of image recognition, where the data is characterised
by extraordinarily high signal-to-noise ratios, combined with an abundant training data availability.
Rudin (2019)’s arguments of introducing interpretable layers into machine learning models, such as
neural networks, follows common sense: if, for example, a model detects the head of a bird and is
capable of comparing the bird’s head to an extensive library of other birds’ heads and, thus, bases its
classification on the most probable commonality, this type of model mechanic is highly interpretable
for humans. On the other hand, datasets in economics and finance are fundamentally different due to
their meagre signal-to-noise ratio. The extent to which a human can fully understand where the data’s
signal is extracted from is, thus, diametrically different compared to other disciplines, such as image
recognition. Therefore, direct comparability of model interpretability across disciplines is limited or at
least difficult.
3

We build on existing research, such as Chen et al. (2019), Gu et al. (2020a) and Freyberger et al.
(2020), that presents compelling evidence in favour of modelling the conditional mean of equation (1)
non-parametrically and nonlinearly. To estimate risk premia, we conflate the seminal cross-sectional regressions of Fama and MacBeth (1973) with machine learning. Machine learning certainly is already an
integral component of modern asset pricing literature. However, to the best of our knowledge, there exists
little insight into the direct implementation of FM regressions using machine learning. The main reason
for this is that previous research has often deemed periodic refitting as computationally too expensive.
However, thanks to easy access to increased and on-demand computing power and significantly reduced
computing costs, those limitations are slowly but surely reduced. Despite being computationally more
expensive, machine learning helps mitigate one of the central and well-documented problems present in
cross-sectional regressions: in-sample overfitting. By drawing on tools such as sample-splitting, hyperparameter tuning or regularisation, machine learning specifically attempts to avoid in-sample overfitting
and seeks to generalise well out-of-sample1 . This shift in focus helps to improve statistical inference
about risk premia.
With this paper, we aim to make four key contributions. First, under the condition of differentiability,
we propose the use of partial derivatives of the conditional mean estimator function with respect to its
inputs (i.e. the input gradients) as a means to estimate nonlinear and time-varying risk premia (e.g. see
Dixon and Polson (2019)). The nonlinearity of the thereby estimated risk premia directly results from
the estimator’s inherent nonlinearity (e.g. see Kapetanios (2007)). Refitting periodically in the style of
Fama-Macbeth regressions induces a time variation in our estimation. Partial derivatives are a natural
way to conceptualise risk premia and distinguish themselves through easy and fast implementation. We
show that our proposed methodology is a generalisation that nests the Fama-Macbeth estimator as a
special case assuming that the conditional mean estimator function is linear. Most importantly, the
combination of nonlinear modelling and input gradients allows for more general pricing factors. Most
cross-sectional research fundamentally draws on the seminal works by Black et al. (1972), Fama and
MacBeth (1973) and Gibbons et al. (1989) and perform statistical inference on the time-series averages
of risk premia estimates, typically with some form of adjustment for multiple testing as proposed by
Newey and West (1987) or Benjamini and Yekutieli (2001). Using averages is an intuitive and wellestablished procedure. However, evaluating an overall average fails to recognise that risk premia may
change over time. We explicitly allow risk premia to vary over time and transition in and out of empirical
importance by constructing tolerance bands. This transition is particularly beneficial when analysing
times of economic stress. Moreover, we show that nonlinear risk premia facilitate much richer insights
as we can conduct analyses, for example, by company size or industry, without refitting the model on
the subset of interest.
1 For an introduction to machine learning in general or machine learning in finance, we refer readers to Bishop (1995),
Hastie et al. (2009), Goodfellow et al. (2016) or De Prado (2018).

4

Second, we introduce Jacobian regularisation as part of the objective function to empirical asset pricing. The objective function generalises variable selection and shrinkage for nonlinear models analogously
to LASSO, Ridge or Elastic Net (e.g. see Tibshirani (1996), Zou and Hastie (2005) or Hastie et al.
(2009)), and thus increases model interpretability. The objective function minimises the residual sum of
squares subject to the regularisation of the gradient norm2 . While the concept of Jacobian regularisation
is not new in the general machine learning literature, and in particular in image recognition (e.g. see
Drucker and Cun (1992), Sokolić et al. (2017), Varga et al. (2018) or Hoffman et al. (2019)), we introduce
it to asset pricing. Due to the constrained loss function’s nature, certain risk premia are set exactly to
(or shrunk towards) zero. Financial machine learning fundamentally differs from machine learning applications in other disciplines, such as image recognition or natural language processing, where machine
learning thrives. Due to limited data availability, the dynamic nature of financial markets and a low
signal-to-noise ratio, out-of-the-box machine learning algorithms show a high failure rate in practice. De
Prado (2018) argues that one of the critical factors to make financial machine learning applications more
successful is to move away from a purely data-driven approach to a quantamental strategy. Quantamental, in this context, describes the combination of data-driven algorithms with economic and financial
theory. Our proposed objective function makes exactly that possible. Economic theory suggests that
the true but unknown asset pricing model is approximately low-dimensional as recognised, for example,
by Kozak et al. (2018) or Barillas and Shanken (2018). By setting specific risk premia to exactly zero,
the objective function performs model selection. The resulting model is, consequently, more easily interpretable due to its reduced dimensionality. Additionally, from a practitioners’ point of view, it is desirable
to achieve competitive out-of-sample performance while manually imposing restrictions on the influence
of specific risk factors or firm characteristics in the model. A significant amount of academic research in
financial machine learning finds that the most influential factors can frequently be attributed to market
frictions or momentum and regularly attests short-term-reversal to be the source of the most significant
signal (e.g. Gu et al. (2020b), or Lewellen (2015)). However, those factors can be difficult to trade on
in real-life, due to high portfolio turnovers and trading cost considerations (e.g. Leung et al. (2021)).
Therefore, there is great demand for performant machine learning models such as deep neural networks
and the possibility of simultaneously imposing individual restrictions on the learning algorithm. While
many practical applications are imaginable, such as ESG (economic, social and governance) restrictions,
we merely focus on the objective function’s theoretical properties and implementation methods in this
paper.
Third, we explicitly do not wish to shift existing out-of-sample performance frontiers with our empirical analysis. Instead, the collectivity of all innovations mentioned above intends to move interpretability
and explainability elements to the foreground. Those aspects have become increasingly important in
2 The flexible form of the objective function also allows for simultaneous weight penalisation.

5

recent years, not least because of asset managers, regulators, and researchers’ fiduciary duty to communicate all associated risks of their models to all relevant stakeholders. For example, we show how the
nonlinear interaction of model sensitivities to changes in the input helps to understand complex inner
model mechanics better. Moreover, our proposed methodology allows for model analysis on the asset
level. Such granular model inspection is still underrepresented in current literature to the best of our
knowledge but offers valuable insights. Examples include detecting individual assets that the model is
not handling well or extreme sensitivity outliers. An inspection on the asset level also helps with software
debugging or the discovery of unwanted model biases. It further allows for computationally inexpensive
and detailed analyses on subgroups of assets, such as by industry or size class, defined by an asset’s
market capitalisation.
Finally, we introduce the concept of double-sorted portfolios based on model sensitivities. Portfolios
sorts are ubiquitous in the empirical asset pricing literature and are used to test fundamental asset
pricing theories, establish pricing anomalies or identify profitable investment strategies (e.g. see Cattaneo
et al. (2020)). Portfolio sorts are most fundamentally based on the idea of sorting firm characteristics
into baskets on which, for example, equal or value-weighted portfolios are constructed and has been
informally recognised as a non-parametric alternative to imposing linearity on the relationship between
the returns of assets and firm characteristics (e.g. see Fama and French (2008), or Cochrane (2011)). We
develop a framework by casting double-sorted out-of-sample portfolios as sorts that are based on firm
characteristics and the expected volatility in expected returns due to changes in firm characteristics.
By incorporating expected out-of-sample return sensitivities to (potentially unexpected) changes in firm
characteristics, we offer a tool to manage a double-sorted portfolio’s expected risk. The critical difference
to existing approaches in current literature is that our methodology fundamentally relies on an out-ofsample estimation that is assumed to generalise well to unseen data. This out-of-sample approach, thus,
is an empirically more difficult task. As a consequence, we do not claim to discover previously unknown
anomalies. Instead, we show that incorporating model sensitivities into the portfolio construction exercise
can help to manage out-of-sample risk.
Our empirical study follows the standard procedure in the empirical asset pricing literature. We source
monthly stock returns from CRSP for firms listed on NYSE, AMEX, and NASDAQ and construct 103
firm characteristics based on fundamental data sourced from Compustat, I/B/E/S, FRED and BLS. The
list of firm characteristics includes current literature’s most relevant candidates. Further, we consider
an alternative scenario in which only 49 of the most commonly used firm characteristics are included.
We refer to these 49 firm characteristics as core characteristics. The data sample spans over 492 months
from January 1980 to December 2020, yielding over four decades of data. We investigate ten different
neural networks, each with a separate objective function. In particular, we consider objective functions
with no model parameter penalisation, model weights regularisation and various forms of Jacobian (input
6

gradient) regularisation.
Moreover, we benchmark the neural networks’ model performances against five standard linear benchmarks, including ordinary least squares (OLS), weighted least squares (WLS), ridge, elastic net and least
absolute shrinkage and selection operator (LASSO). All ten neural networks under consideration outperform the linear benchmarks out-of-sample . We compare the out-of-sample performances using the
two key metrics, cross-sectional mean R2 and predictive R2 . The best performing models in the core
characteristics only case are the neural networks with L1 norm weight, respectively column-wise L1
norm Jacobian penalisation as part of their objective function, with 17.20%, respectively 15.83% crosssectional mean R2 and 0.12%, respectively 0.09% predictive R2 . This paper primarily focuses on the
newly introduced methodological contributions, which is why we do not claim to shift existing stateof-the-art prediction performance frontiers. The competitive empirical performances of neural networks
whose objective function includes Jacobian regularisation are robust across many sample-splitting and
data pre-processing regimes, input dimensionalities, asset size classes (defined by their market capitalisation) and specific microcap considerations (such as excluding microcaps entirely from the investment
universe).
The empirical findings are six-fold. First, we confirm the empirical insights presented by Gu et al.
(2020b) or Chen et al. (2019) that deep neural networks can explain intrinsic return structures due to their
non-parametric and nonlinear form. However, performances can be subpar when using out-of-the-box
neural network training regimes, including objective functions with no model parameter regularisation.
The crucial innovation is the inclusion of the Jacobian or input gradient penalty, as it not only generalises
the linear model selection techniques, such as the LASSO or Ridge- to neural networks. It also introduces
an economically interpretably penalty term.
Second, we confirm the theoretical properties of the Jacobian regularisation term and show that
input gradient penalisation does indeed yield non-parametric model selection. In addition, the empirical
asset pricing literature has long argued that a parsimonious model representation is desirable. Thus,
the marginal benefit of including the Jacobian penalty term in the objective function is relatively more
significant in high-dimensional settings compared to lower dimensionalities. For example, the crosssectional mean R2 in the case of using all 103 firm characteristics is nearly 18 times higher for neural
networks that are trained with an objective function that includes an L1 norm column-wise penalty
term of the Jacobian compared to a neural network with no model parameter regularisation as part of
the objective function (16.12%, compared to 0.92%). However, in the case of manually reducing the
firm characteristic universe from 103 to the most relevant 49 firm characteristics, this marginal benefit
is significantly reduced, as the cross-sectional mean R2 of neural networks with L1 norm column-wise
Jacobian penalisation in their objective function is merely 1.1 times higher compared to neural networks
with no model parameter penalisation in the objective function. In addition, the most critical variables
7

selected by the best performing neural networks are significantly less correlated than, for example, the
linear OLS benchmark. The advantageous correlation structure of the most influential firm characteristics
suggests that the best performing neural networks are better capable of extracting signals from firm
characteristics, even in the presence of mild multicollinearity, making them an empirically robust model
choice.
Third, our applied study provides strong empirical evidence suggesting a time-varying nature of risk
premia. In particular, we show that the unconditional and constant risk premia, which are typically
reported in the empirical asset pricing literature, are far less informative compared to the time-varying
risk premia with tolerance bands that we report in this paper. In our empirical study, we show, for
example, that the estimated risk premium associated with being exposed to the systematic risk factor
return-on-assets spikes during the financial crisis but levels out in the following years. Moreover, the risk
premia estimates are robust across different asset size classes defined by their market capitalisation.
Fourth, similarly to Chen et al. (2019) we show that nonlinear firm characteristic interactions matter,
which is also discussed by Gu et al. (2020b), or Bryzgalova et al. (2019). Moreover, pairwise locally
weighted regressions help to understand the nonlinear interactions between return prediction sensitivities
to changes in input firm characteristics. These nonlinear interactions provide valuable model insights
to all stakeholders. They help explain the expected model prediction sensitivities to changes in one
firm characteristic, given the model prediction sensitivity to changes in another firm characteristic for a
particular asset.
Fifth, the input gradients provide valuable model insights on the asset level. In particular, we show
that they are beneficial for evaluating the general functioning of the objective function of choice. Further,
they can help with software debugging and the detection of unwanted biases in the model estimation.
In an empirical study, we also discuss the concept of prediction stability, which is closely related to
model sensitivity outliers. While individual assets typically do not negatively influence the overall model
performance, model stakeholders may still be interested in stable return predictions for individual assets.
In this context, we stress that the objective function choice should become an integral part of the model
design exercise, such that the general functioning of the objective function adequately fulfils the required
features.
Sixth, double-sorted portfolios on firm characteristics and sensitivities can offer a practical tool that
can help to manage expected out-of-sample portfolio volatilities. We show that for neural networks
with column-wise L1 norm Jacobian regularisation, double-sorted value-weighted portfolios sorted on,
for example, return-on-assets and low expected model prediction sensitivities yield a Sharpe ratio that
is nearly 38% higher than the Sharpe ratio of portfolios that are sorted on high model sensitivities.
A similar pattern emerges for equal-weighted portfolios where the portfolio sorted on low sensitivities
is nearly 28% compared to the portfolio constructed on high sensitivities. We do not claim that such
8

patterns are universally applicable to all firm characteristics and models. A possible explanation for this
pattern – which is also closely related to the concept of prediction stability – is that given an estimator
function g that generalises well out-of-sample, assets with low return prediction sensitivities to changes
in a specific characteristic are expected to be less volatile compared to assets with high sensitivities.
We intend to extend several different strands of the empirical asset pricing and econometrics literature.
This paper seamlessly blends into the extensive asset pricing literature investigating the risk factor
identification and econometrics literature estimating factor models. However, our paper most closely
relates to the newly established literature utilising non-parametric and nonlinear machine learning to
shrink the high-dimensional cross-section in asset pricing.
Starting with the capital asset pricing model (CAPM) of Sharpe (1964), Lintner (1965) and Mossin
(1966), which is building on the mean-variance portfolio optimisation proposed by Markowitz (1952)),
empirical asset pricing has attempted to capture the implications of the SDF: the empirical estimation
of risk exposures β and prices of risk λ3 . The most important theme of this early seminal research are
the testable implications (e.g. see Jensen (1968), Black (1972) or Gibbons et al. (1989)) of the null
hypothesis that the intercept (alpha) and time-series average of risk prices are zero. Subsequently, the
single-factor approach of the CAPM4 has been continuously extended. Fama and French (1993) propose
a three-factor model, while Hou et al. (2015) present a four-factor model. Fama and French (2015)
move on to a five-factor model and Barillas and Shanken (2018) suggest a six-factor model. Similarly to
Lewellen (2015), we aim to shed some light on the navigation of this ever-growing factor zoo.
Methodologically, the seminal contribution by Fama and MacBeth (1973) is the infamous two-step
procedure that combines time-series with cross-sectional regressions in order to estimate β’s and λ’s in
equation (2). FM regressions are still heavily used today. In a first step, excess returns are regressed
on previously constructed risk factor portfolio returns5 using rolling time-series regressions such as for
months t − 60 to t − 1. After estimating the betas, cross-sectional regressions of excess returns on the
betas are utilised to estimate prices of risk, where the null hypothesis is that the mean risk premium
is zero. We translate this fundamental principle into a nonlinear and non-parametric setting where we
allow risk exposures and risk premia to vary over time. Fama and French (2020) show that observable
firm characteristics are equivalent to time-varying risk exposures in cross-sectional regressions.
We also build on extensive econometrics research investigating time-varying models, in contrast to, for
example, static-beta models. Ferson and Harvey (1991), Ghysels (1998), Chaieb et al. (2018), Ilmanen
et al. (2019) or Umlandt (2020) study time-varying risk premia. Similarly, Jagannathan and Wang
3 Other pioneering work includes Black (1972), Merton (1973), Ross (1976), Banz (1981), Basu (1983) or Rosenberg
et al. (1985)
4 Empirical evidence against the static single-factor CAPM is presented, for example, by Banz (1981), Reinganum (1981),
Gibbons (1982), Basu (1983), Chan et al. (1985), Shanken (1985) or Bhandari (1988)
5 Typically, they are constructed as long-short portfolio returns based on firm characteristics. For example, at time t all
firms are ranked based on their characteristic ct,ik where an investor would go long the top decile and short the bottom
decile.

9

(1996a) allow risk exposures and risk premia to vary over time. We extend this approach by discussing
the usefulness of confidence, respectively tolerance bands for time-varying risk premia estimations. In
particular, we discuss theoretical methodologies such as bootstrapping and put them into practical
context by considering their computational cost. To keep computational costs feasible, we provide an
empirical approach that yields empirical tolerance bands, such that risk premia can transition in and out
of empirical importance, where we define risk premia as empirically important if the tolerance bands do
not include zero.
In times of unprecedented data and computing power availability machine learning has emerged as
a powerful alternative in modern asset pricing to study the cross-section of stock returns6 . Gu et al.
(2020b) conduct a large-scale comparison of various different machine learning methods and show the
benefits of non-parametric and nonlinear modelling. In particular, they show the competitive performance of deep neural networks, not least because of their universal approximation capabilities; see, for
example, Cybenko (1989) or Hornik et al. (1989). Similarly, Messmer (2017a), Feng et al. (2018) and Gu
et al. (2020a) predict stock returns using deep neural networks. Chen et al. (2019) propose to further
include macroeconomic time-series in their neural network training. Feng et al. (2020) utilise deep neural
networks in connection with an economic objective function that minimises pricing errors which shows
a significant improvement in the efficient portfolio7 . We build on this strand of literature and also apply
deep neural networks in our empirical application. However, our proposed methodology generalises to a
large number of models under the condition of differentiability.
In addition to deep neural networks, we also build on the newly established strand of literature
studying the dimensionality reduction of the high-dimensional cross-section of returns. Rapach et al.
(2013) apply LASSO to predict returns using lagged information. Similarly, Messmer (2017b), Feng and
Giglio (2017) or Han et al. (2019) propose the LASSO for model selection in order to navigate through
the factor zoo8 . Freyberger et al. (2020) and Kozak et al. (2020) use shrinkage and selection methods
to estimate the SDF. Kelly et al. (2019) propose the instrumented principal components that extends
typical PCA with time-varying loadings. Lettau and Pelger (2020b) study the estimation of latent factors
using a generalisation of PCA. Gu et al. (2020a) propose non-linear PCA using autoencoders.
Last but not least, our paper also relates to Dixon and Polson (2019) as they also draw on partial
derivatives of deep neural networks with respect to their inputs. However, our approach differs from
Dixon and Polson (2019) in three ways: First, we apply partial derivatives in the context of asset
pricing models and, therefore, estimate not only systematic risk factors but also risk compensation.
Second, we directly incorporate the partial derivatives as part of the objective function through Jacobian
6 The rise of machine learning in asset pricing has arguably been fuelled even more by spillover effects from successes in
other disciplines such as such as computer vision (e.g. Goodfellow et al. (2014)), natural language processing (e.g. Kumar
et al. (2016)) or complex gaming (eg. Silver et al. (2016)).
7 Other notable papers include Heaton et al. (2017), Messmer (2017a), Bryzgalova et al. (2019), Imajo et al. (2020),
Harvey and Liu (2014).
8 Other examples include Chernozhukov et al. (2018).

10

penalisation. Third, we use partial derivatives as an instrument to shed light on complex inner model
structures, which are often described as black boxes. Moreover, our empirical application more closely
resembles current state-of-the-art data as we replicate the factor zoo of Green et al. (2017).
The rest of this paper is structured as follows. Section 2 lays out our proposed methodology. Section 3
discusses the empirical estimation strategy. In section 4 we introduce the considered investment universe
and the empirical results. Section 5 presents concluding remarks. We relegate all additional empirical
results and discussion to the appendix.

2

Methodology

In the following, we do not impose any restrictions on the functional form of returns and generalise the
beta representation of excess returns from equation (2) as an additive prediction error model such that
ri,t+1 = Et [ri,t+1 ] + ϵi,t+1 ,

(3)

Et [ri,t+1 ] = gt (xi,t ; Wt , θt ),

(4)

with

where Et [ri,t+1 ] = E[ri,t+1 |It ] denotes the conditional expectation of returns given all available information It that is observable by an investor up until time t = 1, ..., T , and ϵ ∼ i.i.d.(0, σϵ2 ) are random
errors. In its most general form, gt is arbitrary, unknown and only depends on the K-dimensional input
vector xi,t . Potential remaining model (hyper-)parameters are captured by Wt and θt . More specifically, we apply common terminology and refer to the model parameters captured by Wt as learnable
model parameters, as their optimal 9 values are not set manually. Instead they are found as part of the
model fitting process, such as gradient-based optimisation. In contrast, the model parameters captured
by θt cannot be learned through optimisation, but need to be set manually. Their optimal values are
typically found through k-fold cross-validation10 . Individual stocks in equations (3) and (4) are indexed
by i = 1, ..., Nt , where Nt describes the total number of stocks at time t. The time index emphasises
that financial data is generally unbalanced and that the number of stocks can vary over time.

2.1

Generalisation of Fama-Macbeth Two-Pass Procedure

In this section, we review the standard two-step FM regression procedure (see Fama and MacBeth (1973)
or Cochrane (2009)) that remains popular in asset pricing literature to this day due to its simplicity.
Our approach can be seen as a generalisation of the static FM asset pricing approach. What unites the
classical FM and our generalised approach is estimating the risk exposures and the prices of risk in two
9 Where the term optimal does not necessarily refer to a global optimum.
10 We refer readers to Hastie et al. (2009) for a more comprehensive discussion.

11

separate steps. Equation (4) raises the central question of how to approximate and estimate the unknown
estimator function gt . The conditional asset pricing model in equation (1) and the beta representation
in equation (2) form the basis of FM regressions as they assume a linear relationship between the risk
exposures (β) and risk prices (λ) and, thus, defines gt as
Et [ri,t+1 ] = gt (β i,t , λ) = β i,t λt ,
(1)

(k)

(5)

(K)

where β i,t = (βi,t , · · · , βi,t , · · · , βi,t ) denotes a K-vector of risk exposures for asset i = 1, ..., Nt+1 and
λt = (λ1,t , · · · , λk,t , · · · , λK,t )′ is a K-vector of risk prices. The fundamental problem (and the starting
point of the first step in FM regressions) is that neither the risk exposures nor the risk prices are directly
observable and, therefore, must be estimated first. The standard procedure assumes (e.g. see Bai and
Zhou (2015)) that returns are governed by a K-factor model, which – expressed in vector and matrix
notation – yields
(1)

(K)

r t+1 = αt + β t f1,t+1 + · · · + β t

fK,t+1 + ϵt+1 = αt + B t f t+1 + ϵt+1

(6)

for the cross-section of returns, with r t+1 = (r1,t+1 , · · · , rNt+1 ,t+1 )′ is an Nt+1 -vector of excess returns,
(1)

(K)

βt , · · · , βt

(1)

(K)

are Nt+1 -vectors of the multiple-regression betas, B t = (β t , · · · , β t

) is an [Nt+1 × K]

matrix, and f t+1 = (f1,t+1 , · · · , fk,t+1 , · · · , fK,t+1 )′ , αt = (α1,t , · · · , αNt+1 ,t )′ , with Et [αt ] = 0, and let
ϵt+1 = (ϵ1,t+1 , · · · , ϵNt+1 ,t+1 )′ be an Nt+1 -vector of errors.
In the first step of the classical FM procedure, we find the beta estimates through linear time-series
regressions for each asset i = 1, ..., N , where Fama and MacBeth (1973) suggest a rolling regression
approach to induce a time-variation in the betas (see also Chen et al. (1986), Ferson and Harvey (1991)
or Petkova and Zhang (2005))), following a fixed-window look-back period of window size W , such that
(i)

r i,t+1 = ht (F t+1 ) = αi,t + F t+1 β ′i,t + ϵi,t+1 , ∀i = 1, ...., N, ∀t = W, ..., T

(7)

where N denotes the total number of assets in the sample, r i,t+1 denotes a W -vector of excess returns
(1)

(k)

(K)

for asset i, β i,t = (βi,t , · · · , βi,t , · · · , βi,t ) is a K-dimensional vector, and F t+1 is a [W × K] matrix of
observable factors. Consequently, running rolling time-series regressions for each asset across time yields
E[β̂ i,t |F t+1 ] = β̂ i,t = (F ′t+1 F t+1 )−1 F ′t+1 r i,t+1 , ∀i = 1, ..., N, ∀t = W, ..., T

(8)

While equation (8) can also be estimated over the entire sample, there is compelling empirical evidence
in favour of time-varying betas (e.g. see Jagannathan and Wang (1996b) or Adrian et al. (2015)) which is
why we concentrate on a rolling estimation. Most importantly, in the standard Fama-Macbeth procedure,

12

equations (6)-(7) assume that the the factors f t+1 are observable and may or may not be tradable. The
nuance on tradeability is subtle but important: as discussed by Giglio and Xiu (2016), if the factors are
themselves portfolios (i.e. they are tradable), the risk premium may be directly estimated as the timeseries average of the excess return of the factor (see Cochrane (2009) for a detailed discussion). However,
investors may be concerned about non-tradable risks (i.e. risks that are not themselves portfolios, such
as inflation, consumption or liquidity), requiring the second pass in the FM procedure.
Once the betas are estimated, they are considered observable and substituted into equation (6).
Subsequently, the second step consists of T cross-sectional regressions such that the conditional asset
pricing model takes the form
Et [r t+1 ] = gt (β̂ t ) = β̂ t λt

(9)

and the prices of risk are estimated through T cross-sectional regressions
′

′

λ̂t = (β̂ t β̂ t )−1 β̂ t r t+1 , ∀t = 1, ..., T

(10)
(1)

(K)

where λ̂t = (λ̂1,t , · · · , λ̂K,t )′ is a K-vector of estimated risk prices at t, and β t = (β t , · · · , β t

). For

simplicity, note that equation (9) does not include a constant (e.g. see Cochrane (2009)), such that the
intercepts are pricing errors. Typically, the unconditional risk price is estimated as the overall time-series
average (e.g. see Green et al. (2017))
T

λ̂k =

1X
λ̂k,t .
T t=1

(11)

We do not intend to discuss all econometric properties – such as standard errors –– of the estimators in
their entirety as they are well-documented and refer readers to Shanken (1992), Jagannathan and Wang
(1998), Cochrane (2011) and Bai and Zhou (2015).
In the following, we build on Ullah (1988), Dixon and Polson (2019) and Farrell et al. (2021) to
(i)

generalise the first and second pass of the classical FM procedure and assume that ĥ(i) , respectively ĥt

and ĝt , are consistent estimators (which are collective denoted by fˆ) of the true but unknown function
f∗ , that is continuous and differentiable everywhere. Moreover, we require fˆ to be Lipschitz continuous,
meaning that there is a positive real constant A such that ∀x1 , x2 ∈ RK , ∥fˆ(x1 ) − fˆ(x2 )∥ ≤ A∥x1 − x2 ∥.
Then, for the first pass of the two-pass procedure, the partial derivative of the estimator function with
respect to its inputs is bounded and a consistent estimator of the risk exposures such that
(k)

β̂i,t (fk,t+1 ) =

∂ ĥ(i) (F t+1 )
= ∇k ĥ(i) , ∀ i = 1, ..., N ; ∀ k = 1, ..., K
∂fk,t+1

(12)

respectively
(k)

(k)

(k)

(k)

β̂i,t (f t+1 ) = β̄ i,t (f t+1 ),

13

(13)

with
(i)

(k)

(k)

β i,t (f t+1 ) =

∂ ĥt (F t+1 )
(k)

f t+1

(i)

= ∇k ĥt , ∀ i = 1, ..., N ; ∀ k = 1, ..., K; ∀ t = W, ..., T.

(14)

Equation (12) describes the simplest case of estimating h for asset i once over the entire sample, whereas
equations (13) to (14) generalise the rolling window estimation from equation (8). Thus, in equation (14),
(i)

ĥt is a rolling window estimate over a fixed-window look-back period os size W for each asset i = 1, ..., N ,
(k)

where F t+1 denotes a [W × K] matrix of observable factors11 , and f t+1 is a W -dimensional vector of the
(k)

k-th factor. In equation (13), β̂i,t denotes the fixed response estimate, which requires an aggregation of
(k)

the W -vector of partial derivatives summarised by β i,t . The aggregation into a fixed response estimate
is further highlighted by the bar notation in equation (13). We discuss various aggregation forms in
section 2.3, but examples include the expected value (e.g. see Ullah (1988) or the median.
Analogously for the second pass, the partial derivatives of ĝt with respect to the risk exposures yield
consistent and bounded estimates for the prices of risk, where
(k)

(k)

λ̂k,t (β̂ t ) = λ̄k,t (β̂ t ),

(15)

with
(k)

λk,t (β̂ t ) =

∂ĝt (β̂ t )
(k)

∂ β̂ t

= ∇k ĝt , ∀ k = 1, ..., K, ∀ t = 1, ..., T,

(16)

(k)

where ĝt was estimated cross-sectionally at each t, and λ̄k,t (β̂ t ) is a fixed response estimate.
From equations (12)-(16) it is easy to see that if h and g are linear in factors, respectively betas,
the partial derivatives yield the classical Fama-Macbeth regressions as a special case. However, there
is little intuitive justification for the strong assumption of linearity. In contrast, there exists strong
empirical evidence in favour of non-linear return dynamics (e.g. see Campbell and Cochrane (1999), He
and Krishnamurthy (2013), Pohl et al. (2018) or Gu et al. (2020a)). Consequently, we do not intend to
impose any restrictions on the functional forms of h and g, allowing for (potentially strong) nonlinearities,
which offers advantageous properties, especially in the context of non-parametric estimation, which we
discuss in section 2.3.

2.2

Special Case: Observable Risk Exposures – Firm Characteristics

The previous section discussed the generalisation of the Fama-MacBeth two-pass procedure. In particular, we showed that the starting point of the first pass is that neither the betas nor the prices of risk are
observable. Consequently, the betas must be estimated first through time-series regressions. However,
Fama and French (2020) use the insight from Fama (1976), that the coefficients from cross-sectional regressions of excess returns on firm characteristics are equivalent to zero-investment portfolio returns, and
11 Note that in equation (12), F

t+1 denotes a [T × K] matrix of observable factors.

14

show that if the cross-sectional regressions are stacked over t, the setup becomes an asset pricing model.
In other words, firm characteristics are treated as observable and time-varying betas. There exists a
myriad of research that investigates the explanatory power of firm characteristics in the cross-section
of returns (e.g. see Fama and French (1992, 1996), Daniel and Titman (1997), Avramov and Chordia
(2006), Chordia et al. (2015), Lewellen (2015), Kelly and Pruitt (2015) or Freyberger et al. (2020)).
Prominent examples include firm characteristics such book-to-market equity, 1-month momentum, size
or investments, but modern asset pricing literature lists over 300 different firm characteristics candidates
(e.g. see Mclean and Pontiff (2016), Harvey et al. (2016) or Hou et al. (2020)).
When we treat firm characteristics as observable and time-varying betas – analogously to Fama and
French (2020) – we skip the first step of the FM two-step procedure and directly run T cross-sectional
regressions of excess returns on firm characteristics in order to estimate the prices of risk. Consequently,
(1)

(k)

(K)

we substitute β̂ t = ct into equations (15) and (16), with ct = (ct , · · · , ct , · · · , ct

(k)

), where ct

denotes an Nt+1 dimensional vector of the k-th observable firm characteristic for assets i = 1, ..., Nt+1 .
It follows that the k-th time-varying price of risk is
(k)

(k)

λ̂k,t (ct ) = λ̄k,t (ct ),

(17)

with
(k)

λk,t (ct ) =

∂ĝt (ct )
(k)

∂ct

= ∇k ĝt , ∀ k = 1, ..., K, ∀ t = 1, ..., T

(18)

(k)

where ĝt was estimated cross-sectionally at each t, and λ̄k,t (ct ) is a fixed response estimate which
(k)

(k)

can, for example, be defined as λ̄k,t (ct ) = Et [λk,t (ct )] (e.g. see Ullah (1988)), or see 2.3 for a more
comprehensive discussion. Note that the firm characteristics in equation (20) and (18) are treated as
observable to an investor at t.
The general model setup in equation (6) introduces a return forecasting approach using lagged firm
characteristics that is common in the empirical asset pricing literature. Our approach differs from a pure
forecasting exercise in that we are not only interested in estimating gt consistently, but also deriving
prices of risks as well as deeper model insights. The aspect of model interpretability and communication
of inner model mechanics in particular is underrepresented in current literature. We, therefore, intend
to shift the focus from a pure forecasting perspective and introduce elements of model interpretability.
Moreover, our approach yields an asset pricing model analogously to equation (1).
One of the key problems in current literature is that there is an abundance of cross-sectional predictors (firm characteristics) to chose from. Moreover, a large number of proposed predictors are highly
correlated, which calls for the need of a parsimonious model representation to effectively navigate the
current factor zoo. In a first step, equations (20) and (18) do not introduce the possibility for model
selection as they merely focus on a generalised way to estimate prices of risk as the partial derivatives
15

of ĝt with respect to its inputs (firm characteristics). Equally, equations (20) and (18) do not limit the
number of firm characteristics that can be used for return estimations, making the setup very flexible.
However, we introduce nonlinear and non-parametric model selection that is conform with the above
introduced methodology as part of the objective function in section 3.3.
Goyal (2012) points to the problem that in a high-dimensional setting, the amount of independent
information in firm characteristics is unclear. Backed by the empirical analyses conducted by Messmer
(2017a) or Gu et al. (2020a), we follow a non-parametric approach analogously to the second step
discussed in section 2.1, and allow for nonlinear interactions between firm characteristics through the
flexible functional form of gt .
Due to its popularity in the empirical asset pricing literature, we adopt the ”one-step” procedure
introduced in this section, which treats firm characteristics as observable betas, in an empirical study
in section 4, which also makes our results directly comparable to Green et al. (2017). Consequently,
we merely concentrate on the estimation and statistical inference of risk prices assuming we observe the
betas through firm characteristics in the following. However, all subsequently introduced methodologies
are analogously applicable to the beta estimates.

2.3

Economic Interpretation and Discussion

In the following, we discuss the benefits and economic interpretation of the proposed methodology,
starting with the first pass of the two-pass procedure. First, nonlinear estimator functions yield timevarying estimates. Equation (12) shows that even if we estimate h only once over the entire sample, the
resulting risk exposure estimates are still time-varying since they are a function of the time-varying factors
f t+1 (e.g. see Kapetanios (2007)). The same also applies for the estimation of risk prices. In the context
of non-parametric modelling, a single estimation is particularly beneficial as it dramatically reduces
computational costs. Single estimations, rather than periodic model re-fitting, are not uncommon in the
empirical asset pricing literature using machine learning (e.g. see Gu et al. (2020b)). Moreover, there
exists a long-standing debate about the time-varying nature of betas and the prices of risk (e.g. see Fama
and French (1989), Harvey (1989), Chen (1991), Ferson and Harvey (1991, 1993), Ferson and Korajczyk
(1995), Ghysels (1998), Gagliardini et al. (2016) or Umlandt (2020)), which is not fully explored yet.
On top of existing empirical evidence indicating time-variations in the risk exposures and prices of risk,
we argue that due to the ever-changing nature of financial markets, the introduction of novel regulation
and disruptive technologies, allowing for the time-varying estimation of risk exposures and prices of risk
is reasonable. Therefore, the proposed methodology is aimed to contribute to the on-going debate about
the time-varying nature of risk exposures and prices of risk by offering an alternative way of estimation.
However, to make our proposed methodology directly comparable to traditional FM regressions, we
also allow for periodic re-fitting. While this also applies for the estimation of risk exposures (which,
16

for example, could involve a rolling or expanding window approach, see equation (14)), we specifically
emphasise this approach for the cross-sectional risk price estimations as it directly corresponds to the
original FM regressions (see equation (16)).
Equation 16 raises some questions about the economic interpretation of the partial derivatives with
respect to the betas (or firm characteristics in equation (18)) – the Nt -dimensional vector12 λk,t . In the
linear regression case, Fama (1976) and Fama and French (2020) show that the regression coefficients
from cross-sectional regressions ∀t are equivalent to the month t zero-investment portfolio with weights
for the assets that set the month t − 1 portfolio value of that variable to one and zero for all other,
such that the portfolio weights are summarised by W t = (β ′t β t )−1 β ′t . Fama (1976) point out that OLS
PNt+1
implies a zero-investment portfolio, since i+1
wik,t = 0. In other words, ordinary least squares yields
the portfolio weights for forming the k-th risk factor portfolio, with all constraint discussed in Fama
(1976). From this perspective, the partial derivative can be seen as a marginal return. To see why this
is, let us assume returns are linear in firm characteristics as summarised in equations (9) and (10). Thus,
if the conditional estimator function is linear in firm characteristics, it follows that

λ̂k,t = λ̂ik,t =

∂ĝt (Ct )
= (Ct Ct′ )−1 Ct′ Rt+1 ,
∂cik,t

∀i

(19)

where C denoted the [Nt+1 × K] matrix of firm characteristics, and R is an Nt+1 -vector of returns.
Most importantly, however, equation (19) states that the partial derivative conceptually coincides with
a portfolio return. The reason why we refer to this portfolio return as marginal is, that we can estimate
the partial derivative on the asset-level: the partial derivative is also a sensitivity, which quantifies the
expected change in expected return for a particular asset, given a unit change in the k-th characteristic.
In equation (19), however, if g is linear, the marginal portfolio return is invariant across assets.
However, if g is nonlinear, the partial derivatives (and thus the marginal returns) vary across assets
and time, yielding a distribution of marginal portfolio returns, requiring an aggregation into a single
risk portfolio return (e.g. Kapetanios (2007)). Ullah (1988), for example, proposes an expected value
(k)

approach such that Et [λk,t (β̂ t )] denotes an aggregated estimate of the price of risk. This approach is
also followed by Dixon and Polson (2019)13 . In the case of the expected value, and analogously to the
weight matrix in case of linear regressions, the portfolio weight for each asset is assumed to be 1/Nt+1 .
However, the functional form of the aggregated portfolio remains unknown, which is why the aggregation
is not limited to the expected value approach. In this paper, for example, we propose the median as a
form of aggregation, which is a more robust estimate in the case of skewed partial derivative distributions.
Therefore, the economic interpretation of the aggregated derivative or price of risk is the expected
12 Note that for notational simplicity we do not apply the commonly used hat notation to indicate an estimate.
13 While Dixon and Polson (2019) do not explicitly mention the usage of the expected value in their paper, they seem to
apply it in their published code which can be found on https://github.com/mfrdixon/Deep_Fundamental_Factors/blob/
master/DNNs_vs_OLS.ipynb.

17

change in return given a change in risk exposure across all assets. The reference to the risk price’s
conformity to all assets is subtle but important as it aligns the partial derivative approach with classical
asset pricing theory, where we assume that the price of risk must be the same for all assets – merely the
asset-specific risk exposure (or beta) causes cross-sectional variations in expected returns.
Empirically, however, we are confronted with a dilemma as the estimated risk prices naturally depend
on the investment universe choice. This choice induces an unavoidable arbitrariness, as it is up to the
researcher to decide which assets to include or what time horizon to consider in an empirical study. For
example, the empirical US-only investment universe considered by Gu et al. (2020b) includes almost
30,000 assets, while Chen et al. (2019) only consider assets for which all firm characteristics are always
fully observable (with no missing values), yielding an investment universe of 10,000 US-only assets. Other
studies, such as Hou et al. (2020) explicitly account for the influence of micro-caps and also exclude all
financial firms. This problem of fuzziness has recently gained more attention in the empirical asset
pricing literature, in particular in the context of result replication (e.g. see Jensen et al. (2021)).
Our proposed methodology helps to structure some of the above-mentioned empirical difficulties.
Since the estimated risk prices are aggregates of the partial derivatives, we could also estimate λ on
subsamples, such as by industry, market capitalisation or any other grouping of interest. This estimation
would yield an industry-specific (or market capitalisation-specific) price of risk, making our proposed
methodology more flexible. Moreover, a subsample-specific estimation does not require a separate model
re-fit, making the estimation computationally efficient, which is particularly appealing in the context
of machine learning, where the estimation of g might be computationally very expensive. Furthermore,
modern non-parametric models tend to be data-intensive, requiring as much training data as possible.
Consequently, excluding entire industries or size classes from the training data set would reduce the
training set dramatically, making complex model estimations more difficult. Thus, we may wish to
re-write equation (15) as
(s)

(k)

(k)

λ̂k,t (β̂ t ) = λ̄k,t (β̂ t ) i∈S ,

(20)

where S denotes a subsample of choice, for example, industries or groups of market capitalisation, and
the bar notation denotes an aggregate such as the expected value or the median. Appendix I provides
empirical details regarding the concerns mentioned above. As an example, classic FM regressions using
OLS yield an out-of-sample predictive R2 of −0.35% when evaluated across all assets, with manufacturing
assets – which make up the most considerable portion of assets – yielding an out-of-sample predictive
R2 of −0.22% while agricultural assets – which make up a minor portion of all assets – only yield an
out-of-sample predictive R2 of −2.15%. Without providing further empirical details at this point, this
short excursion alone provides initial evidence that we may be interested in a more granular analysis
than an overall aggregate across all assets. Even if we are only interested in an overall estimate of risk
prices that apply to all assets at time t, our proposed methodology still offers valuable attributes due to
18

the more granular model insights resulting from partial derivatives’ distribution. For example, a separate
risk price estimation by industry or market capitalisation may still help detect biases in the estimation,
serve as a general sanity check, or even be helpful for code debugging.
While the proposed risk price estimation in equations (13), (15) and (20) fundamentally rely on
aggregation, we may still be interested in the asset-specific partial derivatives as they offer valuable
insights into the inner model mechanics and significantly improve model interpretability. Those assetspecific partial derivatives are summarised in the Nt × K matrix λt and, besides their interpretation
of being marginal portfolio returns, capture an asset’s return’s sensitivity to changes in the k-th risk
exposure. The possibilities of analyses using those derivatives are too extensive to be discussed in their
entirety in this paper. However, possible avenues include, but are not limited to, pairwise locally-weighted
regression analyses, unsupervised clustering to discover intrinsic sensitivity clusters, correlation analyses
investigating cross-asset sensitivities, long-short portfolio construction based on sorted derivatives, or
distributional analyses.
In this paper, we limit the interpretability discussion to pairwise locally-weighted regressions, distributional analyses, and investigate the k-th partial derivatives in relation to the k-th input variable, or in
relation to the j-th input variable or partial derivative, where j ̸= k. We are particularly interested in the
general distributional properties such as skewness, asset-specific outliers, and the effect of the objective
function which we discuss in section 4.12. Despite the fact that we do not pursue the idea any further,
it is worth mentioning that the estimation and evaluation of risk prices on subsamples naturally leads to
the notion of weak and strong risk factors, where we follow the definition of Lettau and Pelger (2020a)
and define strong risk factors as factors that affect all underlying assets. In contrast, weak factors only
affect a subset of the underlying assets. Moreover, time-varying risk compensation further closely relates
to the topic of factor timing, which is particularly heavily researched by practitioners (e.g. see Dichtl
et al. (2019)).
With this paper, we, therefore, intend to contribute to the on-going debate about the time-varying
estimation of risk exposures and prices of risk and model interpretability. In particular, we do not report
our findings as point estimates as it is typically done, but as time-varying estimates with confidence
(respectively tolerance) intervals. Moreover, in section 3.3 we further introduce non-linear and nonparametric model selection which helps to navigate the factor zoo in current literature. Most importantly,
the methodology above is intended to offer a wide variety of applications that is universal for a large
class of estimator functions and that can be tailored to the specific means of the respective stakeholder.
We discuss a class of estimator functions that fulfil these above-mentioned requirements in section 3.
The overarching theme, however, is that we intend to move elements of model interpretability and
communications of inner model mechanics to the foreground.

19

Figure 1:
Smoothing effect – example: asset-growth
The graph displays the smoothing effect exemplified by asset growth. The solid blue line corresponds to monthly
Fama-Macbeth regressions, while the dotted blue line visualises the respective unconditional estimate. The
solid green line displays the smoothed estimated through a rolling window average, with the green dotted line
corresponding to the unconditional estimate from the smoothed estimate. For direct comparison, the dotted red
line visualises the original estimate reported by Green et al. (2017).

2.4

Smoothing

There is considerable empirical evidence and rational economic intuition that the risk exposures and the
prices of risk vary smoothly over time (e.g. see Adrian et al. (2015) or Ang and Kristensen (2012)). One
of the returning arguments is that asset prices, and hence returns, are connected to economic cycles,
as argued, for example, by Ferson et al. (1987), Fama and French (1989), Harvey (1989) or Campbell
(1999), with economic cycles also evolving smoothly over time. As a consequence, we may wish to
smooth out the risk price estimates to only allow for gradual changes over time. A complete discussion
of various smoothing regimes is beyond the scope of this paper and shall not be the main focus. However,
the previously introduced methodology for risk price (and exposure) estimation can also be smoothed
using standard smoothing techniques, such as a rolling window estimations or backwards-looking rolling
averages. Lewellen (2015), for example, reports a ten-year rolling average of Fama-MacBeth slopes, which
are estimated using standard OLS regressions. As an example, consider the univariate OLS regression
model, where we regress adjusted excess returns on the firm characteristic asset growth analogously to
Green et al. (2017)14 . The dotted lines in figure 1 display the unconditional risk price estimates, which
are derived from the time-series average of the conditional estimates visualised by the solid lines. It can
be seen that our unconditional estimates are almost identical to the original value reported by Green
et al. (2017)15 . However, it can also be seen that the conditional risk price estimate resulting from crosssectional FM regressions for every t = 1, ..., T are not smooth. In contrast, the solid green line visualises
the smoothing effect of an exemplary 5-year rolling window average of the regression coefficients.
In this paper, we follow a similar approach. Section 2.3 discusses the fixed estimation of λ̂k,t , which
14 See appendix A for a detailed variable definition
15 See appendix E for further details on a replication study.

20

is the nonlinear equivalent to the conditional Fama-Macbeth estimates visualised by the solid blue line
in figure 1. We smooth the risk price estimates, similarly to Lewellen (2015), by estimating λ̂k,t through
a five-year backwards looking rolling window approach. The rolling window approach is a balanced
trade-off between achieving a desirable level of smoothness and taking potential economic cycles into
account. We further investigate an expanding window approach as part of the robustness checks, with
results reported in appendix I.

2.5

Empirical Inference

For empirical inference, we diverge from to the traditional approach of reporting unconditional risk prices
as time-series averages following equation (11), where under the null risk prices are zero and the null
hypothesis is traditionally tested using asymptotic test statistics (e.g. see Gibbons et al. (1989)). In
contrast, we propose a data-driven approach to report the fixed response estimates along with timevarying confidence intervals. Thus, risk prices transition in and out of statistical significance over time,
where statistical significance is defined as a confidence interval that does not include zero. This timevarying statistical significance offers the opportunity to analyse estimated prices of risk at different points
in time, which may be of particular interest, for example, times of economic crisis.
In particular, we propose bootstrapping to obtain standard errors for the risk price estimates. However, we acknowledge that bootstrapping can be computationally expensive and, therefore, be an infeasible option in practice. Consequently, we also provide alternative estimation strategies, which are
computationally inexpensive and receive more practical attention at the cost of being less rigorous. As
pointed out by Cochrane (2009), modern asset pricing does not necessarily need to rely on asymptotic
theory to calculate accurate standard errors. Monte Carlo simulations or bootstrapping offer competitive
data-driven alternatives, especially in small sample, with Dixon and Polson (2019) proposing a similar
approach.
Following Kapetanios (2008), we propose non-parametric bootstrapping and define the non-parametric
∗(1)

bootstrap sample as {r ∗t+1 , ct

∗(K)

, · · · , ct

}, where the star notation denotes some form of cross-sectional

resampling from the original return and characteristics data16 . Thus, for example, we denote r ∗t+1 =
(rj1 ,t+1 , · · · , rjj ,t+1 , · · · , rjNt+1 ,t+1 )′ a Nt+1 -dimensional vector of re-sampled returns, where the vector
of indices (j1 , · · · , jNt+1 )′ is obtained by drawing with replacement from (1, · · · , Nt+1 )′ . The same vector
of indices is used to draw from the betas as well. Kapetanios (2008) points out that the re-sampling
can be adjusted for cross-sectional dependence. We further remark, that the re-sampling can also be
adjusted to asset pricing specific data specifics: in section 4, we introduce the standard CRSP and Compustat US-only dataset, which is the gold standard in empirical asset pricing. It is well-known, however,
that the dataset is strongly misbalanced, especially with regard to microcap stocks (e.g. see Hou et al.
16 Note, that parametric bootstrapping would also be possible, but an in-depth discussion is beyond the scope of this
paper.

21

(2020)). The above-mentioned re-sampling can, therefore, also follow a stratified approach, where the
re-sampling is done by sub-groups, such as market capitalisation and (or) industry.
For each bootstrap iteration b = 1, · · · , B, we re-fit gt , such that ĝt,b denotes the estimated function
(b)

using the b-th re-sampled dataset and λ̂k,t the b-th fixed response estimation. Subsequently, standard
q
PB
PB
(b)
(b)
1
1
errors are SE(λ̂k,t ) = B−1
b=1 λ̂k,t − λ̄k,t , with λ̄k,t = B
b=1 λ̂k,t , see Efron and Tibshirani (1986).
We propose confidence intervals to be CIk,t = (−ρ × SE(λ̂k,t ), ρ × SE(λ̂k,t )), where ρ ≥ 3 guided by
Harvey et al. (2016) who argue that higher statistical hurdles are required to account for for multiple
testing and data mining issues.
We acknowledge, that the above-introduced bootstrap approach may be infeasible in practice, if the
estimation of g is computationally expensive and B is large, for example B = 1, 000. With this paper,
we intend to take real-life computational costs into account. In the following, we, therefore, introduce
computationally less expensive alternatives at the cost of being less rigorous. A full description is beyond
the scope of this paper and we refer readers to Kapetanios et al. (2019) for a more detailed discussion.
The alternatives include:
1. Instead of re-fitting ĝ for each bootstrap iteration, keep the original estimate and merely derive B
(b)

(b)

fixed response estimates, using the re-sampled data only such that λ̂k,t = λ̄k,t (ck,t ) denotes the
(b)

fixed response estimate derived from the b-th re-sampled characteristics data denoted by ck,t , with
q
PB
(b)
1
SE(λ̂k,t ) = B−1
b=1 λ̂k,t − λ̄k,t .
2. Instead of bootstrapping return and characteristics data to derive bootstrapped partial derivatives,
(k)

(b)

we can also bootstrap directly from the distribution λk,t (ct ), such that λk,t denotes the b-th crossq
PB
(b)
1
sectionally re-sampled Nt+1 -vector of partial derivatives, with SE(λ̂k,t ) = B−1
b=1 λ̂k,t − λ̄k,t .
3. Instead of performing any type of bootstrap, estimate distributional percentiles directly from
(k)

λk,t (ct ), if Nt+1 is large.
4. Use the backwards-looking rolling standard deviation approach to construct tolerance intervals or
tolerance bands, such that TIk,t = (−ρ × σ(λ̂k,t ), ρ × σ(λ̂k,t )), where σ is estimated through a
backwards-looking rolling window approach.
For all methods, except the fourth alternative, the re-sampling can either be done at time t, time s,
where s ̸= t, or over the entire sample. There are, however, certain real-life data issues which should be
considered. If, for example, g is estimated using machine learning (which we discuss further in section
3 and is common in modern empirical asset pricing), the input data is typically pre-processed in a way
that it is either cross-sectionally rank standardised or cross-sectionally rank-normalised. As a result, the
data is equally distributed across all t. If in addition Nt+1 is very large, it becomes irrelevant at what
t the data was re-sampled. For the purpose of computational ease and simplicity, we follow the fourth
22

alternative in our empirical application. Due to the real-life considerations, we utilise the term tolerance
interval or tolerance bands, where risk premia are considered empirically important when the tolerance
bands do not include zero. To impose a rigorous empirical hurdle, the tolerance bands are constructed
with ρ = 3, in the style of Harvey et al. (2016). We appreciate that the derived empirical importance is an
ad-hoc approach, but more robust estimates are infeasible in practice due to an exponentially increasing
computational cost.

2.6

Comparison to Other Interpretability Measures

There exist a number of interpretability measures in current literature, in particular with regard to
nonlinear estimator functions. We do not intend to provide a holistic overview here and concentrate on a
brief comparison of our proposed methodology to the most commonly used measures and measures that
most closely relate to our methodology, including partial dependence, individual conditional expectation,
accumulated local effect, local interpretable model-agnostic explanations (LIME) and Shapley values. For
a more detailed discussion, we refer readers to Doshi-Velez and Kim (2017), Molnar (2020) and Miller
(2019). In general, there is a differentiation between model-agnostic and model specific interpretability,
where model-agnostic interpretability is universally applicable and, thus, very flexible. We see our
proposed methodology as a hybrid of model-agnostic and model specific interpretability tools, as it
generalises to a large number of models under the conditions discussed in section 2.1 (e.g. see Ribeiro
et al. (2016b)).
Partial dependence plots (PDP), introduced by Friedman (2001), are among the most popular graphical interpretability tools and show the functional relationship between model inputs and the model’s
predictions. In particular, they plot an average effect an input variable has on the prediction. Our
methodology diverges from PDP in two key ways. First, our methodology is not concerned about an
average effect, but instead offers asset-specific model insights, which is, therefore, much more granular.
Second, partial dependence plots assume that the input variables are independent, an assumption that
is not required for our methodology.
Individual conditional expectation (ICE) expand the idea of partial derivative plots to the individual
item level, see Goldstein et al. (2015). In our setting, this would correspond to an individual line per
asset, which can result in overcrowded plots if N is large. While this makes an analysis on asset-level
possible, it does not allow for analyses in relation to other input variables. Moreover, our proposed
methodology offers the opportunity to even investigate sensitivities between each other.
Neither partial dependence plots nor individual conditional expectations can deal with correlated
input data. Accumulated local effect (ALE), introduced by Apley and Zhu (2020), offers a solution to
this problem by visualising how model predictions change in a small window of the input data around
a given point, by using differences in predictions rather than averages and taking changes in another
23

variable into account. However, ALE does not allow for asset-level investigations.
Local interpretable model-agnostic explanations (LIME), proposed by Ribeiro et al. (2016a), intends
to make any (machine learning) classifier or regressor interpretable through local interpretable approximations, which may not generalise globally, such as linear models or devision trees. While the local
approximation offers an intuitive motivation for model interpretability is poses the problem of correctly
defining the neighbourhood in which the approximation is taking place. With our methodology, we
also offer great interpretability through the interpretability lens, but do not need to worry about local
permutation and neighbourhood definitions.
Shapley values originates from game theory and coined by Shapley (1953), is a method for evaluation
”payouts” to ”players”, depending on their contribution to the ”total payout” and is defined as the average marginal contribution of a feature value across all possible coalitions. One of the key disadvantages
is that the computational cost increases exponentially with the number of input variables. While the
methodology has become increasingly popular, the model insights for asset pricing-specific interpretability is limited compared to the partial derivative approach, due to the computational cost (we typically
have to deal with a large number of input variables) and the limited insight on asset-level.
Last but not least, Dimopoulos et al. (1995) propose a neural network specific interpretability tool
that is fundamentally based on the sum of the squared norm of partial derivatives of a neural network
with respect to its input. It is the tool that most closely relates to our methodology, but differs in two key
ways: first, the measure is an aggregate, meaning that it does not allow for asset-specific model insights.
Second, it does not allow for relative analyses in which we compare and contrast model sensitivities in
relation to other sensitivities.
What separates our approach from all the above-mentioned alternatives is, that model interpretability
is not a separate step and, therefore, does not involve a separate estimation. Instead, it is integral part
of the estimation already as we use the partial derivatives to estimate risk prices. This makes our
methodology readily available, computationally inexpensive and easily implementable. In addition, in
section 3.3, we introduce an objective function that directly makes use of the partial derivatives as part
of the optimisation. Furthermore, the possibility of further analyses does not stop with the partial
derivatives themselves. As discussed in the previous section, we can use the partial derivatives and run,
for example, unsupervised learning on top, which makes our methodology much more flexible to common
alternatives.
Although we do not follow this approach further at this point, it is theoretically conceivable, that
interpretability measures, such as Shapley values, are directly incorporated into an objective function,
analogously to the Jacobian in equation (30). We appreciate that such objective function can be computationally expensive and, thus, infeasible in practice. However, our approach naturally eludes to the
concept of penalising other forms of variable significance.
24

3

Machine Learning Estimation

In this section, we concentrate on the estimation of gt and discuss a class of estimator functions G that
satisfies the assumptions addressed in section 2 (e.g. see Farrell et al. (2021)). Although our proposed
methodology generalises to a large number of possible estimator functions under the condition of differentiability, we particularly focus on deep neural networks in this section due to their recent popularity
in empirical asset pricing (e.g. Messmer (2017a), Gu et al. (2020a) and Chen et al. (2019)). However,
we are not necessarily plain supporters of neural networks, as their training can be computationally
expensive and data-intensive, especially if their architecture is complex. Those aspects can be limiting in
practice. In addition, practitioners frequently express their scepticism towards neural networks, partly
because out-of-the-box architectures and training regimes do not necessarily provide significantly better
(computational cost-adjusted) performances. In this context, De Prado (2018) points out what we can
verify empirically: the architectural design and training of a competitively performing neural network requires in-depth domain knowledge in asset pricing, especially with regards to data-specific nuances, and
machine learning theory, with plain vanilla out-of-the-box models performance being underwhelming,
which we discuss in section 4.
We show that extensive hyperparameter tuning and asset pricing-specific training regimes are required to match standard and competitive benchmark models. Most importantly, however, our proposed
methodology sheds light on the complex inner network mechanics that are often perceived as ”black
boxes”. Our methodology, hence, significantly improves model interpretability and communication to
various stakeholders. On the other hand, improved model interpretability can help eradicate some of the
scepticism towards neural networks, making them a viable model choice.
The deep neural network literature is exceptionally vast and interdisciplinary, ranging from fields
such as natural language processing over image recognition to asset pricing. Each domain requires
specific tweaks to the data preprocessing, architectural design and training regimes. We find that asset
pricing-specific solutions currently being underrepresented in literature, as most studies primarily apply
out-of-the-box schemes. With this paper, we intend to transfer and combine interdisciplinary approaches,
by, for example, incorporating estimation strategies stemming from the field of image recognition and
giving them an interpretable meaning in asset pricing (see section 3.3). However, due to the sheer
amount of literature, we cannot provide a holistic overview of all potential model architectures, training
or regularisation schemes, so we primarily focus on fully connected feed-forward networks in this section.
For a more detailed and general discussion of neural networks, we refer readers to Bishop (1995) and
Goodfellow et al. (2016).

25

3.1

Deep Artificial Neural Networks

Fully connected feed-forward neural networks are comprised in the collective term deep learning, a subfield
of machine learning17 . They are among the most potent estimators due to their universal approximation
capabilities, which means that they can approximate an arbitrary function with arbitrary accuracy (e.g.
see Hornik et al. (1989) or Cybenko (1989)), with Farrell et al. (2021) showing that they are consistent
estimators. In the empirical asset pricing literature and other disciplines, they are currently among the
most popular model choices because of their competitive out-of-sample performances (e.g. see Messmer
(2017a), Chen et al. (2019), and Gu et al. (2020a)). Section 1 introduced current challenges in modern
asset pricing, including high-dimensional datasets and pronounced nonlinearities. Neural networks thrive
in this environment through their extreme functional flexibility, which stems from the interconnection
of (potentially many) so-called layers, enabling nonlinear estimations. Figure 2 visualises an arbitrary
fully connected feed-forward neural network and displays its three core hierarchical elements: the input,
hidden, and the output layer(s). While there does not exist an exact definition, the term deep typically
refers to neural networks with more than one hidden layer. This biological imitation is the origin of the
model’s name. In simple terms, the input data flows from the left input layer through all hidden layers,
where the nonlinear transformations take place in the neurons, to the output layer, which results in a
final prediction – in our case, a return prediction analogously to equation (3).
More specifically, the model displayed consists of L + 2 layers, where l = 0 denotes the K-dimensional
input layer, with the dimension of the input layer corresponding to the input data dimension. Further,
the displayed neural network consists of L hidden layers and an output layer, where the dimension of the
output layer corresponds to the dimension of the target variable – in our case, an asset’s return. Each
hidden layer consists of Hl nodes that are all fully connected through weights with all Hl−1 nodes from
the preceding layer. In addition, the shaded nodes in figure 2 display the biases.
All nodes in the hidden layers perform a nonlinear transformation of their inputs through an activation
function, where each node’s input is a linear combination of the outputs from all preceding nodes plus
a bias. There are many potential activation function candidates. Some of the most common activation
functions are visualised in figure 3. Interestingly, and to the best of our knowledge, there does not yet
exist an asset-pricing specific activation function. Thus, the activation functions displayed in figure 3
are universally applicable across domains, including asset pricing.
While there is no clear guideline for which activation function to use for asset pricing applications,
rectified linear unit (ReLU) is the most commonly used activation function in recent empirical asset
pricing literature. Furthermore, Farrell et al. (2021) show that neural networks designed with ReLU as
activation functions in their hidden layers are consistent estimators. However, ReLU is vulnerable to the
17 There exist other types of neural networks, such as recurrent or long-short term memory (LSTM) neural networks,
which also find occasional applications in asset pricing, or at least finance or economics in general, but are beyond the
scope of this paper.

26

<latexit sha1_base64="4Ko4aQV8AkzCW55UQ6EN2dxyiSA=">AAAB63icbVBNTwIxEJ3iF+IX6tFLIzHhRHY5qEeiF4+YCJjAhnRLFxra7qbtmpANf8GLB43x6h/y5r+xC3tQ8CWTvLw3k5l5YSK4sZ73jUobm1vbO+Xdyt7+weFR9fika+JUU9ahsYj1Y0gME1yxjuVWsMdEMyJDwXrh9Db3e09MGx6rBztLWCDJWPGIU2Jz6YYTM6zWvIa3AF4nfkFqUKA9rH4NRjFNJVOWCmJM3/cSG2REW04Fm1cGqWEJoVMyZn1HFZHMBNni1jm+cMoIR7F2pSxeqL8nMiKNmcnQdUpiJ2bVy8X/vH5qo+sg4ypJLVN0uShKBbYxzh/HI64ZtWLmCKGau1sxnRBNqHXxVFwI/urL66TbbPiXDe++WWvVizjKcAbnUAcfrqAFd9CGDlCYwDO8whuS6AW9o49lawkVM6fwB+jzB97Sjgo=</latexit>

Input Layer 2 RK
<latexit sha1_base64="1W1kI21258lQ8ctVxGm3vmLgvoI=">AAACB3icbVDLSsNAFJ3UV62vqEtBBluhq5J0oS4LbhRdVLEPaGKZTCft0MkkzEyEELpz46+4caGIW3/BnX/jpM1CWw8MHM65l7nneBGjUlnWt1FYWl5ZXSuulzY2t7Z3zN29tgxjgUkLhywUXQ9JwignLUUVI91IEBR4jHS88Xnmdx6IkDTkdyqJiBugIac+xUhpqW8eXvIoVvAaJUTAikM5dAKkRp6X3k7uryp9s2zVrCngIrFzUgY5mn3zyxmEOA4IV5ghKXu2FSk3RUJRzMik5MSSRAiP0ZD0NOUoINJNpzkm8FgrA+iHQj+u4FT9vZGiQMok8PRkdqSc9zLxP68XK//MTWkWlXA8+8iPGVQhzEqBAyoIVizRBGFB9a0Qj5BAWOnqSroEez7yImnXa/ZJzbqplxvVvI4iOABHoApscAoa4AI0QQtg8AiewSt4M56MF+Pd+JiNFox8Zx/8gfH5A5KZmGU=</latexit>

<latexit sha1_base64="fCd9eKTGUSKnXo8keNdXxhP078U=">AAACDHicbVC7TsMwFHXKq5RXgZHFokXqVCUdgLESSweGguhDakPlODetVceJbAepivoBLPwKCwMIsfIBbPwNTtsBWo5k6eicc+V7jxdzprRtf1u5tfWNza38dmFnd2//oHh41FZRIim0aMQj2fWIAs4EtDTTHLqxBBJ6HDre+CrzOw8gFYvEnZ7E4IZkKFjAKNFGGhRLDeb7IPA1mYDE5T4TuB8SPfK89HZ6nzYGzrRsUnbVngGvEmdBSmiB5qD41fcjmoQgNOVEqZ5jx9pNidSMcpgW+omCmNAxGULPUEFCUG46O2aKz4zi4yCS5gmNZ+rviZSESk1CzySzRdWyl4n/eb1EB5duykScaBB0/lGQcKwjnDWDfSaBaj4xhFDJzK6YjogkVJv+CqYEZ/nkVdKuVZ3zqn1TK9Urizry6ASdogpy0AWqowZqohai6BE9o1f0Zj1ZL9a79TGP5qzFzDH6A+vzBywcmlg=</latexit>

Hidden Layer 2 RH1

<latexit sha1_base64="PN5dIFMHzsvUWDVqvga7CyNt20k=">AAACDHicbVC7TsMwFHXKq5RXgZHFokXqVCUdgLESSweGguhDakPlOLetVceJbAepivIBLPwKCwMIsfIBbPwNTtsBWo5k6eicc+V7jxdxprRtf1u5tfWNza38dmFnd2//oHh41FZhLCm0aMhD2fWIAs4EtDTTHLqRBBJ4HDre5CrzOw8gFQvFnZ5G4AZkJNiQUaKNNCiWGsz3QeBrMgWJy30mcD8geux5yW16nzQGtbRsUnbVngGvEmdBSmiB5qD41fdDGgcgNOVEqZ5jR9pNiNSMckgL/VhBROiEjKBnqCABKDeZHZPiM6P4eBhK84TGM/X3REICpaaBZ5LZomrZy8T/vF6sh5duwkQUaxB0/tEw5liHOGsG+0wC1XxqCKGSmV0xHRNJqDb9FUwJzvLJq6RdqzrnVfumVqpXFnXk0Qk6RRXkoAtURw3URC1E0SN6Rq/ozXqyXqx362MezVmLmWP0B9bnDy2imlk=</latexit>

Bias

Hidden Layer 2 RH2 · · · Hidden Layer 2 RHL
<latexit sha1_base64="exRlzt8kmDefx+eRp5HN49hCdxs=">AAACDHicbVC7TsMwFHV4lvIqMLJYtEidqqQDMFZi6dChIPqQ2lI5zm1r1XEi20GKonwAC7/CwgBCrHwAG3+D22aAliNZOjrnXPne44acKW3b39ba+sbm1nZuJ7+7t39wWDg6bqsgkhRaNOCB7LpEAWcCWpppDt1QAvFdDh13ej3zOw8gFQvEnY5DGPhkLNiIUaKNNCwU68zzQOAGiUHiUp8J3PeJnrhucpveJ/VhIy2ZlF2x58CrxMlIEWVoDgtffS+gkQ9CU06U6jl2qAcJkZpRDmm+HykICZ2SMfQMFcQHNUjmx6T43CgeHgXSPKHxXP09kRBfqdh3TXK2qFr2ZuJ/Xi/So6tBwkQYaRB08dEo4lgHeNYM9pgEqnlsCKGSmV0xnRBJqDb95U0JzvLJq6RdrTgXFfumWqyVszpy6BSdoTJy0CWqoTpqohai6BE9o1f0Zj1ZL9a79bGIrlnZzAn6A+vzB1U+mnM=</latexit>

<latexit sha1_base64="vJgH5Joe6srUQKmjLaD/9vxqQuY=">AAAB73icbVA9SwNBEJ3zM8avqKXNYSKkCncp1DJgYxnBfEByhL29TbJkb/fcnRPCkT9hY6GIrX/Hzn/jJrlCEx8MPN6bYWZemAhu0PO+nY3Nre2d3cJecf/g8Oi4dHLaNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJ7dzvPDFtuJIPOE1YEJOR5ENOCVqpW+nTSKGpDEplr+Yt4K4TPydlyNEclL76kaJpzCRSQYzp+V6CQUY0cirYrNhPDUsInZAR61kqScxMkC3unbmXVoncodK2JLoL9fdERmJjpnFoO2OCY7PqzcX/vF6Kw5sg4zJJkUm6XDRMhYvKnT/vRlwzimJqCaGa21tdOiaaULQRFW0I/urL66Rdr/lXNe++Xm5U8zgKcA4XUAUfrqEBd9CEFlAQ8Ayv8OY8Oi/Ou/OxbN1w8pkz+APn8wdiDY90</latexit>

Output 2 R
<latexit sha1_base64="2yOZ/hPj2L1StR66ENGw5ILg6dc=">AAACAHicbVC7TsMwFL0pr1JeAQYGFosWqVOVdADGSixsFEQfUhNVjuu2Vh0nsh2kKurCr7AwgBArn8HG3+C0GaDlSJaOzrlXvucEMWdKO863VVhb39jcKm6Xdnb39g/sw6O2ihJJaItEPJLdACvKmaAtzTSn3VhSHAacdoLJdeZ3HqlULBIPehpTP8QjwYaMYG2kvn1ym+g40ajiMYG8EOtxEKT3s0rfLjs1Zw60StyclCFHs29/eYOIJCEVmnCsVM91Yu2nWGpGOJ2VvETRGJMJHtGeoQKHVPnpPMAMnRtlgIaRNE9oNFd/b6Q4VGoaBmYyO1Ete5n4n9dL9PDKT5kwEakgi4+GCUc6QlkbaMAkJZpPDcFEMnMrImMsMdGms5IpwV2OvEra9Zp7UXPu6uVGNa+jCKdwBlVw4RIacANNaAGBGTzDK7xZT9aL9W59LEYLVr5zDH9gff4A3teV2g==</latexit>

Figure 2:
Example of an arbitrary deep neural network architecture
The displayed fully-connected feedfworward neural network consists of an input layer with K input nodes (analogously to a K-dimensional input vector ci,t ), l = 1, ..., L hidden layers, where Hl denotes the number of nodes
in the l-th layer, with H0 = K denoting the input nodes and a single output node. Furthermore, each hidden
layer features a fully-connected bias, visualised by the shaded nodes.

vanishing gradient problem, which is sometimes referred to as dying ReLU problem (e.g. see Hu et al.
(2021)). Section 3.2 introduces an exemplary optimisation algorithm, which can be used to train neural
networks. What unites all training algorithms is that they are fundamentally based on gradient descent
in combination with backpropagation. Thus, effective learning requires the existence of a gradient with
respect to the model parameters, particularly with respect to the neural network’s weight. However,
due to the functional form of ReLU, the vanishing gradient problem describes a scenario in which the
gradients of the network weights approach zero. Consequently, the optimisation is stuck as there is
no clear direction for the next model parameter update. To the best of our knowledge, this well-known
problem associated with ReLU has not yet been well-documented in the empirical asset pricing literature.
Leaky ReLU is an alternative activation function that is specifically designed to avoid the vanishing
gradient problem. Instead of the output being zero (i.e. producing no model parameter gradient) when
the input is negative, leaky ReLU produces a small positive slope (in PyTorch, for example, the default
value is 0.01). That is, leaky ReLU computes f (x) = 1(x < 0)(ax) + 1(x >= 0)(x), where a = 0.01.
Leaky ReLU and other alternatives that are specifically designed to avoid the vanishing gradient problem
have gained popularity in recent years in the general machine learning literature, with examples including,
Maas et al. (2013) or Xu et al. (2015). Due to the popularity and easy implementation, the empirical
application presented in section 4 also applies leaky ReLU. Besides ReLU and leaky ReLU, figure 3

27

(a) Leaky rectified linear
unit (ReLU)

(b) Rectified linear unit
(ReLU)

(c) Sigmoid

(d) Hyperbolic tangent
(tanh)

Figure 3:
Common activation functions
From left to right, the figure displays the most commonly used activation functions in the general machine
learning literature, and particularly in asset pricing. They range from leaky rectified linear unit (Leaky ReLU),
rectified linear unit (ReLU), to sigmoid and hyperbolic tangent. Section 4 primarily focuses on an application
using leaky ReLU.

summarises common activation functions in the literature, which includes the sigmoid and hyperbolic
tangent. However, a full discussion of all possible activation function candidates is beyond the scope of
this paper, and we refer readers to Nwankpa et al. (2018) for a more detailed discussion.
A natural critique for the choice of leaky ReLU in the context of this paper – and in particular
with regard to sections 2.1 and 2.2 – is that leaky ReLU is not continuously differentiable. While
mathematically correct, this issue is negligible in practice, as discussed by Goodfellow et al. (2016),
since software implementations (for example, PyTorch) are prone to rounding errors. Those rounding
errors make it very unlikely to land exactly on the singularity point. Additionally, even if the singularity
point was reached, leaky ReLU is commonly implemented so that the right-hand side derivative is used,
meaning that software implementations always produce a gradient.
The fact that there exists no clear guideline for the choice of the activation function introduces a
fuzziness as part of the empirical application. The term fuzziness in this context describes that there is
no right or wrong when it comes to the activation function choice in empirical asset pricing applications.
In this context, section 2 introduces the term hyperparameter, which describes model parameters that
cannot be optimised algorithmically as part of the optimisation procedure. Thus, hyperparameters
must be set manually instead. Their (locally) optimal value can be found, for example, through crossvalidation. The activation function choice is one of such hyperparameters. Theoretically, each node in
each layer could apply a different activation function, which would require manual tuning to find a (local)
optimum. However, to dramatically reduce the computational cost of the grid search finding (locally)
optimal hyperparameter values, we assume that each node in each hidden layer applies the same activation
function, namely leaky ReLU. Unfortunately, the list of hyperparameters does not only include the
activation function and can be extensive. The list of potential hyperparameters includes but is not limited
to: the learning rate, batch size, number of epochs, number of epochs before early stopping is activated,
penalisation parameters or even architectural hyperparameters, such as the number of hidden layers or
28

the number of nodes in each layer. Section 4.3 discusses the considered hyperparameters and their tuning
regime in detail, with a particular focus on the computational cost, since the computational cost increases
exponentially with the number of hyperparameters. While it is well-known that hyperparameter tuning
is the computational bottleneck of neural networks, we find that current literature pays little attention
to this crucial step. Thus, we intend to be as transparent as possible about our hyperparameter tuning
regime to increase reproducibility.
Mathematically, figure 2 simplifies to

gt (ci,t ; Wt , θt ) = (f1 ◦ · · · ◦ fl ◦ · · · ◦ fL )(ci,t ; Wt , θt ),

(21)

where Wt captures all learnable model parameters, such as weights and biases, while θ collects all
remaining model (hyper-)parameters. For reasons of simplicity, we assume that the activation functions,
denoted by σ, are the same in each hidden layer, such that σ = σl , ∀l. Zooming in on an arbitrary j-th
node in the l-th hidden layer, the nonlinear transformation is summarised by
Hl−1
(l)
fj = σ(

X

(l−1) (l−1)
(l−1)
(l)
xi
+ bji ) = xj ,

wji

(22)

j=1

(l−1)

where wji

denotes the weight connecting the i-th node from the preceding layer l − 1 with the j-th

node in the l-th hidden layer. The same notation applies for the biases. The output of the i-th node from
(l−1)

the preceding l−1-th layer is denoted by xi

, where for the input layer (l = 0) x0 = ci,t . Consequently,

we can rewrite equation (21) as

gt (ci,t ; Wt , θt ) = W(L) σ(· · · σ(W(l) (· · · σ(W(0) ci,t + b0 )) + bl ) · · · ) + bL ,

(23)

where W(l) denotes a Hl−1 × HL matrix, capturing all weights connecting the l − 1-th with the l-th
layer. The same notational logic applies to the biases. Note that equation (23) means that we do not
apply another nonlinear transformation through an activation function in the output node.

3.2

Standard Objective Function and Optimisation

This section reviews the standard objective function and stochastic optimisation algorithm that we use in
this paper to train deep neural networks and are commonly used in the literature. In addition, we propose
an alternative objective function that allows for non-linear model selection and sensitivity penalisation
in section 3.3.
One of the main reasons neural networks have become popular in the empirical asset pricing literature
is their data-driven estimation approach. This form of estimation does not impose any substantial direct

29

restrictions on the functional form of the estimator function. During an iterative process, called learning,
we minimise a stochastic objective function through first-order gradient-based optimisation to find an
optimal set of (learnable18 ) model parameters19 .
Such a data-driven approach is particularly appealing in data-rich environments and when the underlying true data-generating process is unclear, as in return data. However, the main disadvantage is
that fitting a neural network does not necessarily involve any economic or financial theory. Therefore,
letting the data speak for itself is prone to data-snooping or allows for statistical estimators that have
little economic intuition. We discuss solutions to these challenges in section 3.3.
We find that domain-specific knowledge is still crucial regarding the data pre-processing, the architectural design of the network or even the objective function. In this section, we review the standard
procedure of finding an optimal set of learnable model parameters, where learnable parameters refer to
model parameters that are not found manually but iteratively updated during the training procedure.
At the heart of the training of neural networks and finding this set of model parameters is an objective
function that is minimised iteratively through a stochastic optimisation algorithm, evaluated concerning
the actual data generating process (DGP). Algebraically, this is summarised by
Jt (Wt , θt ) = Eci,t ,ri,t+1 ∼pDGP [L(g(ci,t ; Wt , θt ); ri,t+1 )],

(24)

where L denotes a loss function, and the time index indicates that the objective function corresponds to
the T cross-sectional estimations (e.g. see Goodfellow et al. (2016)). However, the actual data generating
process is unknown. Hence, we replace the underlying DGP with the empirical distribution (e.g. see
Messmer (2017a)), such that equation (24) can be re-written as
Ĵt (Wt , θt ) = Eci,t ,ri,t+1 ∼pci,t ,ri,t+1 [L(g(ci,t ; Wt , θt ); ri,t+1 )],

(25)

with the objective function
N

L(c; Wt , θt ) =

t
1 X
(ri,t+1 − gt (ci,t ; Wt , θt ))2 + Ω(Wt ),
2Nt i=1

(26)

where we follow the standard assumption that the loss function L is the mean-squared-error. Equation
(25) extended equation (24) with a penalty term, denoted by Ω(Wt ), which analogously to Friedman
et al. (2010), is defined as


1
1
2
Ω(Wt ) = λ (1 − α) ∥Wt ∥1 + α ∥Wt ∥2 .
2

(27)

18 Not all model parameters are learnable and, therefore, found through training. The optimal values of hyperparameters,

such as the learning rate or penalty terms, are found manually through cross-validation.
19 The term optimal generally refers to model parameters that lead to a local minimum, as a global minimum is elusive.

30

The penalty term in equation (27) allows for L1 , L2 or L1 and L2 norm weight penalisation (note that
the penalty is only applied to the weights, not the biases). In general, any form of regularisation is aimed
at preventing in-sample overfitting in order to achieve better out-of-sample generalisations. There exists
a variety of ways to introduce regularisation, which we discuss further in section 4.3, and include early
stopping or batch normalisation in addition to the weight penalties from equation (27).
The optimal set of weights is found first-order gradient-based optimisation in combination, along with
backpropagation which is used to calculate the gradient of the loss function with respect to the learnable
model parameters (e.g. see Rumelhart et al. (1986)), such that

Wt∗ = min
Wt

Nt
X
i=1

2
ri,t+t − gt (ci,t ; Wt , θ) + Ω(Wt )

(28)

denotes the (learnable) model parameters which minimise the objective function. Any additional model
hyperparameters, such as the penalty parameter λ in equation (27), are tuned through, for example, crossvalidation, which we discuss in section 4.3. It follows that the estimated function becomes ĝ = g(· ; Wt∗ ).
The optimisation of the neural network’s objective function is almost exclusively non-convex (e.g. see
Messmer (2017a)). As a consequence, the objective function is optimised numerically, using gradientbased optimisers. One of the most common optimisers used in current literature is stochastic gradient
descent (SGD). However, there are some challenges regarding the use of SGD, including the difficulty
of finding an optimal learning rate (if the rate is too small, convergence is too slow, if the rate is too
large, it can hinder convergence), and the fact that SGD applies the same learning rate to all learnable
model parameters. To tackle those challenges, we focus on the Adaptive Moment Estimation or Adam,
proposed by Kingma and Ba (2015)), as a gradient descent optimisation algorithm to solve equation (28)
and find that Adam performs well empirically. The algorithm is summarised in algorithm 1.
Adam allows for adaptive learning rates for each learnable model parameter and learning rate decay, which is beneficial when a minimum is approached. The optimisation dynamics of Adam is often
described with the analogy of a heavy ball with friction rolling down the loss function, see Heusel et al.
(2017). Similarly to other optimisers such as Adadelta, Adam stores an exponentially decaying average
of past partial derivatives of the loss function with respect to the model parameters in addition to an
exponentially decaying average of past gradients, similarly to optimisers with momentum (e.g. Nesterov
accelerated gradient descent). For a more detailed overview on gradient-based optimisation, see Ruder
(2016).

31

Algorithm 1: Adam algorithm, analogously to Kingma and Ba (2015). gi2 indicates the elementwise square gi ⊙ gi . Empirically well-performing default values are β1 = 0.9, β2 = 0.999 and
ϵ = 10−8 . With β1i and β2i we denote β1 , respectively β2 to the power i, where i is the iteration
step.
Require: α: stepwise
Require: β1 , β1 ∈ [0, 1): Exponential decay rates for the moment estimates

Require: f (w): Stochastic objective function with learnable parameters w
Require: w0 : Initial parameter vector

m0 ← 0 (Initialise 1st moment vector) v0 ← 0 (Initialise 2nd moment vector)
i ← 0 (Initialise iteration step)
while wi not converged do
i←i+1

gi ← ∇w fi (wi−1 ) (Get gradients w.r.t. stochastic objective at iteration i)
mi ← β1 · mi−1 + (1 − β1 ) · gi (Update biased first moment estimate)

vi ← β1 · vi−1 + (1 − β1 ) · gi2 (Update biased second raw moment estimate)

mi
m̂i ← 1−β
i (Compute bias-corrected first moment estimate)
1

vi
vˆi ← 1−β
i (Compute bias-corrected second raw moment estimate)
2

i
wi ← wi−1 − α · √m̂
vˆ +ϵ
i

end
return wi (Resulting parameters)

3.3

Jacobian Objective Function

Section 3.2 introduced a standard objective function and optimisation strategy. One of the key advantages
of the objective function in equation (26) in combination with gradient descent-based optimisation is its
purely data-driven approach: a local minimum is reached without imposing any economic or financial
theory. The resulting statistical estimator is particularly advantageous in data-rich environments, where
the true data-generating process is unknown, and we wish to let the data speak for itself. There are,
however, asset-pricing specific conditions, which should be considered during training.
First, return data is characterised by a notoriously low signal-to-noise ratio. Therefore, a purely datadriven approach poses the danger of data-snooping and finding apparent statistical relationships even
though they are not there. In combination with our previous discussion about model interpretability,
this is particularly disadvantageous from a communication perspective. For example, a neural network
might find a nonlinear statistical link between specific firm characteristics and returns for which it is
difficult to find an economic justification. In turn, an investor might be sceptical about the model and
decide to use a simpler model with more interpretable and economically meaningful firm characteristic
interactions instead at the cost of inferior model performance.
Second, despite the abundance of potential regressors, there is considerable empirical evidence, such
as presented by Kelly et al. (2018), Kozak et al. (2020) or Freyberger et al. (2020), in favour of an

32

economically motivated sparse representation of the asset pricing model using a reduced number of
regressors. Such a parsimonious representation can either be achieved through prior data pre-selection,
where only a small number of regressors is hand-picked before training the neural network or through
model selection analogously to, for example, Lasso.
Third, as part of the regularisation strategy, we wish to limit an individual asset’s influence on the
predictions to make the model performance more robust. As discussed previously, compared to other
disciplines, the standard datasets used in empirical asset pricing comprise a comparatively small number
of observations. This relative data sparsity is combined with the severe unbalancedness of financial
datasets, where microcaps make up the vast majority of observations but are the least relevant stock
group for investors due to their illiquidity and high transaction costs. We are, therefore, in a dilemma:
do we keep all observations, including microcaps, in our training dataset, or do we throw out a significant
portion of the dataset, making training complex models more difficult. Limiting an asset’s influence on
the model performance can help mitigate this dilemma, as it allows us to keep all observations while also
ensuring that microcaps are not primarily driving model performance.
On this basis, we propose an objective function that helps mitigate those asset pricing specific challenges, particularly by allowing for model selection as part of the training algorithm. Our proposed
objective function can be used for an arbitrary choice of gt under the assumption that gradient descent
optimisation is used in order to determine its optimal parameters and the standard assumptions discussed
in section 2.
Let

N

t
1 X
Lt (c, Wt , θt ; Jt ) =
(ri,t+1 − gt (ci,t ; Wt , θt ))2 + Ω(Jt )
2Nt i=1

(29)

be the objective function, where Ω denotes a ”Jacobian”-based regulariser, defined as


1
1
2
Ω(Jt )) = λ (1 − α) ∥Jt ∥1 + α ∥Jt ∥2 ,
2

(30)

p

where ∥·∥p denotes the Frobenius norm of the ”Jacobian” Jt ∈ RNt ×K . We use quotation marks to
indicate that Jt is not strictly the Jacobian. Instead, each row in Jt corresponds to the partial derivatives
of gt with respect to the asset’s K firm characteristics, such that the dimension of Jt correspond to the
dimension of the input data. We acknowledge the nuances in the Jacobian definition, but for the purpose
of simplicity we omit the quotation marks in the following.
Moreover, we differentiate between two type of Jacobian regularisation: element-wise and columnmean. In the case of element-wise Jacobian regularisation, each element of the Jacobian enters the
PNt PK
1
2
∂gt (·)
2
2
penalty term, for example when p = 2, as ∥Jt ∥2 = ( i=1
k=1 dik,t ) , with dik,t = ∂cik,t . This means
that we penalise the sum of each element or partial derivative in Jt . In contrast, columns-mean Jacobian
regularisation, which we denote by J̄t , penalises the sum of the column-mean for each firm characteristic,
33

PNt
PK
1
2
dik,t )2 ) 2 . Although we do pursue
such that for example, when p = 2, such that J̄t 2 = ( k=1 ( N1t i=1
this path further in this paper, other variants of penalisations are imaginable, such as a value weighted
average penalisation, where the weights correspond to the inverse of an asset’s market capitalisation,
meaning that microcaps are penalised more heavily. Although we do not pursue this idea further in the
empirical application, it is conceivable that the objective function includes a penalty term for both – the
Jacobian and the model parameters – such that
N

t
1 X
Lt (c, Wt , θt ; Jt ) =
(ri,t+1 − gt (ci,t ; Wt , θt ))2 + Ω(Jt , Wt ),
2Nt i=1

(31)

with

Ω(Jt , Wt ) = λW

1
1
2
(1 − αW ) ∥Wt ∥1 + αW ∥Wt ∥2
2




+ λJ

1
1
2
(1 − αJ ) ∥Jt ∥1 + αJ ∥Jt ∥2
2


,

(32)

Gradient regularisation offers three key advantages. First, any form of regularisation is generally
helpful for preventing the model from overfitting. As in equation (28), weight regularisation is intended
to keep the outputs of each hidden layer away from the saturated regions of the activation function,
as, for example, discussed by Goodfellow et al. (2016). On the other hand, gradient regularisation
penalises large output changes due to small changes in input. Thus, Jacobian regularisation enforces a
smoothness prior (e.g. see Drucker and Cun (1992) or Hoffman et al. (2019)) and thereby increases model
robustness. In general, the idea of gradient regularisation as a method to increase model robustness is
not new to machine learning literature. In the field of image recognition, in particular, it is a common
strategy to deal with adversarial examples, where minimal changes in the input can lead to grossly wrong
classifications (e.g. see, Goodfellow et al. (2015), Lyu et al. (2016), Varga et al. (2018) or Barrett and
Dherin (2020)). With this paper, we propose Jacobian-based regularisation for asset pricing, where due
to the low signal-to-noise ratio present in financial data, it is also desirable to be protected against large
changes in output due to small changes in input.
Second, gradient regularisation is deeply rooted in economic theory and motivated by economic
intuition. Unlike image recognition, where it is difficult to interpret the partial derivatives, they have a
precise and well-defined economic meaning in our setup: risk prices. The penalty in equation (30) shrinks
specific risk prices in equation (20) towards zero, or even precisely to zero depending on the norm and size
of the penalty. There exists a large body of literature in asset pricing, such as Kelly et al. (2018), Kozak
et al. (2020) or Freyberger et al. (2020), who argue for an economically motivated sparse representation
of the asset pricing model using a reduced number of risk factors. Our proposed objective function
makes precisely this possible and is, therefore, a generalisation of model selection, where model selection
deals with the question of which firm characteristics have incremental predictive power, given all other
characteristics. Moreover, due to the time-varying nature of our proposed methodology, we allow this
34

variable selection to vary over time. Finally, our proposed objective function has a low computational
cost and is directly implementable as part of the training algorithm.

3.4

Linear Benchmarks

To make the neural networks’ empirical results more comparable, we benchmark them against a range of
commonly used alternative models. In addition to the ordinary least-squares (OLS) and neural network
estimations discussed in sections 2.1 and 3.1, we also investigate weighted least squares (WLS) and
three penalised linear regressions, including Ridge, Lasso and Elastic Net. While the primary numeric
evaluation metric constitutes the out-of-sample performance (see section 2.6 for performance definitions),
we also consider a model’s interpretability and general insights that we can gain from it to paint the
complete model comparison picture. This total evaluation package includes, but is not limited to, the
time-varying model performance or a model’s performance consistency, the firm characteristics the model
puts the most weight on, how those change over time, or the interpretability of a model’s output.
3.4.1

Weighted Least Squares

In the empirical asset pricing literature, it is well-documented that market capitalisation significantly
impacts model performance. For example, in the standard US-only dataset sourced from CRSP and
Compustat, microcaps make up nearly 60% of all stock while their total market capitalisation merely
represents around 3% of the total market capitalisation of all stocks on average. The two main problems
arising from this are that OLS overstates the importance of microcaps, while at the same time, microcaps
are much harder to trade on in practice due to their limited liquidity and higher trading costs. As a
consequence, the least-squares loss function is replaced with
N

L(g(·)) =

t
1 X
wi,t (ri,t+1 − gt (ci,t ))2 ,
2Nt i=1

(33)

where gt is linear in firm characteristics, analogously to standard OLS, with gt = C t λt and λ̂t =
′

′

(Ĉ t W Ĉ t )−1 Ĉ t Cr t+1 , ∀t = 1, ..., T , and W is a diagonal matrix containing the weights.
We use a stock’s market value of equity at time t as the weight wi,t . The weighted least squares
objective in equation (33) allows us to place more weight on statistically or economically informative
observations. In particular, due to the limited liquidity and increased trading costs of microcaps, WLS
tilts estimates towards higher liquidity and lower trading costs, as the objective function underweights
stocks with a smaller market capitalisation in favour of large stocks. Therefore, it helps mitigate the
challenges arising from the unbalancedness present in typical asset pricing datasets while being computationally very cheap at the cost of imposing a strong assumption about the linear functional form of
gt .
35

3.4.2

Penalised Linear Regressions

The main econometric disadvantage of the unregularised linear benchmarks (OLS and WLS) is that they
fail in high-dimensional settings when the number of predictors K reaches the number of cross-sectional
observations Nt , as the model becomes inefficient or even inconsistent. In a low signal-to-noise environment as present in return data, these concerns are particularly troublesome. In addition, and as pointed
out by Han et al. (2019), ordinary or weighted least squares may suffer from the problem of overfitting
in high-dimensional multivariate regressions. To avoid overfitting, parsimonious regression models with
a reduced number of parameters are crucial. As briefly discussed in section 3.2, and more specifically
in equation (26), the most common machine learning solution for imposing parameter parsimony is to
append the objective function with a penalty term, such that the new objective function is
N

Lt (Ct , λt ) =

t
1 X
(ri,t+1 − gt (ci,t ))2 + Ω(λt ),
2Nt i=1

(34)

where Ω(λt ) denotes a penalty, defined as


1
2
1
Ω(λt ) = λ (1 − α) ∥λt ∥1 + α ∥λt ∥2
2

(35)

and gt is a linear estimator function. The hyperparameters, such as the penalty term λ and α, are found
manually through cross-validation (e.g. see Friedman et al. (2010)). This form of regularisation mechanically diminishes a model’s in-sample performance to boost its out-of-sample performance stability. As
argued by Gu et al. (2020b), the model’s performance stability is particularly improved if regularisation
reduces the fit of noise while preserving the signal fit.
In equation (35), the case of α = 0 corresponds to the lasso or L1 penalisation, which is capable
of model selection due to the geometry of the objective function (e.g. see Hastie et al. (2009)). Lasso,
therefore, imposes sparsity. The case of α = 1 corresponds to the ridge regression, or L2 penalisation,
which shrinks parameters towards (but not exactly) to zero. This form of shrinkage limits the magnitude
of coefficients and, hence, prevents coefficients from becoming too large. All intermediate cases, where
0 < α < 1, correspond to the elastic net, a lasso and ridge regression mixture.
The penalised linear regression model and its objective function clarify the link between our proposed
methodology and the linear regression case. The main difference between standard objective functions,
such as equation (26), for neural networks and objective functions for the linear case, such as equation
(34), is that in the linear case, the coefficients are penalised, while in the neural network case we penalise
the network’s weights, which are not equivalent to coefficients. On the other hand, our proposed objective
function in section 3.3 uses the same analogy to the objective function in equation (34) as the penalisation
of the partial derivatives is the non-linear generalisation of the penalisation of the coefficients.
36

Last but not least, we acknowledge that there exists a myriad of potential additional benchmarks
such as principal components, partial least squares or instrumented principal components (e.g. see Kelly
et al. (2018)). However, with this paper, we do not intend to present a comprehensive model horserace.
Instead, we aim to shift the focus to our proposed methodology of increased interpretability using neural
networks and keep the model benchmarks with OLS, WLS, Lasso, Ridge and Elastic Net to an extensive
minimum.

4

An Empirical Study of U.S. Equities

This section provides a detailed overview of the investment universe we consider, consisting of monthly
stock returns and 103 firm characteristics, which we construct analogously to Green et al. (2017). We
show that deep neural networks are a reasonable model choice from a performance perspective and have
substantial interpretability advantages. We explicitly do not wish to shift existing out-of-sample performance frontiers and are not plain supporters of deep neural networks. However, this section shows that
conditional on the model choice, deep neural networks add significantly from an interpretability perspective leveraging our proposed methodology, compared to standard linear benchmarks. In particular, we
show which risk factors help explain cross-sectional returns over time and how our results compare to
existing research such as Green et al. (2017).

4.1

Data

We follow standard procedure, source monthly stock returns from CRSP for firms listed on NYSE,
AMEX, and NASDAQ, and obtain firm-level data from Compustat and I/B/E/S. We use the Effective
Federal Funds rate as a proxy for the risk-free rate of return, which we source from the Federal Reserve
Economic Data repository (FRED) and obtain price level data directly from the US Bureau of Labor
Statistics (BLS). Stock returns are adjusted for delistings and are matched in month t with firm characteristics that are most recently available to an investor at the beginning of month t. We assume that
firm characteristics based on annual accounting information become available at least six months after
the fiscal year ending and firm characteristics based on quarterly accounting data become available at
least four months after the fiscal quarter ending to avoid information leakage.
The sample period spans over 492 months from January 1980 to December 2020, yielding over four
decades of data. The sample period choice in empirical asset pricing is somewhat arbitrary. This paper
primarily focuses on fitting non-linear machine learning models whose training is data-intensive and,
therefore, requires a minimum amount of data. However, data availability varies considerably over time.
For example, the NASDAQ only began its operations in the early 1970s, adding a significant number of
stocks to the sample at that time. Moreover, I/B/E/S data only became available in the late 1980s and
37

early 1990s. Thus, our sample choice yields a balanced trade-off between data availability, comparability
to other existing research and timespan over economically relevant and heterogeneous periods. Figure 4
visualises the time-series properties of an unrestricted US-only investment universe.
We consider stocks with share code20 10 or 11 only and impose no restrictions on the company size,
industry or time since a company has been listed on an exchange to minimise data-snooping biases
(e.g. see Lo and MacKinlay (1990)). We adopt standard company size definitions based on market
capitalisation analogously to Fama and French (1993) and define microcaps as stocks whose market
capitalisation is smaller than the 20th percentile of the market equity of NYSE stocks. In contrast, the
market capitalisation of small stocks is greater than the 20th percentile but smaller than the median (50th
percentile) market capitalisation for NYSE stocks. Subsequently, large stocks are all remaining stocks
whose market capitalisation is larger than the median market equity of NYSE listed stocks. Figure
4 and table 1 reveal that microcaps merely represent around 3% of aggregated market capitalisation
while making up around 60% of the total number of stocks. This well-known misbalance poses specific
difficulties. Conversely, the general machine learning mantra that more data is better than less data
also holds in asset pricing. From this point of view, microcaps provide the vast majority of training
data. On the other hand, empirical asset pricing frequently faces criticism from practitioners as many
findings heavily rely on microcaps, but microcaps are generally less liquid and more expensive to trade.
We acknowledge the existence of such problems and explicitly discuss those in our empirical analysis.
We build a large collection of stock-level firm characteristics based on 

[The evaluation harness truncated this reference: showing the first 120000 of 420994 characters.]
</reference>

<statements>
1. Supervised ML models (trees, random forests, neural nets, AutoML) learn non‑linear mappings from predictors to returns or risk premia; deep RL models directly learn allocation policies that optimize risk‑adjusted reward under dynamic, multi‑period settings.
2. Non-linear, high-dimensional prediction: Deep neural networks and other ML models estimate time-varying equity risk premia or asset returns by learning non-linear functions of large predictor sets, generalizing classic linear regressions like Fama-MacBeth.
3. Performance: Empirical studies show that such models can explain complex cross-sectional return structures and outperform classical factor models in predictive accuracy.
4. Interpretability challenge: These models are typically black boxes; their lack of economic interpretability raises concerns in high-stakes finance and regulatory contexts.
5. Supervised deep / ML: Return prediction / prior: Non‑linear, high‑dimensional prediction of risk premia or returns; can outperform linear factor models.
6. Supervised deep / ML: Main strengths: Captures complex structures and interactions; improved predictive accuracy and flexibility.
7. Supervised deep / ML: Main limitations: Black‑box behavior; fragile under regime shifts; interpretability and economic grounding often weak.
8. This layer is agnostic about the higher-level model but is designed to support both interpretable factors (value, momentum, quality) and high-dimensional signals used by deep models.
9. This yields a traceable decision record that can be explained to stakeholders, addressing the regulatory and practical interpretability concerns around black-box models.
10. Deep and AutoML models improve predictive accuracy for asset pricing relative to classic linear models.
</statements>

Begin the assessment now. Output only the JSON list, without any conversational text or explanations.