You will be provided with a reference and some statements. Please determine whether each statement is 'supported', 'unsupported', or 'unknown' with respect to the reference. Please note:
First, assess whether the reference contains any valid content. If the reference contains no valid information, such as a 'page not found' message, then all statements should be considered 'unknown'.
If the reference is valid, for a given statement: if the facts or data it contains can be found entirely or partially within the reference, it is considered 'supported' (data accepts rounding); if all facts and data in the statement cannot be found in the reference, it is considered 'unsupported'.

You should return the result in a JSON list format, where each item in the list contains the statement's index and the judgment result, for example:
[
    {
        "idx": 1,
        "result": "supported"
    },
    {
        "idx": 2,
        "result": "unsupported"
    }
]

Below are the reference and statements:
<reference>
End-to-End Conformal Calibration for Optimization Under Uncertainty



Report GitHub Issue

×

Title:

Content selection saved. Describe the issue below:

Description:

Submit without GitHub

Submit in GitHub

arXiv is now an independent nonprofit!

Learn more

×

Back to arXiv

Why HTML?

Report Issue

Back to Abstract

Download PDF

Abstract

1
Introduction

2
Problem Statement and Background

Comparison to related work.

3
End-to-End Training of Conformally Calibrated Uncertainty Sets

3.1
Representations of the uncertainty set

Box uncertainty sets.

Ellipsoidal uncertainty sets.

3.2
Conformal uncertainty set calibration

3.3
End-to-end training and calibration

4
Representing General Convex Uncertainty Sets via PICNNs

5
Experiments

5.1
Problem descriptions

Price forecasting for battery storage.

Portfolio optimization.

5.2
Baseline methods

5.3
Battery storage problem results

5.4
Performance under distribution shift

6
Conclusion

References

A
Appendix: Additional experimental results

A.1
Experimental results: Battery storage

Performance.

A.2
Experimental results: Portfolio optimization

B
Appendix: Related Work

“Task-based” or “decision-focused” learning.

Uncertainty Quantification.

Robust and stochastic optimization.

C
Appendix: Maximizing over the uncertainty set

C.1
Maximizing over a box constraint

C.2
Maximizing over an ellipsoid

C.3
Proof of Theorem
: Maximizing over the sublevel set of a PICNN

C.4
Ensuring feasibility of the PICNN maximization problem

Ensuring compact sublevel sets.

Ensuring
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is nonempty.

D
Appendix: Exact differentiable conformal prediction

E
Appendix: Experiment details

E.1
Uncertainty representation

Box uncertainty.

Ellipsoidal uncertainty.

PICNN uncertainty.

E.2
Data

Price forecasting for battery storage.

Portfolio optimization.

License: CC BY 4.0

arXiv:2409.20534v2 [cs.LG] 01 Feb 2026

End-to-End Conformal Calibration for Optimization Under Uncertainty

Christopher Yeh
1
1

1

These authors contributed equally.

cyeh@caltech.edu

Affiliation:
Nicolas Christianson
1
1

1

These authors contributed equally.

nchristi@caltech.edu

Affiliation:
Alan Wu
ywu9@caltech.edu

Affiliation:
Adam Wierman
adamw@caltech.edu

Affiliation:
Yisong Yue
yyue@caltech.edu

Affiliation:
Department of Computing and Mathematical Sciences

Affiliation:
California Institute of Technology

Abstract

Machine learning can significantly improve performance for decision-making under uncertainty across a wide range of domains. However, ensuring robustness guarantees requires well-calibrated uncertainty estimates, which can be difficult to achieve with neural networks. Moreover, in high-dimensional settings, there may be many valid uncertainty estimates, each with its own performance profile—i.e., not all uncertainty is equally valuable for downstream decision-making. To address this problem, this paper develops an end-to-end framework to
learn
uncertainty sets for conditional robust optimization in a way that is informed by the downstream decision-making loss, with robustness and calibration guarantees provided by conformal prediction. In addition, we propose to represent general convex uncertainty sets with partially input-convex neural networks, which are learned as part of our framework. Our approach consistently improves upon two-stage estimate-then-optimize baselines on concrete applications in energy storage arbitrage and portfolio optimization.

Reviewed on OpenReview:
https://openreview.net/forum?id=yM8qkT0f9H

1
Introduction

Well-calibrated estimates of forecast uncertainty are vital for risk-aware decision-making in many real-world systems. For instance, grid-scale battery operators forecast electricity prices to schedule battery charging/discharging to maximize profit, while accounting for forecast uncertainty to mitigate financial or operational risk. Similarly, financial investors use forecasts of asset returns with uncertainty estimates to maximize portfolio returns while minimizing downside risk.

Traditional approaches to decision-making under uncertainty often follow an “estimate-then-optimize” (ETO) paradigm
(
Chenreddy et al., 2022
)
, in which the uncertainty estimation and decision-making stages are decoupled. In the “estimation” stage, a predictive model is trained to forecast a target quantity along with an uncertainty set. In the “optimization” stage, this uncertainty estimate informs a downstream decision. Crucially, the cost or performance of the downstream decision is typically not fed back into the training of the predictive model.

A recent line of work
(
Chenreddy et al., 2022
;
Patel et al., 2024
;
Wang et al., 2024
;
Chenreddy and Delage, 2024
)
has made steps toward bridging the gap between uncertainty quantification and robust optimization-driven decision-making, where optimization problems take a forecast uncertainty set as a parameter, as is common in energy systems
(
Yan et al., 2020
;
Parvar and Nazaripouya, 2022
)
and financial applications
(
Gregory et al., 2011
;
Quaranta and Zaffaroni, 2008
)
.
However, existing approaches are suboptimal for several reasons:

1.

Lack of decision-aware training:
The predictive model is typically not trained with feedback from the downstream objective. Because the downstream objective is often asymmetric with respect to the forecasting model’s errors, prediction models trained with decision-agnostic losses may achieve high predictive accuracy but still perform poorly on the decision-making objective.

2.

Restricted uncertainty set parameterizations:
To preserve tractability of the robust optimization problem, uncertainty sets are often constrained to simple parametric forms (e.g., boxes or ellipsoids), limiting the expressivity of uncertainty estimates.

3.

Calibration challenges:
Because neural network models are often poor at estimating their own uncertainty, the forecasts may not be well-calibrated. Recent approaches such as isotonic regression
(
Kuleshov et al., 2018
)
and conformal prediction
(
Shafer and Vovk, 2008
)
have made progress in providing
calibrated
uncertainty estimates from deep learning models, but such methods are typically applied post-hoc to trained models and are therefore difficult to incorporate into an end-to-end training procedure.

Figure 1
:
Whereas prior “estimate-then-optimize” (ETO, top) approaches separate the model training from the optimization (decision-making) procedure, we propose a framework for end-to-end (E2E, bottom) conformal calibration for optimization under uncertainty that directly trains the machine learning model using gradients from the task loss.

As such, there is as of yet
no
comprehensive methodology for training calibrated uncertainty-aware deep learning models end-to-end with downstream decision-making objectives. In this work, we provide the first such methodology. We make three specific contributions corresponding to the three issues identified above:

1.

We develop a framework for training prediction models end-to-end with downstream decision-making objectives and conformal-calibrated uncertainty sets in the context of the
conditional robust optimization
problem.
This framework is illustrated in
Figure

1
(bottom). By including differentiable conformal calibration in our model during training, we close the loop and ensure that feedback from the uncertainty’s impact on the downstream objective is accounted for in the training process, since not all model errors nor uncertainty estimates will result in the same downstream cost. This end-to-end training enables the model to focus its learning capacity on minimizing error and uncertainty on outputs with the largest decision-making cost, with more leeway for outputs that have lower costs.

2.

We propose using partially input-convex neural networks (PICNNs) as the nonconformity score function for conformal prediction, enabling the approximate parametrization of arbitrary compact, convex uncertainty sets in the conditional robust optimization problem.
To the best of our knowledge, no existing works use PICNNs to parametrize such arbitrary convex uncertainty sets. Due to the universal convex function approximation property these networks enjoy
(
Chen et al., 2019
)
, this approach enables training much more general representations of uncertainty than prior works have considered, which in turn yields substantial improvements on downstream decision-making performance. Importantly, PICNNs are well-matched to our conditional robust optimization problem: we show that the robust problem resulting from this parametrization can be reformulated as a tractable convex optimization problem.

3.

We propose an exact and computationally efficient method to differentiate through the conformal prediction procedure during training.
Unlike prior work
(
Stutz et al., 2022
)
, our method gives exact gradients, without using approximate ranking and sorting methods.

Finally, we evaluate the performance of our approach on two applications: an energy storage arbitrage task and a portfolio optimization problem. We demonstrate that the combination of end-to-end training with the flexibility of the PICNN-based uncertainty sets consistently improves over ETO baseline methods. The performance benefit of our end-to-end method is apparent even under distribution shift.

2
Problem Statement and Background

Our problem is defined formally as follows: suppose that data
(
x
,
y
)
∈
ℝ
m
×
ℝ
n
(x,y)\in\mathbb{R}^{m}\times\mathbb{R}^{n}
is sampled i.i.d. from an unknown joint distribution
𝒫
\mathcal{P}
. Upon observing the input
x
x
(but not the label
y
y
), an agent makes a decision
z
∈
ℝ
p
z\in\mathbb{R}^{p}
. After the decision is made, the true label
y
y
is revealed, and the agent incurs a
task loss

f
⁡
(
x
,
y
,
z
)
f(x,y,z)
, for some known task loss function
f
:
ℝ
m
×
ℝ
n
×
ℝ
p
→
ℝ
f:\mathbb{R}^{m}\times\mathbb{R}^{n}\times\mathbb{R}^{p}\to\mathbb{R}
. In addition, the agent’s decision must satisfy a set of joint constraints
g
⁡
(
x
,
z
)
≤
0
g(x,z)\leq 0
coupling
x
x
and
z
z
.

As an illustrative example, consider an agent who would like to minimize the costs of charging and discharging a battery over 24 hours in a day. The agent may use weather forecasts and historical observations
x
x
to predict future energy prices
y
y
. Based on the predicted prices, the agent decides on the amount
z
z
to charge or discharge the battery. The task loss
f
f
is the cost incurred by the agent, and the constraints
g
g
include limits on how fast the battery can charge as well as the maximum capacity of the battery. This example is explored in more detail in
Section

5
.

Because the agent does not observe the label
y
y
prior to making its decision, ensuring good performance and constraint satisfaction requires that the agent makes decisions
z
z
that are
robust
to the various outcomes of
y
y
. A common objective is to choose
z
z
to robustly minimize the task loss and satisfy the constraints over all realizations of
y
y
within a
(
1
−
α
)
(1-\alpha)
-confidence region
Ω
⁡
(
x
)
⊂
ℝ
n
\Omega(x)\subset\mathbb{R}^{n}
of the true conditional distribution
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
, where
α
∈
(
0
,
1
)
\alpha\in(0,1)
is a fixed risk level based on operational requirements. In this case, the agent’s robust decision can be expressed as the optimal solution to the following
conditional robust optimization (CRO)
problem
(
Chenreddy et al., 2022
)
:

z
⋆
​
(
x
)
:=
arg
​
min
z
∈
ℝ
p
⁡
max
y
^
∈
Ω
⁡
(
x
)
​
f
​
(
x
,
y
^
,
z
)
​
s.t.
​
g
​
(
x
,
z
)
≤
0
.
\hskip-6.00006ptz^{\star}(x):=\argmin_{z\in\mathbb{R}^{p}}\max_{\hat{y}\in\Omega(x)}f(x,\hat{y},z)\ \text{s.t. }g(x,z)\leq 0.

(1)

After the agent decides
z
⋆
​
(
x
)
z^{\star}(x)
, the true label
y
y
is revealed, and the agent incurs the task loss
f
​
(
x
,
y
,
z
⋆
​
(
x
)
)
f(x,y,z^{\star}(x))
. Thus, the agent seeks to minimize expected task loss

𝔼
(
x
,
y
)
∼
𝒫
⁡
[
f
⁡
(
x
,
y
,
z
⋆
​
(
x
)
)
]
.
\E_{(x,y)\sim\mathcal{P}}\left[f(x,y,z^{\star}(x))\right].

(2)

While the joint distribution
𝒫
\mathcal{P}
is unknown, we assume that we have a dataset
D
=
{
(
x
i
,
y
i
)
}
i
=
1
N
D=\{(x_{i},y_{i})\}_{i=1}^{N}
of i.i.d. samples from
𝒫
\mathcal{P}
. Then, our objective is to train a machine learning model to learn an approximate
(
1
−
α
)
(1-\alpha)
-confidence set
Ω
⁡
(
x
)
\Omega(x)
of possible
y
y
values for each input
x
x
. Formally, our learned
Ω
⁡
(
x
)
\Omega(x)
should satisfy the following
marginal coverage
guarantee.

Definition 2.1

(marginal coverage)
.

An uncertainty set
Ω
⁡
(
x
)
\Omega(x)
for the distribution
𝒫
\mathcal{P}
provides
marginal coverage at level
(
1
−
α
)
(1-\alpha)
if

ℙ
(
x
,
y
)
∼
𝒫
​
(
y
∈
Ω
⁡
(
x
)
)
≥
1
−
α
.
\mathbb{P}_{(x,y)\sim\mathcal{P}}\left(y\in\Omega(x)\right)\geq 1-\alpha.

Comparison to related work.

The problem of constructing data-driven and machine-learned uncertainty sets with probabilistic coverage guarantees for use in robust optimization has been widely explored in prior literature (e.g.,
Bertsimas and Thiele (2006)
;
Bertsimas et al. (2018)
;
Alexeenko and Bitar (2020)
;
Goerigk and Kurtz (2023)
).
Chenreddy et al. (2022)
first coined the phrase “conditional robust optimization” for the problem
(
1
)
and considered learning context-dependent uncertainty sets
Ω
⁡
(
x
)
\Omega(x)
in this setting. However, their approach results in a mixed integer optimization that is intractable to solve for large-scale problems. Moreover, they follow the “estimate then optimize” (ETO) paradigm
(
Elmachtoub et al., 2023
)
. As shown in
Figure

1
(top), the ETO paradigm separates the machine learning model training from the decision optimization. The lack of feedback from the downstream task loss during model training in ETO generally leads to uncertainty sets
Ω
⁡
(
x
)
\Omega(x)
which yield suboptimal results. Several other recent papers follow the ETO paradigm using homoskedastic ellipsoidal uncertainty sets
(
Johnstone and Cox, 2021
)
, heteroskedastic box and ellipsoidal uncertainty sets
(
Sun et al., 2023
)
, and a “union of balls” parametrization of uncertainty
(
Patel et al., 2024
)
. In our experiments (
Section

5
), we demonstrate consistent improvements over the methods of
Johnstone and Cox (2021)
and
Sun et al. (2023)
.

Closest to our work is an “end-to-end” formulation of the CRO problem posed by
Chenreddy and Delage (2024)
, which aims to learn conditional uncertainty sets
Ω
⁡
(
x
)
\Omega(x)
using a weighted combination of the downstream task loss along with a “conditional coverage loss” to promote calibrated uncertainty. However, they focus solely on ellipsoidal uncertainty sets, and their conditional coverage loss does not provably ensure coverage for their learned uncertainty sets. In our experiments, we do not compare against
Chenreddy and Delage (2024)
because we only consider other methods with a provable coverage guarantee.
1
1

1

As of the time of writing, the approach of
Chenreddy and Delage (2024)
also suffers from substantial inconsistencies between
their code implementation
and the equations from their paper. In particular, the conditional coverage loss proposed in their paper is not implementable, as it will (almost surely) yield zero gradients.

A concurrent related work by
Wang et al. (2024)
approaches the problem of learning
unconditional
uncertainty sets for robust optimization, while achieving robust constraint satisfaction guarantees. While their method also uses an end-to-end task loss, we find their use of the same uncertainty set
Ω
\Omega
for every problem instance (i.e.,
Ω
\Omega
is independent of
x
x
) to be highly restrictive. For example, in the context of our battery control problem, this restriction would disallow the use of weather forecasts and historical price data to estimate uncertainty in future energy prices.
2
2

2

After the present paper’s submission, the authors of
Wang et al. (2024)
updated their preprint and expanded their formulation to accommodate conditional uncertainty sets.
They also use restrictive uncertainty set parametrizations such as box, ellipsoidal, and polyhedral uncertainty. Moreover, their guarantee on constraint satisfaction requires an assumption on the convergence of their algorithm to an approximate stationary point; however, this convergence is only guaranteed asymptotically, and not in finitely many samples.

An alternative line of research designs post-hoc conformal prediction sets aimed at minimizing decision costs in robust classification problems
(
Kiyani et al., 2025
;
Cortes-Gomez et al., 2024
)
. However, to date, it is unclear how to adapt these methods, which produce discrete uncertainty sets, for regression settings.

In contrast, our work overcomes these limitations: we incorporate differentiable conformal calibration
during training
to ensure that uncertainty is learned end-to-end in a manner that is both calibrated and minimizes task loss. We apply split conformal post-hoc calibration during inference for provable guarantees on coverage. Furthermore, we use partially input-convex neural networks
(
Amos et al., 2017
)
to directly parameterize the nonconformity score function in conformal prediction, enabling a general and expressive representation of arbitrary conditional convex uncertainty regions that can vary with
x
x
and be used efficiently in robust optimization.

Beyond the above closely related work, this paper builds upon and contributes to several different areas in machine learning and robust optimization; see
Appendix

B
for a comprehensive discussion.

3
End-to-End Training of Conformally Calibrated Uncertainty Sets

In this section, we describe our proposed methodological framework for end-to-end task-aware training of predictive models with conformally calibrated uncertainty for the conditional robust optimization problem (
1
). Our overarching goal is to learn uncertainty sets
Ω
⁡
(
x
)
\Omega(x)
which provide
(
1
−
α
)
(1-\alpha)
coverage for any choice of
α
∈
(
0
,
1
)
\alpha\in(0,1)
, and which offer the lowest possible task loss (
2
). To this end, we must consider three primary questions:

1.

How should the family of uncertainty sets
Ω
⁡
(
x
)
\Omega(x)
be parametrized?

2.

How can we guarantee that the uncertainty set
Ω
⁡
(
x
)
\Omega(x)
provides coverage at level
1
−
α
1-\alpha
?

3.

How can the uncertainty set
Ω
⁡
(
x
)
\Omega(x)
be learned to minimize expected task loss?

Figure

1
(bottom) illustrates the key parts of our framework to answer these questions. First, we use a machine learning model to parametrize a nonconformity score function
s
θ
s_{\theta}
, and we define the uncertainty set
Ω
⁡
(
x
)
\Omega(x)
to be a
q
q
-sublevel set of
s
θ
​
(
x
,
⋅
)
s_{\theta}(x,\cdot)
. Second, we use conformal calibration to pick
q
q
to enforce marginal coverage. Third, we backpropagate gradients through both the robust optimization and conformal calibration steps to update the machine learning model, thereby enabling end-to-end learning.
Sections

3.1
,
3.2
and
3.3
describe each of these parts in detail, and
Algorithm

1
shows pseudocode for both training and inference.

For the rest of the paper, we make the following assumptions on the functions
f
f
and
g
g
to ensure tractability of the resulting optimization problem.

Assumption 3.1
.

We assume the task loss has the form
f
⁡
(
x
,
y
,
z
)
=
y
⊤
​
F
​
(
x
,
z
)
+
f
~
​
(
x
,
z
)
f(x,y,z)=y^{\top}F(x,z)+\tilde{f}(x,z)
, where
F
⁡
(
x
,
z
)
F(x,z)
is an affine function of
z
z
and
f
~
​
(
x
,
z
)
\tilde{f}(x,z)
is convex in
z
z
. Furthermore, we assume that
g
⁡
(
x
,
z
)
g(x,z)
is convex in
z
z
.

Technically, the assumption on
f
f
can be relaxed to the more general setting where
f
f
is a
conic-representable saddle function
that is convex in
z
z
and concave in
y
y

(
Juditsky and Nemirovski, 2022
)
, which admits an automated dual representation
(
Schiele et al., 2023
)
. However, we believe the core ideas for our exposition and proofs are much more straightforward with the stronger assumption in
Assumption

3.1
.

3.1
Representations of the uncertainty set

We consider convex uncertainty sets of the form

Ω
θ
​
(
x
)
=
{
y
^
∈
ℝ
n
∣
s
θ
​
(
x
,
y
^
)
≤
q
}
,
\Omega_{\theta}(x)=\left\{\hat{y}\in\mathbb{R}^{n}\mid s_{\theta}(x,\hat{y})\leq q\right\},

(3)

where
s
θ
:
ℝ
m
×
ℝ
n
→
ℝ
s_{\theta}:\mathbb{R}^{m}\times\mathbb{R}^{n}\to\mathbb{R}
is an arbitrary
nonconformity score function
that is convex in
y
^
\hat{y}
,
q
q
is a scalar, and
θ
\theta
collects the parameters of a model that we will seek to learn. Note that this representation loses no generality;
any
family of convex sets
Ω
⁡
(
x
)
\Omega(x)
can be represented as such a collection of sublevel sets of a partially input-convex function
s
⁡
(
x
,
y
^
)
s(x,\hat{y})
. This particular representation is chosen due to the ease of calibrating sets of this form via conformal prediction to ensure marginal coverage, as we will describe in
Section

3.2
.

In choosing a particular score function
s
θ
s_{\theta}
, one must balance two considerations: first, the
generality
of the sets
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
that
s
θ
s_{\theta}
can represent, and second, the
tractability
of the resulting robust optimization problem (
1
). We will now show that our representation (
3
) generalizes commonly-used box and ellipsoidal uncertainty sets, which are known to have tractable robust problems; later, in
Section

4
, we will propose to approximate more general convex uncertainty sets using partially input-convex neural networks.

Box uncertainty sets.

A simple uncertainty representation is box uncertainty where
Ω
⁡
(
x
)
=
[
y
¯
​
(
x
)
,
y
¯
​
(
x
)
]
\Omega(x)=[\underline{y}(x),\overline{y}(x)]
is an
n
n
-dimensional box whose lower and upper bounds depend on
x
x
. Let
h
θ
:
ℝ
m
→
ℝ
n
×
ℝ
n
h_{\theta}:\mathbb{R}^{m}\to\mathbb{R}^{n}\times\mathbb{R}^{n}
be a neural network that estimates lower and upper bounds:
h
θ
​
(
x
)
=
(
h
θ
lo
​
(
x
)
,
h
θ
hi
​
(
x
)
)
h_{\theta}(x)=(h_{\theta}^{\text{lo}}(x),h_{\theta}^{\text{hi}}(x))
. To represent a box uncertainty set in the form (
3
), we define a nonconformity score function that generalizes scalar conformalized quantile regression
(
Romano et al., 2019
)
:

s
θ
​
(
x
,
y
)
=
max
⁡
(
‖
h
θ
lo
​
(
x
)
−
y
‖
∞
,
‖
y
−
h
θ
hi
​
(
x
)
‖
∞
)
.
s_{\theta}(x,y)=\max(\left\lVert h_{\theta}^{\text{lo}}(x)-y\right\rVert_{\infty},\left\lVert y-h_{\theta}^{\text{hi}}(x)\right\rVert_{\infty}).

Then, the uncertainty set (
3
) becomes

Ω
θ
​
(
x
)
=
[
h
θ
lo
​
(
x
)
−
q
​
𝟏
,
h
θ
hi
​
(
x
)
+
q
​
𝟏
]
=
:
[
y
¯
​
(
x
)
,
y
¯
​
(
x
)
]
.
\Omega_{\theta}(x)=\left[h_{\theta}^{\text{lo}}(x)-q\mathbf{1},\ h_{\theta}^{\text{hi}}(x)+q\mathbf{1}\right]=:\Big[\underline{y}(x),\ \overline{y}(x)\Big].

Given a box uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
, we can take the dual of the inner maximization problem (see
Section

C.1
) to transform the robust optimization problem (
1
) into an equivalent form that is convex, and hence tractable, under
Assumption

3.1
:

z
θ
⋆
​
(
x
)
=
arg
​
min
z
∈
ℝ
p
⁡
min
ν
∈
ℝ
n
\displaystyle z_{\theta}^{\star}(x)=\argmin_{z\in\mathbb{R}^{p}}\min_{\nu\in\mathbb{R}^{n}}

(
y
¯
​
(
x
)
−
y
¯
​
(
x
)
)
⊤
​
ν
+
y
¯
​
(
x
)
⊤
​
F
​
(
x
,
z
)
+
f
~
​
(
x
,
z
)
\displaystyle(\overline{y}(x)-\underline{y}(x))^{\top}\nu+\underline{y}(x)^{\top}F(x,z)+\tilde{f}(x,z)

(4)

s.t.
\displaystyle\text{s.t.}

ν
≥
𝟎
,
ν
−
F
(
x
,
z
)
≥
𝟎
,
g
(
x
,
z
)
≤
0
.
\displaystyle\nu\geq\mathbf{0},\ \nu-F(x,z)\geq\mathbf{0},\ g(x,z)\leq 0.

Ellipsoidal uncertainty sets.

Another common form of uncertainty set is ellipsoidal uncertainty. Suppose a neural network model
h
θ
:
ℝ
m
→
ℝ
n
×
𝕊
+
n
h_{\theta}:\mathbb{R}^{m}\to\mathbb{R}^{n}\times\mathbb{S}_{+}^{n}
predicts mean and covariance parameters
h
θ
​
(
x
)
=
(
μ
θ
​
(
x
)
,
Σ
θ
​
(
x
)
)
h_{\theta}(x)=(\mu_{\theta}(x),\Sigma_{\theta}(x))
, so that
𝒫
^
​
(
y
∣
x
;
θ
)
=
𝒩
⁡
(
y
∣
μ
θ
​
(
x
)
,
Σ
θ
​
(
x
)
)
\hat{\mathcal{P}}(y\mid x;\theta)=\Gaussian(y\mid\mu_{\theta}(x),\Sigma_{\theta}(x))
denotes a predicted conditional density, where
𝒩
(
⋅
∣
μ
,
Σ
)
\Gaussian(\cdot\mid\mu,\Sigma)
is the multivariate normal density function. In this case, we define the nonconformity score function based on the squared Mahalanobis distance
(
Johnstone and Cox, 2021
;
Sun et al., 2023
)

s
θ
​
(
x
,
y
)
=
(
y
−
μ
θ
​
(
x
)
)
⊤
​
(
Σ
θ
​
(
x
)
)
−
1
​
(
y
−
μ
θ
​
(
x
)
)
,
s_{\theta}(x,y)=(y-\mu_{\theta}(x))^{\top}(\Sigma_{\theta}(x))^{-1}(y-\mu_{\theta}(x)),

which yields uncertainty sets (
3
) that are ellipsoidal:

Ω
θ
​
(
x
)
=
{
y
^
∣
(
y
^
−
μ
θ
​
(
x
)
)
⊤
​
(
Σ
θ
​
(
x
)
)
−
1
​
(
y
^
−
μ
θ
​
(
x
)
)
≤
q
}
.
\Omega_{\theta}(x)=\{\hat{y}\mid(\hat{y}-\mu_{\theta}(x))^{\top}(\Sigma_{\theta}(x))^{-1}(\hat{y}-\mu_{\theta}(x))\leq q\}.

Let
L
θ
​
(
x
)
L_{\theta}(x)
denote the unique lower-triangular Cholesky factor of
Σ
θ
​
(
x
)
\Sigma_{\theta}(x)
(i.e.,
Σ
θ
​
(
x
)
=
L
θ
​
(
x
)
​
L
θ
​
(
x
)
⊤
\Sigma_{\theta}(x)=L_{\theta}(x)L_{\theta}(x)^{\top}
). Taking the dual of the inner maximization problem and invoking strong duality (see
Section

C.2
), we transform the robust optimization problem (
1
) into an equivalent form that is convex, and hence tractable, under
Assumption

3.1
:

z
θ
⋆
​
(
x
)
=
arg
​
min
z
∈
ℝ
p
\displaystyle z_{\theta}^{\star}(x)=\argmin_{z\in\mathbb{R}^{p}}

q
​
‖
L
θ
​
(
x
)
⊤
​
F
​
(
x
,
z
)
‖
2
+
μ
θ
​
(
x
)
⊤
​
F
​
(
x
,
z
)
+
f
~
​
(
x
,
z
)
\displaystyle\sqrt{q}\|L_{\theta}(x)^{\top}F(x,z)\|_{2}+\mu_{\theta}(x)^{\top}F(x,z)+\tilde{f}(x,z)

(5)

s.t.
\displaystyle\text{s.t.}

g
⁡
(
x
,
z
)
≤
0
,
\displaystyle g(x,z)\leq 0,

3.2
Conformal uncertainty set calibration

As long as the uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
can be expressed in the form (
3
), we can use the split conformal prediction procedure at inference time to choose a value
q
q
that ensures
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
provides marginal coverage (
Definition

2.1
) at any confidence level
1
−
α
1-\alpha
. The split conformal procedure assumes access to a calibration dataset
D
cal
=
{
(
x
i
,
y
i
)
}
i
=
1
M
D_{\text{cal}}=\{(x_{i},y_{i})\}_{i=1}^{M}
drawn exchangeably from
𝒫
\mathcal{P}
. We refer readers to
Angelopoulos et al. (2023)
for details on this procedure.

Lemma 3.2

(from
Angelopoulos et al. (2023)
, Appendix D)
.

Let
D
cal
=
{
(
x
i
,
y
i
)
}
i
=
1
M
D_{\text{cal}}=\{(x_{i},y_{i})\}_{i=1}^{M}
be a calibration dataset drawn exchangeably (e.g., i.i.d.) from
𝒫
\mathcal{P}
, and let
s
i
=
s
θ
​
(
x
i
,
y
i
)
s_{i}=s_{\theta}(x_{i},y_{i})
. If
q
=
Quantile
​
(
{
s
i
}
i
=
1
M
,
1
−
α
)
q=\textsc{Quantile}(\{s_{i}\}_{i=1}^{M},\ 1-\alpha)
(see
Algorithm

1
) is the empirical
⌈
(
M
+
1
)
​
(
1
−
α
)
⌉
M
\frac{\left\lceil(M+1)(1-\alpha)\right\rceil}{M}
-quantile of the set
{
s
i
}
i
=
1
M
\{s_{i}\}_{i=1}^{M}
and
(
x
,
y
)
(x,y)
is drawn exchangeably with
D
cal
D_{\text{cal}}
, then
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
has the marginal coverage guarantee

1
−
α
≤
ℙ
x
,
y
,
D
cal
​
(
y
∈
Ω
θ
​
(
x
)
)
≤
1
−
α
+
1
M
+
1
.
1-\alpha\leq\mathbb{P}_{x,y,D_{\text{cal}}}(y\in\Omega_{\theta}(x))\leq 1-\alpha+\frac{1}{M+1}.

Algorithm 1

End-to-end conformal calibration for robust decisions under uncertainty

function

Train
(training data
D
=
{
(
x
i
,
y
i
)
}
i
=
1
N
D=\{(x_{i},y_{i})\}_{i=1}^{N}
, uncertainty level
α
\alpha
, initial model parameters
θ
\theta
)

for
mini-batch
B
⊂
{
1
,
…
,
N
}
B\subset\{1,\dotsc,N\}

do

Randomly split batch:
B
=
(
B
cal
,
B
pred
)
B=(B_{\text{cal}},B_{\text{pred}})

Compute
q
=
Quantile
​
(
{
s
θ
​
(
x
i
,
y
i
)
}
i
∈
B
cal
,
1
−
α
)
q=\textsc{Quantile}(\{s_{\theta}(x_{i},y_{i})\}_{i\in B_{\text{cal}}},\ 1-\alpha)

for

i
∈
B
pred
i\in B_{\text{pred}}

do

Solve for robust decision
z
θ
⋆
​
(
x
i
)
z_{\theta}^{\star}(x_{i})
using (
4
), (
5
), or (
8
)

Compute gradient of task loss:
d
​
θ
i
=
∂
f
⁡
(
x
i
,
y
i
,
z
θ
⋆
​
(
x
i
)
)
/
∂
θ
d\theta_{i}=\partial f(x_{i},y_{i},z_{\theta}^{\star}(x_{i}))/\partial\theta

Update
θ
\theta
using gradients
∑
i
∈
B
pred
d
​
θ
i
\sum_{i\in B_{\text{pred}}}d\theta_{i}

function

Inference
(model parameters
θ
\theta
, calibration data
D
cal
=
{
(
x
i
,
y
i
)
}
i
=
1
M
D_{\text{cal}}=\{(x_{i},y_{i})\}_{i=1}^{M}
, uncertainty level
α
\alpha
, input
x
x
)

Compute
q
=
Quantile
​
(
{
s
θ
​
(
x
~
,
y
~
)
}
(
x
~
,
y
~
)
∈
D
cal
,
1
−
α
)
q=\textsc{Quantile}(\{s_{\theta}(\tilde{x},\tilde{y})\}_{(\tilde{x},\tilde{y})\in D_{\text{cal}}},\ 1-\alpha)

return
robust decision
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
using (
4
), (
5
), or (
8
)

function

Quantile
(scores
S
=
{
s
i
}
i
=
1
M
S=\{s_{i}\}_{i=1}^{M}
, level
β
\beta
)

s
(
1
)
,
…
,
s
(
M
+
1
)
=
SortAscending
​
(
S
∪
{
+
∞
}
)
s_{(1)},\dotsc,s_{(M+1)}=\textsc{SortAscending}(S\cup\{+\infty\})

return

s
(
⌈
(
M
+
1
)
​
β
⌉
)
s_{(\left\lceil(M+1)\beta\right\rceil)}

We use split conformal prediction, rather than full conformal prediction, both for computational tractability and to avoid the problem of nonconvex uncertainty sets that can arise from the full conformal approach, as noted in
Johnstone and Cox (2021)
. For the rest of this paper, we assume
α
∈
[
1
M
+
1
,
1
)
\alpha\in[\frac{1}{M+1},1)
so that
q
=
Quantile
​
(
{
s
i
}
i
=
1
M
,
1
−
α
)
<
∞
q=\textsc{Quantile}(\{s_{i}\}_{i=1}^{M},\ 1-\alpha)<\infty
is finite. Thus, for appropriate choices of the score function
s
θ
s_{\theta}
, the uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is not unbounded.

While the split conformal prediction procedure in
Lemma

3.2
ensures that the uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
satisfies
(
1
−
α
)
(1-\alpha)
coverage at inference time, this process does not address the question of
training
the uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
(via the score function
s
θ
s_{\theta}
) to ensure optimal task performance while maintaining coverage. In
Section

3.3
, we propose applying a separate differentiable conformal prediction procedure during training to address this challenge.

3.3
End-to-end training and calibration

Thus far, we have discussed how to calibrate an uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
of the form (
3
) to ensure coverage, and we described two choices of score function
s
θ
s_{\theta}
parametrizing common box and ellipsoidal uncertainty sets. However, to ensure that the uncertainty sets
Ω
θ
​
(
x
)
\Omega_{\theta}(x)

both
guarantee coverage
and
ensure optimal downstream task performance, it is necessary to design an end-to-end training methodology that can incorporate both desiderata in a fully differentiable manner. We propose such a methodology in
Algorithm

1
.

Our end-to-end training approach minimizes the empirical task loss
ℓ
⁡
(
θ
)
=
1
N
​
∑
i
=
1
N
ℓ
i
​
(
θ
)
\ell(\theta)=\frac{1}{N}\sum_{i=1}^{N}\ell_{i}(\theta)
using minibatch gradient descent, where
ℓ
i
​
(
θ
)
=
f
⁡
(
x
i
,
y
i
,
z
θ
⋆
​
(
x
i
)
)
\ell_{i}(\theta)=f(x_{i},y_{i},z_{\theta}^{\star}(x_{i}))
. This requires differentiating through both the robust optimization problem as well as the conformal prediction step. The gradient of the task loss on a single instance is
d
​
ℓ
i
d
​
θ
=
∂
f
∂
z
​
(
x
i
,
y
i
,
z
θ
⋆
​
(
x
i
)
)
⋅
∂
z
θ
⋆
∂
θ
​
(
x
i
)
\frac{\mathrm{d}\ell_{i}}{\mathrm{d}\theta}=\frac{\partial f}{\partial z}(x_{i},y_{i},z_{\theta}^{\star}(x_{i}))\cdot\frac{\partial z_{\theta}^{\star}}{\partial\theta}(x_{i})
, where
∂
z
θ
⋆
∂
θ
​
(
x
i
)
\frac{\partial z_{\theta}^{\star}}{\partial\theta}(x_{i})
is computed by differentiating through the Karush–Kuhn–Tucker (KKT) conditions of the convex reformulation of the optimization problem (
1
) (i.e., the problems (
4
), (
5
)) following the approach of
Amos and Kolter (2017)
, under mild assumptions on the differentiability of
f
f
and
g
g
. Note that the gradient of any convex optimization problem can be computed with respect to its parameters as such
(
Agrawal et al., 2019
, Appendix B)
.

To include calibration during training, we assume that for every
(
x
,
y
)
(x,y)
in our training set,
s
θ
​
(
x
,
y
)
s_{\theta}(x,y)
is differentiable w.r.t.
θ
\theta
almost everywhere; this assumption holds for common nonconformity score functions, including those used in this paper. We then adopt the conformal training approach
(
Stutz et al., 2022
)
in which a separate
q
q
is chosen in each minibatch, as shown in
Algorithm

1
. The chosen
q
q
depends on
θ
\theta
(through
s
θ
s_{\theta}
), and
z
θ
⋆
​
(
x
i
)
z_{\theta}^{\star}(x_{i})
depends on the chosen
q
q
. Therefore
∂
z
θ
⋆
∂
θ
\frac{\partial z_{\theta}^{\star}}{\partial\theta}
involves calculating
∂
z
θ
⋆
∂
q
​
∂
q
∂
θ
\frac{\partial z_{\theta}^{\star}}{\partial q}\frac{\partial q}{\partial\theta}
, where
∂
q
∂
θ
\frac{\partial q}{\partial\theta}
requires differentiating through the empirical quantile function. Whereas
Stutz et al. (2022)
uses a smoothed approximate quantile function for calculating
q
q
, we find the smoothing unnecessary, as the gradient of the empirical quantile function is unique and well-defined almost everywhere. Importantly, our exact gradient is both more computationally efficient and simpler to implement than the smoothed approximate quantile approach. See
Appendix

D
for more details.

After training has concluded and we have performed the final conformal calibration step, the resulting model enjoys the following theoretical guarantee on performance (
cf.

Sun et al. (2023, Proposition 1)
).

Proposition 3.3
.

Under the same assumptions as
Lemma

3.2
, the task loss satisfies the following bound with probability at least
1
−
α
1-\alpha
(over
x
,
y
x,y
, and the calibration set
D
cal
D_{\text{cal}}
):

f
(
x
,
y
,
z
θ
⋆
(
x
)
)
≤
(
min
z
∈
ℝ
p
max
y
^
∈
Ω
θ
​
(
x
)
f
(
x
,
y
^
,
z
)
s
.
t
.
g
(
x
,
z
)
≤
0
)
.
f(x,y,z_{\theta}^{\star}(x))\leq\left(\min_{z\in\mathbb{R}^{p}}\max_{\hat{y}\in\Omega_{\theta}(x)}f(x,\hat{y},z)\ \mathrm{s.t.}\ g(x,z)\leq 0\right).

(6)

Proof.

When
y
∈
Ω
θ
​
(
x
)
y\in\Omega_{\theta}(x)
, the definition of
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
(
1
) guarantees that (
6
) holds. By
Lemma

3.2
,
ℙ
x
,
y
,
D
cal
​
(
y
∈
Ω
θ
​
(
x
)
)
≥
1
−
α
\mathbb{P}_{x,y,D_{\text{cal}}}(y\in\Omega_{\theta}(x))\geq 1-\alpha
.
∎

We note that in practice, training models from scratch using only the task loss sometimes leads to poor local optima. Instead, in our experiments, we pretrain our models with a standard loss function (e.g., negative log-likelihood for the mean-variance predictor used in ellipsoidal uncertainty sets), and then we fine-tune the model using the task loss. This is a standard heuristic commonly used in the decision-focused learning and predict-then-optimize literature
(
Donti et al., 2017
;
Mandi et al., 2024
)
.
Appendix

E
provides more details on our training procedure.

4
Representing General Convex Uncertainty Sets via PICNNs

Figure 2
:
Consider a robust portfolio optimization problem with 2 assets, where
y
∈
ℝ
2
y\in\mathbb{R}^{2}
is a random vector of asset returns, and the decision
z
∈
ℝ
2
z\in\mathbb{R}^{2}
represents portfolio weights:
max
z
⁡
min
y
^
∈
Ω
−
z
⊤
​
y
^
​
s.t.
​
z
≥
0
,
1
⊤
​
z
≤
1
\max_{z}\min_{\hat{y}\in\Omega}-z^{\top}\hat{y}\ \text{s.t.}\ z\geq 0,\ \mathbf{1}^{\top}z\leq 1
. Let the distribution of asset returns
y
y
be uniform over 3 discrete points (black). The optimal box (blue), ellipse (orange), and PICNN (green) uncertainty sets are shown with their robust decision vectors
z
⋆
z^{\star}
. The flexibility of the PICNN uncertainty representation allows it to achieve the lowest expected task loss.

The previous section discussed how to train calibrated box and ellipsoidal uncertainty sets end-to-end to optimize the downstream task loss. However,
both box and ellipsoidal uncertainty sets have restrictive shapes which may yield suboptimal task performance. If
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
could represent
any
arbitrary convex uncertainty set, this more expressive class would enable obtaining better task loss.
Figure

2
illustrates an example where a general convex uncertainty set representation provides a clear advantage over box and ellipsoid uncertainty.

To this end, we propose to directly learn a partially-convex nonconformity score function
s
θ
:
ℝ
m
×
ℝ
n
→
ℝ
s_{\theta}:\mathbb{R}^{m}\times\mathbb{R}^{n}\to\mathbb{R}
that is convex only in the second input vector. Fixing
x
x
, any
q
q
-sublevel set
{
y
^
∈
ℝ
n
∣
s
θ
​
(
x
,
y
^
)
≤
q
}
\{\hat{y}\in\mathbb{R}^{n}\mid s_{\theta}(x,\hat{y})\leq q\}
of
s
θ
s_{\theta}
is a convex set, and likewise every family of convex sets can be expressed as the
q
q
-sublevel sets of some partially-convex function. To implement this approach, we are faced with two questions.

1. How should we parametrize the score function
s
θ
s_{\theta}
so
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
can approximate
arbitrary
convex sets?
A natural answer is to parametrize
s
θ
s_{\theta}
with a partially input-convex neural network (PICNN)
(
Amos et al., 2017
)
, which can efficiently approximate any partially-convex function
(
Chen et al., 2019
)
. We consider a PICNN defined as
s
θ
​
(
x
,
y
)
=
W
L
​
σ
L
+
V
L
​
y
+
b
L
s_{\theta}(x,y)=W_{L}\sigma_{L}+V_{L}y+b_{L}
, where

σ
0
\displaystyle\sigma_{0}

=
𝟎
,
u
0
=
x
,
\displaystyle=\mathbf{0},\qquad u_{0}=x,

W
l
\displaystyle W_{l}

=
W
¯
l
​
diag
⁡
(
[
W
^
l
​
u
l
+
w
l
]
+
)
\displaystyle=\bar{W}_{l}\diag([\hat{W}_{l}u_{l}+w_{l}]_{+})

(7)

u
l
+
1
\displaystyle u_{l+1}

=
ReLU
⁡
(
R
l
​
u
l
+
r
l
)
,
\displaystyle=\ReLU\left(R_{l}u_{l}+r_{l}\right),

V
l
\displaystyle V_{l}

=
V
¯
l
​
diag
⁡
(
V
^
l
​
u
l
+
v
l
)
\displaystyle=\bar{V}_{l}\diag(\hat{V}_{l}u_{l}+v_{l})

σ
l
+
1
\displaystyle\sigma_{l+1}

=
ReLU
⁡
(
W
l
​
σ
l
+
V
l
​
y
+
b
l
)
,
\displaystyle=\ReLU\left(W_{l}\sigma_{l}+V_{l}y+b_{l}\right),

b
l
\displaystyle b_{l}

=
B
¯
l
​
u
l
+
b
¯
l
,
\displaystyle=\bar{B}_{l}u_{l}+\bar{b}_{l},

with weights
θ
=
(
R
l
,
r
l
,
W
¯
l
,
W
^
l
,
w
l
,
V
¯
l
,
V
^
l
,
v
l
,
B
¯
l
,
b
¯
l
)
l
=
0
L
\theta=(R_{l},r_{l},\bar{W}_{l},\hat{W}_{l},w_{l},\bar{V}_{l},\hat{V}_{l},v_{l},\bar{B}_{l},\bar{b}_{l})_{l=0}^{L}
. The matrices
W
¯
l
\bar{W}_{l}
are constrained to be entrywise nonnegative to ensure convexity of
s
θ
s_{\theta}
with respect to
y
y
. For ease of notation, we assume all hidden layers
σ
1
,
…
,
σ
L
\sigma_{1},\dotsc,\sigma_{L}
have the same dimension
d
d
.

2. Does the chosen parametrization of
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
(via PICNNs) yield a tractable reformulation of the CRO problem (
1
)?
Fortunately, we show in the following theorem that the answer is yes.

Theorem 4.1
.

Let
Ω
θ
​
(
x
)
=
{
y
^
∈
ℝ
n
∣
s
θ
​
(
x
,
y
^
)
≤
q
}
\Omega_{\theta}(x)=\{\hat{y}\in\mathbb{R}^{n}\mid s_{\theta}(x,\hat{y})\leq q\}
, where
s
θ
s_{\theta}
is a PICNN as defined in (
7
). Then, under
Assumption

3.1
, the CRO problem (
1
) with uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is equivalent to the following convex (and hence tractable) minimization problem:

z
θ
⋆
​
(
x
)
=
arg
​
min
z
∈
ℝ
p
⁡
min
ν
∈
ℝ
2
​
L
​
d
+
1
\displaystyle z_{\theta}^{\star}(x)=\argmin_{z\in\mathbb{R}^{p}}\min_{\nu\in\mathbb{R}^{2Ld+1}}

b
​
(
θ
,
q
)
⊤
​
ν
+
f
~
​
(
x
,
z
)
\displaystyle b(\theta,q)^{\top}\nu+\tilde{f}(x,z)

(8)

s
.
t
.
\displaystyle\mathrm{s.t.}

A
​
(
θ
)
⊤
​
ν
=
[
F
⁡
(
x
,
z
)
𝟎
]
,
ν
≥
𝟎
,
g
⁡
(
x
,
z
)
≤
0
\displaystyle A(\theta)^{\top}\nu=\begin{bmatrix}F(x,z)\\
\mathbf{0}\end{bmatrix},\ \nu\geq\mathbf{0},\ g(x,z)\leq 0

where
A
⁡
(
θ
)
∈
ℝ
(
2
​
L
​
d
+
1
)
×
(
n
+
L
​
d
)
A(\theta)\in\mathbb{R}^{(2Ld+1)\times(n+Ld)}
and
b
⁡
(
θ
,
q
)
∈
ℝ
2
​
L
​
d
+
1
b(\theta,q)\in\mathbb{R}^{2Ld+1}
are constructed from the weights
θ
\theta
of the PICNN (
7
), and
b
b
also depends on
q
q
.

We prove
Theorem

4.1
in
Section

C.3
; the main idea is that when
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is a sublevel set of a PICNN, we can equivalently reformulate the inner maximization problem in (
1
) as a linear program and take the dual to yield a tractable minimization problem.

Since the PICNN uncertainty sets are of the same form as (
3
) and yield a tractable convex reformulation (
8
) of the CRO problem (
1
), we can apply the split conformal procedure detailed in
Section

3.2
to choose
q
∈
ℝ
q\in\mathbb{R}
and obtain coverage guarantees on
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
, and we can employ the same end-to-end training methodology from
Section

3.3
to train calibrated uncertainties end-to-end using the downstream task loss. In some cases during training, the inner maximization problem of (
1
) with PICNN-parametrized uncertainty set may be unbounded (if
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is not compact) or infeasible (if the chosen
q
q
is too small causing
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
to be empty). This will lead, respectively, to an infeasible or unbounded equivalent problem (
8
). We can avoid this concern by adjusting the PICNN architecture to ensure its sublevel sets are compact and by suitably increasing
q
q
when needed to ensure
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is never empty. Such modifications do not change the general form of the problem (
8
) and preserve the marginal coverage guarantee for the uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
; see
Section

C.4
for details.

5
Experiments

In this section, we present experimental results for our E2E method against several ETO baselines. Code to reproduce our results is available on GitHub.
3
3

3

https://github.com/chrisyeh96/e2e-conformal

5.1
Problem descriptions

We consider two tasks: price forecasting for battery storage operation and portfolio optimization. Their task loss functions and constraints satisfy
Assumption

3.1
.

Price forecasting for battery storage.

This problem comes from
Donti et al. (2017)
, where a grid-scale battery operator predicts electricity prices
y
∈
ℝ
T
y\in\mathbb{R}^{T}
over a
T
T
-step horizon and uses the predicted prices to decide a battery charge/discharge schedule for price arbitrage. The input features
x
x
include the past day’s prices and temperature, the next day’s energy load forecast and temperature forecast, binary indicators of weekends or holidays, and yearly sinusoidal features. The operator decides how much to charge (
z
in
∈
ℝ
T
z^{\mathrm{in}}\in\mathbb{R}^{T}
) or discharge (
z
out
∈
ℝ
T
z^{\mathrm{out}}\in\mathbb{R}^{T}
) the battery, which changes the battery’s state of charge (
z
state
∈
ℝ
T
z^{\mathrm{state}}\in\mathbb{R}^{T}
). The battery has capacity
B
B
, charging efficiency
γ
\gamma
, and maximum charging/discharging rates
c
in
c^{\mathrm{in}}
and
c
out
c^{\mathrm{out}}
. The task loss function represents the multiple objectives of maximizing profit, flexibility to participate in other markets by keeping the battery near half its capacity (with weight
λ
\lambda
), and battery health by discouraging rapid charging/discharging (with weight
ϵ
\epsilon
):

f
⁡
(
y
,
z
)
=
∑
t
=
1
T
y
t
​
(
z
in
−
z
out
)
t
+
λ
​
‖
z
state
−
B
2
​
𝟏
‖
2
+
ϵ
​
‖
z
in
‖
2
+
ϵ
​
‖
z
out
‖
2
.
f(y,z)=\sum_{t=1}^{T}y_{t}(z^{\mathrm{in}}-z^{\mathrm{out}})_{t}+\lambda\left\lVert z^{\mathrm{state}}-\frac{B}{2}\mathbf{1}\right\rVert^{2}+\epsilon\left\lVert z^{\mathrm{in}}\right\rVert^{2}+\epsilon\left\lVert z^{\mathrm{out}}\right\rVert^{2}.

The constraints are, for all
t
=
1
,
…
,
T
t=1,\dotsc,T
,

z
0
state
=
B
/
2
,
z
t
state
=
z
t
−
1
state
−
z
t
out
+
γ
​
z
t
in
,
\displaystyle z^{\mathrm{state}}_{0}=B/2,\quad z^{\mathrm{state}}_{t}=z^{\mathrm{state}}_{t-1}-z^{\mathrm{out}}_{t}+\gamma z^{\mathrm{in}}_{t},

0
≤
z
in
≤
c
in
,
0
≤
z
out
≤
c
out
,
0
≤
z
t
state
≤
B
.
\displaystyle 0\leq z^{\mathrm{in}}\leq c^{\mathrm{in}},\quad 0\leq z^{\mathrm{out}}\leq c^{\mathrm{out}},\quad 0\leq z^{\mathrm{state}}_{t}\leq B.

Following
Donti et al. (2017)
, we set
T
=
24
T=24
,
B
=
1
B=1
,
γ
=
0.9
\gamma=0.9
,
c
in
=
0.5
c^{\mathrm{in}}=0.5
,
c
out
=
0.2
c^{\mathrm{out}}=0.2
,
λ
=
0.1
\lambda=0.1
, and
ϵ
=
0.05
\epsilon=0.05
.

Portfolio optimization.

We adopt the portfolio optimization setting and synthetic dataset from
Chenreddy and Delage (2024)
, where the prediction targets
y
∈
ℝ
n
y\in\mathbb{R}^{n}
are the returns of a set of
n
n
securities, and the decision
z
∈
ℝ
n
z\in\mathbb{R}^{n}
sets portfolio weights. The task loss is
f
⁡
(
y
,
z
)
=
−
y
⊤
​
z
f(y,z)=-y^{\top}z
, with constraints
z
≥
0
,
1
⊤
​
z
=
1
z\geq 0,\ \mathbf{1}^{\top}z=1
. The synthetic dataset consists of
(
x
,
y
)
∈
ℝ
2
×
2
(x,y)\in\mathbb{R}^{2\times 2}
drawn from a mixture of three 4-D multivariate normal distributions. We provide data details in
Section

E.2
and experimental results in
Section

A.2
. The results are similar to those for battery storage, except that portfolio optimization is a lower-dimensional and easier problem.

5.2
Baseline methods

We implemented several “estimate-then-optimize” (ETO) baselines, listed below, to compare against our end-to-end (E2E) method. These two-stage ETO baselines are trained using task-agnostic losses such as pinball loss or negative log-likelihood (NLL). To ensure a fair comparison against our E2E method, we also apply conformal calibration to each ETO method after training to satisfy coverage.

•

ETO
denotes models with identical neural network architectures to our E2E models, differing only in the loss function during training. The box uncertainty
ETO
model is trained with pinball loss to predict the
α
2
\frac{\alpha}{2}
and
1
−
α
2
1-\frac{\alpha}{2}
quantiles. The ellipsoidal uncertainty
ETO
model is trained with a multivariate normal negative log-likelihood loss. For the PICNN
ETO
model, we train
s
θ
s_{\theta}
using a negative log-likelihood loss by interpreting
s
θ
s_{\theta}
as an energy function—i.e.,
𝒫
^
θ
​
(
y
∣
x
)
∝
exp
⁡
(
−
s
θ
​
(
x
,
y
)
)
\hat{\mathcal{P}}_{\theta}(y\mid x)\propto\exp(-s_{\theta}(x,y))
—yielding the loss

NLL
⁡
(
θ
)
=
ln
⁡
s
θ
​
(
x
,
y
)
+
ln
⁡
Z
θ
​
(
x
)
,
\NLL(\theta)=\ln s_{\theta}(x,y)+\ln Z_{\theta}(x),

where
Z
θ
​
(
x
)
=
∫
y
~
∈
ℝ
n
exp
⁡
(
−
s
θ
​
(
x
,
y
~
)
)
​
⁡
𝑑
y
~
Z_{\theta}(x)=\int_{\tilde{y}\in\mathbb{R}^{n}}\exp(-s_{\theta}(x,\tilde{y}))\,\mathop{}\!\mathrm{d}\tilde{y}
, following the approach of
Lin and Ba (2023)
. More details of the
ETO
models are given in
Appendix

E
.

•

ETO-SLL
is our implementation of the box and ellipsoid uncertainty ETO methods from
Sun et al. (2023)
. Unlike
ETO
,
ETO-SLL
first trains a point estimate model (without uncertainty) with mean-squared error loss. Then,
ETO-SLL
box and ellipsoidal uncertainty sets are derived from training a separate quantile regressor using pinball loss to predict the
(
1
−
α
)
(1-\alpha)
-quantiles of absolute residuals or
ℓ
2
\ell_{2}
-norm of residuals of the point estimate. Unlike
ETO
which can learn ellipsoidal uncertainty sets with different covariance matrices for each input
x
x
, the
ETO-SLL
ellipsoidal uncertainty sets all share the same covariance matrix (up to scale).

•

ETO-JC
is our implementation of the ellipsoid uncertainty ETO method by
Johnstone and Cox (2021)
. Like
ETO-SLL
,
ETO-JC
also first trains a point estimate model (without uncertainty) with mean-squared error loss.
ETO-JC
uses the same covariance matrix (with the same scale) for each input
x
x
.

5.3
Battery storage problem results

no
distribution
shift

with
distribution
shift

Figure 3
:
Task loss performance (mean
±
\pm
1 stddev across 10 runs) for the battery storage problem with no distribution shift (top) and with distribution shift (bottom). Lower values are better.

no
distribution
shift

with
distribution
shift

Figure 4
:

Coverage (mean
±
\pm
1 stddev across 10 runs) for the battery storage problem with no distribution shift (top) and with distribution shift (bottom). The dotted black line indicates the target coverage level
1
−
α
1-\alpha
. Our E2E models achieve similar coverage to the ETO baselines, confirming that the lower task loss of our E2E models does not come at the expense of worse coverage.

Figure

3
(top) compares task loss performance for different uncertainty levels (
α
∈
{
.01
,
.05
,
.1
,
.2
}
\alpha\in\{.01,.05,.1,.2\}
) and the different uncertainty set representations for the ETO baselines against our proposed E2E methodology on the battery storage problem with no distribution shift. Our E2E approach consistently yields improved performance over all ETO baselines, for all three uncertainty set parametrizations, and over all tested uncertainty levels
α
\alpha
. Moreover, the PICNN uncertainty representation, when trained end-to-end, provides up to 42% relative improvement in performance over the best ETO box uncertainty set and up to 209% relative improvement over the best ETO ellipse uncertainty set. We also show the corresponding coverage obtained by the learned uncertainty sets in
Figure

4
(top); all models obtain coverage close to the target level, confirming that the improvements in task loss performance from our E2E approach do not come at the cost of worse coverage.

In
Appendix

A
, we present additional results on the battery storage problem, including value-at-risk (VaR) and conditional value-at-risk (CVaR) metrics which show that the E2E approach consistently outperforms the ETO baselines on tail risk as well. In addition,
Table

A1
shows the per-epoch computational time required by our E2E approach compared with the two-stage baselines. While E2E requires additional training time, the training time is dominated by time spent solving the conditional robust optimization problem (
1
) for every training example at every epoch. The development of GPU-accelerated solvers
(
Blin, 2024
)
may help significantly speed up the optimization, though we do not explore GPU-accelerated optimization in our implementation.

5.4
Performance under distribution shift

The aforementioned results were produced without distribution shift, where our training and test sets were sampled uniformly at random, thus ensuring exchangeability and guaranteeing marginal coverage. In this section, we evaluate our method on the more realistic setting with distribution shift by splitting our data temporally; our models are trained on the first 80% of days and evaluated on the last 20% of days.
Figure

A4
plots the underlying time series data and visually highlights the distribution shift. Figures
3
(bottom) and
4
(bottom) mirror Figures
3
(top) and
4
(top), except that there is now distribution shift. We again find that our E2E approach consistently yields improved performance over all ETO baselines, for all three uncertainty set parametrizations, and for all tested uncertainty levels
α
\alpha
. Likewise, the PICNN uncertainties, when trained end-to-end, improve on the performance offered by box and ellipsoidal uncertainty. We unsurprisingly find that, under distribution shift, the models do not provide the same level of coverage guaranteed in the i.i.d. case, as the exchangeability assumption needed for conformal prediction no longer holds. The ellipsoidal and PICNN models tend to provide worse coverage than the box uncertainty, which we believe reflects how ellipsoidal and PICNN uncertainty sets offer greater representational power, and thus might be fitting too closely to the pre-shift distribution, which impacts robustness under distribution shift. Devising methods to anticipate distribution shift when training these more expressive models, and in particular the PICNN-based uncertainty, remains an interesting avenue for future work.

6
Conclusion

In this work, we develop the first end-to-end methodology for training predictive models with uncertainty estimates (with calibration enforced differentiably throughout training) that are utilized in downstream conditional robust optimization problems. We demonstrate an approach utilizing partially input-convex neural networks (PICNNs) to represent general convex uncertainty regions, and we perform extensive experiments on a battery storage application and a portfolio optimization task. Whereas prior works on two-stage estimate-then-optimize approaches emphasized “the convenience brought by the disentanglement of the prediction and the uncertainty calibration”
(
Sun et al., 2023
)
, our results highlight that such “convenience” comes at a substantial cost; our end-to-end approach, combined with the expressiveness of the PICNN representation, has clear performance gains over the traditional two-stage methods.

A number of interesting directions for future work on learning decision-aware uncertainty in an end-to-end manner remain. First, while our PICNN-based uncertainty set representation allows the parametrization of general convex uncertainty sets, future work may explore
nonconvex
uncertainty regions. Doing so may require eschewing the analytical methods for differentiating through convex optimization problems and instead use, e.g., policy gradient methods for passing gradients through general stochastic and robust optimization problems. Second, developing end-to-end methods to target conditional calibration (as opposed to marginal calibration) remains an active research direction. Finally, one may explore other types of constraints besides uncertainty set-based robustness, such as value-at-risk (VaR) or conditional value-at-risk (CVaR) constraints.

Acknowledgments

We thank Priya Donti for helpful discussions. The authors acknowledge support from an NSF Graduate Research Fellowship (DGE-2139433); NSF Grants CNS-2146814, CPS-2136197, CNS-2106403, and NGSDI-2105648; gifts from Amazon, OpenAI, and Latitude AI; and the Caltech Resnick Sustainability Institute.

References

Agrawal
et al.
(2019)

A. Agrawal, B. Amos, S. Barratt, S. Boyd, S. Diamond, and J. Z. Kolter

Differentiable Convex Optimization Layers
.

In
Advances in Neural Information Processing Systems
,

Vol.
32
.

External Links:
Link

Cited by:
Appendix B
,

§C.4
,

Appendix E
,

§3.3
.

Alexeenko and Bitar (2020)

P. Alexeenko and E. Bitar

Nonparametric Estimation of Uncertainty Sets for Robust Optimization
.

In
2020 59th IEEE Conference on Decision and Control (CDC)
,

Jeju, Korea (South)
,
pp. 1196–1203
(
en
).

External Links:
ISBN 978-1-72817-447-1
,

Link
,

Document

Cited by:
§2
.

Amos and Kolter (2017)

B. Amos and J. Z. Kolter

OptNet: Differentiable Optimization as a Layer in Neural Networks
.

In
Proceedings of the 34th International Conference on Machine Learning
,

pp. 136–145
(
en
).

External Links:
ISSN 2640-3498
,

Link

Cited by:
Appendix B
,

§3.3
.

Amos
et al.
(2017)

B. Amos, L. Xu, and J. Z. Kolter

Input Convex Neural Networks
.

In
Proceedings of the 34th International Conference on Machine Learning
,

Proceedings of Machine Learning Research
, Vol.
70
,
pp. 146–155
.

External Links:
Link

Cited by:
§2
,

§4
.

Angelopoulos
et al.
(2023)

A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster

Conformal Risk Control
.

arXiv
(
en
).

Note:
arXiv:2208.02814 [cs, math, stat]

External Links:
Link

Cited by:
§3.2
,

Lemma 3.2
.

Ben-Tal
et al.
(2009)

A. Ben-Tal, L. E. Ghaoui, and A. Nemirovski

Robust Optimization
.

Princeton University Press
.

External Links:
ISBN 978-0-691-14368-2
,

Link

Cited by:
Appendix B
.

Bertsimas
et al.
(2018)

D. Bertsimas, V. Gupta, and N. Kallus

Data-driven robust optimization
.

Mathematical Programming

167
(
2
),
pp. 235–292
(
en
).

External Links:
ISSN 1436-4646
,

Link
,

Document

Cited by:
Appendix B
,

§2
.

Bertsimas
et al.
(2013)

D. Bertsimas, E. Litvinov, X. A. Sun, J. Zhao, and T. Zheng

Adaptive Robust Optimization for the Security Constrained Unit Commitment Problem
.

IEEE Transactions on Power Systems

28
(
1
),
pp. 52–63
.

Note:
Conference Name: IEEE Transactions on Power Systems

External Links:
ISSN 1558-0679
,

Link
,

Document

Cited by:
Appendix B
.

Bertsimas and Thiele (2006)

D. Bertsimas and A. Thiele

Robust and Data-Driven Optimization: Modern Decision Making Under Uncertainty
.

In
Models, Methods, and Applications for Innovative Decision Making
,

INFORMS TutORials in Operations Research
,
pp. 95–122
.

Note:
Section: 4

External Links:
ISBN 978-1-877640-20-9
,

Link
,

Document

Cited by:
§2
.

Blin (2024)

N. Blin

Accelerate Large Linear Programming Problems with NVIDIA cuOpt
.

(
en-US
).

External Links:
Link

Cited by:
§A.1
,

§5.3
.

Blundell
et al.
(2015)

C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra

Weight Uncertainty in Neural Networks
.

In
Proceedings of the 32nd International Conference on Machine Learning
,

Proceedings of Machine Learning Research
, Vol.
37
,
pp. 1613–1622
.

External Links:
Link

Cited by:
Appendix B
.

Chen
et al.
(2019)

Y. Chen, Y. Shi, and B. Zhang

Optimal Control Via Neural Networks: A Convex Approach
.

In
International Conference on Learning Representations
,

(
en
).

External Links:
Link

Cited by:
item 2
,

§4
.

Chenreddy
et al.
(2022)

A. R. Chenreddy, N. Bandi, and E. Delage

Data-Driven Conditional Robust Optimization
.

In
Advances in Neural Information Processing Systems
,

Vol.
35
,
pp. 9525–9537
(
en
).

External Links:
Link

Cited by:
§1
,

§1
,

§2
,

§2
.

Chenreddy and Delage (2024)

A. R. Chenreddy and E. Delage

End-to-end Conditional Robust Optimization
.

In
Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence
,

pp. 736–748
(
en
).

External Links:
ISSN 2640-3498
,

Link

Cited by:
§E.2
,

§E.2
,

§1
,

§2
,

§5.1
,

footnote 1
.

Christianson
et al.
(2022)

N. Christianson, L. Werner, A. Wierman, and S. Low

Dispatch-aware planning for feasible power system operation
.

Electric Power Systems Research

212
,
pp. 108597
.

External Links:
ISSN 0378-7796
,

Link
,

Document

Cited by:
Appendix B
.

Cortes-Gomez
et al.
(2024)

S. Cortes-Gomez, C. M. Patiño, Y. Byun, S. Wu, E. Horvitz, and B. Wilder

Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantification
.

In
The Thirteenth International Conference on Learning Representations
,

(
en
).

External Links:
Link

Cited by:
§2
.

Donti
et al.
(2017)

P. L. Donti, B. Amos, and J. Z. Kolter

Task-based End-to-end Model Learning in Stochastic Optimization
.

In
Advances in Neural Information Processing Systems
,

Vol.
30
,
Long Beach, CA, USA
.

External Links:
Link

Cited by:
Appendix B
,

Appendix B
,

§E.2
,

§3.3
,

§5.1
,

§5.1
.

Dvorkin (2020)

Y. Dvorkin

A Chance-Constrained Stochastic Electricity Market
.

IEEE Transactions on Power Systems

35
(
4
),
pp. 2993–3003
.

Note:
Conference Name: IEEE Transactions on Power Systems

External Links:
ISSN 1558-0679
,

Link
,

Document

Cited by:
Appendix B
.

Einbinder
et al.
(2022)

B. Einbinder, Y. Romano, M. Sesia, and Y. Zhou

Training Uncertainty-Aware Classifiers with Conformalized Deep Learning
.

In
Advances in Neural Information Processing Systems
,

New Orleans, LA, USA
.

Note:
arXiv:2205.05878 [cs, stat]

External Links:
Link
,

Document

Cited by:
Appendix B
.

Elmachtoub and Grigas (2022)

A. N. Elmachtoub and P. Grigas

Smart “Predict, then Optimize”
.

Management Science

68
(
1
),
pp. 9–26
.

External Links:
ISSN 0025-1909
,

Link
,

Document

Cited by:
Appendix B
.

Elmachtoub
et al.
(2023)

A. N. Elmachtoub, H. Lam, H. Zhang, and Y. Zhao

Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective
.

arXiv
.

Note:
arXiv:2304.06833 [cs, stat]

External Links:
Link
,

Document

Cited by:
§2
.

Gal and Ghahramani (2016)

Y. Gal and Z. Ghahramani

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
.

In
arXiv:1506.02142 [cs, stat]
,

Note:
arXiv: 1506.02142

External Links:
Link

Cited by:
Appendix B
.

Goerigk and Kurtz (2023)

M. Goerigk and J. Kurtz

Data-driven robust optimization using deep neural networks
.

Computers & Operations Research

151
,
pp. 106087
.

External Links:
ISSN 0305-0548
,

Link
,

Document

Cited by:
§2
.

Gregory
et al.
(2011)

C. Gregory, K. Darby-Dowman, and G. Mitra

Robust optimization and portfolio selection: The cost of robustness
.

European Journal of Operational Research

212
(
2
),
pp. 417–428
.

External Links:
ISSN 0377-2217
,

Link
,

Document

Cited by:
Appendix B
,

§1
.

Johnstone and Cox (2021)

C. Johnstone and B. Cox

Conformal uncertainty sets for robust optimization
.

In
Proceedings of the Tenth Symposium on Conformal and Probabilistic Prediction and Applications
,

pp. 72–90
(
en
).

External Links:
ISSN 2640-3498
,

Link

Cited by:
§2
,

§3.1
,

§3.2
,

3rd item
.

Juditsky and Nemirovski (2022)

A. Juditsky and A. Nemirovski

On well-structured convex–concave saddle point problems and variational inequalities with monotone operators
.

Optimization Methods and Software

37
(
5
),
pp. 1567–1602
.

Note:
_eprint: https://doi.org/10.1080/10556788.2021.1928121

External Links:
ISSN 1055-6788
,

Link
,

Document

Cited by:
§3
.

Kingma and Ba (2015)

D. P. Kingma and J. Ba

Adam: A Method for Stochastic Optimization
.

In
International Conference on Learning Representations
,

San Diego, CA, USA
.

External Links:
Link

Cited by:
Appendix E
.

Kiyani
et al.
(2025)

S. Kiyani, G. Pappas, A. Roth, and H. Hassani

Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents
.

arXiv
.

Note:
arXiv:2502.02561 [cs]

External Links:
Link
,

Document

Cited by:
§2
.

Kuleshov
et al.
(2018)

V. Kuleshov, N. Fenner, and S. Ermon

Accurate Uncertainties for Deep Learning Using Calibrated Regression
.

In
Proceedings of the 35th International Conference on Machine Learning
,

pp. 2796–2804
(
en
).

External Links:
ISSN 2640-3498
,

Link

Cited by:
Appendix B
,

item 3
.

Lakshminarayanan
et al.
(2017)

B. Lakshminarayanan, A. Pritzel, and C. Blundell

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
.

In
Advances in Neural Information Processing Systems
,

Vol.
30
.

External Links:
Link

Cited by:
Appendix B
.

Lin and Ba (2023)

A. Lin and D. E. Ba

How to Train Your FALCON: Learning Log-Concave Densities with Energy-Based Neural Networks
.

In
Fifth Symposium on Advances in Approximate Bayesian Inference
,

(
en
).

External Links:
Link

Cited by:
§E.1
,

1st item
.

Liu
et al.
(2020)

J. Liu, Z. Lin, S. Padhy, D. Tran, T. Bedrax Weiss, and B. Lakshminarayanan

Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
.

Vol.
33
,
pp. 7498–7512
.

External Links:
Link

Cited by:
Appendix B
.

Mandi
et al.
(2024)

J. Mandi, J. Kotary, S. Berden, M. Mulamba, V. Bucarey, T. Guns, and F. Fioretto

Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities
.

Journal of Artificial Intelligence Research

80
,
pp. 1623–1701
(
en
).

External Links:
ISSN 1076-9757
,

Link
,

Document

Cited by:
Appendix B
,

§3.3
.

Nado
et al.
(2022)

Z. Nado, N. Band, M. Collier, J. Djolonga, M. W. Dusenberry, S. Farquhar, Q. Feng, A. Filos, M. Havasi, R. Jenatton, G. Jerfel, J. Liu, Z. Mariet, J. Nixon, S. Padhy, J. Ren, T. G. J. Rudner, F. Sbahi, Y. Wen, F. Wenzel, K. Murphy, D. Sculley, B. Lakshminarayanan, J. Snoek, Y. Gal, and D. Tran

Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
.

Note:
arXiv:2106.04015 [cs]

External Links:
Link
,

Document

Cited by:
Appendix B
.

Ndrio
et al.
(2021)

M. Ndrio, A. N. Madavan, and S. Bose

Pricing Conditional Value at Risk-Sensitive Economic Dispatch
.

In
2021 IEEE Power & Energy Society General Meeting (PESGM)
,

pp. 01–05
.

External Links:
ISSN 1944-9933
,

Link
,

Document

Cited by:
Appendix B
.

Nemirovski and Shapiro (2007)

A. Nemirovski and A. Shapiro

Convex Approximations of Chance Constrained Programs
.

SIAM Journal on Optimization

17
(
4
),
pp. 969–996
(
en
).

External Links:
ISSN 1052-6234, 1095-7189
,

Link
,

Document

Cited by:
Appendix B
.

Parvar and Nazaripouya (2022)

S. S. Parvar and H. Nazaripouya

Optimal Operation of Battery Energy Storage Under Uncertainty Using Data-Driven Distributionally Robust Optimization
.

Electric Power Systems Research

211
,
pp. 108180
.

External Links:
ISSN 0378-7796
,

Link
,

Document

Cited by:
§1
.

Patel
et al.
(2024)

Y. P. Patel, S. Rayan, and A. Tewari

Conformal Contextual Robust Optimization
.

In
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics
,

pp. 2485–2493
(
en
).

External Links:
ISSN 2640-3498
,

Link

Cited by:
§1
,

§2
.

Poolla
et al.
(2021)

B. K. Poolla, A. R. Hota, S. Bolognani, D. S. Callaway, and A. Cherukuri

Wasserstein Distributionally Robust Look-Ahead Economic Dispatch
.

IEEE Transactions on Power Systems

36
(
3
),
pp. 2010–2022
.

Note:
Conference Name: IEEE Transactions on Power Systems

External Links:
ISSN 1558-0679
,

Link
,

Document

Cited by:
Appendix B
.

Quaranta and Zaffaroni (2008)

A. G. Quaranta and A. Zaffaroni

Robust optimization of conditional value at risk and portfolio selection
.

Journal of Banking & Finance

32
(
10
),
pp. 2046–2056
.

External Links:
ISSN 0378-4266
,

Link
,

Document

Cited by:
Appendix B
,

§1
.

Rahimian and Mehrotra (2019)

H. Rahimian and S. Mehrotra

Distributionally Robust Optimization: A Review
.

arXiv:1908.05659 [cs, math, stat]
.

Note:
arXiv: 1908.05659

External Links:
Link

Cited by:
Appendix B
.

Rasmussen and Williams (2005)

C. E. Rasmussen and C. K. I. Williams

Gaussian Processes for Machine Learning
.

The MIT Press
.

External Links:
ISBN 978-0-262-25683-4
,

Link
,

Document

Cited by:
Appendix B
.

Romano
et al.
(2019)

Y. Romano, E. Patterson, and E. Candes

Conformalized Quantile Regression
.

In
Advances in Neural Information Processing Systems
,
H. Wallach, H. Larochelle, A. Beygelzimer, F. d. Alché-Buc, E. Fox, and R. Garnett (Eds.)
,

Vol.
32
.

External Links:
Link

Cited by:
Appendix B
,

§E.1
,

§3.1
.

Schiele
et al.
(2023)

P. Schiele, E. S. Luxenberg, and S. P. Boyd

Disciplined Saddle Programming
.

Transactions on Machine Learning Research
(
en
).

External Links:
ISSN 2835-8856
,

Link

Cited by:
§3
.

Shafer and Vovk (2008)

G. Shafer and V. Vovk

A Tutorial on Conformal Prediction
.

Journal of Machine Learning Research

9
(
12
),
pp. 371–421
.

External Links:
Link

Cited by:
Appendix B
,

item 3
.

Shapiro
et al.
(2009)

A. Shapiro, D. Dentcheva, and A. Ruszczyński

Lectures on Stochastic Programming: Modeling and Theory
.

Society for Industrial and Applied Mathematics
(
en
).

External Links:
ISBN 978-0-89871-687-0 978-0-89871-875-1
,

Link
,

Document

Cited by:
Appendix B
.

Stutz
et al.
(2022)

D. Stutz, Krishnamurthy, Dvijotham, A. T. Cemgil, and A. Doucet

Learning Optimal Conformal Classifiers
.

In
International Conference on Learning Representations
,

External Links:
Link

Cited by:
Appendix B
,

Appendix D
,

item 3
,

§3.3
.

Sun
et al.
(2023)

C. Sun, L. Liu, and X. Li

Predict-then-Calibrate: A New Perspective of Robust Contextual LP
.

In
Advances in Neural Information Processing Systems
,

Vol.
36
,
pp. 17713–17741
(
en
).

External Links:
Link

Cited by:
§2
,

§3.1
,

§3.3
,

2nd item
,

§6
.

Uryasev and Rockafellar (2001)

S. Uryasev and R. T. Rockafellar

Conditional Value-at-Risk: Optimization Approach
.

In
Stochastic Optimization: Algorithms and Applications
,
S. Uryasev and P. M. Pardalos (Eds.)
,

pp. 411–435
(
en
).

External Links:
ISBN 978-1-4757-6594-6
,

Link
,

Document

Cited by:
§A.1
.

Wang
et al.
(2024)

I. Wang, C. Becker, B. Van Parys, and B. Stellato

Learning Decision-Focused Uncertainty Sets in Robust Optimization
.

arXiv
.

Note:
arXiv:2305.19225 [math]

External Links:
Link
,

Document

Cited by:
§1
,

§2
,

footnote 2
.

Warrington
et al.
(2012)

J. Warrington, P. J. Goulart, S. Mariéthoz, and M. Morari

Robust reserve operation in power systems using affine policies
.

In
2012 IEEE 51st IEEE Conference on Decision and Control (CDC)
,

pp. 1111–1117
.

External Links:
ISSN 0743-1546
,

Link
,

Document

Cited by:
Appendix B
.

Wilder
et al.
(2019)

B. Wilder, B. Dilkina, and M. Tambe

Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization
.

Proceedings of the AAAI Conference on Artificial Intelligence

33
(
01
),
pp. 1658–1665
(
en
).

Note:
Number: 01

External Links:
ISSN 2374-3468
,

Link
,

Document

Cited by:
Appendix B
.

Wilson
et al.
(2016)

A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing

Deep Kernel Learning
.

In
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS)
,
A. Gretton and C. C. Robert (Eds.)
,

Cadiz, Spain
,
pp. 370–378
.

External Links:
ISSN 1938-7228
,

Link

Cited by:
Appendix B
.

Yan
et al.
(2020)

X. Yan, C. Gu, X. Zhang, and F. Li

Robust Optimization-Based Energy Storage Operation for System Congestion Management
.

IEEE Systems Journal

14
(
2
),
pp. 2694–2702
(
en
).

External Links:
ISSN 1932-8184, 1937-9234, 2373-7816
,

Link
,

Document

Cited by:
§1
.

Zheng
et al.
(2015)

Q. P. Zheng, J. Wang, and A. L. Liu

Stochastic Optimization for Unit Commitment—A Review
.

IEEE Transactions on Power Systems

30
(
4
),
pp. 1913–1924
.

Note:
Conference Name: IEEE Transactions on Power Systems

External Links:
ISSN 1558-0679
,

Link
,

Document

Cited by:
Appendix B
.

Zhong
et al.
(2021)

W. Zhong, K. Xie, Y. Liu, S. Xie, and L. Xie

Chance Constrained Scheduling and Pricing for Multi-Service Battery Energy Storage
.

IEEE Transactions on Smart Grid

12
(
6
),
pp. 5030–5042
.

Note:
Conference Name: IEEE Transactions on Smart Grid

External Links:
ISSN 1949-3061
,

Link
,

Document

Cited by:
Appendix B
.

Appendix A
Appendix: Additional experimental results

A.1
Experimental results: Battery storage

In
Figures

A1
and
A2
, we plot the
(
1
−
α
)
(1-\alpha)
-value-at-risk (VaR) and
(
1
−
α
)
(1-\alpha)
-conditional value-at-risk (CVaR)
(
Uryasev and Rockafellar, 2001
)
of the task loss obtained via our method on the battery storage problem to evaluate model robustness. In particular, for each target coverage level
1
−
α
1-\alpha
, we evaluate
VaR
1
−
α
​
[
task loss
]
\mathrm{VaR}^{1-\alpha}[\text{task loss}]
and
CVaR
1
−
α
​
[
task loss
]
\mathrm{CVaR}^{1-\alpha}[\text{task loss}]
for both the ETO baseline(s) and our E2E methodologies across box, ellipsoid, and PICNN uncertainty sets. In general, it is clear that our E2E methodology uniformly improves over ETO, and while the performance of E2E ellipsoid is close to that of E2E PICNN, the latter appears to perform better for certain
α
\alpha
. These results highlight that our methodology improves not only the average task loss, but robustness more generally, even when compared with the robust ETO baselines.

The optimal task losses shown in black dotted lines in
Figures

3
,
A1
and
A2
are the lowest achievable task loss on the test set given perfect knowledge of the target
y
y
. The optimal task loss is calculated for each example
(
x
,
y
)
(x,y)
in the test set as
f
⁡
(
x
,
y
,
z
opt
⋆
)
f(x,y,z_{\text{opt}}^{\star})
where

z
opt
⋆
=
arg
​
min
z
∈
ℝ
p
⁡
f
​
(
x
,
y
,
z
)
​
s.t.
​
g
​
(
x
,
z
)
≤
0
.
z_{\text{opt}}^{\star}\ =\ \argmin_{z\in\mathbb{R}^{p}}f(x,y,z)\ \text{s.t.}\ g(x,z)\leq 0.

Performance.

In
Table

A1
, we report the wall-clock time for pretraining, optimization, and end-to-end training. These times were measured on a machine with 2
×
\times
AMD EPYC 7513 32-Core Processors, 1TiB RAM, and 4 NVIDIA A100 GPUs (although only 1 of the GPUs was used in these experiments). The “pretrain” column gives the time per epoch of training with the standard loss function (pinball loss for Box, negative log-likelihood loss for Ellipse and PICNN). The “optimize” column gives the time needed to compute the decision
z
θ
⋆
​
(
x
)
z^{\star}_{\theta}(x)
given a pretrained model. The “E2E” column gives the time per epoch of E2E training, which requires computing
z
θ
⋆
​
(
x
)
z^{\star}_{\theta}(x)
for each training and calibration example. Evidently, the bulk of the time spent on E2E training comes from computing the decision
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
; the additional overhead from computing the gradient through the KKT conditions of the optimization problem is much less than solving the optimization problem itself.

We note that in theory, E2E times should always be higher than the optimization time. The optimization time accounts for the time required to compute
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
, and E2E training additional accounts for the time required to compute the gradient
∂
∂
θ
​
z
θ
⋆
​
(
x
)
\frac{\partial}{\partial\theta}z_{\theta}^{\star}(x)
. However, in practice, the optimization time depends heavily on the choice of numerical solver. In our implementation, we use the default
cvxpy
solver (Clarabel) for the optimization step in ETO, whereas we use the default
cvxpylayers
solver (SCS) during E2E training. In the case of the Ellipse and PICNN uncertainty sets, the SCS solver is generally faster than Clarabel; hence,
Table

A1
counterintuitively reports lower E2E training times for Ellipse and PICNN than optimization times.

We further note that the optimization and E2E training times reflect our particular implementation, but significantly faster times are likely possible. In particular, we solve all of the optimization problems (in both the ETO optimization step and during E2E training) using CPU-based solvers; however, recently developed GPU-accelerated solvers may be able to reduce the optimization time required by orders of magnitude, especially when solving a batch of problems with the same structure
(
Blin, 2024
)
.

no
distribution
shift

with
distribution
shift

Figure A1
:
Similar to
Figure

3
, except that the value plotted is the VaR
[
task loss
]
1
−
α
{}^{1-\alpha}[\text{task loss}]
(mean
±
\pm
1 stddev across 10 runs) on the test set for the battery storage problem with no distribution shift (top) and with distribution shift (bottom). Lower values are better.

no
distribution
shift

with
distribution
shift

Figure A2
:
Similar to
Figure

3
, except that the value plotted is the CVaR
[
task loss
]
1
−
α
{}^{1-\alpha}[\text{task loss}]
(mean
±
\pm
1 stddev across 10 runs) on the test set for the battery storage problem with no distribution shift (top) and with distribution shift (bottom). Lower values are better.

Table A1
:
Time per epoch of pretraining, optimization, and end-to-end training, measured in seconds (wall-clock time). Values are mean
±
\pm
1 stddev across 10 epochs. Lower values are better.

pretrain

optimize (train+val)

E2E training

Box

0.03
±
\pm
0.01

4.29
±
\pm
0.07

6.34
±
\pm
0.32

Ellipse

0.03
±
\pm
0.01

7.79
±
\pm
0.06

7.12
±
\pm
1.28

PICNN

4.20
±
\pm
0.20

50.49
±
\pm
0.19

33.24
±
\pm
0.25

A.2
Experimental results: Portfolio optimization

Tables

A2
and
A3
show the task loss and coverage results for the portfolio optimization problem. We again find that our E2E approach generally improves upon the ETO baselines at all uncertainty levels
α
\alpha
, with the exception of box uncertainty, where all the methods achieve similar performance. Our PICNN-based uncertainty representation, when learned end-to-end, performs better than box uncertainty and comparably with ellipse uncertainty. The similarity in performance between E2E ellipsoidal uncertainty and E2E PICNN uncertainty is likely due to the underlying aleatoric uncertainty (i.e., the uncertainty in
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
) generally taking an ellipsoidal shape—the conditional distribution
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
is a Gaussian mixture model, and it tends to have a dominant mode (e.g., see
Figure

A3
). In terms of coverage, we find that all the models and training methodologies obtain coverage very close to the target level, confirming that the improvements in task loss performance from our E2E approach do not come at the cost of worse coverage.

Because the conditional distribution
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
for the portfolio optimization problem is 2-dimensional, we can visualize the conditional distribution as well as the uncertainty sets estimated by our models.
Figure

A3
plots the conditional density for input
x
=
[
−
1.167
0.024
]
⊤
x=\begin{bmatrix}-1.167&0.024\end{bmatrix}^{\top}
, along with the
α
=
0.1
\alpha=0.1
uncertainty sets
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
and the resulting decision vectors
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
for each uncertainty set parametrization. Uncertainty sets and decision vectors from both
ETO
and E2E models are shown in different colors. The key takeaway from this figure is that smaller uncertainty sets (which is what ETO training tends to produce) do not always result in lower task loss. Furthermore, the more flexible parametrization of the PICNN allows it to learn uncertainty set shapes that may be more amenable to the downstream robust decision task than box or ellipsoidal uncertainty, even if the resulting uncertainty set has a larger or odder shape.

Table A2
:
Task loss performance (mean
±
\pm
1 stddev across 10 runs) for the portfolio optimization problem. Lower values are better, and the best performance for each uncertainty-level
α
\alpha
is highlighted. The results show that our E2E methods consistently outperform the ETO baselines.

uncertainty level
α
\alpha

0.01

0.05

0.1

0.2

ETO

Box

-1.16
±
\pm
0.42

-1.37
±
\pm
0.12

-1.39
±
\pm
0.13

-1.41
±
\pm
0.12

ETO

Ellipse

-1.09
±
\pm
0.12

-1.24
±
\pm
0.11

-1.29
±
\pm
0.10

-1.33
±
\pm
0.10

ETO

PICNN

-0.95
±
\pm
0.24

-1.11
±
\pm
0.24

-1.20
±
\pm
0.22

-1.31
±
\pm
0.16

ETO-SLL

Box

-1.41
±
\pm
0.13

-1.42
±
\pm
0.12

-1.42
±
\pm
0.12

-1.44
±
\pm
0.11

ETO-SLL

Ellipse

-1.12
±
\pm
0.22

-1.37
±
\pm
0.12

-1.40
±
\pm
0.12

-1.43
±
\pm
0.12

ETO-JC

Ellipse

-1.16
±
\pm
0.17

-1.40
±
\pm
0.11

-1.42
±
\pm
0.11

-1.44
±
\pm
0.11

E2E

Box

-1.21
±
\pm
0.44

-1.40
±
\pm
0.14

-1.43
±
\pm
0.11

-1.43
±
\pm
0.10

E2E

Ellipse

-1.48

±
\pm
0.12

-1.47
±
\pm
0.11

-1.48

±
\pm
0.11

-1.47

±
\pm
0.11

E2E

PICNN

-1.45
±
\pm
0.14

-1.48

±
\pm
0.10

-1.48

±
\pm
0.10

-1.47

±
\pm
0.11

Table A3
:
Coverage (mean
±
\pm
1 stddev across 10 runs) for the portfolio optimization problem. Our E2E models achieve similar coverage to the ETO baselines, confirming that the lower task loss of our E2E models does not come at the expense of worse coverage.

uncertainty level
α
\alpha

0.01

0.05

0.1

0.2

ETO

Box

.984
±
\pm
.007

.947
±
\pm
.017

.902
±
\pm
.017

.786
±
\pm
.020

ETO

Ellipse

.988
±
\pm
.004

.944
±
\pm
.020

.894
±
\pm
.022

.794
±
\pm
.027

ETO

PICNN

.989
±
\pm
.006

.949
±
\pm
.014

.901
±
\pm
.019

.801
±
\pm
.034

ETO-SLL

Box

.985
±
\pm
.012

.945
±
\pm
.021

.885
±
\pm
.030

.796
±
\pm
.029

ETO-SLL

Ellipse

.989
±
\pm
.011

.945
±
\pm
.024

.885
±
\pm
.039

.795
±
\pm
.030

ETO-JC

Ellipse

.991
±
\pm
.006

.953
±
\pm
.017

.902
±
\pm
.026

.796
±
\pm
.026

E2E

Box

.989
±
\pm
.006

.949
±
\pm
.012

.903
±
\pm
.016

.785
±
\pm
.019

E2E

Ellipse

.992
±
\pm
.006

.954
±
\pm
.010

.903
±
\pm
.022

.798
±
\pm
.022

E2E

PICNN

.993
±
\pm
.002

.953
±
\pm
.010

.912
±
\pm
.017

.798
±
\pm
.024

Figure A3
:
This figure plots the density of the conditional distribution
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
for
x
=
[
−
1.167
0.024
]
⊤
x=\begin{bmatrix}-1.167&0.024\end{bmatrix}^{\top}
from the portfolio optimization problem, with darker colors indicating higher density. Also plotted are the
α
=
0.1
\alpha=0.1
uncertainty sets
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
(dashed lines) and the resulting decision vectors
z
θ
⋆
​
(
x
)
z_{\theta}^{\star}(x)
(arrows) for each uncertainty set parametrization. Results for
ETO
models are shown in blue, whereas results for E2E are shown in orange. The “true”
y
y
sampled from
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)
is drawn in green, and the task loss for this example is computed using this
y
y
. The decision vectors have been artificially scaled larger to be easier to see.

Appendix B
Appendix: Related Work

“Task-based” or “decision-focused” learning.

The notion of “task-based” end-to-end model learning was introduced by
Donti et al. (2017)
, which proposed to train machine learning models in an end-to-end fashion to minimize a downstream stochastic optimization objective. To achieve this, the authors backpropagate gradients through a stochastic optimization problem, which is made possible for various types of convex optimization problems via the implicit function theorem
(
Donti et al., 2017
;
Amos and Kolter, 2017
;
Agrawal et al., 2019
)
. However,
Donti et al. (2017)
does not train the model to estimate uncertainty and thereby does not provide any explicit guarantees on robustness on their decisions to uncertainty. Our framework improves upon this baseline by yielding calibrated uncertainty sets which can then be used to obtain robust decisions.

Another line of related works relies on surrogate loss functions to approximate the gradient of the downstream task loss, typically in settings where the exact gradient does not exist or is otherwise hard to compute. For example, the “Smart Predict, then Optimize” approach
(
Elmachtoub and Grigas, 2022
)
specifically considers differentiating the solution of linear programs, whereas
Wilder et al. (2019)
differentiates the solution of discrete (combinatorial) optimization problems. As with
Donti et al. (2017)
, though, these approaches focus only on average task loss without any estimation of model uncertainty, thereby lacking explicit guarantees on the robustness of the resulting decisions. A recent survey by
Mandi et al. (2024)
provides a comprehensive review of existing decision-focused learning approaches.

Uncertainty Quantification.

Various designs for deep learning regression models that provide uncertainty estimates have been proposed in the literature, including Bayesian neural networks
(
Blundell et al., 2015
;
Gal and Ghahramani, 2016
)
, Gaussian process regression and deep kernel learning
(
Rasmussen and Williams, 2005
;
Wilson et al., 2016
;
Liu et al., 2020
)
, ensembles of models
(
Lakshminarayanan et al., 2017
)
, and quantile regression
(
Romano et al., 2019
)
, among other techniques. These methods typically only provide heuristic uncertainty estimates that are not necessarily well-calibrated
(
Nado et al., 2022
)
.

Post-hoc methods such as isotonic regression
(
Kuleshov et al., 2018
)
or conformal prediction
(
Shafer and Vovk, 2008
)
may be used to calibrate the uncertainty outputs of deep learning models. These calibration methods generally treat the model as a black box and scale predicted uncertainty levels so that they are calibrated on a held-out calibration set. Isotonic regression guarantees calibrated outputs in the limit of infinite data, whereas conformal methods provide probabilistic, finite-sample calibration guarantees when the calibration set is exchangeable (e.g., drawn i.i.d. from the same distribution) with test data. These calibration methods are generally not included in the model training procedure because they involve non-differentiable operators, such as sorting. However, recent works have proposed differentiable losses
(
Einbinder et al., 2022
;
Stutz et al., 2022
)
that approximate the conformal prediction procedure during training and thus allow end-to-end training of models to output more calibrated uncertainty. As approximations, these methods lose the marginal coverage guarantees that true conformal methods provide. However, such guarantees can be recovered at test time by replacing the approximations with true conformal prediction.

Robust and stochastic optimization.

The optimization community has proposed a number of techniques over the years to improve robust decision-making under uncertainty, including stochastic, risk-sensitive, chance-constrained, distributionally robust, and robust optimization (e.g.,
Ben-Tal et al. (2009)
;
Shapiro et al. (2009)
;
Nemirovski and Shapiro (2007)
;
Rahimian and Mehrotra (2019)
). These techniques have been applied to a wide range of applications, including energy systems operation

(
Zheng et al., 2015
;
Ndrio et al., 2021
;
Dvorkin, 2020
;
Zhong et al., 2021
;
Poolla et al., 2021
;
Warrington et al., 2012
;
Bertsimas et al., 2013
;
Christianson et al., 2022
)
and portfolio optimization
(
Gregory et al., 2011
;
Quaranta and Zaffaroni, 2008
;
Bertsimas et al., 2018
)
. In these works, the robust and stochastic optimization methods enable selecting decisions (grid resource dispatches or portfolio allocations) in a manner that is aware of uncertainty, e.g., so an energy system operator can ensure that sufficient generation is available to meet demand even on a cloudy day without much solar generation. Typically, however, the construction of uncertainty sets, estimated probability distributions over uncertain parameters, or ambiguity sets over distributions takes place offline and is unconnected to the eventual decision-making task. Thus, our proposed end-to-end approach allows for simultaneous calibration of uncertainty sets with optimal decision-making.

Appendix C
Appendix: Maximizing over the uncertainty set

We consider robust optimization problems of the form

min
z
∈
ℝ
p
⁡
max
y
^
∈
ℝ
n
​
y
^
⊤
​
F
​
z
+
f
~
​
(
x
,
z
)
s.t.
y
^
∈
Ω
⁡
(
x
)
,
g
⁡
(
x
,
z
)
≤
0
.
\min_{z\in\mathbb{R}^{p}}\max_{\hat{y}\in\mathbb{R}^{n}}\ \hat{y}^{\top}Fz+\tilde{f}(x,z)\qquad\text{s.t.}\qquad\hat{y}\in\Omega(x),\quad g(x,z)\leq 0.

For fixed
z
z
, the inner maximization problem is

max
y
^
∈
ℝ
n
⁡
y
^
⊤
​
F
​
z
s.t.
y
^
∈
Ω
⁡
(
x
)
,
\max_{\hat{y}\in\mathbb{R}^{n}}\ \hat{y}^{\top}Fz\qquad\text{s.t.}\qquad\hat{y}\in\Omega(x),

which we analyze in the more abstract form

max
y
∈
ℝ
n
⁡
c
⊤
​
y
s.t.
y
∈
Ω
\max_{y\in\mathbb{R}^{n}}c^{\top}y\qquad\text{s.t.}\qquad y\in\Omega

for arbitrary
c
∈
ℝ
n
∖
{
0
}
c\in\mathbb{R}^{n}\setminus\{0\}
. The subsections of this appendix derive the dual form of this maximization problem for specific representations of the uncertainty set
Ω
\Omega
.

Suppose
y
y
is standardized or whitened by an affine transformation with
μ
∈
ℝ
n
\mu\in\mathbb{R}^{n}
and invertible matrix
W
∈
ℝ
n
×
n
W\in\mathbb{R}^{n\times n}

y
transformed
=
W
−
1
​
(
y
−
μ
)
y_{\text{transformed}}=W^{-1}(y-\mu)

so that
Ω
\Omega
is an uncertainty set on the transformed
y
transformed
y_{\text{transformed}}
. Then, the original primal objective can be recovered as

c
⊤
​
y
=
c
⊤
​
(
W
​
y
transformed
+
μ
)
=
(
W
​
c
)
⊤
​
y
transformed
+
c
⊤
​
μ
.
c^{\top}y=c^{\top}(Wy_{\text{transformed}}+\mu)=(Wc)^{\top}y_{\text{transformed}}+c^{\top}\mu.

In our experiments, we use element-wise standardization of
y
y
by setting
W
=
diag
⁡
(
y
std
)
W=\diag(y_{\text{std}})
, where
y
std
∈
ℝ
n
y_{\text{std}}\in\mathbb{R}^{n}
is the element-wise standard-deviation of
y
y
.

C.1
Maximizing over a box constraint

Let
[
y
¯
,
y
¯
]
⊂
ℝ
n
[\underline{y},\overline{y}]\subset\mathbb{R}^{n}
be a box uncertainty set for
y
∈
ℝ
n
y\in\mathbb{R}^{n}
. Then, for any vector
c
∈
ℝ
n
c\in\mathbb{R}^{n}
, the primal linear program

max
y
∈
ℝ
n
⁡
c
⊤
​
y
s.t.
y
¯
≤
y
≤
y
¯
\max_{y\in\mathbb{R}^{n}}\ c^{\top}y\qquad\text{s.t.}\qquad\underline{y}\leq y\leq\overline{y}

has dual problem

min
ν
∈
ℝ
2
​
n
⁡
[
y
¯
⊤
−
y
¯
⊤
]
​
ν
s.t.
[
I
n
−
I
n
]
​
ν
=
c
,
ν
≥
𝟎
,
\min_{\nu\in\mathbb{R}^{2n}}\ \begin{bmatrix}\overline{y}^{\top}&-\underline{y}^{\top}\end{bmatrix}\nu\qquad\text{s.t.}\qquad\begin{bmatrix}I_{n}&-I_{n}\end{bmatrix}\nu=c,\quad\nu\geq\mathbf{0},

which can also be equivalently written as

min
ν
∈
ℝ
n
⁡
(
y
¯
−
y
¯
)
⊤
​
ν
+
y
¯
⊤
​
c
s.t.
ν
≥
𝟎
,
ν
−
c
≥
𝟎
.
\min_{\nu\in\mathbb{R}^{n}}\ (\overline{y}-\underline{y})^{\top}\nu+\underline{y}^{\top}c\qquad\text{s.t.}\qquad\nu\geq\mathbf{0},\quad\nu-c\geq\mathbf{0}.

Since strong duality always holds for linear programs, the optimal values of the primal and dual problems will be equal so long as one of the problems is feasible, e.g., so long as the box
[
y
¯
,
y
¯
]
[\underline{y},\overline{y}]
is nonempty. We can thus incorporate this dual problem into the outer minimization of (
1
) to yield the non-robust form (
4
).

C.2
Maximizing over an ellipsoid

For any
c
∈
ℝ
n
∖
{
0
}
c\in\mathbb{R}^{n}\setminus\{0\}
,
Σ
∈
𝕊
+
⁣
+
n
\Sigma\in\mathbb{S}_{++}^{n}
, and
q
>
0
q>0
, the primal quadratically constrained linear program (QCLP)

max
y
∈
ℝ
n
⁡
c
⊤
​
y
s.t.
(
y
−
μ
)
⊤
​
Σ
−
1
​
(
y
−
μ
)
≤
q
\max_{y\in\mathbb{R}^{n}}\ c^{\top}y\qquad\text{s.t.}\qquad(y-\mu)^{\top}\Sigma^{-1}(y-\mu)\leq q

has dual problem

min
ν
∈
ℝ
⁡
1
4
​
ν
​
c
⊤
​
Σ
​
c
+
μ
⊤
​
c
+
ν
​
q
s.t.
ν
≥
0
.
\min_{\nu\in\mathbb{R}}\ \frac{1}{4\nu}c^{\top}\Sigma c+\mu^{\top}c+\nu q\qquad\text{s.t.}\quad\nu\geq 0.

By Slater’s condition, strong duality holds by virtue of the assumption that
q
>
0
q>0
(which implies strict feasibility of the primal problem), and thus the primal and dual problems have the same optimal value. Moreover, since
Σ
\Sigma
is positive definite and
q
>
0
q>0
, this problem has a unique optimal solution at
ν
⋆
=
1
2
​
q
​
‖
L
⊤
​
c
‖
2
\nu^{\star}=\frac{1}{2\sqrt{q}}\|L^{\top}c\|_{2}
, where
L
L
is the unique lower-triangular Cholesky factor of
Σ
\Sigma
(i.e.,
Σ
=
L
​
L
⊤
\Sigma=LL^{\top}
). Substituting
ν
⋆
\nu^{\star}
into the dual problem yields

q
​
‖
L
⊤
​
c
‖
2
+
μ
⊤
​
c
.
\sqrt{q}\left\lVert L^{\top}c\right\rVert_{2}+\mu^{\top}c.

Plugging this into (
1
) yields the non-robust form (
5
).

We write the dual objective in terms of the Cholesky factor
L
L
because our predictive models for ellipsoidal uncertainty directly output the entries of
L
L
(see
Appendix

E
). Note, however, that the dual problem solution can be equivalently written in terms of the square-root of
Σ
\Sigma
, because

‖
L
⊤
​
c
‖
2
2
=
c
⊤
​
L
​
L
⊤
​
c
=
c
⊤
​
Σ
​
c
=
c
⊤
​
Σ
1
/
2
​
Σ
1
/
2
​
c
=
‖
Σ
1
/
2
​
c
‖
2
2
.
\left\lVert L^{\top}c\right\rVert_{2}^{2}=c^{\top}LL^{\top}c=c^{\top}\Sigma c=c^{\top}\Sigma^{1/2}\Sigma^{1/2}c=\left\lVert\Sigma^{1/2}c\right\rVert_{2}^{2}.

C.3
Proof of Theorem
4.1
: Maximizing over the sublevel set of a PICNN

Let
s
θ
:
ℝ
m
×
ℝ
n
→
ℝ
s_{\theta}:\mathbb{R}^{m}\times\mathbb{R}^{n}\to\mathbb{R}
be a partially input-convex neural network (PICNN) with ReLU activations as described in (
7
), so that
s
θ
​
(
x
,
y
)
s_{\theta}(x,y)
is convex in
y
y
. Suppose that all the hidden layers have the same dimension
d
d
(i.e.,
∀
l
=
0
,
…
,
L
−
1
\forall l=0,\dotsc,L-1
:
W
l
∈
ℝ
d
×
d
W_{l}\in\mathbb{R}^{d\times d}
,
V
l
∈
ℝ
d
×
n
V_{l}\in\mathbb{R}^{d\times n}
,
b
l
∈
ℝ
d
b_{l}\in\mathbb{R}^{d}
), and the final layer
L
L
has
W
L
∈
ℝ
1
×
d
W_{L}\in\mathbb{R}^{1\times d}
,
V
L
∈
ℝ
1
×
n
V_{L}\in\mathbb{R}^{1\times n}
,
b
L
∈
ℝ
b_{L}\in\mathbb{R}
. Let
c
∈
ℝ
n
c\in\mathbb{R}^{n}
be any vector. Then, the optimization problem

max
y
∈
ℝ
n
⁡
c
⊤
​
y
s.t.
s
θ
​
(
x
,
y
)
≤
q
\max_{y\in\mathbb{R}^{n}}\ c^{\top}y\qquad\text{s.t.}\qquad s_{\theta}(x,y)\leq q

(9)

can be equivalently written as

max
y
∈
ℝ
n
,
σ
1
,
…
,
σ
L
∈
ℝ
d
\displaystyle\max_{y\in\mathbb{R}^{n},\ \sigma_{1},\dotsc,\sigma_{L}\in\mathbb{R}^{d}}\quad

c
⊤
​
y
\displaystyle c^{\top}y

(10a)

s.t.

σ
l
≥
𝟎
d
\displaystyle\sigma_{l}\geq\mathbf{0}_{d}

∀
l
=
1
,
…
,
L
\displaystyle\forall l=1,\dotsc,L

(10b)

σ
l
+
1
≥
W
l
​
σ
l
+
V
l
​
y
+
b
l
\displaystyle\sigma_{l+1}\geq W_{l}\sigma_{l}+V_{l}y+b_{l}

∀
l
=
0
,
…
,
L
−
1
\displaystyle\forall l=0,\dotsc,L-1

(10c)

W
L
​
σ
L
+
V
L
​
y
+
b
L
≤
q
.
\displaystyle W_{L}\sigma_{L}+V_{L}y+b_{L}\leq q.

(10d)

To see that this is the case, first note that (
10
) is a relaxed form of (
9
), obtained by replacing the equalities
σ
l
+
1
=
ReLU
⁡
(
W
l
​
σ
l
+
V
l
​
y
+
b
l
)
\sigma_{l+1}=\ReLU\left(W_{l}\sigma_{l}+V_{l}y+b_{l}\right)
in the definition of the PICNN (
7
) with the two separate inequalities
σ
l
+
1
≥
𝟎
d
\sigma_{l+1}\geq\mathbf{0}_{d}
and
σ
l
+
1
≥
W
l
​
σ
l
+
V
l
​
y
+
b
l
\sigma_{l+1}\geq W_{l}\sigma_{l}+V_{l}y+b_{l}
for each
l
=
0
,
…
,
L
−
1
l=0,\ldots,L-1
. As such, the optimal value of (
10
) is no less than that of (
9
). However, given an optimal solution
y
,
σ
1
,
…
,
σ
L
y,\sigma_{1},\ldots,\sigma_{L}
to (
10
), it is possible to obtain another feasible solution
y
,
σ
^
1
,
…
,
σ
^
L
y,\hat{\sigma}_{1},\ldots,\hat{\sigma}_{L}
with the same optimal objective value by iteratively decreasing each component of
σ
l
\sigma_{l}
until one of the two inequality constraints (
10b
), (
10c
) is tight, beginning at
l
=
1
l=1
and incrementing
l
l
once all entries of
σ
l
\sigma_{l}
cannot be decreased further. This procedure of decreasing the entries in each
σ
l
\sigma_{l}
will maintain problem feasibility, since the weight matrices
W
l
W_{l}
are all assumed to be entrywise nonnegative in the PICNN construction; in particular, this procedure will not increase the left-hand side of (
10d
). Moreover, since one of the two constraints (
10b
), (
10c
) will hold for each entry of each
σ
^
l
\hat{\sigma}_{l}
, this immediately implies that
y
y
is feasible for the unrelaxed problem (
9
), and so (
9
) and (
10
) must have the same optimal value.

Having shown that we may replace the convex program (
9
) with a linear equivalent (
10
), we can write this latter problem in the matrix form

max
y
∈
ℝ
n
,
σ
1
,
…
,
σ
L
∈
ℝ
d
⁡
c
⊤
​
y
s.t.
A
​
[
y
σ
1
σ
L
]
≤
b
\max_{y\in\mathbb{R}^{n},\ \sigma_{1},\dotsc,\sigma_{L}\in\mathbb{R}^{d}}\ c^{\top}y\qquad\text{s.t.}\qquad A\begin{bmatrix}y\\
\sigma_{1}\\
\vdots\\
\sigma_{L}\end{bmatrix}\leq b

where

A
=
[
−
I
d
⋱
−
I
d
V
0
−
I
d
W
1
⋱
⋱
−
I
d
V
L
W
L
]
∈
ℝ
(
2
​
L
​
d
+
1
)
×
(
n
+
L
​
d
)
,
b
=
[
𝟎
d
𝟎
d
−
b
0
−
b
L
−
1
q
−
b
L
]
∈
ℝ
2
​
L
​
d
+
1
.
A=\begin{bmatrix}&-I_{d}\\
&&\ddots\\
&&&-I_{d}\\
V_{0}&-I_{d}\\
\vdots&W_{1}&\ddots\\
\vdots&&\ddots&-I_{d}\\
V_{L}&&&W_{L}\end{bmatrix}\in\mathbb{R}^{(2Ld+1)\times(n+Ld)},\qquad b=\begin{bmatrix}\mathbf{0}_{d}\\
\vdots\\
\mathbf{0}_{d}\\
-b_{0}\\
\vdots\\
-b_{L-1}\\
q-b_{L}\end{bmatrix}\in\mathbb{R}^{2Ld+1}.

(11)

By strong duality, if this linear program has an optimal solution, its optimal value is equal to the optimal value of its dual problem:

min
ν
∈
ℝ
2
​
L
​
d
+
1
⁡
b
⊤
​
ν
s.t.
A
⊤
​
ν
=
[
c
𝟎
L
​
d
]
,
ν
≥
0
.
\min_{\nu\in\mathbb{R}^{2Ld+1}}b^{\top}\nu\qquad\text{s.t.}\qquad A^{\top}\nu=\begin{bmatrix}c\\
\mathbf{0}_{Ld}\end{bmatrix},\quad\nu\geq 0.

(12)

We can incorporate this dual problem (
12
) into the outer minimization of (
1
) to yield the non-robust form (
8
). For a more interpretable form of this dual problem, let
ν
(
i
)
\nu^{(i)}
denote the portion of the dual vector
ν
\nu
corresponding to the
i
i
-th block-row of matrix
A
A
, indexed from
0
0
. That is,
ν
(
i
)
=
ν
i
​
d
+
1
:
(
i
+
1
)
​
d
\nu^{(i)}=\nu_{id+1:(i+1)d}
for
i
=
0
,
…
,
2
​
L
−
1
i=0,\dotsc,2L-1
. Furthermore, let
μ
=
ν
2
​
L
​
d
+
1
\mu=\nu_{2Ld+1}
be the last entry of
ν
\nu
. Written out, the dual problem (
12
) becomes

min
ν
(
0
)
,
…
,
ν
(
2
​
L
−
1
)
∈
ℝ
d
,
μ
∈
ℝ
\displaystyle\min_{\nu^{(0)},\dotsc,\nu^{(2L-1)}\in\mathbb{R}^{d},\,\mu\in\mathbb{R}}\quad

μ
⁡
(
q
−
b
L
)
−
∑
l
=
0
L
b
l
⊤
​
ν
(
L
+
l
)
\displaystyle\mu(q-b_{L})-\sum_{l=0}^{L}b_{l}^{\top}\nu^{(L+l)}

s.t.

[
V
0
⊤
⋯
V
L
⊤
]
ν
L
​
d
+
1
:
=
c
\displaystyle\begin{bmatrix}V_{0}^{\top}&\dotsb&V_{L}^{\top}\end{bmatrix}\nu_{Ld+1:}=c

W
l
+
1
⊤
​
ν
(
L
+
l
+
1
)
−
ν
(
L
+
l
)
−
ν
(
l
)
=
𝟎
d
\displaystyle W_{l+1}^{\top}\nu^{(L+l+1)}-\nu^{(L+l)}-\nu^{(l)}=\mathbf{0}_{d}

∀
l
=
0
,
…
,
L
−
1
\displaystyle\forall l=0,\dotsc,L-1

ν
≥
0
.
\displaystyle\nu\geq 0.

C.4
Ensuring feasibility of the PICNN maximization problem

As noted at the end of
Section

4
, it may sometimes be the case that the inner maximization problem of (
1
) is unbounded or infeasible when
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is parametrized by a PICNN, since in general, the sublevel sets of the PICNN might be unbounded, or the
q
q
selected by the split conformal procedure detailed in
Section

3.2
may be sufficiently small that
Ω
θ
​
(
x
)
=
{
y
^
∈
ℝ
n
∣
s
θ
​
(
x
,
y
^
)
≤
q
}
\Omega_{\theta}(x)=\{\hat{y}\in\mathbb{R}^{n}\mid s_{\theta}(x,\hat{y})\leq q\}
is empty for certain inputs
x
x
. We can address each of these concerns using separate techniques.

Ensuring compact sublevel sets.

To ensure that the PICNN-parametrized score function
s
θ
​
(
x
,
y
)
s_{\theta}(x,y)
has compact sublevel sets in
y
y
, we can redefine the output layer by setting
V
L
=
𝟎
1
×
n
V_{L}=\mathbf{0}_{1\times n}
and adding a small
ℓ
∞
\ell^{\infty}
norm term penalizing growth in
y
y
:

s
θ
​
(
x
,
y
)
=
W
L
​
σ
L
+
ϵ
​
‖
y
‖
∞
+
b
L
,
s_{\theta}(x,y)=W_{L}\sigma_{L}+\epsilon\|y\|_{\infty}+b_{L},

(13)

where
ϵ
≥
0
\epsilon\geq 0
is a small penalty term, and where all the remaining parameters and layers remain identical to their definition in (
7
). This modification ensures that, for any fixed
x
x
,
s
θ
​
(
x
,
y
)
s_{\theta}(x,y)
has compact sublevel sets, since
σ
L
≥
0
\sigma_{L}\geq 0
by construction (
7
) and the penalty term
ϵ
​
‖
y
‖
∞
\epsilon\|y\|_{\infty}
will grow unboundedly large as
y
y
goes to infinity in any direction, so long as
ϵ
>
0
\epsilon>0
. Moreover, so long as
ϵ
\epsilon
is chosen sufficiently small and the PICNN is sufficiently deep, this modification should not negatively impact the ability of the PICNN to represent general compact convex uncertainty sets.

Using this modified PICNN (
13
), the maximization problem (
9
) can be written as an equivalent linear program

max
y
∈
ℝ
n
,
σ
1
,
…
,
σ
L
∈
ℝ
d
,
κ
∈
ℝ
\displaystyle\max_{\begin{subarray}{c}y\in\mathbb{R}^{n},\ \sigma_{1},\dotsc,\sigma_{L}\in\mathbb{R}^{d},\\
\ \kappa\in\mathbb{R}\end{subarray}}\quad

c
⊤
​
y
\displaystyle c^{\top}y

(14a)

s.t.

σ
l
≥
𝟎
d
\displaystyle\sigma_{l}\geq\mathbf{0}_{d}

∀
l
=
1
,
…
,
L
\displaystyle\forall l=1,\dotsc,L

(14b)

σ
l
+
1
≥
W
l
​
σ
l
+
V
l
​
y
+
b
l
\displaystyle\sigma_{l+1}\geq W_{l}\sigma_{l}+V_{l}y+b_{l}

∀
l
=
0
,
…
,
L
−
1
\displaystyle\forall l=0,\dotsc,L-1

(14c)

κ
≥
y
i
,
κ
≥
−
y
i
\displaystyle\kappa\geq y_{i},\,\kappa\geq-y_{i}

∀
i
=
1
,
…
,
n
\displaystyle\forall i=1,\ldots,n

(14d)

W
L
​
σ
L
+
ϵ
​
κ
+
b
L
≤
q
\displaystyle W_{L}\sigma_{L}+\epsilon\kappa+b_{L}\leq q

(14e)

where the equivalence between (
9
) and (
14
) follows the same argument as that employed in the previous section when showing the equivalence of (
9
) and (
10
). We can thus likewise apply strong duality to obtain an equivalent minimization form of the problem (
14
) and incorporate this into the outer minimization of (
1
) to yield a non-robust problem of the general form (
8
).

Ensuring
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is nonempty.

If the
q
q
chosen by the split conformal procedure is too small such that
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is empty, i.e.,
q
≤
q
min
q\leq q_{\min}
where

q
min
:=
min
y
^
∈
ℝ
n
⁡
s
θ
​
(
x
,
y
^
)
,
q_{\min}:=\min_{\hat{y}\in\mathbb{R}^{n}}s_{\theta}(x,\hat{y}),

(15)

then we simply increase
q
q
to
q
min
q_{\min}
so that
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
is guaranteed to be nonempty. That is, for input
x
x
, we set

q
=
max
⁡
(
min
y
^
∈
ℝ
n
⁡
s
θ
​
(
x
,
y
^
)
,
Quantile
​
(
{
s
θ
​
(
x
i
,
y
i
)
}
(
x
i
,
y
i
)
∈
D
cal
,
1
−
α
)
)
.
q=\max\left(\min_{\hat{y}\in\mathbb{R}^{n}}s_{\theta}(x,\hat{y}),\ \textsc{Quantile}(\{s_{\theta}(x_{i},y_{i})\}_{(x_{i},y_{i})\in D_{\text{cal}}},\ 1-\alpha)\right).

This preserves the marginal coverage guarantee, as increasing
q
q
can only result in a larger uncertainty set
Ω
θ
​
(
x
)
\Omega_{\theta}(x)
.

In theory,
q
min
q_{\min}
varies as a function of
θ
\theta
, and it is possible to differentiate through the optimization problem (
15
) using the methods from
Agrawal et al. (2019)
since the problem is convex and
s
θ
s_{\theta}
is assumed to be differentiable w.r.t.
θ
\theta
almost everywhere. However, to avoid this added complexity, in practice, we treat
q
min
q_{\min}
as a constant. In other words, on inputs
x
x
where we have to increase
q
q
to
q
min
q_{\min}
, we treat
∂
q
∂
θ
=
0
\frac{\partial q}{\partial\theta}=0
.

Appendix D
Appendix: Exact differentiable conformal prediction

In this section, we prove how to exactly differentiate through the conformal prediction procedure, unlike the approximate derivative first introduced in
Stutz et al. (2022)
.

Theorem D.1
.

Let
α
∈
(
0
,
1
)
\alpha\in(0,1)
be a risk level, and let
s
i
:=
s
θ
​
(
x
i
,
y
i
)
s_{i}:=s_{\theta}(x_{i},y_{i})
denote the scores computed by a score function
s
θ
:
ℝ
m
×
ℝ
n
→
ℝ
s_{\theta}:\mathbb{R}^{m}\times\mathbb{R}^{n}\to\mathbb{R}
over data points
{
(
x
i
,
y
i
)
}
i
=
1
M
\{(x_{i},y_{i})\}_{i=1}^{M}
. Suppose
s
θ
​
(
x
i
,
y
i
)
s_{\theta}(x_{i},y_{i})
is differentiable w.r.t.
θ
\theta
for all
i
=
1
,
…
,
M
i=1,\dotsc,M
.

Define
s
M
+
1
:=
∞
s_{M+1}:=\infty
. Let
σ
:
{
1
,
…
,
M
+
1
}
→
{
1
,
…
,
M
+
1
}
\sigma:\{1,\dotsc,M+1\}\to\{1,\dotsc,M+1\}
denote the permutation that sorts the scores in ascending order, such that
s
σ
⁡
(
i
)
≤
s
σ
⁡
(
j
)
s_{\sigma(i)}\leq s_{\sigma(j)}
for all
i
<
j
i<j
. For simplicity of notation, we may write
s
(
i
)
:=
s
σ
⁡
(
i
)
s_{(i)}:=s_{\sigma(i)}
.

Let
q
=
Quantile
​
(
{
s
i
}
i
=
1
M
,
1
−
α
)
q=\textsc{Quantile}(\{s_{i}\}_{i=1}^{M},\ 1-\alpha)
where the
Quantile
function is as defined in
Algorithm

1
. That is,
q
=
s
(
k
)
q=s_{(k)}
, where
k
:=
⌈
(
M
+
1
)
​
(
1
−
α
)
⌉
∈
{
1
,
…
,
M
,
M
+
1
}
k:=\left\lceil(M+1)(1-\alpha)\right\rceil\in\{1,\dotsc,M,M+1\}
. If
s
(
k
)
s_{(k)}
is unique, then

d
​
q
d
​
θ
=
{
d
d
​
θ
​
s
θ
​
(
x
σ
⁡
(
k
)
,
y
σ
⁡
(
k
)
)
,
if
​
α
≥
1
M
+
1
0
,
otherwise
.
\frac{\mathrm{d}q}{\mathrm{d}\theta}=\begin{cases}\frac{\mathrm{d}}{\mathrm{d}\theta}s_{\theta}(x_{\sigma(k)},y_{\sigma(k)}),&\text{if }\alpha\geq\frac{1}{M+1}\\
0,&\text{otherwise}.\end{cases}

Proof.

First, when
α
∈
(
0
,
1
M
+
1
)
\alpha\in(0,\frac{1}{M+1})
, we have
k
=
M
+
1
k=M+1
, so
q
=
∞
q=\infty
is constant regardless of the choice of
θ
\theta
. Thus,
d
​
q
d
​
θ
=
0
\frac{\mathrm{d}q}{\mathrm{d}\theta}=0
.

Now, suppose
α
≥
1
M
+
1
\alpha\geq\frac{1}{M+1}
. The
Quantile
function returns the
k
k
-th largest value of
{
s
i
}
i
=
1
M
∪
{
∞
}
\{s_{i}\}_{i=1}^{M}\cup\{\infty\}
. Since we assume
s
(
k
)
s_{(k)}
is unique, we have
d
​
q
d
​
s
(
i
)
=
𝟏
[
i
=
k
]
\frac{\mathrm{d}q}{\mathrm{d}s_{(i)}}=\mathbf{1}[i=k]
. Finally, we have

d
​
q
d
​
θ
=
∑
i
=
1
M
d
​
q
d
​
s
i
​
d
​
s
i
d
​
θ
=
∑
i
=
1
M
d
​
q
d
​
s
(
i
)
​
d
​
s
(
i
)
d
​
θ
=
d
​
s
(
k
)
d
​
θ
=
d
d
​
θ
​
s
θ
​
(
x
σ
⁡
(
k
)
,
y
σ
⁡
(
k
)
)
.
\frac{\mathrm{d}q}{\mathrm{d}\theta}=\sum_{i=1}^{M}\frac{\mathrm{d}q}{\mathrm{d}s_{i}}\frac{\mathrm{d}s_{i}}{\mathrm{d}\theta}=\sum_{i=1}^{M}\frac{\mathrm{d}q}{\mathrm{d}s_{(i)}}\frac{\mathrm{d}s_{(i)}}{\mathrm{d}\theta}=\frac{\mathrm{d}s_{(k)}}{\mathrm{d}\theta}=\frac{\mathrm{d}}{\mathrm{d}\theta}s_{\theta}(x_{\sigma(k)},y_{\sigma(k)}).

∎

The two key assumptions in this theorem are that (1)
s
θ
s_{\theta}
is differentiable w.r.t.
θ
\theta
, and (2)
s
(
k
)
s_{(k)}
is unique. When
s
θ
s_{\theta}
is a neural network with a common activation function (e.g., ReLU), (1) holds for inputs
(
x
,
y
)
∈
ℝ
m
×
ℝ
n
(x,y)\in\mathbb{R}^{m}\times\mathbb{R}^{n}
almost everywhere and
θ
\theta
almost everywhere. Regarding (2), in practice, just as the gradient of the
max
\max
function is typically implemented without checking whether its inputs have ties, we do not check whether
s
(
k
)
s_{(k)}
is unique.

Appendix E
Appendix: Experiment details

Our experiments were conducted across a variety of machines, including private servers and Amazon AWS EC2 instances, ranging from 12-core to 128-core machines. Our ETO experiments benefited from GPU acceleration across a combination of NVIDIA GeForce GTX 1080 Ti, Titan RTX, T4, and A100 GPUs. Our E2E experiments did not use GPU acceleration, due to the lack of GPU support in the
cvxpylayers
Python package
(
Agrawal et al., 2019
)
.

In all experiments, we use a batch size of 256 and the Adam optimizer
(
Kingma and Ba, 2015
)
. Models were trained for up to 100 epochs with early stopping if there was no improvement in validation loss for 10 consecutive epochs.

For box and ellipsoid
ETO
baseline models, we performed a hyperparameter grid search over learning rates (
10
−
4.5
10^{-4.5}
,
10
−
4
10^{-4}
,
10
−
3.5
10^{-3.5}
,
10
−
3
10^{-3}
,
10
−
2.5
10^{-2.5}
,
10
−
2
10^{-2}
,
10
−
1.5
10^{-1.5}
) and L2 weight decay values (0,
10
−
4
10^{-4}
,
10
−
3
10^{-3}
,
10
−
2
10^{-2}
). For PICNN
ETO
models we performed a hyperparameter grid search over learning rates (
10
−
4
10^{-4}
,
10
−
3
10^{-3}
,
10
−
2
10^{-2}
) and L2 weight decay values (
10
−
4
10^{-4}
,
10
−
3
10^{-3}
,
10
−
2
10^{-2}
).

E.1
Uncertainty representation

Box uncertainty.

Our box uncertainty model uses a neural network
h
θ
h_{\theta}
with 3 hidden layers of 256 units each and ReLU activations with batch-normalization. The output layer has dimension
2
​
n
2n
, where dimensions
1
:
n
1:n
predict the lower bound. Output dimensions
n
+
1
:
2
​
n
n+1:2n
, after passing through a softplus to ensure positivity, represents the difference between the upper and lower bounds. That is,

[
h
θ
lo
​
(
x
)
h
θ
hi
​
(
x
)
]
=
[
h
θ
(
x
)
1
:
n
h
θ
(
x
)
1
:
n
+
softplus
(
h
θ
(
x
)
n
+
1
:
2
​
n
)
]
.
\begin{bmatrix}h_{\theta}^{\text{lo}}(x)\\
h_{\theta}^{\text{hi}}(x)\end{bmatrix}=\begin{bmatrix}h_{\theta}(x)_{1:n}\\
h_{\theta}(x)_{1:n}+\softplus(h_{\theta}(x)_{n+1:2n})\end{bmatrix}.

This architecture ensures that
h
θ
hi
​
(
x
)
>
h
θ
lo
​
(
x
)
h_{\theta}^{\text{hi}}(x)>h_{\theta}^{\text{lo}}(x)
.

In the two-stage
ETO
baseline, we first train
h
θ
h_{\theta}
to estimate the
α
/
2
\alpha/2
- and
(
1
−
α
/
2
)
(1-\alpha/2)
-quantiles, so that
[
h
θ
lo
​
(
x
)
,
h
θ
hi
​
(
x
)
]
[h_{\theta}^{\text{lo}}(x),h_{\theta}^{\text{hi}}(x)]
represents the centered
(
1
−
α
)
(1-\alpha)
-confidence region. Quantile regression is a common method for generating uncertainty sets for scalar predictions by estimating quantiles of the conditional distribution
𝒫
⁡
(
y
∣
x
)
\mathcal{P}(y\mid x)

(
Romano et al., 2019
)
. For scalar true label
y
y
, quantile regression models are commonly trained to minimize
pinball loss
(a.k.a.
quantile loss
) where
β
\beta
is the quantile level being estimated:

pinball
β
⁡
(
y
^
,
y
)
=
{
β
⋅
(
y
−
y
^
)
,
if
​
y
>
y
^
(
1
−
β
)
⋅
(
y
^
−
y
)
,
if
​
y
≤
y
^
.
\pinball_{\beta}(\hat{y},y)=\begin{cases}\beta\cdot(y-\hat{y}),&\text{if }y>\hat{y}\\
(1-\beta)\cdot(\hat{y}-y),&\text{if }y\leq\hat{y}.\end{cases}

To generalize the pinball loss to our setting of multi-dimensional
y
∈
ℝ
n
y\in\mathbb{R}^{n}
, we sum the pinball loss across the dimensions of
y
y
:

pinball
β
⁡
(
y
^
,
y
)
=
∑
i
=
1
n
pinball
β
⁡
(
y
^
i
,
y
i
)
\pinball_{\beta}(\hat{y},y)=\sum_{i=1}^{n}\pinball_{\beta}(\hat{y}_{i},y_{i})
.

Our end-to-end (E2E) box uncertainty models use the same architecture as above, initialized with weights from the trained
ETO
model. We found it helpful to use a weighted combination of the task loss and pinball loss during training of the E2E models to improve training stability. In our experiments, we used a weight of 0.9 on the task loss and 0.1 on the pinball loss. The E2E models used the best L2 weight decay from the
ETO
models, and the learning rate was tuned across
10
−
2
10^{-2}
,
10
−
3
10^{-3}
, and
10
−
4
10^{-4}
.

Ellipsoidal uncertainty.

Our ellipsoidal uncertainty model uses a neural network
h
θ
h_{\theta}
with 3 hidden layers of 256 units each and ReLU activations with batch-normalization. The output layer has dimension
n
+
n
⁡
(
n
+
1
)
/
2
n+n(n+1)/2
, where dimensions
1
:
n
1:n
predict the mean
μ
θ
​
(
x
)
\mu_{\theta}(x)
and the remaining output dimensions are used to construct a lower-triangular Cholesky factor
L
θ
​
(
x
)
L_{\theta}(x)
of the covariance matrix
Σ
θ
​
(
x
)
=
L
θ
​
(
x
)
​
L
θ
​
(
x
)
⊤
\Sigma_{\theta}(x)=L_{\theta}(x)L_{\theta}(x)^{\top}
. We pass the diagonal entries of
L
θ
​
(
x
)
L_{\theta}(x)
through a softplus function to ensure strict positivity, which then ensures
Σ
θ
​
(
x
)
\Sigma_{\theta}(x)
is positive definite.

For the
ETO
baseline, we trained the model using the negative log-likelihood (NLL) loss

NLL
(
θ
)
=
1
N
∑
(
x
,
y
)
∈
D
−
ln
𝒩
(
y
∣
μ
θ
(
x
)
,
Σ
θ
(
x
)
)
,
\NLL(\theta)=\frac{1}{N}\sum_{(x,y)\in D}-\ln\Gaussian(y\mid\mu_{\theta}(x),\Sigma_{\theta}(x)),

where
𝒩
(
⋅
∣
μ
,
Σ
)
\Gaussian(\cdot\mid\mu,\Sigma)
denotes the density of a multivariate normal distribution with mean
μ
\mu
and covariance matrix
Σ
\Sigma
.

Our end-to-end (E2E) ellipsoidal uncertainty models use the same architecture as above, initialized with weights from the the trained
ETO
model. We found it helpful to use a weighted combination of the task loss and NLL loss during training of the E2E models to improve training stability. In our experiments, we used a weight of 0.9 on the task loss and 0.1 on the NLL loss. The E2E models used the best L2 weight decay from the
ETO
models, and the learning rate was tuned across
10
−
2
10^{-2}
,
10
−
3
10^{-3}
, and
10
−
4
10^{-4}
.

PICNN uncertainty.

Our PICNN has 2 hidden layers with ReLU activations.

For the battery storage problem, we used 64 units per hidden layer. We did not run into any feasibility issues for the PICNN maximization problem, so we did not restrict
V
L
V_{L}
as described in
Section

C.4
, and we set
ϵ
=
0
\epsilon=0
.

For the portfolio optimization problem, we tried 32, 64, and 128 units per hidden layer, finding that 32 units worked best. We did run into feasibility issues for the PICNN maximization problem, which we resolved by setting
V
L
=
𝟎
1
×
n
V_{L}=\mathbf{0}_{1\times n}
as described in
Section

C.4
. This change alone was sufficient, and we set
ϵ
=
0
\epsilon=0
.

For the
ETO
baseline, we take inspiration from the approach by
Lin and Ba (2023)
to give probabilistic interpretation to a PICNN model
s
θ
s_{\theta}
via the energy-based model
𝒫
^
θ
​
(
y
∣
x
)
=
1
Z
θ
​
(
x
)
​
exp
⁡
(
−
s
θ
​
(
x
,
y
)
)
\hat{\mathcal{P}}_{\theta}(y\mid x)=\frac{1}{Z_{\theta}(x)}\exp(-s_{\theta}(x,y))
where
Z
θ
​
(
x
)
:=
∫
y
~
∈
ℝ
n
exp
⁡
(
−
s
θ
​
(
x
,
y
~
)
)
​
⁡
𝑑
y
~
Z_{\theta}(x):=\int_{\tilde{y}\in\mathbb{R}^{n}}\exp(-s_{\theta}(x,\tilde{y}))\mathop{}\!\mathrm{d}\tilde{y}
is the normalizing constant. We train our
ETO
PICNN models with an approximation to the true NLL loss based on samples from the Metropolis-Adjusted Langevin Algorithm (MALA), a Markov Chain Monte Carlo (MCMC) method.
We refer readers to our code for the specific hyperparameters and implementation details we used.

Note that under this energy-based model, adding a scalar constant
c
c
to the PICNN (i.e.,
s
θ
​
(
x
,
y
)
+
c
s_{\theta}(x,y)+c
) does not change the probability distribution. That is,
exp
⁡
(
−
s
θ
​
(
x
,
y
)
)
∝
exp
⁡
(
−
s
θ
​
(
x
,
y
)
+
c
)
\exp(-s_{\theta}(x,y))\propto\exp(-s_{\theta}(x,y)+c)
. To regularize the PICNN model, which has a bias term in its output layer, we therefore introduce a regularization loss of
w
zero
⋅
s
θ
​
(
x
,
y
)
2
w_{\text{zero}}\cdot s_{\theta}(x,y)^{2}
where
w
zero
w_{\text{zero}}
is a regularization weight. This regularization loss encourages
s
θ
​
(
x
,
y
)
s_{\theta}(x,y)
to be close to 0, for all examples in the training set. In our experiments, we set
w
zero
=
1
w_{\text{zero}}=1
.

Our end-to-end (E2E) PICNN uncertainty models use the same architecture as above, initialized with weights from the the trained
ETO
model. Unlike for box and ellipsoidal uncertainty which used a weighted combination of task loss and NLL loss, our E2E PICNN uncertainty models are trained only with the task loss. Similar to the
ETO
PICNN model, we also regularize the E2E PICNN. Here, we add a regularization loss of
w
q
⋅
q
2
w_{\text{q}}\cdot q^{2}
, where
w
q
w_{\text{q}}
is a regularization weight and
q
q
is the conformal prediction threshold computed in each minibatch of E2E training. This regularization loss term aims to keep
q
q
near 0; without this regularization, we found that
q
q
tended to grow dramatically over training epochs with poor task loss. In our experiments, we set
w
q
=
0.01
w_{\text{q}}=0.01
.

The E2E models used the best L2 weight decay from the
ETO
models. For the battery storage problem, we tested learning rates of
10
−
3
10^{-3}
and
10
−
4
10^{-4}
. For the portfolio optimization problem, we used a learning rate of
5
×
10
−
3
5\times 10^{-3}
.

E.2
Data

Price forecasting for battery storage.

We use the same dataset as
Donti et al. (2017)
in our price forecasting for battery storage problem. In this dataset, the target
y
∈
ℝ
24
y\in\mathbb{R}^{24}
is the hourly PJM day-ahead system energy price for 2011-2016, for a total of 2189 days. Unlike
Donti et al. (2017)
, though, we do not exclude any days whose electricity prices are too high (>500$/MWh). Whereas
Donti et al. (2017)
treated these days as outliers, our conditional robust optimization problem is designed to output robust decisions. For predicting target for a given day, the inputs
x
∈
ℝ
101
x\in\mathbb{R}^{101}
include the previous day’s log-prices, the given day’s hourly load forecast, the previous day’s hourly temperature, the given day’s hourly temperature, and several calendar-based features such as whether the given day is a weekend or a US holiday.

For the setting without distribution shift, we take a random 20% subset of the dataset as the test set; because the test set is selected randomly, it is considered exchangeable with the rest of the dataset. For the setting with distribution shift, we take the chronologically last 20% of the dataset as the test set; because load, electricity prices, and temperature all have distribution shifts over time, the test set is not exchangeable with the rest of the dataset.
Figure

A4
illustrates the data from the distribution shift setting, where the electricity price tends to be lower and has lower variability in the test set compared to the training/validation splits. For each seed, we further use a 80/20 random split of the remaining data for training and calibration.

(a)

Month of year vs. electricity price

(b)

Hour of day vs. electricity price (shaded region shows
±
\pm
1 std dev)

(c)

Daily high temperature (rounded to nearest 5
∘
F) vs. daily peak electricity price (shaded region shows
±
\pm
1 std dev)

Figure A4
:
Visualization of data from the battery storage problem under distribution shift. The train/val splits are sampled randomly from the range 2011-01-04 to 2015-10-20, whereas the test set comprises 2015-10-21 to 2016-12-31. These plots evidently show that the electricity price is generally lower in the test set and also has lower variability by both hour of day and the daily maximum temperature.

Portfolio optimization.

For the portfolio optimization task, we used synthetically generated data. We sample
x
∈
ℝ
2
,
y
∈
ℝ
2
x\in\mathbb{R}^{2},y\in\mathbb{R}^{2}
from a mixture of three 4-D multivariate Gaussian distributions as used in
Chenreddy and Delage (2024)
. Formally,

[
x
y
]
∼
p
a
​
𝒩
⁡
(
μ
a
,
Σ
a
)
+
p
b
​
𝒩
⁡
(
μ
b
,
Σ
b
)
+
p
c
​
𝒩
⁡
(
μ
c
,
Σ
c
)
\begin{bmatrix}x\\
y\end{bmatrix}\sim p_{a}\Gaussian(\mu_{a},\Sigma_{a})+p_{b}\Gaussian(\mu_{b},\Sigma_{b})+p_{c}\Gaussian(\mu_{c},\Sigma_{c})

where
p
a
+
p
b
+
p
c
=
1
p_{a}+p_{b}+p_{c}=1
. Specifically,

p
a
\displaystyle p_{a}

=
ϕ
,
\displaystyle=\phi,

p
b
\displaystyle p_{b}

=
1
α
GMM
+
1
​
(
1
−
ϕ
)
,
\displaystyle=\frac{1}{\alpha_{\text{GMM}}+1}(1-\phi),

p
c
\displaystyle p_{c}

=
α
GMM
α
GMM
+
1
​
(
1
−
ϕ
)
,
\displaystyle=\frac{\alpha_{\text{GMM}}}{\alpha_{\text{GMM}}+1}(1-\phi),

μ
a
\displaystyle\mu_{a}

=
𝟎
4
,
\displaystyle=\mathbf{0}_{4},

μ
b
\displaystyle\mu_{b}

=
[
0
5
5
0
]
⊤
,
\displaystyle=\begin{bmatrix}0&5&5&0\end{bmatrix}^{\top},

μ
c
\displaystyle\mu_{c}

=
μ
b
,
\displaystyle=\mu_{b},

Σ
a
\displaystyle\Sigma_{a}

=
[
1
0
0.37
0
0
1.5
0
0
0.37
0
2
0.73
0
0
0.73
3
]
,
\displaystyle=\begin{bmatrix}1&0&0.37&0\\
0&1.5&0&0\\
0.37&0&2&0.73\\
0&0&0.73&3\end{bmatrix},

Σ
b
\displaystyle\Sigma_{b}

=
α
GMM
​
Σ
a
,
\displaystyle=\alpha_{\text{GMM}}\Sigma_{a},

Σ
c
\displaystyle\Sigma_{c}

=
1
α
GMM
​
Σ
a
,
\displaystyle=\frac{1}{\alpha_{\text{GMM}}}\Sigma_{a},

for some
ϕ
∈
[
0
,
1
]
\phi\in[0,1]
and
α
GMM
∈
[
0
,
1
]
\alpha_{\text{GMM}}\in[0,1]
. In our experiments, we used
ϕ
=
0.7
\phi=0.7
and
α
GMM
=
0.9
\alpha_{\text{GMM}}=0.9
. (
Chenreddy and Delage (2024)
do not disclose the values of
ϕ
\phi
and
α
GMM
\alpha_{\text{GMM}}
chosen for their experiments.) For each random seed, we generate 2000 samples and use a (train, calibration, test) split of (600, 400, 1000).

Experimental support, please

view the build logs

for errors. Generated by

L

A

T

E

xml

.

Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile
support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the
methods listed below:

Click the "Report Issue"
(
)
button, located in the page header.

Tip:
You can select the relevant text first, to include it in your report.

Our team has already identified
the following issues
. We appreciate your time reviewing and reporting rendering errors we
may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability
should not be a barrier to accessing research. Thank you for your continued support in championing open access for
all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a
list of packages that need conversion
, and welcome
developer contributions
.

We gratefully acknowledge support from
our
major funders
,

member institutions
,
,
and all contributors.

About

·

Help

·

Contact

·

Subscribe

·

Copyright

·

Privacy

·

Accessibility

·

Operational Status
(opens in new tab)

Major funding support from
</reference>

<statements>
1. By employing distribution-free techniques such as Conformal Prediction and Partially Input-Convex Neural Networks (PICNNs), modern architectures can construct calibrated uncertainty sets that provide valid finite-sample coverage without requiring Gaussian or elliptical assumptions
2. In portfolio selection, however, the loss landscape is highly asymmetric and constrained
3. where \(w^*(y)\) is the optimal decision under realized parameters, and \(w^*(\hat{y})\) is the decision computed using model predictions
4. Conformal nonconformity scores establish prediction intervals that guarantee finite-sample coverage at a confidence level of \(1 - \alpha\)
5. Dual linear prediction heads; split-conformal quantile regression for distribution-free bounds
</statements>

Begin the assessment now. Output only the JSON list, without any conversational text or explanations.