https://doi.org/10.5281/zenodo.20711000
Abstract:
Conditional value-at-risk (CVaR), or expected shortfall, leads to large-scale constrained quadratic programs in finance, energy, logistics, contrl, and learning. Recent algorithms for CVaR-constraint quadratic programs identify projection onto a top-k-sum sublevel set as the central computational primitive. This paper studies the complementary problem: how to differentiate this projection, and how to use it as a differentiable layer.
We show that the top-k-sum projection is piecewise affine and that an active-face certificate from the forward projection determines its backward pass. On each fixed face, the Jacobian is the orthogonal projector onto the corresponding tangent space, giving exact vector-Jacobian products in O(m) time once the certificate is available. The method handles degenerate tied plateaus and provides adjoints with respect to both the projection input and the CVaR risk budget, without forming a deterministic-equivalent epigraph quadratic program. For full CVaR-constrained quadratic programs, accurate solves are differentiated through a reduced active-face KKT system, while early-stopped solves are differentiated by unrolling.
Numerical tests validate the adjoints against finite differences and an independent solver, including degenerate cases. Batched CPU/GPU implementations scale to 200 million scenarios, differentiating a 200-million-scenario projection in under 0.12 seconds on a single GPU, while deterministic-equivalent differentiation becomes impractical near 100,000 scenarios. Pre-specified reproducible experiments show that this scale is decision-relevant: hard CVaR constraints place realized out-of-sample tail risk on budget as the scenario count grows, including under distribution shift, whereas calibrated penalty methods drift or retain substantial per-instance dispersion. Code and a one-command reproduction are provided.
Keywords:
conditional value-at-risk (CVaR); top-k-sum projection; differentiable optimization; vector-Jacobian product; ADMM; risk-constrained quadratic programming