Folio Lab/Documentation

Sparse Markowitz (L1)

Mean-variance optimisation with an L1 penalty on the weight vector. Brodie, Daubechies, De Mol, Giannone and Loris (2009) showed that adding an norm to the Markowitz objective produces sparse, stable portfolios that out-perform the unregularised optimum on out-of-sample data — the "lasso" idea (Tibshirani, 1996) ported to portfolio construction.

Overview

Classical mean-variance is notoriously ill-conditioned: small perturbations of the input mean and covariance produce large changes in the optimal weights (Michaud, 1989). Brodie et al. (2009) showed that adding an regularisation term to the Markowitz objective produces a portfolio that (a) is sparse: most weights are exactly zero, and (b) generalises better out-of-sample. The mechanism is the same as in the lasso (Tibshirani, 1996): the penalty shrinks small weights to zero, eliminating noise-driven positions.

DeMiguel, Garlappi, Nogales and Uppal (2009) generalised the idea to a family of norm-constrained portfolio optimisations and showed that imposing a norm ball on the weights subsumes several classical regularisations (no short-selling, Jagannathan-Ma 2003 covariance shrinkage, etc.). The L1 norm sits naturally between "long-only with no further structure" and "long-only with explicit cardinality cap".

FolioLab implements the Brodie et al. formulation as a tractable quadratic-programming problem. The penalty strength is the only hyperparameter and can be tuned by cross-validation or selected to hit a target portfolio cardinality.

Mathematical Formulation

Objective

The first two terms are the standard quadratic-utility formulation of MVO with risk-aversion . The third term is the penalty with strength . As the solution collapses to standard MVO; as the solution collapses to a single asset (or a small basket).

Long-only and budget constraints

Under the long-only simplex constraint , so the L1 penalty is constant and the regularisation has no effect. Brodie et al. therefore relax the long-only constraint, allowing some short positions, so that the L1 norm becomes informative. FolioLab's implementation follows the same pattern: the L1 penalty is informative when short selling is allowed, and the user can control the maximum allowed short via a leverage parameter.

Equivalent constrained form

The Lagrangian penalty form above is equivalent (by convex duality) to the constrained form:

caps the total gross leverage; setting recovers long-only. This is exactly the norm-constrained portfolio in DeMiguel, Garlappi, Nogales and Uppal (2009).

Choosing the penalty strength

The hyperparameter trades off in-sample fit against out-of-sample stability. Standard practice (DeMiguel, Garlappi, Nogales and Uppal, 2009; Brodie et al., 2009) is k-fold cross-validation on historical returns, optimising for out-of-sample Sharpe or out-of-sample volatility. Alternatively, can be set by targeting a desired portfolio cardinality: the L1 path is piecewise linear in , and walking up the path produces a sequence of portfolios of decreasing size.

FolioLab's default selects a moderate chosen to keep the optimisation visibly different from sample-MVO without driving the solution to a single name.

Advantages & Limitations

Advantages

  • Sparse weights: Most positions are exactly zero — cleaner to trade and explain.
  • Out-of-sample stable: L1 shrinkage suppresses noise-driven positions.
  • Convex QP: Solves quickly with off-the-shelf solvers.
  • Convex duality: Penalty and constrained forms are interchangeable.

Limitations

  • Needs short-selling: Under the long-only simplex constraint the L1 penalty is uninformative.
  • Penalty tuning: A poor dominates the solution.
  • Sample mean is still noisy: The expected-return input remains the bottleneck.
  • Discontinuous in : Cardinality jumps at kinks of the L1 path.

References

  • Brodie, J., Daubechies, I., De Mol, C., Giannone, D., & Loris, I. (2009). "Sparse and Stable Markowitz Portfolios." Proceedings of the National Academy of Sciences, 106(30), 12267-12272.
  • DeMiguel, V., Garlappi, L., Nogales, F. J., & Uppal, R. (2009). "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms." Management Science, 55(5), 798-812.
  • Tibshirani, R. (1996). "Regression Shrinkage and Selection via the Lasso." Journal of the Royal Statistical Society: Series B, 58(1), 267-288.
  • Jagannathan, R., & Ma, T. (2003). "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps." The Journal of Finance, 58(4), 1651-1683.
  • Michaud, R. O. (1989). "The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?" Financial Analysts Journal, 45(1), 31-42.