Folio Lab/Documentation

Sparse Markowitz (L1)

Mean-variance optimisation with an L1 penalty on the weight vector. Brodie, Daubechies, De Mol, Giannone and Loris (2009) showed that adding an norm to the Markowitz objective produces sparse, stable portfolios that out-perform the unregularised optimum on out-of-sample data - the "lasso" idea (Tibshirani, 1996) ported to portfolio construction.

Overview

Classical mean-variance is notoriously ill-conditioned: small perturbations of the input mean and covariance produce large changes in the optimal weights (Michaud, 1989). Brodie et al. (2009) showed that adding an regularisation term to the Markowitz objective produces a portfolio that (a) is sparse: most weights are exactly zero, and (b) generalises better out-of-sample. The mechanism is the same as in the lasso (Tibshirani, 1996): the penalty shrinks small weights to zero, eliminating noise-driven positions.

DeMiguel, Garlappi, Nogales and Uppal (2009) generalised the idea to a family of norm-constrained portfolio optimisations and showed that imposing a norm ball on the weights subsumes several classical regularisations (no short-selling, Jagannathan-Ma 2003 covariance shrinkage, etc.). The L1 norm sits naturally between "long-only with no further structure" and "long-only with explicit cardinality cap".

FolioLab implements the Brodie et al. objective as a tractable quadratic-programming problem, under a long-only, fully invested feasible set. Both coefficients are fixed in the dispatch: the penalty strength is l1_coef = 0.02, and the coefficient on the expected-return term is return_tradeoff = 8.0. Neither is a request field, and neither is tuned by cross-validation or fitted to a target cardinality.

Read the next section before you read the objective. The shipped feasible set keeps the long-only simplex, and under it the L1 term is a constant. It therefore cannot drive sparsity on this platform, whatever value takes.

Mathematical Formulation

Objective

The first two terms are the quadratic-utility formulation of MVO. The third term is the penalty with strength , fixed at 0.02.

is not a risk-aversion parameter here, although the textbook form of this objective calls it one. The code names it return_tradeoff, and it scales the expected-return term, so a higher value seeks more return and accepts more variance. That is the opposite direction to a risk-aversion reading. It is fixed at 8.0.

The limiting behaviour in described in the paper needs a feasible set where can vary. The shipped feasible set is not one, as the next section states, so the solution does not move toward a single asset as grows.

Long-only and budget constraints

Under the long-only simplex constraint , so the L1 penalty is constant and the regularisation has no effect. Brodie et al. therefore relax the long-only constraint, allowing some short positions, so that the L1 norm becomes informative.

FolioLab does not relax it. The program fixes and , and the leverage row is , which the simplex already satisfies with equality. There is no short-selling option and no leverage field on the request. So the L1 term is a constant on every run of this method, and it contributes nothing to the solution. What you get is the quadratic-utility solve under a long-only budget, with the shrinkage the penalty would supply absent.

Equivalent constrained form

The Lagrangian penalty form above is equivalent (by convex duality) to the constrained form:

caps the total gross leverage; setting recovers long-only. This is exactly the norm-constrained portfolio in DeMiguel, Garlappi, Nogales and Uppal (2009).

Choosing the penalty strength

The hyperparameter trades off in-sample fit against out-of-sample stability. Standard practice (DeMiguel, Garlappi, Nogales and Uppal, 2009; Brodie et al., 2009) is k-fold cross-validation on historical returns, optimising for out-of-sample Sharpe or out-of-sample volatility. Alternatively, can be set by targeting a desired portfolio cardinality: the L1 path is piecewise linear in , and walking up the path produces a sequence of portfolios of decreasing size.

FolioLab's default selects a moderate chosen to keep the optimisation visibly different from sample-MVO without driving the solution to a single name.

Advantages & Limitations

Advantages

  • Sparse weights, in the published method: Where the L1 term is informative, most positions come out exactly zero, which is cleaner to trade and explain. The shipped long-only configuration does not produce that, because the penalty is a constant on the simplex.
  • Out-of-sample stability, in the published method: L1 shrinkage suppresses noise-driven positions. The shipped configuration does not deliver that shrinkage, for the same reason.
  • Convex QP: Solves quickly with off-the-shelf solvers.
  • Convex duality: Penalty and constrained forms are interchangeable.

Limitations

  • Needs short-selling, and does not get it here: Under the long-only simplex constraint the L1 penalty is uninformative. This method ships long-only and fully invested, so the penalty is uninformative on every run.
  • Penalty is fixed: is pinned at 0.02 and cannot be tuned per run.
  • Sample mean is still noisy: The expected-return input remains the bottleneck.
  • Discontinuous in : Cardinality jumps at kinks of the L1 path.

References

  • Brodie, J., Daubechies, I., De Mol, C., Giannone, D., & Loris, I. (2009). "Sparse and Stable Markowitz Portfolios." Proceedings of the National Academy of Sciences, 106(30), 12267-12272.
  • DeMiguel, V., Garlappi, L., Nogales, F. J., & Uppal, R. (2009). "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms." Management Science, 55(5), 798-812.
  • Tibshirani, R. (1996). "Regression Shrinkage and Selection via the Lasso." Journal of the Royal Statistical Society: Series B, 58(1), 267-288.
  • Jagannathan, R., & Ma, T. (2003). "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps." The Journal of Finance, 58(4), 1651-1683.
  • Michaud, R. O. (1989). "The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?" Financial Analysts Journal, 45(1), 31-42.

Not investment advice. Past performance is not indicative of future results.