HMM Regime MVO
Markov-switching regime-conditioned mean-variance optimisation. Returns are modelled as draws from a small set of latent regimes (typically two: a calm regime and a stress regime), and the optimiser uses regime-dependent moments to construct a portfolio that adapts to the prevailing market state.
Overview
Equity-return distributions are not stationary: long calm periods are punctuated by short, violent stress episodes with very different volatility, correlation, and skewness profiles. A single sample mean and a single sample covariance fitted across the full history will average over these regimes and produce moments that are wrong for both. The Markov-switching framework of Hamilton (1989, 1990) provides a principled treatment: a hidden state variable governs which of regimes is active at each time, and the asset returns are conditionally Gaussian given the regime.
Ang and Bekaert (2002) applied the framework to international asset allocation; Guidolin and Timmermann (2007) studied multi-regime allocation with Bayesian inference; Nystrup, Madsen and Lindstrom (2018) gave the formulation closest to the FolioLab implementation, where regime probabilities are estimated by EM and used to compute regime-mixed moments for the MVO step.
The most important practical question is the number of regimes. Two regimes (calm and stress) are interpretable and statistically identifiable on most equity samples. Three regimes (calm, transition, stress) sometimes give better in-sample fit but are harder to identify out of sample. FolioLab fixes the count at two. The number of regimes is pinned to n_regimes = 2 in the dispatch and is not a request field.
Two further points decide how you read a result from this method. The regime model is fitted on one univariate series, not on the asset panel: the benchmark return series when a benchmark is supplied, and the cross-sectional mean of the asset returns when it is not. The regime probabilities are filtered, on every run, not only inside a rolling backtest.
Mathematical Formulation
Model
- - latent regime at time
- - transition probabilities
- - conditional Gaussian returns
- - filtered regime probability at the optimisation date, the last row of the filtered sequence
- here is the single series the regime model is fitted on, not the asset panel
EM estimation
The parameters are estimated by the Expectation-Maximisation algorithm: the Forward-Backward (Baum-Welch) recursions compute the smoothed regime probabilities given the parameters, and the M-step updates each regime's mean and covariance as the probability-weighted sample moments. EM converges in a few dozen iterations for two-state models.
The library is statsmodels, through its MarkovRegression estimator. It is not hmmlearn. The same fit powers the forward Monte Carlo engine, so a regime split here and a regime split there come from one deterministic, multi-start, fail-closed routine.
Smoothed probabilities are used inside the fit, which is what Baum-Welch requires. They are not what the allocation reads. The allocation reads the filtered probabilities, as the next section states.
Regime-mixed moments
The optimisation-date moments are mixtures of the regime-specific moments, weighted by the filtered regime probabilities. The covariance term picks up both the within-regime variance and the between-regime variance - the latter is exactly what a single-regime estimator misses.
Plug-in MVO
The mixture moments , are substituted directly into the MVO problem. The objective is the Sharpe ratio, not the quadratic utility a textbook plug-in MVO usually writes. There is no risk-aversion coefficient on this program, and no field sets one:
The regime model estimates the inputs; the allocation underneath is the same constrained max-Sharpe solve the other frontier methods run. When the filter indicates the calm regime is dominant the portfolio looks like a typical MVO; when it indicates the stress regime is dominant the portfolio shifts toward defensive names and lower vol-uncorrelated clusters.
Identifying regimes on Indian equities
On Nifty / Sensex daily returns, a two-state HMM typically identifies a high-mean low-vol calm regime and a low-mean high-vol stress regime. Stress regimes correspond to identifiable historical episodes (the 2008 crisis, the 2013 taper tantrum, the 2020 COVID drawdown, the 2022 risk-off). Transition probabilities are usually highly persistent, with regime durations from weeks to several months.
Out-of-sample regime classification is the hardest part. A smoothed probability is two-sided: it conditions on the whole sample, including data after the date it describes. A filtered probability is one-sided, so it conditions only on data up to that date, and it lags the smoothed estimate by roughly the regime's persistence horizon.
FolioLab uses the filtered probability on every run, not only inside a rolling backtest. The probability that weights the mixture moments is the last row of the filtered sequence at the optimisation date. That avoids look-ahead in the allocation. It does not make the regime split itself out of sample: the model parameters are still fitted on the whole submitted window.
Advantages & Limitations
Advantages
- Adaptive moments: Captures regime structure that single-window estimators miss.
- Interpretable states: Two-regime models map cleanly to calm vs stress.
- Principled mixture covariance: Includes between-regime variance.
- Plug-in compatible: Drops into any downstream MVO formulation.
Limitations
- Identifiability: EM can fall into local optima, especially with many regimes.
- Latency: Regime changes are detected with a lag.
- Gaussian regimes: Within-regime distribution is still parametric.
- Look-ahead risk: Smoothed probabilities use future data; only filtered probabilities are out-of-sample valid.
References
- Hamilton, J. D. (1989). "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle." Econometrica, 57(2), 357-384.
- Hamilton, J. D. (1990). "Analysis of Time Series Subject to Changes in Regime." Journal of Econometrics, 45(1-2), 39-70.
- Ang, A., & Bekaert, G. (2002). "International Asset Allocation with Regime Shifts." The Review of Financial Studies, 15(4), 1137-1187.
- Guidolin, M., & Timmermann, A. (2007). "Asset Allocation under Multivariate Regime Switching." Journal of Economic Dynamics and Control, 31(11), 3503-3544.
- Nystrup, P., Madsen, H., & Lindstrom, E. (2018). "Dynamic Portfolio Optimization across Hidden Market Regimes." Quantitative Finance, 18(1), 83-95.
- Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970). "A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains." The Annals of Mathematical Statistics, 41(1), 164-171.
Not investment advice. Past performance is not indicative of future results.