All posts

What a Monte Carlo probability is conditional on

A simulated probability is an exact statement about a model and a vague one about the world. Here is the boundary, drawn precisely, for the engine behind Folio Lab.

Pranav Surampudi5 min readMonte CarloMethodology

A Monte Carlo run in Folio Lab ends with a sentence like there is a 63% chance this portfolio is worth more than ₹1 crore in ten years. That sentence is doing two very different jobs at once, and the difference between them is the entire subject of this post.

The first job is arithmetic, and it is exact. Out of the paths the engine generated, 63% of them ended above the target. Nothing about that is uncertain. Run it again with the same seed and the same inputs and you get the same number, to the last digit.

The second job is inference, and it is not exact at all. It is the leap from 63% of simulated paths to 63% chance, and every bit of the risk in that sentence lives in the leap.

The estimator is the easy part

Let WHW_H be terminal wealth at horizon HH and ww the target. The engine draws NN paths and reports the sample proportion:

p^=P^(WHw)=1Ni=1N1 ⁣{WH(i)w}\widehat{p} = \widehat{\mathrm{P}}(W_H \ge w) = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\!\left\{ W_H^{(i)} \ge w \right\}

This is a binomial proportion, so its Monte Carlo standard error is closed-form:

se(p^)=p(1p)N\mathrm{se}(\widehat{p}) = \sqrt{\frac{p(1-p)}{N}}

At N=10,000N = 10{,}000 and p0.5p \approx 0.5, that is se0.005\mathrm{se} \approx 0.005: a 95% interval about one percentage point wide either side. Simulation noise, in other words, is already small enough to ignore relative to everything else on this page.1

That is worth sitting with. The part of the number we can quantify precisely is the part that barely matters. The part that matters cannot be quantified from inside the simulation at all.

Five things the number is conditional on

Every probability the engine reports is conditional on a specific, recorded configuration. These are not caveats bolted on afterwards; they are inputs, and the output is a function of them.

ConditionWhat it fixesWhat moves if you change it
Return processThe joint law paths are drawn fromTail mass, clustering, drawdown depth
Optimizer-time inputsWeights, frozen at the parent runThe entire allocation being simulated
Effective configurationHorizon, contributions, tax, inflationThe wealth accounting, not the returns
Engine versionSimulation semanticsAnything, by definition
SeedThe specific drawThe last digit, and nothing structural

The second row is the one people miss. A Monte Carlo run does not re-optimize. It takes the weights a completed optimization produced, freezes them, and asks what happens next. That is a deliberate design decision: it keeps the simulation a statement about this portfolio rather than a statement about the optimizer, and it is why the two live on separate queues with separate workers.

OPTIMIZEMONTE CARLOsubmitenqueueconsumepersistenqueueconsumepersistparent run id, frozen weightsBrowseroptimize formFastAPIPOST /optimizeRedisoptimize queueWorker31 methodsNeon + R2run, artifactFastAPIPOST /monte-carloRedismc queuemc-worker2 GiB, dedicatedNeon + R2paths, summary
One optimize request, then one Monte Carlo request against it. The lanes are joined by a run id, not by a shared process.

Where the estimate ends and the model begins

The default return process is a stationary block bootstrap. It resamples contiguous blocks of historical returns, which preserves autocorrelation and cross-sectional structure inside a block and destroys it across block boundaries.

the shape of the resampling, not the implementation
# Blocks are contiguous in time, so a 2008 October block carries
# its own correlation structure with it. What the bootstrap cannot
# do is invent a regime the sample never contained.
for path in range(n_paths):
    t = 0
    while t < horizon:
        start = rng.integers(0, n_obs - block_len)
        block = returns[start : start + block_len]
        emit(block[: horizon - t])
        t += block_len

The honest description of what that buys is narrow: the simulated distribution is the historical distribution, rearranged. If the sample window contains no episode like the one that actually arrives, no number of paths will produce one. Increasing NN samples the same empirical law more finely; it does not widen it.

Why none of this is called a forecast

Folio Lab does not describe these probabilities as calibrated forecasts, and the reason is specific rather than rhetorical: no generator here has been validated out of sample.

Calibration is a testable property. A generator is calibrated at horizon HH if, across many rolling origins, the realised outcome falls below the predicted qq-th quantile about qq of the time. Testing it needs a rolling-origin harness that re-fits at each origin, holds the next HH days out, and scores coverage against published thresholds decided before the results are seen.

Rolling-origin validation: fit on what was knowable, score on what happened next, never the reverse.

That harness exists and has been run. It has not passed. The best-performing arm calibrated to roughly 3.2% against its threshold and was still rejected, and at the annual horizon the gate is not reachable by any generator we have tried, which is a statement about the study design as much as about the models.

So the language stays where the evidence is. The engine reports what its model implies. It does not claim the model is right, and it will not claim one generator beats another until something measured says so.

What would change the claim

Three things, in order of how much they would change:

  1. A generator passing rolling-origin validation at a stated horizon. Then, and only then, the word "calibrated" becomes available, scoped to that horizon and that generator.
  2. A validation gate whose power is understood. A test nothing can pass is not evidence of anything except the test.
  3. Out-of-sample evidence that one rebalancing policy beats another. Currently unclaimed, deliberately.

Until then the right way to read a Folio Lab probability is as a precise answer to a question you fully specified: given this return process, these frozen weights, this configuration and this engine, what fraction of paths clears the target? That question has an exact answer, and the platform gives it to you along with everything it was conditional on.

The question you actually wanted answered is harder, and nobody has it.

Footnotes

  1. Which is why adding paths is the cheapest and least useful lever available. Going from 10,000 to 100,000 paths shrinks the interval by a factor of 10\sqrt{10} and changes nothing about whether the model was right.

Share this post

Subscribe

No email list, no account, no algorithm deciding what reaches you. Point a reader at the feed and every post arrives in full, math included.