Nested Clustered Optimization

Distance Metric

The first step converts the correlation matrix into a proper distance metric. The standard angular distance transforms correlations into distances that satisfy the metric axioms (non-negativity, symmetry, triangle inequality):

Perfectly correlated assets () have distance zero, uncorrelated assets () have distance , and perfectly anti-correlated assets () have distance one.

Hierarchical Clustering

Using the distance matrix, agglomerative hierarchical clustering groups assets into clusters. Common linkage methods include single linkage, complete linkage, and Ward's method. The number of clusters can be determined by the gap statistic, silhouette scores, or a fixed threshold on the dendrogram. Each cluster contains a subset of assets that are more correlated with each other than with assets in other clusters.

For each cluster , a standard portfolio optimization is performed using only the assets in that cluster. Let and denote the expected return vector and covariance matrix restricted to the assets in cluster . The intra-cluster optimal weights are:

where is the chosen objective function (e.g., maximum Sharpe ratio, minimum variance, or risk parity). The key advantage is that each cluster contains far fewer assets than the full universe, so the covariance sub-matrix is much smaller and better conditioned.

Cluster-Level Statistics

Once the intra-cluster weights are determined, each cluster is reduced to a single "meta-asset" with the following return and variance:

These meta-asset statistics form the inputs for the inter-cluster optimization. The covariance between meta-assets and is computed as:

where is the cross-covariance block between clusters and .

The inter-cluster optimization determines the allocation across the meta-assets. Let and denote the -dimensional return vector and covariance matrix of the meta-assets. The inter-cluster weights are:

This is a much smaller optimization problem (dimension instead of ), which is faster to solve and far less prone to numerical instability. The same objective function can be used at both levels, or different objectives can be chosen for each stage.