Smart Portfolios 2 – Uncertainty and Optimisation

Smart Portfolios 2 – Uncertainty and Optimisation

Today’s post is our second visit to Rob Carver’s book Smart Portfolios.


Chapter Two of Rob’s book is about uncertainty, and how you can use what happened in the past to make predictions about the future.

  • Inspired by Ed Thorp’s success in card counting, Rob begins by comparing investment to blackjack.

Note that Rob makes no distinction between investing, trading and gambling, since they all involve risking money with an uncertain outcome.

  • The big mistakes are betting when you will lose on average (no edge / odds against you) and putting too much money into a single bet.

In blackjack, the key is betting more when the odds are in your favour (based on what kind of cards are left in the pack).

  • Rob imagines investment as a game with a very large deck of cards whose contents are not known.

Each card has a number which indicates an annual return, some of which are negative.

  • Each round (year) you need to decide how much to bet (or not to bet at all).

So we need to predict the value of the next card, based on a model which includes an expected average and some measure of the variation in outcome (volatility, sometimes called risk).

The advantage of the investment game over blackjack is that we can see all the cards that have already been dealt.

  • The disadvantage is that this doesn’t tell us which cards are left in the deck.

Blackjack vs investment

Blackjack vs investment

The most common approach used in real life investment is to assume that history repeats itself.

This is of course, the opposite of what the regulatory warnings say.

If the starting deck is extremely large, the undealt deck is going to look almost exactly like the starting deck, no matter how many cards have already been dealt.

This assumption leads to the opposite strategy to blackjack, where dealing a card (from a multi-deck shoe) means that card is less likely to turn up in future.

  • In investment, more good cards already played means more good cards to come.

Investment with assumption

Investment with assumption

Almost everyone involved in the markets has their eyes fixed firmly on the rear-view mirror.


To decide whether to play you need to know your expected average mean return. If the average was negative there would be no point participating.

In the distribution in [the chart above] the mean return is positive: 0.08571%. The geometric mean is slightly lower: 0.08521%. As this is positive it’s worth playing the next round.

The correct size of your bet depends on the expected variation in returns.
There are many ways of doing this but I prefer to use standard deviation: the average deviation from the mean return.

Rob refers to the standard deviation of returns (SD) as volatility.

  • In the chart below this is 0.322%.

Individual equities have a standard deviation of around 25% to 30% a year (a little more in emerging markets, a little less in more developed markets).

Bonds tend to be safer. A typical two-year bond issued by a safe government has an annual volatility of around 1.5%, and a ten-year bond would be more like 8%.

As I use geometric means, I’ll also be using geometric standard deviations. However it’s usually safe to assume the two versions are equal.

There is a useful formula for finding the approximate geometric mean, given the arithmetic mean and the standard deviation.

geometric mean = arithmetic mean - 0.5 (standard deviation)2
Gaussian returns

The dealt cards can be summarised as a histogram (line graph) which can then be modelled according to the Gaussian or normal distribution.

  • Note that this is not an accurate model, but it makes the maths simpler.

Past returns

Past returns

Using a statistical model with a Gaussian distribution makes your life much easier since you only need to estimate two parameters: the mean and the standard deviation.

Gaussian models are notoriously bad at modelling extreme returns in financial markets. With a Gaussian distribution you’d get a 4 standard deviation fall on only 1 out of 31,500 days (roughly once every century).

But from 1914 to 2014 the Dow Jones Index had a 4 standard deviation fall on more than 30 occasions!

If your returns are Gaussian normal, then the mean and standard deviation alone are sufficient to say how likely certain returns will be.

There is a difference between the actual past returns [solid line] and the statistical model of past returns (the dotted line). Although they have the same mean and standard deviation, they have different distributions.



Values one standard deviation or less around the average 68.2% of the time, and returns two standard deviations or less about 95.4% of the time.

In 2.3% of periods you’d see a change two standard deviations better than the average. You’d also see a return which is two standard deviations worse than the average 2.3% of the time, because the Gaussian distribution is symmetric.

Uncertainty of the past

As well as the Gaussian problem, we have the problem that the future may not be like the past.

  • And we don’t even know if the cards dealt so far are representative of the pack that they come from.

Unfortunately there are numerous sets of parameters that could possibly have generated the historical returns that you’ve observed. You’ll never be entirely sure exactly which set of parameters to use.

All you can do is look at the possible range of parameter values and work out which are the most plausible candidates.

We can ask this question: How likely is it that the average return of the cards in the deck at the start of the investment game was positive, given the cards that have already been dealt?

You need to know how uncertain you are about your estimate of the geometric mean (and therefore how likely it is that is really is positive).

To work this out I used a statistical technique called bootstrapping. First of all I took the historical data, then I created a series of `alternative histories’ by randomly selecting returns from the data.

I then measured the mean return in the alternative history. This process was repeated many thousands of times. This gave me a distribution of mean estimates.

As you would expect, the mean of this distribution is the same as the arithmetic mean of the cards that have been dealt: 0.0857%. In half the alternative histories the mean is higher than this, in the other half lower.

There is a 95% chance [two SD] that the arithmetic mean of the starting deck was between -0.023% and 0.195%. That is quite a wide range, which reflects the fact that you’ve only seen 35 cards.

There is a 94.2% chance that the average of the starting pack was above zero. This is pretty high; if I was you I’d happily start to play this game.


Rob is not a great believer in the standard portfolio optimisation techniques.

[The] techniques are tricky to use correctly, often come out with crazy recommendations, and don’t deal with the problem of uncertainty of the past.

I think it’s unnecessary and even dangerous for most investors to use these techniques.

You start with a list of possible assets to include in your portfolio, and some expectations of future asset returns. You then consider different portfolio weights, and look for the portfolio that is expected to be the best.

The best portfolio will have the highest expected geometric mean return for some level of expected risk [standard deviation of returns].

Historical returns

Historical returns

Rob begins with the classic stocks and bonds portfolio.

The best portfolio will have a large proportion of assets with higher expected returns: so clearly more equities.

But reducing the risk of a portfolio will also make it more attractive, making bonds look like a better option.

These two goals are clearly contradictory. To find the best weights for each asset you need to find the portfolio with the best expected return adjusted for risk.

To do this, Rob uses the Sharpe Ratio (SR), which we’ve written about previously.

  • The SR measures returns per unit risk (volatility).

To simplify the maths, Rob does not subtract the risk-free return from the asset return before calculating the SR.

With the low levels of interest rates prevailing as I write this in early 2017, there is no meaningful difference.

The Sharpe Ratio of US bonds is 0.266 [The geometric SR is 0.225]

Geometric Sharpe Ratios are usually lower than the arithmetic version, because geometric means are lower than arithmetic means.

Sharpe ratio

Sharpe ratio

Equities have the highest Sharpe Ratio as well as the highest geometric mean. Does this mean that the best portfolio should be entirely in equities?

Of course not – diversification (with non-correlated assets) lowers risk.

Because the geometric mean increases when risk falls, it will also be slightly higher for more diversified portfolios; assuming of course that arithmetic returns do not fall too much at the same time.



Portfolio effects

Returns fall as bonds are added:

Stock bond geometric return

Stock bond geometric return

Volatility falls as bonds are added, but the low point uses less than 100% bonds (because of diversification):

Stock bond standard deviation

Stock bond standard deviation

The SR increases until around 65% bonds, but 100% bonds is worse than 100% stocks.

Stock bond sharpe ratio

Stock bond sharpe ratio

So the best portfolio is not obvious:

Three portfolios

Three portfolios

Some people might want the highest SR, but others (including me) will want a higher return than that.

Example portfolios

In the next section, Rob runs through some imaginary portfolios with imaginary assets.

Portfolio 1 has three almost identical assets with high correlations, and the optimal portfolio is an even split.

Portfolio 2 has identical assets with no correlation, and is also split evenly.

  • Rob uses this result later to explain his “hand-crafting” approach to portfolio allocations.

Portfolio 3 has an uncorrelated asset and two highly correlated ones.

  • The uncorrelated asset picks up almost half of the total cash, with the rest split between the other two.

Portfolio 4 has uncorrelated assets, one of which has a higher volatility.

  • The allocation to this asset is lower, but Rob is uncomfortable that it’s hard to see how much lower it should be.

Portfolio 5 has highly correlated assets with one slightly riskier.

  • This riskier asset now gets a zero weighting!

Portfolio 6 has highly correlated assets, one of which has a slightly higher return.

  • Now the entire portfolio is in that asset!

Portfolio 7 has uncorrelated assets, one with a much higher return.

  • Now the weights change almost in proportion to the returns.

We can see that small variations between highly correlated assets leads to massive changes in weightings.

  • But this “tipping point” problem is not so large with uncorrelated assets.
Unstable portfolios

“Using extreme weights and putting all your eggs in a relatively small number of baskets is inherently dangerous.

In the next section, Rob repeats the bootstrapping exercise he carried out to calculate the uncertainty of the historic mean.

  • This time he looks at standard deviation, correlation and the Sharpe Ratio.

Geometric means

Geometric means

The variation in means between bonds and equities is many times larger than the small (0.5%) difference in means that can produce extreme portfolio weights.

Standard deviations

Standard deviations

Standard deviations (volatilities) are less uncertain, and we can be confident that stocks are riskier than bonds (the distributions don’t overlap).

  • We should be less confident that the higher mean compensates for the higher volatility.

Correlations also have less uncertainty than means.

Sharpe ratios

Sharpe ratios

Most importantly, it’s almost impossible to distinguish between the historic Sharpe ratios of stocks and bonds.

  • The average SR is around 0.2

The key problem that this throws up is that standard optimisation techniques can produce a wide range of allocations to equities.

  • In the experiment that Rob ran (which I think optimised for ST rather than maximum return) the range was between 8% and 56% equities.

Things would be even more volatile with assets more highly correlated than stocks and bonds, as we saw earlier.

Rob has three ways of dealing with this:

  1. risk-weighting
  2. “hand-crafting”
  3. assuming that risk-adjusted returns (SRs) are identical for all assets.
Lower future returns

Future returns of most asset classes are likely to be considerably lower than they were in the past.

Rob’s first point is that inflation in the future will be lower than the past, and so nominal returns will be lower.

  • But he also expects real returns to be lower.

The fantastic real returns in stocks and bonds over the last 40 years or so were driven by macroeconomic trends that are unlikely to repeat themselves.

A large fall in inflation reduced interest rates and increased stock price-to-earnings (bond yields to rise.

As Rob notes:

With lower expectations of returns, costs become much more important.

Unrealistic return expectations can also drive overtrading.

Rob provides his own adjusted expected future returns:

Future returns

Future returns

He’s going for 3% real for equities, with an SR of 0.15.

I think this is plausible, but it’s also problematic.

  • It’s below our failsafe SWR of 3.25% pa (and that retirement portfolio is only 75% equities).

Given that the SWR is derived from historical data, I think that Rob’s approach may have ignored the sequencing effects in equity returns.

  • We are not quite at historic highs, so there’s no reason to expect historically low future returns.

Or it could be that the SWR includes the fallback option of running down your capital over 40+ years, in the event that returns are lower than required.

  • I think that real equity returns could be as high as 4% pa over the long-term, but I wouldn’t be shocked if Rob turns out to be right and not me, or if returns were even lower than Rob thinks.

Rob’s adjusted numbers produce a max SR portfolio that is 68% bonds and 32% equities.

This is far from any portfolio that I would ever hold.

  • The SWR retirement portfolio (max long-term returns) is 75% equities.

Note that Rob’s allocation is for a US-only portfolio.

  • International diversification will lower risk, and support higher equity allocations.

It’s been another interesting visit to Rob’s book.

The contrast between blackjack and investing was illuminating, but I’m especially pleased to discover that portfolio optimisation doesn’t work in practice.

  • I’ve been using my own version of “hand-crafting” asset allocations for decades.

I hadn’t realised that SRs are broadly equivalent across assets, but then I have my doubts about SR:

  • it includes upside volatility (or rather, it counts it as bad rather than good), and
  • more importantly, I don’t want the max SR portfolio

I want the highest returns for the level of volatility than I can tolerate.

  • Sortino is a better measure, but less easy to calculate (and so less often calculated).

We’ve also learned that historic (relative) volatility and (relative) correlations are broadly reliable.

Until next time.

Share this with , Google+, Pinterest, LinkedIn, Tumblr, Reddit and StumbleUpon.

Mike Rawson

Mike is the owner of 7 Circles, and a private investor living in London. He has been managing his own money for 35 years, with some success.

Article credit to:

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.