(un-) Importance of directional accuracy and DV2

02Dec10

So far, we could achieve about 52% D_stat. That means we could correctly predict the direction only 52% of the time. That is not too much you may reckon. It only means that during a 250 trading days of a year, our edge is only 2%. That is 5 days. So, we may speculate that there is a very tiny difference between our ‘clever’ prediction and a random monkey throwing darts for direction selection. However, there are a couple of things worth considering.
In spite of the 52% directional accuracy looks tiny, it is valuable.

1.
Let’s suppose that our random monkey (with uniform random distribution) would predict Up direction 50% of the time and Down direction 50% of the time. Nothing wit here, we only predict blindly with pseudorandom numbers. That would be random, but our directional accuracy (D_stat) would be less than 50%. The reason is that Up days are more frequent than 50%. In long term Up days occurs about 56% of the time. So a random predictor prediction Up days with 50% frequency will be worse than average.
Do you see now how difficult is to reach even 50% D_stat? It is very rarely considered. One can easily think that a random predictor (like a toss of a coin) is accurate about 50% of the time, but it is false.

2.
We can achieve 50% D_stat easily by this blind random monkey predictor if we create a uniformly distributed random predictor with 56% probability of predicting Up days. Because real life has also 56% Up days, our predictor and real life match, and we can achieve 50% D_stat.

3.
One can say that we can easily achieve even 56% D_stat. How? It is easy. The predictor should forecast every day as an Up day deterministically. 🙂 This will surely achieve 56% D_stat, but it is fundamentally a buy&hold approach, and we know how bad its performance is sometimes. (lost decade after 2000)
And this exactly reveals why the D_stat is just partly a good performance measurement. We can easily have a 56% D_stat, but it is not the aim. It is not the aim to be right on the direction for most of the days. The aim is to be right on those days when we expect high Up or Down movements. Catching only those high moving days, we can achieve 5 times more CAGR than buy&hold, while our D_stat can be even less than 40%. Like George Soros, we don’t have to be right many times, but when we are right, we should gain huge, when we are wrong, we should lose petty.

I hope this post reveals why we should always consider the CAGR and D_stat measurements in unity when we consider that one strategy is better than another.
A high D_stat alone doesn’t mean it is a profitable strategy (buy&hold has 56% D_stat)
A high CAGR alone doesn’t mean that it is a consistently good strategy. The Varadi DV2 has a huge CAGR in 2008 (90%?), but was mediocre in another years. Having high CAGR means that regime was favourable to the strategy, but it is only a lucky period, subperiod, and there is no guarantee that lucky period will last. (MR stopped working in 2009, 2010).

4.
We were curios what is the D_stat of the successful DV2 (DVB) strategy from Varadi. In the last 5 years backtest, the CAGR was 29.57%, while D_stat is 51.88%. We were disappointed a little bit about this D_stat. So, the DV2 is not as good after all. Most of its CAGR gain is contributed only in the 2008 period. Without the 2008 period, we guess, its CAGR would be about 18%.

In spite of what we thought about our petty D_stat performance, the DV2 51.88% D_stat tells us that when our backtest show 52% D_stat and 15% CAGR, we are quite good. And we shouldn’t forget that our ANN is adaptive that cannot be said to most of those rule based strategies like DV2. So even if we achieve exactly the same performance numbers as a rule based strategy, I would still sleep better with an adaptive approach.

Advertisements


No Responses Yet to “(un-) Importance of directional accuracy and DV2”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: