Ensemble strategy: Play if All Agree

30May11

We decided to run various combinations of the ensembling when the participating members should all agree on the next day Up or Down decision. Obviously, if not all of them agrees, the strategy stays in cash.

Parameters:

NensembleGroupMembers = 5;
int nTest = 5;

Note that we also run the NensembleGroupMembers = 1; experiments (that took another week), but we don’t present them here in the blog. Those wore inferior results (as expected).

We run 11 different experiments with various ANN combinations; the result can be found in the next tables.
PV (portfolio value):

dStat:

maxDD:

PvMAPEfrMul (PV Mean Average Percentage Error From Multiplicative Approximation, the smaller the better; the smoother the PV):

Note that the performance numbers here in the next section is the average of 4 cells: the maxEpoch=49, 99; and nNeurons=1,2 combos. Because the most likely is that we would play these parameters.

The strategies and — in a little chaotic mode — some performance numbers that lead us to pick the winner. (Sorry, it is quite chaotic. Feel free to jump to the notes section.)

A. only day and T-0 inputs (for clarity)
– ANN(T-0) agree ANN(day), nEnsembleRepeat = 1 (PV:50.9, dStat:59.16%, maxDD:41.39%)*
– ANN(T-0) agree ANN(day), nEnsembleRepeat = 11 (PV:70.6, dStat:58.16%, maxDD:42.55%) (PV70: better than nEnsembleRepeat = 1 PV60; maxDD42.55% (worse than the repeat=1, but OK) the PV compensates; zeroFrq: 25.20%)
– ANN(T-0) agree ANN(T-0), nEnsembleRepeat = 1 (PV:427.5, dStat:57.02%, maxDD:50.88%)
– ANN(T-0) agree ANN(T-0) agree ANN(T-0), nEnsembleRepeat = 1 (PV:446.8, dStat:57.22%, maxDD:48.8%)

B. T-1 input too
– ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1 (PV:20.91, dStat:59.63%, maxDD:35.08%)**
– ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 11 (PV:30.51, dStat:58.57%, maxDD:39.33%)
– ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1 (PV:23.6, dStat:60.14%, maxDD:32.79%)** (considering only the 1neuron case: PV: 29.22)
– ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 11 (PV:30.92, dStat:58.91%, maxDD:40.68%)

C. Can we improve our best 2D ANN?
– ANN(T-0, T-1) with ANN(day), nEnsembleRepeat = 1 (PV:69.5, dStat:59.3%, maxDD:35.71%)** // Avg PvMAPEfrMul is good
– ANN(T-0, T-1) with ANN(T-0), nEnsembleRepeat = 1 (PV:591.2, dStat:57.95%, maxDD:41.01%)
– ANN(T-0, T-1) with ANN(T-1), nEnsembleRepeat = 1 (PV:53.25, dStat:57.07%, maxDD:38.29%)

1.
Notes about nEnsembleRepeat = 11:

– in General nEnsembleRepeat = 11 (instead of 1) plays the market on more days. It reduces zeroFrq, and dStat, but increases the PV and maxDD. Because it generally increases the maxDD, I would like to avoid it for this safe strategy. Currently we would like to obtain a very safe, smooth strategy with as less maxDD as possible. We don’t really care about PV. For that, we have other high-performing strategies.

– in our previous study that we did for Matlab, we concluded that nEnsembleRepeat = 11 is helpful based on 4 years backtest in Matlab.

At that time, we concluded that nEnsembleRepeat=11 improves the performance. However, we haven’t considered other effects, like maxDD. That conclusion is still valid in this 24 years backtest.
+ It improves the performance (PV), mostly because it is on the market much more. And because it is on the market more, and we have an edge of 60% dStat, after a while, we can accumulate more profit.
+ However, it is detrimental to the maxDD and smoothness. Why? Mostly because of randomness. (see the excel table that compares the repeat=111 case for nNeurons 1 and 2: with 2 neurons the 111 ensembles can give very random results. With 1 neuron the randomness is acceptable. This randomness generates hectic trades and more volatility in PV)
So, in this safe strategy, we will omit using it. (using nEnsembleRepeat=11)

– probably, with the AllAgree strategies, increasing the nEnsembleRepeat to 11 or 111 is not a good idea, because, in that 111 prediction,
It can happen that most of them is ZERO, but
many of them can be random; and it is the SUM(SIGN()) can be randomly positive or negative.
Also, what I found, that nNeurons=1, significantly decreased the randomness over nNeuorons2.
see The screenshot here:

Considering the screenshot, our conclusion of it is here:

prediction for the endday 2011-05-27
Conclusion: 1 neuron significantly decreased randomness
Code:
public int maxEpoch = 49; // use 40-50
static int generalNensembleGroupMembers = 5;
public EnsembleGroupSetup[] ensembleGroups = new EnsembleGroupSetup[] {
new EnsembleGroupSetup() { Nneurons = 1, NNInputDesc = NNInputDesc.BarChange, BarChangeLookbackDaysInds= new int[] { 0, 1 }, NensembleGroupMembers = generalNensembleGroupMembers },
new EnsembleGroupSetup() { Nneurons = 1, NNInputDesc = NNInputDesc.WeekDays, NensembleGroupMembers = generalNensembleGroupMembers },
};
public EnsembleAggregationStrategy ensembleAggregation = EnsembleAggregationStrategy.PlayOnlyIfAllGroupAgree_onGroupSumSign;
public int nEnsembleRepeat = 111;

2.
Notes about selecting our safe strategy:

– T-1 input is generally helpful in reducing maxDD. So, I would like to have it in the final (safe) strategy.

– weekDay input: we would also like to have this input in the strategy, because it gives an orthogonal opinion (not technical, but calendar factors) into our decision. And we like to aggregate different orthogonal opinions.

– surprising to see such a good maxDD performance for ‘ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1’.
maxDD: 32.79%.
Why? Because it has the best dStat: 60.14%.
Why does it have the best dStat?
Because it is the combination of 4 ANNs. And it plays only if all 4 agree. So, it is on the market quite rarely. If we really want a safe strategy, pick this one.

– based on maxDD, we have 3 good candidates left. Compare them by the PvMAPEfrMul:
– ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1 (PV:20.91, dStat:59.63%, maxDD:35.08%, PvMAPEfrMul: 65.74%)
– ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1 (PV:23.6, dStat:60.14%, maxDD:32.79%, PvMAPEfrMul: 57.72%)***
– ANN(T-0, T-1) with ANN(day), nEnsembleRepeat = 1 (PV:69.5, dStat:59.3%, maxDD:35.71%, PvMAPEfrMul: 57.4%)***

– very difficult to choose between the last two, but let’s compare the maxDD, based only the 1neuron measurements (omit 2 neuron cases), because the 1 neuron case is what we would play, because that is the least affected by randomness.
– ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1 (PV:23.6, dStat:60.14%, maxDD:32.79%, PvMAPEfrMul: 57.72%, zeroFrq: about 30%, maxDD_1neuron:30.1%)****
– ANN(T-0, T-1) with ANN(day), nEnsembleRepeat = 1 (PV:69.5, dStat:59.3%, maxDD:35.71%, PvMAPEfrMul: 57.4%, , zeroFrq: about 40%, maxDD_1neuron:34.01%)
So, it seems the 4 ANN version is smoother, more stable. It is less time on the market, but still with that the 1 neuron case it achieved PV: 29.22. (that is about 16% CAGR without any leverage)
We would like to play this one as our safe strategy. We expect about 30% maxDD from this strategy in the future, which is moderately good.

-As a visual proof, compare the PV chart of the 2 strategies:

“ANN(T-0, T-1) with ANN(day), nEnsembleRepeat = 1”:

“ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1”:

– note the amazing dStat: like 60.4% directional accuracy. With a daily prediction! It means that we are significantly better than random (50%). That is our edge!
It means that every 10 days when we are on the market, we can correctly predict at least 1 day direction. Isn’t it excellent? (other successful ANN applications are happy with 70% dStat with a monthly prediction. But note how much more profit you can accumulate with a 60% accurate daily rebalancing than with a 70% accurate monthly rebalancing.)
With one caveat: that 40% of the time we are in cash, because we are not confident enough about the prediction. But this is fine. 60% directional accuracy on a daily prediction is significant.

compare this strategy to the high performance strategy of the last post.
That was the ANN(T-0 %change, T-1 %change) with PV: $865 (investing $1 in 1987), that is 35% CAGR. MaxDD: 40%. PvMAPEfrMul: 130%.
Here, we developed a low performance strategy (CAGR 16%, maxDD: 30%, dStat: 60%, PvMAPEfrMul: 57.72%). The aim was stability here.

As a conclusion: ‘play if all agree’ ensembling can decrease maxDD (by -10%), MAPE (good), but the cost is some performance. (by -19%)
However, we feel it still good to play; the degraded performance is OK. It is a trade-off we accept.
We would like to play both of them concurrently. One would double our portfolio every 3 years; the other one would give us stability in bad times. Just look at how smooth the chart of the low performing strategy is.

– somehow, we have the feeling that
‘ANN(T-0) agree ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1’ ensembling can be almost equivalent to the case
‘ANN(T-0) agree ANN(T-1) agree ANN(day), nEnsembleRepeat = 1’, when we double the ANN(T-0) nEnsembleGroupMembers.
If we double the ANN(T-0) nEnsembleGroupMembers, we could have achieved the same results. And it is conceptually simpler strategy, so in the future, we may compare the two, and switch to this conceptually simpler one.

– basically we got the same result as in 2010 December study, in our previous post of the All-Agree ensembling: (in the 4years backtest) (24 years backtest is simply not possible with Matlab) (takes months)
– the best is the pure currDayChange: 28%CAGR (with T-1 input too)
– the 3ANN version (or the 4ANN version) (if all agree) could achieve 12%CAGR or 19%CAGR; (or here 16%)
the All-Agree combined ANN cannot be better in profit than the best ANN (than the best expert), but by sharing the decision among different ‘experts’, we can mitigate the risk (maxDD is 30% instead of 40%)
– We can even mitigate very serous maxDDs by the All-Agree strategy: for example, the pure ANN(T-0) (without T-1 input) can give 65% maxDD(nNeurons=2, ensemble5, repeat11 case)), by combining this with another ANN by the All-Agree strategy this 65% maxdDD can be decreased significantly.

Advertisements


No Responses Yet to “Ensemble strategy: Play if All Agree”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: