I Know First Algorithm Performance Analysis: Part 1 of the backtests (updated)

dr roitmanCo-Founder & CTO of I Know First Ltd. With over 35 years of research in AI and machine learning. Dr. Roitman earned a Ph.D  from the Weizmann Institute of Science

 I Know First Algorithm Performance Analysis (Part 1)
Date: 09/21/2014

We are now exploring the possibility of managing a hedge fund. To get an idea of potential profitability, we are developing and conducting backtests.

The system forecasts multiple stocks, each in six different time ranges, and each signal comes with the predictability. The number of variables per stock is 12, and we have hundreds of stocks to consider. Thus there is a multitude of possible ways to trade this system. We do not know of any standard backtest that fits our system, and have to develop our own.

We analyze here the results of our Top 10 Stocks selection, and compare them with the non-selected pool. The Top 10 list is based on the double-filtered selection: First, the 95 stocks with the highest average predictability are selected from the general pool, then they are arranged in the order of decreasing signals. Positive signals go on top, negative on the bottom. Then we blank out all the weak signals in the middle of the table, leaving the top strongest signals.

Back test conditions:

  1. We used the actual “Top 10” stocks forecasts sent daily to our customers over 190 days ending at the end of July 2014. To simplify the study, we ignored the trading costs and possible overnight price gaps: buy at the next close price (after the signal), and likewise, sell at the next close price after the opposite signal.
  2. To simplify the study, we combined the 30 days, 90 days and the year signals in one by adding them together.
  3. To reduce the amount of signal swings, we used the past 5 days majority signal: if more than half of the last five forecasts were pointing up, then the signal was up. We would stay in the market until the signal changes direction.

For the first study of 190 days of data, we tried three different rules for trading:  

a) Signal combined with price movement trigger: 5 days majority signal with (in the same direction as) the 5 days stock trend. Exit when either majority signal or the trend change direction.

b) 5 days majority signal against the 5 days stock trend

c) Enter on signal change and exit on signal change.

Top 10 Stocks Strategy

Here are the results for the Top 10 Stocks simulated trades ($1.00 invested per trade):

TOP 10 Strategy

 

 

 

Signal
with trend

Signal
against trend

Just
the signal

Total wins in $$

39.4

42.2

26.6

Total losses in $$

-16.3

-22.4

-7.2

Win/loss ratio in $$

2.42

1.88

3.69

Net Profit

23.1

19.8

19.4

Number of winning trades

287

270

148

Number of losing trades

135

150

45

Total Trades

422

420

193

Win/loss ratio in # of trades

2.13

1.8

3.29

Av. Profit per trade (%)

5.47

4.71

10.05

From this simulation, we can see that simply following the signal is the best strategy. For every dollar lost there are 3.7 dollar won. For every losing trade there are 3.3 winning trades. On average, every such trade brought 10.05% profit.

Using signal/trend combinations as triggers only increases the number of trades and decreases profit per trade (5.47%). Still, such trading in the same direction of trend is better than against trend (4.72% profit).

Comparison Strategy:

For comparison, here are the results for the group of an “All”, our pool of hundreds not selected simulated trades, for which we had signals over the same period of 190 days. ($1.00 invested per trade):

“All”, no selection strategy

 

 

Signal
with trend

Signal
against trend

Just
the signal

Total wins in $$

57.9

49.8

44.1

Total losses in $$

-25.5

-44.9

-20.7

Win/loss ratio in $$

2.27

1.11

2.13

Net Profit

32.4

4.9

23.4

Number of winning trades

387

350

356

Number of losing trades

246

283

212

Total Trades

633

633

568

Win/loss ratio in # of trades

1.57

1.24

1.68

Av. Profit per trade (%)

5.118

0.774

4.12

What is interesting, the second table shows, when trading on signals from general “All”, not screened pool, using signal-trend combination becomes better strategy than “just the signal” (5.1% vs 4.12%). The worst results are if trading with weak signal against the trend (0.77%).

One can see the results of non-selected general trades are still positive, but far less than the Top 10. Thus, our “Top 10” stock list is the clear winner.

Conclusions:

1. The best strategy from this study is to buy when the majority of the last 5 days signals are pointing up, and sell when pointing down.

2. The signal, the basis of the Top 10 selection is important. One should trade when the signal is strong. When trading with weaker signals, (non-top-10 stocks), one should consider the direction of the recent stock movement, and go with the trend, and not against it.

We are conducting now a number of additional, more sophisticated simulations, where we take into account the signal strength, predictability and trends, with more statistical details.

Update:

We are conducting now a number of additional, more sophisticated simulations, where we take into account the signal strength, predictability and trends, with more statistical details. Some preliminary results:

The signal tested was an arithmetic sum (including signal strength) of all signals from 3 days ahead to a year.

Predictability vs. Signal:

The signal and the predictability for each time range were filtered out at different levels.
In one study, if one of the signals had negative predictability, it was ignored, in another study it was not. The result is:

If the signal for a time range is stronger than 256, and the predictability is negative, then the signal is important, if less, it should be ignored.

If the predictability is positive, then stronger signal gives better results (up to a limit).

If the predictability is more than 0.2, then any strength signal gives good results.

Interpretation: strong signal overcomes negative predictability, stronger positive predictability supports weaker signal.

Clarification:

1. The update part is unrelated to the first part of the article. The first part analyses the Top-10 data set, and compares it to a large, non-selected group of signals.

The update part is using the large data set only, and shows only the preliminary findings, which may take time to finalize. We will update you as new findings are confirmed.

2. The first part has also a different definition of a signal: from 30 days to a year (sum of up to three signals, if they appear in the top-10 table. If the signal does not appear in, for instance, in the 90 days forecast top ten table of that day, then it is not counted. If three of the five last days have a positive signal, then the majority signal is positive.

The update part summarizes all the signals, from 3 days to a year ahead, regardless of the top 10 status.

3. The rules of entry and exit are also different. The first part has two conditions, which have to be met for entry: the signal and the trigger, which is a “trend”. The trend here is simply defined as “the last price, minus the price 5 days earlier”.
The update part the entry condition is “”the last price, minus the average of the last 7 days.” Among the variables studied is this difference, between the last average price and the current one, the effect of entering the market at a discount to the average, or at a premium price.

4. Again the update part is tentative. Many additional factors are being studied. The purpose is to find a “sweet spot”, the one rule that works for most of the market most of the time, and shows less variation between different equities.