I Know First Live Forecast Evaluation: Based On New Global Volatility Adjusted Approach

I Know First Live Forecast Evaluation

Executive Summary

As of December 2018, I Know First finished a performance evaluation of the live AI-based predictive forecasts for the new S&P 500 stock strategy sent to customers. This evaluation clearly demonstrates the consistent out-performance of I Know First’s forecasts vs S&P 500 index and previously used stock picking methods. This evaluation is part of our continuous studies of live I Know First’s AI predictive algorithm performance.


S&P 500 Highlights:

  • Selecting the S&P 500 stocks with the top 5 adjusted signal strength provides the highest return consistently beating other filters for each time horizon.
  • There is a clear correlation between the length of the time horizon and the return as the length increases the accuracy and performance of the I Know First algorithm dramatically improves.
  • For the 3-month time horizon all signal filters outperform the benchmark with the top 5 signal providing a return of 42% more than the benchmark.

Evaluation Period and Universe

In this Live Forecast Evaluation Report, we will examine the performance of the forecasts generated by the I Know First AI Algorithm for S&P 500 stock universe. Our analysis covers the time period from January 1, 2018 – December 1, 2018. We present below the optimal stock picking strategy for this period. This period was chosen since this was a milestone in which I Know First introduced a major new version of the stock picking method for S&P 500 investment universe. It includes analysis of all the predictions generated by the algorithm and sent daily to I Know First’s clients over this time period. The results of this study illustrate the significant positive effects of the algorithm’s continuous improvement utilizing advanced machine learning and AI capabilities.

The asset universe under consideration is the full 10,200 set of assets that I Know First forecast these include stocks, commodities and currency pairs. Each daily forecast sent to I Know First’s clients includes predictions for the following time horizons: 3 days, 7 days, 14 days, 1 month, 3 month and 1 year.

About the I Know First Algorithm

The I Know First self-learning algorithm analyses, models, and predicts the stock market. The algorithm is based on Artificial Intelligence (AI) and Machine Learning (ML) and incorporates elements of Artificial Neural Networks and Genetic Algorithms.

The system outputs the predicted trend as a number, positive or negative, along with a wave chart that predicts how the waves will overlap the trend. This helps the trader to decide which direction to trade, at what point to enter the trade, and when to exit. Since the model is 100% empirical, the results are based only on factual data, thereby avoiding any biases or emotions that may accompany human derived assumptions. The human factor is only involved in building the mathematical framework and providing the initial set of inputs and outputs to the system. The algorithm produces a forecast with a signal and a predictability indicator. The signal is the number in the middle of the box. The predictability is the number at the bottom of the box. At the top, a specific asset is identified. This format is consistent across all predictions.

Our algorithm provides two independent indicators for each asset – Signal and Predictability.

The Signal is the predicted strength and direction of movement of the asset. Measured from -inf to +inf.

The predictability indicates our confidence in that result. It is a Pearson correlation coefficient between past algorithmic performance and actual market movement. Measured from -1 to 1.

Here is the detailed description of the heatmap.

Our New Stock Picking Method

The new stock picking method takes all 10,200 assets, the global set, that are forecasted by I Know First. The assets are then filtered by predictability and the top 30 most predictable assets are selected. A volatility measurement is then conducted. These top 30 assets are then adjusted for this volatility measurement. The S&P 500 stocks are then selected from this final set of top 30 global most predictable assets with volatility adjusted signal.

As mentioned before, the asset universe under consideration is the full 10,200 set of assets covered by I Know First forecast and includes stocks, commodities and currency pairs. By implementing this new stock picking method, we are selecting only the most predictable assets and the ones with the highest signals. However, there could be occasions where no S&P 500 stocks fall in this final set of 30 assets. If this was the case, a stock selection will not take place on such occasion.

The Performance Evaluation Method

We perform evaluations on the individual forecast level. It means that we calculate what would be the return of each forecast we have issued for each horizon in the testing period. Then, we take the average of those results by strategy and forecast horizon.

For example, to evaluate the performance of our 1-month forecasts, we calculate the return of each trade by using this formula:

This simulates a client purchasing the asset based on our prediction and selling it exactly 1 month in the future.

We iterate this calculation for all trading days in the analysed period and average the results.

Note that this evaluation does not take a set portfolio and follow it. This is a different evaluation method at the individual forecast level.

The Benchmarking Method

The theory behind our benchmarking method is the “Null hypothesis“, meaning buying every stock in the particular asset universe regardless of our I Know First indicators.

In comparison, only when our signals are of high signal strength and high predictability for a specific stock, then it should be bought (or shorted).

The ratio of our signal’s trading results to benchmark results indicates the quality of the system and our indicators.

Example: A benchmark for the 3 days horizon means buy on each day and sell exactly 3 business days afterwards. We then average the results to get the benchmark. This is to get an “apples to apples” comparison.

Performance evaluations

In the following charts you will see proof that our predictability and signal indicators can be used to beat the benchmark and obtain alpha.

We will start by investigating the influence of the I Know First signal indicator on the trade results.

In the following chart you can see our out-performance of our top 50 strategy compared to the S&P 500 index benchmark.

Performance Summary

We calculated the out-performance by using the following formula:


Average returns per time horizon for different signal filters (3 days to 14 days)


Average returns per time horizon for different signal filters (1 Month to 3 Months)


S&P 500 – out performance for 3-14 day time horizon


S&P 500 – out performance for 1-3 month time horizon


Average hit ratio per time horizon, predictability & signal filters

The above clearly shows to what extent the hit ratio can exceed 50% when following the I Know First signal and predictability filters and it further shows that for the longer time frames the top 5 and top 10 filters consistently outperform the top 20 and all signals filter with higher hit ratios.


The goal of this analysis is to demonstrate the out-performance of the new stock picking method outlined above in the report where the entire global set of assets are filtered by predictability, following this a volatility measurement is taken and the signals of the top 30 most predictable assets are then adjusted based on this volatility measure. The S&P 500 stocks are then selected from this.

This analysis shows that for all the different time horizons, using this new method with I Know First’s indicators significantly outperforms the benchmark, especially on the longer time horizons, such as the one month and three-month time frame. It also has a significant hit ratio particularly for the top 5 signal filter which is 80% or above in each time frame aside from the 2-week horizon where the hit ratio fell short of 80% by just over 2% with a hit ratio of 77.78%.