Executive Summary

In this stock market forecast evaluation report, we will examine the performance of the forecasts generated by the I Know First AI Algorithm for top S&P 500 stocks which were daily sent to our customers. Our analysis covers the time period from 19 June, 2019, to December 8, 2019. The report also demonstrates how our Global Stock Picking method compares to our Daily Forecast model and how those generate predictions resulting in returns that beat S&P 500 even during the volatile times in 2019.

Top S&P 500 Stocks Evaluation Highlights:

• Stock market forecasts that were generated by our Daily Forecast model obtained positive returns for all time horizons and outperformed the S&P 500 for all time horizons (3-days to 1-year).
• We observed a consistent positive effect of signal-based filtering, besides one exception in the 90 days time horizon, additional screening by signal always resulted in higher performance.
• The Global Model improved the performance even further – our algorithm’s forecast according to this new model enabled a more flexible trading strategy that generated robust returns compared to both the S&P 500 Benchmark and the traditional model performance on longer horizons.
• The highest return of 6.86% was obtained by applying the Global Model on the 90-days time stock market forecast horizon.

The above results were obtained based on the stock market forecast evaluation over the specific time period using consecutive filtering approach – by predictability, then by signal, to give general overview of the forecasting capabilities of the algorithm for specific stock universe. In addition, we conducted alternative filtering based on our Global Model which is based on special signals filtering and will be explained further in a respective section. We will start with an introduction to our stock picking and benchmarking methods and then apply it to the stock universe of the S&P 500 index covered by us in the “S&P 500 Companies” Package. After introducing the evaluation methodology in detail, we develop the analysis and present the evaluation results with relevant conclusions. This stock market forecast evaluation is part of our continuous studies of live I Know First’s AI predictive algorithm performance.

Evaluating S&P 500 Companies Forecasts: The Stock Picking Method

In this evaluation analysis, we use two stock picking methodologies – our Daily Forecast model and our new Global Model forecast method:

The Daily Forecast Model

We take the top 30 most predictable assets, and then we apply a set of signal-based filters: top 20, 10 and 5 based on predictability.

By doing so we focus on the most predictable assets on the one hand, while capturing the ones with the highest signal on the other.

We use absolute signals since these strategies are long and short ones. If the signal is positive, then we buy and, if negative, we short.

For example, a top 30 predictability filter with a top 10 signal filter means that on each day we take only the 30 most predictable assets, and then we pick from them the top 10 assets with the highest absolute signals.

Our Global Model Method For Top S&P 500 Stocks Market Forecast

The new stock-picking method takes all 10,200 assets, the global set, that are forecasted by I Know First. The assets are then filtered by predictability and the top 30 most predictable assets are selected. Then these 30 stocks are scanned to see if there are any S&P 500 stocks – if there are such stocks, they are selected and then, based on the number of available assets, the signal filtering is applied at available levels to arrive at the top S&P 500 stock picks.

As mentioned before, the asset universe under consideration is the full 10,200 set of assets covered by I Know First forecast and includes stocks, commodities and currency pairs. By implementing this new stock-picking method, we are selecting only the most predictable assets and the ones with the highest signals. Therefore, unlike our Daily Forecast picking method, the Global Model method provides significantly stricter filtered forecasts for with narrower assets’ set. Note, that in case the Global Model filtering procedure is not passed by any of the S&P 500 stocks on a specific day, the system does not generate the forecast on this date until the stronger predictions come in.

About the I Know First Algorithm

The I Know First self-learning algorithm analyzes, models, and predicts the capital market, including stocks, bonds, currencies, commodities and interest rates markets. The algorithm is based on Artificial Intelligence (AI) and Machine Learning (ML) and incorporates elements of Artificial Neural Networks and Genetic Algorithms.

The I Know First Market Prediction System models and predicts the flow of money between the markets. It separates the predictable information from any “random noise”. It then creates a model that projects the future trajectory of the given market in the multidimensional space of other markets.

Since the model is 100% empirical, the results are based only on factual data, thereby avoiding any biases or emotions that may accompany human derived assumptions. The human factor is only involved in building the mathematical framework and providing the initial set of inputs and outputs to the system. The algorithm produces a stock market forecast with a signal and a predictability indicator. The signal is the number in the middle of the box. The predictability is the number at the bottom of the box. At the top, a specific asset is identified. This format is consistent across all predictions.

Our algorithm provides two independent indicators for each asset – signal and predictability. The signal is the predicted strength and direction of movement of the asset. This is measured from -inf to +inf. The predictability indicates our confidence in that result. It is a Pearson correlation coefficient between past algorithmic performance and actual market movement. This is measured from -1 to 1.

You can find a detailed description of our heatmap here.

The Stock Market Forecast Performance Evaluation Method

We perform a stock market forecast evaluation on the individual forecast level. This means that we calculate the return of each forecast we have issued for each horizon in the testing period. We then take the average of those results by strategy and forecast horizon.

For example, to evaluate the performance of our 1-month forecasts, we calculate the return of each trade by using this formula:

This simulates a client purchasing the asset on the day we issue our prediction and selling it exactly 1 month in the future from that day.

We iterate this calculation for all trading days in the analyzed period and average the results.

Note that this stock forecast evaluation does not take a set portfolio and follow it. This is a different stock forecast evaluation method at the individual forecast level.

The Hit Ratio Method

The hit ratio helps us to identify the accuracy of our algorithm’s predictions.

Using our Daily Forecast and Global Model asset filtering, we predict the direction of movement of different assets. Our predictions are then compared against actual movements of these assets within the same time horizon.

The hit ratio is then calculated as follows:

For instance, a 90% hit ratio for a top 30 predictability filter with a top 10 signal filter would imply that the algorithm correctly predicted the price movements of 9 out of 10 assets within this particular set of assets.

The Benchmarking Method – S&P 500 Index

In order to evaluate our algorithm’s performance in comparison to the US market, we used S&P 500 as a benchmark.

The S&P 500 measures the stock performance of the largest 500 companies by market cap listed on different stock exchanges in the United States. It is one of the most followed equity indices and is frequently used as the best indicator for the overall performance of US public companies, and the US market as a whole. S&P 500 is a capitalization-weighted index, the weight of each company in the index is determined based on its market cap divided by the aggregate market cap of all the S&P 500 companies.

For each time horizon, we compare the S&P 500 performance with the performance of our forecasts after the filtering processes described above. For evaluating the performance of the forecasts that are filtered based on Global Model method, we compare those with the average S&P 500 performance only when the Global Model filtering resulted in forecasts for S&P 500 companies. For example, if after our Global filtering the model recommended buying a given asset and hold it three days on five different occasions, the S&P 500 performance we will use as a benchmark will be the average of returns only in these five intervals of three days.

Performance Evaluation – Overview

We conduct our research for the period from 19 June (date of the first ever forecast generated for this stock universe with Global Model) to 8 December, 2019 for both Daily Forecast method and for Global Model. Following the methodology, as described in the previous sections, we start our analysis by computing the performance of the algorithm’s signals for time horizons ranging from 3 to 90 days with Daily Forecast model using predictability and signal filtering consecutively applied on S&P 500 stock universe. Then we continue with the evaluation results for stocks picked with Global Model approach and consider what are the advantages and disadvantages of these models.

Performance Evaluation – The Signal Indicator Effect

Once filtering by predictability is done, we utilize the signal indicator in our asset picking method to achieve the maximum forecast performance. It is important to measure it with respect to the benchmark, i.e. how the selected assets out-perform the benchmark, and for that, we will apply the formula:

We further filter and rank the assets based on absolute signal strength to the Top 20 assets under Daily Forecast model, which were already filtered by predictability.

In this article, we examine the kind of effect the signal filter has. To do so, we have filtered the stocks by predictability, selected the 30 most predictable stocks from S&P 500 stock universe. The stocks are not necessarily all long or short positions, but they are a mix of both. That is because the I Know First algorithm believes in getting a good return for both bull and bear markets. Therefore, we applied filtering by signal strength to the top 30 assets filtered by predictability.

In parallel, we apply the Global Model method and compare the returns to S&P 500 performance – the average return of the S&P 500 measured for the same dates that the S&P 500 stocks passed through Global Model filtering.

Top S&P 500 Stock Results: Average Return, Hit Ratio and Out-performance

As can be seen above, by applying the top 30 predictability-based filtering method, we achieved positive returns for every time horizon, even without further filtering. The effect of almost all filters in the signal-based filtering set is positive, meaning that for every time horizon, filtering by signals almost always results in higher returns. An addition of ‘top 5’ signal-based filter provides the best performing with higher returns relative to all other filtering rules in the set across all time horizons. Those results indicate that the signal effect on forecast return is strong and consistent.

As for Global Model forecasts’ performance, the observed average returns are far better than S&P 500’s for the respective time frames. We see the results’ improvements with expansion of the time horizon, and most importantly, the marginal performance improvement represents a trend of out-performing the benchmark by at least 2 times.

The signal effect on the hit ratio is less significant, but still representative of the general trend. For most time horizons, changes are low and sometimes negative after additional filtering. We observe the peak hit ratio for Daily Forecast Model on 30-days and 90-days horizons, while Global Model arrives at its peaking hit ratios on 7-days and 90-days horizons.

Finally, in accordance with the methodology described in the above section the out-performance of each model against S&P 500 performance was calculated. One can see that the Daily Forecast model successfully and consistently out-performs the benchmark and we record that the signal filtering achieves the optimal results at the strictest filtering level of top 5 assets. As expected, the returns generated when Global Model approach is used provide significantly higher out-performance as it allows the forecast to focus only on the most predictable and most promising market opportunities tailored for to specific forecast horizon. As a result, we see amazing out-performance of 550% and 323% on 3-days and 7-days horizons, respectively. An interesting observation is that with the expansion of the time horizon the out-performance presented a declining trend, although still showing 117% out-performance on 90-days horizon above S&P 500. Due to the freshness of Global Model approach it is early to judge about the consistency of this trend and more data may correct it.

Conclusions

This evaluation report presents the performance of I Know First’s algorithm as reflected in the average returns for all time horizons after the relevant filtering processes. The results of this analysis demonstrated a strong overall performance of forecasts that are filtered by predictability, together with significant and consistent improvement that was obtained by the signal-based filtering method. The analysis also demonstrates the high effect of the Global Model method as reflected across all relevant metrics: return, hit ratio and out-performance.

The Global filtering method generated high returns and outperformed S&P 500 for each time horizon by a high margin. Due to the bullish rally the US stock market experienced during 2019, fund managers have been struggling to out-perform the market. Nevertheless, I Know First Algorithm’s forecasts, combined with the Global Model filtering method developed by our research team, out-performed the S&P 500 index for each time horizon, from 3 to 90 days. We look forward to new market data in the following months and will monitor the changes in performance trends that are going to be communicated to our investors and subscribers in the follow-up reports.