In this post we’ll describe some experiments that we have done recently on backtesting Stockflix recommendations. As you know, Stockflix has three major - highly correlated - functionalities to:
- Identify the pattern exhibited by a given stock chart (and searching for stocks with similar patterns) [Search]
- Forecast price movement scenarios of a specific stock by running a statistical analysis of the historical matches of the pattern that the stock currently has [Forecast]
- Pick the most interesting stocks for the day, by sorting and ranking the results of calculating the forecasts for the whole stock universe and extracting those exhibiting a well defined behaviour [Today’s Picks]
Stock picking is certainly one of the most intriguing functions of Stockflix for potential investors. In this experiment, we wanted to simulate how ‘buying’ stocks picked by Stockflix algorithms would have performed over a given period of time, subject to certain conditions described below.
Model and Assumptions
As backtesting is a tricky subject which can get very complex, we have made a number of simplifying assumptions in our experiments.
Basically, we have chosen a limited stock universe, i.e. the components of the Dow Jones Industrial Average index (30 stocks), and a specific time period, from November 2007 to July 2012. Also, we only considered a single type of ‘positive’ signals as provided by Today’s Picks function, i.e. those grouped under the Continue to Positive tag in the service. The period considered includes both bullish and bearish stretches for the overall market.
Over that time period, using historical data we have extracted the signals that Stockflix would have provided for the Dow stocks. Given this is a small universe consisting of only 30 stocks, signals are not always available (on average, 20% of the dates in the considered time period show a signal).
Based on those signals, we simulated trading the stock in this pre-defined universe over a period of one year. As the signals we start from are positive we didn’t take into account short-selling strategies.
As we focused on a time period of over four and a half years overall, it turns out there are over 900 rolling years to consider. The following picture should give you the idea.
Therefore, for each year considered, starting from Nov. 15, 2007 to Nov. 15, 2008, and moving forward one day at a time, we ran a simulation of buying, holding and selling a stock list based on parameters provided by Stockflix.
We made a number of additional assumptions to simulate buying and selling stocks:
- Buying and selling is done at end-of-day price
- No trading costs are considered
- Each list holds one stock at any given time
The model is summarized in the following example flow:
Buy and Sell Criteria
We adopted the following criteria for simulating buy and sell operations:
- At any given day, buy a single stock; the selected stock is the one showing a Continue to Positive signal within the Dow Index constituents for that day; if there is more than one stock available, choose the one with higher Up probability
- Sell the stock when either one of these conditions has been met: 1) the stock has reached a target price; the target price is calculated as a percentage (target factor) of the weighted average of Stockflix up probability forecast, or 2) the stock has reached a stop price; the price is calculated as a percentage (stop factor) of the weighted average of Stockflix down probability forecast
- Keep the stock for at least w days before selling; w is calculated as a percentage of the average length of a leg of the pattern the stock had at the time it was selected
Here is an example of target factor calculation. Suppose we have this forecast:
The current price is 120.85. The continuation target price weighted by the upward (green) probabilities calculated by Stockflix is 145.7. The target factor is a percentage between 0 and 100 applied to the difference between the target and current price.
A target factor of 50% means that the stock is sold as it reaches 133.27 (i.e. an increase of 0.5 x (145.7 - 120.85) on current price). A similar definition holds for the stop factor, using the downward (red) probabilities.
We have introduced the target factor and the stop factor as we noticed that using a percentage of the target price as calculated by Stockflix instead of the plain average or a specific value yields better results overall.
Results
Given the above (admittedly strong) hypothesis, we have run a number of simulation batches, repeatedly varying the two parameters driving the target price and the stop price.
Each batch consisted of the above 900+ steps, each step in turn corresponding to trading the stocks in the Dow Jones universe for one specific year. The result of a simulation batch is how many times the Stockflix selection beat the corresponding Dow Jones Industrial Average index out of the 900+ years tested.
After extensive simulation, it turns out that the best parameter combination has a 70% target factor and a 30% stop factor. With this settings, lists based on Stockflix picks outperform the index in about 93% of the cases.
The following is a chart that compares the performance of Stockflix picks (in red) to the index (blue) for one random year with the above parameters:
Further testing should be done, particularly to extend the model to include multiple stock holdings, to analyze other signals such as Reverse to Positive, to test different stock indexes, to include trading costs, to carry out simulations over longer and shorter periods, etc.
However, these results provide us with increased confidence that the forecast method based on statistical analysis of pattern matching as modeled by the Stockflix service is indeed valuable and can provide to subscribers new investment ideas which are worth exploring.
