The backtesting process is one of the most important steps that you should perform before investing any money in a trading system. It will tell you whether your trading system has been profitable in the past or not. This step is essential in order to validate and gauge the effectiveness of your strategy. The problem with backtesting is that any small mistake may result in inaccurate and unreliable outcomes, which will ultimately lead to bad trading decisions. This is why, I am going to show you some mistakes you should avoid in order to get simulation results that are as accurate and reliable as possible. Before and after performing any backtest, you should make sure that every point is addressed, verified and fixed. These mistakes have to do with data, report and statistics. Few trades generated by your trading strategy Statistical significance is a very important concept is statistics. A simulation report is called "statistically significant" if it is unlikely to have occurred by chance. You want to minimize as possible the part of chance in your simulation results. The total number of trades is an important measure here. The more trades we have the more our results are statistically significant. There is no single level at which the statistical significance of a trading system changes, but we strongly suggest that your trading system generates at least 50 trades during the simulation. If it does not then try the following: - Add more stocks to backtest - Choose an older start date - Tweak your buy/short rules (make them less restrictive) The number of trades is available under the "Statistics" tab of the trading system report generated by QuantShare.  Ignoring the top performing stocks After backtesting your strategy, get the list of top performing trades ("Trades" tab in QuantShare) and check the performance of each one. We want to make sure that the strategy performance is not due to some outperforming trades. Here is what you should do: - Take 5% of your trades (In case you have 100 trades, get the five top performing ones) and note each stock - Create a copy of your trading system - In "Symbols Selection" control, select a new "Custom Symbols" list then enter the different stock names - Click on the cell under the first column to ignore these stocks - Backtest your trading system again and make sure that it is still profitable If the result dramatically decreases, ignore this strategy or perform some changes to make it perform better.  Not using a properly adjusted historical data Using unadjusted stock data with some analysis techniques such as backtesting is likely to produce wrong results. The backtesting process requires that your data be adjusted for splits, dividends and any other corporate actions. In order to avoid this mistake, you must use a data downloader that gets adjusted data. Here are some of these downloaders: Historical Stock Market Data Quotemedia Historical Quotes Market Data List of recent corporate actions: https://stockcharts.com/charts/adjusthist.html Investing in illiquid stocks Here, we want to make sure that our backtester is not buying illiquid stocks that would be impossible to buy in practice. To do this, we can add a filter rule that prevents the trading system from purchasing any stock if its current volume is lower than a certain amount. Let us say, we have $200,000 and we are planning to invest in 20 stocks. This will result in about $10,000 investment in each stock. I usually want the backtester to buy a maximum of 10% of the average daily volume. In our example, we will have to add a rule than prevents us from buying a certain stock if its average volume is lower than $100,000. buy = buy and sma(volume, 10) * close > 100000; Penny stocks Penny stocks are very volatile; therefore trading these stocks is very risky and the backtesting results of penny stocks-based trading systems are more error-prone. For this reason, I tend to avoid penny stocks by adding the following rule: buy = buy and close > 2; If you goal is to trade penny stocks then simply ignore this paragraph.  Ignore In-Sample & Out-of-sample Analysis This topic is particularly important when optimizing a trading strategy. Out-of-sample testing is a technique used to minimize curve fitting. Curve fitting refers to over-optimized and over-optimistic backtesting results. This can happen when the trading system's indicators and rules combination produce good results by pure chance. The technique consists of backtesting a trading system on a given period (in-sample) and then verifying the trading system results on another period (out-of-sample). For example: In-sample: Year: 2000 / 2008 Out-of-sample: Year: 2008 / 2012 In order to validate your strategy, the out-of-sample testing should produce an acceptable return (compared with the in-sample result). The biggest challenge here is to choose the right periods. Make sure there are at least one bullish and one bearish period in each sample. Survivorship bias The survivorship bias is the tendency to exclude failed or delisted companies during a simulation. Consequently, your simulation result will be probably inflated. To avoid this bias, you need to download end-of-day data for these delisted companies and thus construct a survivorship bias free database. QuantShare has a list of delisted stocks that can be found here: US delisted symbols. Once the symbols are added to your database, use the following item to download historical data: Historical Quotes for Delisted US Stocks. Look-ahead bias This bias is created when you use data that would not have been known during the simulation period. This often leads to inaccurate and wrong results. An example would be to reference future bars in your formula: buy = ref(close, -1) > 10; The above line references a future bar (close price at next bar). You should also make sure that the indicators you are using do not have the look-ahead bias. An example of such as indicator is the ZigZag. This indicator makes use of future bars to create high and low swings. How to avoid look-ahead bias: - Right click on a chart - Select "Edit Formula" - Type your formula - Click on "Look-ahead" button - Make sure that the "Bars to the future" is equal to 0. Be aware, that some custom function (rare) may reference future bars and returns 0 in "Bars to the future" field. If you have any doubt, please post your function name (+ link to the trading item) on the forum. Ignoring broker commissions Each time you trade a stock, an ETF or a futures contract, you pay a commission to your broker. Ignoring these commissions in your trading system always lead to inflated simulation results. Although, it is often said that past performance is not indicative of future results, simulating or backtesting a strategy with historical data remains an important technique in the trader's arsenal.
|
|
|
|
|
