Multi-Timeframe Backtesting in eSignal

Most eSignal users are probably aware that EFS allows for the development of multi-timeframe indicators, meaning that different components of the indicator can be running in different bar intervals. 

"Multi-timeframe backtesting can be hazardous to your health"

So, for example, you could create a moving average indicator that will plot a moving average based on 10-minute bars and a second moving average based on 30-minute bars, and this indicator can then be run in a 1-minute chart (or a 15-second chart, or whatever).

This same multi-timeframe logic can be used when developing trading systems. So you might develop a trading system that you intend to run in a 1-min chart and, as part of the trade-entry logic, you might require that the current price bar in a higher interval (let's say 15-minute) is currently biased in the direction of the trade (i.e., close>open for a long signal or close<open for a short signal).

Now while this logic will work perfectly well when your system is running in realtime, you can run into problems when backtesting this same system via the eSignal Backtester using historical data. The issue has to do with the synchronization of historical bars. For purposes of this discussion, a historical bar is any price bar that is not currently open (i.e., any bar that has closed).

Going back to our example, all of this works in realtime simply because bars on every interval are evolving at exactly the same pace. So if we are looking at the 1-minute and 15-minute bars at 9:37am (in realtime), the 15-min bar will accurately reflect the current state (i.e., as of 9:37am) if we check to see if the close>open or the close<open.

However, when we move to historical price bars it is an entirely different ballgame. Consider the fact that, with historical price bars, all we know is the open, the high, the low, and the close. We have no way of getting inside a historical price bar to see what it looked like at some point between the time the bar opened and the bar closed...all we do know is what it looked like as of the time that the bar closed.

This poses a problem when backtesting multi-timeframe systems because it introduces a potential look-ahead effect. By this I mean we are making a decision at time A based on data that will not actually be available until time B.

Consider the chart snapshot on the left. The upper section is comprised of three 15-minute bars while the lower section is comprised of forty-five 1-minute bars. The 1-minute bars are enclosed in 15-minute blocks using our Box Script indicator so you can clearly see which 15-minute bar is associated with which block of 1-minute bars.

Let's assume we are backtesting our hypothetical trading system and let's also assume that our system generates a potential long signal in the first bar of the very last block of 1-minute bars in the graphic above, and let's assume that the timestamp of this bar is 1:00pm. So the next step is to confirm this long signal by checking the status of the 15-minute bar.

Well, when our script accesses the 15-minute data during the backtest it is going to report the status as close<open (i.e., a down bar) and it will not take the trade since the 15-minute bar direction does not confirm the trading signal. The problem is that this "status" is based on the close of the 15-minute bar at 1:15pm rather than the actual state of the 15-minute bar as of the time of our trade entry signal, which, as mentioned above, is 1:00pm. So, for purposes of our backtest, we are making a trading decision at 1:00pm based on information that will not actually be available until 1:15pm (i.e., a look-ahead).

In reality (in realtime) our hypothetical trade probably would have been confirmed since the 15-minute bar would have been bullish (i.e., close>open) as of 1:00pm, and it would have probably been a losing trade since price began to drop after 1:00pm.

So the net effect of this look-ahead issue is that the backtester will skip a lot of losing trades and will take a lot of winning trades that, in realtime, might not have even been confirmed in the first place. This, in turn, will give you a false sense of security regarding your trading logic which could have a disastrous impact on your bottom-line.

Now, our example above is fairly simplistic and an obvious workaround would be to use the 1-minute bars to build a synthetic 15-minute bar based on the number of 1-minute bars that have actually elapsed as of the time of our trading signal. Things can get a lot more complicated, though, when you actually build a collection of indicators based on different bar intervals and then use these as part of your trading logic.

The takeaway from all of this is that you need to be extremely careful when building and testing multi-timeframe systems, and you have to have a clear understanding of what is going on in the background when you backtest. Failure to do so can be hazardous to your health.

Divergence Software, Inc.
www.sr-analyst.com
support@sr-analyst.com
Contact Us/Subscribe