To judge how well a given trading system should work in the future, we backtest it on past market data. Backtesting applies a set of trading rules to historical data to estimate how those rules would have performed if we actually had traded them.
Good hypothetical historical results do not guarantee that a set of rules will work well in the future. However, poor hypothetical historical results almost certainly mean a system should not be traded in real time. The perceived value of backtesting is rooted in the belief that historical tendencies repeat. Traders have been testing strategies on historical data for generations.
However, the practice became popular with the advent of personal computers and purpose-built system-testing software , such as System Writer, which evolved into TradeStation.
This software and a database of historical data allowed those without a code-writing background to test trading system ideas. The broader understanding and acceptance of trading systems, as well as the frustration many encountered when trying to build trading systems on their own, helped the market of third-party systems flourish throughout the s.
Futures Truth is an independent company that has tracked commercially available trading systems since the s. Currently, it tracks more than systems. Futures Truth tests trading systems in real time, not on historical data.
Trading Platform Software for backtesting & auto trading, for stock, futures and forex trading systems » Seer
This prevents the modification of rules over time and better simulates rule execution in actual market conditions, such as periods of high volatility. So many systems fail because they lack a valid premise. Instead, the entry and exit parameters are derived from data mining. Data mining simply scans historical data for rules that would have worked in the past. Often, such rules are fit precisely to the past and have no hope of working any better than random on unseen data. Instead, system development should start with a theory that can be tested, analyzed and fine-tuned for application.
This concept also implies a different perspective on system testing itself: The goal of backtesting is not to produce a collection of hypothetical profit and loss statistics. It is to test the validity of the theory and the accuracy of the rules in capturing the premise.
System testing is a multifaceted process from the data, to the time scale, to order entry assumptions, to contract specifics and risk control.
Failing at any of these can ruin an otherwise valid test — or, manipulating them can generate results that are far superior than what we would achieve in real time. You need to do it right if you hope to validate — or when appropriate, invalidate — your system.
There are two elements to backtesting: The proper tools — software and data — and a scientific method to develop systems using those tools. Many options are available for testing your ideas. They differ in the ease of turning ideas into code and in how they handle the details, which can have a major impact on the results. For example, if a system enters on a limit order, some software records a fill if that price is touched. Entering on stops guarantees an entry, but not a price.
Another issue is recording real prices. While most professionally developed software no longer has this issue, it is still a concern for those who manually test systems in spreadsheets, such as Microsoft Excel. For example, if a system buys on a stop equal to the close plus one-third of the average range over the last three periods, and if the average range is 10, then we are buying at the close plus 3. This means the entry differential must round up to 3.
Over time, such an error could add up to a sizable discrepancy. E-mail him at ruggieroassoc aol.
Free Newsletter Modern Trader Follow. We asked traders what FBI Director Comey's testimony means for stocks and other markets. Silver holding huge commercial short. Retail is in trouble because of economic conditions.
What does this mean for the markets? Election play in gold options.
How to backtest trading systems and avoid curve fitting FROM ISSUE. Tools of the trade There are two elements to backtesting: In the big picture, however, such procedural details are minor.
The big issue is data. Page 1 of 3. Short-term signals in E-minis, crude and gold. The age old question. A better measure of risk. Trading stocks with stochastics. New tools for the forex trader. Microsoft Futures Classroom TradeStation Murray A.
The age old question Sortino ratio: Previous Trading stocks with stochastics. Next New tools for the forex trader. Related Terms Microsoft Futures Classroom TradeStation Murray A.