Backtesting with TradeSim

A question I get asked often is that what is the point of backtesting when the future won't be the same as the past. A valid question.


There are 3 ways to know whether you are trading with a valid (profitable) strategy. A system that has a positive expectancy.

1. Backtest it.
2. Paper trade it.
3. Trade it in real time.


Do either of the 3 over a long enough time frame to get a statistically significant number of trades, with documentation of average win, average loss, and the win percentage, and expectancy can be easily calculated. As discussed previously, trade frequency is also important.


Okay, so let's go through the options. Paper trading is fine, though some say it is boring and doesn't test the emotions the same way that real trading does, but at least with this method, dollars are not risked. Still, it would take a long time to get a statistically significant number of trades. And very hard to be able to test the system over a variety of market conditions. Same problems with real time trading, but the main concern with this is, why would you trade a system and risk real dollars if you have no idea of the probabilities of whether your system will make a profit or not?

Backtesting, with correct validation techniques (i.e. walk forward analysis) can provide a very good indication is a system is likely to profitable when traded in real time.

The software I use for all my backtesting is TradeSim. It was initially designed to be a plugin for MetaStock (MS) but also be used with Amibroker and Bullcharts though the integration isnt as seamless as it with MS.

TradeSim is a trading simulator and backtester that is design to test a system exactly as if you were trading it. So trades will not be taken if there is insufficient capital, brokerage fees are factored in, there are different models of slippage to choose from, several position sizing strategies can be explored, and volume information can be also considered.

But the standout feature for me was monte carlo analysis capabilities. Because many trading systems do not have enough capital to be able to take every trade that is signalled, is it of paramount importance that the system be exhaustively tested through as many combinations and permutations of trades and portfolios that is possible.

David, the developer of TradeSim gives this example in his excellent User Manual. Take the heavily publicised MACD crossover system, where the entry is when the fast moving average crosses the slower one to the upside, and the exit when the fast moving average average crosses the slower one to the downside. If you only do a single portfolio simulation, it is possible that the results show that the system makes 22%pa. But if you run the simulation again, its a 2% gain. And then run it again its a 38% loss. Its only when you run it through 10,000 or 20,000+ simulations that you realise its a break-even system at best.

Another feature of TradeSim which is useful is the options for slippage. Slippage is the difference between the price you want and the price you actually get. Say for example, your system is set so that if you get a signal today, then you will buy tomorrow's open. If you set TradeSim to default order, then it will buy what the next day's opening price actually was. No slippage is taken into account here.

If you set TradeSim to market order, the program, through a random algorithm, with randomly choose a price between the day's hi price and lo price as the entry price. Slippage is the difference between the price you are getting and the price you wanted (the actual open). One of the other options for slippage is worst case, where when you buy, you pay the day's hi.

One way of testing the robustness of the method is to play with different slippage models. If a change from default order to market order, or market order to worst case, puts your very profitable system into the red, then you have some work to do.

With the system I'm currently developing, which is a weekly system, i was designing it with flexibility in mind. I may not be able to buy Monday's open all the time. So I set market orders during backtesting which gives me the flexibility to buy anytime during the week.