Walk forward analysis

I have completed a walk forward analysis on my system, and tested it on the out-of-sample data from 01-01-2004 until 30-06-2007 (3.5 years).

This was previously untouched data that the system has never seen. It can be considered as "tomorrow's data".

The results were promising and have validated the robustness of the system.

Robert Pardo, in his excellent book, included in my recommended texts, mentions that one method of assessing out-of-sample testing results is by a ratio known as the walk forward efficiency (WFE).

WFE is simply the ratio between the objective function score on the out-of-sample testing and the objective function score on the in-sample testing. For a system to be considered validated, the WFE should be >50%.

An objective function is the outcome that we try to maximise when trading. My objective function is (profit/maximum peak-to-valley drawdown). This will differ for everybody. The WFE for my system was 0.89, or 89%.

Out-of-sample data testing should always be worse than in-sample data testing, because the system was optimised on the in-sample data. We did not stop testing and optimising until the results on the in-sample data were acceptable. But if the out-of-sample results are not acceptable, this is a sign that the system has been curve-fitted to the in-sample data, and we must start the testing process again.

Thus, 89% is a rather pleasing result, and is a sign of the robustness of the system, indicating that it has not been curve-fitted.

I'm in the middle of creating an Excel spreadsheet that will be used for trade monitoring and position sizing. Live trading should start in a few weeks.