Friday, December 28, 2007
Sunday, December 23, 2007
Saturday, December 15, 2007
Saturday, December 8, 2007
Saturday, December 1, 2007
Backtested equity curves of current tickers
Saturday, November 24, 2007
Portfolio Update 23/11/2007
Saturday, November 17, 2007
Saturday, November 10, 2007
Portfolio Update 9/11/2007
The latter two have already finalised everything regarding the takeover. So I figured if I don't cash out now, then I will end up taking either cash or shares in a few weeks/months time. Might as well put the money to use elsewhere.
The funny thing is, every week since that first week I've been thoroughly checking the candidates from the weekly scans if they are currently under takeover, and there's been none! Yet I got 3/10 on the first week!!
Friday, November 2, 2007
REX and BKN
Sunday, October 28, 2007
Customised systems
One of the benefits of designing your own system -- as opposed to buying one -- is that you can include the features you want and exclude that which you do not. The system can be, and should be, customised to suit your personal preferences. This also helps you to stick to the plan in those inevitable periods of drawdown.
So from the word go, from when you begin the systems design process, you should already know what sort of system you want to design, and which features are more important to you than others.
If you like a smoother equity curve, then have a shorter holding period (higher stock turnover) for your positions. This can be accomplished by having a tighter trailing exit (probably at the expense of profits).
If you cannot handle long losing streaks, then a way around this is to design a system with an entry which will give you a higher win%.
I was looking at improving my win% yesterday by using different universes as I currently use the whole market. So I tried the all ordinaries, ASX100/200/300, small ordinaries, and dividend paying securities.
Testing was the first 3 quarters of this year, 01-01-2007 until 30-09-2007, in an attempt to minimise inclusion bias (indices couldn't have changed too much throughout the course of the year). Nevertheless, it's only for relative measure anyway.
Results were as follows:
All Ords (488 stocks)
#Trades: 32
MaxDD: 2.73%
Return: 17.83%
Win% = 59
R/R = 2.06
Small Ords (199)
#Trades: 26
MaxDD: 3.97
Return: 14.58%
Win% = 50
R/R = 2.60
ASX100
#Trades: 37
MaxDD: 17.33%
Return: -15.79%
Win% = 23
R/R = 0.63
ASX200
#Trades: 41
MaxDD: 9.87%
Return: 6.84%
Win% = 37
R/R = 2.58
ASX300
#Trades: 39
MaxDD: 3.39%
Return: 21.51%
Win% = 61
R/R = 2.30
Div paying (645)
#Trades: 37
MaxDD: 4.65%
Return: 23.13%
Win% = 54
R/R = 3.08
Whole market (1951)
#Trades: 43
MaxDD: 4.57%
Return: 23.75%
Win% = 63
R/R = 1.93
The best returns were from the whole market, dividend paying securities, and the ASX300.
So I did 1000 monte carlo runs of each of them with the results as follows:
The format for profit is max/average/min/std deviation/%of portfolios profitable.
And for maxDD max/average/min.
Whole market
Profit: 55/18/-7/9.68/97.8
MaxDD: 17/6/1
MaxDD: 17/6/1
The last figure on the profit column is one I pay the most attention to. 100% would be ideal.
To give some sort of benchmark the XAO (All ordinaries) was up by 12.5% over this 9-month period.
So it seems from this test that we can't expect an improvement by trading an alternative universe, at least from the options tested.
I will be adding $20,000 of trading capital to the system this week just to make it compound quicker as we are approaching the month of November. November to April is traditionally a very strong period for equities.
From the weekend scan, the full list of possible candidates are as follows:
AQA, BEC, BLY, BKN, CPB, CEY, DWS, FLT, GMI, JBH, LEI, LGL, MSL, MIN, NCM, NOD, RCR, SMX, SDG, UGL, WTP, WOR, WTF.
I applied 3 filters. The chart has to be good. Clear uptrend and not in a trading range.
This narrowed it down to AQA, BEC, BLY, LGL, BKN, MIN, NOD, RCR, SMX.
Then I checked if any were under takeover. None were.
Then I picked the lower price securities or the ones I liked. LGL a gold play I just had to buy it.
MIN and BKN even if they didn't trigger, I could pyramid because i have reached 15% profit (that's what was backtested).
Since I could probably afford 3 more positions, I'm going with LGL, NOD, and BLY.
Here is a website I check to see Broker consensus. It's another filter I've added. I like to see a minimum rating of buy or strong buy.
Remember the system is robust and well tested so it doesn't matter HOW we pick the stocks from the candidates spat out by the scan. All the testing initially was done by picking random trades (monte carlo analysis).
Ranking by price and even pyramiding trades, only adds 1-2%p.a.
Friday, October 26, 2007
Friday, October 19, 2007
Seek and you shall find
It really is amazing how much good information is out there on the internet. And the best part is that most of it is free. All we have to do is look for it. I feel I have learnt a great deal from online forums and other resources available over the internet. This post is simply a list of material that I have used and that others may find useful.
I have started numerous threads over at Aussie Stock Forums, including topics such as:
*System Robustness
*Drawdowns
*Would you trade this system?
*Outside the "Blueprint"
*Systems testing
And read the whole thread. Don't focus on what I write, but rather, on the responses, which are mostly from seasoned professionals.
This is an interview with author and hedge fund manager Robert Pardo. I think it's very good and I found it useful. He covers several important aspects of systems development.
Chuck LeBeau's presentation on exits is another resource I would highly recommend.
On Nick Radge's forum, there is so much knowledge you would not believe.
Tech/a has shared the discussion regarding the full design and development of his system and traded it live and answered questions for the last 5 years. Any question you have on systems development chances are somebody has asked it and it has been responded to. I would recommend reading the whole folder (about 50 threads).
Also, in the Trading Systems section, go to the very end, and read anything by Stevo and OPM and tech/a. A search may be helpful here.
And lastly, I just stumbled across another forum recently, called Elite traders. This thread by Acracy is pure gold. I would regard it as the next level of system development. He goes beyond entries, exits, and money management.
Tuesday, October 16, 2007
Don't be fooled by randomness
The whole purpose of systems development is to find a method of trading that gives us an edge. An edge in terms of trading can be defined as probability of making profit which is greater than random chance.
If our probability of being profitable is the same as random chance, then any gains we make can be attributed to luck. That said, luck is also an important and often underrated factor when it comes to trading. We can see this from the difference in the results between the minimum and maximum of thousands of monte carlo simulations. This is why we should aim to design systems with enough rigidity so that the distribution of results is nice and tight. But this is another topic.
Here is nice definition of an edge: "If you make only bets on which you have an edge, you will win and you will lose but in the long haul your winnings will overwhelm your losses."
A good way to test a system is to compare it to another system which is based on randomness. ASX.G has done significant work on this. I ran one test but it was enough to confirm the work he had done. An random system with 10% chance of entering on any bar and 20% of exiting on any bar returns about 1%p.a. greater than the market, on average. The luckiest bastard made many thousands of percent.
In addition to this, I did a bit of testing to attempt to see whether the edge in my system was in the exit or the entry. And from my work, I can say that entries are very underrated.
The testing period was from 01-01-1998 to 31-12-2003.
The market as a whole underperformed it's long term average significantly during this time, with a compounded annual return (CAR) of only 3.82%p.a.
My whole system returned 38.90%p.a.
My system with my entry and random exit: 28.00%p.a.
My system with my exit and random entry: 7.74%p.a.
Random entry/exit was defined as a 10% chance of entering/exiting on any given bar.
My system with my entry and random exit: 15.95%p.a.
Random exit here was defined as a 5% chance of exiting on any given bar in an attempt to extend holding time to that similar to the original system which is 20 bars. It looks like the trailing stop becomes increasingly important in this instance.
But still, it seems most of the edge lies in the entry.
Sunday, October 14, 2007
Speaking my language
It's nice to see somebody else speak your language. I was reading this week's BRW and there was an interview (pg.78-82) with a hedge fund manager named Karl Siegling. Below are some of the things he had to say about the way he trades. His fund's gross return is up 75.98% over the last 15 months (inception) and about 32% for the last 6 months for Australian shares.
"A falling share price is not a buying opportunity, but a clear warning from the market that something is wrong. Repeated new highs for a stock are not evidence of a missed opportunity or an overpriced company, but rather an invitation to buy more shares."
"I believe that there is no sense, no matter how compelling our fundamental belief may be, in buying into a stock as it is falling in value."
"I believe that the words "contrarian investor" are often used out of context, and that a contrarian strategy of constantly buying and adding to a falling position and constantly selling or shorting a rising market leads an inexperienced investor into making costly mistakes."
"The market can be irrational and sell down a stock for no reason - no reason based on the fundamentals of a business. But it's not for us to say when the market will stop being irrational."
"I will buy shares at, for example, $1, then $1.10, and then $1.12, but if I have just paid $1.12 for a stock then I will not pay $1.10 if the price falls a few days later. That's because I like buying shares that are going up."
So do I, Karl!
Based on these sounds principles, his strong performance comes as no surprise.
Saturday, October 13, 2007
Not an exact science
Backtesting is not an exact science, and has its limitations. At times, it can be said that the best computer is the the one between the ears.
One limitation that I only discovered recently was regarding company M&A. I realised yesterday that 2 out the 10 companies that I purchased this week were under takeover offers. So I was wondering how TradeSim handles these situations, because, I want to stick to the plan which was backtested with the least degree of deviation as possible; so I have the most chance of realising similar results.
Well TradeSim is only as good as the data that you feed to it; and data is simply 5 values for each bar. It cannot in any way account for company decisions. So when a company is delisted i.e. stops trading for any reason (merger with another company, script takeover, cash takeover, change of stock code, bankcruptcy), then TradeSim will close the position on the last bar it traded, the trade will be found in the TradeSim trade database as an open trade, and the price will be at the price it last traded (regardless of how long ago that was).
It will be marked as an open trade to differentiate from those trades that were closed out because they hit the protective stop or triggered a normal exit.
So it is upto me how to deal with these company M&A situations. I think at least for now, I will just leave the positions as they are, but will reassess the situation periodically.
I will not be updating the portfolio this week because all exits (as are entries) are delayed by one bar, I will have to check at the close of the next bar, which is next week, to see if any stops or exits were triggered. And I won't be running a scan (exploration) for any new candidates because all my capital is already in open trades anyway.
I notice there's some media articles regarding next week being the 20th anniversary of the 1987 crash. Some commentators say that this may well be a cause of market "jitters" next week. I guess that sort of news sells newpapers.
Thursday, October 11, 2007
Exits
I can relate somewhat to what Eckhardt is saying in the above quote because I tried profit targets for my system sometime ago. They didn't work very well. What was initially supposed to increase win % in fact did not, but instead, reduced the size of average winners substantially. I was effectively cutting winners short.
My whole thinking behind it was because I don't like seeing winners becoming losses, as this (sometimes) happens in long-term trend following systems, that by definition give back alot of open profits. But clearly, the flip side of the coin of cutting winners short made the trade-off rather unfavourable.
Thinking about it now, the idea did not make sense at all, as I'm cutting (potentially) big winners in an attempt stop small losses. Losses are always small because of the my stop loss. Big winners, however, can often be the difference between a great system and an average one. If I were to remove the best 10 trades from the trade database, returns fall by almost 10%p.a.
What could work is a profit exit based on rate of change (ROC), designed for stocks that rise very rapidly. If a stock rises X% in a N number of bars, then the exit is taken. But more work had to be done here before I can make any conclusions.
Howard Bandy discusses exits in a section of his book. He mentions there are 5 types;
1. By the action of an indicator or recognition of a pattern, similar to what caused the entry.
a. The parameters can be the same as those that caused the entry, but in the other direction.
b. The parameters can be different for exit than for entry.
c. Some other indicator can be used.
2. By the price reaching a profit target.
3. By the time in the trade reaching a maximum holding period.
4. By the price falling back to the level of a trailing stop.
5. By the price falling back to the level of a maximum-loss stop.
He goes onto mention that the fifth type is the worst i.e. the maximum stop loss level. Those that have designed and traded mechanical systems would agree with this. During testing, we try and aim for the highest percent possible of trades exited in profit. Exits taken with a loss, and these are mostly those that hit the protective stop, should ideally be no more than 10-15% of the total exits.
Initial stops don't always help the system. For example, for my system, that I've just began trading live on this blog, I tried several but couldn't find an initial stop that improved the bottom line (doesn't mean there isn't one). I wanted an initial stop to calculate position sizing so ultimately I decided to place the initial stop at the trailing exit.
Maximum holding periods I must admit I haven't put enough research into this area to have an informed opinion but it just doesn't make sense to me. I believe the time in a stock should be determined by its trend. You want to spend the most time in the winners because profit increases as time in trade increases. At the same time, you want to spend as little time as you can in the losers, so you can take the loss and use the money for another trade (an opportunity cost issue). The problem is that you never know beforehand whether the trade will be a win or a loss, so it would be difficult (impossible?) to deduce the optimal holding time which would outperform other types of exits.
Wednesday, October 10, 2007
Randomly skipping some trades part 2
I didn't really explain the chart in my previous post as well as I should have.
The test was conducted over 9.5 years. In that time, there were about 2900 trades in the trade database. Those are all the possible trades picked by the system over that time.
Due to capital restraints and position sizing, I only traded between 400-500. I say and position sizing, because this is what keeps the number of trades down. Even if I started with 500k, if my fixed percent risk was 1.5%, I would still be taking about the same number of trades.
As you can see, I'm only taking about 15% of all the trades that are triggered anyway. So it doesn't matter how I pick them or even if ignore a vast majority of the trades, the results aren't affected significantly, as monte carlo runs have shown.
Randomly skipping some trades
Tuesday, October 9, 2007
Every great journey begins with the first step
Start up is hard for long term systems because the trades that are closed out first are often the losers, and we need to give winners time to run. From testing, the average holding time for a winner was about 200 days, compared to 60 days for a loss. And the really big winners (which are the trends we are trying to catch) often run for much longer than a year.
The Total Trading Capital column in the spreadsheet is the money allocated to existing trades minus brokerage fees plus money in the bank. This is what TradeSim uses to calculate position sizing so I will do the same. MIN I paid double in brokerage because I made a mistake with the position sizing and bought less than I should have the first time around.
Monday, October 8, 2007
Waiting for opportunity
This post will have nothing to do with the title but I really liked that quote and has to fit it in somewhere!
There was another tweak I made to the system over the weekend that added a few percent to the results, that I haven't yet mentioned here.
All this time, I had the box in the TradeSim Preferences window "Accept partial trades if inadequate capital", unchecked. This only came to mind when I was thinking about this week's buy orders. After buying 9 stocks, each with a parcel size of about $4,500-4,700 (worked out from fixed percent risk), I would not actually take the 10th trade because i would fall sort of the required parcel size by a few hundred dollars. Then I thought why not take the trade anyway. But that hasn't been backtested. So I ran the tests again, to quantify the benefit (if any).
As suspected, the profit results were greater, by 3-4%p.a. over the various timeframes. Drawdowns were not affected significantly, about 1% higher. I'm happy with that. Number of trades increased by about 20%, naturally, so instead of 149 trades in the 6 year test, there were 179 trades. And instead of 84 trades in the 3.5 year test, there were 101 trades.
So now, the method will allow me to clean up the account if I don't have enough cash to take the full sized trade. This means that my money is in the market more often, which is a more efficient use of trading capital. And 3-4%p.a. is nothing to scoff at. Over a number of years, due to the effect of compounding, the increase in profit is quite substantial.
Maximising performance part 3
I found a flaw in my testing method when ranking trades due to price, well, not a flaw as such, but the study design could've been structured better. Because I was ranking by lower priced stocks, this should mean, in theory, that there is only one possible route of trades to take. So everytime I ran a single portfolio simulation (with original ordering), all else being equal, the results should be exactly the same.
But they weren't, and I soon realised this was due to slippage. The variability would come from the randomness generated by the market orders which would buy a price anywhere between the low and the high of the entry bar and exit through any price between the low and the high of the exit bar.
As this wasn't a sound method of testing, I started again. For the purposes of this test, I ran monte carlo simulations (20,000) this time using default order slippage. Then I ran a single simulation through with the ranking giving preference to lower priced securities, also using default order slippage. So there's only 1 possible outcome. Then I compared this to the average portfolio result generated from monte carlo. The results are as follows.
01-01-1998 to 31-12-2003
Monte carlo average: 37.4%p.a.
With trade ranking: 38.9%p.a.
01-07-2001 to 01-03-2003 (The worst - XAO loses 18%, peak to trough of the bearmarket)
Monte carlo average: 1.5% (non-annualised)
With trade ranking: 13.56% (non-annualised)
01-01-2004 to 30-06-2007
Monte carlo average: 31.1%p.a.
With trade ranking: 44.6%p.a.
I don't think its coincidence that the more recent we go, the more wider the gulf becomes. I suspect it's due to not many stocks back in the 90s getting past my liquidity filter, though this hypothesis has not been tested. You can see this from the trade database. For the 6 year test between 1998 and 2003, there were 1234 possible trades, and for the 3.5 year test between 2004 and 2007, there were 1647 possible trades.
Sunday, October 7, 2007
Maximising performance part 2
Thanks to an idea by rnr over at ASF, I decided to run some more simulations in my attempt to find a systematic method that will allow me to make gains similar to those at the higher end of the monte carlo range.
This time I ranked the stocks by their volatility (ATR(10)) and told TradeSim to give preference to the stocks with higher volatility. The results were less than flattering. Out of 50 runs, only 60% produced an annual return greater than the average from monte carlo. And only 4 (8%) produced gains which were significantly higher than average.
These results, unlike those produced by price ranking (see previous post), are inconclusive at best. In fact, if I were to run another 50 simulations, it could well be that the we could conclude that this method provides no edge over random selection.
Saturday, October 6, 2007
Maximising performance part 1
In a previous post, I mentioned how I had not yet quantified the benefit of picking lower priced stocks over those with a higher price. This post will address this issue.
From this week's candidates, after excluding 3 with my eye-ball filters, I have 16 left to choose from. From those 16, my capital would only be sufficient for 9 or 10. I had initially planned to trade the way TradeSim does by default (for single portfolio simulations), which is alphabetically. Then I thought it would be nice if I could test what benefit, if any, there is, by picking lower priced stocks.
In theory it makes sense that, say, a $3 stock, would get to $6 much quicker than you can see CSL (currently around $107) reach $220. But, being the trader I have now become, I don't go for hunches or opinions anymore, everything needs to be tested, validated, and quantified.
Using TradeSim, you can test this by using the SetVariableTradeRank function (see manual pg.115). I told TradeSim to rank the trades in the trade database by their closing prices. And then to give preference to lower priced securities. What I wanted to see is WHERE the results fell relative to the average profit values generated by the monte carlo analysis of the same system over the same timeframe.
So I ran 50 single simulations, and the results were conclusive. Of the annual returns from these single simulations, 49 (98%) were greater than the average return of the monte carlo. Also, 19 of the simulations (38%), gave returns that were significantly higher than the average return. Significantly higher was defined as an increase of more than 5%p.a. in annualised returns.
The above simulations were done on the 6-year period from 01-01-1998 until 31-12-2003. I also did some tests on the previously out-of-sample period which was used for the walk forward analysis. From 50 simulations using the ranking function to give preference to lower priced securities, 39 (78%) produced annual returns greater than the average from monte carlo analysis, and 23 (46%) produced gains that were significantly higher than the average return.
So clearly, there is a benefit in choosing lower priced securities, and the bang for buck theory has been verified and quantified for my system.
Now I have a very easy method of choosing stocks. Go for the lower priced share. Though I should clarify that rightly or wrongly I will still do my eyeball filters first! I just can't buy a share with a disgusting chart!
Next week's purchases will be, in order of preference, LRF, REX, SSX, STS, MIN, CSM, COA, PWK, IWL, BKN.
BKN may have to miss out. I'll see how I go with the position sizing. It looks to be either 9 or 10.
**EDIT: It has been brought to my attention that SSX has in fact been delisted (has merged with OneSteel). So the question must be asked, why was it still picked up by the scan? Because the last time it did trade, it did give a valid entry signal. The most recent data loaded for this stock was 3rd August 2007. So, i must keep in mind for the future to check the date the stock last traded. So BKN will be surely included now. Next in line would be JBH.**
Friday, October 5, 2007
Next weeks candidates
Thursday, October 4, 2007
Trend Following
I searched "trend following" on google the other day and one of the responses was from Wikipedia:
In finance, trend following is an investment strategy that tries to take advantage of long-term moves that seem to play out in various markets. ... Traders who subscribe to a trend following strategy do not aim to forecast or predict markets or price levels; they simply jump on the trend and ride it.
The last sentence is one of the main factors which drew me towards trend following, and mechanical systems trading in general. Prediction does not play a role. You do not have to know what is going to happen next in order to be profitable.
Prediction, or forecasting, is something that is subjective, or opinion-based. With trend following (or system development of any style) everything we do, every rule, every parameter, its not subjective or opinion based, but rather, its evidence based. If it can't be tested then you don't trade it.
The other day, just for practice, I ran a scan of the ASX using my entry criteria and I got about 20 signals! Due to the limited capital available, I can only afford maybe 12 or 13, so I decided to (hypothetically) buy the lower priced shares. Then i thought, hold on, i didn't factor a price filter into my entry filter during backtesting, so I have no reason to believe that lower priced shares will provide better gains. Hold on, let me rephrase that, while I actually do have reason to believe that lower priced shares provide better gains (through Nick Radge's "Bang for Buck" study), the benefit of employing this strategy for my system has not yet been quantified.
Regardless of which method I choose to pick the stocks to buy (out of those that gave entry signals), it shouldn't matter too much, and discretion can be applied here. This is the reason we do monte carlo analysis. Whether I study the charts or flip a coin, no matter how I choose, allowing for a margin of error (10-20%), the results should still fall within the minimum and maximum amounts returned from the monte carlo analysis of the out-of-sample data. Backtesting is not an exact science, but its the best tool we have. And applied correctly, it can give us a good idea of how a system is expected to perform.
Michael Covel has done some great work on trend following. In addition to his first excellent book, he has a second book coming out soon titled: "The complete turtle trader". Michael Covel also did a presentation about the turtles and trend following at a conference in Tokyo earlier this year. The videos can be found on YouTube. Well worth a watch.
Wednesday, October 3, 2007
Riding those big winners
Monday, October 1, 2007
Rigorous testing methods
A great trader once told me: "Try to make your system fail".
I didn't go into this in much detail beforehand, so will do now.
When I said my system was initially back tested in the 1998-2003 six year period, and optimised on this period, the testing did not stop there.
I also run the test through different 2-year blocks and even single year tests, though single year tests do not give an accurate representation of the profitability of long term weekly systems such as this one, as the big winners often run for much more than 1 year.
The testing through 1 and 2-year blocks was an attempt to try and address start and end date biases. You don't want 6 years to look good because of one or two great years.
All my testing always includes delisted securities. I never test on the current All Ordinaries or ASX300 Indices. Quite pointless, it will give you great results, but not realistic results, which is what we are after. What I was after when I started testing was dynamic indices, with historical index constituents. Now that would be handy. We include delisted securities to overcome survivorship biases. For example, if we test using the current ASX100, which includes ZFX, we would've bought ZFX a few years ago at $2. But ZFX actually wasn't included in the ASX100 in 2005, so we wouldn't have picked it up during actual trading as its not in our universe. Stocks that are listed today can easily become delisted. In fact, my system performs better with delisted securities than without. This test was only done for my own personal interest. Shows that those dogs were once champions. The trailing stop/exit is what saves you.
The system was tested through a period I'd like to call THE WORST. Its the closest we've ever come to a bearmarket in the last 15 years on the ASX. From 01-07-2001 until 01-03-2003, is as bad as its been for sometime. XAO lost 18% during this time (21 months), its pretty much peak to trough. My system was put through 20,000 simulations. More than 82% were profitable. The average portfolio gained 8%. Not bad.
During 2001, there were 2 price shocks. First the dot.com crash and then 9/11. So a very volatile year. Market actually gained 6.98% though. For my system, more than 99% of the portfolios were profitable, with the average portfolio gaining 16%.
During 2002, the market lost about 10%. Solid downtrend. 99.97% of my 20,000 portfolios were profitable. The average portfolio gaining more than 12%.
I also test the system using worst case slippage. I always buy the high of the week and sell on the low of the week. Profitability declines by about 40%.
Another thing I like to do is remove the best 3 and worse 3 trades from the TradeSim trade database. Hardly any difference in the results. I don't want one or two champion stocks to be responsible for me making money. The largest winner contributes about 10% of the overall profit of the system over the 6 year test. I'm happy with that. You don't want 20%+ coming from just one stock.
Sunday, September 30, 2007
Walk forward analysis
This was previously untouched data that the system has never seen. It can be considered as "tomorrow's data".
The results were promising and have validated the robustness of the system.
Robert Pardo, in his excellent book, included in my recommended texts, mentions that one method of assessing out-of-sample testing results is by a ratio known as the walk forward efficiency (WFE).
WFE is simply the ratio between the objective function score on the out-of-sample testing and the objective function score on the in-sample testing. For a system to be considered validated, the WFE should be >50%.
An objective function is the outcome that we try to maximise when trading. My objective function is (profit/maximum peak-to-valley drawdown). This will differ for everybody. The WFE for my system was 0.89, or 89%.
Out-of-sample data testing should always be worse than in-sample data testing, because the system was optimised on the in-sample data. We did not stop testing and optimising until the results on the in-sample data were acceptable. But if the out-of-sample results are not acceptable, this is a sign that the system has been curve-fitted to the in-sample data, and we must start the testing process again.
Thus, 89% is a rather pleasing result, and is a sign of the robustness of the system, indicating that it has not been curve-fitted.
I'm in the middle of creating an Excel spreadsheet that will be used for trade monitoring and position sizing. Live trading should start in a few weeks.
Saturday, September 29, 2007
Testing part 2
Position value limit means no position can make up more than 15% of the portfolio. This does not differentiate between stocks. So if I pyramided into one stock 3 times, this is regarded by this value as 3 positions, the same as 3 positions into 3 different stocks.
Portfolio heat means how much i were to lose if i was stopped out of all my positions. I have set this to 20% but in the trading timeframe the maximum has been 17%.
Entry and order type is market order, which means the actual entry will be a randomly generated figure between the high and the low of the price on the entry bar (weekly). Testing with these parameters gives me the flexibility to enter and exit stocks at any time during the week, not just Monday's open.
Trade processed by each simulation was 1265. This is how many trades were triggered by the system's entry. The number of executed trades is how many trades were taken, and this is governed by the amount of available trading capital.
Profit Summary
Profit Status: PROFITABLE
Starting Capital: $45,000.00
Finishing Capital: $281,900.17
Maximum Equity/(Date):
$236,900.17 (24/12/2003)
Minimum Equity/(Date):
$687.26 (9/04/1998)
Gross Trade Profit: $328,373.16 (729.72%)
Gross Trade Loss: -$91,472.99 (-203.27%)
Total Net Profit: $236,900.17 (526.44%)
Average Profit per Trade: $1,548.37
Profit Factor: 3.5898
Profit Index: 72.14%
Total Transaction Cost: $6,732.00
Total Slippage: $77,130.18
Total Trade Interest: $0.00
Daily Compound Interest Rate: 0.0841%
Annualized Compound Interest Rate: 35.9254%
Trade Statistics
Trades Processed: 1271
Trades Taken: 153
Partial Trades Taken: 0
Trades Rejected: 739
Winning Trades: 82 (53.59%)
Losing Trades: 71 (46.41%)
Breakeven Trades: 0 (0.00%)
Largest Winning Trade/(Date): $27,505.82 (7/12/2001)
Largest Losing Trade/(Date): -$4,233.32 (28/06/2002)
Average Winning Trade: $4,004.55
Average Losing Trade: -$1,288.35
Average Win/Average Loss: 3.1083
Trade Breakdown
Long Trades
Normal Exit: 121 (79.08%)
Protective Stop: 18 (11.76%)
Open Trade: 14 (9.15%)
Trade Duration Statistics
Winning Trades
Maximum: 245 days
Maximum consecutive winning trades: 8
Maximum consecutive losing trades: 6
Average consecutive winning trades: 2.22
Average consecutive losing trades: 1.97
Trade Expectation Statistics
Normalized Expectation per dollar risked: $1.33
Maximum Reward/Risk ratio: 35.43
Minimum Reward/Risk ratio: -1.23
Average Positive Reward/Risk ratio: $3.00
Average Negative Reward/Risk ratio: -$0.59
Relative Drawdown
Maximum Dollar Drawdown/(Date): $13,533.74 (5/07/2002)
Maximum Percentage Drawdown/(Date): 5.9480% (5/07/2002)
Maximum Dollar Drawdown: $24,202.46 (10.5600%)
Capital Peak/(Date): $229,178.79 (5/04/2002)
Capital Valley/(Date): $204,976.33 (14/03/2003)
Absolute (Peak-to-Valley) Percent Drawdown
Maximum Percentage Drawdown: 10.5600% ($24,202.46)
Capital Peak/(Date): $229,178.79 (5/04/2002)
Capital Valley/(Date): $204,976.33 (14/03/2003)
Profit factor: Ratio of gross trade profit to gross trade loss.
Profit Index: Total net profit to gross trade profit. 100% would mean no losses.
Slippage is the difference between the price you want and the price you actually pay. In our case, its the difference between the opening price of the stock compared to our actual entry price. Because our system in a trend following system, a fair bit of slippage is to be expected as we are entering traded in the direction of the trend.
Profit factor: Ratio of gross trade profit to gross trade loss.
Profit Index: Total net profit to gross trade profit. 100% would mean no losses.
Slippage is the difference between the price you want and the price you actually pay. In our case, its the difference between the opening price of the stock compared to our actual entry price. Because our system in a trend following system, a fair bit of slippage is to be expected as we are entering traded in the direction of the trend.
Amount of trades that are exited through the initial protective stop is 11%. This is healthy and means our stop isn't too tight. 10-15% is good for this figure. Lower the better.
Average trade duration. The longer you are in a trade generally the more profit you make. If winning trades are banked quicker this generally provides a smoother equity curve.
Relative drawdown is the loss in equity from a previous equity high.
Maximum string of winners and losses. Best if they are equal or favouring the winners (which is the case here). This system shouldn't have a problem here because it picks more winners (52%) than losers. This statistic would be of concern to those trading methods that pick 30-40% winners.
Testing part 1
Monte Carlo Report
Trade Database Filename
C:\TradeSimData\Eagle Egg ORIGINAL.trb
Simulation Summary
Simulation Date: 23/09/2007
Simulation Time: 8:37:23 AM
Simulation Duration: 286.52 seconds
Trade Parameters
Initial Capital: $45,000.00
Portfolio Limit: 100.00%
Maximum number of open positions: 15
Position Size Model: Fixed Percent Risk
Percentage of capital risked per trade: 1.50%
Position size limit: 15.00%
Portfolio Heat: 20.00%
Pyramid profits: Yes
Transaction cost (Trade Entry): $22.00
Transaction cost (Trade Exit): $22.00
Margin Requirement: 100.00%
Magnify Position Size(& Risk) according to Margin Req: No
Margin Requirement Daily Interest Rate (Long Trades): 0.0000%
Margin Requirement Yearly Interest Rate (Long Trades): 0.0000%
Margin Requirement Daily Interest Rate (Short Trades): 0.0000%
Margin Requirement Yearly Interest Rate (Short Trades): 0.0000%
Trade Preferences
Trading Instrument: Stocks
Break Even Trades: Process separately
Trade Position Type: Process all trades
Entry Order Type: Market Order
Exit Order Type: Market Order
Minimum Trade Size: $500.00
Accept Partial Trades: No
Volume Filter: Reject Trades if Position Size is greater than
10.00% of the maximum traded volume
Pyramid Trades: Yes
Favour Trade Pyramid: Yes
Start Pyramid at any level up to level: N/A
Maximum Pyramid Level Limited to: N/A
Maximum Pyramid Count Limited to: N/A
Simulation Stats
Number of trade simulations: 20000
Trades processed per simulation: 1246
Maximum Number of Trades Executed: 175
Average Number of Trades Executed: 149
Minimum Number of Trades Executed: 130
Standard Deviation: 6.87
Profit Stats
Maximum Profit: $404,529.75 (898.96%)
Average Profit: $243,636.83 (541.42%)
Minimum Profit: $108,794.38 (241.77%)
Standard Deviation: $34,823.52 (77.39%)
Probability of Profit: 100.00%
Probability of Loss: 100.00%
Percent Winning Trade Stats
Maximum percentage of winning trades: 62.58%
Average percentage of winning trades: 53.33%
Minimum percentage of winning trades: 41.62%
Standard Deviation: 2.64%
Percent Losing Trade Stats
Maximum percentage of losing trades: 58.38%
Average percentage of losing Trades: 46.67%
Minimum percentage of losing trades: 37.42%
Standard Deviation: 2.64%
Average Relative Dollar Drawdown Stats
Maximum of the Average Relative Dollar Drawdown: $2,503.85
Average of the Average Relative Dollar Drawdown: $1,119.33
Minimum of the Average Relative Dollar Drawdown: $584.95
Standard Deviation: $213.27
Average Relative Percent Drawdown Stats
Maximum of the Average Relative Percent Drawdown: 1.7230%
Average of the Average Relative Percent Drawdown: 0.9085%
Minimum of the Average Relative Percent Drawdown: 0.5263%
Standard Deviation: 0.1387%
Maximum Peak-to-Valley Dollar Drawdown Stats
Maximum Absolute Dollar Drawdown: $60,067.90
Average Absolute Dollar Drawdown: $16,956.65
Minimum Absolute Dollar Drawdown: $4,375.91
Standard Deviation: $6,799.79
Maximum Peak-to-Valley Percent Drawdown Stats
Maximum Absolute Percent Drawdown: 16.3946%
Average Absolute Percent Drawdown: 8.0761%
Minimum Absolute Percent Drawdown: 4.1432%
Standard Deviation: 1.5186%
Friday, September 28, 2007
Cutting those losses
The popular adage to success in the markets is: Cut your losses short, and let your winners run.
Though in theory it may seem simple, for most of those in the trading arena, it is rather difficult. And that's why we have that statistic that 95% (or was it 97%) of traders are net losers over the long term. So the question must be asked; Why?
William Gallacher, in his book "Winner take all" says its very easy to spot traders that lose, simply take a look at their open and closed trades. Most of the open trades are losses, some of them substantial, and the closed trades are always positive. Usually small in magnitude.
So from this it can be said that most traders do exactly opposite to what is required to be successful, they take quick, small profits, and never take losses.
Why do they trade this way? Because of their ego. Taking a loss is like admitting you were wrong. Some people, and I don't mean Average Joe's here, I mean professional money managers, think that a loss is not a loss until you sell.
That's why I think fundamental analysis is so dangerous. Because of the hours spent analysing the company and its prospects, you are more likely to hold onto a loser for longer than you should, rather than somebody who looked at a chart for a few seconds or bought the stock because it triggered an entry on the system.
I was having a conversation with my dad the other day and I asked him if he thought anybody has held PDN all the way, and he said probably not. Yet there are a few that I know personally that are still holding the script of dot.com stock, bought for a few dollars, and now if the company is not delisted then its worth a few cents at best.
So it seems it is human nature so trade EXACTLY opposite of what is required to win.
The only thing I can think of which is worse than letting your losses run is adding to losing positions. If you just think about what this will achieve, by the end of it, you will have many shares of a dog. If you pyramid as the share price goes up, there's a good chance what you'll end up with is many shares of an absolute champion.
The reason why profits seem to be taken quickly by those who lose is that they take the view that you better lock in a profit while its there. Problem with that thinking is that you will never have any big winners.
It's not hard to be in the 3-5% of those that win. As William Eckhardt said:
"Anyone with average intelligence can learn to trade. This is not rocket science".
I'll end this post with a few quotes from Michael Covel's "Trend Following".
Referring to $8.4million in WorldCom stock now only worth about $492,000: "Until you actually sell it, you haven't lost it" - Robert Leggett, Kentucky Retirement Systems.
"If you had Enron in your portfolio and didn't sell it at $90 or even at $10, don't feel embarrassed. On the surface it had always seemed to be a fairly good growth stock. We bought it all the way down" - Alfred Harrison, money manager at Alliance Capital Management Holding LP.
(The above 2 are professional money managers, these guys actually get paid to manage other people's money).
Enron stock was rated as "Can't miss" until it became clear that the company was in desperate trouble, at which point analysts lowered the rating to "sure thing". Only when Enron went completely under did a few bold analysts demote its stock to the lowest possible Wall Street analyst rating, "Hot Buy".
(Gotta love analysts).
Wednesday, September 26, 2007
Money Management
Money management is a very overlooked component of a trading system. This is unfortunate because, arguably, it is your money management that can have much more an impact on your results than entries or exits.
It's money management, often referred to as position sizing, which dictates how quickly you will compound your account when you are winning, and how slowly you will kill your account when you are losing.
Van Tharp, in his book Trade your way to financial freedom, tells an analogy of a test given to a number of Ph.D students (It was either 20 or 40). Each student was given $1000 to begin with and were told to play a game where they had a 60% chance of winning each time they played, and the size of each win and the size of each loss was the same. Obviously a positive expectancy system. There was no fee for playing and they could play as many times as they liked. None of the students made money except for two!
Why?
1. Either their position size was too large and/or
2. They followed the Martingale system.
Now, remember how I mentioned how trade frequency was very important in assessing the profitability of a system. A fair coin has a 50% chance of landing on heads and an equal chance of landing on tails when flipped. But if you flip the coin 10 times, there's more than a slim chance that you could get 8 or 9 heads consecutively. But flip it 100 times, and the results are more likely to be closer to 50%.
The Martingale system originates from gambling, and says that, you should double your betting size after you lose. Those that have even a basic understanding of probabilities know that the probability of a win is something that does not change irrespective of whether the previous bet was a win or a loss.
And note that once a trading system is in a period of drawdown, its very hard to even claw back to breakeven. If you lose 50%, you have to double your money, i.e., make 100% to just breakeven.
A common method of position sizing used by those that win are either the:
*Fixed percent risk, or
*Percent of total equity, models.
With these methods, when your account size (closed equity) increases, you position size increased, and when your account is in drawdown, your position size decreases. These methods, in essence, can be considered an anti-Martingale approaches. They work to decrease the risk of ruin.
Remember what Larry Hite said: If you lose all your chips, you can't bet.
To conclude, I just want to demonstrate the effect that money management can have on a system.
Systems 1 and 2 are exactly the same system. The only difference is in their money management (and their results!).
System 1 uses $10k equal dollar units.
System 2 uses 10% of total available capital.
Starting capital is $100k.
5000 Monte Carlo simulations were run.
System 1:
Average Number of Trades Executed: 346
Average Profit: $372,838.43 (372.84%)
Average Absolute Percent Drawdown: 9.4199%
System 2:
Average Number of Trades Executed: 152
Average Profit: $498,482.84 (498.48%)
Average Absolute Percent Drawdown: 8.2077%
Backtesting with TradeSim
There are 3 ways to know whether you are trading with a valid (profitable) strategy. A system that has a positive expectancy.
1. Backtest it.
2. Paper trade it.
3. Trade it in real time.
Do either of the 3 over a long enough time frame to get a statistically significant number of trades, with documentation of average win, average loss, and the win percentage, and expectancy can be easily calculated. As discussed previously, trade frequency is also important.
Okay, so let's go through the options. Paper trading is fine, though some say it is boring and doesn't test the emotions the same way that real trading does, but at least with this method, dollars are not risked. Still, it would take a long time to get a statistically significant number of trades. And very hard to be able to test the system over a variety of market conditions. Same problems with real time trading, but the main concern with this is, why would you trade a system and risk real dollars if you have no idea of the probabilities of whether your system will make a profit or not?
Backtesting, with correct validation techniques (i.e. walk forward analysis) can provide a very good indication is a system is likely to profitable when traded in real time.
The software I use for all my backtesting is TradeSim. It was initially designed to be a plugin for MetaStock (MS) but also be used with Amibroker and Bullcharts though the integration isnt as seamless as it with MS.
TradeSim is a trading simulator and backtester that is design to test a system exactly as if you were trading it. So trades will not be taken if there is insufficient capital, brokerage fees are factored in, there are different models of slippage to choose from, several position sizing strategies can be explored, and volume information can be also considered.
But the standout feature for me was monte carlo analysis capabilities. Because many trading systems do not have enough capital to be able to take every trade that is signalled, is it of paramount importance that the system be exhaustively tested through as many combinations and permutations of trades and portfolios that is possible.
David, the developer of TradeSim gives this example in his excellent User Manual. Take the heavily publicised MACD crossover system, where the entry is when the fast moving average crosses the slower one to the upside, and the exit when the fast moving average average crosses the slower one to the downside. If you only do a single portfolio simulation, it is possible that the results show that the system makes 22%pa. But if you run the simulation again, its a 2% gain. And then run it again its a 38% loss. Its only when you run it through 10,000 or 20,000+ simulations that you realise its a break-even system at best.
Another feature of TradeSim which is useful is the options for slippage. Slippage is the difference between the price you want and the price you actually get. Say for example, your system is set so that if you get a signal today, then you will buy tomorrow's open. If you set TradeSim to default order, then it will buy what the next day's opening price actually was. No slippage is taken into account here.
If you set TradeSim to market order, the program, through a random algorithm, with randomly choose a price between the day's hi price and lo price as the entry price. Slippage is the difference between the price you are getting and the price you wanted (the actual open). One of the other options for slippage is worst case, where when you buy, you pay the day's hi.
One way of testing the robustness of the method is to play with different slippage models. If a change from default order to market order, or market order to worst case, puts your very profitable system into the red, then you have some work to do.
With the system I'm currently developing, which is a weekly system, i was designing it with flexibility in mind. I may not be able to buy Monday's open all the time. So I set market orders during backtesting which gives me the flexibility to buy anytime during the week.
Tuesday, September 25, 2007
Trading with an Edge
An edge can also be referred to as a positive expectancy.
My aim and I'm sure this is the aim of all those that design and develop trading systems, is to develop a model which has a mathematical positive expectancy.
Having a system with a positive expectancy basically means that over the long run, over X number of trades and N number of years, the system will make money.
The formula for Expectancy is.
(Average win * Percent winners) - (Average loss * Percent losses).
This figure, if positive, tells you how much you will make, per trade.
Expectancy can also be calculated as a % per dollar risk.
Now "per trade" is the key part, and this is something that I only fully understood a few days ago.
I was testing two systems, both systems were exactly the same, and what I was trying to do was optimise the ATR multiplier.
When I was testing the system with a 2.5*ATR trailing exit, these were the relevant stats.
- Average win/Average loss = 5.4654
- Percent winners = 42%
- Average profit per trade = $1,285.02 (Expectancy)
- Annualised compound interest rate = 22.6859%
When testing with a 2*ATR trailing exit, these were the relevant stats.
- Average win/Average loss = 3.0486
- Percent winners = 49%
- Average profit per trade = $754.89 (Expectancy)
- Annualised compound interest rate = 31.5157%
Now, note how both systems have a positive expectancy. The first system has a greater expectancy than the second one YET the rate of return of the second system is much higher.
Why?
(Note that even though system 2 has more percent winners than system 1, this is taken into account when working out the average profit per trade)
The answer to the WHY lies in what i was highlighting before.
Trade frequency.
System 1 in fact made only 109 trades in the testing period (6 years).
System 2 made 247 trades.
In summary, whilst its crucial to trade a system with a positive expectancy, its also important to keep in mind that trade frequency is what determines how often you can exploit that edge.
Its useless having a system that picks 80% winners and has an average win/average loss of 12 if it only trades 8 times a year.
Van Tharp's Trade Your Way to Financial Freedom is one of the best texts I've read on the topic of expectancy.