Backtesting
It has become an everyday word……backtesting. You can hardly avoid seeing it, because so many marketing efforts are convincing traders during this choppy market that backtesting is the magic bullet for successful trading. I’m not so sure I agree.
Maybe a PC tells you that a given trading system would have made or lost X dollars over the past 6 months. So what? Isn’t the market always in motion, always changing? If it is so easy to just figure out what worked over the past month and apply it toward the coming month, then wouldn’t the market be showing us some phenomenal trends right now with everyone pushing stocks in the same direction rather than the back-and-forth choppy action we see so much of?
Instead, we’re getting a LOT of program trading that keeps things choppy as dips are bought and rallies are sold. With the trendless market, it has truly become a market of stocks. Good swing trading right now is about doing your homework to locate good technical patterns and then keeping your risk/reward in check.
Don’t let glossy advertising from software vendors fool you into thinking that your trading strategy just isn’t complicated enough. Remember Gartman’s rule and the simplicity it offers. Keep tabs on the market environment and adjust your trading plan accordingly. If a good trend exists, then look for some continuation setups like flag patterns and triangle patterns. If prices are stuck in a range, then consider some reversal setups like the double top or trade a channeling stock. Your ability to adapt to existing conditions is what will add to your P&L, not your ability to backtest effectively.
Successful trading is about recognizing what kind of market you’re in and how to trade it most effectively, even if it means that you stand aside at times. Hoping that you could spend the next long weekend backtesting a magic formula based on past conditions may just be a shortcut that will only leave you more frustrated.
Jeff White
President, The Stock Bandit, Inc.
www.TheStockBandit.com
Michael Taylor | Mar 8, 2006 | Reply
I’m sure part of this post was a jab to testers like myself to get some response. Well, you’re getting one. Ha ha. 🙂
I think you’re right in some ways and wrong in others. What’s happening now in the retail trading world is the same thing that happened in the corporate world back in the 90’s.
Back in the 90’s, everyone in the corporate world just threw money at technology…didn’t matter what it did or whether the results or ROI was better or worse than before. If the project contained one of the buzzwords…then another million thrown. Now, things are different…the buzzwords don’t mean as much…substance is more important than some slick ad or salesman.
And I think retail trading is at this technological stage where everyone wants to backtest, forward test, fundamentally rank, or any other buzzword that is hot. All without knowing what any of it really means or how to evaluate the results these things can produce.
An example of this is the Wealth-Lab forum has a very funny topic where new users are wanting to trade this fantastic system that peaks a bar ahead. And cannot understand why they cannot trade this peak ahead system because the backtested results are so amazing! 🙂
Anyway, back to your main point…that your trading strategy should be as simple as can be…but not any simpler. Markets change and bring about new environments and your trading strategy should adjust. I agree 100% with all of this. But, please, keep one thing in mind…something one of my programming mentors told me back in the day. If it can be explained…it can be programmed. If you can come up with a trading strategy…it can be coded and tested…and more importantly should be coded and tested.
Trading systems can follow trends. Flags, triangles, double tops, and channeling stock patterns can be coded. And all can be tested and quantified. If this can be done…shouldn’t it be done?
I believe it should but with one word of caution…you’ve got to know how to test and evaluate the results all these wonderful backtests produce. Without that…you might as well keep to your golden gut.
Oh, and most importantly, you’ve got to be able to logically express ideas into computer code. There’s an art to that as well. Some people wouldn’t know the first step in converting patterns such as a double top into logic a computer can understand. But, that doesn’t mean it cannot be done. 🙂
Just my two cents…keep the change.
Take care,
MT
Administrator | Mar 8, 2006 | Reply
Hey Michael,
Thanks for your two cents! I think that your input is certainly valid, so let me explain the point I was trying to get across.
I agree with you completely in that being able to code systems to locate patterns should be done, especially by those who are able to do it (those with the ability to mathematically express just what a chart pattern like a flag looks like). Perhaps I should have clarified better what my point was…..that the weekend warrior who reads in a trading magazine at Barnes & Noble that he should backtest a strategy can go home, download the software and start tinkering that afternoon. This hypothetical trader will likely continue to adjust the inputs & variables to improve performance, and by Saturday at dinner he’s got random inputs that would have tripled his account in the last month. Come Monday’s opening bell, he might put it into action and start taking trading signals based on what his 3 hours generated! That’s poor judgement and exactly the kind of trader I’m referring to.
The glossy ads seem aimed at the guy who’s been buying random stocks while the market chops around and he can’t understand why his account is lower than it was on Jan. 1. So, the thought of backtesting appeals greatly to him as he wants to be able to quit his day job.
I should have done better to clarify that the “backtester” I was making a jab at is the amateur part-timer who wants to know what the perfect trading system would have made him in the past, and how he might apply that towards tomorrow’s session. THAT is the guy that I think is a victim of the ideal that he can put a little elbow grease into his strategy software and POOF, have a wonderful and profitable strategy.
When it comes to locating patterns based on a mathematical interpretation and having a PC locate those for you, I’m all for that.
I just don’t think that backtesting to buy the 4th red bar after a moving average crossover with the TRIN at lows is going to make anybody any money, even if it might have worked like a charm last week!
Jeff
Michael Taylor | Mar 8, 2006 | Reply
Agree 100% with your comments. Thanks for the clarification.
Now, was that the 4th red bar after or before the moving average crossover? Ha ha. 🙂
Here’s that funny forum topic I was referring to: http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/topic?id=7536&start=0
Have a good week,
MT
Dino | Mar 9, 2006 | Reply
Interesting thoughts on backtesting. The biggest problem that I have found with backtesting is simple. Time. Many systems simply do not go back far enough. For example, let us assume that the current market that we are in (flat) is due for some type of 10% correction. What good is a backtesting program that only goes back a few months, or even a few years? NONE. I can see the significance in backtesting programs that give data going back 5 to 10 years, where someone can test there ideas in both bull and bear markets. I am currently playing around with the backtesting features in Tradestation and with Zacks. I find them interesting. No matter what results they may come up with, a trader needs to be able to read a chart and make that finally dicision before buying. Good luck all.
john | Mar 12, 2006 | Reply
Backtesting should always be done with current data and across several different timeframes. I like using 90 days of data in 4 increments – 90 days ago, 60 days ago, 30 days ago and today. By doing this I am overlapping several increments and testing two totally fresh batches. If I can get consistent results in both win rate and roi across all tests I feel that I have a good, current indicator.
Would that indicator have worked 5 years ago? Maybe, but, really, who cares? Markets change, chop is the norm, the 90’s will probably never exist again. So if I test a system that works great from 99 to 2000 should I use it today? Only if it tests well today.
EMB | Oct 9, 2006 | Reply
I’ve done a lot of work that testing both in the commodities and the stock markets, and I’ve seen the pitfalls. However, it is possible to avoid them with the right kind of analysis. I’ve been doing very well over the past two-plus years in the stock market using a strategy that I developed through backtesting the previous five years of markets. The actual results (so far, anyway) are fairly consistent with the backtesting results. So I wouldn’t dismiss backtesting out of hand if I were you.