Gonzaga --
YES, walk forward testing is useful. Typically, it's purpose is to
verify, or not, that your system is robust and not over optimized. I
say 'typically' only because I might imagine that some one might want to
try to determine if his system might work better using variable values
that worked in the most recent past. This use would not be 'typical'.
You say, "changing continuously the variables using the walk forward
with a 1 month step". If, you mean that you are using only one month IS
for each month OS, your results are telling you that only one month of
IS to adjust variables is not sufficient (this is most often the case).
You should probably use a much longer IS, maybe a year or more.
An interesting experiment would be to use 'anchored' IS and compare OS
results with with some non-anchored runs.
IS is not a 'test', but training (sorta like being in the students seat
next to a flight instructor). OS is a test (in a flight simulator
safely anchored to the ground), which, hopefully, will help prepare you
for flying over Manhattan among large flocks of geese.
-- Keith
Gonzaga wrote:
I am using the walk forward backtest in order to obtain a realistic
CAR data, but I am thinking, since some time ago, that perhaps the
walk forward study is not as good as everybody say.
I have a system that gets, during the last 10 years, a 68% annual
profit, and a 16% DD.
Of course, I obtain that CAR optimizing one variable, during the last
10 years, and using simultaneous stocks; I mean, is a very very big
sample..
The system has few variables, most of them comes from the only
indicator I use, with the standard parameters.
Another variable is what I optimize, obtaining the best result in 0.96.
When I perform a walk forward simulation, the CAR goes down to 29%.
Mr Howard Bandy say in his very good book `Quantitative Trading
systems' that the in-sample studies are not useful, and we have to
consider only out-of-sample studies.
But when I see my in-sample backtest, with a result of 68%, I see that
the value of 0.96 has a very good performance every year.
I mean, if I optimize only from 99 to 2003, e.g., the 0.96 value of
the variable has a very good performance. And occurs the same in
2003-2007, or in 2007-2009.
In the last two years, that has been so volatile, the 0.96 value is
not the best value by far. But, it still has a very good CAR of 105%,
although another values has better results.
So I wonder... Is really worth the walk forward? If an optimized
variable has a good performance over many years, not always the best,
but always a good performance..
Is preferably to use it instead of the madness of changing
continuously the variables using the walk forward with a 1 month step???
Thanks, and sorry for my English..
