Steve,
that's an interesting observation. So far I had exclusively used CMAE,
but now I've applied SPSO to some of my trading systems by performing an
extensive walk-forward optimization.
My (surely not representative) observations are that the IS results are
rather similar regardless if I use CMAE and SPSO. However, I found that
the OOS results with SPSO were always worse than with CMAE. This seems
to support your findings that CMAE tends to find the broad plateaus - and
this leads eventually to better OOS performance (through better
generalization). Since I never trade a system without thorough WF
testing, that's the crucial aspect for me as IS results alone are more
or less worthless.
I should add that I used OptimizerSetOption("Runs", 1 ) and
OptimizerSetOption("MaxEval", 1000 ). Different parameters might lead to
different conclusions.
Greetings,
Thomas
Steve Dugas wrote:
> To test the built-in IO engines, I ran a few exhausive opts and saved
> the results, then ran lots of IO tests and compared them to the
> exhaustive results to see what the IO's found and also what they
> missed. You could say that CMAE seems to take the "safe" approach,
> IMHO it finds the broad plateaus pretty well but as you might guess
> they are usually far from the most profitable. In my experience, the
> other two IO engines will generally find those too but they also find
> a lot of the smaller and more profitable ones, which you can then run
> a mini exhaustive opt on to get a more complete picture.
