Hello,
I have a very unusual situation with an SVM and wanted to get the
group's opinion.
We developed an experiment where we train the SVM with one set of data
(train data) and then test with a completely independent set of data
(test data). The results were VERY good.
I found and error in how we generate one of or training variables. We
discovered that it was indirectly influenced by future events. Clearly
that needed to be fixed. Fixing the variable immediately changed our
results from good to terrible. (Not a surprise since the erroneous
variable had future influence.)
A friend, who knows NOTHING of statistics or math, innocently asked,
"Why don't you just keep that variable since it seems to make your
results so much better." The idea, while naive, led me to thinking. We
can include future data in the training set, since it occurred in the
past, but what to do with the test data from today? As a test, I tried
simply setting the variable to the average of the value in the training
data. The results were great! Now since the data is scaled, and we set
the variable to the same value (constant from average of training data.)
it scaled to 0. Still, great results.
To summarize:
Bad var in training + Bad var in testing = great results
Good var in training + Good var in testing = bad results
Bad var in training + Constant in testing = great results.
I'm not an expert with the internals of the SVM, but clearly the bad
variable is setting some kind of threshhold or intercept when defining
the model. Can someone help me figure out why/how this is working?
Thanks!
--
N
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
