May I ask why are you giving the dont care examples to the algorithm. Cant you weed them out. Is adaptive lr the same as weighted lr.. which is used when you have unbalanced training examples?
On Wednesday, December 5, 2012, Raman Srinivasan <raman.sriniva...@gmail.com> wrote: > I am trying to classify a set of short text descriptions (1 - 15 words > long) into a handful of classes. I am following the approach in the 20 > newsgroup example using Adaptive Logistic Regression. > > There are a couple of twists to the problem I am solving: > 1) Only a small set of descriptions result in useful classifications - bit > like finding a needle in a haystack. My training data has a grab bag > classification called "DON'T CARE" into which 90-95% of the descriptions > end up. The remaining 5-10% are classified into roughly 10 classes. > 2) There are certain words (features) in the description that immediately > imply its classification unambiguously. However, they do not occur very > frequently in the data set. > > When I train and test with this data set, the overall classification > accuracy is very high (98%) except a high proportion of the incorrect > classifications occur for the descriptions I am most interested in. I guess > one should expect this intuitively! > > What's the best way to model this scenario? Is it better to exclude the > "DON'T CARE" descriptions from the training set for SGD? Because when the > proportions accurately reflect the real data set, the classification error > rate for the interesting subset is too high! > > Appreciate any ideas... > -- Mohit "When you want success as badly as you want the air, then you will get it. There is no other secret of success." -Socrates
