>>>> X = np.array([[0, 10000], [10000, 0]]) >>>> observed = X >>>> Y = np.array([[0,1], [1,0]]) >>>> expected = np.dot(np.atleast_2d(Y.mean(axis=0)).T, >>>> np.atleast_2d(X.sum(axis=0))) >>>> chisquare(observed, expected) > (array([ 10000., 10000.]), array([ 0., 0.])) > > It may not be pretty, but it is expected behavior since the p-value > will be too small to represent. If this is bothering you in your own > code, then you could binarize your features using a cutoff before > handing them to chi2, or use logarithmic scaling on your frequencies > with > > TfidfTransformer(sublinear_tf=True, use_idf=False) Thanks, I'll give that a try.
------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
