Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Joel Nothman
I was all too glad to put together a patch: https://github.com/scikit-learn/scikit-learn/pull/3580 On 21 August 2014 01:34, Vlad Niculae wrote: > It has confused me as well, +1. > > It's counterintuitive and broken, in my opinion. > > Vlad > > On Wed, Aug 20, 2014 at 2:31 PM, Gael Varoquaux >

Re: [Scikit-learn-general] ValueError: numpy.dtype has the wrong size, try recompiling

2014-08-20 Thread Veck Hsiao
I have no idea about recompiling. But I found the solution. I remove scikit-learn installed by pip and install with https://github.com/scikit-learn/scikit-learn. And the problem was resolved. Thanks everybody! *Veck Hsiao @ PLSM Lab in Dept. of CS in NCCU* *About me: http://fbukevin.github.io

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Vlad Niculae
It has confused me as well, +1. It's counterintuitive and broken, in my opinion. Vlad On Wed, Aug 20, 2014 at 2:31 PM, Gael Varoquaux wrote: >> It's been around for so long, but it's also hard to believe that anyone >> exploited this behaviour intentionally. Shall we be bold and just fix >> it

Re: [Scikit-learn-general] ValueError: numpy.dtype has the wrong size, try recompiling

2014-08-20 Thread km
yes.. try recompiling numpy then scipy and then scikit-learn Regards, Krishna On Wed, Aug 20, 2014 at 5:34 PM, federico vaggi wrote: > Are you sure you are 32/64 bit libraries uniformly? > > > On Wed, Aug 20, 2014 at 9:00 AM, Veck Hsiao wrote: > >> Hi ! all, >> >> I just installed scikit-lear

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Gael Varoquaux
> It's been around for so long, but it's also hard to believe that anyone > exploited this behaviour intentionally. Shall we be bold and just fix > it with a warning? +1 on my side. It has confused many people that I work with. -

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Joel Nothman
On 20 August 2014 21:41, Gael Varoquaux wrote: > On Wed, Aug 20, 2014 at 01:37:36PM +0200, federico vaggi wrote: > > Are there any reasons at all for keeping score function in its current > form? > > No. I think that it is a bug. I'd like it changed, but we need to agree > on a way that is not to

Re: [Scikit-learn-general] ValueError: numpy.dtype has the wrong size, try recompiling

2014-08-20 Thread federico vaggi
Are you sure you are 32/64 bit libraries uniformly? On Wed, Aug 20, 2014 at 9:00 AM, Veck Hsiao wrote: > Hi ! all, > > I just installed scikit-learn and followed with > http://scikit-learn.org/stable/tutorial/basic/tutorial.html > > Then I faced a problem when I tried to load dataset from sklea

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Gael Varoquaux
On Wed, Aug 20, 2014 at 01:37:36PM +0200, federico vaggi wrote: > Are there any reasons at all for keeping score function in its current form? No. I think that it is a bug. I'd like it changed, but we need to agree on a way that is not to disruptive to users (or simply decide that the current solu

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread federico vaggi
Ok - now it makes sense. As long as it uses the correct one internally during 'fit' - that's what matters. Are there any reasons at all for keeping score function in its current form? On Wed, Aug 20, 2014 at 12:55 PM, Joel Nothman wrote: > It's actually simpler than that issue, Michael. Gri

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Michael Eickenberg
Aah yes -- thanks for this rather importance piece of information! -- so it is using the scoring of the underlying estimator in precedence before the gridsearchcv's own one. Since we can't really change the internal default scorer of ExtraTreesClassifier, except by doing something horrible like, et

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Joel Nothman
It's actually simpler than that issue, Michael. GridSearchCV (and RandomizedSearchCV) has a score method that is unintuitive. It will generally not use the metric passed to `scoring`. But yes, in `fit`, it has used the correct scoring metric. IMO, it should be changed. But it's been this way since

Re: [Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread Michael Eickenberg
Hi Federico, I recall an issue at the beginning of this year stating that internally GridSearchCV sometimes defaulted to accuracy scoring even though a different scorer was passed. I am not sure though if this is what you have encountered. There is some code in https://github.com/scikit-learn/sci

[Scikit-learn-general] Custom Scoring Functions for Grid Search

2014-08-20 Thread federico vaggi
Hi everyone, I'm working on a classification task with ExtraTreesClassifier that deals with somewhat imbalanced datasets, so instead of using accuracy as a metric, I'm using MCC. However - there's some behaviour which doesn't make perfect sense to me, for example - after doing this: score_func =

[Scikit-learn-general] ValueError: numpy.dtype has the wrong size, try recompiling

2014-08-20 Thread Veck Hsiao
Hi ! all, I just installed scikit-learn and followed with http://scikit-learn.org/stable/tutorial/basic/tutorial.html Then I faced a problem when I tried to load dataset from sklearn. Python 2.7.5 (