Re: [Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Matthieu Brucher
Hi, I think some of the algorithms already offer this (Laplacian Eigenmaps for instance). I'm -1 for LLE as LLE does not compute distances, but weights based on the points directly. Matthieu 2011/9/21 Jacob VanderPlas > Hello, > I recently was contacted by someone interested in using manifold

Re: [Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Mathieu Blondel
On Wed, Sep 21, 2011 at 8:56 AM, Robert Layton wrote: > I do really like the metric='precomputed' concept, which allows both > implementing actual metrics (euclidean, manhattan) as well as passing a > precomputed array in. If the algorithm doesn't allow it for whatever reason* > throw an error. T

Re: [Scikit-learn-general] sklearn doc assumptions

2011-09-20 Thread Gael Varoquaux
Hi Satra, On Tue, Sep 20, 2011 at 01:14:46AM -0400, Satrajit Ghosh wrote: > exchange between me and a student on current documentation. i don't know > what level you are expecting people to be at. this is somebody who is using > sklearn and understands cross-validation in principle (a 2nd year > u

Re: [Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Robert Layton
On 21 September 2011 09:50, Gael Varoquaux wrote: > On Tue, Sep 20, 2011 at 04:25:58PM -0700, Jacob VanderPlas wrote: > > I recently was contacted by someone interested in using manifold > > learning methods on abstract metric spaces: that is, the training data > > is a matrix of pairwise distance

Re: [Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Gael Varoquaux
On Tue, Sep 20, 2011 at 04:25:58PM -0700, Jacob VanderPlas wrote: > I recently was contacted by someone interested in using manifold > learning methods on abstract metric spaces: that is, the training data > is a matrix of pairwise distances rather than a set of points. It would > be fairly str

Re: [Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Robert Layton
On 21 September 2011 09:25, Jacob VanderPlas < vanderp...@astro.washington.edu> wrote: > Hello, > I recently was contacted by someone interested in using manifold > learning methods on abstract metric spaces: that is, the training data > is a matrix of pairwise distances rather than a set of point

[Scikit-learn-general] manifold learning on abstract metric spaces

2011-09-20 Thread Jacob VanderPlas
Hello, I recently was contacted by someone interested in using manifold learning methods on abstract metric spaces: that is, the training data is a matrix of pairwise distances rather than a set of points. It would be fairly straightforward to implement this for basic LLE and Isomap, and could

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Olivier Grisel
2011/9/20 Gael Varoquaux : > On Tue, Sep 20, 2011 at 11:20:03AM +0200, Olivier Grisel wrote: >> > Look at it this way: this is an opportunity to teach people how to >> > properly structure their scripts for properly for use with joblib on >> > all platforms. :-) > > Yes, I agree with this. I'll add

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Gael Varoquaux
On Tue, Sep 20, 2011 at 04:35:36PM +0200, Lars Buitinck wrote: > There's also a minor efficiency issue where as_float_array is called > on the result of atleast_2d, which might incur two copies. We may want > to rethink this, maybe add a safe_ replacement for atleast_2d with a > boolean copy argume

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Lars Buitinck
2011/9/20 Olivier Grisel : > 2011/9/20 Fabian Pedregosa : > > I find the opposite: > > $ git grep "\bcopy=" | wc -l > 72 > $ git grep "\boverwrite_X=" | wc -l > 44 Well, there's also a few overwrite_Gram instances. > I prefer the copy={True,False} naming convention. It sounds friendlier, doesn't

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Gael Varoquaux
On Tue, Sep 20, 2011 at 04:27:58PM +0200, Fabian Pedregosa wrote: > So it's OK to change it. However, a quick grep shows that there are > more overwrite_? uses than copy_? in sklearn. That's however contrary to conventions in numpy. My personnal opinion is that when there is no ambiguity on what i

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Olivier Grisel
2011/9/20 Fabian Pedregosa : > On Tue, Sep 20, 2011 at 4:10 PM, Lars Buitinck wrote: >> Hi all, >> >> I only just spotted sklearn.utils.as_float_array. I might have missed >> a discussion about this, but I notice it and its clients use an >> overwrite_Whatever convention, while older modules commo

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Gael Varoquaux
On Tue, Sep 20, 2011 at 11:20:03AM +0200, Olivier Grisel wrote: > > Look at it this way: this is an opportunity to teach people how to > > properly structure their scripts for properly for use with joblib on > > all platforms. :-) Yes, I agree with this. I'll add a statement in the docs of joblib

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Fabian Pedregosa
On Tue, Sep 20, 2011 at 4:10 PM, Lars Buitinck wrote: > Hi all, > > I only just spotted sklearn.utils.as_float_array. I might have missed > a discussion about this, but I notice it and its clients use an > overwrite_Whatever convention, while older modules commonly use copy > and copy_Whatever. Wh

Re: [Scikit-learn-general] why as_float_array?

2011-09-20 Thread Gael Varoquaux
On Tue, Sep 20, 2011 at 04:09:54PM +0200, Lars Buitinck wrote: > I only just spotted sklearn.utils.as_float_array. I might have missed > a discussion about this, but I notice it and its clients use an > overwrite_Whatever convention, while older modules commonly use copy > and copy_Whatever. When h

[Scikit-learn-general] why as_float_array?

2011-09-20 Thread Lars Buitinck
Hi all, I only just spotted sklearn.utils.as_float_array. I might have missed a discussion about this, but I notice it and its clients use an overwrite_Whatever convention, while older modules commonly use copy and copy_Whatever. When has as_float_array been introduced (wrt. releases) and can we c

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread xinfan meng
Yes, the problem is gone! Thanks, and the discussion is great. On Tue, Sep 20, 2011 at 5:20 PM, Olivier Grisel wrote: > 2011/9/20 Robert Kern : > > On Tue, Sep 20, 2011 at 02:49, Olivier Grisel > wrote: > >> 2011/9/20 Robert Kern : > > > >>> The fix is straightforward on sklearn's part: move the

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Olivier Grisel
2011/9/20 Robert Kern : > On Tue, Sep 20, 2011 at 02:49, Olivier Grisel > wrote: >> 2011/9/20 Robert Kern : > >>> The fix is straightforward on sklearn's part: move the code into a >>> function and call that function under an "if __name__ == '__main__':" >>> test. Or just move everything under th

Re: [Scikit-learn-general] Python.exe keep spawning when njobs inGridSearchCV is not 1

2011-09-20 Thread Olivier Grisel
2011/9/20 Robert Kern : > > I would additionally suggest that the test-running script also be > fixed to run correctly under Windows. Vlad, how you running the tests on windows? Do you experiment the "infinite dot" issue using nosetests? Or using the command: python -c "import sklearn; sklearn.

Re: [Scikit-learn-general] Python.exe keep spawning when njobs inGridSearchCV is not 1

2011-09-20 Thread Robert Kern
On Tue, Sep 20, 2011 at 03:17, wrote: >>But then people will just copy the file and modify it to set n_jobs to > be greater, and then ask here why it doesn't work on Windows. It's a > teaching moment. Embrace it! > > Yes and no. If I download sklearn on windows and the run make, it doesn't > fin

Re: [Scikit-learn-general] Python.exe keep spawning when njobs inGridSearchCV is not 1

2011-09-20 Thread bdholt1
>But then people will just copy the file and modify it to set n_jobs to be greater, and then ask here why it doesn't work on Windows. It's a teaching moment. Embrace it! Yes and no. If I download sklearn on windows and the run make, it doesn't finish the tests. To then tell me that I need to mod

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Robert Kern
On Tue, Sep 20, 2011 at 02:49, Olivier Grisel wrote: > 2011/9/20 Robert Kern : >> The fix is straightforward on sklearn's part: move the code into a >> function and call that function under an "if __name__ == '__main__':" >> test. Or just move everything under that __main__ test. > > Indeed I kee

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Vlad Niculae
Also in test suites, it restarts the suite in every process and you get infinite dots. On Tue, Sep 20, 2011 at 10:49 AM, Olivier Grisel wrote: > 2011/9/20 Robert Kern : >> On Mon, Sep 19, 2011 at 20:00, xinfan meng wrote: >>> Hi: >>>     I ran the sklearn sentiment classification codes >>> (http

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Olivier Grisel
2011/9/20 Robert Kern : > On Mon, Sep 19, 2011 at 20:00, xinfan meng wrote: >> Hi: >>     I ran the sklearn sentiment classification codes >> (https://github.com/scikit-learn/scikit-learn-tutorial/blob/master/solutions/exercise_02_sentiment.py) >> and found it keep creating new python.exe instance

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Robert Kern
On Mon, Sep 19, 2011 at 20:00, xinfan meng wrote: > Hi: >     I ran the sklearn sentiment classification codes > (https://github.com/scikit-learn/scikit-learn-tutorial/blob/master/solutions/exercise_02_sentiment.py) > and found it keep creating new python.exe instance, and then my computer > crash

Re: [Scikit-learn-general] Python.exe keep spawning when njobs inGridSearchCV is not 1

2011-09-20 Thread bdholt1
One of the drawcards of using python is that it is relatively platform independent - so windows users have a reasonable expectation that their code will run. Windows itself is not a problem - I've written many C++ programs using boost::thread that work just fine on windows. Perhaps this require

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Olivier Grisel
2011/9/20 xinfan meng : > Hi: >     I ran the sklearn sentiment classification codes > (https://github.com/scikit-learn/scikit-learn-tutorial/blob/master/solutions/exercise_02_sentiment.py) > and found it keep creating new python.exe instance, and then my computer > crashed because ran out of memor

Re: [Scikit-learn-general] Python.exe keep spawning when njobs in GridSearchCV is not 1

2011-09-20 Thread Vlad Niculae
Hello That is the way joblib multiprocessing behaves on Windows, as far as I know. Gaël is aware of this, but there are not that many scikit-learn users running windows on the mailing list. Best, Vlad On Tue, Sep 20, 2011 at 4:00 AM, xinfan meng wrote: > Hi: >     I ran the sklearn sentiment cl