Re: [Scikit-learn-general] : SVM and Sparse Data in (latest) version of sklearn

2012-09-11 Thread Olivier Grisel
2012/9/11 Dimitrios Pritsos : > > Thank you very much Olivier. > > Yes I ve seen the api ref, however, I ve seen the svm.sparse folder in > the git, too. And I was confused. It's just to preserver backward compatibility for a couple of versions. You should get a deprecation warning if you use this

Re: [Scikit-learn-general] : SVM and Sparse Data in (latest) version of sklearn

2012-09-11 Thread Dimitrios Pritsos
Thank you very much Olivier. Yes I ve seen the api ref, however, I ve seen the svm.sparse folder in the git, too. And I was confused. Best Regards, Dimitrios On 09/11/2012 01:30 PM, Olivier Grisel wrote: > 2012/9/11 Dimitrios Pritsos : >> Hello, >> >> Could you please tell me how sklearn tre

Re: [Scikit-learn-general] : SVM and Sparse Data in (latest) version of sklearn

2012-09-11 Thread Olivier Grisel
2012/9/11 Dimitrios Pritsos : > Hello, > > Could you please tell me how sklearn treats sparse matrices when one is > using SVM API in the latest fromGithub version. In previous version > there was a .sparse, which it is still there, however it has been > removed from the 0.12 API Ref. Should I trea

[Scikit-learn-general] : SVM and Sparse Data in (latest) version of sklearn

2012-09-11 Thread Dimitrios Pritsos
Hello, Could you please tell me how sklearn treats sparse matrices when one is using SVM API in the latest fromGithub version. In previous version there was a .sparse, which it is still there, however it has been removed from the 0.12 API Ref. Should I treat my sparse matrices, using the old s

Re: [Scikit-learn-general] Memory explosion with GridSearchCV

2012-09-11 Thread Olivier Grisel
2012/9/11 Olivier Grisel : > > I will open an issue to track this. Done: https://github.com/scikit-learn/scikit-learn/issues/1137 -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel -- Live Security Virtu

Re: [Scikit-learn-general] Memory explosion with GridSearchCV

2012-09-11 Thread Olivier Grisel
2012/9/11 Olivier Grisel : > > Anyway this simulation is probably not representative of your scenario > as integers can be unboxed in the array datastructure hence the copy > while string objects cannot and the array will only store references > to the original string objects. Actually I was wrong

Re: [Scikit-learn-general] Memory explosion with GridSearchCV

2012-09-11 Thread Andreas Mueller
> Well you still have the original x in memory so you should at least > expect a doubling of the memory. The remaining memory might be > temporary stuff allocated during the conversion although that seems > weird. You can add: > > del x > import gc > gc.collect() > > Anyway this simulation is prob

Re: [Scikit-learn-general] Memory explosion with GridSearchCV

2012-09-11 Thread Olivier Grisel
2012/9/11 Christian Jauvin : > Hi, > > I'm working on a text classification problem, and the strategy I'm > currently studying is based on this example: > > http://scikit-learn.org/dev/auto_examples/grid_search_text_feature_extraction.html > > When I replace the data component by my own, I have fou