> It would be nice to do something else instead of crash and burn, but for the
> moment that's on the user.
I think that in recent Python versions segfault can be captured.
--
Dive into the World of Parallel Programming T
Implementing this directly in the estimators seems very messy.
If we had decent logging, we could try that. Unfortunately we don't.
Pretty printing could also be achieved via a logging mechanism, so that
people could define it themselves.
I don't think it is something we necessary want to provide.
Hi,
I think some memory monitoring/warning stuff would be very helpful in general.
As far as I know, memory usage via e.g,. psutil is not supported by every OS or
machine, but we could add an optional "monitor_memory" parameter to
estimators/transformers like
SomeEstimator(..., monitor_memory=
It would be nice to do something else instead of crash and burn, but for
the moment that's on the user.
Well the kernel approximation should make it work. If you are after
visualization I'd also recommend the T-SNE
from this branch:
https://github.com/scikit-learn/scikit-learn/pull/4025
On 0
Hey Andy,
Hmmm, that might be it. My machine only has 8GB of RAM - why didn't I
think of that? Indeed the RAM usage seems to have pretty large
fluctuations for the process, and when I re-run now instead of just
silently dying its choking up my whole computer - indicative of a RAM issue.
Thank y
Hi Steve.
Can you monitor the RAM usage before it fails?
Because of the complexity of the algorithm, and as we don't truncate the
rbf kernel, this will take 16GB of ram.
If the process starts swapping, your OS might just kill it. There is
nothing much we can do about that.
A solution to runnin
Hey Andy,
Sorry, yes, by failing I mean it never finishes, and the python process
dies without raising any exceptions.
The shape of the data is (46196,114).
Also numpy.all(numpy.isfinite(my_data)) returns True before I call
transformer.fit()
I'm running on python 2.7.8 numpy 1.9.1 sklearn 0.15.2
Hi Steve.
So by failing, you mean it never finishes?
Or the python process dies?
What is the shape of your data?
Andy
On 03/23/2015 02:14 PM, Stephen O'Neill wrote:
Hi Sklearn,
I'm using Kernel PCA with the rbf kernel for projecting data into 3
dimensions for viewing alongside normal PCA and
Hi Sklearn,
I'm using Kernel PCA with the rbf kernel for projecting data into 3
dimensions for viewing alongside normal PCA and a stereographic projection
class that I wrote myself. Both the PCA and SGP classes seem to be
functioning correctly on this data set, but when I get to the .fit() method
