On Sun, Oct 9, 2011 at 10:57 AM, Gael Varoquaux <[email protected]> wrote: > The problem is that the IPython parallel utilities integrate quite poorly > with batch-driven engines, like SGE. I haven't tried it out myself, but > this is really what I hear around me.
Well, if it integrates 'quite poorly', it would be nice to hear this as a bug report or a question on the ipython lists, because PBS/SGE support is precisely one of the current goals... Stefan just pointed me to this thread, otherwise I'd have no idea your office chatter has seen this problem. Min has been very responsive when people have given feedback, and there's *explicit* PBS explanation in our docs: http://ipython.org/ipython-doc/dev/parallel/parallel_process.html#using-ipcluster-in-pbs-mode because large-cluster support is precisely one of the targets we have. But fixing unreported bugs we don't know about and that don't get reported is impossible. If in addition what you decide to do is to spread the notion that it 'works poorly' without providing any specifics nor feedback, and without even saying it on our own lists, it's really not helpful. It's simply classic FUD, and I'm quite surprised and saddened to hear it coming from a core IPython developer. Supporting all the large-cluster environments out there robustly is super hard, but we do have infrastructure for it already in place, and for many people it *does work very well*. With the right feedback, we'll be happy to try to do better for the cases where it doesn't. Cheers, f ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2dcopy2 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
