RE: private methods in mllib

I just want things to get running and sometimes it is the easiest and fastest to adapt the code a bit here and there. Scala implicits are lovely to do so, however, if things are private there is no way to add extra functionality. Currently I am just copying the spark classes which I need access

Re: private methods in mllib

So I want to be clear, many of these things are private in org.apache.spark.ml as well. The implementation details of the algorithms may change, so changing parts of the internals isn't easily supported. What are you trying to change or add? On Fri, Dec 1, 2017 at 10:29 AM, Jacek Laskowski

Re: private methods in mllib

Hi Sahm, Unless I'm mistaken [1], but org.apache.spark.mllib is put on hold and is considered @deprecated these days. That'd explain why "so many things made private". [1] https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/package.scala#L21 Pozdrawiam, Jacek

Re: Rolling policy in Spark event logs for long living streaming applications

There's really no current solution to this. There's a brief discussion about it on SPARK-12140. Here we recommend people disable event logs for streaming apps, as sub-optimal as that might be... On Fri, Dec 1, 2017 at 3:27 AM, kankalapti omkar naidu wrote: > Dear spark

Rolling policy in Spark event logs for long living streaming applications

Dear spark developers, I did not find rolling policy for spark event logs,for long living streaming applications the event log is becoming huge with in few hours. The only way I found is fixing the existing event logging spark listener(EventLoggingListener), but I believe there should be already

private methods in mllib

Dear spark developers, just tried to modify the MultivariateGaussian from org.apache.spark.mllib.stat.distribution and failed miserably because there are so many things made private. Why? What would be a good way to deal with this? Rewrite everything? thanks a lot, best, Stephan Stephan

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

Congrats. On Fri, Dec 1, 2017 at 12:10 AM, Felix Cheung wrote: > This vote passes. Thanks everyone for testing this release. > > > +1: > > Sean Owen (binding) > > Herman van Hövell tot Westerflier (binding) > > Wenchen Fan (binding) > > Shivaram Venkataraman (binding) >

[RESULT][VOTE] Spark 2.2.1 (RC2)

This vote passes. Thanks everyone for testing this release. +1: Sean Owen (binding) Herman van Hövell tot Westerflier (binding) Wenchen Fan (binding) Shivaram Venkataraman (binding) Felix Cheung Henry Robinson Hyukjin Kwon Dongjoon Hyun Kazuaki Ishizaki Holden Karau Weichen Xu 0: