MLeap and Spark ML SQLTransformer

2020-01-02 Thread femibyte
1 I have a question. I am trying to serialize a PySpark ML model to mleap. However, the model makes use of the SQLTransformer to do some column-based transformations e.g. adding log-scaled versions of some columns. As we all know, Mleap doesn't support SQLTransformer - see here :

unsubscribe

2020-01-02 Thread Amit Jain
unsubscribe

Re: 'ExecutorTaskSummary' alternative in Spark 2.3 onwards

2020-01-02 Thread Ninja Coder
Any advise/help here is much appreciated. On Mon, Dec 30, 2019 at 4:16 PM Ninja Coder wrote: > I have a spark streaming application (currently spark 2.2) which is using > `org.apache.spark.ui.exec.ExecutorTaskSummary` to grab executor details > like > duration, tasks failed, tasks completed, GC

Re: Java 11 support in Spark 2.5

2020-01-02 Thread Jatin Puri
>From this >(http://apache-spark-developers-list.1001551.n3.nabble.com/DISCUSS-Spark-2-5-release-td27963.html#a27966), > looks like there is no confirmation yet if at all Spark 2.5 would have JDK 11 >support. Spark 3 would most likely be out soon (tentatively this quarter as per mailing list).