Powered by Spark

2018-06-05 Thread Sudhir Menon
SnappyData Inc. http://snappydata.io SnappyData is a high performance in-memory data platform for mixed workload applications. Built on Apache Spark, SnappyData provides a unified programming model for streaming, transactions, machine learning and SQL Analytics in a single cluster. SnappyData

Re: TextSocketMicroBatchReader no longer supports nc utility

2018-06-05 Thread Jungtaek Lim
FYI: Filed https://issues.apache.org/jira/browse/SPARK-24466 and provided the patch https://github.com/apache/spark/pull/21497 2018년 6월 5일 (화) 오전 11:30, Jungtaek Lim 님이 작성: > Yeah that's why I initiated this thread, especially socket source is > expected to be used from examples on official

Re: Revisiting Online serving of Spark models?

2018-06-05 Thread Nick Pentreath
I will aim to join up at 4pm tomorrow (Wed) too. Look forward to it. On Sun, 3 Jun 2018 at 00:24 Holden Karau wrote: > On Sat, Jun 2, 2018 at 8:39 PM, Maximiliano Felice < > maximilianofel...@gmail.com> wrote: > >> Hi! >> >> We're already in San Francisco waiting for the summit. We even think

Re: [build system] meet your build engineer @ spark ai summit SF 2018

2018-06-05 Thread Holden Karau
That's awesome! On Tue, Jun 5, 2018 at 12:23 PM, shane knapp wrote: > just a reminder to come meet your build engineer! > > we'll also be having a couple of demos of current projects in the lab: > tuesday (today) 130pm -- pandas on ray (https://rise.cs.berkeley.edu/ > blog/pandas-on-ray/) >

Re: [build system] meet your build engineer @ spark ai summit SF 2018

2018-06-05 Thread shane knapp
just a reminder to come meet your build engineer! we'll also be having a couple of demos of current projects in the lab: tuesday (today) 130pm -- pandas on ray ( https://rise.cs.berkeley.edu/blog/pandas-on-ray/) wednesday (tomorrow) 130pm -- ray (http://ray.readthedocs.io/) shane On Wed, May 2,

Why can per task‘s memory only reach 1 / numTasks , not greater than 1 / numTasks in ExecutionMemoryPool ?

2018-06-05 Thread John Fang
In fact not all tasks belong to the same stage. Thus, per task may be is deferent for the dependence of memory. For example, the executor are running two tasks(A and B), and the ExecutionMemoryPool own 1000M. We can hope the task-A occupy 900M, and task-B occupy 100M due to the task-A need much