Re: java.util.NoSuchElementException when serializing Map with default value

Hi Jakob, I had multiple versions of Spark installed in my machine. The code now works without issues in spark-shell and the IDE. I have verified this with Spark 1.6 and 2.0. Cheers, Kabeer. On Mon, 3 Oct, 2016 at 7:30 PM, Jakob Odersky wrote: Hi Kabeer, which version of Spark are you us

Re: [ANNOUNCE] Announcing Spark 2.0.1

There seems to be no 2.0.1 tag? https://github.com/apache/spark/tags On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > They have been published yesterday, but can take a while to propagate. > > > On Tue, Oct 4, 2016 at 12:58 PM, Prajwal Tuladhar > wrote: > >> Hi, >> >> It seems like, 2.0.1

Re: [ANNOUNCE] Announcing Spark 2.0.1

There is now. Thanks for the email. On Wed, Oct 5, 2016 at 12:06 PM, Michael Gummelt wrote: > There seems to be no 2.0.1 tag? > > https://github.com/apache/spark/tags > > On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > >> They have been published yesterday, but can take a while to propagat

Re: [ANNOUNCE] Announcing Spark 2.0.1

https://github.com/apache/spark/releases/tag/v2.0.1 ? On Wed, Oct 5, 2016 at 8:06 PM Michael Gummelt wrote: > There seems to be no 2.0.1 tag? > > https://github.com/apache/spark/tags > > On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > > They have been published yesterday, but can take a wh

PySpark UDF Performance Exploration w/Jython (Early/rough 2~3X improvement*) [SPARK-15369]

Hi Python Spark Developers & Users, As Datasets/DataFrames are becoming the core building block of Spark, and as someone who cares about Python Spark performance, I've been looking more at PySpark UDF performance. I've got an early WIP/request for comments pull request open

Re: welcoming Xiao Li as a committer

Congratulations, Xiao! Fred On Tuesday, October 4, 2016, Joseph Bradley wrote: > Congrats! > > On Tue, Oct 4, 2016 at 4:09 PM, Kousuke Saruta > wrote: > >> Congratulations Xiao! >> >> - Kousuke >> On 2016/10/05 7:44, Bryan Cutler wrote: >> >> Congrats Xiao! >> >> On Tue, Oct 4, 2016 at 11:14 A

Re: welcoming Xiao Li as a committer

Congrats, Xiao! Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0x9DCC1DBD7FC7BBB2 On Wed, Oct 5, 2016 at 2:36 PM, Fred Reiss wrote: > Congratulations, Xiao! > > Fred > > > On Tuesday, October 4, 2016, Joseph Bradley wrote:

Re: StructuredStreaming Custom Sinks (motivated by Structured Streaming Machine Learning)

Thanks for the thoughtful comments, Michael and Shivaram. From what I’ve seen in this thread and on JIRA, it looks like the current plan with regard to application-facing APIs for sinks is roughly: 1. Rewrite incremental query compilation for Structured Streaming. 2. Redesign Structured Streaming's

Re: welcoming Xiao Li as a committer

Congratulations, Xiao! Cheers, Liwei On Thu, Oct 6, 2016 at 5:38 AM, DB Tsai wrote: > Congrats, Xiao! > > Sincerely, > > DB Tsai > -- > Web: https://www.dbtsai.com > PGP Key ID: 0x9DCC1DBD7FC7BBB2 > > > On Wed, Oct 5, 2016 at 2:36 PM, Fred

Re: [ANNOUNCE] Announcing Spark 2.0.1

It usually don't take that long to be synced, I still don't see any 2.0.1 related artifacts on maven central http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.spark%22%20AND%20v%3A%222.0.1%22 On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > They have been published yesterday, bu

Re: [discuss] separate API annotation into two components: InterfaceAudience & InterfaceStability

I think this is fairly important to do so I went ahead and created a PR for the first mini step: https://github.com/apache/spark/pull/15374 On Wed, Aug 24, 2016 at 9:48 AM, Reynold Xin wrote: > Looks like I'm general people like it. Next step is for somebody to take > the lead and implement it

Re: [ANNOUNCE] Announcing Spark 2.0.1

Yeah I see the apache maven repos have the 2.0.1 artifacts at https://repository.apache.org/content/repositories/releases/org/apache/spark/spark-core_2.11/ -- Not sure why they haven't synced to maven central yet Shivaram On Wed, Oct 5, 2016 at 8:37 PM, Luciano Resende wrote: > It usually don't