Re: Two major versions?

2016-11-27 Thread Dongjoon Hyun
Thank you, Reynold. Clear enough as a backporting and as a new release criteria. Yes. Definitely. If there is no new commit on `branch-1.6`, there is no need to release. However, Apache Spark 1.6.3 was a single release *accumulating* those kind of 52 patches. Given that `branch-1.5` has over

Re: Two major versions?

2016-11-27 Thread Reynold Xin
I think this highly depends on what issues are found, e.g. critical bugs that impact wide use cases, or security bugs. On Sun, Nov 27, 2016 at 12:49 PM, Dongjoon Hyun wrote: > Hi, All. > > Do we have a release plan of Apache Spark 1.6.4? > > Up to my knowledge, Apache

Two major versions?

2016-11-27 Thread Dongjoon Hyun
Hi, All. Do we have a release plan of Apache Spark 1.6.4? Up to my knowledge, Apache Spark community has been focusing on latest two versions. There was no official release of Apache Spark *X.X.4* so far. It's also well-documented on Apache Spark home page (Versioning policy;

Typo in the programming guide?

2016-11-27 Thread Anton Okolnychyi
Hi guys, I am looking at the Accumulator section in the latest programming guide. Is there a typo in the sample code? Shouldn't the add() method accept only one param in Spark 2.0? It looks like the signature is inherited from AccumulatorParam, which was there before. object VectorAccumulatorV2

Use of BroadcastFactory interface (after SPARK-12588 Remove HTTPBroadcast)

2016-11-27 Thread Jacek Laskowski
Hi, After SPARK-12588 Remove HTTPBroadcast [1], the one and only implementation of BroadcastFactory is TorrentBroadcastFactory. No code in Spark 2 uses BroadcastFactory (but TorrentBroadcastFactory) however the scaladoc says [2]: /** * An interface for all the broadcast implementations in Spark