Andrew, I put up 1.1.1 branch and I am getting shuffle failures while doing flatMap followed by groupBy...My cluster memory is less than the memory I need and therefore flatMap does around 400 GB of shuffle...memory is around 120 GB...
14/11/13 23:10:49 WARN TaskSetManager: Lost task 22.1 in stage 191.0 (TID 4084, istgbd020.hadoop.istg.verizon.com): FetchFailed(null, shuffleId=4, mapId=-1, reduceId=22) I searched on user-list and this issue has been found over there: http://apache-spark-user-list.1001560.n3.nabble.com/Issues-with-partitionBy-FetchFailed-td14760.html I wanted to make sure whether 1.1.1 does not have the same bug...-1 from me till we figure out the root cause... Thanks. Deb On Mon, Nov 17, 2014 at 10:33 AM, Andrew Or <and...@databricks.com> wrote: > This seems like a legitimate blocker. We will cut another RC to include the > revert. > > 2014-11-16 17:29 GMT-08:00 Kousuke Saruta <saru...@oss.nttdata.co.jp>: > > > Now I've finished to revert for SPARK-4434 and opened PR. > > > > > > (2014/11/16 17:08), Josh Rosen wrote: > > > >> -1 > >> > >> I found a potential regression in 1.1.1 related to spark-submit and > >> cluster > >> deploy mode: https://issues.apache.org/jira/browse/SPARK-4434 > >> > >> I think that this is worth fixing. > >> > >> On Fri, Nov 14, 2014 at 7:28 PM, Cheng Lian <lian.cs....@gmail.com> > >> wrote: > >> > >> +1 > >>> > >>> Tested HiveThriftServer2 against Hive 0.12.0 on Mac OS X. Known issues > >>> are > >>> fixed. Hive version inspection works as expected. > >>> > >>> > >>> On 11/15/14 8:25 AM, Zach Fry wrote: > >>> > >>> +0 > >>>> > >>>> I expect to start testing on Monday but won't have enough results to > >>>> change > >>>> my vote from +0 > >>>> until Monday night or Tuesday morning. > >>>> > >>>> Thanks, > >>>> Zach > >>>> > >>>> > >>>> > >>>> -- > >>>> View this message in context: http://apache-spark- > >>>> developers-list.1001551.n3.nabble.com/VOTE-Release- > >>>> Apache-Spark-1-1-1-RC1-tp9311p9370.html > >>>> Sent from the Apache Spark Developers List mailing list archive at > >>>> Nabble.com. > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > >>>> For additional commands, e-mail: dev-h...@spark.apache.org > >>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > >>> For additional commands, e-mail: dev-h...@spark.apache.org > >>> > >>> > >>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > For additional commands, e-mail: dev-h...@spark.apache.org > > > > >
