SPARK-23221 fixes an issue specific to KafkaContinuousSourceStressForDontFailOnDataLossSuite; I don't think it could cause other suites to deadlock.
Do note that the previous hang issues we saw caused by SPARK-23055 were correctly marked as failures. On Thu, Jan 25, 2018 at 3:40 PM, Shixiong(Ryan) Zhu <shixi...@databricks.com > wrote: > + Jose > > On Thu, Jan 25, 2018 at 2:18 PM, Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >> SPARK-23221 is one of the reasons for Kafka-test-suite deadlock issue. >> >> For the hang issues, it seems not to be marked as a failure correctly in >> Apache Spark Jenkins history. >> >> >> On Thu, Jan 25, 2018 at 1:03 PM, Marcelo Vanzin <van...@cloudera.com> >> wrote: >> >>> On Thu, Jan 25, 2018 at 12:29 PM, Sean Owen <so...@cloudera.com> wrote: >>> > I am still seeing these tests fail or hang: >>> > >>> > - subscribing topic by name from earliest offsets (failOnDataLoss: >>> false) >>> > - subscribing topic by name from earliest offsets (failOnDataLoss: >>> true) >>> >>> This is something that we are seeing internally on a different version >>> Spark, and we're currently investigating with our Kafka people. Not >>> sure it's the same issue (we have a newer version of Kafka libraries), >>> but this is just another way of saying that I don't think those hangs >>> are new in 2.3, at least. >>> >>> -- >>> Marcelo >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>> >>> >> >
