Re: [VOTE] Spark 2.2.1 (RC2)

2017-12-26 Thread zzc
Hi Felix Cheung: When to pulish the new version 2.2.1 of spark doc to the website, now it's still the version 2.2.0. -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubs

Re: Result obtained before the completion of Stages

2017-12-26 Thread ckhari4u
Hi Reynold, I am running a Spark SQL query. val df = spark.sql("select * from table1 t1 join table2 t2 on t1.col1=t2.col1") df.count() -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscrib

Re: Result obtained before the completion of Stages

2017-12-26 Thread Reynold Xin
What did you run? On Tue, Dec 26, 2017 at 10:21 PM, ckhari4u wrote: > Hi Sean, > > Thanks for the reply. I believe I am not facing the scenarios you > mentioned. > > Timestamp conflict: I see the Spark driver logs on the console (tried with > INFO and DEBUG). In all the scenarios, I see the res

Re: Result obtained before the completion of Stages

2017-12-26 Thread ckhari4u
Hi Sean, Thanks for the reply. I believe I am not facing the scenarios you mentioned. Timestamp conflict: I see the Spark driver logs on the console (tried with INFO and DEBUG). In all the scenarios, I see the result getting printed and the application execution continues for 4 more minutes. ie

Re: Result obtained before the completion of Stages

2017-12-26 Thread Sean Owen
My guess is that either they haven't actually finished before the result and something about timestamps you're comparing is misleading, or else, you're looking at stages executing that are part of a later part of the program. On Tue, Dec 26, 2017 at 3:49 PM ckhari4u wrote: > I found this interes

Result obtained before the completion of Stages

2017-12-26 Thread ckhari4u
I found this interesting behavior while running some adhoc analysis query. I have a Spark SQL query where I am joining 2 tables and then performing a count operation. In the Spark Web UI, I see there are 4 Stages getting launched. The interesting behavior I see here is that I see the result befor

[JDBC-WRITER] Support for Updates

2017-12-26 Thread Diogo Munaro Vieira
Hey guys, Nowadays JDBC writer only support inserts. Are there plans for update support on jdbc writer? -- Diogo Munaro Vieira