[ 
https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276743#comment-15276743
 ] 

Davies Liu commented on SPARK-14946:
------------------------------------

[~raymond.honderd...@sizmek.com] It seems that the second job (scan the bigger 
table) did not get started, could you try to disable the broadcast join by set 
spark.sql.autoBroadcastJoinThreshold to 0?

> Spark 2.0 vs 1.6.1 Query Time(out)
> ----------------------------------
>
>                 Key: SPARK-14946
>                 URL: https://issues.apache.org/jira/browse/SPARK-14946
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.0.0
>            Reporter: Raymond Honderdors
>            Priority: Critical
>         Attachments: Query Plan 1.6.1.png, screenshot-spark_2.0.png, 
> spark-defaults.conf, spark-env.sh, version 1.6.1 screen 1 - thrift collect = 
> true.png, version 1.6.1 screen 1 thrift collect = false.png, version 1.6.1 
> screen 2 thrift collect =false.png, version 2.0 -screen 1 thrift collect = 
> false.png, version 2.0 screen 2 thrift collect = true.png, versiuon 2.0 
> screen 1 thrift collect = true.png
>
>
> I run a query using JDBC driver running it on version 1.6.1 it return after 5 
> – 6 min , the same query against version 2.0 fails after 2h (due to timeout) 
> for details on how to reproduce (also see comments below)
> here is what I tried
> I run the following query: select * from pe_servingdata sd inner join 
> pe_campaigns_gzip c on sd.campaignid = c.campaign_id ;
> (with and without a counter and group by on campaigne_id)
> I run spark 1.6.1 and Thriftserver
> then running the sql from beeline or squirrel, after a few min I get answer 
> (0 row) it is correct due to the fact my data did not have matching campaign 
> ids in both tables
> when I run spark 2.0 and Thriftserver, I once again run the sql statement and 
> after 2:30 min it gives up, bit already after 30/60 sec I stop seeing 
> activity on the spark ui
> (sorry for the delay in competing the description of the bug, I was on and 
> off work due to national holidays)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to