[
https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045024#comment-15045024
]
Tao Li commented on SPARK-12179:
--------------------------------
For example:
1. First Time Run, I got:
20151204 main google google 1234
20151204 main facebook facebook 12345
20151204 main github github 123456
2. Second Time Run, I got:
20151204 main google google 1234
20151204 main facebook facebook 22345
20151204 main twitter twitter 12345
You can see:
1. "google" is same
2. "facebook" is different
3. the first run has "github" but no "twitter", the second run has "twitter"
but no "github"
> Spark SQL get different result with the same code
> -------------------------------------------------
>
> Key: SPARK-12179
> URL: https://issues.apache.org/jira/browse/SPARK-12179
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, SQL
> Affects Versions: 1.3.0, 1.3.1, 1.3.2, 1.4.0, 1.4.1, 1.4.2, 1.5.0, 1.5.1,
> 1.5.2, 1.5.3
> Environment: hadoop version: 2.5.0-cdh5.3.2
> spark version: 1.5.3
> run mode: yarn-client
> Reporter: Tao Li
> Priority: Minor
>
> I run the sql in yarn-client mode, but get different result each time.
> As you can see the example, I get the different shuffle write with the same
> shuffle read in two jobs with the same code.
> Some of my spark app runs well, but some always met this problem. And I met
> this problem on spark 1.3, 1.4 and 1.5 version.
> Can you give me some suggestions about the possible causes or how do I figure
> out the problem?
> 1. First Run
> Details for Stage 9 (Attempt 0)
> Total Time Across All Tasks: 5.8 min
> Shuffle Read: 24.4 MB / 205399
> Shuffle Write: 6.8 MB / 54934
> 2. Second Run
> Details for Stage 9 (Attempt 0)
> Total Time Across All Tasks: 5.6 min
> Shuffle Read: 24.4 MB / 205399
> Shuffle Write: 6.8 MB / 54905
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
