[jira] [Commented] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame
[
https://issues.apache.org/jira/browse/SPARK-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568554#comment-14568554
]
Apache Spark commented on SPARK-8023:
-
User 'rxin' has created a pull request for this issue:
https://github.com/apache/spark/pull/6573
> Random Number Generation inconsistent in projections in DataFrame
> -
>
> Key: SPARK-8023
> URL: https://issues.apache.org/jira/browse/SPARK-8023
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.4.0
>Reporter: Burak Yavuz
>Assignee: Yin Huai
>Priority: Blocker
>
> to reproduce (in python):
> {code}
> df = sqlContext.range(0, 10).withColumn('uniform', rand(seed=10))
> df.select('uniform', 'uniform' + 1)
> {code}
> You should see that the first column + 1 doesn't equal the second column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame
[
https://issues.apache.org/jira/browse/SPARK-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568471#comment-14568471
]
Apache Spark commented on SPARK-8023:
-
User 'yhuai' has created a pull request for this issue:
https://github.com/apache/spark/pull/6570
> Random Number Generation inconsistent in projections in DataFrame
> -
>
> Key: SPARK-8023
> URL: https://issues.apache.org/jira/browse/SPARK-8023
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Reporter: Burak Yavuz
>Assignee: Yin Huai
>Priority: Blocker
>
> to reproduce (in python):
> {code}
> df = sqlContext.range(0, 10).withColumn('uniform', rand(seed=10))
> df.select('uniform', 'uniform' + 1)
> {code}
> You should see that the first column + 1 doesn't equal the second column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame
[
https://issues.apache.org/jira/browse/SPARK-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568470#comment-14568470
]
Yin Huai commented on SPARK-8023:
-
To fix this issue, I am adding a {{def deterministic: Boolean}} to
{{Expression}}.
> Random Number Generation inconsistent in projections in DataFrame
> -
>
> Key: SPARK-8023
> URL: https://issues.apache.org/jira/browse/SPARK-8023
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Reporter: Burak Yavuz
>Assignee: Yin Huai
>Priority: Blocker
>
> to reproduce (in python):
> {code}
> df = sqlContext.range(0, 10).withColumn('uniform', rand(seed=10))
> df.select('uniform', 'uniform' + 1)
> {code}
> You should see that the first column + 1 doesn't equal the second column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8023) Random Number Generation inconsistent in projections in DataFrame
[
https://issues.apache.org/jira/browse/SPARK-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568265#comment-14568265
]
Burak Yavuz commented on SPARK-8023:
cc [~yhuai]
> Random Number Generation inconsistent in projections in DataFrame
> -
>
> Key: SPARK-8023
> URL: https://issues.apache.org/jira/browse/SPARK-8023
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Reporter: Burak Yavuz
>Priority: Blocker
>
> to reproduce (in python):
> {code}
> df = sqlContext.range(0, 10).withColumn('uniform', rand(seed=10))
> df.select('uniform', 'uniform' + 1)
> {code}
> You should see that the first column + 1 doesn't equal the second column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
