[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253826#comment-15253826 ] Apache Spark commented on SPARK-14848: -- User 'robbinspg' has created a pull request for this issue: https://github.com/apache/spark/pull/12610 > DatasetSuite - Java encoder fails on Big Endian platforms > - > > Key: SPARK-14848 > URL: https://issues.apache.org/jira/browse/SPARK-14848 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.0.0 >Reporter: Pete Robbins > > Since this PR https://github.com/apache/spark/pull/10703 for > https://issues.apache.org/jira/browse/SPARK-12756 the "Java encoder" test in > DatasetSuite has been failing on big endian platforms: > - Java encoder *** FAILED *** > Array((JavaData(2),1), (JavaData(1),1)) did not equal List((JavaData(1),1), > (JavaData(2),1)) (DatasetSuite.scala:478) > I note that the code for the "Kryo encoder" test was changed in the PR to use > toSet and compare results against a Set to stop it failing in the same way > whereas the Java encoder test still uses toSeq. > Is it that the order is not guaranteed (but happens to be in the expected > order on little endian) and this is a test issue? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253821#comment-15253821 ] Wenchen Fan commented on SPARK-14848: - Yea, according to SQL specific, the result order of aggregation is undefined. Feel free to send a PR to fix it. > DatasetSuite - Java encoder fails on Big Endian platforms > - > > Key: SPARK-14848 > URL: https://issues.apache.org/jira/browse/SPARK-14848 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.0.0 >Reporter: Pete Robbins > > Since this PR https://github.com/apache/spark/pull/10703 for > https://issues.apache.org/jira/browse/SPARK-12756 the "Java encoder" test in > DatasetSuite has been failing on big endian platforms: > - Java encoder *** FAILED *** > Array((JavaData(2),1), (JavaData(1),1)) did not equal List((JavaData(1),1), > (JavaData(2),1)) (DatasetSuite.scala:478) > I note that the code for the "Kryo encoder" test was changed in the PR to use > toSet and compare results against a Set to stop it failing in the same way > whereas the Java encoder test still uses toSeq. > Is it that the order is not guaranteed (but happens to be in the expected > order on little endian) and this is a test issue? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14848) DatasetSuite - Java encoder fails on Big Endian platforms
[ https://issues.apache.org/jira/browse/SPARK-14848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253816#comment-15253816 ] Pete Robbins commented on SPARK-14848: -- changing the Java encoder test to use toSet and compare against Set(...) makes the test pass on both little endian and big endian platforms. I will submit a PR. [~cloud_fan] can you confirm my thoughts? > DatasetSuite - Java encoder fails on Big Endian platforms > - > > Key: SPARK-14848 > URL: https://issues.apache.org/jira/browse/SPARK-14848 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.0.0 >Reporter: Pete Robbins > > Since this PR https://github.com/apache/spark/pull/10703 for > https://issues.apache.org/jira/browse/SPARK-12756 the "Java encoder" test in > DatasetSuite has been failing on big endian platforms: > - Java encoder *** FAILED *** > Array((JavaData(2),1), (JavaData(1),1)) did not equal List((JavaData(1),1), > (JavaData(2),1)) (DatasetSuite.scala:478) > I note that the code for the "Kyro encoder" test was changed in the PR to use > toSet and compare results against a Set to stop it failing in the same way > whereas the Java encoder test still uses toSeq. > Is it that the order is not guaranteed (but happens to be in the expected > order on little endian) and this is a test issue? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
