[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907841#comment-14907841
]
Pete Robbins commented on SPARK-6873:
-
I no longer see these errors in my 1.5 branch Java 8 build. Did someone fix
them, remove the tests or is it just chance?
> Some Hive-Catalyst comparison tests fail due to unimportant order of some
> printed elements
> --
>
> Key: SPARK-6873
> URL: https://issues.apache.org/jira/browse/SPARK-6873
> Project: Spark
> Issue Type: Bug
> Components: SQL, Tests
>Affects Versions: 1.3.1
>Reporter: Sean Owen
>Assignee: Cheng Lian
>Priority: Minor
>
> As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
> and actually it still affects master. I think it's a superficial problem that
> only turns up when running on Java 8, but still, would probably be an easy
> fix and good to fix.
> Specifically, here are four tests and the bit that fails the comparison,
> below. I tried to diagnose this but had trouble even finding where some of
> this occurs, like the list of synonyms?
> {code}
> - show_tblproperties *** FAILED ***
> Results do not match for show_tblproperties:
> ...
> !== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
> !tmptruebar bar value
> !barbar value tmp true (HiveComparisonTest.scala:391)
> {code}
> {code}
> - show_create_table_serde *** FAILED ***
> Results do not match for show_create_table_serde:
> ...
>WITH SERDEPROPERTIES ( WITH
> SERDEPROPERTIES (
> ! 'serialization.format'='$',
> 'field.delim'=',',
> ! 'field.delim'=',')
> 'serialization.format'='$')
> {code}
> {code}
> - udf_std *** FAILED ***
> Results do not match for udf_std:
> ...
> !== HIVE - 2 row(s) == == CATALYST
> - 2 row(s) ==
>std(x) - Returns the standard deviation of a set of numbers std(x) -
> Returns the standard deviation of a set of numbers
> !Synonyms: stddev_pop, stddev Synonyms:
> stddev, stddev_pop (HiveComparisonTest.scala:391)
> {code}
> {code}
> - udf_stddev *** FAILED ***
> Results do not match for udf_stddev:
> ...
> !== HIVE - 2 row(s) ====
> CATALYST - 2 row(s) ==
>stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
> - Returns the standard deviation of a set of numbers
> !Synonyms: stddev_pop, stdSynonyms:
> std, stddev_pop (HiveComparisonTest.scala:391)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650268#comment-14650268
]
Cheng Lian commented on SPARK-6873:
---
It's not important. Internally, Hive just traverses a hash map and dumps
everything in it. So the order is decided by the implementation of the hash map.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650480#comment-14650480
]
Sean Owen commented on SPARK-6873:
--
[~rxin] [~lian cheng] It's still a problem. Yes I'm sure it's just a test
issue, not a problem with the code, but ideally the test must not rely on the
ordering. Right now tests don't actually pass in Java 8 because of things like
...
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
== Parsed Logical Plan ==
HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
== Analyzed Logical Plan ==
result: string
HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
== Optimized Logical Plan ==
HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
== Physical Plan ==
ExecutedCommand (HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1)
Code Generation: true
== RDD ==
result
!== HIVE - 13 row(s) == == CATALYST
- 13 row(s) ==
CREATE EXTERNAL TABLE `tmp_showcrt1`( CREATE
EXTERNAL TABLE `tmp_showcrt1`(
`key` string, `key`
string,
`value` boolean)`value`
boolean)
ROW FORMAT SERDEROW FORMAT
SERDE
'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'
'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'
STORED BY STORED BY
'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler'
'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler'
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
LOCATIONLOCATION
'PATHtmp_showcrt1'
'PATHtmp_showcrt1'
TBLPROPERTIES (
TBLPROPERTIES ( (HiveComparisonTest.scala:397)
{code}
I build with {{-Pyarn -Phive}} from master.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647322#comment-14647322
]
Pete Robbins commented on SPARK-6873:
-
We've been trying to get a clean build/test using Java 8 and we see these
errors so I think this is still a problem.
It looks like the Catalyst output changes from Java 7 to Java 8. Is the
ordering supposed to be defined for this or is the ordering really unimportant?
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633002#comment-14633002
]
Reynold Xin commented on SPARK-6873:
[~srowen] Sean - is this still a problem? If not, can you close the ticket?
Thanks.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526019#comment-14526019
]
Sean Owen commented on SPARK-6873:
--
Sorry for the late reply, missed this. Yes, HashMap ordering did change in Java
8. Hm, so it sounds like the test can't pass on both Java 7 and 8 at the same
time then, since the golden output would fail to match one or the other?
or are you saying there's a way to ignore this part of the output?
Otherwise we can track this for a later version like Spark 2.x where we may
require Java 8.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526218#comment-14526218
]
Cheng Lian commented on SPARK-6873:
---
The UDF synonyms lines are easy to be filtered out. Table properties lines can
be fixed by sorting the results. The SerDe lines can be problematic, maybe we
can ignore this case for now. If we plan to add Java 8 Jenkins builder, I can
try to find some time to work on this after 1.4 release. Otherwise it's not
that harmful for now. The time consuming part is regenerating all golden files
since we need to change golden file generation and comparison strategies.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Assignee: Cheng Lian
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494351#comment-14494351
]
Cheng Lian commented on SPARK-6873:
---
I had once hit this recently, but unfortunately forgot to create a JIRA for
this. The reason is that, when printing function synonymns and table/serde
properties, Hive code uses a hash map to store strings to be printed. My guess
is that Java 8 uses a different hash map implementation and thus causes
different iteration order.
To fix this, we can ignore corresponding output lines and regenerate all golden
answer files.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[
https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491567#comment-14491567
]
Sean Owen commented on SPARK-6873:
--
CC [~lian cheng] [~marmbrus] as I bet this would be fairly easy to diagnose for
someone close to the query planner / catalyst bits.
Some Hive-Catalyst comparison tests fail due to unimportant order of some
printed elements
--
Key: SPARK-6873
URL: https://issues.apache.org/jira/browse/SPARK-6873
Project: Spark
Issue Type: Bug
Components: SQL, Tests
Affects Versions: 1.3.1
Reporter: Sean Owen
Priority: Minor
As I mentioned, I've been seeing 4 test failures in Hive tests for a while,
and actually it still affects master. I think it's a superficial problem that
only turns up when running on Java 8, but still, would probably be an easy
fix and good to fix.
Specifically, here are four tests and the bit that fails the comparison,
below. I tried to diagnose this but had trouble even finding where some of
this occurs, like the list of synonyms?
{code}
- show_tblproperties *** FAILED ***
Results do not match for show_tblproperties:
...
!== HIVE - 2 row(s) == == CATALYST - 2 row(s) ==
!tmptruebar bar value
!barbar value tmp true (HiveComparisonTest.scala:391)
{code}
{code}
- show_create_table_serde *** FAILED ***
Results do not match for show_create_table_serde:
...
WITH SERDEPROPERTIES ( WITH
SERDEPROPERTIES (
! 'serialization.format'='$',
'field.delim'=',',
! 'field.delim'=',')
'serialization.format'='$')
{code}
{code}
- udf_std *** FAILED ***
Results do not match for udf_std:
...
!== HIVE - 2 row(s) == == CATALYST
- 2 row(s) ==
std(x) - Returns the standard deviation of a set of numbers std(x) -
Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stddev Synonyms:
stddev, stddev_pop (HiveComparisonTest.scala:391)
{code}
{code}
- udf_stddev *** FAILED ***
Results do not match for udf_stddev:
...
!== HIVE - 2 row(s) ====
CATALYST - 2 row(s) ==
stddev(x) - Returns the standard deviation of a set of numbers stddev(x)
- Returns the standard deviation of a set of numbers
!Synonyms: stddev_pop, stdSynonyms:
std, stddev_pop (HiveComparisonTest.scala:391)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
