[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257218#comment-14257218
]
Brock Noland commented on HIVE-9161:
Hi [~spena],
This looks good to me! I did note one thing related the comments. I believe
[~mohitsabharwal] was suggesting we put the doc comment in the places where we
change a {{HashMap}} to {{LinkedHashMap}} and {{HashSet}} to {{LinkedHashSet}}
as the {{Linked}} versions are the ones which give us deterministic order. Thus
I think we can remove the statements in {{StandardMapObjctInspector}} and add
the comments to the other classes.
Thank you!
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch, HIVE-9161.3.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[ https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257580#comment-14257580 ] Brock Noland commented on HIVE-9161: +1 pending tests Fix ordering differences on UDF functions due to Java8 -- Key: HIVE-9161 URL: https://issues.apache.org/jira/browse/HIVE-9161 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.1 Reporter: Sergio Peña Assignee: Sergio Peña Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch, HIVE-9161.3.patch, HIVE-9161.4.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See Java8 vs Java7) This part is related to UDF functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14258010#comment-14258010
]
Hive QA commented on HIVE-9161:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688925/HIVE-9161.4.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6722 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2182/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2182/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2182/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688925 - PreCommit-HIVE-TRUNK-Build
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch, HIVE-9161.3.patch,
HIVE-9161.4.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256643#comment-14256643
]
Hive QA commented on HIVE-9161:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688680/HIVE-9161.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6719 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_lvj_mapjoin
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2166/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2166/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2166/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688680 - PreCommit-HIVE-TRUNK-Build
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch, HIVE-9161.3.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[ https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254383#comment-14254383 ] Mohit Sabharwal commented on HIVE-9161: --- Another request, [~spena]. Could you add a comment like // Must be deterministic order map for consistent q-test output across Java versions - see HIVE-9161 ? This is so that these changes are not inadvertently reverted. Thank you! Fix ordering differences on UDF functions due to Java8 -- Key: HIVE-9161 URL: https://issues.apache.org/jira/browse/HIVE-9161 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.1 Reporter: Sergio Peña Assignee: Sergio Peña Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See Java8 vs Java7) This part is related to UDF functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[ https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252159#comment-14252159 ] Szehon Ho commented on HIVE-9161: - Looks good to me, +1 pending tests Fix ordering differences on UDF functions due to Java8 -- Key: HIVE-9161 URL: https://issues.apache.org/jira/browse/HIVE-9161 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.1 Reporter: Sergio Peña Assignee: Sergio Peña Attachments: HIVE-9161.1.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See Java8 vs Java7) This part is related to UDF functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252165#comment-14252165
]
Hive QA commented on HIVE-9161:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688078/HIVE-9161.1.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6714 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_varchar_udf1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2129/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2129/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2129/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688078 - PreCommit-HIVE-TRUNK-Build
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252268#comment-14252268
]
Mohit Sabharwal commented on HIVE-9161:
---
Hi [~spena], [~brocknoland] and I were discussing this patch offline. Since
{{StandardMapObjectInspector}} map can get arbitrarily large and is extensively
used for serializing/deserializing results back to the end user for queries
with maps, it might be better to avoid adding the additional memory cost
associated with {{LinkedHashMap}}. To provide an indirect way of fixing such
test failures, we added Java version specific q-test support in HIVE-9109.
(That jira also addresses {{varchar_udf1.q}} test failure.)
As such, I think we can remove {{StandardMapObjectInspector}} related changes
from this patch. What do you think ?
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[ https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252389#comment-14252389 ] Sergio Peña commented on HIVE-9161: --- Hi [~mohitsabharwal] It is good for me. I'll remove the changes on the StandardMapObjctInspector, and update new changes. Thanks. Fix ordering differences on UDF functions due to Java8 -- Key: HIVE-9161 URL: https://issues.apache.org/jira/browse/HIVE-9161 Project: Hive Issue Type: Sub-task Affects Versions: 0.13.1 Reporter: Sergio Peña Assignee: Sergio Peña Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch Java 8 uses a different hash function for HashMap, which is leading to iteration order differences in several cases. (See Java8 vs Java7) This part is related to UDF functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9161) Fix ordering differences on UDF functions due to Java8
[
https://issues.apache.org/jira/browse/HIVE-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252594#comment-14252594
]
Hive QA commented on HIVE-9161:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688119/HIVE-9161.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6716 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2132/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2132/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2132/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688119 - PreCommit-HIVE-TRUNK-Build
Fix ordering differences on UDF functions due to Java8
--
Key: HIVE-9161
URL: https://issues.apache.org/jira/browse/HIVE-9161
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña
Attachments: HIVE-9161.1.patch, HIVE-9161.2.patch
Java 8 uses a different hash function for HashMap, which is leading to
iteration order differences in several cases. (See Java8 vs Java7)
This part is related to UDF functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
