[jira] [Commented] (HIVE-5199) Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3
[ https://issues.apache.org/jira/browse/HIVE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758269#comment-13758269 ] Brock Noland commented on HIVE-5199: Hari, Thanks for the contribution! On small suggestion, in this JIRA project we discuss the Apache Hive project, not a distribution, therefore it's useful to discuss a regression in 0.9., 0.10, or 0.11 which caused this. > Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3 > -- > > Key: HIVE-5199 > URL: https://issues.apache.org/jira/browse/HIVE-5199 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan >Priority: Critical > Attachments: HIVE-5199.patch.txt > > > Custom serdes which used to work in HDP 1.1 is no longer working with HDP 1.3. > The issue happens when the partition serde is not of settable type in HDP > 1.3. The below exception happens via FetchOperator as well as MapOperator. > Inside FetchOperator consider the following call: > getRecordReader()->ObjectInspectorConverters. getConverter() > The output object inspector is of settable type(because it is generated via > ObjectInspectorConverters.getConvertedOI()) where as the input object > inspector that gets passed as serde.getObjectorInspector() and is > non-settable. Inside getConverter(), the (inputOI.equals(outputOI)) check > fails and the switch statement tries to cast the non-settable object > inspector to a settable object inspector. > The stack trace as follows: > 2013-08-28 17:57:25,307 ERROR CliDriver (SessionState.java:printError(432)) - > Failed with exception java.io.IOException:java.lang.ClassCastException: > com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be > cast to > org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector > java.io.IOException: java.lang.ClassCastException: > com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be > cast to > org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector > at > org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544) > at > org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488) > at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) > at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412) > at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.util.RunJar.main(RunJar.java:160) > Caused by: java.lang.ClassCastException: > com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be > cast to > org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.(ObjectInspectorConverters.java:307) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138) > at > org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:406) > at > org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5199) Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3
[ https://issues.apache.org/jira/browse/HIVE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758412#comment-13758412 ] Hive QA commented on HIVE-5199: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12601447/HIVE-5199.patch.txt {color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 2909 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_vs_table_metadata org.apache.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_serde_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fileformat_mix org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_neg_float org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partInit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15 org.apache.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/615/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/615/console Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 24 tests failed {noformat} This message is automatically generated. > Read Only Custom SerDe works with HDP 1.1 but not with HDP 1.3 > -- > > Key: HIVE-5199 > URL: https://issues.apache.org/jira/browse/HIVE-5199 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan >Priority: Critical > Attachments: HIVE-5199.patch.txt > > > Custom serdes which used to work in HDP 1.1 is no longer working with HDP 1.3. > The issue happens when the partition serde is not of settable type in HDP > 1.3. The below exception happens via FetchOperator as well as MapOperator. > Inside FetchOperator consider the following call: > getRecordReader()->ObjectInspectorConverters. getConverter() > The output object inspector is of settable type(because it is generated via > ObjectInspectorConverters.getConvertedOI()) where as the input object > inspector that gets passed as serde.getObjectorInspector() and is > non-settable. Inside getConverter(), the (inputOI.equals(outputOI)) check > fails and the switch statement tries to cast the non-settable object > inspector to a settable object inspector. > The stack trace as follows: > 2013-08-28 17:57:25,307 ERROR CliDriver (SessionState.java:printError(432)) - > Failed with exception java.io.IOException:java.lang.ClassCastException: > com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be > cast to > org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector > java.io.IOException: java.lang.ClassCastException: > com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be > cast to > org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector > at > org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544) > at > org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488) > at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136) > at org.apache.hadoop.hive.ql.Driver.getResult