[jira] [Updated] (HIVE-9350) Add ability for HiveAuthorizer implementations to filter out results of 'show tables', 'show databases'
[ https://issues.apache.org/jira/browse/HIVE-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-9350: - Labels: TODOC1.2 (was: ) > Add ability for HiveAuthorizer implementations to filter out results of 'show > tables', 'show databases' > --- > > Key: HIVE-9350 > URL: https://issues.apache.org/jira/browse/HIVE-9350 > Project: Hive > Issue Type: Bug > Components: Authorization >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Labels: TODOC1.2 > Fix For: 1.2.0 > > Attachments: HIVE-9350.1.patch, HIVE-9350.2.patch, HIVE-9350.3.patch, > HIVE-9350.4.patch > > > It should be possible for HiveAuthorizer implementations to control if a user > is able to see a table or database in results of 'show tables' and 'show > databases' respectively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-9556 started by Alexander Pivovarov. - > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = (12 -1) / 12 = > 0.917f > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Description: algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance {code} --one edit operation, greatest str len = 12 str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 {code} was: algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance {code} --one edit operation, greatest str len = 12 str_sim_levenshtein('Test String1', 'Test String2') = (12 -1) / 12 = 0.917f {code} > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9350) Add ability for HiveAuthorizer implementations to filter out results of 'show tables', 'show databases'
[ https://issues.apache.org/jira/browse/HIVE-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310610#comment-14310610 ] Lefty Leverenz commented on HIVE-9350: -- Doc note: This changes the description of *hive.metastore.filter.hook* which was created by HIVE-8612 in 1.1.0 (aka 0.15.0) and documented in the Metastore Administration wikidoc. So the wiki needs to be updated for 1.2.0. * [Hive Metastore Administration -- Additional Configuration Parameters | https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-AdditionalConfigurationParameters] Does this need any other documentation? > Add ability for HiveAuthorizer implementations to filter out results of 'show > tables', 'show databases' > --- > > Key: HIVE-9350 > URL: https://issues.apache.org/jira/browse/HIVE-9350 > Project: Hive > Issue Type: Bug > Components: Authorization >Reporter: Thejas M Nair >Assignee: Thejas M Nair > Labels: TODOC1.2 > Fix For: 1.2.0 > > Attachments: HIVE-9350.1.patch, HIVE-9350.2.patch, HIVE-9350.3.patch, > HIVE-9350.4.patch > > > It should be possible for HiveAuthorizer implementations to control if a user > is able to see a table or database in results of 'show tables' and 'show > databases' respectively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Status: Patch Available (was: In Progress) > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Attachment: HIVE-9556.1.patch patch #1 > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Status: In Progress (was: Patch Available) > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Attachment: HIVE-9556.2.patch patch #2 - fixed checkArgsLength method > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch, HIVE-9556.2.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9556: -- Status: Patch Available (was: In Progress) > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch, HIVE-9556.2.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9569) Enable more unit tests for UNION ALL [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310623#comment-14310623 ] Hive QA commented on HIVE-9569: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697067/HIVE-9569.1.patch {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7497 tests executed *Failed tests:* {noformat} TestSSL - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union17 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union31 {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2695/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2695/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2695/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697067 - PreCommit-HIVE-TRUNK-Build > Enable more unit tests for UNION ALL [Spark Branch] > --- > > Key: HIVE-9569 > URL: https://issues.apache.org/jira/browse/HIVE-9569 > Project: Hive > Issue Type: Sub-task > Components: Spark >Affects Versions: spark-branch >Reporter: Chao >Assignee: Chao > Attachments: HIVE-9569.1-spark.patch, HIVE-9569.1.patch > > > Currently, we only enabled a subset of all the union tests. We should try to > enable the rest, and see if there's any issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9560) When hive.stats.collect.rawdatasize=true, 'rawDataSize' for an ORC table will result in value '0' after running 'analyze table TABLE_NAME compute statistics;'
[ https://issues.apache.org/jira/browse/HIVE-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-9560: Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) Committed to trunk. > When hive.stats.collect.rawdatasize=true, 'rawDataSize' for an ORC table will > result in value '0' after running 'analyze table TABLE_NAME compute > statistics;' > -- > > Key: HIVE-9560 > URL: https://issues.apache.org/jira/browse/HIVE-9560 > Project: Hive > Issue Type: Bug >Reporter: Xin Hao >Assignee: Prasanth Jayachandran > Fix For: 1.2.0 > > Attachments: HIVE-9560.1.patch, HIVE-9560.2.patch > > > When hive.stats.collect.rawdatasize=true, 'rawDataSize' for an ORC table will > result in value '0' after running 'analyze table TABLE_NAME compute > statistics;' > Reproduce step: > (1) set hive.stats.collect.rawdatasize=true; > (2) Generate an ORC table in hive, and the value of its 'rawDataSize' is NOT > zero. > You can find the value of 'rawDataSize' (NOT zero) by executing 'describe > extended TABLE_NAME;' > (4) Execute 'analyze table TABLE_NAME compute statistics;' > (5) Execute 'describe extended TABLE_NAME;' again, and you will find that > the value of 'rawDataSize' will be changed to '0'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310631#comment-14310631 ] nicolas maillard commented on HIVE-9556: would it be a good time to add also the Damerau–Levenshtein_distance and jaro distance? > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch, HIVE-9556.2.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9188) BloomFilter support in ORC
[ https://issues.apache.org/jira/browse/HIVE-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-9188: Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks [~owen.omalley], [~hagleitn] and [~gopalv] for reviews and feedback! > BloomFilter support in ORC > -- > > Key: HIVE-9188 > URL: https://issues.apache.org/jira/browse/HIVE-9188 > Project: Hive > Issue Type: New Feature > Components: File Formats >Affects Versions: 0.15.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Labels: orcfile > Fix For: 1.2.0 > > Attachments: HIVE-9188.1.patch, HIVE-9188.10.patch, > HIVE-9188.11.patch, HIVE-9188.2.patch, HIVE-9188.3.patch, HIVE-9188.4.patch, > HIVE-9188.5.patch, HIVE-9188.6.patch, HIVE-9188.7.patch, HIVE-9188.8.patch, > HIVE-9188.9.patch > > > BloomFilters are well known probabilistic data structure for set membership > checking. We can use bloom filters in ORC index for better row group pruning. > Currently, ORC row group index uses min/max statistics to eliminate row > groups (stripes as well) that do not satisfy predicate condition specified in > the query. But in some cases, the efficiency of min/max based elimination is > not optimal (unsorted columns with wide range of entries). Bloom filters can > be an effective and efficient alternative for row group/split elimination for > point queries or queries with IN clause. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9517) UNION ALL query failed with ArrayIndexOutOfBoundsException [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310673#comment-14310673 ] Hive QA commented on HIVE-9517: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697075/HIVE-9517.2.patch {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 7483 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore org.apache.hive.hcatalog.streaming.TestStreaming.testMultipleTransactionBatchCommits org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Delimited {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2696/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2696/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2696/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697075 - PreCommit-HIVE-TRUNK-Build > UNION ALL query failed with ArrayIndexOutOfBoundsException [Spark Branch] > - > > Key: HIVE-9517 > URL: https://issues.apache.org/jira/browse/HIVE-9517 > Project: Hive > Issue Type: Sub-task >Affects Versions: spark-branch >Reporter: Chao >Assignee: Chao > Attachments: HIVE-9517.1.patch, HIVE-9517.2.patch > > > I was running a query from cbo_gby_empty.q: > {code} > select unionsrc.key, unionsrc.value FROM (select 'max' as key, max(c_int) as > value from cbo_t3 s1 > UNION ALL > select 'min' as key, min(c_int) as value from cbo_t3 s2 > UNION ALL > select 'avg' as key, avg(c_int) as value from cbo_t3 s3) unionsrc > order by unionsrc.key; > {code} > and got the following exception: > {noformat} > 2015-01-29 15:57:55,948 ERROR [Executor task launch worker-1]: > spark.SparkReduceRecordHandler > (SparkReduceRecordHandler.java:processRow(299)) - Fatal error: > org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row > (tag=0) {"key":{"reducesinkkey0":"max"},"value":{"_col0":1.5}} > org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row > (tag=0) {"key":{"reducesinkkey0":"max"},"value":{"_col0":1.5}} > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processKeyValues(SparkReduceRecordHandler.java:339) > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:289) > at > org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:49) > at > org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:28) > at > org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:98) > at > scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41) > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > at > org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$2.apply(AsyncRDDActions.scala:115) > at > org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$2.apply(AsyncRDDActions.scala:115) > at org.apache.spark.SparkContext$$anonfun$30.apply(SparkContext.scala:1390) > at org.apache.spark.SparkContext$$anonfun$30.apply(SparkContext.scala:1390) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61) > at org.apache.spark.scheduler.Task.run(Task.scala:56) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating > VALUE._col0 > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82) > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processKeyValues(SparkReduceRecordHandler.java:330) > ... 17 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 3 > at > org.apache.hadoop.hive.serde
[jira] [Commented] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310694#comment-14310694 ] Hive QA commented on HIVE-9596: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697094/HIVE-9596.2.patch {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_instr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_locate org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2697/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2697/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2697/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697094 - PreCommit-HIVE-TRUNK-Build > move standard getDisplayString impl to GenericUDF > - > > Key: HIVE-9596 > URL: https://issues.apache.org/jira/browse/HIVE-9596 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9596.1.patch, HIVE-9596.2.patch > > > 54 GenericUDF derived classes have very similar getDisplayString impl which > returns "fname(child1, child2, childn)" > instr() and locate() have bugs in their implementation (no comma btw child) > Instead of having 54 implementations of the same method it's better to move > standard implementation to the base class. > affected UDF classes: > {code} > contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java > ql/src/java/org/apache/ha
[jira] [Updated] (HIVE-9500) Support nested structs over 24 levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-9500: --- Attachment: (was: HIVE-9500.3.patch) > Support nested structs over 24 levels. > -- > > Key: HIVE-9500 > URL: https://issues.apache.org/jira/browse/HIVE-9500 > Project: Hive > Issue Type: Improvement >Reporter: Aihua Xu >Assignee: Aihua Xu > Labels: SerDe > Fix For: 1.2.0 > > Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch > > > Customer has deeply nested avro structure and is receiving the following > error when performing queries. > 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException > org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting > supported for LazySimpleSerde is 23 Unable to work with level 24 > Currently we support up to 24 levels of nested structs when > hive.serialization.extend.nesting.levels is set to true, while the customers > have the requirement to support more than that. > It would be better to make the supported levels configurable or completely > removed (i.e., we can support any number of levels). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9500) Support nested structs over 24 levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-9500: --- Status: In Progress (was: Patch Available) > Support nested structs over 24 levels. > -- > > Key: HIVE-9500 > URL: https://issues.apache.org/jira/browse/HIVE-9500 > Project: Hive > Issue Type: Improvement >Reporter: Aihua Xu >Assignee: Aihua Xu > Labels: SerDe > Fix For: 1.2.0 > > Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch > > > Customer has deeply nested avro structure and is receiving the following > error when performing queries. > 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException > org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting > supported for LazySimpleSerde is 23 Unable to work with level 24 > Currently we support up to 24 levels of nested structs when > hive.serialization.extend.nesting.levels is set to true, while the customers > have the requirement to support more than that. > It would be better to make the supported levels configurable or completely > removed (i.e., we can support any number of levels). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9500) Support nested structs over 24 levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-9500: --- Attachment: HIVE-9500.3.patch The machine ran out of space. Rerun the patch. Unfortunately even without *.q.out, still can't create the review. > Support nested structs over 24 levels. > -- > > Key: HIVE-9500 > URL: https://issues.apache.org/jira/browse/HIVE-9500 > Project: Hive > Issue Type: Improvement >Reporter: Aihua Xu >Assignee: Aihua Xu > Labels: SerDe > Fix For: 1.2.0 > > Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch, HIVE-9500.3.patch > > > Customer has deeply nested avro structure and is receiving the following > error when performing queries. > 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException > org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting > supported for LazySimpleSerde is 23 Unable to work with level 24 > Currently we support up to 24 levels of nested structs when > hive.serialization.extend.nesting.levels is set to true, while the customers > have the requirement to support more than that. > It would be better to make the supported levels configurable or completely > removed (i.e., we can support any number of levels). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9500) Support nested structs over 24 levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-9500: --- Status: Patch Available (was: In Progress) > Support nested structs over 24 levels. > -- > > Key: HIVE-9500 > URL: https://issues.apache.org/jira/browse/HIVE-9500 > Project: Hive > Issue Type: Improvement >Reporter: Aihua Xu >Assignee: Aihua Xu > Labels: SerDe > Fix For: 1.2.0 > > Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch, HIVE-9500.3.patch > > > Customer has deeply nested avro structure and is receiving the following > error when performing queries. > 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException > org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting > supported for LazySimpleSerde is 23 Unable to work with level 24 > Currently we support up to 24 levels of nested structs when > hive.serialization.extend.nesting.levels is set to true, while the customers > have the requirement to support more than that. > It would be better to make the supported levels configurable or completely > removed (i.e., we can support any number of levels). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8471) Cleanup TestSSL
[ https://issues.apache.org/jira/browse/HIVE-8471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310798#comment-14310798 ] Hive QA commented on HIVE-8471: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697186/HIVE-8471.patch {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 7524 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23 org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler org.apache.hive.jdbc.TestSSL.testSSLConnectionWithProperty org.apache.hive.jdbc.TestSSL.testSSLConnectionWithURL org.apache.hive.jdbc.TestSSL.testSSLFetch org.apache.hive.jdbc.TestSSL.testSSLFetchHttp {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2698/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2698/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2698/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697186 - PreCommit-HIVE-TRUNK-Build > Cleanup TestSSL > --- > > Key: HIVE-8471 > URL: https://issues.apache.org/jira/browse/HIVE-8471 > Project: Hive > Issue Type: Improvement >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Attachments: HIVE-8471.patch, HIVE-8471.patch > > > conf is static and there are some other minor cleanups -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9607) Remove unnecessary attach-jdbc-driver execution from package/pom.xml
[ https://issues.apache.org/jira/browse/HIVE-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310841#comment-14310841 ] Hive QA commented on HIVE-9607: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697215/HIVE-9607.1.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} org.apache.hive.jdbc.TestSSL.testSSLFetchHttp {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2699/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2699/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2699/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697215 - PreCommit-HIVE-TRUNK-Build > Remove unnecessary attach-jdbc-driver execution from package/pom.xml > > > Key: HIVE-9607 > URL: https://issues.apache.org/jira/browse/HIVE-9607 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9607.1.patch > > > Looks like build-helper-maven-plugin block which has execution > attach-jdbc-driver is not needed in package/pom.xml > package/pom.xml has maven-dependency-plugin which copies hive-jdbc-standalone > to project.build.directory > I removed build-helper-maven-plugin block and rebuilt hive > hive-jdbc-standalone.jar is still placed to project.build.directory > {code} > $ mvn clean install -Phadoop-2 -Pdist -DskipTests > $ find . -name "apache-hive*jdbc.jar" -exec ls -la {} \; > 16844023 Feb 6 17:45 ./packaging/target/apache-hive-1.2.0-SNAPSHOT-jdbc.jar > $ find . -name "hive-jdbc*standalone.jar" -exec ls -la {} \; > 16844023 Feb 6 17:45 > ./packaging/target/apache-hive-1.2.0-SNAPSHOT-bin/apache-hive-1.2.0-SNAPSHOT-bin/lib/hive-jdbc-1.2.0-SNAPSHOT-standalone.jar > 16844023 Feb 6 17:45 ./jdbc/target/hive-jdbc-1.2.0-SNAPSHOT-standalone.jar > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9481) allow column list specification in INSERT statement
[ https://issues.apache.org/jira/browse/HIVE-9481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310889#comment-14310889 ] Hive QA commented on HIVE-9481: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697214/HIVE-9481.2.patch {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 7520 tests executed *Failed tests:* {noformat} TestSparkClient - did not produce a TEST-*.xml file org.apache.hive.jdbc.TestSSL.testSSLFetchHttp {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2700/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2700/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2700/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697214 - PreCommit-HIVE-TRUNK-Build > allow column list specification in INSERT statement > --- > > Key: HIVE-9481 > URL: https://issues.apache.org/jira/browse/HIVE-9481 > Project: Hive > Issue Type: Bug > Components: Parser, Query Processor, SQL >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-9481.2.patch, HIVE-9481.patch > > > Given a table FOO(a int, b int, c int), ANSI SQL supports insert into > FOO(c,b) select x,y from T. The expectation is that 'x' is written to column > 'c' and 'y' is written column 'b' and 'a' is set to NULL, assuming column 'a' > is NULLABLE. > Hive does not support this. In Hive one has to ensure that the data > producing statement has a schema that matches target table schema. > Since Hive doesn't support DEFAULT value for columns in CREATE TABLE, when > target schema is explicitly provided, missing columns will be set to NULL if > they are NULLABLE, otherwise an error will be raised. > If/when DEFAULT clause is supported, this can be enhanced to set default > value rather than NULL. > Thus, given {noformat} > create table source (a int, b int); > create table target (x int, y int, z int); > create table target2 (x int, y int, z int); > {noformat} > {noformat}insert into target(y,z) select * from source;{noformat} > will mean > {noformat}insert into target select null as x, a, b from source;{noformat} > and > {noformat}insert into target(z,y) select * from source;{noformat} > will meant > {noformat}insert into target select null as x, b, a from source;{noformat} > Also, > {noformat} > from source insert into target(y,z) select null as x, * insert into > target2(y,z) select null as x, source.*; > {noformat} > and for partitioned tables, given > {noformat} > Given: > CREATE TABLE pageviews (userid VARCHAR(64), link STRING, "from" STRING) > PARTITIONED BY (datestamp STRING) CLUSTERED BY (userid) INTO 256 BUCKETS > STORED AS ORC; > INSERT INTO TABLE pageviews PARTITION (datestamp = '2014-09-23')(userid,link) > VALUES ('jsmith', 'mail.com'); > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9608) Define SPARK_HOME if not defined automagically
Brock Noland created HIVE-9608: -- Summary: Define SPARK_HOME if not defined automagically Key: HIVE-9608 URL: https://issues.apache.org/jira/browse/HIVE-9608 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Priority: Minor many hadoop installs are in {{dir/\{spark,hive,hadoop,..\}}}. We can infer {{SPARK_HOME}} in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9608) Define SPARK_HOME if not defined automagically
[ https://issues.apache.org/jira/browse/HIVE-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9608: --- Status: Patch Available (was: Open) > Define SPARK_HOME if not defined automagically > -- > > Key: HIVE-9608 > URL: https://issues.apache.org/jira/browse/HIVE-9608 > Project: Hive > Issue Type: Improvement >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Attachments: HIVE-9608.patch > > > many hadoop installs are in {{dir/\{spark,hive,hadoop,..\}}}. We can infer > {{SPARK_HOME}} in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9608) Define SPARK_HOME if not defined automagically
[ https://issues.apache.org/jira/browse/HIVE-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9608: --- Attachment: HIVE-9608.patch > Define SPARK_HOME if not defined automagically > -- > > Key: HIVE-9608 > URL: https://issues.apache.org/jira/browse/HIVE-9608 > Project: Hive > Issue Type: Improvement >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Attachments: HIVE-9608.patch > > > many hadoop installs are in {{dir/\{spark,hive,hadoop,..\}}}. We can infer > {{SPARK_HOME}} in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9517) UNION ALL query failed with ArrayIndexOutOfBoundsException [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-9517: -- Resolution: Fixed Fix Version/s: spark-branch Status: Resolved (was: Patch Available) Committed to Spark branch. Thanks, Chao. > UNION ALL query failed with ArrayIndexOutOfBoundsException [Spark Branch] > - > > Key: HIVE-9517 > URL: https://issues.apache.org/jira/browse/HIVE-9517 > Project: Hive > Issue Type: Sub-task >Affects Versions: spark-branch >Reporter: Chao >Assignee: Chao > Fix For: spark-branch > > Attachments: HIVE-9517.1.patch, HIVE-9517.2.patch > > > I was running a query from cbo_gby_empty.q: > {code} > select unionsrc.key, unionsrc.value FROM (select 'max' as key, max(c_int) as > value from cbo_t3 s1 > UNION ALL > select 'min' as key, min(c_int) as value from cbo_t3 s2 > UNION ALL > select 'avg' as key, avg(c_int) as value from cbo_t3 s3) unionsrc > order by unionsrc.key; > {code} > and got the following exception: > {noformat} > 2015-01-29 15:57:55,948 ERROR [Executor task launch worker-1]: > spark.SparkReduceRecordHandler > (SparkReduceRecordHandler.java:processRow(299)) - Fatal error: > org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row > (tag=0) {"key":{"reducesinkkey0":"max"},"value":{"_col0":1.5}} > org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row > (tag=0) {"key":{"reducesinkkey0":"max"},"value":{"_col0":1.5}} > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processKeyValues(SparkReduceRecordHandler.java:339) > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:289) > at > org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:49) > at > org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:28) > at > org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:98) > at > scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41) > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > at > org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$2.apply(AsyncRDDActions.scala:115) > at > org.apache.spark.rdd.AsyncRDDActions$$anonfun$foreachAsync$2.apply(AsyncRDDActions.scala:115) > at org.apache.spark.SparkContext$$anonfun$30.apply(SparkContext.scala:1390) > at org.apache.spark.SparkContext$$anonfun$30.apply(SparkContext.scala:1390) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61) > at org.apache.spark.scheduler.Task.run(Task.scala:56) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating > VALUE._col0 > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82) > at > org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processKeyValues(SparkReduceRecordHandler.java:330) > ... 17 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 3 > at > org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.byteArrayToLong(LazyBinaryUtils.java:84) > at > org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryDouble.init(LazyBinaryDouble.java:43) > at > org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264) > at > org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201) > at > org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64) > at > org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:98) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77) > at > org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:77) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9596: -- Status: Patch Available (was: In Progress) > move standard getDisplayString impl to GenericUDF > - > > Key: HIVE-9596 > URL: https://issues.apache.org/jira/browse/HIVE-9596 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9596.1.patch, HIVE-9596.2.patch, HIVE-9596.3.patch > > > 54 GenericUDF derived classes have very similar getDisplayString impl which > returns "fname(child1, child2, childn)" > instr() and locate() have bugs in their implementation (no comma btw child) > Instead of having 54 implementations of the same method it's better to move > standard implementation to the base class. > affected UDF classes: > {code} > contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInFile.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInitCap.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLeadLag.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLocate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapKeys.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapValues.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFNamedStruct.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSentences.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSize.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSplit.java > ql/src/java/org/apac
[jira] [Updated] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9596: -- Status: In Progress (was: Patch Available) > move standard getDisplayString impl to GenericUDF > - > > Key: HIVE-9596 > URL: https://issues.apache.org/jira/browse/HIVE-9596 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9596.1.patch, HIVE-9596.2.patch, HIVE-9596.3.patch > > > 54 GenericUDF derived classes have very similar getDisplayString impl which > returns "fname(child1, child2, childn)" > instr() and locate() have bugs in their implementation (no comma btw child) > Instead of having 54 implementations of the same method it's better to move > standard implementation to the base class. > affected UDF classes: > {code} > contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInFile.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInitCap.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLeadLag.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLocate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapKeys.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapValues.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFNamedStruct.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSentences.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSize.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSplit.java > ql/src/java/org/apac
[jira] [Updated] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9596: -- Attachment: HIVE-9596.3.patch patch #3 - fixed udf_instr.q.out and udf_locate.q.out > move standard getDisplayString impl to GenericUDF > - > > Key: HIVE-9596 > URL: https://issues.apache.org/jira/browse/HIVE-9596 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9596.1.patch, HIVE-9596.2.patch, HIVE-9596.3.patch > > > 54 GenericUDF derived classes have very similar getDisplayString impl which > returns "fname(child1, child2, childn)" > instr() and locate() have bugs in their implementation (no comma btw child) > Instead of having 54 implementations of the same method it's better to move > standard implementation to the base class. > affected UDF classes: > {code} > contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInFile.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInitCap.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLeadLag.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLocate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapKeys.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapValues.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFNamedStruct.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPower.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSentences.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSize.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/Ge
[jira] [Updated] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Pivovarov updated HIVE-9596: -- Description: 54 GenericUDF derived classes have very similar getDisplayString impl which returns "fname(child1, child2, childn)" instr() and locate() have bugs in their implementation (no comma btw children) Instead of having 54 implementations of the same method it's better to move standard implementation to the base class. affected UDF classes: {code} contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInFile.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInitCap.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLastDay.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLeadLag.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLocate.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLower.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapKeys.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMapValues.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFNamedStruct.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPower.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPrintf.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSentences.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSize.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSplit.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFStringToMap.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFStruct.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTranslate.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFUpper.java {code} was: 54 GenericUDF derived classes have very similar getDisplayString impl which returns "fname(child1, child2, childn)" instr() and locate() have bugs in their implementation (no comma btw child) Instead of having 54 implementations of the same method it's better to move standard implementation to the base class. affected UDF classes: {code} cont
[jira] [Updated] (HIVE-3454) Problem with CAST(BIGINT as TIMESTAMP)
[ https://issues.apache.org/jira/browse/HIVE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-3454: --- Status: In Progress (was: Patch Available) > Problem with CAST(BIGINT as TIMESTAMP) > -- > > Key: HIVE-3454 > URL: https://issues.apache.org/jira/browse/HIVE-3454 > Project: Hive > Issue Type: Bug > Components: Types, UDF >Affects Versions: 0.13.1, 0.13.0, 0.12.0, 0.11.0, 0.10.0, 0.9.0, 0.8.1, > 0.8.0 >Reporter: Ryan Harris >Assignee: Aihua Xu > Labels: newbie, newdev, patch > Attachments: HIVE-3454.1.patch.txt, HIVE-3454.patch > > > Ran into an issue while working with timestamp conversion. > CAST(unix_timestamp() as TIMESTAMP) should create a timestamp for the current > time from the BIGINT returned by unix_timestamp() > Instead, however, a 1970-01-16 timestamp is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-3454) Problem with CAST(BIGINT as TIMESTAMP)
[ https://issues.apache.org/jira/browse/HIVE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-3454: --- Attachment: HIVE-3454.2.patch Update the primitive types including boolean, byte, short ,int, long to be consistently represent the time in seconds when we convert them into timestamp. > Problem with CAST(BIGINT as TIMESTAMP) > -- > > Key: HIVE-3454 > URL: https://issues.apache.org/jira/browse/HIVE-3454 > Project: Hive > Issue Type: Bug > Components: Types, UDF >Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.13.0, > 0.13.1 >Reporter: Ryan Harris >Assignee: Aihua Xu > Labels: newbie, newdev, patch > Attachments: HIVE-3454.1.patch.txt, HIVE-3454.2.patch, HIVE-3454.patch > > > Ran into an issue while working with timestamp conversion. > CAST(unix_timestamp() as TIMESTAMP) should create a timestamp for the current > time from the BIGINT returned by unix_timestamp() > Instead, however, a 1970-01-16 timestamp is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-3454) Problem with CAST(BIGINT as TIMESTAMP)
[ https://issues.apache.org/jira/browse/HIVE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-3454: --- Status: Patch Available (was: In Progress) > Problem with CAST(BIGINT as TIMESTAMP) > -- > > Key: HIVE-3454 > URL: https://issues.apache.org/jira/browse/HIVE-3454 > Project: Hive > Issue Type: Bug > Components: Types, UDF >Affects Versions: 0.13.1, 0.13.0, 0.12.0, 0.11.0, 0.10.0, 0.9.0, 0.8.1, > 0.8.0 >Reporter: Ryan Harris >Assignee: Aihua Xu > Labels: newbie, newdev, patch > Attachments: HIVE-3454.1.patch.txt, HIVE-3454.2.patch, HIVE-3454.patch > > > Ran into an issue while working with timestamp conversion. > CAST(unix_timestamp() as TIMESTAMP) should create a timestamp for the current > time from the BIGINT returned by unix_timestamp() > Instead, however, a 1970-01-16 timestamp is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9481) allow column list specification in INSERT statement
[ https://issues.apache.org/jira/browse/HIVE-9481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310930#comment-14310930 ] Eugene Koifman commented on HIVE-9481: -- the error for the failed test is not related to the patch {noformat} java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:53698/default;ssl=true;sslTrustStore=%2Fhome%2Fhiveptest%2F54.144.54.75-hiveptest-2%2Fapache-svn-trunk-source%2Fitests%2Fhive-unit%2F..%2F..%2F%2Fdata%2Ffiles%2Ftruststore.jks;trustStorePassword=HiveJdbc?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice;: Could not create http connection to jdbc:hive2://localhost:53698/default;ssl=true;sslTrustStore=%2Fhome%2Fhiveptest%2F54.144.54.75-hiveptest-2%2Fapache-svn-trunk-source%2Fitests%2Fhive-unit%2F..%2F..%2F%2Fdata%2Ffiles%2Ftruststore.jks;trustStorePassword=HiveJdbc?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice;. org.apache.http.conn.HttpHostConnectException: Connection to https://localhost:53698 refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:414) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827) at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251) at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65) at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:150) at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:142) at org.apache.hive.jdbc.HiveConnection.createHttpTransport(HiveConnection.java:249) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187) at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:163) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.hive.jdbc.TestSSL.testSSLFetchHttp(TestSSL.java:360) {noformat} > allow column list specification in INSERT statement > --- > > Key: HIVE-9481 > URL: https://issues.apache.org/jira/browse/HIVE-9481 > Project: Hive > Issue Type: Bug > Components: Parser, Query Processor, SQL >Affects Versions: 0.14.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-9481.2.patch, HIVE-9481.patch > > > Given a table FOO(a int, b int, c int), ANSI SQL supports insert into > FOO(c,b) select x,y from T. The expectation is that 'x' is written to column > 'c' and 'y' is written column 'b' and 'a' is set to NULL, assuming column 'a' > is NULLABLE. > Hive does not support this. In Hive one has to ensure that the data > producing statement has a schema that matches target table schema. > Since Hive doesn't support DEFAULT value for columns in CREATE TABLE, when > target schema is explicitly provided, missing columns will be set to NULL if > they are NULLABLE, otherwise an error will be raised. > If/when DEFAULT clause is supported, this can be enhanced to set default > value rather than NULL. > Thus, given {noformat} > create table source (a int, b int); > create table target (x int, y int, z int); > create table target2 (x int, y int, z int); > {noformat} > {noformat}insert into target(y,z) select * from source;{noformat} > will mean > {noformat}insert into target select null as x, a, b from source;{noformat} > and > {noformat}insert into target(z,y) select * from source;{no
[jira] [Commented] (HIVE-9556) create UDF to measure strings similarity using Levenshtein Distance algo
[ https://issues.apache.org/jira/browse/HIVE-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310934#comment-14310934 ] Hive QA commented on HIVE-9556: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697234/HIVE-9556.2.patch {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 7530 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_stats_counter org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore org.apache.hive.spark.client.TestSparkClient.testSyncRpc {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2701/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2701/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2701/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697234 - PreCommit-HIVE-TRUNK-Build > create UDF to measure strings similarity using Levenshtein Distance algo > > > Key: HIVE-9556 > URL: https://issues.apache.org/jira/browse/HIVE-9556 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Attachments: HIVE-9556.1.patch, HIVE-9556.2.patch > > > algorithm description http://en.wikipedia.org/wiki/Levenshtein_distance > {code} > --one edit operation, greatest str len = 12 > str_sim_levenshtein('Test String1', 'Test String2') = 1 - 1 / 12 = 0.9167 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9500) Support nested structs over 24 levels.
[ https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310987#comment-14310987 ] Hive QA commented on HIVE-9500: --- {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697258/HIVE-9500.3.patch {color:green}SUCCESS:{color} +1 7527 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2702/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2702/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2702/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12697258 - PreCommit-HIVE-TRUNK-Build > Support nested structs over 24 levels. > -- > > Key: HIVE-9500 > URL: https://issues.apache.org/jira/browse/HIVE-9500 > Project: Hive > Issue Type: Improvement >Reporter: Aihua Xu >Assignee: Aihua Xu > Labels: SerDe > Fix For: 1.2.0 > > Attachments: HIVE-9500.1.patch, HIVE-9500.2.patch, HIVE-9500.3.patch > > > Customer has deeply nested avro structure and is receiving the following > error when performing queries. > 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException > org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting > supported for LazySimpleSerde is 23 Unable to work with level 24 > Currently we support up to 24 levels of nested structs when > hive.serialization.extend.nesting.levels is set to true, while the customers > have the requirement to support more than that. > It would be better to make the supported levels configurable or completely > removed (i.e., we can support any number of levels). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4639) Add has null flag to ORC internal index
[ https://issues.apache.org/jira/browse/HIVE-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311024#comment-14311024 ] Lefty Leverenz commented on HIVE-4639: -- Doc note: [~prasanth_j] documented this in the ORC wiki. * [ORC -- Column Statistics | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC#LanguageManualORC-ColumnStatistics] But it says the hasNull flag is added in 1.2.0 -- shouldn't that be 1.1.0, since this jira's fix version is 0.15? > Add has null flag to ORC internal index > --- > > Key: HIVE-4639 > URL: https://issues.apache.org/jira/browse/HIVE-4639 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Owen O'Malley >Assignee: Prasanth Jayachandran > Fix For: 0.15.0 > > Attachments: HIVE-4639.1.patch, HIVE-4639.2.patch, HIVE-4639.3.patch > > > It would enable more predicate pushdown if we added a flag to the index entry > recording if there were any null values in the column for the 10k rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-4639) Add has null flag to ORC internal index
[ https://issues.apache.org/jira/browse/HIVE-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311026#comment-14311026 ] Prasanth Jayachandran commented on HIVE-4639: - Good catch! [~leftylev]. Updated the docs! > Add has null flag to ORC internal index > --- > > Key: HIVE-4639 > URL: https://issues.apache.org/jira/browse/HIVE-4639 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Owen O'Malley >Assignee: Prasanth Jayachandran > Fix For: 0.15.0 > > Attachments: HIVE-4639.1.patch, HIVE-4639.2.patch, HIVE-4639.3.patch > > > It would enable more predicate pushdown if we added a flag to the index entry > recording if there were any null values in the column for the 10k rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9608) Define SPARK_HOME if not defined automagically
[ https://issues.apache.org/jira/browse/HIVE-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311045#comment-14311045 ] Hive QA commented on HIVE-9608: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697284/HIVE-9608.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2703/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2703/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2703/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: NonZeroExitCodeException Command 'cd /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2703/ && tar -zvcf test-results.tar.gz test-results/' failed with exit status 141 and output 'test-results/ test-results/TEST-TestMiniSparkOnYarnCliDriver-TEST-org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.xml test-results/TEST-TestStorageBasedMetastoreAuthorizationProvider-TEST-org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.xml test-results/TEST-TestGenericUDFLastDay-TEST-org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFLastDay.xml test-results/TEST-TestMetaStoreInitListener-TEST-org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.xml test-results/TEST-TestCliDriver-udf_min.q-union12.q-udf6.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml test-results/TEST-TestHiveAuthorizerShowFilters-TEST-org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.xml test-results/TEST-TestMetaStoreListenersError-TEST-org.apache.hadoop.hive.metastore.TestMetaStoreListenersError.xml test-results/TEST-TestFolderPermissions-TEST-org.apache.hadoop.hive.ql.security.TestFolderPermissions.xml test-results/TEST-TestCliDriver-authorization_cli_createtab.q-groupby_ppr.q-partition_wise_fileformat12.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml test-results/TEST-TestInputJobInfo-TEST-org.apache.hive.hcatalog.mapreduce.TestInputJobInfo.xml test-results/TEST-TestVectorTimestampExpressions-TEST-org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTimestampExpressions.xml test-results/TEST-TestPerformTestRCFileAndSeqFile-TEST-org.apache.hadoop.hive.ql.io.TestPerformTestRCFileAndSeqFile.xml test-results/TEST-TestHiveDecimalParse-TEST-org.apache.hadoop.hive.ql.parse.TestHiveDecimalParse.xml test-results/TEST-TestCliDriver-lateral_view_noalias.q-cast1.q-bucketmapjoin_negative.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml test-results/TEST-TestLockRequestBuilder-TEST-org.apache.hadoop.hive.metastore.TestLockRequestBuilder.xml test-results/TEST-TestDefaultAccumuloRowIdFactory-TEST-org.apache.hadoop.hive.accumulo.serde.TestDefaultAccumuloRowIdFactory.xml test-results/TEST-TestHiveBinarySearchRecordReader-TEST-org.apache.hadoop.hive.ql.io.TestHiveBinarySearchRecordReader.xml test-results/TEST-TestSparkCliDriver-join_cond_pushdown_unqual4.q-vectorization_16.q-groupby10.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestSparkCliDriver.xml test-results/TEST-TestMiniTezCliDriver-vector_decimal_10_0.q-vector_decimal_trailing.q-lvj_mapjoin.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestMiniTezCliDriver.xml test-results/TEST-TestTempletonUtils-TEST-org.apache.hive.hcatalog.templeton.tool.TestTempletonUtils.xml test-results/TEST-TestCliDriver-enforce_order.q-bucketcontext_4.q-stats_publisher_error_1.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml test-results/TEST-TestGenericAvroRecordWritable-TEST-org.apache.hadoop.hive.serde2.avro.TestGenericAvroRecordWritable.xml test-results/TEST-TestHCatMutableDynamicPartitioned-TEST-org.apache.hive.hcatalog.mapreduce.TestHCatMutableDynamicPartitioned.xml test-results/TEST-TestSetUGIOnOnlyClient-TEST-org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.xml test-results/TEST-TestCliDriver-ppd_transform.q-list_bucket_dml_1.q-groupby_sort_6.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml test-results/TEST-TestDateWritable-TEST-org.apache.hadoop.hive.serde2.io.TestDateWritable.xml test-results/TEST-TestObjectInspectorConverters-TEST-org.apache.hadoop.hive.serde2.objectinspector.TestObjectInspectorConverters.xml test-results/TEST-TestMiniTezCliDriver-dynpart_sort_optimization2.q-tez_bmj_schema_evolution.q-vector_char_mapjoin1.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestMiniTezCliDriver.xml test-results/
[jira] [Commented] (HIVE-9593) ORC Reader should ignore unknown metadata streams
[ https://issues.apache.org/jira/browse/HIVE-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311047#comment-14311047 ] Gopal V commented on HIVE-9593: --- +1 - Confirmed that the change does not affect any encodings on disk. During testing, I merged HIVE-9188 + -Pprotobuf to regenerate generated code - that's probably necessary now that JIRA is in. /cc: [~prasanth_j] > ORC Reader should ignore unknown metadata streams > -- > > Key: HIVE-9593 > URL: https://issues.apache.org/jira/browse/HIVE-9593 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.11.0, 0.12.0, 0.13.1, 1.0.0, 1.2.0, 1.1.0 >Reporter: Gopal V >Assignee: Owen O'Malley > Attachments: hive-9593.patch > > > ORC readers should ignore metadata streams which are non-essential additions > to the main data streams. > This will include additional indices, histograms or anything we add as an > optional stream. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9608) Define SPARK_HOME if not defined automagically
[ https://issues.apache.org/jira/browse/HIVE-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311056#comment-14311056 ] Xuefu Zhang commented on HIVE-9608: --- +1 > Define SPARK_HOME if not defined automagically > -- > > Key: HIVE-9608 > URL: https://issues.apache.org/jira/browse/HIVE-9608 > Project: Hive > Issue Type: Improvement >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Attachments: HIVE-9608.patch > > > many hadoop installs are in {{dir/\{spark,hive,hadoop,..\}}}. We can infer > {{SPARK_HOME}} in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9596) move standard getDisplayString impl to GenericUDF
[ https://issues.apache.org/jira/browse/HIVE-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311066#comment-14311066 ] Hive QA commented on HIVE-9596: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697286/HIVE-9596.3.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2704/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2704/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2704/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697286 - PreCommit-HIVE-TRUNK-Build > move standard getDisplayString impl to GenericUDF > - > > Key: HIVE-9596 > URL: https://issues.apache.org/jira/browse/HIVE-9596 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > Attachments: HIVE-9596.1.patch, HIVE-9596.2.patch, HIVE-9596.3.patch > > > 54 GenericUDF derived classes have very similar getDisplayString impl which > returns "fname(child1, child2, childn)" > instr() and locate() have bugs in their implementation (no comma btw children) > Instead of having 54 implementations of the same method it's better to move > standard implementation to the base class. > affected UDF classes: > {code} > contrib/src/java/org/apache/hadoop/hive/contrib/genericudf/example/GenericUDFDBOutput.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEvaluateNPE.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaBoolean.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestGetJavaString.java > itests/util/src/main/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTestTranslate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapBop.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAbs.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAddMonths.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArray.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrue.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFCoalesce.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcat.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFConcatWS.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateAdd.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateDiff.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDateSub.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFField.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFloorCeilBase.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFormatNumber.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFGreatest.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFHash.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInFile.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInitCap.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInstr.java > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLast
[jira] [Created] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
Sushanth Sowmyan created HIVE-9609: -- Summary: AddPartitionMessage.getPartitions() can return null Key: HIVE-9609 URL: https://issues.apache.org/jira/browse/HIVE-9609 Project: Hive Issue Type: Bug Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan DbNotificationListener and NotificationListener both depend on AddPartitionEvent.getPartitions() to get their partitions to trigger a message, but this can be null if an AddPartitionEvent was initialized on a PartitionSpec rather than a List. Also, AddPartitionEvent seems to have a duality, where getPartitions() works only if instantiated on a List, and getPartitionIterator() works only if instantiated on a PartitionSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
[ https://issues.apache.org/jira/browse/HIVE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311094#comment-14311094 ] Sushanth Sowmyan commented on HIVE-9609: Ideally, I'd like to fix this by changing AddPartitionEvent.getPartitions() to return an Iterable, and make that work either way, but I'm not certain if I will tread on any toes if I change that, since this has been public interface for a while - also, I'm not certain if expecting "null" from getPartitions might be used in any code to determine if this is a List based or PartitionSpec-based AddPartitionEvent. So, I've not messed with the current implementation of getPartitions. That said, [~mithun], could you please comment if you're okay with me fixing getPartitions so that it doesn't return null in the case where it has been instantiated from a PartitionSpec? I could at the very least do that. Also, to handle the base problem, we should fix AddPartitionEvent.getPartitionIterator to correctly work in correctly in both cases - this should at least not be controversial. After that, we should change MessageFactory.buildAddPartitionMessage to work on Iterator rather than List - this is trivially fixable, and have JSONMessageFactory use that instead, thereby solving our initial problem of getPartitions call from AddPartitionEvent not being usable in cases of events fired with PartitionSpec rather than List > AddPartitionMessage.getPartitions() can return null > --- > > Key: HIVE-9609 > URL: https://issues.apache.org/jira/browse/HIVE-9609 > Project: Hive > Issue Type: Bug >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > > DbNotificationListener and NotificationListener both depend on > AddPartitionEvent.getPartitions() to get their partitions to trigger a > message, but this can be null if an AddPartitionEvent was initialized on a > PartitionSpec rather than a List. > Also, AddPartitionEvent seems to have a duality, where getPartitions() works > only if instantiated on a List, and getPartitionIterator() works > only if instantiated on a PartitionSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
[ https://issues.apache.org/jira/browse/HIVE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-9609: --- Status: Patch Available (was: Open) > AddPartitionMessage.getPartitions() can return null > --- > > Key: HIVE-9609 > URL: https://issues.apache.org/jira/browse/HIVE-9609 > Project: Hive > Issue Type: Bug >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-9609.patch > > > DbNotificationListener and NotificationListener both depend on > AddPartitionEvent.getPartitions() to get their partitions to trigger a > message, but this can be null if an AddPartitionEvent was initialized on a > PartitionSpec rather than a List. > Also, AddPartitionEvent seems to have a duality, where getPartitions() works > only if instantiated on a List, and getPartitionIterator() works > only if instantiated on a PartitionSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
[ https://issues.apache.org/jira/browse/HIVE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-9609: --- Attachment: HIVE-9609.patch Patch attached. > AddPartitionMessage.getPartitions() can return null > --- > > Key: HIVE-9609 > URL: https://issues.apache.org/jira/browse/HIVE-9609 > Project: Hive > Issue Type: Bug >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-9609.patch > > > DbNotificationListener and NotificationListener both depend on > AddPartitionEvent.getPartitions() to get their partitions to trigger a > message, but this can be null if an AddPartitionEvent was initialized on a > PartitionSpec rather than a List. > Also, AddPartitionEvent seems to have a duality, where getPartitions() works > only if instantiated on a List, and getPartitionIterator() works > only if instantiated on a PartitionSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-3454) Problem with CAST(BIGINT as TIMESTAMP)
[ https://issues.apache.org/jira/browse/HIVE-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311104#comment-14311104 ] Hive QA commented on HIVE-3454: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697289/HIVE-3454.2.patch {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} TestCustomAuthentication - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_timestamp org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_casts org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_decimal_date {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2705/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2705/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2705/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697289 - PreCommit-HIVE-TRUNK-Build > Problem with CAST(BIGINT as TIMESTAMP) > -- > > Key: HIVE-3454 > URL: https://issues.apache.org/jira/browse/HIVE-3454 > Project: Hive > Issue Type: Bug > Components: Types, UDF >Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.13.0, > 0.13.1 >Reporter: Ryan Harris >Assignee: Aihua Xu > Labels: newbie, newdev, patch > Attachments: HIVE-3454.1.patch.txt, HIVE-3454.2.patch, HIVE-3454.patch > > > Ran into an issue while working with timestamp conversion. > CAST(unix_timestamp() as TIMESTAMP) should create a timestamp for the current > time from the BIGINT returned by unix_timestamp() > Instead, however, a 1970-01-16 timestamp is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9585) AlterPartitionMessage should return getKeyValues instead of getValues
[ https://issues.apache.org/jira/browse/HIVE-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-9585: --- Status: Patch Available (was: Open) > AlterPartitionMessage should return getKeyValues instead of getValues > - > > Key: HIVE-9585 > URL: https://issues.apache.org/jira/browse/HIVE-9585 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-9585.patch > > > HIVE-9175 added in AlterPartitionMessage to use in notification events. > However, on trying to write a ReplicationTask implementation on top of that > event, I see that I need the key-values from the message, and from a context > where I might not have access to a hive client to fetch it myself. > Thus, the AlterPartitionMessage needs to be changed so as to return > getKeyValues as a primary, and we can remove getValues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
[ https://issues.apache.org/jira/browse/HIVE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-9609: --- Affects Version/s: 1.2.0 > AddPartitionMessage.getPartitions() can return null > --- > > Key: HIVE-9609 > URL: https://issues.apache.org/jira/browse/HIVE-9609 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-9609.patch > > > DbNotificationListener and NotificationListener both depend on > AddPartitionEvent.getPartitions() to get their partitions to trigger a > message, but this can be null if an AddPartitionEvent was initialized on a > PartitionSpec rather than a List. > Also, AddPartitionEvent seems to have a duality, where getPartitions() works > only if instantiated on a List, and getPartitionIterator() works > only if instantiated on a PartitionSpec. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
Brock Noland created HIVE-9610: -- Summary: Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes Key: HIVE-9610 URL: https://issues.apache.org/jira/browse/HIVE-9610 Project: Hive Issue Type: Bug Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9610.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9610: --- Attachment: HIVE-9610.patch > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9610: --- Affects Version/s: 1.1.0 Status: Patch Available (was: Open) > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1438#comment-1438 ] Vaibhav Gumashta commented on HIVE-9610: +1. I'll try to look at reorganizing the code a bit so that we can reduce the jar size. > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9610: --- Description: We've not had success only including specific shim classes as part of the standalone jdbc jar. Since all shim classes shouldn't be too large we'll include them all. > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > > We've not had success only including specific shim classes as part of the > standalone jdbc jar. Since all shim classes shouldn't be too large we'll > include them all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1439#comment-1439 ] Alexander Pivovarov commented on HIVE-9610: --- I did some investigation on what packages are not needed in hive-jdbc-standalone https://issues.apache.org/jira/browse/HIVE-9599 > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > > We've not had success only including specific shim classes as part of the > standalone jdbc jar. Since all shim classes shouldn't be too large we'll > include them all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9599) remove derby, datanucleus and other not related to jdbc client classes from hive-jdbc-standalone.jar
[ https://issues.apache.org/jira/browse/HIVE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311123#comment-14311123 ] Vaibhav Gumashta commented on HIVE-9599: [~apivovarov] Did you try on a secure cluster as well? How much was the reduced size? > remove derby, datanucleus and other not related to jdbc client classes from > hive-jdbc-standalone.jar > > > Key: HIVE-9599 > URL: https://issues.apache.org/jira/browse/HIVE-9599 > Project: Hive > Issue Type: Improvement > Components: JDBC >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > > Looks like the following packages (included to hive-jdbc-standalone.jar) are > not used when jdbc client opens jdbc connection and runs queries: > {code} > antlr/ > antlr/actions/cpp/ > antlr/actions/csharp/ > antlr/actions/java/ > antlr/actions/python/ > antlr/ASdebug/ > antlr/build/ > antlr/collections/ > antlr/collections/impl/ > antlr/debug/ > antlr/debug/misc/ > antlr/preprocessor/ > com/google/gson/ > com/google/gson/annotations/ > com/google/gson/internal/ > com/google/gson/internal/bind/ > com/google/gson/reflect/ > com/google/gson/stream/ > com/google/inject/ > com/google/inject/binder/ > com/google/inject/internal/ > com/google/inject/internal/asm/ > com/google/inject/internal/cglib/core/ > com/google/inject/internal/cglib/proxy/ > com/google/inject/internal/cglib/reflect/ > com/google/inject/internal/util/ > com/google/inject/matcher/ > com/google/inject/name/ > com/google/inject/servlet/ > com/google/inject/spi/ > com/google/inject/util/ > com/jamesmurty/utils/ > com/jcraft/jsch/ > com/jcraft/jsch/jce/ > com/jcraft/jsch/jcraft/ > com/jcraft/jsch/jgss/ > com/jolbox/bonecp/ > com/jolbox/bonecp/hooks/ > com/jolbox/bonecp/proxy/ > com/sun/activation/registries/ > com/sun/activation/viewers/ > com/sun/istack/ > com/sun/istack/localization/ > com/sun/istack/logging/ > com/sun/mail/handlers/ > com/sun/mail/iap/ > com/sun/mail/imap/ > com/sun/mail/imap/protocol/ > com/sun/mail/mbox/ > com/sun/mail/pop3/ > com/sun/mail/smtp/ > com/sun/mail/util/ > com/sun/xml/bind/ > com/sun/xml/bind/annotation/ > com/sun/xml/bind/api/ > com/sun/xml/bind/api/impl/ > com/sun/xml/bind/marshaller/ > com/sun/xml/bind/unmarshaller/ > com/sun/xml/bind/util/ > com/sun/xml/bind/v2/ > com/sun/xml/bind/v2/bytecode/ > com/sun/xml/bind/v2/model/annotation/ > com/sun/xml/bind/v2/model/core/ > com/sun/xml/bind/v2/model/impl/ > com/sun/xml/bind/v2/model/nav/ > com/sun/xml/bind/v2/model/runtime/ > com/sun/xml/bind/v2/runtime/ > com/sun/xml/bind/v2/runtime/output/ > com/sun/xml/bind/v2/runtime/property/ > com/sun/xml/bind/v2/runtime/reflect/ > com/sun/xml/bind/v2/runtime/reflect/opt/ > com/sun/xml/bind/v2/runtime/unmarshaller/ > com/sun/xml/bind/v2/schemagen/ > com/sun/xml/bind/v2/schemagen/episode/ > com/sun/xml/bind/v2/schemagen/xmlschema/ > com/sun/xml/bind/v2/util/ > com/sun/xml/txw2/ > com/sun/xml/txw2/annotation/ > com/sun/xml/txw2/output/ > com/thoughtworks/paranamer/ > contribs/mx/ > javax/activation/ > javax/annotation/ > javax/annotation/concurrent/ > javax/annotation/meta/ > javax/annotation/security/ > javax/el/ > javax/inject/ > javax/jdo/ > javax/jdo/annotations/ > javax/jdo/datastore/ > javax/jdo/identity/ > javax/jdo/listener/ > javax/jdo/metadata/ > javax/jdo/spi/ > javax/mail/ > javax/mail/event/ > javax/mail/internet/ > javax/mail/search/ > javax/mail/util/ > javax/security/auth/message/ > javax/security/auth/message/callback/ > javax/security/auth/message/config/ > javax/security/auth/message/module/ > javax/servlet/ > javax/servlet/http/ > javax/servlet/jsp/ > javax/servlet/jsp/el/ > javax/servlet/jsp/tagext/ > javax/transaction/ > javax/transaction/xa/ > javax/xml/bind/ > javax/xml/bind/annotation/ > javax/xml/bind/annotation/adapters/ > javax/xml/bind/attachment/ > javax/xml/bind/helpers/ > javax/xml/bind/util/ > javax/xml/stream/ > javax/xml/stream/events/ > javax/xml/stream/util/ > jline/ > jline/console/ > jline/console/completer/ > jline/console/history/ > jline/console/internal/ > jline/internal/ > net/iharder/base64/ > org/aopalliance/aop/ > org/aopalliance/intercept/ > org/apache/commons/beanutils/ > org/apache/commons/beanutils/converters/ > org/apache/commons/beanutils/expression/ > org/apache/commons/beanutils/locale/ > org/apache/commons/beanutils/locale/converters/ > org/apache/commons/cli/ > org/apache/commons/codec/ > org/apache/commons/codec/binary/ > org/apache/commons/codec/digest/ > org/apache/commons/codec/language/ > org/apache/commons/codec/net/ > org/apache/commons/collections/ > org/apache/commons/collections/bag/ > org/apache/commons/collections/bidimap/ > org/apache/commons/collections/buffer/ > org/apache/commons/collections/collectio
[jira] [Commented] (HIVE-9599) remove derby, datanucleus and other not related to jdbc client classes from hive-jdbc-standalone.jar
[ https://issues.apache.org/jira/browse/HIVE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311126#comment-14311126 ] Alexander Pivovarov commented on HIVE-9599: --- I tried with hive.server2.authentication=LDAP min list of jars required for jdbc client are {code} commons-logging-1.1.3.jar hive-exec-1.2.0-SNAPSHOT.jar hive-service-1.2.0-SNAPSHOT.jar httpcore-4.2.5.jar libthrift-0.9.2.jar hadoop-common-2.6.0.jar hive-jdbc-1.2.0-SNAPSHOT.jar httpclient-4.2.5.jar libfb303-0.9.2.jar slf4j-api-1.7.5.jar {code} > remove derby, datanucleus and other not related to jdbc client classes from > hive-jdbc-standalone.jar > > > Key: HIVE-9599 > URL: https://issues.apache.org/jira/browse/HIVE-9599 > Project: Hive > Issue Type: Improvement > Components: JDBC >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > > Looks like the following packages (included to hive-jdbc-standalone.jar) are > not used when jdbc client opens jdbc connection and runs queries: > {code} > antlr/ > antlr/actions/cpp/ > antlr/actions/csharp/ > antlr/actions/java/ > antlr/actions/python/ > antlr/ASdebug/ > antlr/build/ > antlr/collections/ > antlr/collections/impl/ > antlr/debug/ > antlr/debug/misc/ > antlr/preprocessor/ > com/google/gson/ > com/google/gson/annotations/ > com/google/gson/internal/ > com/google/gson/internal/bind/ > com/google/gson/reflect/ > com/google/gson/stream/ > com/google/inject/ > com/google/inject/binder/ > com/google/inject/internal/ > com/google/inject/internal/asm/ > com/google/inject/internal/cglib/core/ > com/google/inject/internal/cglib/proxy/ > com/google/inject/internal/cglib/reflect/ > com/google/inject/internal/util/ > com/google/inject/matcher/ > com/google/inject/name/ > com/google/inject/servlet/ > com/google/inject/spi/ > com/google/inject/util/ > com/jamesmurty/utils/ > com/jcraft/jsch/ > com/jcraft/jsch/jce/ > com/jcraft/jsch/jcraft/ > com/jcraft/jsch/jgss/ > com/jolbox/bonecp/ > com/jolbox/bonecp/hooks/ > com/jolbox/bonecp/proxy/ > com/sun/activation/registries/ > com/sun/activation/viewers/ > com/sun/istack/ > com/sun/istack/localization/ > com/sun/istack/logging/ > com/sun/mail/handlers/ > com/sun/mail/iap/ > com/sun/mail/imap/ > com/sun/mail/imap/protocol/ > com/sun/mail/mbox/ > com/sun/mail/pop3/ > com/sun/mail/smtp/ > com/sun/mail/util/ > com/sun/xml/bind/ > com/sun/xml/bind/annotation/ > com/sun/xml/bind/api/ > com/sun/xml/bind/api/impl/ > com/sun/xml/bind/marshaller/ > com/sun/xml/bind/unmarshaller/ > com/sun/xml/bind/util/ > com/sun/xml/bind/v2/ > com/sun/xml/bind/v2/bytecode/ > com/sun/xml/bind/v2/model/annotation/ > com/sun/xml/bind/v2/model/core/ > com/sun/xml/bind/v2/model/impl/ > com/sun/xml/bind/v2/model/nav/ > com/sun/xml/bind/v2/model/runtime/ > com/sun/xml/bind/v2/runtime/ > com/sun/xml/bind/v2/runtime/output/ > com/sun/xml/bind/v2/runtime/property/ > com/sun/xml/bind/v2/runtime/reflect/ > com/sun/xml/bind/v2/runtime/reflect/opt/ > com/sun/xml/bind/v2/runtime/unmarshaller/ > com/sun/xml/bind/v2/schemagen/ > com/sun/xml/bind/v2/schemagen/episode/ > com/sun/xml/bind/v2/schemagen/xmlschema/ > com/sun/xml/bind/v2/util/ > com/sun/xml/txw2/ > com/sun/xml/txw2/annotation/ > com/sun/xml/txw2/output/ > com/thoughtworks/paranamer/ > contribs/mx/ > javax/activation/ > javax/annotation/ > javax/annotation/concurrent/ > javax/annotation/meta/ > javax/annotation/security/ > javax/el/ > javax/inject/ > javax/jdo/ > javax/jdo/annotations/ > javax/jdo/datastore/ > javax/jdo/identity/ > javax/jdo/listener/ > javax/jdo/metadata/ > javax/jdo/spi/ > javax/mail/ > javax/mail/event/ > javax/mail/internet/ > javax/mail/search/ > javax/mail/util/ > javax/security/auth/message/ > javax/security/auth/message/callback/ > javax/security/auth/message/config/ > javax/security/auth/message/module/ > javax/servlet/ > javax/servlet/http/ > javax/servlet/jsp/ > javax/servlet/jsp/el/ > javax/servlet/jsp/tagext/ > javax/transaction/ > javax/transaction/xa/ > javax/xml/bind/ > javax/xml/bind/annotation/ > javax/xml/bind/annotation/adapters/ > javax/xml/bind/attachment/ > javax/xml/bind/helpers/ > javax/xml/bind/util/ > javax/xml/stream/ > javax/xml/stream/events/ > javax/xml/stream/util/ > jline/ > jline/console/ > jline/console/completer/ > jline/console/history/ > jline/console/internal/ > jline/internal/ > net/iharder/base64/ > org/aopalliance/aop/ > org/aopalliance/intercept/ > org/apache/commons/beanutils/ > org/apache/commons/beanutils/converters/ > org/apache/commons/beanutils/expression/ > org/apache/commons/beanutils/locale/ > org/apache/commons/beanutils/locale/converters/ > org/apache/commons/cli/ > org/apache/commons/codec/ > org/apache/commons/codec/binary/ > org/apache/commons/codec/digest/
[jira] [Commented] (HIVE-9599) remove derby, datanucleus and other not related to jdbc client classes from hive-jdbc-standalone.jar
[ https://issues.apache.org/jira/browse/HIVE-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311127#comment-14311127 ] Vaibhav Gumashta commented on HIVE-9599: I think with a kerberos setup, there are a few more. > remove derby, datanucleus and other not related to jdbc client classes from > hive-jdbc-standalone.jar > > > Key: HIVE-9599 > URL: https://issues.apache.org/jira/browse/HIVE-9599 > Project: Hive > Issue Type: Improvement > Components: JDBC >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov >Priority: Minor > > Looks like the following packages (included to hive-jdbc-standalone.jar) are > not used when jdbc client opens jdbc connection and runs queries: > {code} > antlr/ > antlr/actions/cpp/ > antlr/actions/csharp/ > antlr/actions/java/ > antlr/actions/python/ > antlr/ASdebug/ > antlr/build/ > antlr/collections/ > antlr/collections/impl/ > antlr/debug/ > antlr/debug/misc/ > antlr/preprocessor/ > com/google/gson/ > com/google/gson/annotations/ > com/google/gson/internal/ > com/google/gson/internal/bind/ > com/google/gson/reflect/ > com/google/gson/stream/ > com/google/inject/ > com/google/inject/binder/ > com/google/inject/internal/ > com/google/inject/internal/asm/ > com/google/inject/internal/cglib/core/ > com/google/inject/internal/cglib/proxy/ > com/google/inject/internal/cglib/reflect/ > com/google/inject/internal/util/ > com/google/inject/matcher/ > com/google/inject/name/ > com/google/inject/servlet/ > com/google/inject/spi/ > com/google/inject/util/ > com/jamesmurty/utils/ > com/jcraft/jsch/ > com/jcraft/jsch/jce/ > com/jcraft/jsch/jcraft/ > com/jcraft/jsch/jgss/ > com/jolbox/bonecp/ > com/jolbox/bonecp/hooks/ > com/jolbox/bonecp/proxy/ > com/sun/activation/registries/ > com/sun/activation/viewers/ > com/sun/istack/ > com/sun/istack/localization/ > com/sun/istack/logging/ > com/sun/mail/handlers/ > com/sun/mail/iap/ > com/sun/mail/imap/ > com/sun/mail/imap/protocol/ > com/sun/mail/mbox/ > com/sun/mail/pop3/ > com/sun/mail/smtp/ > com/sun/mail/util/ > com/sun/xml/bind/ > com/sun/xml/bind/annotation/ > com/sun/xml/bind/api/ > com/sun/xml/bind/api/impl/ > com/sun/xml/bind/marshaller/ > com/sun/xml/bind/unmarshaller/ > com/sun/xml/bind/util/ > com/sun/xml/bind/v2/ > com/sun/xml/bind/v2/bytecode/ > com/sun/xml/bind/v2/model/annotation/ > com/sun/xml/bind/v2/model/core/ > com/sun/xml/bind/v2/model/impl/ > com/sun/xml/bind/v2/model/nav/ > com/sun/xml/bind/v2/model/runtime/ > com/sun/xml/bind/v2/runtime/ > com/sun/xml/bind/v2/runtime/output/ > com/sun/xml/bind/v2/runtime/property/ > com/sun/xml/bind/v2/runtime/reflect/ > com/sun/xml/bind/v2/runtime/reflect/opt/ > com/sun/xml/bind/v2/runtime/unmarshaller/ > com/sun/xml/bind/v2/schemagen/ > com/sun/xml/bind/v2/schemagen/episode/ > com/sun/xml/bind/v2/schemagen/xmlschema/ > com/sun/xml/bind/v2/util/ > com/sun/xml/txw2/ > com/sun/xml/txw2/annotation/ > com/sun/xml/txw2/output/ > com/thoughtworks/paranamer/ > contribs/mx/ > javax/activation/ > javax/annotation/ > javax/annotation/concurrent/ > javax/annotation/meta/ > javax/annotation/security/ > javax/el/ > javax/inject/ > javax/jdo/ > javax/jdo/annotations/ > javax/jdo/datastore/ > javax/jdo/identity/ > javax/jdo/listener/ > javax/jdo/metadata/ > javax/jdo/spi/ > javax/mail/ > javax/mail/event/ > javax/mail/internet/ > javax/mail/search/ > javax/mail/util/ > javax/security/auth/message/ > javax/security/auth/message/callback/ > javax/security/auth/message/config/ > javax/security/auth/message/module/ > javax/servlet/ > javax/servlet/http/ > javax/servlet/jsp/ > javax/servlet/jsp/el/ > javax/servlet/jsp/tagext/ > javax/transaction/ > javax/transaction/xa/ > javax/xml/bind/ > javax/xml/bind/annotation/ > javax/xml/bind/annotation/adapters/ > javax/xml/bind/attachment/ > javax/xml/bind/helpers/ > javax/xml/bind/util/ > javax/xml/stream/ > javax/xml/stream/events/ > javax/xml/stream/util/ > jline/ > jline/console/ > jline/console/completer/ > jline/console/history/ > jline/console/internal/ > jline/internal/ > net/iharder/base64/ > org/aopalliance/aop/ > org/aopalliance/intercept/ > org/apache/commons/beanutils/ > org/apache/commons/beanutils/converters/ > org/apache/commons/beanutils/expression/ > org/apache/commons/beanutils/locale/ > org/apache/commons/beanutils/locale/converters/ > org/apache/commons/cli/ > org/apache/commons/codec/ > org/apache/commons/codec/binary/ > org/apache/commons/codec/digest/ > org/apache/commons/codec/language/ > org/apache/commons/codec/net/ > org/apache/commons/collections/ > org/apache/commons/collections/bag/ > org/apache/commons/collections/bidimap/ > org/apache/commons/collections/buffer/ > org/apache/commons/collections/collection/ > org/apache/commons/collectio
[jira] [Commented] (HIVE-9609) AddPartitionMessage.getPartitions() can return null
[ https://issues.apache.org/jira/browse/HIVE-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311130#comment-14311130 ] Hive QA commented on HIVE-9609: --- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697307/HIVE-9609.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2706/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2706/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2706/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests exited with: ExecutionException: org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult [localFile=/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver, remoteFile=/home/hiveptest/54.196.66.93-hiveptest-0/logs/, getExitCode()=12, getException()=null, getUser()=hiveptest, getHost()=54.196.66.93, getInstance()=0]: 'Address 54.196.66.93 maps to ec2-54-196-66-93.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ TEST-TestHBaseMinimrCliDriver-TEST-org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.xml 0 0%0.00kB/s0:00:00 4839 100%4.61MB/s0:00:00 (xfer#1, to-check=3/5) hive.log 0 0%0.00kB/s0:00:00 44040192 22% 42.00MB/s0:00:03 rsync: write failed on "/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (213 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.196.66.93 maps to ec2-54-196-66-93.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on "/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (213 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.196.66.93 maps to ec2-54-196-66-93.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on "/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (213 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.196.66.93 maps to ec2-54-196-66-93.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on "/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (213 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] Address 54.196.66.93 maps to ec2-54-196-66-93.compute-1.amazonaws.com, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT! receiving incremental file list ./ hive.log 0 0%0.00kB/s0:00:00 rsync: write failed on "/data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2706/succeeded/TestHBaseMinimrCliDriver/hive.log": No space left on device (28) rsync error: error in file IO (code 11) at receiver.c(301) [receiver=3.0.6] rsync: connection unexpectedly closed (213 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] ' {noformat} This message is automatically generated. ATTACHMENT ID: 12697307 - PreCommit-HIVE-TRUNK-Build > AddPartitionMessage.getPartitions() can return null > --- > > Key: HIVE-9609 > URL: https://issues.apache.org/jira/browse/HIVE-9609 > Project: Hive >
[jira] [Commented] (HIVE-9585) AlterPartitionMessage should return getKeyValues instead of getValues
[ https://issues.apache.org/jira/browse/HIVE-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311132#comment-14311132 ] Hive QA commented on HIVE-9585: --- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12696618/HIVE-9585.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2707/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2707/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2707/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]] + export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + export PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2707/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java' Reverted 'hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java' Reverted 'hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/NotificationListener.java' Reverted 'hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/messaging/json/JSONMessageFactory.java' Reverted 'hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/messaging/MessageFactory.java' ++ egrep -v '^X|^Performing status on external' ++ awk '{print $2}' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/scheduler/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/thirdparty itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target itests/hive-jmh/target itests/hive-unit/target itests/custom-serde/target itests/util/target itests/qtest-spark/target hcatalog/target hcatalog/core/target hcatalog/streaming/target hcatalog/server-extensions/target hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/NotificationListener.java.orig hcatalog/hcatalog-pig-adapter/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target accumulo-handler/target hwi/target common/target common/src/gen spark-client/target service/target contrib/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1658135. At revision 1658135. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12696618 - PreCommit-HIVE-TRUNK-Build > AlterPartitionMessage should return getKeyValues instead of getValues > - > > Key: HIVE-9585 > URL: https://issues.apache.org/jira/browse/HIVE-9585 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-9585.pat
[jira] [Updated] (HIVE-9608) Define SPARK_HOME if not defined automagically
[ https://issues.apache.org/jira/browse/HIVE-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9608: --- Attachment: HIVE-9608.patch > Define SPARK_HOME if not defined automagically > -- > > Key: HIVE-9608 > URL: https://issues.apache.org/jira/browse/HIVE-9608 > Project: Hive > Issue Type: Improvement >Reporter: Brock Noland >Assignee: Brock Noland >Priority: Minor > Attachments: HIVE-9608.patch, HIVE-9608.patch > > > many hadoop installs are in {{dir/\{spark,hive,hadoop,..\}}}. We can infer > {{SPARK_HOME}} in these cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9610) Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes
[ https://issues.apache.org/jira/browse/HIVE-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311159#comment-14311159 ] Hive QA commented on HIVE-9610: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12697310/HIVE-9610.patch {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testMetastoreProxyUser org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge.testSaslWithHiveMetaStore org.apache.hive.spark.client.TestSparkClient.testSyncRpc {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2708/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2708/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2708/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12697310 - PreCommit-HIVE-TRUNK-Build > Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes > > > Key: HIVE-9610 > URL: https://issues.apache.org/jira/browse/HIVE-9610 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Brock Noland >Assignee: Brock Noland > Attachments: HIVE-9610.patch > > > We've not had success only including specific shim classes as part of the > standalone jdbc jar. Since all shim classes shouldn't be too large we'll > include them all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)