[jira] [Updated] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni updated HIVE-6290: --- Status: Patch Available (was: Open) > Add support for hbase filters for composite keys > > > Key: HIVE-6290 > URL: https://issues.apache.org/jira/browse/HIVE-6290 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 0.12.0 >Reporter: Swarnim Kulkarni >Assignee: Swarnim Kulkarni > Attachments: HIVE-6290.1.patch.txt > > > Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Review Request 17370: Support for hbase filters for composite keys
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17370/ --- Review request for hive. Bugs: HIVE-6290 https://issues.apache.org/jira/browse/HIVE-6290 Repository: hive-git Description --- Support to be able to provide scan filter for hive/hbase integration for composite keys. Diffs - hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java 5008f15 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 899ccce hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 704fcb9 hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestCompositeKey.java 13c344b ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java d39ee2e ql/src/java/org/apache/hadoop/hive/ql/index/IndexSearchCondition.java 5f1329c ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java e50026b Diff: https://reviews.apache.org/r/17370/diff/ Testing --- Tested using locally created data and debug logs that the filter gets applied on the scan. Thanks, Swarnim Kulkarni
[jira] [Commented] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882192#comment-13882192 ] Swarnim Kulkarni commented on HIVE-6290: Review request: https://reviews.apache.org/r/17370/ > Add support for hbase filters for composite keys > > > Key: HIVE-6290 > URL: https://issues.apache.org/jira/browse/HIVE-6290 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 0.12.0 >Reporter: Swarnim Kulkarni >Assignee: Swarnim Kulkarni > Attachments: HIVE-6290.1.patch.txt > > > Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni updated HIVE-6290: --- Attachment: HIVE-6290.1.patch.txt > Add support for hbase filters for composite keys > > > Key: HIVE-6290 > URL: https://issues.apache.org/jira/browse/HIVE-6290 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 0.12.0 >Reporter: Swarnim Kulkarni >Assignee: Swarnim Kulkarni > Attachments: HIVE-6290.1.patch.txt > > > Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (HIVE-6132) Support hbase filters for Hive HBase Integration
[ https://issues.apache.org/jira/browse/HIVE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni reassigned HIVE-6132: -- Assignee: Swarnim Kulkarni > Support hbase filters for Hive HBase Integration > > > Key: HIVE-6132 > URL: https://issues.apache.org/jira/browse/HIVE-6132 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 0.12.0 >Reporter: Swarnim Kulkarni >Assignee: Swarnim Kulkarni > > Along with predicate pushdown to optimize HBase table scans, we should > support HBase filters as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882153#comment-13882153 ] Lefty Leverenz commented on HIVE-6300: -- Good detailed descriptions. Just some nit-picks and a few points of confusion: # Please limit the line lengths to 100 chars. (hive-default.xml.template is far from perfect on this convention, but I'm planning to tidy it up someday.) # "hive/tez" should be Hive/Tez and "java" should be Java in these descriptions: #* hive.stats.max.variable.length #* hive.stats.list.num.entries #* hive.stats.map.num.entries # In hive.stats.map.parallelism description: #* "through each of the operator" should be "operators" or "through each operator" #* "Some operators like GROUPBY, generates more number of rows that corresponds to the number of mappers." -- omit the comma, make "generates" singular, and I'm not sure what you mean by "more number of rows that corresponds to the number of mappers" -- what's the correspondence, more rows means more parallelism? At first I thought "that" should be "than" but now I don't know. The comment in HiveConf.java is simpler: "to accurately compute statistics for GROUPBY map side parallelism needs to be known". #* "hive" should be Hive # In hive.stats.fetch.column.stats description, "for each needed columns" should be "column" and "when the number of columns are high" should be "is high". Also, why does the comment in HiveConf.java mention partitions too? Maybe it's left over from previous behavior, before hive.stats.fetch.partition.stats was created: #* +// statistics annotation fetches column statistics for all required columns and for all +// required partitions which can be very expensive sometimes # In hive.stats.fetch.partition.stats description, "paritition" should be "partition" and "when the number of partitions are high" should be "is high". Also, does this information mean the same as what's in HiveConf.java? #* "When this flag is disabled, Hive will make calls to filesystem to get file sizes and will estimate the number of rows from row schema." #* HiveConf.java: "basic sizes being fetched from namenode" # In hive.stats.avg.row.size description: #* again, "through each of the operator" should be "operators" or "through each operator" #* "LIMIT operator (which knows the number of rows) will use this value to estimate the size of data flowing through LIMIT operator" left me wondering what's done to estimate data flowing through other operators. (But now I realize they're estimated using other configs. But isn't it the optimizer that uses this value, not the LIMIT operator?) Also, this description doesn't seem to match what's in HiveConf.java -- "average row size will be used to estimate the number of rows/data size" -- is number of rows known or not? # In hive.stats.join.factor description: #* again, "through each of the operator" should be "operators" or "through each operator" #* by the way, in HiveConf.java the comment is slightly garbled: "in the absence of column statistics, the estimated number of rows/data size that will emitted from join operator will depend on t factor" # In hive.stats.deserialization.factor description: #* again, "through each of the operator" should be "operators" or "through each operator" #* "Since files in table/partitions are ..." should be "tables/partitions" (micro-nit) Whew. Sorry about the number of nits. If you like, I can make these changes in a temporary patch and let you remove the ones you don't like and clear up confusions in a third patch. > Add documentation for stats configs to hive-default.xml.template > > > Key: HIVE-6300 > URL: https://issues.apache.org/jira/browse/HIVE-6300 > Project: Hive > Issue Type: Sub-task > Components: Query Processor, Statistics >Affects Versions: 0.13.0 >Reporter: Prasanth J >Assignee: Prasanth J >Priority: Minor > Fix For: 0.13.0 > > Attachments: HIVE-6300.1.patch > > > Add documentation for the following configs > hive.stats.max.variable.length > hive.stats.list.num.entries > hive.stats.map.num.entries > hive.stats.map.parallelism > hive.stats.fetch.column.stats > hive.stats.avg.row.size > hive.stats.join.factor > hive.stats.deserialization.factor > hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5783) Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882125#comment-13882125
]
Hive QA commented on HIVE-5783:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625200/HIVE-5783.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4990 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1032/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1032/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625200
> Native Parquet Support in Hive
> --
>
> Key: HIVE-5783
> URL: https://issues.apache.org/jira/browse/HIVE-5783
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
>Reporter: Justin Coffey
>Assignee: Justin Coffey
>Priority: Minor
> Fix For: 0.13.0
>
> Attachments: HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch,
> HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch,
> HIVE-5783.patch
>
>
> Problem Statement:
> Hive would be easier to use if it had native Parquet support. Our
> organization, Criteo, uses Hive extensively. Therefore we built the Parquet
> Hive integration and would like to now contribute that integration to Hive.
> About Parquet:
> Parquet is a columnar storage format for Hadoop and integrates with many
> Hadoop ecosystem tools such as Thrift, Avro, Hadoop MapReduce, Cascading,
> Pig, Drill, Crunch, and Hive. Pig, Crunch, and Drill all contain native
> Parquet integration.
> Changes Details:
> Parquet was built with dependency management in mind and therefore only a
> single Parquet jar will be added as a dependency.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6226) It should be possible to get hadoop, hive, and pig version being used by WebHCat
[
https://issues.apache.org/jira/browse/HIVE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882121#comment-13882121
]
Thejas M Nair commented on HIVE-6226:
-
+1
> It should be possible to get hadoop, hive, and pig version being used by
> WebHCat
>
>
> Key: HIVE-6226
> URL: https://issues.apache.org/jira/browse/HIVE-6226
> Project: Hive
> Issue Type: New Feature
> Components: WebHCat
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6226.2.patch, HIVE-6226.patch
>
>
> Calling /version on WebHCat tells the caller the protocol verison, but there
> is no way to determine the versions of software being run by the applications
> that WebHCat spawns.
> I propose to add an end-point: /version/\{module\} where module could be pig,
> hive, or hadoop. The response will then be:
> {code}
> {
> "module" : _module_name_,
> "version" : _version_string_
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Hive-branch-0.12-hadoop2 - Build # 27 - Failure
Changes for Build #26 Changes for Build #27 No tests ran. The Apache Jenkins build system has built Hive-branch-0.12-hadoop2 (build #27) Status: Failure Check console output at https://builds.apache.org/job/Hive-branch-0.12-hadoop2/27/ to view the results.
[jira] [Commented] (HIVE-6293) Not all minimr tests are executed or reported in precommit test run
[
https://issues.apache.org/jira/browse/HIVE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882108#comment-13882108
]
Xuefu Zhang commented on HIVE-6293:
---
Latest miniMR test run shows 3 failures after all tests are enabled, all
complaining ${system:test.dfs.mkdir} in the script. However, these tests passed
locally, and the system var is defined in qtest pom.xml. What's wrong with
ptest for not taking this var?
> Not all minimr tests are executed or reported in precommit test run
> ---
>
> Key: HIVE-6293
> URL: https://issues.apache.org/jira/browse/HIVE-6293
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Affects Versions: 0.13.0
>Reporter: Xuefu Zhang
>
> It seems that not all q file tests for minimr are executed or reported in the
> pre-commit test run. Here is an example:
> http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/987/testReport/org.apache.hadoop.hive.cli/TestMinimrCliDriver/
> This might be due to ptest because manually running test TestMinimrCliDriver
> seems executing all tests. My last run shows 38 tests run, with 8 test
> failures.
> This is identified in HIVE-5446. It needs to be fixed to have broader
> coverage.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882100#comment-13882100
]
Hive QA commented on HIVE-4558:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12583228/HIVE-4558.1.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1029/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1029/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1029/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
service/target contrib/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
ql/src/test/results/clientpositive/orc_vectorization_ppd.q.out
ql/src/test/queries/clientpositive/orc_vectorization_ppd.q
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1561419.
At revision 1561419.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12583228
> mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
> --
>
> Key: HIVE-4558
> URL: https://issues.apache.org/jira/browse/HIVE-4558
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 0.11.0
> Environment: Windows
>Reporter: Xi Fang
>Assignee: Xi Fang
> Attachments: HIVE-4558.1.patch
>
>
> testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
> information is printed out on Windows. Here is the example of
> mapreduce_stack_trace_hadoop20.q.out.orig:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> PREHOOK: Output:
> hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
> Ended Job = job_20130514152027587_0001 with errors
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> groups found for user Administrators
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
> However, it is supposed to look like:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> \ A masked pattern was here
> FATAL ExecMapper: org.a
[jira] [Commented] (HIVE-6287) batchSize computation in Vectorized ORC reader can cause BufferUnderFlowException when PPD is enabled
[
https://issues.apache.org/jira/browse/HIVE-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882099#comment-13882099
]
Hive QA commented on HIVE-6287:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625092/HIVE-6287.2.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4959 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_vectorization_ppd
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1025/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1025/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625092
> batchSize computation in Vectorized ORC reader can cause
> BufferUnderFlowException when PPD is enabled
> -
>
> Key: HIVE-6287
> URL: https://issues.apache.org/jira/browse/HIVE-6287
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 0.13.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Labels: orcfile, vectorization
> Attachments: HIVE-6287.1.patch, HIVE-6287.2.patch, HIVE-6287.WIP.patch
>
>
> nextBatch() method that computes the batchSize is only aware of stripe
> boundaries. This will not work when predicate pushdown (PPD) in ORC is
> enabled as PPD works at row group level (stripe contains multiple row
> groups). By default, row group stride is 1. When PPD is enabled, some row
> groups may get eliminated. After row group elimination, disk ranges are
> computed based on the selected row groups. If batchSize computation is not
> aware of this, it will lead to BufferUnderFlowException (reading beyond disk
> range). Following scenario should illustrate it more clearly
> {code}
> |- STRIPE 1
> |
> |-- row grp 1 --|-- row grp 2 --|-- row grp 3 --|-- row grp 4 --|-- row grp 5
> --|
> |- diskrange 1 -| |- diskrange
> 2 -|
> ^
> (marker)
> {code}
> diskrange1 will have 2 rows and diskrange 2 will have 1 rows. Since
> nextBatch() was not aware of row groups and hence the diskranges, it tries to
> read 1024 values from the end of diskrange 1 where it should only read 2
> % 1024 = 544 values. This will result in BufferUnderFlowException.
> To fix this, a marker is placed at the end of each range and batchSize is
> computed accordingly. {code}batchSize =
> Math.min(VectorizedRowBatch.DEFAULT_SIZE, (markerPosition -
> rowInStripe));{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6226) It should be possible to get hadoop, hive, and pig version being used by WebHCat
[
https://issues.apache.org/jira/browse/HIVE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882053#comment-13882053
]
Hive QA commented on HIVE-6226:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625101/HIVE-6226.2.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4961 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
org.apache.hadoop.hive.jdbc.TestJdbcDriver.testExplainStmt
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1023/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1023/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625101
> It should be possible to get hadoop, hive, and pig version being used by
> WebHCat
>
>
> Key: HIVE-6226
> URL: https://issues.apache.org/jira/browse/HIVE-6226
> Project: Hive
> Issue Type: New Feature
> Components: WebHCat
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-6226.2.patch, HIVE-6226.patch
>
>
> Calling /version on WebHCat tells the caller the protocol verison, but there
> is no way to determine the versions of software being run by the applications
> that WebHCat spawns.
> I propose to add an end-point: /version/\{module\} where module could be pig,
> hive, or hadoop. The response will then be:
> {code}
> {
> "module" : _module_name_,
> "version" : _version_string_
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Patch committed to trunk. Thanks for the contribution Navis!
> alter partition column throws NPE in authorization
> --
>
> Key: HIVE-6205
> URL: https://issues.apache.org/jira/browse/HIVE-6205
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Reporter: Navis
>Assignee: Navis
> Fix For: 0.13.0
>
> Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
> HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
>
>
> alter table alter_coltype partition column (dt int);
> {noformat}
> 2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
> FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> {noformat}
> Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6268) Network resource leak with HiveClientCache when using HCatInputFormat
[
https://issues.apache.org/jira/browse/HIVE-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882005#comment-13882005
]
Hive QA commented on HIVE-6268:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624843/HIVE-6268.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1022/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1022/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624843
> Network resource leak with HiveClientCache when using HCatInputFormat
> -
>
> Key: HIVE-6268
> URL: https://issues.apache.org/jira/browse/HIVE-6268
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.12.0
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-6268.patch
>
>
> HCatInputFormat has a cache feature that allows HCat to cache hive client
> connections to the metastore, so as to not keep reinstantiating a new hive
> server every single time. This uses a guava cache of hive clients, which only
> evicts entries from cache on the next write, or by manually managing the
> cache.
> So, in a single threaded case, where we reuse the hive client, the cache
> works well, but in a massively multithreaded case, where each thread might
> perform one action, and then is never used, there are no more writes to the
> cache, and all the clients stay alive, thus keeping ports open.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-5783) Native Parquet Support in Hive
[ https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5783: --- Attachment: HIVE-5783.patch I believe the test issues have been resolved. Uploading same patch for another round of testing. > Native Parquet Support in Hive > -- > > Key: HIVE-5783 > URL: https://issues.apache.org/jira/browse/HIVE-5783 > Project: Hive > Issue Type: New Feature > Components: Serializers/Deserializers >Reporter: Justin Coffey >Assignee: Justin Coffey >Priority: Minor > Fix For: 0.13.0 > > Attachments: HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, > HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, > HIVE-5783.patch > > > Problem Statement: > Hive would be easier to use if it had native Parquet support. Our > organization, Criteo, uses Hive extensively. Therefore we built the Parquet > Hive integration and would like to now contribute that integration to Hive. > About Parquet: > Parquet is a columnar storage format for Hadoop and integrates with many > Hadoop ecosystem tools such as Thrift, Avro, Hadoop MapReduce, Cascading, > Pig, Drill, Crunch, and Hive. Pig, Crunch, and Drill all contain native > Parquet integration. > Changes Details: > Parquet was built with dependency management in mind and therefore only a > single Parquet jar will be added as a dependency. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names
[ https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881973#comment-13881973 ] Harish Butani commented on HIVE-6013: - looks good. thanks for doing this > Supporting Quoted Identifiers in Column Names > - > > Key: HIVE-6013 > URL: https://issues.apache.org/jira/browse/HIVE-6013 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Harish Butani >Assignee: Harish Butani > Fix For: 0.13.0 > > Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, > HIVE-6013.4.patch, HIVE-6013.5.patch, HIVE-6013.6.patch, HIVE-6013.7.patch, > QuotedIdentifier.html > > > Hive's current behavior on Quoted Identifiers is different from the normal > interpretation. Quoted Identifier (using backticks) has a special > interpretation for Select expressions(as Regular Expressions). Have > documented current behavior and proposed a solution in attached doc. > Summary of solution is: > - Introduce 'standard' quoted identifiers for columns only. > - At the langauage level this is turned on by a flag. > - At the metadata level we relax the constraint on column names. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881972#comment-13881972 ] Harish Butani commented on HIVE-6300: - looks good > Add documentation for stats configs to hive-default.xml.template > > > Key: HIVE-6300 > URL: https://issues.apache.org/jira/browse/HIVE-6300 > Project: Hive > Issue Type: Sub-task > Components: Query Processor, Statistics >Affects Versions: 0.13.0 >Reporter: Prasanth J >Assignee: Prasanth J >Priority: Minor > Fix For: 0.13.0 > > Attachments: HIVE-6300.1.patch > > > Add documentation for the following configs > hive.stats.max.variable.length > hive.stats.list.num.entries > hive.stats.map.num.entries > hive.stats.map.parallelism > hive.stats.fetch.column.stats > hive.stats.avg.row.size > hive.stats.join.factor > hive.stats.deserialization.factor > hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5843) Transaction manager for Hive
[
https://issues.apache.org/jira/browse/HIVE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881960#comment-13881960
]
Hive QA commented on HIVE-5843:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624960/HIVE-5843.4.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5052 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1017/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1017/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624960
> Transaction manager for Hive
>
>
> Key: HIVE-5843
> URL: https://issues.apache.org/jira/browse/HIVE-5843
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 0.12.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 0.13.0
>
> Attachments: HIVE-5843-src-only.patch, HIVE-5843.2.patch,
> HIVE-5843.3-src.path, HIVE-5843.3.patch, HIVE-5843.4-src.patch,
> HIVE-5843.4.patch, HIVE-5843.patch, HiveTransactionManagerDetailedDesign
> (1).pdf
>
>
> As part of the ACID work proposed in HIVE-5317 a transaction manager is
> required.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6205) alter partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881933#comment-13881933
]
Hive QA commented on HIVE-6205:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624966/HIVE-6205.5.patch.txt
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1016/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1016/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624966
> alter partition column throws NPE in authorization
> --
>
> Key: HIVE-6205
> URL: https://issues.apache.org/jira/browse/HIVE-6205
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Reporter: Navis
>Assignee: Navis
> Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
> HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
>
>
> alter table alter_coltype partition column (dt int);
> {noformat}
> 2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
> FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> {noformat}
> Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6264) Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
[
https://issues.apache.org/jira/browse/HIVE-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881924#comment-13881924
]
Harish Butani commented on HIVE-6264:
-
looks good. Can you check why the 5 minimr tests are failing.
> Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
> -
>
> Key: HIVE-6264
> URL: https://issues.apache.org/jira/browse/HIVE-6264
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-6264.1.patch, HIVE-6264.2.patch
>
>
> HiveParser pushes/pops messages describing the current parse rule like so:
> {noformat}
> joinSource
> @init { gParent.msgs.push("join source"); }
> @after { gParent.msgs.pop(); }
> ...
> {noformat}
> The ANTLR generated code for the init/after actions looks like this:
> {noformat}
> gParent.msgs.push("join source");
> ...
> if ( state.backtracking==0 ) { gParent.msgs.pop(); }
> {noformat}
> If we have a parse rule that does some lookahead, the message is always
> pushed onto the message stack since the init action has no check of
> state.backtracking. But that message is never popped because the after
> action does check state.backtracking. As a result there can be a bunch of
> parser context messages added to the stack which are never taken off.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881904#comment-13881904
]
Hive QA commented on HIVE-6012:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624984/HIVE-6012.1.patch
{color:red}ERROR:{color} -1 due to 93 failed/errored test(s), 4952 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.initializationError
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_autogen_colalias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cast1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_column_access_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_hadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_insert_common_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join38
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_leadlag_queries
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_load_dyn_part12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_hook
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_lateral_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_num_op_type_conv
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_empty_strings
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_constant_expr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_extended
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook_hadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin_having
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_table_access_keys_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliD
[jira] [Commented] (HIVE-6264) Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
[
https://issues.apache.org/jira/browse/HIVE-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881846#comment-13881846
]
Hive QA commented on HIVE-6264:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624985/HIVE-6264.2.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1014/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1014/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624985
> Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
> -
>
> Key: HIVE-6264
> URL: https://issues.apache.org/jira/browse/HIVE-6264
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-6264.1.patch, HIVE-6264.2.patch
>
>
> HiveParser pushes/pops messages describing the current parse rule like so:
> {noformat}
> joinSource
> @init { gParent.msgs.push("join source"); }
> @after { gParent.msgs.pop(); }
> ...
> {noformat}
> The ANTLR generated code for the init/after actions looks like this:
> {noformat}
> gParent.msgs.push("join source");
> ...
> if ( state.backtracking==0 ) { gParent.msgs.pop(); }
> {noformat}
> If we have a parse rule that does some lookahead, the message is always
> pushed onto the message stack since the init action has no check of
> state.backtracking. But that message is never popped because the after
> action does check state.backtracking. As a result there can be a bunch of
> parser context messages added to the stack which are never taken off.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Status: Patch Available (was: Open)
> alter partition column throws NPE in authorization
> --
>
> Key: HIVE-6205
> URL: https://issues.apache.org/jira/browse/HIVE-6205
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Reporter: Navis
>Assignee: Navis
> Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
> HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
>
>
> alter table alter_coltype partition column (dt int);
> {noformat}
> 2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
> FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> {noformat}
> Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Status: Open (was: Patch Available)
> alter partition column throws NPE in authorization
> --
>
> Key: HIVE-6205
> URL: https://issues.apache.org/jira/browse/HIVE-6205
> Project: Hive
> Issue Type: Bug
> Components: Authorization
>Reporter: Navis
>Assignee: Navis
> Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
> HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
>
>
> alter table alter_coltype partition column (dt int);
> {noformat}
> 2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
> FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> {noformat}
> Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5929) SQL std auth - Access control statement updates
[
https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881808#comment-13881808
]
Hive QA commented on HIVE-5929:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624926/HIVE-5929.4.patch
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 4959 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_part_inherit_tbl_props
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_part_inherit_tbl_props_with_star
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_url_hook
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_add_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_disallow_incompatible_type_change_on1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1012/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1012/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624926
> SQL std auth - Access control statement updates
> ---
>
> Key: HIVE-5929
> URL: https://issues.apache.org/jira/browse/HIVE-5929
> Project: Hive
> Issue Type: Sub-task
> Components: Authorization
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch,
> HIVE-5929.4.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Subtask for sql standard based auth, for performing the updates to metastore
> from newly supported access control statements .
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-4996) unbalanced calls to openTransaction/commitTransaction
[ https://issues.apache.org/jira/browse/HIVE-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881806#comment-13881806 ] Lefty Leverenz commented on HIVE-4996: -- The patch is misnamed: hive-4996.path should be hive-4996.patch. (It downloads instead of opening when I click the link.) > unbalanced calls to openTransaction/commitTransaction > - > > Key: HIVE-4996 > URL: https://issues.apache.org/jira/browse/HIVE-4996 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 0.10.0, 0.11.0, 0.12.0 > Environment: hiveserver1 Java HotSpot(TM) 64-Bit Server VM (build > 20.6-b01, mixed mode) >Reporter: wangfeng >Priority: Critical > Labels: hive, metastore > Attachments: hive-4996.path > > Original Estimate: 504h > Remaining Estimate: 504h > > when we used hiveserver1 based on hive-0.10.0, we found the Exception > thrown.It was: > FAILED: Error in metadata: MetaException(message:java.lang.RuntimeException: > commitTransaction was called but openTransactionCalls = 0. This probably > indicates that the > re are unbalanced calls to openTransaction/commitTransaction) > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > help -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names
[ https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881803#comment-13881803 ] Lefty Leverenz commented on HIVE-6013: -- Please review the naming information in these two sections of the DDL wikidoc: * [Create Table |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable] (right after the syntax) * [Alter Column |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterColumn] Still to come: adding hive.support.quoted.identifiers to the Configuration Properties wikidoc. > Supporting Quoted Identifiers in Column Names > - > > Key: HIVE-6013 > URL: https://issues.apache.org/jira/browse/HIVE-6013 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Harish Butani >Assignee: Harish Butani > Fix For: 0.13.0 > > Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, > HIVE-6013.4.patch, HIVE-6013.5.patch, HIVE-6013.6.patch, HIVE-6013.7.patch, > QuotedIdentifier.html > > > Hive's current behavior on Quoted Identifiers is different from the normal > interpretation. Quoted Identifier (using backticks) has a special > interpretation for Select expressions(as Regular Expressions). Have > documented current behavior and proposed a solution in attached doc. > Summary of solution is: > - Introduce 'standard' quoted identifiers for columns only. > - At the langauage level this is turned on by a flag. > - At the metadata level we relax the constraint on column names. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names
[ https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881793#comment-13881793 ] Lefty Leverenz commented on HIVE-6013: -- Are backticks required for column names? If not, then do column names specified without backticks have the old limitations (alphanumeric plus underscore) even when hive.support.quoted.identifiers is set to 'column'? > Supporting Quoted Identifiers in Column Names > - > > Key: HIVE-6013 > URL: https://issues.apache.org/jira/browse/HIVE-6013 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Harish Butani >Assignee: Harish Butani > Fix For: 0.13.0 > > Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, > HIVE-6013.4.patch, HIVE-6013.5.patch, HIVE-6013.6.patch, HIVE-6013.7.patch, > QuotedIdentifier.html > > > Hive's current behavior on Quoted Identifiers is different from the normal > interpretation. Quoted Identifier (using backticks) has a special > interpretation for Select expressions(as Regular Expressions). Have > documented current behavior and proposed a solution in attached doc. > Summary of solution is: > - Introduce 'standard' quoted identifiers for columns only. > - At the langauage level this is turned on by a flag. > - At the metadata level we relax the constraint on column names. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6302) annotate_stats_*.q are failing on trunk
[
https://issues.apache.org/jira/browse/HIVE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881771#comment-13881771
]
Hive QA commented on HIVE-6302:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624995/HIVE-6302.1.patch.txt
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1011/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1011/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1011/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java'
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java'
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java'
Reverted 'metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java'
Reverted 'metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java'
Reverted 'metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py'
Reverted 'metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py'
Reverted
'metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote'
Reverted 'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp'
Reverted 'metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp'
Reverted 'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h'
Reverted 'metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h'
Reverted
'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp'
Reverted 'metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb'
Reverted 'metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java'
Reverted 'metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php'
Reverted 'metastore/src/gen/thrift/gen-php/metastore/Types.php'
Reverted 'metastore/if/hive_metastore.thrift'
Reverted
'itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java'
Reverted 'ql/src/test/results/clientpositive/metadataonly1.q.out'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-
[jira] [Commented] (HIVE-6157) Fetching column stats slower than the 101 during rush hour
[
https://issues.apache.org/jira/browse/HIVE-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881770#comment-13881770
]
Hive QA commented on HIVE-6157:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625080/HIVE-6157.03.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1010/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1010/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625080
> Fetching column stats slower than the 101 during rush hour
> --
>
> Key: HIVE-6157
> URL: https://issues.apache.org/jira/browse/HIVE-6157
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Gunther Hagleitner
>Assignee: Sergey Shelukhin
> Attachments: HIVE-6157.01.patch, HIVE-6157.01.patch,
> HIVE-6157.03.patch, HIVE-6157.03.patch, HIVE-6157.nogen.patch,
> HIVE-6157.nogen.patch, HIVE-6157.prelim.patch
>
>
> "hive.stats.fetch.column.stats" controls whether the column stats for a table
> are fetched during explain (in Tez: during query planning). On my setup (1
> table 4000 partitions, 24 columns) the time spent in semantic analyze goes
> from ~1 second to ~66 seconds when turning the flag on. 65 seconds spent
> fetching column stats...
> The reason is probably that the APIs force you to make separate metastore
> calls for each column in each partition. That's probably the first thing that
> has to change. The question is if in addition to that we need to cache this
> in the client or store the stats as a single blob in the database to further
> cut down on the time. However, the way it stands right now column stats seem
> unusable.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-4445) Fix the Hive unit test failures on Windows when Linux scripts or commands are used in test cases
[ https://issues.apache.org/jira/browse/HIVE-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4445: - Assignee: Xi Fang > Fix the Hive unit test failures on Windows when Linux scripts or commands are > used in test cases > > > Key: HIVE-4445 > URL: https://issues.apache.org/jira/browse/HIVE-4445 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 0.11.0 > Environment: Windows >Reporter: Xi Fang >Assignee: Xi Fang > Attachments: HIVE-4445.1.patch > > > The following unit tests fail on Windows because Linux scripts or commands > are used in the test cases or .q files: > 1. TestMinimrCliDriver: scriptfile1.q > 2. TestNegativeMinimrCliDriver: mapreduce_stack_trace_hadoop20.q, > minimr_broken_pipe.q > 3. TestCliDriver: hiveprofiler_script0.q -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-3129) Create windows native scripts (CMD files) to run hive on windows without Cygwin
[ https://issues.apache.org/jira/browse/HIVE-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3129: - Assignee: Xi Fang > Create windows native scripts (CMD files) to run hive on windows without > Cygwin > > > Key: HIVE-3129 > URL: https://issues.apache.org/jira/browse/HIVE-3129 > Project: Hive > Issue Type: Bug > Components: CLI, Windows >Affects Versions: 0.11.0 >Reporter: Kanna Karanam >Assignee: Xi Fang > Labels: Windows > Attachments: HIVE-3129.1.patch, HIVE-3129.2.patch, > HIVE-3129.unittest.2.patch, HIVE-3129.unittest.patch > > > Create the cmd files equivalent to > a)Bin\hive > b)Bin\hive-config.sh > c)Bin\Init-hive-dfs.sh > d)Bin\ext\cli.sh > e)Bin\ext\debug.sh > f)Bin\ext\help.sh > g)Bin\ext\hiveserver.sh > h)Bin\ext\jar.sh > i)Bin\ext\hwi.sh > j)Bin\ext\lineage.sh > k)Bin\ext\metastore.sh > l)Bin\ext\rcfilecat.sh -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-4349) Fix the Hive unit test failures when the Hive enlistment root path is longer than ~12 characters
[ https://issues.apache.org/jira/browse/HIVE-4349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4349: - Assignee: Xi Fang > Fix the Hive unit test failures when the Hive enlistment root path is longer > than ~12 characters > > > Key: HIVE-4349 > URL: https://issues.apache.org/jira/browse/HIVE-4349 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0 >Reporter: Xi Fang >Assignee: Xi Fang > Attachments: HIVE-4349.1.patch > > > If the Hive enlistment root path is longer than 12 chars then test classpath > “hadoop.testcp” is exceeding the 8K chars so we are unable to run most of the > Hive unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: Xi Fang
> mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
> --
>
> Key: HIVE-4558
> URL: https://issues.apache.org/jira/browse/HIVE-4558
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 0.11.0
> Environment: Windows
>Reporter: Xi Fang
>Assignee: Xi Fang
> Attachments: HIVE-4558.1.patch
>
>
> testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
> information is printed out on Windows. Here is the example of
> mapreduce_stack_trace_hadoop20.q.out.orig:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> PREHOOK: Output:
> hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
> Ended Job = job_20130514152027587_0001 with errors
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> groups found for user Administrators
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
> However, it is supposed to look like:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> \ A masked pattern was here
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: (was: Carl Steinbach)
> mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
> --
>
> Key: HIVE-4558
> URL: https://issues.apache.org/jira/browse/HIVE-4558
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 0.11.0
> Environment: Windows
>Reporter: Xi Fang
> Attachments: HIVE-4558.1.patch
>
>
> testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
> information is printed out on Windows. Here is the example of
> mapreduce_stack_trace_hadoop20.q.out.orig:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> PREHOOK: Output:
> hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
> Ended Job = job_20130514152027587_0001 with errors
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> groups found for user Administrators
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
> However, it is supposed to look like:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> \ A masked pattern was here
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: Carl Steinbach
> mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
> --
>
> Key: HIVE-4558
> URL: https://issues.apache.org/jira/browse/HIVE-4558
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 0.11.0
> Environment: Windows
>Reporter: Xi Fang
>Assignee: Carl Steinbach
> Attachments: HIVE-4558.1.patch
>
>
> testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
> information is printed out on Windows. Here is the example of
> mapreduce_stack_trace_hadoop20.q.out.orig:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> PREHOOK: Output:
> hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
> Ended Job = job_20130514152027587_0001 with errors
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> groups found for user Administrators
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
> However, it is supposed to look like:
> --
> PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
> 'script_does_not_exist' AS (key, value)
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> \ A masked pattern was here
> FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
> Runtime Error while processing row {"key":"238","value":"val_238"}
> Hive Runtime Error while processing row {"key":"238","value":"val_238"}
> --
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
