[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: (was: Carl Steinbach)
mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
--
Key: HIVE-4558
URL: https://issues.apache.org/jira/browse/HIVE-4558
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.11.0
Environment: Windows
Reporter: Xi Fang
Attachments: HIVE-4558.1.patch
testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
information is printed out on Windows. Here is the example of
mapreduce_stack_trace_hadoop20.q.out.orig:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output:
hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
Ended Job = job_20130514152027587_0001 with errors
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
groups found for user Administrators
Hive Runtime Error while processing row {key:238,value:val_238}
--
However, it is supposed to look like:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
\ A masked pattern was here
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
Hive Runtime Error while processing row {key:238,value:val_238}
--
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: Carl Steinbach
mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
--
Key: HIVE-4558
URL: https://issues.apache.org/jira/browse/HIVE-4558
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.11.0
Environment: Windows
Reporter: Xi Fang
Assignee: Carl Steinbach
Attachments: HIVE-4558.1.patch
testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
information is printed out on Windows. Here is the example of
mapreduce_stack_trace_hadoop20.q.out.orig:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output:
hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
Ended Job = job_20130514152027587_0001 with errors
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
groups found for user Administrators
Hive Runtime Error while processing row {key:238,value:val_238}
--
However, it is supposed to look like:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
\ A masked pattern was here
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
Hive Runtime Error while processing row {key:238,value:val_238}
--
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-4558:
-
Assignee: Xi Fang
mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
--
Key: HIVE-4558
URL: https://issues.apache.org/jira/browse/HIVE-4558
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.11.0
Environment: Windows
Reporter: Xi Fang
Assignee: Xi Fang
Attachments: HIVE-4558.1.patch
testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
information is printed out on Windows. Here is the example of
mapreduce_stack_trace_hadoop20.q.out.orig:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output:
hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
Ended Job = job_20130514152027587_0001 with errors
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
groups found for user Administrators
Hive Runtime Error while processing row {key:238,value:val_238}
--
However, it is supposed to look like:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
\ A masked pattern was here
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
Hive Runtime Error while processing row {key:238,value:val_238}
--
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-3129) Create windows native scripts (CMD files) to run hive on windows without Cygwin
[ https://issues.apache.org/jira/browse/HIVE-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3129: - Assignee: Xi Fang Create windows native scripts (CMD files) to run hive on windows without Cygwin Key: HIVE-3129 URL: https://issues.apache.org/jira/browse/HIVE-3129 Project: Hive Issue Type: Bug Components: CLI, Windows Affects Versions: 0.11.0 Reporter: Kanna Karanam Assignee: Xi Fang Labels: Windows Attachments: HIVE-3129.1.patch, HIVE-3129.2.patch, HIVE-3129.unittest.2.patch, HIVE-3129.unittest.patch Create the cmd files equivalent to a)Bin\hive b)Bin\hive-config.sh c)Bin\Init-hive-dfs.sh d)Bin\ext\cli.sh e)Bin\ext\debug.sh f)Bin\ext\help.sh g)Bin\ext\hiveserver.sh h)Bin\ext\jar.sh i)Bin\ext\hwi.sh j)Bin\ext\lineage.sh k)Bin\ext\metastore.sh l)Bin\ext\rcfilecat.sh -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-4349) Fix the Hive unit test failures when the Hive enlistment root path is longer than ~12 characters
[ https://issues.apache.org/jira/browse/HIVE-4349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4349: - Assignee: Xi Fang Fix the Hive unit test failures when the Hive enlistment root path is longer than ~12 characters Key: HIVE-4349 URL: https://issues.apache.org/jira/browse/HIVE-4349 Project: Hive Issue Type: Bug Affects Versions: 0.11.0 Reporter: Xi Fang Assignee: Xi Fang Attachments: HIVE-4349.1.patch If the Hive enlistment root path is longer than 12 chars then test classpath “hadoop.testcp” is exceeding the 8K chars so we are unable to run most of the Hive unit tests on Windows. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-4445) Fix the Hive unit test failures on Windows when Linux scripts or commands are used in test cases
[ https://issues.apache.org/jira/browse/HIVE-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-4445: - Assignee: Xi Fang Fix the Hive unit test failures on Windows when Linux scripts or commands are used in test cases Key: HIVE-4445 URL: https://issues.apache.org/jira/browse/HIVE-4445 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.11.0 Environment: Windows Reporter: Xi Fang Assignee: Xi Fang Attachments: HIVE-4445.1.patch The following unit tests fail on Windows because Linux scripts or commands are used in the test cases or .q files: 1. TestMinimrCliDriver: scriptfile1.q 2. TestNegativeMinimrCliDriver: mapreduce_stack_trace_hadoop20.q, minimr_broken_pipe.q 3. TestCliDriver: hiveprofiler_script0.q -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6302) annotate_stats_*.q are failing on trunk
[
https://issues.apache.org/jira/browse/HIVE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881771#comment-13881771
]
Hive QA commented on HIVE-6302:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624995/HIVE-6302.1.patch.txt
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1011/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1011/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1011/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java'
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java'
Reverted
'metastore/src/test/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java'
Reverted 'metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java'
Reverted 'metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java'
Reverted
'metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java'
Reverted 'metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py'
Reverted 'metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py'
Reverted
'metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote'
Reverted 'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp'
Reverted 'metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp'
Reverted 'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h'
Reverted 'metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h'
Reverted
'metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp'
Reverted 'metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb'
Reverted 'metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsResult.java'
Reverted
'metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AddPartitionsRequest.java'
Reverted 'metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php'
Reverted 'metastore/src/gen/thrift/gen-php/metastore/Types.php'
Reverted 'metastore/if/hive_metastore.thrift'
Reverted
'itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java'
Reverted 'ql/src/test/results/clientpositive/metadataonly1.q.out'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsResult.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/TableStatsRequest.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsRequest.java
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PartitionsStatsResult.java
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target
[jira] [Commented] (HIVE-6157) Fetching column stats slower than the 101 during rush hour
[
https://issues.apache.org/jira/browse/HIVE-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881770#comment-13881770
]
Hive QA commented on HIVE-6157:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625080/HIVE-6157.03.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1010/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1010/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625080
Fetching column stats slower than the 101 during rush hour
--
Key: HIVE-6157
URL: https://issues.apache.org/jira/browse/HIVE-6157
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Gunther Hagleitner
Assignee: Sergey Shelukhin
Attachments: HIVE-6157.01.patch, HIVE-6157.01.patch,
HIVE-6157.03.patch, HIVE-6157.03.patch, HIVE-6157.nogen.patch,
HIVE-6157.nogen.patch, HIVE-6157.prelim.patch
hive.stats.fetch.column.stats controls whether the column stats for a table
are fetched during explain (in Tez: during query planning). On my setup (1
table 4000 partitions, 24 columns) the time spent in semantic analyze goes
from ~1 second to ~66 seconds when turning the flag on. 65 seconds spent
fetching column stats...
The reason is probably that the APIs force you to make separate metastore
calls for each column in each partition. That's probably the first thing that
has to change. The question is if in addition to that we need to cache this
in the client or store the stats as a single blob in the database to further
cut down on the time. However, the way it stands right now column stats seem
unusable.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names
[ https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881793#comment-13881793 ] Lefty Leverenz commented on HIVE-6013: -- Are backticks required for column names? If not, then do column names specified without backticks have the old limitations (alphanumeric plus underscore) even when hive.support.quoted.identifiers is set to 'column'? Supporting Quoted Identifiers in Column Names - Key: HIVE-6013 URL: https://issues.apache.org/jira/browse/HIVE-6013 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, HIVE-6013.4.patch, HIVE-6013.5.patch, HIVE-6013.6.patch, HIVE-6013.7.patch, QuotedIdentifier.html Hive's current behavior on Quoted Identifiers is different from the normal interpretation. Quoted Identifier (using backticks) has a special interpretation for Select expressions(as Regular Expressions). Have documented current behavior and proposed a solution in attached doc. Summary of solution is: - Introduce 'standard' quoted identifiers for columns only. - At the langauage level this is turned on by a flag. - At the metadata level we relax the constraint on column names. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6013) Supporting Quoted Identifiers in Column Names
[ https://issues.apache.org/jira/browse/HIVE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881803#comment-13881803 ] Lefty Leverenz commented on HIVE-6013: -- Please review the naming information in these two sections of the DDL wikidoc: * [Create Table |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable] (right after the syntax) * [Alter Column |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterColumn] Still to come: adding hive.support.quoted.identifiers to the Configuration Properties wikidoc. Supporting Quoted Identifiers in Column Names - Key: HIVE-6013 URL: https://issues.apache.org/jira/browse/HIVE-6013 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6013.1.patch, HIVE-6013.2.patch, HIVE-6013.3.patch, HIVE-6013.4.patch, HIVE-6013.5.patch, HIVE-6013.6.patch, HIVE-6013.7.patch, QuotedIdentifier.html Hive's current behavior on Quoted Identifiers is different from the normal interpretation. Quoted Identifier (using backticks) has a special interpretation for Select expressions(as Regular Expressions). Have documented current behavior and proposed a solution in attached doc. Summary of solution is: - Introduce 'standard' quoted identifiers for columns only. - At the langauage level this is turned on by a flag. - At the metadata level we relax the constraint on column names. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-4996) unbalanced calls to openTransaction/commitTransaction
[ https://issues.apache.org/jira/browse/HIVE-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881806#comment-13881806 ] Lefty Leverenz commented on HIVE-4996: -- The patch is misnamed: hive-4996.path should be hive-4996.patch. (It downloads instead of opening when I click the link.) unbalanced calls to openTransaction/commitTransaction - Key: HIVE-4996 URL: https://issues.apache.org/jira/browse/HIVE-4996 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0, 0.11.0, 0.12.0 Environment: hiveserver1 Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Reporter: wangfeng Priority: Critical Labels: hive, metastore Attachments: hive-4996.path Original Estimate: 504h Remaining Estimate: 504h when we used hiveserver1 based on hive-0.10.0, we found the Exception thrown.It was: FAILED: Error in metadata: MetaException(message:java.lang.RuntimeException: commitTransaction was called but openTransactionCalls = 0. This probably indicates that the re are unbalanced calls to openTransaction/commitTransaction) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask help -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5929) SQL std auth - Access control statement updates
[
https://issues.apache.org/jira/browse/HIVE-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881808#comment-13881808
]
Hive QA commented on HIVE-5929:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624926/HIVE-5929.4.patch
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 4959 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_part_inherit_tbl_props
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_part_inherit_tbl_props_with_star
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testNegativeCliDriver_url_hook
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_add_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_partition_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_disallow_incompatible_type_change_on1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1012/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1012/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624926
SQL std auth - Access control statement updates
---
Key: HIVE-5929
URL: https://issues.apache.org/jira/browse/HIVE-5929
Project: Hive
Issue Type: Sub-task
Components: Authorization
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Attachments: HIVE-5929.1.patch, HIVE-5929.2.patch, HIVE-5929.3.patch,
HIVE-5929.4.patch
Original Estimate: 48h
Remaining Estimate: 48h
Subtask for sql standard based auth, for performing the updates to metastore
from newly supported access control statements .
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter table partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Status: Open (was: Patch Available)
alter table partition column throws NPE in authorization
--
Key: HIVE-6205
URL: https://issues.apache.org/jira/browse/HIVE-6205
Project: Hive
Issue Type: Bug
Components: Authorization
Reporter: Navis
Assignee: Navis
Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
alter table alter_coltype partition column (dt int);
{noformat}
2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
{noformat}
Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter table partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Status: Patch Available (was: Open)
alter table partition column throws NPE in authorization
--
Key: HIVE-6205
URL: https://issues.apache.org/jira/browse/HIVE-6205
Project: Hive
Issue Type: Bug
Components: Authorization
Reporter: Navis
Assignee: Navis
Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
alter table alter_coltype partition column (dt int);
{noformat}
2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
{noformat}
Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6264) Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
[
https://issues.apache.org/jira/browse/HIVE-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881846#comment-13881846
]
Hive QA commented on HIVE-6264:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624985/HIVE-6264.2.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1014/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1014/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624985
Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
-
Key: HIVE-6264
URL: https://issues.apache.org/jira/browse/HIVE-6264
Project: Hive
Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6264.1.patch, HIVE-6264.2.patch
HiveParser pushes/pops messages describing the current parse rule like so:
{noformat}
joinSource
@init { gParent.msgs.push(join source); }
@after { gParent.msgs.pop(); }
...
{noformat}
The ANTLR generated code for the init/after actions looks like this:
{noformat}
gParent.msgs.push(join source);
...
if ( state.backtracking==0 ) { gParent.msgs.pop(); }
{noformat}
If we have a parse rule that does some lookahead, the message is always
pushed onto the message stack since the init action has no check of
state.backtracking. But that message is never popped because the after
action does check state.backtracking. As a result there can be a bunch of
parser context messages added to the stack which are never taken off.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6012) restore backward compatibility of arithmetic operations
[
https://issues.apache.org/jira/browse/HIVE-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881904#comment-13881904
]
Hive QA commented on HIVE-6012:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624984/HIVE-6012.1.patch
{color:red}ERROR:{color} -1 due to 93 failed/errored test(s), 4952 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.initializationError
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_autogen_colalias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cast1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_column_access_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_hadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_insert_common_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join38
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_leadlag_queries
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_load_dyn_part12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_hook
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapreduce8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_dynamic_partition5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_lateral_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_num_op_type_conv
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_empty_strings
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_constant_expr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_extended
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook_hadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin_having
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_table_access_keys_stats
[jira] [Commented] (HIVE-6264) Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
[
https://issues.apache.org/jira/browse/HIVE-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881924#comment-13881924
]
Harish Butani commented on HIVE-6264:
-
looks good. Can you check why the 5 minimr tests are failing.
Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead
-
Key: HIVE-6264
URL: https://issues.apache.org/jira/browse/HIVE-6264
Project: Hive
Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6264.1.patch, HIVE-6264.2.patch
HiveParser pushes/pops messages describing the current parse rule like so:
{noformat}
joinSource
@init { gParent.msgs.push(join source); }
@after { gParent.msgs.pop(); }
...
{noformat}
The ANTLR generated code for the init/after actions looks like this:
{noformat}
gParent.msgs.push(join source);
...
if ( state.backtracking==0 ) { gParent.msgs.pop(); }
{noformat}
If we have a parse rule that does some lookahead, the message is always
pushed onto the message stack since the init action has no check of
state.backtracking. But that message is never popped because the after
action does check state.backtracking. As a result there can be a bunch of
parser context messages added to the stack which are never taken off.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6205) alter table partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881933#comment-13881933
]
Hive QA commented on HIVE-6205:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624966/HIVE-6205.5.patch.txt
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1016/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1016/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624966
alter table partition column throws NPE in authorization
--
Key: HIVE-6205
URL: https://issues.apache.org/jira/browse/HIVE-6205
Project: Hive
Issue Type: Bug
Components: Authorization
Reporter: Navis
Assignee: Navis
Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
alter table alter_coltype partition column (dt int);
{noformat}
2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
{noformat}
Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5843) Transaction manager for Hive
[
https://issues.apache.org/jira/browse/HIVE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881960#comment-13881960
]
Hive QA commented on HIVE-5843:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624960/HIVE-5843.4.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5052 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1017/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1017/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624960
Transaction manager for Hive
Key: HIVE-5843
URL: https://issues.apache.org/jira/browse/HIVE-5843
Project: Hive
Issue Type: Sub-task
Affects Versions: 0.12.0
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: 0.13.0
Attachments: HIVE-5843-src-only.patch, HIVE-5843.2.patch,
HIVE-5843.3-src.path, HIVE-5843.3.patch, HIVE-5843.4-src.patch,
HIVE-5843.4.patch, HIVE-5843.patch, HiveTransactionManagerDetailedDesign
(1).pdf
As part of the ACID work proposed in HIVE-5317 a transaction manager is
required.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881972#comment-13881972 ] Harish Butani commented on HIVE-6300: - looks good Add documentation for stats configs to hive-default.xml.template Key: HIVE-6300 URL: https://issues.apache.org/jira/browse/HIVE-6300 Project: Hive Issue Type: Sub-task Components: Query Processor, Statistics Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6300.1.patch Add documentation for the following configs hive.stats.max.variable.length hive.stats.list.num.entries hive.stats.map.num.entries hive.stats.map.parallelism hive.stats.fetch.column.stats hive.stats.avg.row.size hive.stats.join.factor hive.stats.deserialization.factor hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5783) Native Parquet Support in Hive
[ https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5783: --- Attachment: HIVE-5783.patch I believe the test issues have been resolved. Uploading same patch for another round of testing. Native Parquet Support in Hive -- Key: HIVE-5783 URL: https://issues.apache.org/jira/browse/HIVE-5783 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Justin Coffey Assignee: Justin Coffey Priority: Minor Fix For: 0.13.0 Attachments: HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch Problem Statement: Hive would be easier to use if it had native Parquet support. Our organization, Criteo, uses Hive extensively. Therefore we built the Parquet Hive integration and would like to now contribute that integration to Hive. About Parquet: Parquet is a columnar storage format for Hadoop and integrates with many Hadoop ecosystem tools such as Thrift, Avro, Hadoop MapReduce, Cascading, Pig, Drill, Crunch, and Hive. Pig, Crunch, and Drill all contain native Parquet integration. Changes Details: Parquet was built with dependency management in mind and therefore only a single Parquet jar will be added as a dependency. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6268) Network resource leak with HiveClientCache when using HCatInputFormat
[
https://issues.apache.org/jira/browse/HIVE-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882005#comment-13882005
]
Hive QA commented on HIVE-6268:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12624843/HIVE-6268.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4958 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1022/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1022/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12624843
Network resource leak with HiveClientCache when using HCatInputFormat
-
Key: HIVE-6268
URL: https://issues.apache.org/jira/browse/HIVE-6268
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.12.0
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-6268.patch
HCatInputFormat has a cache feature that allows HCat to cache hive client
connections to the metastore, so as to not keep reinstantiating a new hive
server every single time. This uses a guava cache of hive clients, which only
evicts entries from cache on the next write, or by manually managing the
cache.
So, in a single threaded case, where we reuse the hive client, the cache
works well, but in a massively multithreaded case, where each thread might
perform one action, and then is never used, there are no more writes to the
cache, and all the clients stay alive, thus keeping ports open.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6205) alter table partition column throws NPE in authorization
[
https://issues.apache.org/jira/browse/HIVE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-6205:
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Patch committed to trunk. Thanks for the contribution Navis!
alter table partition column throws NPE in authorization
--
Key: HIVE-6205
URL: https://issues.apache.org/jira/browse/HIVE-6205
Project: Hive
Issue Type: Bug
Components: Authorization
Reporter: Navis
Assignee: Navis
Fix For: 0.13.0
Attachments: HIVE-6205.1.patch.txt, HIVE-6205.2.patch.txt,
HIVE-6205.3.patch.txt, HIVE-6205.4.patch.txt, HIVE-6205.5.patch.txt
alter table alter_coltype partition column (dt int);
{noformat}
2014-01-15 15:53:40,364 ERROR ql.Driver (SessionState.java:printError(457)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:599)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:996)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1039)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:922)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
{noformat}
Operation for TOK_ALTERTABLE_ALTERPARTS is not defined.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6226) It should be possible to get hadoop, hive, and pig version being used by WebHCat
[
https://issues.apache.org/jira/browse/HIVE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882053#comment-13882053
]
Hive QA commented on HIVE-6226:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625101/HIVE-6226.2.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4961 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
org.apache.hadoop.hive.jdbc.TestJdbcDriver.testExplainStmt
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1023/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1023/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625101
It should be possible to get hadoop, hive, and pig version being used by
WebHCat
Key: HIVE-6226
URL: https://issues.apache.org/jira/browse/HIVE-6226
Project: Hive
Issue Type: New Feature
Components: WebHCat
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: 0.13.0
Attachments: HIVE-6226.2.patch, HIVE-6226.patch
Calling /version on WebHCat tells the caller the protocol verison, but there
is no way to determine the versions of software being run by the applications
that WebHCat spawns.
I propose to add an end-point: /version/\{module\} where module could be pig,
hive, or hadoop. The response will then be:
{code}
{
module : _module_name_,
version : _version_string_
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6287) batchSize computation in Vectorized ORC reader can cause BufferUnderFlowException when PPD is enabled
[
https://issues.apache.org/jira/browse/HIVE-6287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882099#comment-13882099
]
Hive QA commented on HIVE-6287:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625092/HIVE-6287.2.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4959 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_vectorization_ppd
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1025/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1025/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625092
batchSize computation in Vectorized ORC reader can cause
BufferUnderFlowException when PPD is enabled
-
Key: HIVE-6287
URL: https://issues.apache.org/jira/browse/HIVE-6287
Project: Hive
Issue Type: Bug
Components: Vectorization
Affects Versions: 0.13.0
Reporter: Prasanth J
Assignee: Prasanth J
Labels: orcfile, vectorization
Attachments: HIVE-6287.1.patch, HIVE-6287.2.patch, HIVE-6287.WIP.patch
nextBatch() method that computes the batchSize is only aware of stripe
boundaries. This will not work when predicate pushdown (PPD) in ORC is
enabled as PPD works at row group level (stripe contains multiple row
groups). By default, row group stride is 1. When PPD is enabled, some row
groups may get eliminated. After row group elimination, disk ranges are
computed based on the selected row groups. If batchSize computation is not
aware of this, it will lead to BufferUnderFlowException (reading beyond disk
range). Following scenario should illustrate it more clearly
{code}
|- STRIPE 1
|
|-- row grp 1 --|-- row grp 2 --|-- row grp 3 --|-- row grp 4 --|-- row grp 5
--|
|- diskrange 1 -| |- diskrange
2 -|
^
(marker)
{code}
diskrange1 will have 2 rows and diskrange 2 will have 1 rows. Since
nextBatch() was not aware of row groups and hence the diskranges, it tries to
read 1024 values from the end of diskrange 1 where it should only read 2
% 1024 = 544 values. This will result in BufferUnderFlowException.
To fix this, a marker is placed at the end of each range and batchSize is
computed accordingly. {code}batchSize =
Math.min(VectorizedRowBatch.DEFAULT_SIZE, (markerPosition -
rowInStripe));{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-4558) mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882100#comment-13882100
]
Hive QA commented on HIVE-4558:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12583228/HIVE-4558.1.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1029/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1029/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1029/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
service/target contrib/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
ql/src/test/results/clientpositive/orc_vectorization_ppd.q.out
ql/src/test/queries/clientpositive/orc_vectorization_ppd.q
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1561419.
At revision 1561419.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12583228
mapreduce_stack_trace_hadoop20 in TestNegativeMinimrCliDriver fails on Windows
--
Key: HIVE-4558
URL: https://issues.apache.org/jira/browse/HIVE-4558
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.11.0
Environment: Windows
Reporter: Xi Fang
Assignee: Xi Fang
Attachments: HIVE-4558.1.patch
testNegativeCliDriver_mapreduce_stack_trace_hadoop20 fails because group
information is printed out on Windows. Here is the example of
mapreduce_stack_trace_hadoop20.q.out.orig:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
PREHOOK: Output:
hdfs://127.0.0.1:25477/code/HWX/hive-monarch/build/ql/scratchdir/hive_2013-05-14_15-21-00_075_593034964465269090/-mr-1
Ended Job = job_20130514152027587_0001 with errors
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {key:238,value:val_238}
groups found for user Administrators
Hive Runtime Error while processing row {key:238,value:val_238}
--
However, it is supposed to look like:
--
PREHOOK: query: FROM src SELECT TRANSFORM(key, value) USING
'script_does_not_exist' AS (key, value)
PREHOOK: type: QUERY
PREHOOK: Input: default@src
\ A masked pattern was here
FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
[jira] [Commented] (HIVE-6293) Not all minimr tests are executed or reported in precommit test run
[
https://issues.apache.org/jira/browse/HIVE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882108#comment-13882108
]
Xuefu Zhang commented on HIVE-6293:
---
Latest miniMR test run shows 3 failures after all tests are enabled, all
complaining ${system:test.dfs.mkdir} in the script. However, these tests passed
locally, and the system var is defined in qtest pom.xml. What's wrong with
ptest for not taking this var?
Not all minimr tests are executed or reported in precommit test run
---
Key: HIVE-6293
URL: https://issues.apache.org/jira/browse/HIVE-6293
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Affects Versions: 0.13.0
Reporter: Xuefu Zhang
It seems that not all q file tests for minimr are executed or reported in the
pre-commit test run. Here is an example:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/987/testReport/org.apache.hadoop.hive.cli/TestMinimrCliDriver/
This might be due to ptest because manually running test TestMinimrCliDriver
seems executing all tests. My last run shows 38 tests run, with 8 test
failures.
This is identified in HIVE-5446. It needs to be fixed to have broader
coverage.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Hive-branch-0.12-hadoop2 - Build # 27 - Failure
Changes for Build #26 Changes for Build #27 No tests ran. The Apache Jenkins build system has built Hive-branch-0.12-hadoop2 (build #27) Status: Failure Check console output at https://builds.apache.org/job/Hive-branch-0.12-hadoop2/27/ to view the results.
[jira] [Commented] (HIVE-6226) It should be possible to get hadoop, hive, and pig version being used by WebHCat
[
https://issues.apache.org/jira/browse/HIVE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882121#comment-13882121
]
Thejas M Nair commented on HIVE-6226:
-
+1
It should be possible to get hadoop, hive, and pig version being used by
WebHCat
Key: HIVE-6226
URL: https://issues.apache.org/jira/browse/HIVE-6226
Project: Hive
Issue Type: New Feature
Components: WebHCat
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: 0.13.0
Attachments: HIVE-6226.2.patch, HIVE-6226.patch
Calling /version on WebHCat tells the caller the protocol verison, but there
is no way to determine the versions of software being run by the applications
that WebHCat spawns.
I propose to add an end-point: /version/\{module\} where module could be pig,
hive, or hadoop. The response will then be:
{code}
{
module : _module_name_,
version : _version_string_
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5783) Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882125#comment-13882125
]
Hive QA commented on HIVE-5783:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12625200/HIVE-5783.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 4990 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1032/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1032/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12625200
Native Parquet Support in Hive
--
Key: HIVE-5783
URL: https://issues.apache.org/jira/browse/HIVE-5783
Project: Hive
Issue Type: New Feature
Components: Serializers/Deserializers
Reporter: Justin Coffey
Assignee: Justin Coffey
Priority: Minor
Fix For: 0.13.0
Attachments: HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch,
HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch, HIVE-5783.patch,
HIVE-5783.patch
Problem Statement:
Hive would be easier to use if it had native Parquet support. Our
organization, Criteo, uses Hive extensively. Therefore we built the Parquet
Hive integration and would like to now contribute that integration to Hive.
About Parquet:
Parquet is a columnar storage format for Hadoop and integrates with many
Hadoop ecosystem tools such as Thrift, Avro, Hadoop MapReduce, Cascading,
Pig, Drill, Crunch, and Hive. Pig, Crunch, and Drill all contain native
Parquet integration.
Changes Details:
Parquet was built with dependency management in mind and therefore only a
single Parquet jar will be added as a dependency.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6300) Add documentation for stats configs to hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882153#comment-13882153 ] Lefty Leverenz commented on HIVE-6300: -- Good detailed descriptions. Just some nit-picks and a few points of confusion: # Please limit the line lengths to 100 chars. (hive-default.xml.template is far from perfect on this convention, but I'm planning to tidy it up someday.) # hive/tez should be Hive/Tez and java should be Java in these descriptions: #* hive.stats.max.variable.length #* hive.stats.list.num.entries #* hive.stats.map.num.entries # In hive.stats.map.parallelism description: #* through each of the operator should be operators or through each operator #* Some operators like GROUPBY, generates more number of rows that corresponds to the number of mappers. -- omit the comma, make generates singular, and I'm not sure what you mean by more number of rows that corresponds to the number of mappers -- what's the correspondence, more rows means more parallelism? At first I thought that should be than but now I don't know. The comment in HiveConf.java is simpler: to accurately compute statistics for GROUPBY map side parallelism needs to be known. #* hive should be Hive # In hive.stats.fetch.column.stats description, for each needed columns should be column and when the number of columns are high should be is high. Also, why does the comment in HiveConf.java mention partitions too? Maybe it's left over from previous behavior, before hive.stats.fetch.partition.stats was created: #* +// statistics annotation fetches column statistics for all required columns and for all +// required partitions which can be very expensive sometimes # In hive.stats.fetch.partition.stats description, paritition should be partition and when the number of partitions are high should be is high. Also, does this information mean the same as what's in HiveConf.java? #* When this flag is disabled, Hive will make calls to filesystem to get file sizes and will estimate the number of rows from row schema. #* HiveConf.java: basic sizes being fetched from namenode # In hive.stats.avg.row.size description: #* again, through each of the operator should be operators or through each operator #* LIMIT operator (which knows the number of rows) will use this value to estimate the size of data flowing through LIMIT operator left me wondering what's done to estimate data flowing through other operators. (But now I realize they're estimated using other configs. But isn't it the optimizer that uses this value, not the LIMIT operator?) Also, this description doesn't seem to match what's in HiveConf.java -- average row size will be used to estimate the number of rows/data size -- is number of rows known or not? # In hive.stats.join.factor description: #* again, through each of the operator should be operators or through each operator #* by the way, in HiveConf.java the comment is slightly garbled: in the absence of column statistics, the estimated number of rows/data size that will be emitted from join operator will depend on t this factor # In hive.stats.deserialization.factor description: #* again, through each of the operator should be operators or through each operator #* Since files in table/partitions are ... should be tables/partitions (micro-nit) Whew. Sorry about the number of nits. If you like, I can make these changes in a temporary patch and let you remove the ones you don't like and clear up confusions in a third patch. Add documentation for stats configs to hive-default.xml.template Key: HIVE-6300 URL: https://issues.apache.org/jira/browse/HIVE-6300 Project: Hive Issue Type: Sub-task Components: Query Processor, Statistics Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6300.1.patch Add documentation for the following configs hive.stats.max.variable.length hive.stats.list.num.entries hive.stats.map.num.entries hive.stats.map.parallelism hive.stats.fetch.column.stats hive.stats.avg.row.size hive.stats.join.factor hive.stats.deserialization.factor hive.stats.fetch.partition.stats -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni updated HIVE-6290: --- Attachment: HIVE-6290.1.patch.txt Add support for hbase filters for composite keys Key: HIVE-6290 URL: https://issues.apache.org/jira/browse/HIVE-6290 Project: Hive Issue Type: Sub-task Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6290.1.patch.txt Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (HIVE-6132) Support hbase filters for Hive HBase Integration
[ https://issues.apache.org/jira/browse/HIVE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni reassigned HIVE-6132: -- Assignee: Swarnim Kulkarni Support hbase filters for Hive HBase Integration Key: HIVE-6132 URL: https://issues.apache.org/jira/browse/HIVE-6132 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Along with predicate pushdown to optimize HBase table scans, we should support HBase filters as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Review Request 17370: Support for hbase filters for composite keys
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17370/ --- Review request for hive. Bugs: HIVE-6290 https://issues.apache.org/jira/browse/HIVE-6290 Repository: hive-git Description --- Support to be able to provide scan filter for hive/hbase integration for composite keys. Diffs - hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java 5008f15 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 899ccce hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 704fcb9 hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestCompositeKey.java 13c344b ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java d39ee2e ql/src/java/org/apache/hadoop/hive/ql/index/IndexSearchCondition.java 5f1329c ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java e50026b Diff: https://reviews.apache.org/r/17370/diff/ Testing --- Tested using locally created data and debug logs that the filter gets applied on the scan. Thanks, Swarnim Kulkarni
[jira] [Updated] (HIVE-6290) Add support for hbase filters for composite keys
[ https://issues.apache.org/jira/browse/HIVE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni updated HIVE-6290: --- Status: Patch Available (was: Open) Add support for hbase filters for composite keys Key: HIVE-6290 URL: https://issues.apache.org/jira/browse/HIVE-6290 Project: Hive Issue Type: Sub-task Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6290.1.patch.txt Add support for filters to be provided via the composite key class -- This message was sent by Atlassian JIRA (v6.1.5#6160)
