[jira] [Created] (HIVE-6431) Hive table name start with underscore (_)
Kuek Chiew Yea created HIVE-6431: Summary: Hive table name start with underscore (_) Key: HIVE-6431 URL: https://issues.apache.org/jira/browse/HIVE-6431 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.12.0 Reporter: Kuek Chiew Yea When I create a hive table with table name start with underscore, I am able to create successfully. The command that I am using is as the following: *CREATE TABLE `_testtable` AS SELECT * FROM `dimdate`;* Once created, I issue a command (as following) to query all records from that table. *SELECT * FROM `_testtable`;* However, I got the error as the following: *Failed with exception java.io.IOException:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://sandbox:8020/apps/hive/warehouse/_testtable* When I run the hdfs command to list the file in directory */apps/hive/warehouse/_testtable*, I am able to get list of files in that directory. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6330) Metastore support for permanent UDFs
[
https://issues.apache.org/jira/browse/HIVE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901199#comment-13901199
]
Hive QA commented on HIVE-6330:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1262/HIVE-6330.5.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5105 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleFunction
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1319/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1319/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1262
Metastore support for permanent UDFs
Key: HIVE-6330
URL: https://issues.apache.org/jira/browse/HIVE-6330
Project: Hive
Issue Type: Sub-task
Components: UDF
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6330.1.patch, HIVE-6330.2.patch, HIVE-6330.3.patch,
HIVE-6330.4.patch, HIVE-6330.5.patch
Allow CREATE FUNCTION to add metastore entry for the created function, so
that it only needs to be added to Hive once.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6399: --- Status: Open (was: Patch Available) bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Remus Rusanu Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.3.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6399: --- Attachment: HIVE-6399.4.patch This is better patch: the multiply algorithm is directly in LittleEndian logic and the patch contains significant more testing for the multiplyArrays4And4To8 cases. bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Remus Rusanu Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 18071: bug in high-precision Decimal128 multiply
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18071/ --- (Updated Feb. 14, 2014, 10:15 a.m.) Review request for hive, Eric Hanson and Jitendra Pandey. Bugs: HIVE-6399 https://issues.apache.org/jira/browse/HIVE-6399 Repository: hive-git Description --- See HIVE-6399 comments Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java fb3c346 common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java 6824cd7 Diff: https://reviews.apache.org/r/18071/diff/ Testing --- regression test case added Thanks, Remus Rusanu
[jira] [Updated] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6399: --- Status: Patch Available (was: Open) bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Remus Rusanu Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6391) Use pre-warm APIs in Tez to improve hive query startup
[ https://issues.apache.org/jira/browse/HIVE-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6391: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch. Thanks Gopal. Use pre-warm APIs in Tez to improve hive query startup -- Key: HIVE-6391 URL: https://issues.apache.org/jira/browse/HIVE-6391 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: tez-branch Reporter: Gopal V Assignee: Gopal V Priority: Minor Labels: optimization Fix For: tez-branch Attachments: HIVE-6391.1-tez.patch, HIVE-6391.2-tez.patch, HIVE-6391.3-tez.patch With the addition of TEZ-766, Tez supports pre-warmed containers within the Tez session. Allow hive users to enable and use this feature from within the hive shell. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6325) Enable using multiple concurrent sessions in tez
[
https://issues.apache.org/jira/browse/HIVE-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901315#comment-13901315
]
Rajesh Balamohan commented on HIVE-6325:
if (!session.isDefault()) {
if
(existingConf.get(tez.queue.name).compareTo(conf.get(tez.queue.name)) != 0)
{
return false;
}
} else {
// this session should never be a default session unless something has
messed up.
throw new HiveException(Default queue should always be returned. Hence we
should not be here.);
}
There is a possibility of existingConf.get(tez.queue.name) throwing NPE
when HIVE_SERVER2_TEZ_INITIALIZE_DEFAULT_SESSIONS is not set.
tez.queue.name is set in the conf in startPool() which is internally called
by HS2 when we enable HIVE_SERVER2_TEZ_INITIALIZE_DEFAULT_SESSIONS. If it is
not enabled, HS2 will not call startPool(); And apparently
existingConf.get(tez.queue.name) can throw NPE.
Enable using multiple concurrent sessions in tez
Key: HIVE-6325
URL: https://issues.apache.org/jira/browse/HIVE-6325
Project: Hive
Issue Type: Improvement
Components: Tez
Affects Versions: 0.13.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Attachments: HIVE-6325.1.patch, HIVE-6325.2.patch, HIVE-6325.3.patch,
HIVE-6325.4.patch, HIVE-6325.5.patch
We would like to enable multiple concurrent sessions in tez via hive server
2. This will enable users to make efficient use of the cluster when it has
been partitioned using yarn queues.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: Review Request 17899: HIVE-5998 Add vectorized reader for Parquet files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17899/ --- (Updated Feb. 14, 2014, 11:07 a.m.) Review request for hive, Brock Noland, Eric Hanson, and Jitendra Pandey. Bugs: HIVE-5998 https://issues.apache.org/jira/browse/HIVE-5998 Repository: hive-git Description --- Implementation is straight forward and very simple, but offers all benefits of vectorization possible with a 'shallow' vectorized reader (ie. one that doe not got into parquet-mr project changes). the only complication arrised because of discrepancies between the object inspector seen by the inputformat and the actual output provided by the Parquet readers (eg. OI declares 'byte' primitives but the Parquet reader outputs IntWritable). I had to create a just-in-time VectorColumnAssigner colelciton base don whatever writers the Parquet record reader provides. It is assumed the reader does not change it's output during the iteration. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java d1a75df ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java 0b504de ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java d409d44 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java d3412df ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java PRE-CREATION ql/src/test/queries/clientpositive/vectorized_parquet.q PRE-CREATION ql/src/test/results/clientpositive/vectorized_parquet.q.out PRE-CREATION Diff: https://reviews.apache.org/r/17899/diff/ Testing --- Manually tested. New query .q added. Thanks, Remus Rusanu
[jira] [Updated] (HIVE-5998) Add vectorized reader for Parquet files
[ https://issues.apache.org/jira/browse/HIVE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-5998: --- Status: Open (was: Patch Available) Add vectorized reader for Parquet files --- Key: HIVE-5998 URL: https://issues.apache.org/jira/browse/HIVE-5998 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers, Vectorization Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: Parquet, vectorization Attachments: HIVE-5998.1.patch, HIVE-5998.2.patch HIVE-5783 is adding native Parquet support in Hive. As Parquet is a columnar format, it makes sense to provide a vectorized reader, similar to how RC and ORC formats have, to benefit from vectorized execution engine. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5998) Add vectorized reader for Parquet files
[ https://issues.apache.org/jira/browse/HIVE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-5998: --- Status: Patch Available (was: Open) Add vectorized reader for Parquet files --- Key: HIVE-5998 URL: https://issues.apache.org/jira/browse/HIVE-5998 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers, Vectorization Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: Parquet, vectorization Attachments: HIVE-5998.1.patch, HIVE-5998.2.patch, HIVE-5998.3.patch HIVE-5783 is adding native Parquet support in Hive. As Parquet is a columnar format, it makes sense to provide a vectorized reader, similar to how RC and ORC formats have, to benefit from vectorized execution engine. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5998) Add vectorized reader for Parquet files
[ https://issues.apache.org/jira/browse/HIVE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-5998: --- Attachment: HIVE-5998.3.patch .3 patch has trailinig ws removed and consistent tab space (I did indeed change the tab setting couple of days ago...) Add vectorized reader for Parquet files --- Key: HIVE-5998 URL: https://issues.apache.org/jira/browse/HIVE-5998 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers, Vectorization Reporter: Remus Rusanu Assignee: Remus Rusanu Priority: Minor Labels: Parquet, vectorization Attachments: HIVE-5998.1.patch, HIVE-5998.2.patch, HIVE-5998.3.patch HIVE-5783 is adding native Parquet support in Hive. As Parquet is a columnar format, it makes sense to provide a vectorized reader, similar to how RC and ORC formats have, to benefit from vectorized execution engine. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
A question about the derivation of intermediate sum field for decimal average aggregates
Hi, With HIVE-5872 the intermediate sum field for decimal aggregates was changed to increase scale by 4. I understand the reasoning for having accurate precision/scale for the aggregate output. However, for the intermediate sum field of AVG, I believe we should increase precision w/o increasing scale. The sum can grow large, but cannot increase digits in the fractional part, so we should increase the precision of the sum, but not the scale. When sum is divided by count to get the average on the reduce side then we should indeed project value with higher scale. Opinions? Thanks, ~Remus
[jira] [Commented] (HIVE-6339) Implement new JDK7 schema management APIs in java.sql.Connection
[
https://issues.apache.org/jira/browse/HIVE-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901378#comment-13901378
]
Hive QA commented on HIVE-6339:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628889/HIVE-6339.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5096 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1320/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1320/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628889
Implement new JDK7 schema management APIs in java.sql.Connection
-
Key: HIVE-6339
URL: https://issues.apache.org/jira/browse/HIVE-6339
Project: Hive
Issue Type: Improvement
Components: JDBC
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-6339.1.patch, HIVE-6339.2.patch
JDK7 has added a few metadata methods in
[java.sql.Conntion|http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html]
{noformat}
getSchema()
setSchema()
getCatalog()
setCatalog()
{noformat}
Currently Hive JDBC just has stub implementation for all these methods throws
unsupported exception. This needs to be fixed.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6418) MapJoinRowContainer has large memory overhead in typical cases
[
https://issues.apache.org/jira/browse/HIVE-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901456#comment-13901456
]
Hive QA commented on HIVE-6418:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628945/HIVE-6418.02.patch
{color:green}SUCCESS:{color} +1 5095 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1321/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1321/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628945
MapJoinRowContainer has large memory overhead in typical cases
--
Key: HIVE-6418
URL: https://issues.apache.org/jira/browse/HIVE-6418
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-6418.01.patch, HIVE-6418.02.patch,
HIVE-6418.WIP.patch, HIVE-6418.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Review Request 18122: Support more generic way of using composite key for HBaseHandler
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18122/ --- Review request for hive, Brock Noland, Navis Ryu, and Swarnim Kulkarni. Bugs: HIVE-6411 https://issues.apache.org/jira/browse/HIVE-6411 Repository: hive-git Description --- Refer to description on HIVE-6411. Diffs - hbase-handler/pom.xml 7c3524c hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java 5008f15 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyFactory.java PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseLazyObjectFactory.java PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 2cd65cb hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 8cd594b hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java fc40195 hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseKeyFactory.java PRE-CREATION hbase-handler/src/test/queries/positive/hbase_custom_key.q PRE-CREATION hbase-handler/src/test/results/positive/hbase_custom_key.q.out PRE-CREATION itests/util/pom.xml 9885c53 serde/src/java/org/apache/hadoop/hive/serde2/StructObject.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/StructObjectBaseInspector.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStructBase.java 1fd6853 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyObject.java 10f4c05 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyObjectBase.java 3334dff serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java 8a1ea46 serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazySimpleStructObjectInspector.java 8a5386a serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryObject.java 598683f serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517 Diff: https://reviews.apache.org/r/18122/diff/ Testing --- Thanks, Swarnim Kulkarni
[jira] [Commented] (HIVE-6411) Support more generic way of using composite key for HBaseHandler
[
https://issues.apache.org/jira/browse/HIVE-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901558#comment-13901558
]
Swarnim Kulkarni commented on HIVE-6411:
[~navis] [~brocknoland] I created a RB review here to make it easier to look at
the diff. I'll start digging into the changes.
[1] https://reviews.apache.org/r/18122/
Support more generic way of using composite key for HBaseHandler
Key: HIVE-6411
URL: https://issues.apache.org/jira/browse/HIVE-6411
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-6411.1.patch.txt
HIVE-2599 introduced using custom object for the row key. But it forces key
objects to extend HBaseCompositeKey, which is again extension of LazyStruct.
If user provides proper Object and OI, we can replace internal key and keyOI
with those.
Initial implementation is based on factory interface.
{code}
public interface HBaseKeyFactory {
void init(SerDeParameters parameters, Properties properties) throws
SerDeException;
ObjectInspector createObjectInspector(TypeInfo type) throws SerDeException;
LazyObjectBase createObject(ObjectInspector inspector) throws
SerDeException;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6386) sql std auth - database should have an owner
[
https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901571#comment-13901571
]
Hive QA commented on HIVE-6386:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628894/HIVE-6386.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5100 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1322/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1322/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628894
sql std auth - database should have an owner
Key: HIVE-6386
URL: https://issues.apache.org/jira/browse/HIVE-6386
Project: Hive
Issue Type: Sub-task
Components: Authorization, Metastore
Reporter: Thejas M Nair
Assignee: Ashutosh Chauhan
Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.patch
Database in metastore does not have owner associated with it. Database owner
is needed for sql std authorization rules.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: Review Request 18103: Adds owner for DB.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/#review34499 --- trunk/metastore/if/hive_metastore.thrift https://reviews.apache.org/r/18103/#comment64593 There is a PrincipalType which used in other metastore apis (see grant_role for example). I think it is better to be consistent and use that as the datatype for ownerType. If we reuse PrincipalType, then we don't need to move HivePrincipal class to the common package. - Thejas Nair On Feb. 13, 2014, 11:24 p.m., Ashutosh Chauhan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/ --- (Updated Feb. 13, 2014, 11:24 p.m.) Review request for hive. Bugs: HIVE-6386 https://issues.apache.org/jira/browse/HIVE-6386 Repository: hive Description --- Adds owner for DB. Diffs - trunk/common/src/java/org/apache/hive/common/HivePrincipal.java PRE-CREATION trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1568122 trunk/metastore/if/hive_metastore.thrift 1568122 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 1568122 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 1568122 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 1568122 trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php 1568122 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 1568122 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1568122 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1568122 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1568122 trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java 1568122 trunk/metastore/src/model/package.jdo 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/AuthorizationUtils.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAccessController.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizer.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveAuthorizerImpl.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrincipal.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HivePrivilegeInfo.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/GrantPrivAuthUtils.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/RevokePrivAuthUtils.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLAuthorizationUtils.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAccessController.java 1568122 trunk/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidator.java 1568122 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1568122 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1568122 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1568122 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 1568122 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 1568122 trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 1568122 trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1568122 trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1568122 Diff: https://reviews.apache.org/r/18103/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
Re: Review Request 18122: Support more generic way of using composite key for HBaseHandler
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18122/#review34498 --- hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java https://reviews.apache.org/r/18122/#comment64592 Javadoc on this factory class would be very helpful for consumers. serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyObjectBase.java https://reviews.apache.org/r/18122/#comment64594 Are we not breaking our consumers with this non-passive change? If we want to go this route, may be we should deprecate out the existing abstract class. serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java https://reviews.apache.org/r/18122/#comment64595 Nit: Could change this to SerDeException to catch that specific checked exception - Swarnim Kulkarni On Feb. 14, 2014, 3:19 p.m., Swarnim Kulkarni wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18122/ --- (Updated Feb. 14, 2014, 3:19 p.m.) Review request for hive, Brock Noland, Navis Ryu, and Swarnim Kulkarni. Bugs: HIVE-6411 https://issues.apache.org/jira/browse/HIVE-6411 Repository: hive-git Description --- Refer to description on HIVE-6411. Diffs - hbase-handler/pom.xml 7c3524c hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java 5008f15 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyFactory.java PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseLazyObjectFactory.java PRE-CREATION hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 2cd65cb hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 8cd594b hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java fc40195 hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseKeyFactory.java PRE-CREATION hbase-handler/src/test/queries/positive/hbase_custom_key.q PRE-CREATION hbase-handler/src/test/results/positive/hbase_custom_key.q.out PRE-CREATION itests/util/pom.xml 9885c53 serde/src/java/org/apache/hadoop/hive/serde2/StructObject.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/StructObjectBaseInspector.java PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStructBase.java 1fd6853 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyObject.java 10f4c05 serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyObjectBase.java 3334dff serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java 8a1ea46 serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazySimpleStructObjectInspector.java 8a5386a serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryObject.java 598683f serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryStruct.java caf3517 Diff: https://reviews.apache.org/r/18122/diff/ Testing --- Thanks, Swarnim Kulkarni
[jira] [Commented] (HIVE-6400) Add Apache Rat plugin to pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901629#comment-13901629 ] Ashutosh Chauhan commented on HIVE-6400: +1 Add Apache Rat plugin to pom.xml Key: HIVE-6400 URL: https://issues.apache.org/jira/browse/HIVE-6400 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-6400.1.patch So we can generate the Release Audit report. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6250) sql std auth - view authorization should not underlying table. More tests and fixes.
[ https://issues.apache.org/jira/browse/HIVE-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6250: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thejas! sql std auth - view authorization should not underlying table. More tests and fixes. Key: HIVE-6250 URL: https://issues.apache.org/jira/browse/HIVE-6250 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.13.0 Attachments: HIVE-6250.1.patch, HIVE-6250.2.patch Original Estimate: 24h Remaining Estimate: 24h This patch adds more tests for table and view authorization and also fixes a number of issues found during testing - - View authorization should happen on only on the view, and not the underlying table (Change in ReadEntity to indicate if it is a direct/indirect dependency) - table owner in metadata should be the user as per SessionState authentication provider - added utility function for finding the session state authentication provider user - authorization should be based on current roles - admin user should have all permissions - error message improvements -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: A question about the derivation of intermediate sum field for decimal average aggregates
Remus, Thanks for looking into this. You're right that sum() result doesn't increase the scale, but have you seen that sum UDF returns wrong scale? As to the implementation of avg UDF, the object inspector for sum field is initialized with a scale + 4, which might not be necessary, but perhaps harmless. The same object inspector is also used for the average result, which gives correct type. I guess it's possible to separate this into two object inspectors, one for sum field and one for the avg result, but the difference might be subtle and questionable. This is because the data may not comply to the metadata specified for Hive tables. Thus, I'm not sure if truncating data before it's summed if the right behavior. Do you have a use case that suggests one is better than the other? --Xuefu On Fri, Feb 14, 2014 at 3:55 AM, Remus Rusanu rem...@microsoft.com wrote: Hi, With HIVE-5872 the intermediate sum field for decimal aggregates was changed to increase scale by 4. I understand the reasoning for having accurate precision/scale for the aggregate output. However, for the intermediate sum field of AVG, I believe we should increase precision w/o increasing scale. The sum can grow large, but cannot increase digits in the fractional part, so we should increase the precision of the sum, but not the scale. When sum is divided by count to get the average on the reduce side then we should indeed project value with higher scale. Opinions? Thanks, ~Remus
[jira] [Commented] (HIVE-5759) Implement vectorized support for COALESCE conditional expression
[
https://issues.apache.org/jira/browse/HIVE-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901676#comment-13901676
]
Hive QA commented on HIVE-5759:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628920/HIVE-5759.2.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5096 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.common.type.TestDecimal128.testHighPrecisionDecimal128Multiply
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1323/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1323/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628920
Implement vectorized support for COALESCE conditional expression
Key: HIVE-5759
URL: https://issues.apache.org/jira/browse/HIVE-5759
Project: Hive
Issue Type: Sub-task
Reporter: Eric Hanson
Assignee: Jitendra Nath Pandey
Attachments: HIVE-5759.1.patch, HIVE-5759.2.patch
Implement full, end-to-end support for COALESCE in vectorized mode, including
new VectorExpression class(es), VectorizationContext translation to a
VectorExpression, and unit tests for these, as well as end-to-end ad hoc
testing. An end-to-end .q test is recommended.
This is lower priority than IF and CASE but it is still a fairly popular
expression.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
Re: Review Request 17899: HIVE-5998 Add vectorized reader for Parquet files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17899/#review34512 --- Seems fine to me! Someone familiar with vectorization should probably do the +1. - Brock Noland On Feb. 14, 2014, 11:07 a.m., Remus Rusanu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17899/ --- (Updated Feb. 14, 2014, 11:07 a.m.) Review request for hive, Brock Noland, Eric Hanson, and Jitendra Pandey. Bugs: HIVE-5998 https://issues.apache.org/jira/browse/HIVE-5998 Repository: hive-git Description --- Implementation is straight forward and very simple, but offers all benefits of vectorization possible with a 'shallow' vectorized reader (ie. one that doe not got into parquet-mr project changes). the only complication arrised because of discrepancies between the object inspector seen by the inputformat and the actual output provided by the Parquet readers (eg. OI declares 'byte' primitives but the Parquet reader outputs IntWritable). I had to create a just-in-time VectorColumnAssigner colelciton base don whatever writers the Parquet record reader provides. It is assumed the reader does not change it's output during the iteration. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java d1a75df ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java 0b504de ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java d409d44 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java d3412df ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java PRE-CREATION ql/src/test/queries/clientpositive/vectorized_parquet.q PRE-CREATION ql/src/test/results/clientpositive/vectorized_parquet.q.out PRE-CREATION Diff: https://reviews.apache.org/r/17899/diff/ Testing --- Manually tested. New query .q added. Thanks, Remus Rusanu
[jira] [Commented] (HIVE-5176) Wincompat : Changes for allowing various path compatibilities with Windows
[ https://issues.apache.org/jira/browse/HIVE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901732#comment-13901732 ] Sushanth Sowmyan commented on HIVE-5176: Actually, never mind - it doesn't even patch now, those files have moved/been refactored. Wincompat : Changes for allowing various path compatibilities with Windows -- Key: HIVE-5176 URL: https://issues.apache.org/jira/browse/HIVE-5176 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5176.2.patch, HIVE-5176.patch We need to make certain changes across the board to allow us to read/parse windows paths. Some are escaping changes, some are being strict about how we read paths (through URL.encode/decode, etc) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5176) Wincompat : Changes for allowing various path compatibilities with Windows
[ https://issues.apache.org/jira/browse/HIVE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901731#comment-13901731 ] Sushanth Sowmyan commented on HIVE-5176: I'm going to mark this as patch-unavailable and then set available again, so that a newer run of the precommit tests kick off again on this to see if it's fresh enough. Wincompat : Changes for allowing various path compatibilities with Windows -- Key: HIVE-5176 URL: https://issues.apache.org/jira/browse/HIVE-5176 Project: Hive Issue Type: Sub-task Components: Windows Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5176.2.patch, HIVE-5176.patch We need to make certain changes across the board to allow us to read/parse windows paths. Some are escaping changes, some are being strict about how we read paths (through URL.encode/decode, etc) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5759) Implement vectorized support for COALESCE conditional expression
[ https://issues.apache.org/jira/browse/HIVE-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901749#comment-13901749 ] Eric Hanson commented on HIVE-5759: --- +1 Also, the failure in testHighPrecisionDecimal128Multiply is external to this patch. Implement vectorized support for COALESCE conditional expression Key: HIVE-5759 URL: https://issues.apache.org/jira/browse/HIVE-5759 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Jitendra Nath Pandey Attachments: HIVE-5759.1.patch, HIVE-5759.2.patch Implement full, end-to-end support for COALESCE in vectorized mode, including new VectorExpression class(es), VectorizationContext translation to a VectorExpression, and unit tests for these, as well as end-to-end ad hoc testing. An end-to-end .q test is recommended. This is lower priority than IF and CASE but it is still a fairly popular expression. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6330) Metastore support for permanent UDFs
[ https://issues.apache.org/jira/browse/HIVE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6330: - Status: Open (was: Patch Available) TestRemoteHiveMetaStore.testSimpleFunction() is failing because HiveMetaStore actually overwrites the create time when doing the create function request, so we shouldn't be testing the create time that we pass in when creating a function. Metastore support for permanent UDFs Key: HIVE-6330 URL: https://issues.apache.org/jira/browse/HIVE-6330 Project: Hive Issue Type: Sub-task Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6330.1.patch, HIVE-6330.2.patch, HIVE-6330.3.patch, HIVE-6330.4.patch, HIVE-6330.5.patch Allow CREATE FUNCTION to add metastore entry for the created function, so that it only needs to be added to Hive once. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6379) webhcat is not operational due to Slf4j conflicts
[ https://issues.apache.org/jira/browse/HIVE-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901770#comment-13901770 ] Eugene Koifman commented on HIVE-6379: -- most likely we should use the hadoop-1/hadoop-2 build profiles to make sure webhcat uses the version of jetty that matches (at least approximately) the one used in hadoop webhcat is not operational due to Slf4j conflicts - Key: HIVE-6379 URL: https://issues.apache.org/jira/browse/HIVE-6379 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman 1. build current trunk: mvn clean install -Pdist,hadoop-1 -DskipTests 2. /Users/ekoifman/dev/hive/packaging/target/apache-hive-0.13.0-SNAPSHOT-bin/apache-hive-0.13.0-SNAPSHOT-bin/hcatalog 3. ./sbin/webhcat_server.sh start 4. http://localhost:50111/templeton/v1/ddl/database?user.name=ekoifman webhcat-console-error.log has log4j:WARN No appenders could be found for logger (org.apache.hive.hcatalog.templeton.AppConfig). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception in thread main java.lang.NoSuchMethodError: org.slf4j.helpers.MessageFormatter.arrayFormat(Ljava/lang/String;[Ljava/lang/Object;)Lorg/slf4j/helpers/FormattingTuple; at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:613) at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:200) at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:94) at org.eclipse.jetty.util.component.Container.add(Container.java:201) at org.eclipse.jetty.util.component.Container.update(Container.java:164) at org.eclipse.jetty.util.component.Container.update(Container.java:106) at org.eclipse.jetty.server.Server.setConnectors(Server.java:184) at org.eclipse.jetty.server.Server.init(Server.java:97) at org.apache.hive.hcatalog.templeton.Main.runServer(Main.java:162) at org.apache.hive.hcatalog.templeton.Main.run(Main.java:114) at org.apache.hive.hcatalog.templeton.Main.main(Main.java:259) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:160) webhcat-console-error.log (END) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6330) Metastore support for permanent UDFs
[ https://issues.apache.org/jira/browse/HIVE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6330: - Attachment: HIVE-6330.6.patch patch v6 - fix TestHiveMetaStore Metastore support for permanent UDFs Key: HIVE-6330 URL: https://issues.apache.org/jira/browse/HIVE-6330 Project: Hive Issue Type: Sub-task Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6330.1.patch, HIVE-6330.2.patch, HIVE-6330.3.patch, HIVE-6330.4.patch, HIVE-6330.5.patch, HIVE-6330.6.patch Allow CREATE FUNCTION to add metastore entry for the created function, so that it only needs to be added to Hive once. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6330) Metastore support for permanent UDFs
[ https://issues.apache.org/jira/browse/HIVE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6330: - Status: Patch Available (was: Open) Metastore support for permanent UDFs Key: HIVE-6330 URL: https://issues.apache.org/jira/browse/HIVE-6330 Project: Hive Issue Type: Sub-task Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6330.1.patch, HIVE-6330.2.patch, HIVE-6330.3.patch, HIVE-6330.4.patch, HIVE-6330.5.patch, HIVE-6330.6.patch Allow CREATE FUNCTION to add metastore entry for the created function, so that it only needs to be added to Hive once. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6415) Disallow transform clause in sql std authorization mode
[
https://issues.apache.org/jira/browse/HIVE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901812#comment-13901812
]
Hive QA commented on HIVE-6415:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628897/HIVE-6415.2.patch
{color:green}SUCCESS:{color} +1 5120 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1324/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1324/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628897
Disallow transform clause in sql std authorization mode
---
Key: HIVE-6415
URL: https://issues.apache.org/jira/browse/HIVE-6415
Project: Hive
Issue Type: Sub-task
Components: Authorization
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-6415.1.patch, HIVE-6415.2.patch, HIVE-6415.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6380) Specify jars/files when creating permanent UDFs
[ https://issues.apache.org/jira/browse/HIVE-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6380: - Attachment: HIVE-6380.2.patch updated patch, including updated changes from HIVE-6330. Specify jars/files when creating permanent UDFs --- Key: HIVE-6380 URL: https://issues.apache.org/jira/browse/HIVE-6380 Project: Hive Issue Type: Sub-task Components: UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6380.1.patch, HIVE-6380.2.patch Need a way for a permanent UDF to reference jars/files. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 18094: HIVE-6339: Implement new JDK7 schema management APIs in java.sql.Connection
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18094/ --- (Updated Feb. 14, 2014, 7:51 p.m.) Review request for hive. Changes --- Updated the patch to close the statement and resultset in the schema functions. Bugs: HIVE-6339 https://issues.apache.org/jira/browse/HIVE-6339 Repository: hive-git Description --- Implement new JDK7 schema management APIs in java.sql.Connection. Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java e68a5a9 jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 13fc19b Diff: https://reviews.apache.org/r/18094/diff/ Testing --- Added new test case. Thanks, Prasad Mujumdar
[jira] [Updated] (HIVE-6339) Implement new JDK7 schema management APIs in java.sql.Connection
[
https://issues.apache.org/jira/browse/HIVE-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar updated HIVE-6339:
--
Attachment: HIVE-6339.3.patch
The test failure looks unrelated to the patch.
Adding a minor change to close the statements in the new schema functions to
release the resources early.
Implement new JDK7 schema management APIs in java.sql.Connection
-
Key: HIVE-6339
URL: https://issues.apache.org/jira/browse/HIVE-6339
Project: Hive
Issue Type: Improvement
Components: JDBC
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-6339.1.patch, HIVE-6339.2.patch, HIVE-6339.3.patch
JDK7 has added a few metadata methods in
[java.sql.Conntion|http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html]
{noformat}
getSchema()
setSchema()
getCatalog()
setCatalog()
{noformat}
Currently Hive JDBC just has stub implementation for all these methods throws
unsupported exception. This needs to be fixed.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
RE: Review Request 17899: HIVE-5998 Add vectorized reader for Parquet files
Hey Jitendra, can you double check the Parquet vectorized record reader is OK with regard to partitioning? Thanks, ~Remus From: Brock Noland [mailto:nore...@reviews.apache.org] On Behalf Of Brock Noland Sent: Friday, February 14, 2014 7:50 PM To: Jitendra Pandey; Eric Hanson (BIG DATA); Brock Noland Cc: Remus Rusanu; hive Subject: Re: Review Request 17899: HIVE-5998 Add vectorized reader for Parquet files This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17899/ Seems fine to me! Someone familiar with vectorization should probably do the +1. - Brock Noland On February 14th, 2014, 11:07 a.m. UTC, Remus Rusanu wrote: Review request for hive, Brock Noland, Eric Hanson, and Jitendra Pandey. By Remus Rusanu. Updated Feb. 14, 2014, 11:07 a.m. Bugs: HIVE-5998https://issues.apache.org/jira/browse/HIVE-5998 Repository: hive-git Description Implementation is straight forward and very simple, but offers all benefits of vectorization possible with a 'shallow' vectorized reader (ie. one that doe not got into parquet-mr project changes). the only complication arrised because of discrepancies between the object inspector seen by the inputformat and the actual output provided by the Parquet readers (eg. OI declares 'byte' primitives but the Parquet reader outputs IntWritable). I had to create a just-in-time VectorColumnAssigner colelciton base don whatever writers the Parquet record reader provides. It is assumed the reader does not change it's output during the iteration. Testing Manually tested. New query .q added. Diffs * ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java (d1a75df) * ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java (0b504de) * ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java (d409d44) * ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java (d3412df) * ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java (PRE-CREATION) * ql/src/test/queries/clientpositive/vectorized_parquet.q (PRE-CREATION) * ql/src/test/results/clientpositive/vectorized_parquet.q.out (PRE-CREATION) View Diffhttps://reviews.apache.org/r/17899/diff/
Re: Review Request 18094: HIVE-6339: Implement new JDK7 schema management APIs in java.sql.Connection
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18094/#review34541 --- itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java https://reviews.apache.org/r/18094/#comment64651 It would be nice if these tab/spaces can be removed. Same below. - Xuefu Zhang On Feb. 14, 2014, 7:51 p.m., Prasad Mujumdar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18094/ --- (Updated Feb. 14, 2014, 7:51 p.m.) Review request for hive. Bugs: HIVE-6339 https://issues.apache.org/jira/browse/HIVE-6339 Repository: hive-git Description --- Implement new JDK7 schema management APIs in java.sql.Connection. Diffs - itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java e68a5a9 jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 13fc19b Diff: https://reviews.apache.org/r/18094/diff/ Testing --- Added new test case. Thanks, Prasad Mujumdar
Re: Review Request 18143: HIVE-6046: add UDF for converting date time from one presentation to another
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18143/ --- (Updated Feb. 14, 2014, 8:18 p.m.) Review request for hive. Bugs: HIVE-6046 https://issues.apache.org/jira/browse/HIVE-6046 Repository: hive-git Description --- Created this review request on behalf the JIRA owner. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 92fee9b ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFormatDate.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFFormatDate.java PRE-CREATION ql/src/test/queries/clientpositive/udf_format_date.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 3400ac2 ql/src/test/results/clientpositive/udf_format_date.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18143/diff/ Testing --- Thanks, Xuefu Zhang
Review Request 18143: HIVE-6046: add UDF for converting date time from one presentation to another
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18143/ --- Review request for hive. Bugs: HIVE-6046 https://issues.apache.org/jira/browse/HIVE-6046 Repository: hive-git Description --- Created this review request on behalf the JIRA owner. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 92fee9b ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFormatDate.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFFormatDate.java PRE-CREATION ql/src/test/queries/clientpositive/udf_format_date.q PRE-CREATION ql/src/test/results/clientpositive/show_functions.q.out 3400ac2 ql/src/test/results/clientpositive/udf_format_date.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18143/diff/ Testing --- Thanks, Xuefu Zhang
[jira] [Updated] (HIVE-6046) add UDF for converting date time from one presentation to another
[
https://issues.apache.org/jira/browse/HIVE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6046:
--
Attachment: HIVE-6046.patch
Regenerated patch.
add UDF for converting date time from one presentation to another
--
Key: HIVE-6046
URL: https://issues.apache.org/jira/browse/HIVE-6046
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.13.0
Reporter: Kostiantyn Kudriavtsev
Assignee: Kostiantyn Kudriavtsev
Attachments: HIVE-6046.patch, Hive-6046.patch
it'd be nice to have function for converting datetime to different formats,
for example:
format_date('2013-12-12 00:00:00.0', '-MM-dd HH:mm:ss.S', '/MM/dd')
There are two signatures to facilitate further using:
format_date(datetime, fromFormat, toFormat)
format_date(timestamp, toFormat)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6406) Introduce immutable-table table property and if set, disallow insert-into
[ https://issues.apache.org/jira/browse/HIVE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901890#comment-13901890 ] Sushanth Sowmyan commented on HIVE-6406: [~thejas]/[~ashutoshc]/[~brocknoland] , could I please get a review of this patch? The error reported above is not connected to this patch. Introduce immutable-table table property and if set, disallow insert-into - Key: HIVE-6406 URL: https://issues.apache.org/jira/browse/HIVE-6406 Project: Hive Issue Type: Sub-task Components: HCatalog, Metastore, Query Processor, Thrift API Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6406.2.patch, HIVE-6406.patch As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow INSERT INTO into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5504) OrcOutputFormat honors compression properties only from within hive
[ https://issues.apache.org/jira/browse/HIVE-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901888#comment-13901888 ] Sushanth Sowmyan commented on HIVE-5504: [~thejas]/[~ashutoshc] , could I please get a review of this patch? OrcOutputFormat honors compression properties only from within hive - Key: HIVE-5504 URL: https://issues.apache.org/jira/browse/HIVE-5504 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0, 0.12.0 Reporter: Venkat Ranganathan Assignee: Sushanth Sowmyan Attachments: HIVE-5504.patch When we import data into a HCatalog table created with the following storage description .. stored as orc tblproperties (orc.compress=SNAPPY) the resultant orc file still uses the default zlib compression It looks like HCatOutputFormat is ignoring the tblproperties specified. show tblproperties shows that the table indeed has the properties properly saved. An insert/select into the table has the resulting orc file honor the tbl property. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6400) Add Apache Rat plugin to pom.xml
[
https://issues.apache.org/jira/browse/HIVE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901935#comment-13901935
]
Hive QA commented on HIVE-6400:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628949/HIVE-6400.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5119 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_quotedid_smb
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1325/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1325/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628949
Add Apache Rat plugin to pom.xml
Key: HIVE-6400
URL: https://issues.apache.org/jira/browse/HIVE-6400
Project: Hive
Issue Type: Bug
Reporter: Harish Butani
Assignee: Harish Butani
Attachments: HIVE-6400.1.patch
So we can generate the Release Audit report.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6046) add UDF for converting date time from one presentation to another
[
https://issues.apache.org/jira/browse/HIVE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901937#comment-13901937
]
Xuefu Zhang commented on HIVE-6046:
---
[~kostiantyn] Thanks for working on this. The code changes seem fine, but I
have a couple of high-level questions:
1. The UDF in the patch takes only a date in a string form, which seems too
restrictive as Hive has a the data type of Date. Hive should be able to output
the date to any format the user specifies. The udf isn't able to achieve this.
2. The name of the UDF, format_date is questionable, because major DBs provide
date_format() that does a different thing. This can cause unnecessary confusion.
Thus, to have a bigger usage, I'd suggest the following:
1. provide a UDF, such as to_date(string dateString, String format), that
converts a given date in the given format to a Date date. to_date is also
available in Oracle, while mysql has STR_TO_DATE.
2. provide UDF, date_format(Date date, String format) as what Oracle or MySQL
has.
With these two, you should be able to achieve what you want while retaining
bigger flexibility and usage.
add UDF for converting date time from one presentation to another
--
Key: HIVE-6046
URL: https://issues.apache.org/jira/browse/HIVE-6046
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.13.0
Reporter: Kostiantyn Kudriavtsev
Assignee: Kostiantyn Kudriavtsev
Attachments: HIVE-6046.patch, Hive-6046.patch
it'd be nice to have function for converting datetime to different formats,
for example:
format_date('2013-12-12 00:00:00.0', '-MM-dd HH:mm:ss.S', '/MM/dd')
There are two signatures to facilitate further using:
format_date(datetime, fromFormat, toFormat)
format_date(timestamp, toFormat)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6046) add UDF for converting date time from one presentation to another
[
https://issues.apache.org/jira/browse/HIVE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6046:
--
Status: Open (was: Patch Available)
add UDF for converting date time from one presentation to another
--
Key: HIVE-6046
URL: https://issues.apache.org/jira/browse/HIVE-6046
Project: Hive
Issue Type: New Feature
Components: UDF
Affects Versions: 0.13.0
Reporter: Kostiantyn Kudriavtsev
Assignee: Kostiantyn Kudriavtsev
Attachments: HIVE-6046.patch, Hive-6046.patch
it'd be nice to have function for converting datetime to different formats,
for example:
format_date('2013-12-12 00:00:00.0', '-MM-dd HH:mm:ss.S', '/MM/dd')
There are two signatures to facilitate further using:
format_date(datetime, fromFormat, toFormat)
format_date(timestamp, toFormat)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6403) uncorrelated subquery is failing with auto.convert.join=true
[
https://issues.apache.org/jira/browse/HIVE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harish Butani updated HIVE-6403:
Assignee: Navis (was: Harish Butani)
uncorrelated subquery is failing with auto.convert.join=true
Key: HIVE-6403
URL: https://issues.apache.org/jira/browse/HIVE-6403
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-6403.1.patch, HIVE-6403.2.patch, navis.patch,
navis2.patch
Fixing HIVE-5690, I've found query in subquery_multiinsert.q is not working
with hive.auto.convert.join=true
{noformat}
set hive.auto.convert.join=true;
hive explain
from src b
INSERT OVERWRITE TABLE src_4
select *
where b.key in
(select a.key
from src a
where b.value = a.value and a.key '9'
)
INSERT OVERWRITE TABLE src_5
select *
where b.key not in ( select key from src s1 where s1.key '2')
order by key
;
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinLocalWork(MapJoinProcessor.java:149)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genLocalWorkForMapJoin(MapJoinProcessor.java:256)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinOpAndLocalWork(MapJoinProcessor.java:248)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.convertTaskToMapJoinTask(CommonJoinTaskDispatcher.java:191)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:481)
at
org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
at
org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:100)
at
org.apache.hadoop.hive.ql.parse.MapReduceCompiler.optimizeTaskPlan(MapReduceCompiler.java:290)
at
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:216)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9167)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:64)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:346)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1056)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1099)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:992)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:687)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
org.apache.hadoop.hive.ql.parse.SemanticException: Failed to generate new
mapJoin operator by exception : Index: 0, Size: 0
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genLocalWorkForMapJoin(MapJoinProcessor.java:266)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinOpAndLocalWork(MapJoinProcessor.java:248)
at
[jira] [Commented] (HIVE-6403) uncorrelated subquery is failing with auto.convert.join=true
[
https://issues.apache.org/jira/browse/HIVE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901947#comment-13901947
]
Harish Butani commented on HIVE-6403:
-
[~navis] i went through the patch; I think I understand the reason for the
change. Couple of questions:
- i don't see a union e.g. Can an e.g. be added that exercises the map-side
union case.
- i see several plans changed in .out files to scan src1 in place of src2. But
the check at CommonJoinTaskDispatcher::cannotConvert line 447, should favor the
right alias as the big table. So I don't understand why these plans are changed
with your patch. For my understanding, can you please explain this change.
Assigning the jira to you; you have made most of the changes, my contribution
is very tiny; and I am just becoming familiar with the pieces you have touched
in this patch.
uncorrelated subquery is failing with auto.convert.join=true
Key: HIVE-6403
URL: https://issues.apache.org/jira/browse/HIVE-6403
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Harish Butani
Attachments: HIVE-6403.1.patch, HIVE-6403.2.patch, navis.patch,
navis2.patch
Fixing HIVE-5690, I've found query in subquery_multiinsert.q is not working
with hive.auto.convert.join=true
{noformat}
set hive.auto.convert.join=true;
hive explain
from src b
INSERT OVERWRITE TABLE src_4
select *
where b.key in
(select a.key
from src a
where b.value = a.value and a.key '9'
)
INSERT OVERWRITE TABLE src_5
select *
where b.key not in ( select key from src s1 where s1.key '2')
order by key
;
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinLocalWork(MapJoinProcessor.java:149)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genLocalWorkForMapJoin(MapJoinProcessor.java:256)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.genMapJoinOpAndLocalWork(MapJoinProcessor.java:248)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.convertTaskToMapJoinTask(CommonJoinTaskDispatcher.java:191)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:481)
at
org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
at
org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
at
org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
at
org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:100)
at
org.apache.hadoop.hive.ql.parse.MapReduceCompiler.optimizeTaskPlan(MapReduceCompiler.java:290)
at
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:216)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9167)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:64)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:346)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1056)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1099)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:992)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:687)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
Re: Review Request 18094: HIVE-6339: Implement new JDK7 schema management APIs in java.sql.Connection
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18094/ --- (Updated Feb. 14, 2014, 9:41 p.m.) Review request for hive. Changes --- Fixed whitespaces Bugs: HIVE-6339 https://issues.apache.org/jira/browse/HIVE-6339 Repository: hive-git Description --- Implement new JDK7 schema management APIs in java.sql.Connection. Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java e68a5a9 jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 13fc19b Diff: https://reviews.apache.org/r/18094/diff/ Testing --- Added new test case. Thanks, Prasad Mujumdar
[jira] [Updated] (HIVE-6339) Implement new JDK7 schema management APIs in java.sql.Connection
[
https://issues.apache.org/jira/browse/HIVE-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar updated HIVE-6339:
--
Attachment: HIVE-6339.4.patch
Implement new JDK7 schema management APIs in java.sql.Connection
-
Key: HIVE-6339
URL: https://issues.apache.org/jira/browse/HIVE-6339
Project: Hive
Issue Type: Improvement
Components: JDBC
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-6339.1.patch, HIVE-6339.2.patch, HIVE-6339.3.patch,
HIVE-6339.4.patch
JDK7 has added a few metadata methods in
[java.sql.Conntion|http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html]
{noformat}
getSchema()
setSchema()
getCatalog()
setCatalog()
{noformat}
Currently Hive JDBC just has stub implementation for all these methods throws
unsupported exception. This needs to be fixed.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6339) Implement new JDK7 schema management APIs in java.sql.Connection
[
https://issues.apache.org/jira/browse/HIVE-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasad Mujumdar updated HIVE-6339:
--
Attachment: (was: HIVE-6339.3.patch)
Implement new JDK7 schema management APIs in java.sql.Connection
-
Key: HIVE-6339
URL: https://issues.apache.org/jira/browse/HIVE-6339
Project: Hive
Issue Type: Improvement
Components: JDBC
Affects Versions: 0.13.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-6339.1.patch, HIVE-6339.2.patch, HIVE-6339.4.patch
JDK7 has added a few metadata methods in
[java.sql.Conntion|http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html]
{noformat}
getSchema()
setSchema()
getCatalog()
setCatalog()
{noformat}
Currently Hive JDBC just has stub implementation for all these methods throws
unsupported exception. This needs to be fixed.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6406) Introduce immutable-table table property and if set, disallow insert-into
[
https://issues.apache.org/jira/browse/HIVE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901988#comment-13901988
]
Lefty Leverenz commented on HIVE-6406:
--
Here's a Valentine nit for you in ErrorMsg.java:
bq. INSERT_INTO_IMMUTABLE_TABLE(10256, Inserting into an non-empty immutable
table is not allowed),
... should be a non-empty (also fix in insert_into5.q.out
insert_into6.q.out).
And another in hive_metastore.thrift:
{quote}
+// Whether or not the table is considered immutable - immutable tables can
only be
+// overwritten or created if unpartitioned, or if partitioned, partitions
inside it
+// can only be overwritten or created. Immutability supports write-once and
replace
+// semantics, but not append.
{quote}
... immutable tables doesn't match inside it so change to an immutable
table (or them).
Introduce immutable-table table property and if set, disallow insert-into
-
Key: HIVE-6406
URL: https://issues.apache.org/jira/browse/HIVE-6406
Project: Hive
Issue Type: Sub-task
Components: HCatalog, Metastore, Query Processor, Thrift API
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Attachments: HIVE-6406.2.patch, HIVE-6406.patch
As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar
ways with regards to immutable tables, this is a companion task to introduce
the notion of an immutable table, wherein all tables are not immutable by
default, and have this be a table property. If this property is set for a
table, and we attempt to write to a table that already has data (or a
partition), disallow INSERT INTO into it from hive(if destination directory
is non-empty). This property being set will allow hive to mimic HCatalog's
current immutable-table property.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Assigned] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson reassigned HIVE-6399: - Assignee: Eric Hanson (was: Remus Rusanu) bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Eric Hanson Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901995#comment-13901995 ] Eric Hanson commented on HIVE-6399: --- Remus' patch is technically good. I have a question I'll raise with the PMC though about the comment about using the algorithm from BigInteger.multiplyToLen. For now I'm going to promote my original patch to get it in so we can get the bug failure out of trunk. bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Eric Hanson Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson updated HIVE-6399: -- Attachment: HIVE-6399.05.patch Promoting patch 02 to first position to get committed, now as 05. bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Eric Hanson Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.05.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6400) Add Apache Rat plugin to pom.xml
[ https://issues.apache.org/jira/browse/HIVE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6400: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Harish! Add Apache Rat plugin to pom.xml Key: HIVE-6400 URL: https://issues.apache.org/jira/browse/HIVE-6400 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.13.0 Attachments: HIVE-6400.1.patch So we can generate the Release Audit report. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6415) Disallow transform clause in sql std authorization mode
[ https://issues.apache.org/jira/browse/HIVE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6415: --- Resolution: Fixed Fix Version/s: 0.13.0 Status: Resolved (was: Patch Available) Committed to trunk. Disallow transform clause in sql std authorization mode --- Key: HIVE-6415 URL: https://issues.apache.org/jira/browse/HIVE-6415 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.13.0 Attachments: HIVE-6415.1.patch, HIVE-6415.2.patch, HIVE-6415.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
RE: A question about the derivation of intermediate sum field for decimal average aggregates
Hi Xuefu, I do not have any particular use case in mind. I've noticed the problem when I implemented the vectorized AVG for decimal, which must match your implementation (since we vectorized only the map side operator, it better produce the output expected by the reduce side...). I thought that since we alter the precission/scale for the result, we may as well alter it for the intermediate sum field. But if this complicates the use of object inspectors and introduces maintenance risks, then is probably not worth it. Thanks, ~Remus -Original Message- From: Xuefu Zhang [mailto:xzh...@cloudera.com] Sent: Friday, February 14, 2014 7:18 PM To: dev@hive.apache.org Cc: xu...@apache.org; Eric Hanson (BIG DATA) Subject: Re: A question about the derivation of intermediate sum field for decimal average aggregates Remus, Thanks for looking into this. You're right that sum() result doesn't increase the scale, but have you seen that sum UDF returns wrong scale? As to the implementation of avg UDF, the object inspector for sum field is initialized with a scale + 4, which might not be necessary, but perhaps harmless. The same object inspector is also used for the average result, which gives correct type. I guess it's possible to separate this into two object inspectors, one for sum field and one for the avg result, but the difference might be subtle and questionable. This is because the data may not comply to the metadata specified for Hive tables. Thus, I'm not sure if truncating data before it's summed if the right behavior. Do you have a use case that suggests one is better than the other? --Xuefu On Fri, Feb 14, 2014 at 3:55 AM, Remus Rusanu rem...@microsoft.com wrote: Hi, With HIVE-5872 the intermediate sum field for decimal aggregates was changed to increase scale by 4. I understand the reasoning for having accurate precision/scale for the aggregate output. However, for the intermediate sum field of AVG, I believe we should increase precision w/o increasing scale. The sum can grow large, but cannot increase digits in the fractional part, so we should increase the precision of the sum, but not the scale. When sum is divided by count to get the average on the reduce side then we should indeed project value with higher scale. Opinions? Thanks, ~Remus
[jira] [Updated] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6386: --- Status: Patch Available (was: Open) sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6386: --- Attachment: HIVE-6386.3.patch Incorporating Thejas feedback. sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6386: --- Status: Open (was: Patch Available) sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6415) Disallow transform clause in sql std authorization mode
[ https://issues.apache.org/jira/browse/HIVE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902024#comment-13902024 ] Lefty Leverenz commented on HIVE-6415: -- When SQL standard authorization mode is available, this needs to be documented in the wiki: * [Language Manual -- Transform |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Transform] * [Language Manual -- Authorization |https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Authorization] Disallow transform clause in sql std authorization mode --- Key: HIVE-6415 URL: https://issues.apache.org/jira/browse/HIVE-6415 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.13.0 Attachments: HIVE-6415.1.patch, HIVE-6415.2.patch, HIVE-6415.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 18103: Adds owner for DB.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/ --- (Updated Feb. 14, 2014, 10:18 p.m.) Review request for hive. Changes --- Incorporated Thejas feedback. Bugs: HIVE-6386 https://issues.apache.org/jira/browse/HIVE-6386 Repository: hive Description --- Adds owner for DB. Diffs (updated) - trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1568535 trunk/metastore/if/hive_metastore.thrift 1568535 trunk/metastore/scripts/upgrade/derby/016-HIVE-6386.derby.sql PRE-CREATION trunk/metastore/scripts/upgrade/mysql/016-HIVE-6386.mysql.sql PRE-CREATION trunk/metastore/scripts/upgrade/oracle/016-HIVE-6386.oracle.sql PRE-CREATION trunk/metastore/scripts/upgrade/postgres/016-HIVE-6386.postgres.sql PRE-CREATION trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1568535 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 1568535 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 1568535 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 1568535 trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php 1568535 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 1568535 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1568535 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1568535 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1568535 trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java 1568535 trunk/metastore/src/model/package.jdo 1568535 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1568535 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java 1568535 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java 1568535 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java 1568535 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1568535 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1568535 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1568535 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 1568535 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 1568535 trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 1568535 trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1568535 trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1568535 Diff: https://reviews.apache.org/r/18103/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
[jira] [Commented] (HIVE-6399) bug in high-precision Decimal128 multiply
[ https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902029#comment-13902029 ] Jitendra Nath Pandey commented on HIVE-6399: +1 for the latest patch. bug in high-precision Decimal128 multiply - Key: HIVE-6399 URL: https://issues.apache.org/jira/browse/HIVE-6399 Project: Hive Issue Type: Sub-task Components: Query Processor, Vectorization Reporter: Eric Hanson Assignee: Eric Hanson Labels: vectorization Fix For: 0.13.0 Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch, HIVE-6399.02.patch, HIVE-6399.05.patch, HIVE-6399.3.patch, HIVE-6399.4.patch For operation -605044214913338382 * 55269579109718297360 expected: -33440539101030154945490585226577271520 but was: -33440539021801992431226247633033321184 More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6386: --- Status: Open (was: Patch Available) sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6406) Introduce immutable-table table property and if set, disallow insert-into
[ https://issues.apache.org/jira/browse/HIVE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6406: --- Attachment: HIVE-6406.3.patch /doh. Thanks Lefty. :) Updated patch with fix. Introduce immutable-table table property and if set, disallow insert-into - Key: HIVE-6406 URL: https://issues.apache.org/jira/browse/HIVE-6406 Project: Hive Issue Type: Sub-task Components: HCatalog, Metastore, Query Processor, Thrift API Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6406.2.patch, HIVE-6406.3.patch, HIVE-6406.patch As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow INSERT INTO into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6406) Introduce immutable-table table property and if set, disallow insert-into
[ https://issues.apache.org/jira/browse/HIVE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6406: --- Attachment: (was: HIVE-6406.3.patch) Introduce immutable-table table property and if set, disallow insert-into - Key: HIVE-6406 URL: https://issues.apache.org/jira/browse/HIVE-6406 Project: Hive Issue Type: Sub-task Components: HCatalog, Metastore, Query Processor, Thrift API Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6406.2.patch, HIVE-6406.patch As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow INSERT INTO into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6406) Introduce immutable-table table property and if set, disallow insert-into
[ https://issues.apache.org/jira/browse/HIVE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6406: --- Attachment: HIVE-6406.3.patch Introduce immutable-table table property and if set, disallow insert-into - Key: HIVE-6406 URL: https://issues.apache.org/jira/browse/HIVE-6406 Project: Hive Issue Type: Sub-task Components: HCatalog, Metastore, Query Processor, Thrift API Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-6406.2.patch, HIVE-6406.3.patch, HIVE-6406.patch As part of HIVE-6405's attempt to make HCatalog and Hive behave in similar ways with regards to immutable tables, this is a companion task to introduce the notion of an immutable table, wherein all tables are not immutable by default, and have this be a table property. If this property is set for a table, and we attempt to write to a table that already has data (or a partition), disallow INSERT INTO into it from hive(if destination directory is non-empty). This property being set will allow hive to mimic HCatalog's current immutable-table property. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6332) HCatConstants Documentation needed
[ https://issues.apache.org/jira/browse/HIVE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902051#comment-13902051 ] Sushanth Sowmyan commented on HIVE-6332: Just commenting to note that I wasn't able to get to this last week or this, but I'll still definitely try to get this in before we fork for 0.13 HCatConstants Documentation needed -- Key: HIVE-6332 URL: https://issues.apache.org/jira/browse/HIVE-6332 Project: Hive Issue Type: Task Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan HCatConstants documentation is near non-existent, being defined only as comments in code for the various parameters. Given that a lot of api winds up being implemented as knobs that can be tweaked here, we should have a public facing doc for this. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6386: --- Attachment: HIVE-6386.4.patch sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.4.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: Review Request 18103: Adds owner for DB.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/ --- (Updated Feb. 14, 2014, 10:41 p.m.) Review request for hive. Changes --- Some minor fixes. Bugs: HIVE-6386 https://issues.apache.org/jira/browse/HIVE-6386 Repository: hive Description --- Adds owner for DB. Diffs (updated) - trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1568537 trunk/metastore/if/hive_metastore.thrift 1568537 trunk/metastore/scripts/upgrade/derby/016-HIVE-6386.derby.sql PRE-CREATION trunk/metastore/scripts/upgrade/mysql/016-HIVE-6386.mysql.sql PRE-CREATION trunk/metastore/scripts/upgrade/oracle/016-HIVE-6386.oracle.sql PRE-CREATION trunk/metastore/scripts/upgrade/postgres/016-HIVE-6386.postgres.sql PRE-CREATION trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1568537 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 1568537 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 1568537 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 1568537 trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php 1568537 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 1568537 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1568537 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1568537 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1568537 trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java 1568537 trunk/metastore/src/model/package.jdo 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java 1568537 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1568537 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 1568537 trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 1568537 trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1568537 trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1568537 Diff: https://reviews.apache.org/r/18103/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
Re: A question about the derivation of intermediate sum field for decimal average aggregates
Hi Remus, Thank you for the info. I now realized that it might be good to address this, as the object inspector for the sum field might be problematic because it has the same number of integer digits as the summed field, which may overflow easily. What about we have sum filed precision/scale as (p+10, s), which is consistent with UDF sum, and have (p+4, s+4) for the avg result only? If you agree, I can make the change. Thanks, Xuefu On Fri, Feb 14, 2014 at 2:09 PM, Remus Rusanu rem...@microsoft.com wrote: Hi Xuefu, I do not have any particular use case in mind. I've noticed the problem when I implemented the vectorized AVG for decimal, which must match your implementation (since we vectorized only the map side operator, it better produce the output expected by the reduce side...). I thought that since we alter the precission/scale for the result, we may as well alter it for the intermediate sum field. But if this complicates the use of object inspectors and introduces maintenance risks, then is probably not worth it. Thanks, ~Remus -Original Message- From: Xuefu Zhang [mailto:xzh...@cloudera.com] Sent: Friday, February 14, 2014 7:18 PM To: dev@hive.apache.org Cc: xu...@apache.org; Eric Hanson (BIG DATA) Subject: Re: A question about the derivation of intermediate sum field for decimal average aggregates Remus, Thanks for looking into this. You're right that sum() result doesn't increase the scale, but have you seen that sum UDF returns wrong scale? As to the implementation of avg UDF, the object inspector for sum field is initialized with a scale + 4, which might not be necessary, but perhaps harmless. The same object inspector is also used for the average result, which gives correct type. I guess it's possible to separate this into two object inspectors, one for sum field and one for the avg result, but the difference might be subtle and questionable. This is because the data may not comply to the metadata specified for Hive tables. Thus, I'm not sure if truncating data before it's summed if the right behavior. Do you have a use case that suggests one is better than the other? --Xuefu On Fri, Feb 14, 2014 at 3:55 AM, Remus Rusanu rem...@microsoft.com wrote: Hi, With HIVE-5872 the intermediate sum field for decimal aggregates was changed to increase scale by 4. I understand the reasoning for having accurate precision/scale for the aggregate output. However, for the intermediate sum field of AVG, I believe we should increase precision w/o increasing scale. The sum can grow large, but cannot increase digits in the fractional part, so we should increase the precision of the sum, but not the scale. When sum is divided by count to get the average on the reduce side then we should indeed project value with higher scale. Opinions? Thanks, ~Remus
[jira] [Commented] (HIVE-6037) Synchronize HiveConf with hive-default.xml.template and support show conf
[
https://issues.apache.org/jira/browse/HIVE-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902070#comment-13902070
]
Hive QA commented on HIVE-6037:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628958/HIVE-6037.9.patch.txt
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5120 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_conf
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_groupby2
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveOperationType.checkHiveOperationTypeMatch
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1329/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1329/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12628958
Synchronize HiveConf with hive-default.xml.template and support show conf
-
Key: HIVE-6037
URL: https://issues.apache.org/jira/browse/HIVE-6037
Project: Hive
Issue Type: Improvement
Components: Configuration
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: CHIVE-6037.3.patch.txt, HIVE-6037.1.patch.txt,
HIVE-6037.2.patch.txt, HIVE-6037.4.patch.txt, HIVE-6037.5.patch.txt,
HIVE-6037.6.patch.txt, HIVE-6037.7.patch.txt, HIVE-6037.8.patch.txt,
HIVE-6037.9.patch.txt
see HIVE-5879
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6203) Privileges of role granted indrectily to user is not applied
[
https://issues.apache.org/jira/browse/HIVE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902071#comment-13902071
]
Hive QA commented on HIVE-6203:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12628956/HIVE-6203.2.patch.txt
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1330/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1330/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1330/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'conf/hive-default.xml.template'
Reverted
'itests/hive-unit/src/test/java/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java'
Reverted
'itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java'
Reverted 'common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java'
Reverted 'common/src/test/org/apache/hadoop/hive/conf/TestHiveLogging.java'
Reverted
'common/src/test/org/apache/hadoop/hive/conf/TestHiveConfRestrictList.java'
Reverted 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
Reverted 'common/pom.xml'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/VariableSubstitution.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFile.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
common/src/java/org/apache/hadoop/hive/ant
common/src/java/org/apache/hadoop/hive/conf/Validator.java
common/src/java/org/apache/hive/common/util/SystemVariables.java contrib/target
service/target serde/target beeline/target odbc/target cli/target
ql/dependency-reduced-pom.xml ql/target
ql/src/test/results/clientpositive/show_conf.q.out
ql/src/test/queries/clientpositive/show_conf.q
ql/src/java/org/apache/hadoop/hive/ql/plan/ShowConfDesc.java
+ svn update
Upom.xml
Aql/src/test/queries/clientnegative/authorization_disallow_transform.q
Aql/src/test/results/clientnegative/authorization_disallow_transform.q.out
A
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/DisallowTransformHook.java
Uql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1568539.
Updated to revision 1568539.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0,
Re: Review Request 18103: Adds owner for DB.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/#review34553 --- Ship it! Ship It! - Thejas Nair On Feb. 14, 2014, 10:41 p.m., Ashutosh Chauhan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18103/ --- (Updated Feb. 14, 2014, 10:41 p.m.) Review request for hive. Bugs: HIVE-6386 https://issues.apache.org/jira/browse/HIVE-6386 Repository: hive Description --- Adds owner for DB. Diffs - trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1568537 trunk/metastore/if/hive_metastore.thrift 1568537 trunk/metastore/scripts/upgrade/derby/016-HIVE-6386.derby.sql PRE-CREATION trunk/metastore/scripts/upgrade/mysql/016-HIVE-6386.mysql.sql PRE-CREATION trunk/metastore/scripts/upgrade/oracle/016-HIVE-6386.oracle.sql PRE-CREATION trunk/metastore/scripts/upgrade/postgres/016-HIVE-6386.postgres.sql PRE-CREATION trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1568537 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 1568537 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 1568537 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 1568537 trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php 1568537 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 1568537 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1568537 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1568537 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1568537 trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java 1568537 trunk/metastore/src/model/package.jdo 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java 1568537 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java 1568537 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1568537 trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/test/ThriftTestObj.java 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/Complex.java 1568537 trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java 1568537 trunk/serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php 1568537 trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1568537 trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1568537 Diff: https://reviews.apache.org/r/18103/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
[jira] [Commented] (HIVE-6386) sql std auth - database should have an owner
[ https://issues.apache.org/jira/browse/HIVE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902076#comment-13902076 ] Thejas M Nair commented on HIVE-6386: - +1 sql std auth - database should have an owner Key: HIVE-6386 URL: https://issues.apache.org/jira/browse/HIVE-6386 Project: Hive Issue Type: Sub-task Components: Authorization, Metastore Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Attachments: HIVE-6386.1.patch, HIVE-6386.2.patch, HIVE-6386.3.patch, HIVE-6386.4.patch, HIVE-6386.patch Database in metastore does not have owner associated with it. Database owner is needed for sql std authorization rules. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6326) Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes
[ https://issues.apache.org/jira/browse/HIVE-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-6326: - Attachment: HIVE-6326.3.patch The earlier patch made split computation unnecessarily complicated. Thanks [~owen.omalley] for pointing out. Uploading a new one. Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes --- Key: HIVE-6326 URL: https://issues.apache.org/jira/browse/HIVE-6326 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Labels: orcfile Attachments: HIVE-6326.1.patch, HIVE-6326.2.patch, HIVE-6326.3.patch HIVE-5091 added padding to ORC files to avoid ORC stripes straddling HDFS blocks. The length of this padded bytes are not stored in stripe information. OrcInputFormat.getSplits() uses stripeInformation.getLength() for split computation. stripeInformation.getLength() is sum of index length, data length and stripe footer length. It does not account for the length of padded bytes which may result in wrong split boundary. The fix for this is to use the offset of next stripe as the length of current stripe which includes the padded bytes as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5636) Introduce getPartitionColumns() functionality from HCatInputFormat
[ https://issues.apache.org/jira/browse/HIVE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-5636: --- Attachment: HIVE-5636.2.patch Attached rebased updated patch Introduce getPartitionColumns() functionality from HCatInputFormat -- Key: HIVE-5636 URL: https://issues.apache.org/jira/browse/HIVE-5636 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5636.2.patch, HIVE-5636.patch As of HCat 0.5, we made the class InputJobInfo private for hcatalog use only, and we made it so that setInput would not modify the InputJobInfo being passed in. However, if a user of HCatInputFormat wants to get what Partitioning columns or Data columns exist for the job, they are not able to do so directly from HCatInputFormat and are forced to use InputJobInfo, which currently does not work. Thus, we need to expose this functionality. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-5636) Introduce getPartitionColumns() functionality from HCatInputFormat
[ https://issues.apache.org/jira/browse/HIVE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-5636: --- Status: Patch Available (was: Open) Introduce getPartitionColumns() functionality from HCatInputFormat -- Key: HIVE-5636 URL: https://issues.apache.org/jira/browse/HIVE-5636 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5636.2.patch, HIVE-5636.patch As of HCat 0.5, we made the class InputJobInfo private for hcatalog use only, and we made it so that setInput would not modify the InputJobInfo being passed in. However, if a user of HCatInputFormat wants to get what Partitioning columns or Data columns exist for the job, they are not able to do so directly from HCatInputFormat and are forced to use InputJobInfo, which currently does not work. Thus, we need to expose this functionality. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5636) Introduce getPartitionColumns() functionality from HCatInputFormat
[ https://issues.apache.org/jira/browse/HIVE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902133#comment-13902133 ] Sushanth Sowmyan commented on HIVE-5636: In my previous patch, I'd removed some of the older deprecated methods, but I've avoided doing that in this current patch given that it's so late in this release cycle - I think it's better to approach that after our next release. Introduce getPartitionColumns() functionality from HCatInputFormat -- Key: HIVE-5636 URL: https://issues.apache.org/jira/browse/HIVE-5636 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5636.2.patch, HIVE-5636.patch As of HCat 0.5, we made the class InputJobInfo private for hcatalog use only, and we made it so that setInput would not modify the InputJobInfo being passed in. However, if a user of HCatInputFormat wants to get what Partitioning columns or Data columns exist for the job, they are not able to do so directly from HCatInputFormat and are forced to use InputJobInfo, which currently does not work. Thus, we need to expose this functionality. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5636) Introduce getPartitionColumns() functionality from HCatInputFormat
[ https://issues.apache.org/jira/browse/HIVE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902137#comment-13902137 ] Sushanth Sowmyan commented on HIVE-5636: Reviewboard link : https://reviews.apache.org/r/18151/ [~thejas]/[~daijy] , can I bug either of you for a review? Introduce getPartitionColumns() functionality from HCatInputFormat -- Key: HIVE-5636 URL: https://issues.apache.org/jira/browse/HIVE-5636 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5636.2.patch, HIVE-5636.patch As of HCat 0.5, we made the class InputJobInfo private for hcatalog use only, and we made it so that setInput would not modify the InputJobInfo being passed in. However, if a user of HCatInputFormat wants to get what Partitioning columns or Data columns exist for the job, they are not able to do so directly from HCatInputFormat and are forced to use InputJobInfo, which currently does not work. Thus, we need to expose this functionality. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
User doc for table properties
The user doc for TBLPROPERTIES needs work. Currently the DDL wikidoc only says this: The TBLPROPERTIES clause allows you to tag the table definition with your own metadata key/value pairs. But some table properties have predefined keys and values. HIVE-6406https://issues.apache.org/jira/browse/HIVE-6406will add immutable -- how many others already exist? Are they all listed in one file and distinguishable from internal parameters, or just scattered throughout the code? A quick search found orc.compress (example in HIVE-6083https://issues.apache.org/jira/browse/HIVE-6083) and hbase.table.name hbase.table.default.storage.type (in TestPigHBaseStorageHandler.java). OrcFile.java has several more listed after orc.compress (some mentioned in HIVE-4221https://issues.apache.org/jira/browse/HIVE-4221 comments). This might be a can of worms but the wiki should list all predefined keys and their possible values, with version information where needed. I suggest a new subsection in the Create Table section of DDL: - Language Manual - DDL - Create Tablehttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTable Then particular table properties can be mentioned in their topic docs (like ORC) with links to the DDL doc. This message can be converted to a JIRA ticket later, but now I'm just looking for information. Hearts flowers chocolate to all on Valentine's Day. -- Lefty
[jira] [Commented] (HIVE-6256) add batch dropping of partitions to Hive metastore (as well as to dropTable)
[ https://issues.apache.org/jira/browse/HIVE-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902149#comment-13902149 ] Lefty Leverenz commented on HIVE-6256: -- This is all internal, right? No user doc needed? add batch dropping of partitions to Hive metastore (as well as to dropTable) Key: HIVE-6256 URL: https://issues.apache.org/jira/browse/HIVE-6256 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6256.01.patch, HIVE-6256.02.patch, HIVE-6256.03.patch, HIVE-6256.04.patch, HIVE-6256.05.patch, HIVE-6256.06.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.patch Metastore drop partitions call drops one partition; when many are being dropped this can be slow. Partitions could be dropped in batch instead, if multiple are dropped via one command. Drop table can also use that. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HIVE-6432) Remove deprecated methods in HCatalog
Sushanth Sowmyan created HIVE-6432: -- Summary: Remove deprecated methods in HCatalog Key: HIVE-6432 URL: https://issues.apache.org/jira/browse/HIVE-6432 Project: Hive Issue Type: Task Components: HCatalog Reporter: Sushanth Sowmyan There are a lot of methods in HCatalog that have been deprecated in HCatalog 0.5, and some that were recently deprecated in Hive 0.11 (joint release with HCatalog). The goal for HCatalog deprecation is that in general, after something has been deprecated, it is expected to stay around for 2 releases, which means hive-0.13 will be the last release to ship with all the methods that were deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be removed afterwards), and it is also good for us to clean out and nuke all other older deprecated methods. We should take this on early in a dev/release cycle to allow us time to resolve all fallout, so I propose that we remove all HCatalog deprecated methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6256) add batch dropping of partitions to Hive metastore (as well as to dropTable)
[ https://issues.apache.org/jira/browse/HIVE-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902163#comment-13902163 ] Ashutosh Chauhan commented on HIVE-6256: Yup, performance enhancement, no functional changes visible to user. add batch dropping of partitions to Hive metastore (as well as to dropTable) Key: HIVE-6256 URL: https://issues.apache.org/jira/browse/HIVE-6256 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6256.01.patch, HIVE-6256.02.patch, HIVE-6256.03.patch, HIVE-6256.04.patch, HIVE-6256.05.patch, HIVE-6256.06.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.nogen.patch, HIVE-6256.patch Metastore drop partitions call drops one partition; when many are being dropped this can be slow. Partitions could be dropped in batch instead, if multiple are dropped via one command. Drop table can also use that. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5636) Introduce getPartitionColumns() functionality from HCatInputFormat
[ https://issues.apache.org/jira/browse/HIVE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902164#comment-13902164 ] Sushanth Sowmyan commented on HIVE-5636: (I've created https://issues.apache.org/jira/browse/HIVE-6432 to track removal of all deprecations, and is something we can do right after we fork off 0.13 and trunk becomes 0.14) Introduce getPartitionColumns() functionality from HCatInputFormat -- Key: HIVE-5636 URL: https://issues.apache.org/jira/browse/HIVE-5636 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Sushanth Sowmyan Assignee: Sushanth Sowmyan Attachments: HIVE-5636.2.patch, HIVE-5636.patch As of HCat 0.5, we made the class InputJobInfo private for hcatalog use only, and we made it so that setInput would not modify the InputJobInfo being passed in. However, if a user of HCatInputFormat wants to get what Partitioning columns or Data columns exist for the job, they are not able to do so directly from HCatInputFormat and are forced to use InputJobInfo, which currently does not work. Thus, we need to expose this functionality. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HIVE-6433) SQL std auth - allow grant/revoke roles if user has ADMIN OPTION
Thejas M Nair created HIVE-6433: --- Summary: SQL std auth - allow grant/revoke roles if user has ADMIN OPTION Key: HIVE-6433 URL: https://issues.apache.org/jira/browse/HIVE-6433 Project: Hive Issue Type: Bug Reporter: Thejas M Nair Follow up jira for HIVE-5952. If a user/role has admin option on a role, then user should be able to grant /revoke other users to/from the role. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6433) SQL std auth - allow grant/revoke roles if user has ADMIN OPTION
[ https://issues.apache.org/jira/browse/HIVE-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6433: Issue Type: Sub-task (was: Bug) Parent: HIVE-5837 SQL std auth - allow grant/revoke roles if user has ADMIN OPTION Key: HIVE-6433 URL: https://issues.apache.org/jira/browse/HIVE-6433 Project: Hive Issue Type: Sub-task Reporter: Thejas M Nair Follow up jira for HIVE-5952. If a user/role has admin option on a role, then user should be able to grant /revoke other users to/from the role. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6326) Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes
[ https://issues.apache.org/jira/browse/HIVE-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902183#comment-13902183 ] Owen O'Malley commented on HIVE-6326: - You don't need the change to the first line, since it is functionally equivalent to the new version. Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes --- Key: HIVE-6326 URL: https://issues.apache.org/jira/browse/HIVE-6326 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Labels: orcfile Attachments: HIVE-6326.1.patch, HIVE-6326.2.patch, HIVE-6326.3.patch HIVE-5091 added padding to ORC files to avoid ORC stripes straddling HDFS blocks. The length of this padded bytes are not stored in stripe information. OrcInputFormat.getSplits() uses stripeInformation.getLength() for split computation. stripeInformation.getLength() is sum of index length, data length and stripe footer length. It does not account for the length of padded bytes which may result in wrong split boundary. The fix for this is to use the offset of next stripe as the length of current stripe which includes the padded bytes as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6326) Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes
[
https://issues.apache.org/jira/browse/HIVE-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902193#comment-13902193
]
Owen O'Malley commented on HIVE-6326:
-
You may also want to protect line 732 with
{code}
if (sarg != null
stripeStats != null
idx stripeStats.size()
!isStripeSatisfyPredicate(...) {
{code}
Split generation in ORC may generate wrong split boundaries because of
unaccounted padded bytes
---
Key: HIVE-6326
URL: https://issues.apache.org/jira/browse/HIVE-6326
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Prasanth J
Assignee: Prasanth J
Labels: orcfile
Attachments: HIVE-6326.1.patch, HIVE-6326.2.patch, HIVE-6326.3.patch
HIVE-5091 added padding to ORC files to avoid ORC stripes straddling HDFS
blocks. The length of this padded bytes are not stored in stripe information.
OrcInputFormat.getSplits() uses stripeInformation.getLength() for split
computation. stripeInformation.getLength() is sum of index length, data
length and stripe footer length. It does not account for the length of padded
bytes which may result in wrong split boundary.
The fix for this is to use the offset of next stripe as the length of current
stripe which includes the padded bytes as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6399) bug in high-precision Decimal128 multiply
[
https://issues.apache.org/jira/browse/HIVE-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902208#comment-13902208
]
Hive QA commented on HIVE-6399:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12629126/HIVE-6399.05.patch
{color:green}SUCCESS:{color} +1 5120 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12629126
bug in high-precision Decimal128 multiply
-
Key: HIVE-6399
URL: https://issues.apache.org/jira/browse/HIVE-6399
Project: Hive
Issue Type: Sub-task
Components: Query Processor, Vectorization
Reporter: Eric Hanson
Assignee: Eric Hanson
Labels: vectorization
Fix For: 0.13.0
Attachments: HIVE-6399.01.patch, HIVE-6399.02.patch,
HIVE-6399.02.patch, HIVE-6399.05.patch, HIVE-6399.3.patch, HIVE-6399.4.patch
For operation -605044214913338382 * 55269579109718297360
expected: -33440539101030154945490585226577271520
but was: -33440539021801992431226247633033321184
More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply
many times, you'll get an occasional failure. This is one example of such a
failure.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (HIVE-6434) Restrict function create/drop to admin roles
Jason Dere created HIVE-6434: Summary: Restrict function create/drop to admin roles Key: HIVE-6434 URL: https://issues.apache.org/jira/browse/HIVE-6434 Project: Hive Issue Type: Sub-task Components: Authorization, UDF Reporter: Jason Dere Assignee: Jason Dere -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902210#comment-13902210 ] Eugene Koifman commented on HIVE-6432: -- +1 on the idea. we'll also be able to remove a may of unit tests that cover old API Remove deprecated methods in HCatalog - Key: HIVE-6432 URL: https://issues.apache.org/jira/browse/HIVE-6432 Project: Hive Issue Type: Task Components: HCatalog Reporter: Sushanth Sowmyan There are a lot of methods in HCatalog that have been deprecated in HCatalog 0.5, and some that were recently deprecated in Hive 0.11 (joint release with HCatalog). The goal for HCatalog deprecation is that in general, after something has been deprecated, it is expected to stay around for 2 releases, which means hive-0.13 will be the last release to ship with all the methods that were deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be removed afterwards), and it is also good for us to clean out and nuke all other older deprecated methods. We should take this on early in a dev/release cycle to allow us time to resolve all fallout, so I propose that we remove all HCatalog deprecated methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6326) Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes
[ https://issues.apache.org/jira/browse/HIVE-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated HIVE-6326: - Attachment: HIVE-6326.4.patch Incorporated [~owen.omalley] review comments. Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes --- Key: HIVE-6326 URL: https://issues.apache.org/jira/browse/HIVE-6326 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Prasanth J Assignee: Prasanth J Labels: orcfile Attachments: HIVE-6326.1.patch, HIVE-6326.2.patch, HIVE-6326.3.patch, HIVE-6326.4.patch HIVE-5091 added padding to ORC files to avoid ORC stripes straddling HDFS blocks. The length of this padded bytes are not stored in stripe information. OrcInputFormat.getSplits() uses stripeInformation.getLength() for split computation. stripeInformation.getLength() is sum of index length, data length and stripe footer length. It does not account for the length of padded bytes which may result in wrong split boundary. The fix for this is to use the offset of next stripe as the length of current stripe which includes the padded bytes as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HIVE-6435) Allow specification of alternate metastore in WebHCat job
Eric Hanson created HIVE-6435: - Summary: Allow specification of alternate metastore in WebHCat job Key: HIVE-6435 URL: https://issues.apache.org/jira/browse/HIVE-6435 Project: Hive Issue Type: Improvement Components: CLI, WebHCat Reporter: Eric Hanson Assignee: Eric Hanson Allow a user to specify with their WebHCat jobs a metastore database JDBC connection string. For the job, this overrides the default metastore configured for the cluster. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HIVE-6434) Restrict function create/drop to admin roles
[ https://issues.apache.org/jira/browse/HIVE-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6434: - Attachment: HIVE-6434.1.patch patch v1. Restrict function create/drop to admin roles Key: HIVE-6434 URL: https://issues.apache.org/jira/browse/HIVE-6434 Project: Hive Issue Type: Sub-task Components: Authorization, UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6434.1.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HIVE-6436) Allow specification of one or more additional Windows Azure storage accounts in WebHCat job
Eric Hanson created HIVE-6436: - Summary: Allow specification of one or more additional Windows Azure storage accounts in WebHCat job Key: HIVE-6436 URL: https://issues.apache.org/jira/browse/HIVE-6436 Project: Hive Issue Type: Improvement Components: CLI, WebHCat Reporter: Eric Hanson Allow a user to specify one or more additional Windows Azure storage accounts, including account name and key, in a WebHCat Hive job submission. These would be in addition to any that were specified in the default cluster configuration. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (HIVE-6436) Allow specification of one or more additional Windows Azure storage accounts in WebHCat job
[ https://issues.apache.org/jira/browse/HIVE-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Hanson reassigned HIVE-6436: - Assignee: Eric Hanson Allow specification of one or more additional Windows Azure storage accounts in WebHCat job --- Key: HIVE-6436 URL: https://issues.apache.org/jira/browse/HIVE-6436 Project: Hive Issue Type: Improvement Components: CLI, WebHCat Reporter: Eric Hanson Assignee: Eric Hanson Allow a user to specify one or more additional Windows Azure storage accounts, including account name and key, in a WebHCat Hive job submission. These would be in addition to any that were specified in the default cluster configuration. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Review Request 18162: HIVE-6434: Restrict function create/drop to admin roles
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18162/ --- Review request for hive and Thejas Nair. Bugs: HIVE-6434 https://issues.apache.org/jira/browse/HIVE-6434 Repository: hive-git Description --- Add output entity of DB object to make sure only admin roles can add/drop functions/macros. Diffs - ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java cc12f30 ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java 36f8f71 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/Operation2Privilege.java fae6844 ql/src/test/queries/clientnegative/authorization_create_func1.q PRE-CREATION ql/src/test/queries/clientnegative/authorization_create_func2.q PRE-CREATION ql/src/test/queries/clientnegative/authorization_create_macro1.q PRE-CREATION ql/src/test/queries/clientpositive/authorization_create_func1.q PRE-CREATION ql/src/test/queries/clientpositive/authorization_create_macro1.q PRE-CREATION ql/src/test/results/clientnegative/authorization_create_func1.q.out PRE-CREATION ql/src/test/results/clientnegative/authorization_create_func2.q.out PRE-CREATION ql/src/test/results/clientnegative/authorization_create_macro1.q.out PRE-CREATION ql/src/test/results/clientpositive/authorization_create_func1.q.out PRE-CREATION ql/src/test/results/clientpositive/authorization_create_macro1.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18162/diff/ Testing --- positive/negative q files added Thanks, Jason Dere
[jira] [Commented] (HIVE-6434) Restrict function create/drop to admin roles
[ https://issues.apache.org/jira/browse/HIVE-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902254#comment-13902254 ] Jason Dere commented on HIVE-6434: -- https://reviews.apache.org/r/18162/ Restrict function create/drop to admin roles Key: HIVE-6434 URL: https://issues.apache.org/jira/browse/HIVE-6434 Project: Hive Issue Type: Sub-task Components: Authorization, UDF Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6434.1.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
