[jira] [Updated] (HIVE-6986) MatchPath fails with small resultExprString

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6986:
---

   Resolution: Fixed
Fix Version/s: 0.14.0
 Assignee: Furcy Pin
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Furcy!

> MatchPath fails with small resultExprString
> ---
>
> Key: HIVE-6986
> URL: https://issues.apache.org/jira/browse/HIVE-6986
> Project: Hive
>  Issue Type: Bug
>  Components: UDF
>Reporter: Furcy Pin
>Assignee: Furcy Pin
>Priority: Trivial
> Fix For: 0.14.0
>
> Attachments: HIVE-6986.1.patch
>
>
> When using MatchPath, a query like this:
> select year
> from matchpath(on 
> flights_tiny 
> sort by fl_num, year, month, day_of_month  
>   arg1('LATE.LATE+'), 
>   arg2('LATE'), arg3(arr_delay > 15), 
> arg4('year') 
>)
> ;
> will fail with error message 
> "FAILED: StringIndexOutOfBoundsException String index out of range: 6"



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Status: Patch Available  (was: In Progress)

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, 
> HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006861#comment-14006861
 ] 

Ashish Kumar Singh commented on HIVE-7117:
--

[~swarnim] Done! The name now is bit long, but shouts out the intent. Thanks 
for reviewing!

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, 
> HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: HIVE-7117.4.patch

Rename fileutils.rename to fileutils.renameAndInheritParentPermissions

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, 
> HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7113) Support fully managed partitioned table, which does not allow partitions to have different properties with table

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006860#comment-14006860
 ] 

Hive QA commented on HIVE-7113:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646198/HIVE-7113.1.patch.txt

{color:red}ERROR:{color} -1 due to 48 failed/errored test(s), 5456 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.ql.parse.TestParse.testParse_case_sensitivity
org.apache.hadoop.hive.ql.parse.TestParse.testParse_cast1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input20
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input8
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input9
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input_testsequencefile
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input_testxpath
org.apache.hadoop.hive.ql.parse.TestParse.testParse_input_testxpath2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_join8
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample2
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample3
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample5
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_sample7
org.apache.hadoop.hive.ql.parse.TestParse.testParse_subq
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf1
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf4
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf6
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_case
org.apache.hadoop.hive.ql.parse.TestParse.testParse_udf_when
org.apache.hadoop.hive.ql.parse.TestParse.testParse_union
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/267/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/267/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-267/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 48 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12646198

> Support fully managed partitioned table, which does not allow partitions to 
> have different properties with table
> 
>
> Key: HIVE-7113
> URL: https://issues.apache.org/jira/browse/HIVE-7113
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7113.1.patch.txt
>
>
> Mostly, partitioned table has the same storage description with mother table 
> just except data location. If partitions can be forced not to have different 
> schema with table by configuration, the time for retrieving partition objects 
> from metastore and for crea

Re: Review Request 21846: Rename fileutils.rename to fileutils.renameAndInheritParentPermissions

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/
---

(Updated May 23, 2014, 6:13 a.m.)


Review request for hive and Szehon Ho.


Changes
---

Rename fileutils.rename to fileutils.renameAndInheritParentPermissions


Summary (updated)
-

Rename fileutils.rename to fileutils.renameAndInheritParentPermissions


Repository: hive-git


Description (updated)
---

Rename fileutils.rename to fileutils.renameAndInheritParentPermissions


Check for filesystem.rename return types


HIVE-7117: Partitions not inheriting table permissions after alter rename 
partition


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
ee61350ab1d32db96a234f6444836f76e23f3251 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
 4f566d2a2999eecea7018836505208ae80662e97 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
221b01043b3520211643847c00b17b328ad2b9c0 
  metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
c1790b427e7fa64780fdf0a580f28135d6a4ff8d 

Diff: https://reviews.apache.org/r/21846/diff/


Testing
---

Tested with itests.


Thanks,

Ashish Singh

[jira] [Updated] (HIVE-6910) Invalid column access info for partitioned table

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6910:
---

Status: Open  (was: Patch Available)

Patch needs to be rebased.

> Invalid column access info for partitioned table
> 
>
> Key: HIVE-6910
> URL: https://issues.apache.org/jira/browse/HIVE-6910
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.12.0, 0.11.0
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-6910.1.patch.txt, HIVE-6910.2.patch.txt, 
> HIVE-6910.3.patch.txt, HIVE-6910.4.patch.txt, HIVE-6910.5.patch.txt
>
>
> From http://www.mail-archive.com/user@hive.apache.org/msg11324.html
> neededColumnIDs in TS is only for non-partition columns. But 
> ColumnAccessAnalyzer is calculating it on all columns.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-4561) Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-4561:
---

Assignee: Navis  (was: caofangkun)
  Status: Open  (was: Patch Available)

> Column stats :  LOW_VALUE (or HIGH_VALUE) will always be 0. ,if all the 
> column values larger than 0.0 (or if all column values smaller than 0.0)
> 
>
> Key: HIVE-4561
> URL: https://issues.apache.org/jira/browse/HIVE-4561
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.12.0
>Reporter: caofangkun
>Assignee: Navis
> Attachments: HIVE-4561.1.patch, HIVE-4561.2.patch, HIVE-4561.3.patch, 
> HIVE-4561.4.patch.txt
>
>
> if all column values larger than 0.0  DOUBLE_LOW_VALUE always will be 0.0 
> or  if all column values less than 0.0,  DOUBLE_HIGH_VALUE will always be 
> hive (default)> create table src_test (price double);
> hive (default)> load data local inpath './test.txt' into table src_test;
> hive (default)> select * from src_test;
> OK
> 1.0
> 2.0
> 3.0
> Time taken: 0.313 seconds, Fetched: 3 row(s)
> hive (default)> analyze table src_test compute statistics for columns price;
> mysql> select * from TAB_COL_STATS \G;
>  CS_ID: 16
>DB_NAME: default
> TABLE_NAME: src_test
>COLUMN_NAME: price
>COLUMN_TYPE: double
> TBL_ID: 2586
> LONG_LOW_VALUE: 0
>LONG_HIGH_VALUE: 0
>   DOUBLE_LOW_VALUE: 0.   # Wrong Result ! Expected is 1.
>  DOUBLE_HIGH_VALUE: 3.
>  BIG_DECIMAL_LOW_VALUE: NULL
> BIG_DECIMAL_HIGH_VALUE: NULL
>  NUM_NULLS: 0
>  NUM_DISTINCTS: 1
>AVG_COL_LEN: 0.
>MAX_COL_LEN: 0
>  NUM_TRUES: 0
> NUM_FALSES: 0
>  LAST_ANALYZED: 1368596151
> 2 rows in set (0.00 sec)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7109) Resource leak in HBaseStorageHandler

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006846#comment-14006846
 ] 

Ashutosh Chauhan commented on HIVE-7109:


+1

> Resource leak in HBaseStorageHandler
> 
>
> Key: HIVE-7109
> URL: https://issues.apache.org/jira/browse/HIVE-7109
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Attachments: HIVE-7109.1.patch.txt
>
>
> The "preCreateTable" method in the HBaseStorageHandler checks that the HBase 
> table is still online by creating a new instance of HTable
> {code}
> // ensure the table is online
> new HTable(hbaseConf, tableDesc.getName());
> {code}
> However this instance is never closed. So if this test succeeds, we would 
> have a resource leak in the code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006842#comment-14006842
 ] 

Swarnim Kulkarni commented on HIVE-7117:


Thanks Ashish. Left one more minor comment after that.

+1(non-binding) after that.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21846: Check for filesystem.rename return types

[Swarnim Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/#review43813
---



common/src/java/org/apache/hadoop/hive/common/FileUtils.java


Minor but this method seems to be doing more than just renaming. Should 
probably update the method name to be descriptive of what it does.


- Swarnim Kulkarni


On May 23, 2014, 5:43 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21846/
> ---
> 
> (Updated May 23, 2014, 5:43 a.m.)
> 
> 
> Review request for hive and Szehon Ho.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-7117: Partitions not inheriting table permissions after alter rename 
> partition
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
> ee61350ab1d32db96a234f6444836f76e23f3251 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
>  4f566d2a2999eecea7018836505208ae80662e97 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
> 221b01043b3520211643847c00b17b328ad2b9c0 
>   metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
> c1790b427e7fa64780fdf0a580f28135d6a4ff8d 
> 
> Diff: https://reviews.apache.org/r/21846/diff/
> 
> 
> Testing
> ---
> 
> Tested with itests.
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

[jira] [Updated] (HIVE-7109) Resource leak in HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7109:
---

Affects Version/s: 0.13.0
   Status: Patch Available  (was: Open)

> Resource leak in HBaseStorageHandler
> 
>
> Key: HIVE-7109
> URL: https://issues.apache.org/jira/browse/HIVE-7109
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Attachments: HIVE-7109.1.patch.txt
>
>
> The "preCreateTable" method in the HBaseStorageHandler checks that the HBase 
> table is still online by creating a new instance of HTable
> {code}
> // ensure the table is online
> new HTable(hbaseConf, tableDesc.getName());
> {code}
> However this instance is never closed. So if this test succeeds, we would 
> have a resource leak in the code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7109) Resource leak in HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006836#comment-14006836
 ] 

Swarnim Kulkarni commented on HIVE-7109:


This is now available for review.

> Resource leak in HBaseStorageHandler
> 
>
> Key: HIVE-7109
> URL: https://issues.apache.org/jira/browse/HIVE-7109
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Attachments: HIVE-7109.1.patch.txt
>
>
> The "preCreateTable" method in the HBaseStorageHandler checks that the HBase 
> table is still online by creating a new instance of HTable
> {code}
> // ensure the table is online
> new HTable(hbaseConf, tableDesc.getName());
> {code}
> However this instance is never closed. So if this test succeeds, we would 
> have a resource leak in the code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7109) Resource leak in HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7109:
---

Attachment: HIVE-7109.1.patch.txt

Patch attached.

> Resource leak in HBaseStorageHandler
> 
>
> Key: HIVE-7109
> URL: https://issues.apache.org/jira/browse/HIVE-7109
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Attachments: HIVE-7109.1.patch.txt
>
>
> The "preCreateTable" method in the HBaseStorageHandler checks that the HBase 
> table is still online by creating a new instance of HTable
> {code}
> // ensure the table is online
> new HTable(hbaseConf, tableDesc.getName());
> {code}
> However this instance is never closed. So if this test succeeds, we would 
> have a resource leak in the code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006834#comment-14006834
 ] 

Ashish Kumar Singh commented on HIVE-7117:
--

[~swarnim] updated patch and addressed you reviews at rb.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: HIVE-7117.3.patch

Add checks for filesystem.rename return value

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21846: Check for filesystem.rename return types

[Ashish Singh]

> On May 23, 2014, 5:12 a.m., Swarnim Kulkarni wrote:
> > common/src/java/org/apache/hadoop/hive/common/FileUtils.java, line 612
> > 
> >
> > Any specific reason why this exception is not allowed to propagate?

Idea is to try to inherit permissions, but not fail if inheriting permissions 
fail.


- Ashish


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/#review43810
---


On May 23, 2014, 5:43 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21846/
> ---
> 
> (Updated May 23, 2014, 5:43 a.m.)
> 
> 
> Review request for hive and Szehon Ho.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-7117: Partitions not inheriting table permissions after alter rename 
> partition
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
> ee61350ab1d32db96a234f6444836f76e23f3251 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
>  4f566d2a2999eecea7018836505208ae80662e97 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
> 221b01043b3520211643847c00b17b328ad2b9c0 
>   metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
> c1790b427e7fa64780fdf0a580f28135d6a4ff8d 
> 
> Diff: https://reviews.apache.org/r/21846/diff/
> 
> 
> Testing
> ---
> 
> Tested with itests.
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

Re: Review Request 21846: Check for filesystem.rename return types

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/
---

(Updated May 23, 2014, 5:43 a.m.)


Review request for hive and Szehon Ho.


Changes
---

Add checks for filesystem.rename return value


Summary (updated)
-

Check for filesystem.rename return types


Repository: hive-git


Description
---

HIVE-7117: Partitions not inheriting table permissions after alter rename 
partition


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
ee61350ab1d32db96a234f6444836f76e23f3251 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
 4f566d2a2999eecea7018836505208ae80662e97 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
221b01043b3520211643847c00b17b328ad2b9c0 
  metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
c1790b427e7fa64780fdf0a580f28135d6a4ff8d 

Diff: https://reviews.apache.org/r/21846/diff/


Testing
---

Tested with itests.


Thanks,

Ashish Singh

[jira] [Commented] (HIVE-7108) Cleanup HBaseStorageHandler

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006832#comment-14006832
 ] 

Ashutosh Chauhan commented on HIVE-7108:


+1

> Cleanup HBaseStorageHandler
> ---
>
> Key: HIVE-7108
> URL: https://issues.apache.org/jira/browse/HIVE-7108
> Project: Hive
>  Issue Type: Task
>  Components: HBase Handler
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Fix For: 0.13.0
>
> Attachments: HIVE-7108.1.patch.txt
>
>
> HBaseStorageHandler has bunch of variables and unused imports that should be 
> cleaned up. We should probably also revisit our findbugs and checkstyle 
> plugins to see why they did not catch these issues in first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7113) Support fully managed partitioned table, which does not allow partitions to have different properties with table

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006831#comment-14006831
 ] 

Gunther Hagleitner commented on HIVE-7113:
--

Are you failing "alter table" statements on a fully managed table? Or are you 
removing the property as soon as the schema changes?

I've been bitten by the cost of fetching too many partitions before, but schema 
evolution is typically important on tables with many partitions. Any chance you 
looked at some sort of de-duping partition information before sending it over 
thrift? Is it very expensive/difficult to do that?

> Support fully managed partitioned table, which does not allow partitions to 
> have different properties with table
> 
>
> Key: HIVE-7113
> URL: https://issues.apache.org/jira/browse/HIVE-7113
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7113.1.patch.txt
>
>
> Mostly, partitioned table has the same storage description with mother table 
> just except data location. If partitions can be forced not to have different 
> schema with table by configuration, the time for retrieving partition objects 
> from metastore and for creating serde for each partition could be nullified.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7108) Cleanup HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006827#comment-14006827
 ] 

Swarnim Kulkarni commented on HIVE-7108:


This is up for review.

> Cleanup HBaseStorageHandler
> ---
>
> Key: HIVE-7108
> URL: https://issues.apache.org/jira/browse/HIVE-7108
> Project: Hive
>  Issue Type: Task
>  Components: HBase Handler
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Fix For: 0.13.0
>
> Attachments: HIVE-7108.1.patch.txt
>
>
> HBaseStorageHandler has bunch of variables and unused imports that should be 
> cleaned up. We should probably also revisit our findbugs and checkstyle 
> plugins to see why they did not catch these issues in first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7108) Cleanup HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7108:
---

Attachment: HIVE-7108.1.patch.txt

Patch attached.

> Cleanup HBaseStorageHandler
> ---
>
> Key: HIVE-7108
> URL: https://issues.apache.org/jira/browse/HIVE-7108
> Project: Hive
>  Issue Type: Task
>  Components: HBase Handler
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Fix For: 0.13.0
>
> Attachments: HIVE-7108.1.patch.txt
>
>
> HBaseStorageHandler has bunch of variables and unused imports that should be 
> cleaned up. We should probably also revisit our findbugs and checkstyle 
> plugins to see why they did not catch these issues in first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7108) Cleanup HBaseStorageHandler

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7108:
---

Fix Version/s: 0.13.0
   Status: Patch Available  (was: Open)

> Cleanup HBaseStorageHandler
> ---
>
> Key: HIVE-7108
> URL: https://issues.apache.org/jira/browse/HIVE-7108
> Project: Hive
>  Issue Type: Task
>  Components: HBase Handler
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Fix For: 0.13.0
>
> Attachments: HIVE-7108.1.patch.txt
>
>
> HBaseStorageHandler has bunch of variables and unused imports that should be 
> cleaned up. We should probably also revisit our findbugs and checkstyle 
> plugins to see why they did not catch these issues in first place.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7087) Remove lineage information after query completion

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-7087:
---

Status: Patch Available  (was: Open)

> Remove lineage information after query completion
> -
>
> Key: HIVE-7087
> URL: https://issues.apache.org/jira/browse/HIVE-7087
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt
>
>
> Lineage information is stacked in session and is not cleared before the 
> session is closed. That also makes redundant lineage logs in q.out files for 
> all of the queries after any inserts, which should be available only for 
> insert queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7087) Remove lineage information after query completion

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006825#comment-14006825
 ] 

Ashutosh Chauhan commented on HIVE-7087:


+1

> Remove lineage information after query completion
> -
>
> Key: HIVE-7087
> URL: https://issues.apache.org/jira/browse/HIVE-7087
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt
>
>
> Lineage information is stacked in session and is not cleared before the 
> session is closed. That also makes redundant lineage logs in q.out files for 
> all of the queries after any inserts, which should be available only for 
> insert queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7048) CompositeKeyHBaseFactory should not use FamilyFilter

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7048:
---

Attachment: HIVE-7048.3.patch.txt

Addressed Xuefu's comments on RB. 

New patch attached.

> CompositeKeyHBaseFactory should not use FamilyFilter
> 
>
> Key: HIVE-7048
> URL: https://issues.apache.org/jira/browse/HIVE-7048
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>Priority: Blocker
> Attachments: HIVE-7048.1.patch.txt, HIVE-7048.2.patch.txt, 
> HIVE-7048.3.patch.txt
>
>
> HIVE-6411 introduced a more generic way to provide composite key 
> implementations via custom factory implementations. However it seems like the 
> CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
> which doesn't seem appropriate. This should be investigated further and if 
> possible replaced with a RowRangeScanFilter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006818#comment-14006818
 ] 

Swarnim Kulkarni commented on HIVE-7117:


Left comments on RB.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21846: HIVE-7117: Partitions not inheriting table permissions after alter rename partition

[Swarnim Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/#review43810
---



common/src/java/org/apache/hadoop/hive/common/FileUtils.java


Should check the return flag on the rename to ensure that the rename was 
successful. If not bail out/log warning as appropriate.



common/src/java/org/apache/hadoop/hive/common/FileUtils.java


Similar comment here. Check that the return type is true to ensure that 
renaming happened.



common/src/java/org/apache/hadoop/hive/common/FileUtils.java


Does this have to be logged at an info level? Or a debug be more 
appropriate here?



common/src/java/org/apache/hadoop/hive/common/FileUtils.java


Any specific reason why this exception is not allowed to propagate?


- Swarnim Kulkarni


On May 23, 2014, 4:42 a.m., Ashish Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21846/
> ---
> 
> (Updated May 23, 2014, 4:42 a.m.)
> 
> 
> Review request for hive and Szehon Ho.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-7117: Partitions not inheriting table permissions after alter rename 
> partition
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
> ee61350ab1d32db96a234f6444836f76e23f3251 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
>  4f566d2a2999eecea7018836505208ae80662e97 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
> 221b01043b3520211643847c00b17b328ad2b9c0 
>   metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
> c1790b427e7fa64780fdf0a580f28135d6a4ff8d 
> 
> Diff: https://reviews.apache.org/r/21846/diff/
> 
> 
> Testing
> ---
> 
> Tested with itests.
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>

[jira] [Work started] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-7117 started by Ashish Kumar Singh.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006790#comment-14006790
 ] 

Swarnim Kulkarni commented on HIVE-7116:


Should the fs.close() be in a finally so that an exception during mkdirs 
doesn't still cause a resource leak? We might as well use IOUtils.closeQuietly 
if we do unnecessarily want the fs.close() exception to propagate up the stack.

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch, HIVE-7116.2.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006788#comment-14006788
 ] 

Ashish Kumar Singh commented on HIVE-7117:
--

[~szehon] Updated patch and posted rb at https://reviews.apache.org/r/21846/.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21846: HIVE-7117: Partitions not inheriting table permissions after alter rename partition

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/
---

(Updated May 23, 2014, 4:42 a.m.)


Review request for hive and Szehon Ho.


Changes
---

Added testing done.


Repository: hive-git


Description
---

HIVE-7117: Partitions not inheriting table permissions after alter rename 
partition


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
ee61350ab1d32db96a234f6444836f76e23f3251 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
 4f566d2a2999eecea7018836505208ae80662e97 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
221b01043b3520211643847c00b17b328ad2b9c0 
  metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
c1790b427e7fa64780fdf0a580f28135d6a4ff8d 

Diff: https://reviews.apache.org/r/21846/diff/


Testing (updated)
---

Tested with itests.


Thanks,

Ashish Singh

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: HIVE-7117.2.patch

Removed code formatting changes.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.2.patch, HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 21846: HIVE-7117: Partitions not inheriting table permissions after alter rename partition

[Ashish Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21846/
---

Review request for hive and Szehon Ho.


Repository: hive-git


Description
---

HIVE-7117: Partitions not inheriting table permissions after alter rename 
partition


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java 
ee61350ab1d32db96a234f6444836f76e23f3251 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java
 4f566d2a2999eecea7018836505208ae80662e97 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
221b01043b3520211643847c00b17b328ad2b9c0 
  metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 
c1790b427e7fa64780fdf0a580f28135d6a4ff8d 

Diff: https://reviews.apache.org/r/21846/diff/


Testing
---


Thanks,

Ashish Singh

[jira] [Updated] (HIVE-7118) Oracle upgrade schema scripts do not map Java long datatype columns correctly for transaction related tables

[Deepesh Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepesh Khandelwal updated HIVE-7118:
-

Fix Version/s: 0.14.0
   Status: Patch Available  (was: Open)

> Oracle upgrade schema scripts do not map Java long datatype columns correctly 
> for transaction related tables
> 
>
> Key: HIVE-7118
> URL: https://issues.apache.org/jira/browse/HIVE-7118
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema
>Affects Versions: 0.14.0
> Environment: Oracle DB
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Fix For: 0.14.0
>
> Attachments: HIVE-7118.1.patch
>
>
> In Transaction related tables, Java long column fields are mapped to 
> NUMBER(10) which results in failure to persist the transaction ids which are 
> incompatible. Following error is seen:
> {noformat}
> ORA-01438: value larger than specified precision allowed for this column
> {noformat}
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7118) Oracle upgrade schema scripts do not map Java long datatype columns correctly for transaction related tables

[Deepesh Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepesh Khandelwal updated HIVE-7118:
-

Attachment: HIVE-7118.1.patch

Attaching a patch that changes NUMBER(10) to NUMBER(19) datatype for long 
columns for Oracle metastore schema upgrade scripts.

> Oracle upgrade schema scripts do not map Java long datatype columns correctly 
> for transaction related tables
> 
>
> Key: HIVE-7118
> URL: https://issues.apache.org/jira/browse/HIVE-7118
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema
>Affects Versions: 0.14.0
> Environment: Oracle DB
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-7118.1.patch
>
>
> In Transaction related tables, Java long column fields are mapped to 
> NUMBER(10) which results in failure to persist the transaction ids which are 
> incompatible. Following error is seen:
> {noformat}
> ORA-01438: value larger than specified precision allowed for this column
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7118) Oracle upgrade schema scripts do not map Java long datatype columns correctly for transaction related tables

[Deepesh Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepesh Khandelwal updated HIVE-7118:
-

Description: 
In Transaction related tables, Java long column fields are mapped to NUMBER(10) 
which results in failure to persist the transaction ids which are incompatible. 
Following error is seen:
{noformat}
ORA-01438: value larger than specified precision allowed for this column
{noformat}
NO PRECOMMIT TESTS

  was:
In Transaction related tables, Java long column fields are mapped to NUMBER(10) 
which results in failure to persist the transaction ids which are incompatible. 
Following error is seen:
{noformat}
ORA-01438: value larger than specified precision allowed for this column
{noformat}


> Oracle upgrade schema scripts do not map Java long datatype columns correctly 
> for transaction related tables
> 
>
> Key: HIVE-7118
> URL: https://issues.apache.org/jira/browse/HIVE-7118
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema
>Affects Versions: 0.14.0
> Environment: Oracle DB
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-7118.1.patch
>
>
> In Transaction related tables, Java long column fields are mapped to 
> NUMBER(10) which results in failure to persist the transaction ids which are 
> incompatible. Following error is seen:
> {noformat}
> ORA-01438: value larger than specified precision allowed for this column
> {noformat}
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7119) Extended ACL's should be inherited if warehouse perm inheritance enabled

[Szehon Ho (JIRA)]

Szehon Ho created HIVE-7119:
---

 Summary: Extended ACL's should be inherited if warehouse perm 
inheritance enabled
 Key: HIVE-7119
 URL: https://issues.apache.org/jira/browse/HIVE-7119
 Project: Hive
  Issue Type: Bug
Reporter: Szehon Ho


HDFS recently came out with support for extended ACL's, ie permission for 
specific group/user in addition to the general owner/group/other permission.

Hive permission inheritance should also inherit those as well, if user has set 
them at any point in the warehouse directory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21841: Hive should support adding multiple resources at once

[Szehon Ho]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21841/#review43807
---



ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java


Why do we need to make a copy?



ql/src/java/org/apache/hadoop/hive/ql/processors/DeleteResourceProcessor.java


Why do we need to make a copy?



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Looks awkward to declare it only here, don't need to in my opinion as its a 
RuntimeException.



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Same as above



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Don't need to declare this kind of exception



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Don't need to declare this kind of exception



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Same


- Szehon Ho


On May 23, 2014, 2:14 a.m., Navis Ryu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21841/
> ---
> 
> (Updated May 23, 2014, 2:14 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-3907
> https://issues.apache.org/jira/browse/HIVE-3907
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Currently hive adds resources in one by one manner. And for JAR resources, 
> one classloader is created for each jar file, which seemed not good idea.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 49eb83f 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 
> f3f3619 
>   ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java 
> 5ed1e6c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/processors/DeleteResourceProcessor.java 
> 83fadeb 
>   ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java dab8610 
>   ql/src/test/org/apache/hadoop/hive/ql/session/TestSessionState.java 7df7ecc 
> 
> Diff: https://reviews.apache.org/r/21841/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Navis Ryu
> 
>

Re: Review Request 21841: Hive should support adding multiple resources at once

[Szehon Ho]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21841/#review43808
---


Good change, left some minor code considerations.

- Szehon Ho


On May 23, 2014, 2:14 a.m., Navis Ryu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21841/
> ---
> 
> (Updated May 23, 2014, 2:14 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-3907
> https://issues.apache.org/jira/browse/HIVE-3907
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Currently hive adds resources in one by one manner. And for JAR resources, 
> one classloader is created for each jar file, which seemed not good idea.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 49eb83f 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 
> f3f3619 
>   ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java 
> 5ed1e6c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/processors/DeleteResourceProcessor.java 
> 83fadeb 
>   ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java dab8610 
>   ql/src/test/org/apache/hadoop/hive/ql/session/TestSessionState.java 7df7ecc 
> 
> Diff: https://reviews.apache.org/r/21841/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Navis Ryu
> 
>

[jira] [Assigned] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh reassigned HIVE-7117:


Assignee: Ashish Kumar Singh

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>Assignee: Ashish Kumar Singh
> Attachments: HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7118) Oracle upgrade schema scripts do not map Java long datatype columns correctly for transaction related tables

[Deepesh Khandelwal (JIRA)]

Deepesh Khandelwal created HIVE-7118:


 Summary: Oracle upgrade schema scripts do not map Java long 
datatype columns correctly for transaction related tables
 Key: HIVE-7118
 URL: https://issues.apache.org/jira/browse/HIVE-7118
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema
Affects Versions: 0.14.0
 Environment: Oracle DB
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal


In Transaction related tables, Java long column fields are mapped to NUMBER(10) 
which results in failure to persist the transaction ids which are incompatible. 
Following error is seen:
{noformat}
ORA-01438: value larger than specified precision allowed for this column
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7087) Remove lineage information after query completion

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7087:


Attachment: HIVE-7087.2.patch.txt

with diff (partial)

> Remove lineage information after query completion
> -
>
> Key: HIVE-7087
> URL: https://issues.apache.org/jira/browse/HIVE-7087
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt
>
>
> Lineage information is stacked in session and is not cleared before the 
> session is closed. That also makes redundant lineage logs in q.out files for 
> all of the queries after any inserts, which should be available only for 
> insert queries.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7112) Tez processor swallows errors

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006722#comment-14006722
 ] 

Hive QA commented on HIVE-7112:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646346/HIVE-7112.2.patch

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5529 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/266/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/266/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-266/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12646346

> Tez processor swallows errors
> -
>
> Key: HIVE-7112
> URL: https://issues.apache.org/jira/browse/HIVE-7112
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-7112.1.patch, HIVE-7112.2.patch
>
>
> If a OOM or other occurs during initializing a TezProcessor the original 
> stack trace/throwable is swallowed and replaced by a misleading NPE caused by 
> trying to close an uninitialized processor.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-3054) Simple reconnection support for JDBC

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis resolved HIVE-3054.
-

Resolution: Won't Fix

hiveserver1 will be deprecated. closing the issue.

> Simple reconnection support for JDBC
> 
>
> Key: HIVE-3054
> URL: https://issues.apache.org/jira/browse/HIVE-3054
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.10.0
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
>
> Somehow when hive server is down, JDBC connections should be re-established 
> by hand and it's not clean to implement on the outside of JDBC code.
> Session recovery could be also implemented if HiveServer2(HIVE-2935) is ready.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7075) JsonSerde raises NullPointerException when object key is not lower case

[Harsh J (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006721#comment-14006721
 ] 

Harsh J commented on HIVE-7075:
---

Thanks for clarifying Navis (and the test) and Yibing!

> JsonSerde raises NullPointerException when object key is not lower case
> ---
>
> Key: HIVE-7075
> URL: https://issues.apache.org/jira/browse/HIVE-7075
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog, Serializers/Deserializers
>Affects Versions: 0.12.0
>Reporter: Yibing Shi
> Attachments: HIVE-7075.1.patch.txt, HIVE-7075.2.patch.txt
>
>
> We have noticed that the JsonSerde produces a NullPointerException if a JSON 
> object has a key value that is not lower case. For example. Assume we have 
> the file "one.json": 
> { "empId" : 123, "name" : "John" } 
> { "empId" : 456, "name" : "Jane" } 
> hive> CREATE TABLE emps (empId INT, name STRING) 
> ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 
> hive> LOAD DATA LOCAL INPATH 'one.json' INTO TABLE emps; 
> hive> SELECT * FROM emps; 
> Failed with exception java.io.IOException:java.lang.NullPointerException 
>  
> Notice, it seems to work if the keys are lower case. Assume we have the file 
> 'two.json': 
> { "empid" : 123, "name" : "John" } 
> { "empid" : 456, "name" : "Jane" } 
> hive> DROP TABLE emps; 
> hive> CREATE TABLE emps (empId INT, name STRING) 
> ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 
> hive> LOAD DATA LOCAL INPATH 'two.json' INTO TABLE emps;
> hive> SELECT * FROM emps; 
> OK 
> 123   John 
> 456   Jane



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-3907) Hive should support adding multiple resources at once

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-3907:


Status: Patch Available  (was: Open)

> Hive should support adding multiple resources at once
> -
>
> Key: HIVE-3907
> URL: https://issues.apache.org/jira/browse/HIVE-3907
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3907.2.patch.txt, HIVE-3907.D7971.1.patch
>
>
> Currently hive adds resources in one by one manner. And for JAR resources, 
> one classloader is created for each jar file, which seemed not good idea.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 21841: Hive should support adding multiple resources at once

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21841/
---

Review request for hive.


Bugs: HIVE-3907
https://issues.apache.org/jira/browse/HIVE-3907


Repository: hive-git


Description
---

Currently hive adds resources in one by one manner. And for JAR resources, one 
classloader is created for each jar file, which seemed not good idea.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 49eb83f 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 
f3f3619 
  ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java 
5ed1e6c 
  ql/src/java/org/apache/hadoop/hive/ql/processors/DeleteResourceProcessor.java 
83fadeb 
  ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java dab8610 
  ql/src/test/org/apache/hadoop/hive/ql/session/TestSessionState.java 7df7ecc 

Diff: https://reviews.apache.org/r/21841/diff/


Testing
---


Thanks,

Navis Ryu

[jira] [Updated] (HIVE-3907) Hive should support adding multiple resources at once

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-3907:


Attachment: HIVE-3907.2.patch.txt

> Hive should support adding multiple resources at once
> -
>
> Key: HIVE-3907
> URL: https://issues.apache.org/jira/browse/HIVE-3907
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3907.2.patch.txt, HIVE-3907.D7971.1.patch
>
>
> Currently hive adds resources in one by one manner. And for JAR resources, 
> one classloader is created for each jar file, which seemed not good idea.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Szehon Ho (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006713#comment-14006713
 ] 

Szehon Ho commented on HIVE-7117:
-

Thanks for the fix, can you please fix the formatting diffs and create a review 
board request with the patch?  Probably its clearer if the patch just contain 
code diffs.  Thanks!

[https://reviews.apache.org/dashboard/|https://reviews.apache.org/dashboard/]



> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
> Attachments: HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: HIVE-7117.patch

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
> Attachments: HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: HIVE-7117.diff

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
> Attachments: HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Attachment: (was: HIVE-7117.diff)

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
> Attachments: HIVE-7117.patch
>
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Kumar Singh updated HIVE-7117:
-

Description: On altering/renaming a partition it must inherit permission of 
the parent directory, if the flag hive.warehouse.subdir.inherit.perms is set.

> Partitions not inheriting table permissions after alter rename partition
> 
>
> Key: HIVE-7117
> URL: https://issues.apache.org/jira/browse/HIVE-7117
> Project: Hive
>  Issue Type: Bug
>  Components: Security
>Reporter: Ashish Kumar Singh
>
> On altering/renaming a partition it must inherit permission of the parent 
> directory, if the flag hive.warehouse.subdir.inherit.perms is set.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-4790) MapredLocalTask task does not make virtual columns

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-4790:


Attachment: HIVE-4790.8.patch.txt

> MapredLocalTask task does not make virtual columns
> --
>
> Key: HIVE-4790
> URL: https://issues.apache.org/jira/browse/HIVE-4790
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: D11511.3.patch, D11511.4.patch, HIVE-4790.5.patch.txt, 
> HIVE-4790.6.patch.txt, HIVE-4790.7.patch.txt, HIVE-4790.8.patch.txt, 
> HIVE-4790.D11511.1.patch, HIVE-4790.D11511.2.patch
>
>
> From mailing list, 
> http://www.mail-archive.com/user@hive.apache.org/msg08264.html
> {noformat}
> SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON 
> b.rownumber = a.number;
> fails with this error:
>  
> > SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = 
> a.number;
> Automatically selecting local only mode for query
> Total MapReduce jobs = 1
> setting HADOOP_USER_NAMEpmarron
> 13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property 
> hive.metastore.local no longer has any effect. Make sure to provide a valid 
> value for hive.metastore.uris if you are connecting to a remote metastore.
> Execution log at: /tmp/pmarron/.log
> 2013-06-25 10:52:56 Starting to launch local task to process map join;
>   maximum memory = 932118528
> java.lang.RuntimeException: cannot find field block__offset__inside__file 
> from [0:rownumber, 1:offset]
> at 
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
> at 
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
> at 
> org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
> at 
> org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
> at 
> org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
> at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at 
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
> at 
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Execution failed with exit status: 2
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-4790) MapredLocalTask task does not make virtual columns

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-4790:


Status: Patch Available  (was: Open)

> MapredLocalTask task does not make virtual columns
> --
>
> Key: HIVE-4790
> URL: https://issues.apache.org/jira/browse/HIVE-4790
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: D11511.3.patch, D11511.4.patch, HIVE-4790.5.patch.txt, 
> HIVE-4790.6.patch.txt, HIVE-4790.7.patch.txt, HIVE-4790.8.patch.txt, 
> HIVE-4790.D11511.1.patch, HIVE-4790.D11511.2.patch
>
>
> From mailing list, 
> http://www.mail-archive.com/user@hive.apache.org/msg08264.html
> {noformat}
> SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON 
> b.rownumber = a.number;
> fails with this error:
>  
> > SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = 
> a.number;
> Automatically selecting local only mode for query
> Total MapReduce jobs = 1
> setting HADOOP_USER_NAMEpmarron
> 13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property 
> hive.metastore.local no longer has any effect. Make sure to provide a valid 
> value for hive.metastore.uris if you are connecting to a remote metastore.
> Execution log at: /tmp/pmarron/.log
> 2013-06-25 10:52:56 Starting to launch local task to process map join;
>   maximum memory = 932118528
> java.lang.RuntimeException: cannot find field block__offset__inside__file 
> from [0:rownumber, 1:offset]
> at 
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
> at 
> org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
> at 
> org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
> at 
> org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
> at 
> org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
> at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
> at 
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at 
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
> at 
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Execution failed with exit status: 2
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7095) Fix test fails for both hadoop-1 and hadoop-2

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-7095:
---

   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Navis!

> Fix test fails for both hadoop-1 and hadoop-2
> -
>
> Key: HIVE-7095
> URL: https://issues.apache.org/jira/browse/HIVE-7095
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Reporter: Navis
>Assignee: Navis
> Fix For: 0.14.0
>
> Attachments: HIVE-7095.1.patch.txt, HIVE-7095.2.patch.txt, 
> HIVE-7095.3.patch.txt
>
>
> There are some try to fix test results for hadoop-2(HIVE-6838, HIVE-6979, 
> etc.). But these remained problems for hadoop-1. Let's finish this here, not 
> prolonging output ping-pongs between two versions.
> 1. Data size in explain result is different
> bucketizedhiveinputformat.q
> 2. forced bucketing(=2) does not make two files in hadoop-1. 
> groupby_sort_1.q
> groupby_sort_skew_1.q
> 3. Column stats of srcpart, which has loaded a file rather than inserted, has 
> COLUMN_STATS_ACCURATE=true, numRows=0, rawDataSize=0 for hadoop-2, which 
> seemed wrong.
> sample8.q
> transform_ppr1.q
> transform_ppr2.q
> union_ppr.q
> 4. Iteration order of table properties for show create table is different.
> nullformatCTAS.q
> show_create_table_alter.q
> 5. Iteration order of show table properties is different
> unset_table_view_property.q



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-4923) Provide compile + run method for HiveServer2

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis resolved HIVE-4923.
-

Resolution: Won't Fix

Seemed replaced by asynchronous execution in hiveserver2. Closing the issue.

> Provide compile + run method for HiveServer2
> 
>
> Key: HIVE-4923
> URL: https://issues.apache.org/jira/browse/HIVE-4923
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-4923.D11781.1.patch
>
>
> Make additional thrift method for asynchronous execution.
> execute=compile+run
> This addresses same thing with HIVE-4569 but with simpler implementation. 
> It's just for reference (We are using this for months).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 21834: JsonSerde raises NullPointerException when object key is not lower case

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21834/
---

Review request for hive.


Bugs: HIVE-7075
https://issues.apache.org/jira/browse/HIVE-7075


Repository: hive-git


Description
---

We have noticed that the JsonSerde produces a NullPointerException if a JSON 
object has a key value that is not lower case. For example. Assume we have the 
file "one.json": 

{ "empId" : 123, "name" : "John" } 
{ "empId" : 456, "name" : "Jane" } 

hive> CREATE TABLE emps (empId INT, name STRING) 
ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 

hive> LOAD DATA LOCAL INPATH 'one.json' INTO TABLE emps; 

hive> SELECT * FROM emps; 

Failed with exception java.io.IOException:java.lang.NullPointerException 

 

Notice, it seems to work if the keys are lower case. Assume we have the file 
'two.json': 
{ "empid" : 123, "name" : "John" } 
{ "empid" : 456, "name" : "Jane" } 

hive> DROP TABLE emps; 

hive> CREATE TABLE emps (empId INT, name STRING) 
ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 

hive> LOAD DATA LOCAL INPATH 'two.json' INTO TABLE emps;

hive> SELECT * FROM emps; 
OK 
123 John 
456 Jane


Diffs
-

  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/schema/HCatSchema.java
 59e18b4 
  hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java 
da5ae97 

Diff: https://reviews.apache.org/r/21834/diff/


Testing
---


Thanks,

Navis Ryu

[jira] [Updated] (HIVE-7075) JsonSerde raises NullPointerException when object key is not lower case

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7075:


Attachment: HIVE-7075.2.patch.txt

> JsonSerde raises NullPointerException when object key is not lower case
> ---
>
> Key: HIVE-7075
> URL: https://issues.apache.org/jira/browse/HIVE-7075
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog, Serializers/Deserializers
>Affects Versions: 0.12.0
>Reporter: Yibing Shi
> Attachments: HIVE-7075.1.patch.txt, HIVE-7075.2.patch.txt
>
>
> We have noticed that the JsonSerde produces a NullPointerException if a JSON 
> object has a key value that is not lower case. For example. Assume we have 
> the file "one.json": 
> { "empId" : 123, "name" : "John" } 
> { "empId" : 456, "name" : "Jane" } 
> hive> CREATE TABLE emps (empId INT, name STRING) 
> ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 
> hive> LOAD DATA LOCAL INPATH 'one.json' INTO TABLE emps; 
> hive> SELECT * FROM emps; 
> Failed with exception java.io.IOException:java.lang.NullPointerException 
>  
> Notice, it seems to work if the keys are lower case. Assume we have the file 
> 'two.json': 
> { "empid" : 123, "name" : "John" } 
> { "empid" : 456, "name" : "Jane" } 
> hive> DROP TABLE emps; 
> hive> CREATE TABLE emps (empId INT, name STRING) 
> ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"; 
> hive> LOAD DATA LOCAL INPATH 'two.json' INTO TABLE emps;
> hive> SELECT * FROM emps; 
> OK 
> 123   John 
> 456   Jane



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6132) Support hbase filters for Hive HBase Integration

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-6132:
---

Fix Version/s: 0.14.0

> Support hbase filters for Hive HBase Integration
> 
>
> Key: HIVE-6132
> URL: https://issues.apache.org/jira/browse/HIVE-6132
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.12.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
> Fix For: 0.14.0
>
>
> Along with predicate pushdown to optimize HBase table scans, we should 
> support HBase filters as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition

[Ashish Kumar Singh (JIRA)]

Ashish Kumar Singh created HIVE-7117:


 Summary: Partitions not inheriting table permissions after alter 
rename partition
 Key: HIVE-7117
 URL: https://issues.apache.org/jira/browse/HIVE-7117
 Project: Hive
  Issue Type: Bug
  Components: Security
Reporter: Ashish Kumar Singh






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-7116:
-

Attachment: HIVE-7116.2.patch

[~swarnim] Changing the log to that format affects readability as it is the way 
log has been done in Hive. I agree that is not a strong argument but, there is 
no advantage either ways.

The fs.close() is a good catch! Thanks for your quick review!

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch, HIVE-7116.2.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-7116:
-

Status: Patch Available  (was: Open)

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch, HIVE-7116.2.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-7116:
-

Status: Open  (was: Patch Available)

Address Swarnim's comment.

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7095) Fix test fails for both hadoop-1 and hadoop-2

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7095:


Attachment: HIVE-7095.3.patch.txt

Should be a last one.

> Fix test fails for both hadoop-1 and hadoop-2
> -
>
> Key: HIVE-7095
> URL: https://issues.apache.org/jira/browse/HIVE-7095
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Reporter: Navis
>Assignee: Navis
> Attachments: HIVE-7095.1.patch.txt, HIVE-7095.2.patch.txt, 
> HIVE-7095.3.patch.txt
>
>
> There are some try to fix test results for hadoop-2(HIVE-6838, HIVE-6979, 
> etc.). But these remained problems for hadoop-1. Let's finish this here, not 
> prolonging output ping-pongs between two versions.
> 1. Data size in explain result is different
> bucketizedhiveinputformat.q
> 2. forced bucketing(=2) does not make two files in hadoop-1. 
> groupby_sort_1.q
> groupby_sort_skew_1.q
> 3. Column stats of srcpart, which has loaded a file rather than inserted, has 
> COLUMN_STATS_ACCURATE=true, numRows=0, rawDataSize=0 for hadoop-2, which 
> seemed wrong.
> sample8.q
> transform_ppr1.q
> transform_ppr2.q
> union_ppr.q
> 4. Iteration order of table properties for show create table is different.
> nullformatCTAS.q
> show_create_table_alter.q
> 5. Iteration order of show table properties is different
> unset_table_view_property.q



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21831: Make the behavior of CompositeKeyHBaseFactory consistent

[Xuefu Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21831/#review43800
---



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java


License header



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java


This class name seems needing to be improved. Howe about 
AbstractHBaseKeyPredicateDecomposer?



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java


How about naming it decomposePredicate()



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java


This should probably be protected.



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java


Same as above.



hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeParameters.java


I don't see any use of this.


- Xuefu Zhang


On May 22, 2014, 10:43 p.m., Swarnim Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21831/
> ---
> 
> (Updated May 22, 2014, 10:43 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-7048
> https://issues.apache.org/jira/browse/HIVE-7048
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-6411 introduced a more generic way to provide composite key 
> implementations via custom factory implementations. However it seems like the 
> CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
> which doesn't seem appropriate. This should be investigated further and if 
> possible replaced with a RowRangeScanFilter.
> 
> 
> Diffs
> -
> 
>   
> hbase-handler/src/java/org/apache/hadoop/hive/hbase/CompositeHBaseKeyFactory.java
>  53fa995 
>   
> hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java
>  PRE-CREATION 
>   
> hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeParameters.java 
> 07db3af 
>   
> hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseKeyFactory3.java 
> PRE-CREATION 
>   hbase-handler/src/test/queries/positive/hbase_custom_key3.q PRE-CREATION 
>   hbase-handler/src/test/results/positive/hbase_custom_key3.q.out 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/21831/diff/
> 
> 
> Testing
> ---
> 
> Tests included with the patch.
> 
> 
> Thanks,
> 
> Swarnim Kulkarni
> 
>

[jira] [Resolved] (HIVE-6132) Support hbase filters for Hive HBase Integration

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni resolved HIVE-6132.


Resolution: Fixed

> Support hbase filters for Hive HBase Integration
> 
>
> Key: HIVE-6132
> URL: https://issues.apache.org/jira/browse/HIVE-6132
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.12.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>
> Along with predicate pushdown to optimize HBase table scans, we should 
> support HBase filters as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6132) Support hbase filters for Hive HBase Integration

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006627#comment-14006627
 ] 

Swarnim Kulkarni commented on HIVE-6132:


Marking as resolved. The patches for this have been merged with HIVE-6411.

> Support hbase filters for Hive HBase Integration
> 
>
> Key: HIVE-6132
> URL: https://issues.apache.org/jira/browse/HIVE-6132
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.12.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>
> Along with predicate pushdown to optimize HBase table scans, we should 
> support HBase filters as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HIVE-6291) Add support to provide hbase filters dynamically

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni resolved HIVE-6291.


Resolution: Fixed

> Add support to provide hbase filters dynamically
> 
>
> Key: HIVE-6291
> URL: https://issues.apache.org/jira/browse/HIVE-6291
> Project: Hive
>  Issue Type: Sub-task
>  Components: HBase Handler
>Affects Versions: 0.12.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>
> We should be able to provide an hbase filter class dynamically which can be 
> used instead of start/stop keys on the underlying scan.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6291) Add support to provide hbase filters dynamically

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006626#comment-14006626
 ] 

Swarnim Kulkarni commented on HIVE-6291:


This functionality was provided as a part of HIVE-6411. Marking as resolved.

> Add support to provide hbase filters dynamically
> 
>
> Key: HIVE-6291
> URL: https://issues.apache.org/jira/browse/HIVE-6291
> Project: Hive
>  Issue Type: Sub-task
>  Components: HBase Handler
>Affects Versions: 0.12.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>
> We should be able to provide an hbase filter class dynamically which can be 
> used instead of start/stop keys on the underlying scan.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006623#comment-14006623
 ] 

Swarnim Kulkarni commented on HIVE-7116:


Hi Vikram,

Couple of smaller comments on the patch.

{code}
LOG.info("Create dirs " + mkdirPath + " with permission " + fsPermission + " 
recursive " + recursive);
{code}

You probably don't need to manually concatenate the string but let log4j do it 
for you.

{code}
LOG.info("Create dirs {} with permission {} recursive {}", mkdirPath, 
fsPermission, recursive);
{code}

Secondly, I think that if you create a FileSystem using the newInstance method, 
it should be closed to avoid any resource leaks. Previosuly with the cached FS 
instance, the API was automatically handling that for you.

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7095) Fix test fails for both hadoop-1 and hadoop-2

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006605#comment-14006605
 ] 

Hive QA commented on HIVE-7095:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646149/HIVE-7095.2.patch.txt

{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 5453 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHadoopVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getPigVersion
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getStatus
org.apache.hive.hcatalog.templeton.TestWebHCatE2e.invalidPath
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/265/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/265/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-265/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12646149

> Fix test fails for both hadoop-1 and hadoop-2
> -
>
> Key: HIVE-7095
> URL: https://issues.apache.org/jira/browse/HIVE-7095
> Project: Hive
>  Issue Type: Task
>  Components: Tests
>Reporter: Navis
>Assignee: Navis
> Attachments: HIVE-7095.1.patch.txt, HIVE-7095.2.patch.txt
>
>
> There are some try to fix test results for hadoop-2(HIVE-6838, HIVE-6979, 
> etc.). But these remained problems for hadoop-1. Let's finish this here, not 
> prolonging output ping-pongs between two versions.
> 1. Data size in explain result is different
> bucketizedhiveinputformat.q
> 2. forced bucketing(=2) does not make two files in hadoop-1. 
> groupby_sort_1.q
> groupby_sort_skew_1.q
> 3. Column stats of srcpart, which has loaded a file rather than inserted, has 
> COLUMN_STATS_ACCURATE=true, numRows=0, rawDataSize=0 for hadoop-2, which 
> seemed wrong.
> sample8.q
> transform_ppr1.q
> transform_ppr2.q
> union_ppr.q
> 4. Iteration order of table properties for show create table is different.
> nullformatCTAS.q
> show_create_table_alter.q
> 5. Iteration order of show table properties is different
> unset_table_view_property.q



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7099) Add Decimal datatype support for Windowing

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-7099:
---

   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Harish!

> Add Decimal datatype support for Windowing
> --
>
> Key: HIVE-7099
> URL: https://issues.apache.org/jira/browse/HIVE-7099
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Harish Butani
>Assignee: Harish Butani
> Fix For: 0.14.0
>
> Attachments: HIVE-7099.1.patch, HIVE-7099.2.patch
>
>
> Decimal datatype is not handled by Windowing



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6367) Implement Decimal in ParquetSerde

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6367:
--

Attachment: HIVE-6367.patch

Initial patch. One of new test .q file is expected to failure due to missing 
binary input file, which was already attached in this JIRA.

> Implement Decimal in ParquetSerde
> -
>
> Key: HIVE-6367
> URL: https://issues.apache.org/jira/browse/HIVE-6367
> Project: Hive
>  Issue Type: Sub-task
>  Components: Serializers/Deserializers
>Affects Versions: 0.13.0
>Reporter: Brock Noland
>Assignee: Xuefu Zhang
>  Labels: Parquet
> Attachments: HIVE-6367.patch, dec.parq
>
>
> Some code in the Parquet Serde deals with decimal and other does not. For 
> example in ETypeConverter we convert Decimal to double (which is invalid) 
> whereas in DataWritableWriter and other locations we throw an exception if 
> decimal is used.
> This JIRA is to implement decimal support.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6367) Implement Decimal in ParquetSerde

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6367:
--

Affects Version/s: 0.13.0
   Status: Patch Available  (was: Open)

> Implement Decimal in ParquetSerde
> -
>
> Key: HIVE-6367
> URL: https://issues.apache.org/jira/browse/HIVE-6367
> Project: Hive
>  Issue Type: Sub-task
>  Components: Serializers/Deserializers
>Affects Versions: 0.13.0
>Reporter: Brock Noland
>Assignee: Xuefu Zhang
>  Labels: Parquet
> Attachments: HIVE-6367.patch, dec.parq
>
>
> Some code in the Parquet Serde deals with decimal and other does not. For 
> example in ETypeConverter we convert Decimal to double (which is invalid) 
> whereas in DataWritableWriter and other locations we throw an exception if 
> decimal is used.
> This JIRA is to implement decimal support.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7114) Extra Tez session is started during HiveServer2 startup

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006566#comment-14006566
 ] 

Vikram Dixit K commented on HIVE-7114:
--

+1 LGTM.

> Extra Tez session is started during HiveServer2 startup
> ---
>
> Key: HIVE-7114
> URL: https://issues.apache.org/jira/browse/HIVE-7114
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Deepesh Khandelwal
>Assignee: Thejas M Nair
> Attachments: HIVE-7114.1.patch
>
>
> When starting the HiveServer2 we are seeing an extra Tez AM launched.
> This is where it is getting created .
> {noformat}
> 2014-05-09 23:11:22,261 INFO  [main]: metastore.HiveMetaStore 
> (HiveMetaStore.java:addAdminUsers(588)) - No user is added in admin role, 
> since config is empty
> java.lang.Exception: Opening session
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:356)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
> at 
> org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
> at 
> org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at 
> org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:68)
> at 
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:100)
> at 
> org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7114) Extra Tez session is started during HiveServer2 startup

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-7114:


Status: Patch Available  (was: Open)

> Extra Tez session is started during HiveServer2 startup
> ---
>
> Key: HIVE-7114
> URL: https://issues.apache.org/jira/browse/HIVE-7114
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Deepesh Khandelwal
>Assignee: Thejas M Nair
> Attachments: HIVE-7114.1.patch
>
>
> When starting the HiveServer2 we are seeing an extra Tez AM launched.
> This is where it is getting created .
> {noformat}
> 2014-05-09 23:11:22,261 INFO  [main]: metastore.HiveMetaStore 
> (HiveMetaStore.java:addAdminUsers(588)) - No user is added in admin role, 
> since config is empty
> java.lang.Exception: Opening session
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:356)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
> at 
> org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
> at 
> org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at 
> org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:68)
> at 
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:100)
> at 
> org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7114) Extra Tez session is started during HiveServer2 startup

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-7114:


Attachment: HIVE-7114.1.patch

> Extra Tez session is started during HiveServer2 startup
> ---
>
> Key: HIVE-7114
> URL: https://issues.apache.org/jira/browse/HIVE-7114
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Deepesh Khandelwal
>Assignee: Thejas M Nair
> Attachments: HIVE-7114.1.patch
>
>
> When starting the HiveServer2 we are seeing an extra Tez AM launched.
> This is where it is getting created .
> {noformat}
> 2014-05-09 23:11:22,261 INFO  [main]: metastore.HiveMetaStore 
> (HiveMetaStore.java:addAdminUsers(588)) - No user is added in admin role, 
> since config is empty
> java.lang.Exception: Opening session
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:356)
> at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
> at 
> org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
> at 
> org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
> at 
> org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at 
> org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:68)
> at 
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:100)
> at 
> org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7048) CompositeKeyHBaseFactory should not use FamilyFilter

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7048:
---

Affects Version/s: 0.13.0
   Status: Patch Available  (was: In Progress)

> CompositeKeyHBaseFactory should not use FamilyFilter
> 
>
> Key: HIVE-7048
> URL: https://issues.apache.org/jira/browse/HIVE-7048
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>Priority: Blocker
> Attachments: HIVE-7048.1.patch.txt, HIVE-7048.2.patch.txt
>
>
> HIVE-6411 introduced a more generic way to provide composite key 
> implementations via custom factory implementations. However it seems like the 
> CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
> which doesn't seem appropriate. This should be investigated further and if 
> possible replaced with a RowRangeScanFilter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7048) CompositeKeyHBaseFactory should not use FamilyFilter

[Swarnim Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006558#comment-14006558
 ] 

Swarnim Kulkarni commented on HIVE-7048:


RB: https://reviews.apache.org/r/21831/

> CompositeKeyHBaseFactory should not use FamilyFilter
> 
>
> Key: HIVE-7048
> URL: https://issues.apache.org/jira/browse/HIVE-7048
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.13.0
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>Priority: Blocker
> Attachments: HIVE-7048.1.patch.txt, HIVE-7048.2.patch.txt
>
>
> HIVE-6411 introduced a more generic way to provide composite key 
> implementations via custom factory implementations. However it seems like the 
> CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
> which doesn't seem appropriate. This should be investigated further and if 
> possible replaced with a RowRangeScanFilter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 21831: Make the behavior of CompositeKeyHBaseFactory consistent

[Swarnim Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21831/
---

Review request for hive.


Bugs: HIVE-7048
https://issues.apache.org/jira/browse/HIVE-7048


Repository: hive-git


Description
---

HIVE-6411 introduced a more generic way to provide composite key 
implementations via custom factory implementations. However it seems like the 
CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
which doesn't seem appropriate. This should be investigated further and if 
possible replaced with a RowRangeScanFilter.


Diffs
-

  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/CompositeHBaseKeyFactory.java
 53fa995 
  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseKeyDecomposedPredicate.java
 PRE-CREATION 
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeParameters.java 
07db3af 
  hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseKeyFactory3.java 
PRE-CREATION 
  hbase-handler/src/test/queries/positive/hbase_custom_key3.q PRE-CREATION 
  hbase-handler/src/test/results/positive/hbase_custom_key3.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/21831/diff/


Testing
---

Tests included with the patch.


Thanks,

Swarnim Kulkarni

[jira] [Updated] (HIVE-7048) CompositeKeyHBaseFactory should not use FamilyFilter

[Swarnim Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swarnim Kulkarni updated HIVE-7048:
---

Attachment: HIVE-7048.2.patch.txt

Thanks [~xuefuz] for the explanation. 

Attached is the patch with the fix.

> CompositeKeyHBaseFactory should not use FamilyFilter
> 
>
> Key: HIVE-7048
> URL: https://issues.apache.org/jira/browse/HIVE-7048
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Reporter: Swarnim Kulkarni
>Assignee: Swarnim Kulkarni
>Priority: Blocker
> Attachments: HIVE-7048.1.patch.txt, HIVE-7048.2.patch.txt
>
>
> HIVE-6411 introduced a more generic way to provide composite key 
> implementations via custom factory implementations. However it seems like the 
> CompositeHBaseKeyFactory implementation uses a FamilyFilter for row key scans 
> which doesn't seem appropriate. This should be investigated further and if 
> possible replaced with a RowRangeScanFilter.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-7116:
-

Attachment: HIVE-7116.1.patch

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-7116:
-

Status: Patch Available  (was: Open)

> HDFS FileSystem object cache causes permission issues in creating tmp 
> directories
> -
>
> Key: HIVE-7116
> URL: https://issues.apache.org/jira/browse/HIVE-7116
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tez
>Affects Versions: 0.13.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-7116.1.patch
>
>
> We change permissions of the directory creation to 777 for HiveServer 2 
> operation and it turns out that because of HDFS caching, it does not reflect 
> once created. We need to use the non-cached version of the API to get a 
> FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HIVE-7116) HDFS FileSystem object cache causes permission issues in creating tmp directories

[Vikram Dixit K (JIRA)]

Vikram Dixit K created HIVE-7116:


 Summary: HDFS FileSystem object cache causes permission issues in 
creating tmp directories
 Key: HIVE-7116
 URL: https://issues.apache.org/jira/browse/HIVE-7116
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, Tez
Affects Versions: 0.13.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K


We change permissions of the directory creation to 777 for HiveServer 2 
operation and it turns out that because of HDFS caching, it does not reflect 
once created. We need to use the non-cached version of the API to get a 
FileSystem object to fix this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-7092) Insert overwrite should not delete the original directory

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-7092:
--

   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Patch committed to trunk. Thanks Szehon for the contribution.

> Insert overwrite should not delete the original directory
> -
>
> Key: HIVE-7092
> URL: https://issues.apache.org/jira/browse/HIVE-7092
> Project: Hive
>  Issue Type: Bug
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Fix For: 0.14.0
>
> Attachments: HIVE-7092.patch
>
>
> Today the implementation of "insert overwrite" table or partition deletes the 
> entire directory of the table/partition recursively using the HDFS shell 
> (-rmr) and then re-creates it.
> This makes it get rid of certain user-set attributes of the directory, like 
> permission, owner, group, and will become more important with introduction of 
> HDFS extended-ACL's.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6684) Beeline does not accept comments that are preceded by spaces

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-6684:
--

   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Patch committed to trunk. Thanks to Jeremy for the contribution.

> Beeline does not accept comments that are preceded by spaces
> 
>
> Key: HIVE-6684
> URL: https://issues.apache.org/jira/browse/HIVE-6684
> Project: Hive
>  Issue Type: Bug
>  Components: CLI
>Affects Versions: 0.10.0
>Reporter: Jeremy Beard
> Fix For: 0.14.0
>
> Attachments: HIVE-6684.1.patch, HIVE-6684.2.patch
>
>
> Beeline throws an error if single-line comments are indented with spaces. 
> This works in the embedded Hive CLI.
> For example:
> SELECT
>-- this is the field we want
>field
> FROM
>table;
> Error: Error while processing statement: FAILED: ParseException line 1:71 
> cannot recognize input near '' '' '' in select clause 
> (state=42000,code=4)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006523#comment-14006523
 ] 

Ashutosh Chauhan commented on HIVE-6936:


+1

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch, HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6936) Provide table properties to InputFormats

[Owen O'Malley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Owen O'Malley updated HIVE-6936:


Attachment: HIVE-6936.patch

Removed the table prefix and added the check to ensure the property wasn't 
already defined in the configuration.

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch, HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6936) Provide table properties to InputFormats

[Owen O'Malley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Owen O'Malley updated HIVE-6936:


Status: Patch Available  (was: Reopened)

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch, HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006512#comment-14006512
 ] 

Ashutosh Chauhan commented on HIVE-6936:


Sounds good to me.

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats

[Owen O'Malley (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006505#comment-14006505
 ] 

Owen O'Malley commented on HIVE-6936:
-

My concern with removing the "table." prefix is that it becomes possible to 
clobber security settings. For example, creating a table with the 
"hive.security.authorization.manager" table property shouldn't allow the query 
to override the setting.

For a compromise, how about removing the table prefix, but we check to see if 
that key has a value in the configuration and won't override such a value. Does 
that make sense?

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HIVE-6936) Provide table properties to InputFormats

[Owen O'Malley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Owen O'Malley updated HIVE-6936:


Attachment: HIVE-6936.patch

Added tab quoting code.

> Provide table properties to InputFormats
> 
>
> Key: HIVE-6936
> URL: https://issues.apache.org/jira/browse/HIVE-6936
> Project: Hive
>  Issue Type: Bug
>  Components: File Formats
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, 
> HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. 
> Additionally, it would be convenient to provide a unique id for fetch 
> operators and the complete list of directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-7099) Add Decimal datatype support for Windowing

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006434#comment-14006434
 ] 

Ashutosh Chauhan commented on HIVE-7099:


+1

> Add Decimal datatype support for Windowing
> --
>
> Key: HIVE-7099
> URL: https://issues.apache.org/jira/browse/HIVE-7099
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-7099.1.patch, HIVE-7099.2.patch
>
>
> Decimal datatype is not handled by Windowing



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: wiki editing permissions

[Ashutosh Chauhan]

Done.


On Thu, May 22, 2014 at 12:57 PM, Eugene Koifman
wrote:

> Hi,
> could I get edit permissions for
> https://cwiki.apache.org/confluence/display/Hive/HCatalog?
>
> my username is 'ekoifman'
>
> --
>
> Thanks,
> Eugene
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

[jira] [Commented] (HIVE-7000) Several issues with javadoc generation

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006405#comment-14006405
 ] 

Ashutosh Chauhan commented on HIVE-7000:


I think its better to take this up either as part of HIVE-6929 or after that 
(if its not addressed there)

> Several issues with javadoc generation
> --
>
> Key: HIVE-7000
> URL: https://issues.apache.org/jira/browse/HIVE-7000
> Project: Hive
>  Issue Type: Improvement
>Reporter: Harish Butani
>Assignee: Harish Butani
> Fix For: 0.14.0
>
> Attachments: HIVE-7000.1.patch
>
>
> 1.
> Ran 'mvn  javadoc:javadoc -Phadoop-2'.  Encountered several issues
> - Generated classes are included in the javadoc
> - generation fails in the top level hcatalog folder because its src folder  
> contains  no java files.
> Patch attached to fix these issues.
> 2.
> Tried mvn javadoc:aggregate -Phadoop-2 
> - cannot get an aggregated javadoc for all of hive
> - tried setting 'aggregate' parameter to true. Didn't work
> There are several questions in StackOverflow about multiple project javadoc. 
> Seems like this is broken. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Reopened] (HIVE-7000) Several issues with javadoc generation

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan reopened HIVE-7000:



Reverted this in HIVE-7084 because it broke webhcat tests. As a result, javadoc 
target doesnt work correctly.

> Several issues with javadoc generation
> --
>
> Key: HIVE-7000
> URL: https://issues.apache.org/jira/browse/HIVE-7000
> Project: Hive
>  Issue Type: Improvement
>Reporter: Harish Butani
>Assignee: Harish Butani
> Fix For: 0.14.0
>
> Attachments: HIVE-7000.1.patch
>
>
> 1.
> Ran 'mvn  javadoc:javadoc -Phadoop-2'.  Encountered several issues
> - Generated classes are included in the javadoc
> - generation fails in the top level hcatalog folder because its src folder  
> contains  no java files.
> Patch attached to fix these issues.
> 2.
> Tried mvn javadoc:aggregate -Phadoop-2 
> - cannot get an aggregated javadoc for all of hive
> - tried setting 'aggregate' parameter to true. Didn't work
> There are several questions in StackOverflow about multiple project javadoc. 
> Seems like this is broken. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: I'd like to change verifyAndSet in HiveConf

[Thejas Nair]

If the verifyAndSet in HiveConf is changed to log a warning instead of
throwing an error, it would mean that any "set
=abc;" would result in a warning instead of an
error.
This would not be the right behavior IMO, a command that did not
succeed should throw an error.

I think a better approach would be to support ability to have a
beeline specific property, which if set does not use the common HIVE
env properties.


On Thu, May 22, 2014 at 9:36 AM, Lars Francke  wrote:
> Hi,
>
> I've just run into a problem with verifyAndSet in HiveConf. It throws
> an exception when someone tries to use a blacklisted property. We've
> blacklisted hive.aux.jars.path amongst others.
>
> But we're using the same environment variables for client and server.
> With Hadoop this is no problem, it logs a warning when one tries to
> override a final parameter but Hive now fails on connection startup
> because the client tries to set hive.aux.jars.path making beeline
> pretty much unusable.
>
> This only happens since
> https://issues.apache.org/jira/browse/HIVE-6340 was fixed as it wasn't
> passed on before.
>
> Is there any good reason why we can't change this behavior and instead
> of throwing an exception just log and ignore?
>
> I'm happy to create an issue and submit a patch just wanted to ask
> first if I'm missing anything.
>
> Cheers,
> Lars

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Updated] (HIVE-7084) TestWebHCatE2e is failing on trunk

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-7084:
---

   Resolution: Fixed
Fix Version/s: 0.14.0
   Status: Resolved  (was: Patch Available)

Committed to trunk.

> TestWebHCatE2e is failing on trunk
> --
>
> Key: HIVE-7084
> URL: https://issues.apache.org/jira/browse/HIVE-7084
> Project: Hive
>  Issue Type: Test
>  Components: WebHCat
>Affects Versions: 0.14.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Fix For: 0.14.0
>
> Attachments: HIVE-7084.patch
>
>
> I am able to repro it consistently on fresh checkout.



--
This message was sent by Atlassian JIRA
(v6.2#6252)