[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[
https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967714#comment-13967714
]
Hive QA commented on HIVE-6891:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639963/HIVE-6891.4.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5619 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.thrift.TestDBTokenStore.testDBTokenStore
org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12639963
> Alter rename partition Perm inheritance and general partition/table group
> inheritance
> -
>
> Key: HIVE-6891
> URL: https://issues.apache.org/jira/browse/HIVE-6891
> Project: Hive
> Issue Type: Bug
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch,
> HIVE-6891.patch
>
>
> Found this issue while looking at the method mentioned by HIVE-6648.
> 'alter table .. partition .. rename to ..' and other commands calling
> Warehouse.mkdirs() doesn't inherit permission on the partition directories
> and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set.
> Also, in these scenarios of directory creation, group is not being inherited.
> Data files are already inheriting group by HIVE-3756.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 20243: HIVE-6891 - Alter rename partition Perm inheritance and general partition/table owner inheritance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20243/#review40220 --- Ship it! +1 pending tests - Brock Noland On April 12, 2014, 11:50 p.m., Szehon Ho wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20243/ > --- > > (Updated April 12, 2014, 11:50 p.m.) > > > Review request for hive. > > > Bugs: HIVE-6891 > https://issues.apache.org/jira/browse/HIVE-6891 > > > Repository: hive-git > > > Description > --- > > This is a follow-up of HIVE-6648. > > Extending the fix to other partition/table operations as well, by refactoring > the fixed code in HIVE-6648 into a common FileUtils helper method, and then > using it for all table/partition directory creation operation, when > "hive.warehouse.subdir.inherit.perms" flag is set. > > Another part of this change is to add ownership inheritance in this code as > well when creating directories. Ownership was already inherited for data > (HIVE-3756), but not at the table/partitioned-table level. > > > Diffs > - > > common/src/java/org/apache/hadoop/hive/common/FileUtils.java ad82f62 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java > 47e94ea > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java > f1c7b7b > metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java > 8345d70 > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > c62e085 > metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java f731dab > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 2559e0e > > Diff: https://reviews.apache.org/r/20243/diff/ > > > Testing > --- > > Extending the unit test TestFolderPermission to handle all the new cases of > directory creation (create table, external table, static partition, dynamic > partition, rename partition). > > Unfortunately due to the test using the local file system, I cannot add the > ownership inheritance to unit-testing. I can probably look into using > MiniDFS for that, in a follow-up JIRA. > > > Thanks, > > Szehon Ho > >
[jira] [Updated] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6891: Attachment: HIVE-6891.4.patch That seems to work, fixing a minor whitespace missed in previous patch and uploading to rb. > Alter rename partition Perm inheritance and general partition/table group > inheritance > - > > Key: HIVE-6891 > URL: https://issues.apache.org/jira/browse/HIVE-6891 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch, > HIVE-6891.patch > > > Found this issue while looking at the method mentioned by HIVE-6648. > 'alter table .. partition .. rename to ..' and other commands calling > Warehouse.mkdirs() doesn't inherit permission on the partition directories > and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set. > Also, in these scenarios of directory creation, group is not being inherited. > Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6891: Attachment: (was: HIVE-6891.4.patch) > Alter rename partition Perm inheritance and general partition/table group > inheritance > - > > Key: HIVE-6891 > URL: https://issues.apache.org/jira/browse/HIVE-6891 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch, > HIVE-6891.patch > > > Found this issue while looking at the method mentioned by HIVE-6648. > 'alter table .. partition .. rename to ..' and other commands calling > Warehouse.mkdirs() doesn't inherit permission on the partition directories > and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set. > Also, in these scenarios of directory creation, group is not being inherited. > Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20243: HIVE-6891 - Alter rename partition Perm inheritance and general partition/table owner inheritance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20243/ --- (Updated April 12, 2014, 11:50 p.m.) Review request for hive. Changes --- This patch passes the test for both hadoop profiles. Making the test use the hadoop-shims, to handle listFile non-existent API in hadoop 0.20. Bugs: HIVE-6891 https://issues.apache.org/jira/browse/HIVE-6891 Repository: hive-git Description --- This is a follow-up of HIVE-6648. Extending the fix to other partition/table operations as well, by refactoring the fixed code in HIVE-6648 into a common FileUtils helper method, and then using it for all table/partition directory creation operation, when "hive.warehouse.subdir.inherit.perms" flag is set. Another part of this change is to add ownership inheritance in this code as well when creating directories. Ownership was already inherited for data (HIVE-3756), but not at the table/partitioned-table level. Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/FileUtils.java ad82f62 itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 47e94ea itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java f1c7b7b metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 8345d70 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c62e085 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java f731dab ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 2559e0e Diff: https://reviews.apache.org/r/20243/diff/ Testing --- Extending the unit test TestFolderPermission to handle all the new cases of directory creation (create table, external table, static partition, dynamic partition, rename partition). Unfortunately due to the test using the local file system, I cannot add the ownership inheritance to unit-testing. I can probably look into using MiniDFS for that, in a follow-up JIRA. Thanks, Szehon Ho
[jira] [Commented] (HIVE-6480) Metastore server startup script ignores ENV settings
[ https://issues.apache.org/jira/browse/HIVE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967657#comment-13967657 ] Lefty Leverenz commented on HIVE-6480: -- Good to know. Thanks, [~sushanth]. > Metastore server startup script ignores ENV settings > > > Key: HIVE-6480 > URL: https://issues.apache.org/jira/browse/HIVE-6480 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: Adam Faris >Assignee: Adam Faris >Priority: Minor > Fix For: 0.14.0 > > Attachments: HIVE-6480.01.patch > > > This is a minor issue with hcat_server.sh. Currently the startup script has > HADOOP_HEAPSIZE hardcoded to 2048, causing administrators to hand edit the > script. As hcat_server.sh reads hcat-env.sh, it makes sense to allow an > administrator to define HADOOP_HEAPSIZE in hcat-env.sh (or other location > like /etc/profile). > Secondly, there is no defined default for METASTORE_PORT in hcat_server.sh. > If METASTORE_PORT is missing, the metastore server fails to start. > I will attach a patch in my next update, once this jira is opened. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[
https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967653#comment-13967653
]
Hive QA commented on HIVE-6891:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639948/HIVE-6891.4.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5619 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12639948
> Alter rename partition Perm inheritance and general partition/table group
> inheritance
> -
>
> Key: HIVE-6891
> URL: https://issues.apache.org/jira/browse/HIVE-6891
> Project: Hive
> Issue Type: Bug
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch,
> HIVE-6891.patch
>
>
> Found this issue while looking at the method mentioned by HIVE-6648.
> 'alter table .. partition .. rename to ..' and other commands calling
> Warehouse.mkdirs() doesn't inherit permission on the partition directories
> and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set.
> Also, in these scenarios of directory creation, group is not being inherited.
> Data files are already inheriting group by HIVE-3756.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5725) Separate out ql code from exec jar
[ https://issues.apache.org/jira/browse/HIVE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967610#comment-13967610 ] Shevek commented on HIVE-5725: -- +1 - we are losing days of development work working around this packaging misfeature. > Separate out ql code from exec jar > -- > > Key: HIVE-5725 > URL: https://issues.apache.org/jira/browse/HIVE-5725 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Owen O'Malley >Assignee: Owen O'Malley > > We should publish our code independently from our dependencies. Since the > exec jar has to include the runtime dependencies, I'd propose that we make > two jars, a ql jar and and exec jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6891: Attachment: HIVE-6891.4.patch Using HadoopShims for listing the files works in both profiles works for me, lets see how far it goes. > Alter rename partition Perm inheritance and general partition/table group > inheritance > - > > Key: HIVE-6891 > URL: https://issues.apache.org/jira/browse/HIVE-6891 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.4.patch, > HIVE-6891.patch > > > Found this issue while looking at the method mentioned by HIVE-6648. > 'alter table .. partition .. rename to ..' and other commands calling > Warehouse.mkdirs() doesn't inherit permission on the partition directories > and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set. > Also, in these scenarios of directory creation, group is not being inherited. > Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967600#comment-13967600 ] Szehon Ho commented on HIVE-6891: - I see, was using hadoop-2 profile to build the test, and pre commit test is using hadoop-1.. > Alter rename partition Perm inheritance and general partition/table group > inheritance > - > > Key: HIVE-6891 > URL: https://issues.apache.org/jira/browse/HIVE-6891 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.patch > > > Found this issue while looking at the method mentioned by HIVE-6648. > 'alter table .. partition .. rename to ..' and other commands calling > Warehouse.mkdirs() doesn't inherit permission on the partition directories > and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set. > Also, in these scenarios of directory creation, group is not being inherited. > Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5847) DatabaseMetadata.getColumns() doesn't show correct column size for char/varchar/decimal
[ https://issues.apache.org/jira/browse/HIVE-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5847: -- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks, Jason. > DatabaseMetadata.getColumns() doesn't show correct column size for > char/varchar/decimal > --- > > Key: HIVE-5847 > URL: https://issues.apache.org/jira/browse/HIVE-5847 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Jason Dere >Assignee: Jason Dere > Fix For: 0.14.0 > > Attachments: HIVE-5847.1.patch, HIVE-5847.1.patch > > > column_size, decimal_digits, num_prec_radix should be set appropriately based > on the type qualifiers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6868) Create table in HCatalog sets different SerDe defaults than what is set through the CLI
[ https://issues.apache.org/jira/browse/HIVE-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967577#comment-13967577 ] Sushanth Sowmyan commented on HIVE-6868: [~ashutoshc], could we get a review on this too? :) > Create table in HCatalog sets different SerDe defaults than what is set > through the CLI > --- > > Key: HIVE-6868 > URL: https://issues.apache.org/jira/browse/HIVE-6868 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Harish Butani > Attachments: HIVE-6868.1.patch, HIVE-6868.2.patch, HIVE-6868.3.patch > > > HCatCreateTableDesc doesn't invoke the getEmptyTable function on > org.apache.hadoop.hive.ql.metadata.Table -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6745) HCat MultiOutputFormat hardcodes DistributedCache keynames
[ https://issues.apache.org/jira/browse/HIVE-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967578#comment-13967578 ] Sushanth Sowmyan commented on HIVE-6745: Note : the failed tests are not connected to this patch. > HCat MultiOutputFormat hardcodes DistributedCache keynames > -- > > Key: HIVE-6745 > URL: https://issues.apache.org/jira/browse/HIVE-6745 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.13.0, 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Attachments: HIVE-6745.patch > > > There's a bug in how MultiOutputFormat deals with DistributedCache, in that > it hardcodes the parameter name to merge for distributed cache entries in the > jobconf. This parameter name has changed with recent builds of 2.x, thus > causing a test failure. These parameters need to be properly shimmed out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6035) Windows: percentComplete returned by job status from WebHCat is null
[ https://issues.apache.org/jira/browse/HIVE-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6035: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Fixed on 0.14 trunk. Thanks, Shanyu! > Windows: percentComplete returned by job status from WebHCat is null > > > Key: HIVE-6035 > URL: https://issues.apache.org/jira/browse/HIVE-6035 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: shanyu zhao >Assignee: shanyu zhao > Fix For: 0.14.0 > > Attachments: HIVE-6035.1.patch, HIVE-6035.2.patch, HIVE-6035.3.patch, > HIVE-6035.patch > > > HIVE-5511 fixed the same problem on Linux, but it still broke on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967574#comment-13967574 ] Sushanth Sowmyan commented on HIVE-6432: (and by that, I mean the failures reported now are unrelated to this patch :D ) > Remove deprecated methods in HCatalog > - > > Key: HIVE-6432 > URL: https://issues.apache.org/jira/browse/HIVE-6432 > Project: Hive > Issue Type: Task > Components: HCatalog >Affects Versions: 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Fix For: 0.14.0 > > Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, > HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch, > HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out > > > There are a lot of methods in HCatalog that have been deprecated in HCatalog > 0.5, and some that were recently deprecated in Hive 0.11 (joint release with > HCatalog). > The goal for HCatalog deprecation is that in general, after something has > been deprecated, it is expected to stay around for 2 releases, which means > hive-0.13 will be the last release to ship with all the methods that were > deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be > removed afterwards), and it is also good for us to clean out and nuke all > other older deprecated methods. > We should take this on early in a dev/release cycle to allow us time to > resolve all fallout, so I propose that we remove all HCatalog deprecated > methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967573#comment-13967573 ] Sushanth Sowmyan commented on HIVE-6432: Yay, fifth time's the charm. > Remove deprecated methods in HCatalog > - > > Key: HIVE-6432 > URL: https://issues.apache.org/jira/browse/HIVE-6432 > Project: Hive > Issue Type: Task > Components: HCatalog >Affects Versions: 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Fix For: 0.14.0 > > Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, > HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch, > HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out > > > There are a lot of methods in HCatalog that have been deprecated in HCatalog > 0.5, and some that were recently deprecated in Hive 0.11 (joint release with > HCatalog). > The goal for HCatalog deprecation is that in general, after something has > been deprecated, it is expected to stay around for 2 releases, which means > hive-0.13 will be the last release to ship with all the methods that were > deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be > removed afterwards), and it is also good for us to clean out and nuke all > other older deprecated methods. > We should take this on early in a dev/release cycle to allow us time to > resolve all fallout, so I propose that we remove all HCatalog deprecated > methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6432) Remove deprecated methods in HCatalog
[
https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967524#comment-13967524
]
Hive QA commented on HIVE-6432:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639928/HIVE-6432.5.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5401 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2236/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2236/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12639928
> Remove deprecated methods in HCatalog
> -
>
> Key: HIVE-6432
> URL: https://issues.apache.org/jira/browse/HIVE-6432
> Project: Hive
> Issue Type: Task
> Components: HCatalog
>Affects Versions: 0.14.0
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
> Fix For: 0.14.0
>
> Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch,
> HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch,
> HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out
>
>
> There are a lot of methods in HCatalog that have been deprecated in HCatalog
> 0.5, and some that were recently deprecated in Hive 0.11 (joint release with
> HCatalog).
> The goal for HCatalog deprecation is that in general, after something has
> been deprecated, it is expected to stay around for 2 releases, which means
> hive-0.13 will be the last release to ship with all the methods that were
> deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be
> removed afterwards), and it is also good for us to clean out and nuke all
> other older deprecated methods.
> We should take this on early in a dev/release cycle to allow us time to
> resolve all fallout, so I propose that we remove all HCatalog deprecated
> methods after we branch out 0.13 and 0.14 becomes trunk.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[
https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967496#comment-13967496
]
Hive QA commented on HIVE-6891:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639923/HIVE-6891.3.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/console
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @
hive-hcatalog-it-unit ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit ---
[INFO] Building jar:
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
hive-hcatalog-it-unit ---
[INFO]
[INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit ---
[INFO] Building jar:
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
hive-hcatalog-it-unit ---
[INFO] Installing
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
[INFO] Installing
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom
[INFO] Installing
/data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
[INFO]
[INFO]
[INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util
---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
hive-it-util ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util
---
[INFO] Compiling 45 source files to
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
hive-it-util ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
[copy] Copying 5 files to
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-it-util ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util ---
[INFO] Building jar:
/data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-d
[jira] [Commented] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967493#comment-13967493
]
Hive QA commented on HIVE-6901:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639911/HIVE-6901.patch
{color:red}ERROR:{color} -1 due to 476 failed/errored test(s), 5615 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alias_casted_column
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ambiguous_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ansi_sql_arithmetic
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join0
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join18_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join27
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join28
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join29
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_without_localtask
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binarysortable_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
org.apache.hadoop.hive.cli.TestCliDriver.testC
[jira] [Commented] (HIVE-6035) Windows: percentComplete returned by job status from WebHCat is null
[
https://issues.apache.org/jira/browse/HIVE-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967472#comment-13967472
]
Hive QA commented on HIVE-6035:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639904/HIVE-6035.3.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5614 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2232/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2232/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12639904
> Windows: percentComplete returned by job status from WebHCat is null
>
>
> Key: HIVE-6035
> URL: https://issues.apache.org/jira/browse/HIVE-6035
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.12.0
>Reporter: shanyu zhao
>Assignee: shanyu zhao
> Attachments: HIVE-6035.1.patch, HIVE-6035.2.patch, HIVE-6035.3.patch,
> HIVE-6035.patch
>
>
> HIVE-5511 fixed the same problem on Linux, but it still broke on Windows.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6432: --- Status: Patch Available (was: Open) > Remove deprecated methods in HCatalog > - > > Key: HIVE-6432 > URL: https://issues.apache.org/jira/browse/HIVE-6432 > Project: Hive > Issue Type: Task > Components: HCatalog >Affects Versions: 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Fix For: 0.14.0 > > Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, > HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch, > HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out > > > There are a lot of methods in HCatalog that have been deprecated in HCatalog > 0.5, and some that were recently deprecated in Hive 0.11 (joint release with > HCatalog). > The goal for HCatalog deprecation is that in general, after something has > been deprecated, it is expected to stay around for 2 releases, which means > hive-0.13 will be the last release to ship with all the methods that were > deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be > removed afterwards), and it is also good for us to clean out and nuke all > other older deprecated methods. > We should take this on early in a dev/release cycle to allow us time to > resolve all fallout, so I propose that we remove all HCatalog deprecated > methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6432: --- Attachment: HIVE-6432.5.patch One more attempt, this should catch all the stragglers. > Remove deprecated methods in HCatalog > - > > Key: HIVE-6432 > URL: https://issues.apache.org/jira/browse/HIVE-6432 > Project: Hive > Issue Type: Task > Components: HCatalog >Affects Versions: 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Fix For: 0.14.0 > > Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, > HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.5.patch, HIVE-6432.patch, > HIVE-6432.wip.1.patch, HIVE-6432.wip.2.patch, hcat.6432.test.out > > > There are a lot of methods in HCatalog that have been deprecated in HCatalog > 0.5, and some that were recently deprecated in Hive 0.11 (joint release with > HCatalog). > The goal for HCatalog deprecation is that in general, after something has > been deprecated, it is expected to stay around for 2 releases, which means > hive-0.13 will be the last release to ship with all the methods that were > deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be > removed afterwards), and it is also good for us to clean out and nuke all > other older deprecated methods. > We should take this on early in a dev/release cycle to allow us time to > resolve all fallout, so I propose that we remove all HCatalog deprecated > methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6432) Remove deprecated methods in HCatalog
[ https://issues.apache.org/jira/browse/HIVE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-6432: --- Status: Open (was: Patch Available) > Remove deprecated methods in HCatalog > - > > Key: HIVE-6432 > URL: https://issues.apache.org/jira/browse/HIVE-6432 > Project: Hive > Issue Type: Task > Components: HCatalog >Affects Versions: 0.14.0 >Reporter: Sushanth Sowmyan >Assignee: Sushanth Sowmyan > Fix For: 0.14.0 > > Attachments: 6432-addendum.patch, 6432-full.patch, HIVE-6432.2.patch, > HIVE-6432.3.patch, HIVE-6432.4.patch, HIVE-6432.patch, HIVE-6432.wip.1.patch, > HIVE-6432.wip.2.patch, hcat.6432.test.out > > > There are a lot of methods in HCatalog that have been deprecated in HCatalog > 0.5, and some that were recently deprecated in Hive 0.11 (joint release with > HCatalog). > The goal for HCatalog deprecation is that in general, after something has > been deprecated, it is expected to stay around for 2 releases, which means > hive-0.13 will be the last release to ship with all the methods that were > deprecated in hive-0.11 (the org.apache.hcatalog.* files should all be > removed afterwards), and it is also good for us to clean out and nuke all > other older deprecated methods. > We should take this on early in a dev/release cycle to allow us time to > resolve all fallout, so I propose that we remove all HCatalog deprecated > methods after we branch out 0.13 and 0.14 becomes trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6480) Metastore server startup script ignores ENV settings
[ https://issues.apache.org/jira/browse/HIVE-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967452#comment-13967452 ] Sushanth Sowmyan commented on HIVE-6480: Shouldn't need any userdoc, this simply changes the semantic to a "set if not already set" mode rather than a blind set. Most people will not even run this script, since the bigtop installations of hive won't use this script to start the metastore - this is a leftover script from the days hcat was a separate project - it's use now is primarily so devs can use for testing the metastore without too much setup. > Metastore server startup script ignores ENV settings > > > Key: HIVE-6480 > URL: https://issues.apache.org/jira/browse/HIVE-6480 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 0.12.0 >Reporter: Adam Faris >Assignee: Adam Faris >Priority: Minor > Fix For: 0.14.0 > > Attachments: HIVE-6480.01.patch > > > This is a minor issue with hcat_server.sh. Currently the startup script has > HADOOP_HEAPSIZE hardcoded to 2048, causing administrators to hand edit the > script. As hcat_server.sh reads hcat-env.sh, it makes sense to allow an > administrator to define HADOOP_HEAPSIZE in hcat-env.sh (or other location > like /etc/profile). > Secondly, there is no defined default for METASTORE_PORT in hcat_server.sh. > If METASTORE_PORT is missing, the metastore server fails to start. > I will attach a patch in my next update, once this jira is opened. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5336) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the fieldPositionMap and the fieldPositionMap should not be cached by the end user
[ https://issues.apache.org/jira/browse/HIVE-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-5336: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the > fieldPositionMap and the fieldPositionMap should not be cached by the end user > -- > > Key: HIVE-5336 > URL: https://issues.apache.org/jira/browse/HIVE-5336 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Fix For: 0.14.0 > > Attachments: HIVE-5336.1.patch.txt, HIVE-5336.2.patch.txt, > HIVE-5336.3.patch > > > HCatSchema.remove currently does not renumber the fieldPositionMap which can > be a problem when there are interleaving append() and remove() calls. > 1. We should document that fieldPositionMap should not be cached by the > end-user > 2. We should make sure that the fieldPositionMap gets renumbered after > remove() because HcatSchema.get will otherwise return wrong FieldSchemas. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5336) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the fieldPositionMap and the fieldPositionMap should not be cached by the end user
[ https://issues.apache.org/jira/browse/HIVE-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967451#comment-13967451 ] Sushanth Sowmyan commented on HIVE-5336: Committed to trunk. Thanks for the fix, Hari! > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the > fieldPositionMap and the fieldPositionMap should not be cached by the end user > -- > > Key: HIVE-5336 > URL: https://issues.apache.org/jira/browse/HIVE-5336 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Fix For: 0.14.0 > > Attachments: HIVE-5336.1.patch.txt, HIVE-5336.2.patch.txt, > HIVE-5336.3.patch > > > HCatSchema.remove currently does not renumber the fieldPositionMap which can > be a problem when there are interleaving append() and remove() calls. > 1. We should document that fieldPositionMap should not be cached by the > end-user > 2. We should make sure that the fieldPositionMap gets renumbered after > remove() because HcatSchema.get will otherwise return wrong FieldSchemas. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5336) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the fieldPositionMap and the fieldPositionMap should not be cached by the end user
[ https://issues.apache.org/jira/browse/HIVE-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967450#comment-13967450 ] Sushanth Sowmyan commented on HIVE-5336: The failed tests are not related to this issue, I can go ahead and commit. > HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the > fieldPositionMap and the fieldPositionMap should not be cached by the end user > -- > > Key: HIVE-5336 > URL: https://issues.apache.org/jira/browse/HIVE-5336 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-5336.1.patch.txt, HIVE-5336.2.patch.txt, > HIVE-5336.3.patch > > > HCatSchema.remove currently does not renumber the fieldPositionMap which can > be a problem when there are interleaving append() and remove() calls. > 1. We should document that fieldPositionMap should not be cached by the > end-user > 2. We should make sure that the fieldPositionMap gets renumbered after > remove() because HcatSchema.get will otherwise return wrong FieldSchemas. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5336) HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the fieldPositionMap and the fieldPositionMap should not be cached by the end user
[
https://issues.apache.org/jira/browse/HIVE-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967448#comment-13967448
]
Hive QA commented on HIVE-5336:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12639900/HIVE-5336.3.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5615 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_map_operators
org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2231/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2231/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12639900
> HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the
> fieldPositionMap and the fieldPositionMap should not be cached by the end user
> --
>
> Key: HIVE-5336
> URL: https://issues.apache.org/jira/browse/HIVE-5336
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-5336.1.patch.txt, HIVE-5336.2.patch.txt,
> HIVE-5336.3.patch
>
>
> HCatSchema.remove currently does not renumber the fieldPositionMap which can
> be a problem when there are interleaving append() and remove() calls.
> 1. We should document that fieldPositionMap should not be cached by the
> end-user
> 2. We should make sure that the fieldPositionMap gets renumbered after
> remove() because HcatSchema.get will otherwise return wrong FieldSchemas.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6891) Alter rename partition Perm inheritance and general partition/table group inheritance
[ https://issues.apache.org/jira/browse/HIVE-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-6891: Attachment: HIVE-6891.3.patch Fixing TestHiveMetaStore > Alter rename partition Perm inheritance and general partition/table group > inheritance > - > > Key: HIVE-6891 > URL: https://issues.apache.org/jira/browse/HIVE-6891 > Project: Hive > Issue Type: Bug >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-6891.2.patch, HIVE-6891.3.patch, HIVE-6891.patch > > > Found this issue while looking at the method mentioned by HIVE-6648. > 'alter table .. partition .. rename to ..' and other commands calling > Warehouse.mkdirs() doesn't inherit permission on the partition directories > and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set. > Also, in these scenarios of directory creation, group is not being inherited. > Data files are already inheriting group by HIVE-3756. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20243: HIVE-6891 - Alter rename partition Perm inheritance and general partition/table owner inheritance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20243/ --- (Updated April 12, 2014, 7:59 a.m.) Review request for hive. Changes --- Fixing a unit test, that used Warehouse.mkdir(). Bugs: HIVE-6891 https://issues.apache.org/jira/browse/HIVE-6891 Repository: hive-git Description --- This is a follow-up of HIVE-6648. Extending the fix to other partition/table operations as well, by refactoring the fixed code in HIVE-6648 into a common FileUtils helper method, and then using it for all table/partition directory creation operation, when "hive.warehouse.subdir.inherit.perms" flag is set. Another part of this change is to add ownership inheritance in this code as well when creating directories. Ownership was already inherited for data (HIVE-3756), but not at the table/partitioned-table level. Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/FileUtils.java ad82f62 itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 47e94ea itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java f1c7b7b metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 8345d70 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c62e085 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java f731dab ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 2559e0e Diff: https://reviews.apache.org/r/20243/diff/ Testing --- Extending the unit test TestFolderPermission to handle all the new cases of directory creation (create table, external table, static partition, dynamic partition, rename partition). Unfortunately due to the test using the local file system, I cannot add the ownership inheritance to unit-testing. I can probably look into using MiniDFS for that, in a follow-up JIRA. Thanks, Szehon Ho
