Re: Review Request 69642: HIVE-20977: Lazy evaluate the table object in PreReadTableEvent to improve get_partition performance

2019-01-03 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69642/#review211643
---




standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 4549-4555 (original), 4560-4569 (patched)


Isn't simpler to have a constructor that accepts the (catName, dbName, 
tblName, this) instead of using a Supplier? It will reduce code and make the 
code more readable.


- Sergio Pena


On Jan. 3, 2019, 1:40 a.m., Karthik Manamcheri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69642/
> ---
> 
> (Updated Jan. 3, 2019, 1:40 a.m.)
> 
> 
> Review request for hive, Adam Holley, Na Li, Morio Ramdenbourg, Naveen 
> Gangam, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20977: Lazy evaluate the table object in PreReadTableEvent to improve 
> get_partition performance
> 
> 
> Diffs
> -
> 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  a9398ae1e7 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/events/PreReadTableEvent.java
>  beec72bc12 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/ThrowingSupplier.java
>  PRE-CREATION 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  7429d18226 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
>  fe64a91b56 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestGetPartitions.java
>  4d7f7c1220 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java
>  a338bd4032 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/events/TestPreReadTableEvent.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69642/diff/4/
> 
> 
> Testing
> ---
> 
> Unit tests.
> Manual performance test with Cloudera BDR to notice improved backup 
> performance.
> 
> 
> Thanks,
> 
> Karthik Manamcheri
> 
>

Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side in addition to client-side

2018-12-20 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review211465
---




standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
Lines 1837 (patched)


Is this the only place where is called? Btw, the patch is meant fo the 
server-side only, we could enhance the client-side another patch to keep this 
simpler.



standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
Lines 220-221 (patched)


These two variables are not necessary. To disable the filter, the admin can 
just remove the filter hooks from the configuration and voilà, filter disabled. 
This applies to both sides, server-side and client-side.



standalone-metastore/metastore-server/pom.xml
Lines 242-246 (patched)


Why is this dep necessary? I wonder if this conflicts with the standalone 
solution where all the Hive dependences were removed.



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1405 (patched)


I thought we were going to use the PreReadEvent instead. Also, I'm 
concerned the implementation of filterDatabase does not throw 
NoSuchObjectException, and instead returns null. We should check both cases to 
guarantee get_database and others methods will deny the access to this database.

I think we should use the PreReadEvent which is meant for authorization 
hooks.



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 2934 (patched)


What is this extra space?



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 2910-2911 (original), 2943-2946 (patched)


Why is this change needed? I don't see anything new except splitting it in 
two lines. This will stay in the git history.



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Line 2931 (original), 2966-2970 (patched)


Notice the firePreEvent before the filter. If we use that for authorization 
checks, then the filter is not required here.



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 3050-3053 (patched)


Same question, why splitting the lines where if this patch doesn't need it?



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 3016-3017 (original), 3059-3062 (patched)


Same question, why splitting the lines where if this patch doesn't need it?


- Sergio Pena


On Dec. 19, 2018, 10:50 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> ---
> 
> (Updated Dec. 19, 2018, 10:50 p.m.)
> 
> 
> Review request for hive, Adam Holley, Morio Ramdenbourg, Peter Vary, Sergio 
> Pena, and Vihang Karajgaonkar.
> 
> 
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> add filtering to read result at HMS server, so user cannot see metadata 
> he/she has no privileges. Filtering is enabled/disabled based on 
> configuration.
> 
> 
> Diffs
> -
> 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
>  9eb1193a27120b5167f92daf67bf6a1c4e1d9927 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
>  fb0b2fe6fb9fd4b4c92a6a39f06f39a4641aaabd 
>   standalone-metastore/metastore-server/pom.xml 
> 895abfc423f00b121ee63e40904f5b3e57aea8ed 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  0a1b96dcf62d3536cab2ce074d27a6225b2d3443 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/69585/diff/3/
> 
> 
> Testing
> ---
> 
> Existing unit tests passed. 
> add new unit tests for filtering at HMS server and HMS client
> add code to enabled/disable filtering at HMS client based on configuration
> 
> 
>

Re: Review Request 67186: HIVE-19585: Add UNKNOWN to PrincipalType

2018-05-17 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67186/#review203369
---




standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalType.java
Lines 18 (patched)


This file is auto-generated by Thrift. To add a new field, you need to edit 
the hive_metastore.thrift file and generate the new thrift files.

Btw, this might add a behavior to all the authorization commands like: 
ALTER TABLE ... SET OWNER UNKNOWN 

Do we want to support that? Do we need the UNKNOWN on the PrincipalType?


- Sergio Pena


On May 17, 2018, 3:19 p.m., Arjun Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67186/
> ---
> 
> (Updated May 17, 2018, 3:19 p.m.)
> 
> 
> Review request for hive and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> We need to include type UNKNOWN to PrincipalType to match with 
> HivePrincipal.HivePrincipalType.UKNOWN
> 
> 
> Diffs
> -
> 
>   
> standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalType.java
>  82eb8fd700 
> 
> 
> Diff: https://reviews.apache.org/r/67186/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Arjun Mishra
> 
>

Review Request 67121: HIVE-19527: Preparing for 2.4 development

2018-05-14 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67121/
---

Review request for hive and Peter Vary.


Bugs: HIVE-19527
https://issues.apache.org/jira/browse/HIVE-19527


Repository: hive-git


Description
---

branch-2 is now being used for 2.4 development. 
The build files will need to reflect this change.


Diffs
-

  accumulo-handler/pom.xml f1acb444d9cd4b88ac8c0b9d4b64b8195f5a9b41 
  beeline/pom.xml ea5f9475735c495638eed050f83e81e379afca39 
  cli/pom.xml c59ce3fadaba2fbc74a36b28141d9278185b4cf6 
  common/pom.xml 84bb1e5c93a687be5db21597dedaa2ea9223e852 
  contrib/pom.xml d913fae767490dfff4aa0d66c87735801288de73 
  druid-handler/pom.xml cbf52fc1988945289763d45b20f66137e69913ee 
  hbase-handler/pom.xml 15862dbea23f5eb4b3fed9d7390680206a967f9c 
  hcatalog/core/pom.xml d9054e6a812aa29b33076f62d9af0fd5184de996 
  hcatalog/hcatalog-pig-adapter/pom.xml 
ca93b312cca91c635e650329f8adee8028f08e28 
  hcatalog/pom.xml 713777b8e84d5590a89f89b29e18ce0cb335f750 
  hcatalog/server-extensions/pom.xml 9c28626a1236a00a38df27d2dd8ff123c3eb51a5 
  hcatalog/streaming/pom.xml 9377c3c2a79a47a6906df241b2ce85d062d0 
  hcatalog/webhcat/java-client/pom.xml 90c1c9dd87825037c84115cac506545bcbb8772e 
  hcatalog/webhcat/svr/pom.xml 3aefb92bc29a1f73b97be41277c717115ec77bfb 
  hplsql/pom.xml 6b5b917a254692533c51688caccb65831fe91872 
  itests/custom-serde/pom.xml 7328d92fa96419cc9b6ccf1c9a91cdf5b8900d16 
  itests/custom-udfs/pom.xml c184c1478142e1ff0c62fe9bf4ba129143c98ed6 
  itests/custom-udfs/udf-classloader-udf1/pom.xml 
97b5d40f9c20d023125f81101cd2cf693cf0c756 
  itests/custom-udfs/udf-classloader-udf2/pom.xml 
b0a9aa05ec8fce5b0f363621d43312490144e823 
  itests/custom-udfs/udf-classloader-util/pom.xml 
f77958c8fc34de2fcd7ab1f0858f59de24fb8170 
  itests/custom-udfs/udf-vectorized-badexample/pom.xml 
64f37faa66ec94c7e1d1d24faa2a4b934d44719c 
  itests/hcatalog-unit/pom.xml c5eda767bcbcc7aaf67870e6f8e2a3d20df16a73 
  itests/hive-blobstore/pom.xml e7656d68375019003354d69d43027541ec653347 
  itests/hive-jmh/pom.xml ab7de9f64e3acd1dd88bb263e74da75e6fe9c608 
  itests/hive-minikdc/pom.xml cd457b13e53bbb292c53e43718e997334504da48 
  itests/hive-unit-hadoop2/pom.xml fcdf25448c04b9c20392570b05fc4a0edfdef5fa 
  itests/hive-unit/pom.xml 8790062ff7dfb9c0b011d2289ec847ea3939df55 
  itests/pom.xml 7b215e46c1cab25468e43af369f77a901f448e7f 
  itests/qtest-accumulo/pom.xml c5dbfcab82014e5ae1c599984537d4762611b4ae 
  itests/qtest-spark/pom.xml 049bc54c77b0a1e827ca54f382da882af26f6ab9 
  itests/qtest/pom.xml 7d1dcc690e5ec251bb4ba212035ab388728a2c72 
  itests/test-serde/pom.xml fc03959784ea8e92ec9ac7c2f5128f699ca8904d 
  itests/util/pom.xml 748c2c73ced4dcb1621f6e1a64af867275f9dfb7 
  jdbc-handler/pom.xml 2547265794a2e1eb152c635b6ed22cbaa1a0552e 
  jdbc/pom.xml 1a818c4adfd0bc96cc1c6357264f1f87df821408 
  llap-client/pom.xml a9e5bf186e0c5890d706016a50644869cceccf8f 
  llap-common/pom.xml efd3530bb3e57be6813e544c28c1efe3bbccf7f9 
  llap-ext-client/pom.xml 75ae399c01169dd0bedbccb9e76953adb9fe5026 
  llap-server/pom.xml b97c9830603dc7cc524feb0059c9cb06ce66ff2a 
  llap-tez/pom.xml 08305e5b928c4b044fd17b5591601fb88026c608 
  metastore/pom.xml fddc5ca6784db2eea0a3cb05ae86e6c6c9ec1e22 
  metastore/scripts/upgrade/derby/hive-schema-2.4.0.derby.sql PRE-CREATION 
  metastore/scripts/upgrade/derby/hive-txn-schema-2.4.0.derby.sql PRE-CREATION 
  metastore/scripts/upgrade/derby/upgrade-2.3.0-to-2.4.0.derby.sql PRE-CREATION 
  metastore/scripts/upgrade/derby/upgrade.order.derby 
583bfac44506eee0427d090995d1de2035d59a76 
  metastore/scripts/upgrade/mssql/hive-schema-2.4.0.mssql.sql PRE-CREATION 
  metastore/scripts/upgrade/mssql/upgrade-2.3.0-to-2.4.0.mssql.sql PRE-CREATION 
  metastore/scripts/upgrade/mssql/upgrade.order.mssql 
0bce7eec1d9969203b0dcc1934a65f420b10351a 
  metastore/scripts/upgrade/mysql/hive-schema-2.4.0.mysql.sql PRE-CREATION 
  metastore/scripts/upgrade/mysql/hive-txn-schema-2.4.0.mysql.sql PRE-CREATION 
  metastore/scripts/upgrade/mysql/upgrade-2.3.0-to-2.4.0.mysql.sql PRE-CREATION 
  metastore/scripts/upgrade/mysql/upgrade.order.mysql 
583bfac44506eee0427d090995d1de2035d59a76 
  metastore/scripts/upgrade/oracle/hive-schema-2.4.0.oracle.sql PRE-CREATION 
  metastore/scripts/upgrade/oracle/hive-txn-schema-2.4.0.oracle.sql 
PRE-CREATION 
  metastore/scripts/upgrade/oracle/upgrade-2.3.0-to-2.4.0.oracle.sql 
PRE-CREATION 
  metastore/scripts/upgrade/oracle/upgrade.order.oracle 
624534d068ac971da9621b2dc5635c21690d505f 
  metastore/scripts/upgrade/postgres/hive-schema-2.4.0.postgres.sql 
PRE-CREATION 
  metastore/scripts/upgrade/postgres/hive-txn-schema-2.4.0.postgres.sql 
PRE-CREATION 
  metastore/scripts/upgrade/postgres/upgrade-2.3.0-to-2.4.0.postgres.sql 
PRE-CREATION 
  metastore/scripts/upgrade/postgres/upgrade.order.postgres 
583bfac44506eee0427d090995d1de2035d59a76

Re: Review Request 66979: HIVE-19374: Parse and process ALTER TABLE SET OWNER command syntax

2018-05-08 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66979/
---

(Updated May 8, 2018, 11:57 p.m.)


Review request for hive and Vihang Karajgaonkar.


Changes
---

- Check if ownerType is null on JsonMetaDataFormatted and MetaDataFormatUtils
- Write a test case that verifies the owner and owner type are changed on HMS 
with the alter table set owner command


Bugs: HIVE-19374
https://issues.apache.org/jira/browse/HIVE-19374


Repository: hive-git


Description
---

This patch implements the new ALTER SET ... SET OWNER command and calls the HMS 
api calls to change the owner of the table.

The command syntax is:
> ALTER TABLE  SET OWNER { USER |GROUP |ROLE 
>  }

Currently, Hive sets the owner of a table to the user who created that table. 
With this command, we will be able to change it to another user, group of role 
(as ALTER DATABASE does).

The changes are:
- HiveParser.g which adds the new syntax
- HiveOperation.java which adds the new ALTERTABLE_OWNER operation
- Table.java which gets/sets the owner type
- SemanticAnalyzer.java which returns the DDLSemanticAnalyzer if an 
ALTERTABLE_OWNER operation is detected
- DDLSemanticAnalyzer.java which analyzes the ALTERTABLE_OWNER Operation
- AlterTableDesc.java uses by the DDL semantic analyzer to change the new owner 
information
- MetaDataFormatUtils which displays the owner type when the DESCRIBE command 
is called
- JsonMetaDataFormatted which is another implementation to display the owner 
type in Json format


Diffs (updated)
-

  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/metadata/TestAlterTableMetadata.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
3141a7e981eb35a9fbc7f367f38f8ad420f1f928 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 
879b4224494c3a9adb0713f319e586db4865fb17 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
 cd70eee26c06ee6476964508c54c2bb10b167530 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
 af283e693b5a0fc68e35221b2005fcf1910bdb8e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
defb8becdb5d767ae71d5c962afac43f0c068c3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
a837d67b9615ca1ee359c7aa26f79b6f2504dd99 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388adbc65664ae36b08aaba72943ccb6af 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 
a767796a949da3c23ebe6d8c78b995c8638ebfef 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 
cd4c206a89f1bc1a6195b0f1f39d3c4b462dc027 


Diff: https://reviews.apache.org/r/66979/diff/2/

Changes: https://reviews.apache.org/r/66979/diff/1-2/


Testing
---

Waiting for HiveQA

- alter_table_set_owner.q which verifies that the new command works. Describe 
is not tested because the .q tests files mask the owner information.
- the describe command verified manually in my local hive environment


Thanks,

Sergio Pena

Review Request 66979: HIVE-19374: Parse and process ALTER TABLE SET OWNER command syntax

2018-05-06 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66979/
---

Review request for hive and Vihang Karajgaonkar.


Bugs: HIVE-19374
https://issues.apache.org/jira/browse/HIVE-19374


Repository: hive-git


Description
---

This patch implements the new ALTER SET ... SET OWNER command and calls the HMS 
api calls to change the owner of the table.

The command syntax is:
> ALTER TABLE  SET OWNER { USER |GROUP |ROLE 
>  }

Currently, Hive sets the owner of a table to the user who created that table. 
With this command, we will be able to change it to another user, group of role 
(as ALTER DATABASE does).

The changes are:
- HiveParser.g which adds the new syntax
- HiveOperation.java which adds the new ALTERTABLE_OWNER operation
- Table.java which gets/sets the owner type
- SemanticAnalyzer.java which returns the DDLSemanticAnalyzer if an 
ALTERTABLE_OWNER operation is detected
- DDLSemanticAnalyzer.java which analyzes the ALTERTABLE_OWNER Operation
- AlterTableDesc.java uses by the DDL semantic analyzer to change the new owner 
information
- MetaDataFormatUtils which displays the owner type when the DESCRIBE command 
is called
- JsonMetaDataFormatted which is another implementation to display the owner 
type in Json format


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
3141a7e981eb35a9fbc7f367f38f8ad420f1f928 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 
879b4224494c3a9adb0713f319e586db4865fb17 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
 cd70eee26c06ee6476964508c54c2bb10b167530 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
 af283e693b5a0fc68e35221b2005fcf1910bdb8e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
defb8becdb5d767ae71d5c962afac43f0c068c3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
a837d67b9615ca1ee359c7aa26f79b6f2504dd99 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
820046388adbc65664ae36b08aaba72943ccb6af 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java 
a767796a949da3c23ebe6d8c78b995c8638ebfef 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 
cd4c206a89f1bc1a6195b0f1f39d3c4b462dc027 
  ql/src/test/queries/clientpositive/alter_table_set_owner.q PRE-CREATION 
  ql/src/test/results/clientpositive/alter_table_set_owner.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/66979/diff/1/


Testing
---

Waiting for HiveQA

- alter_table_set_owner.q which verifies that the new command works. Describe 
is not tested because the .q tests files mask the owner information.
- the describe command verified manually in my local hive environment


Thanks,

Sergio Pena

Re: Review Request 66890: HIVE-19371: Add table ownerType to HMS thrift API

2018-05-02 Thread Sergio Pena via Review Board 


> On May 2, 2018, 5:49 p.m., Vihang Karajgaonkar wrote:
> > standalone-metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
> > Lines 1071 (patched)
> > 
> >
> > Is it possible to write a test to make sure this works as expected?

This is Ruby on Rails code. I don't know how to create a test to verify Ruby.
Should we verify it?

Btw, there are other tests in a follow-up patch which implements this Thrift 
API on the ObjectStore.

See https://reviews.apache.org/r/66909/ (needs review too)


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66890/#review202291
---


On May 1, 2018, 8:30 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66890/
> ---
> 
> (Updated May 1, 2018, 8:30 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Bugs: HIVE-19371
> https://issues.apache.org/jira/browse/HIVE-19371
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This is a subtask part of 'HIVE-18762 Support ALTER TABLE SET OWNER command' 
> which adds a new ownerType field on the Table object of the Thrift API of HMS.
> 
> The only file updated (before generting the thrift code) is:
> - hive_metastore.thrift
> 
> The new field is added as 'optional' at the end of the 'struct Table' in 
> order to be backward compatible with older clients of HMS.
> The ownerType filed will be set to a USER as the default value as current 
> tables created are using the owner name as the user.
> 
> 
> Diffs
> -
> 
>   standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
> 47877034746a1cf81cc6731bedb8a6da9cee5b6d 
>   standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
> 629889389e29dcbf9802c348f3b73f6c65695f6f 
>   standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
> c6fadf8a24a32bb208ae6819f159b9b4d5301a2b 
>   
> standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
>  8dfec980d9e14ad01fda27ad97e2e6a91846f782 
>   standalone-metastore/src/gen/thrift/gen-php/metastore/Types.php 
> 9f6cc0e871754afc4fcdbdfb0907d78a0298634c 
>   standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
> 69579e2f593d35c4d52ae7c81d5186d0ae3d379a 
>   standalone-metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
> d7ebaaf914997371201a123f3fc28fac63464ce3 
>   standalone-metastore/src/main/thrift/hive_metastore.thrift 
> ccc3c93bcec88b96e524eeb6d47b7c77ccf49d34 
> 
> 
> Diff: https://reviews.apache.org/r/66890/diff/1/
> 
> 
> Testing
> ---
> 
> Waiting for Hive QA.
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Review Request 66909: HIVE-19372: Add table ownerType to JDO/SQL and ObjectStore

2018-05-02 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66909/
---

Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
Karajgaonkar.


Bugs: HIVE-19372
https://issues.apache.org/jira/browse/HIVE-19372


Repository: hive-git


Description
---

This is a subtask part of 'HIVE-18762 Support ALTER TABLE SET OWNER command' 
which adds a new ownerType field to the JDO and SQL schemas, and does read and 
write the field on the ObjectStore.

The ObjectStore will set the ownerType to USER in case the value read is null 
or empty. This is needed for backward compatibility due to the new schema 
upgrades.

Modify a couple of tests to verify the ownerType is correctly altered to a ROLE 
as well.


Diffs
-

  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 6645e551c2299427c61bcf630caeaf8aae6faa82 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/model/MTable.java
 a38a1250e0ede2b7e294aa1fc326a31fdd8e59f3 
  standalone-metastore/src/main/resources/package.jdo 
221192e37668b4af8dc94add71c148a101f8969e 
  standalone-metastore/src/main/sql/derby/hive-schema-3.0.0.derby.sql 
48d28cb7d8650e988e13950d693fa929849c3b40 
  standalone-metastore/src/main/sql/derby/upgrade-2.3.0-to-3.0.0.derby.sql 
b4c46cd237716b86c3f8dca945b81c6f112969fb 
  standalone-metastore/src/main/sql/mssql/hive-schema-3.0.0.mssql.sql 
4c5739f8b37b1b63b59ea3f7211ac53744654442 
  standalone-metastore/src/main/sql/mssql/upgrade-2.3.0-to-3.0.0.mssql.sql 
b338f0d061815719447f9bf9d641fffab96543c3 
  standalone-metastore/src/main/sql/mysql/hive-schema-3.0.0.mysql.sql 
fd71a1d9b02735770f3885c9cfb67114ce26b542 
  standalone-metastore/src/main/sql/mysql/upgrade-2.3.0-to-3.0.0.mysql.sql 
b0d19271e938752f13125c71689ad59aaa775830 
  standalone-metastore/src/main/sql/oracle/hive-schema-3.0.0.oracle.sql 
a45c7bbb0fa01969bffdf7bd70b8e8162e973d68 
  standalone-metastore/src/main/sql/oracle/upgrade-2.3.0-to-3.0.0.oracle.sql 
0b7c793f16463d35fab3f4f4968bb642421c12e8 
  standalone-metastore/src/main/sql/postgres/hive-schema-3.0.0.postgres.sql 
2484744adfa0b7ba5163d24c37c1d38f30b44054 
  
standalone-metastore/src/main/sql/postgres/upgrade-2.3.0-to-3.0.0.postgres.sql 
f06f0dd34e08bfc271e9acd3fa4dde96c695dd98 
  
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java
 9490586aafac6452c8265a6c0b9410f5aeda2e7d 
  
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
 bdc387e08135102db199776e0b90da2a5616aaf3 


Diff: https://reviews.apache.org/r/66909/diff/1/


Testing
---

Waiting for Hive QA


Thanks,

Sergio Pena

Review Request 66890: HIVE-19371: Add table ownerType to HMS thrift API

2018-05-01 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66890/
---

Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
Karajgaonkar.


Bugs: HIVE-19371
https://issues.apache.org/jira/browse/HIVE-19371


Repository: hive-git


Description
---

This is a subtask part of 'HIVE-18762 Support ALTER TABLE SET OWNER command' 
which adds a new ownerType field on the Table object of the Thrift API of HMS.

The only file updated (before generting the thrift code) is:
- hive_metastore.thrift

The new field is added as 'optional' at the end of the 'struct Table' in order 
to be backward compatible with older clients of HMS.
The ownerType filed will be set to a USER as the default value as current 
tables created are using the owner name as the user.


Diffs
-

  standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
47877034746a1cf81cc6731bedb8a6da9cee5b6d 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
629889389e29dcbf9802c348f3b73f6c65695f6f 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
c6fadf8a24a32bb208ae6819f159b9b4d5301a2b 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
 8dfec980d9e14ad01fda27ad97e2e6a91846f782 
  standalone-metastore/src/gen/thrift/gen-php/metastore/Types.php 
9f6cc0e871754afc4fcdbdfb0907d78a0298634c 
  standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
69579e2f593d35c4d52ae7c81d5186d0ae3d379a 
  standalone-metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
d7ebaaf914997371201a123f3fc28fac63464ce3 
  standalone-metastore/src/main/thrift/hive_metastore.thrift 
ccc3c93bcec88b96e524eeb6d47b7c77ccf49d34 


Diff: https://reviews.apache.org/r/66890/diff/1/


Testing
---

Waiting for Hive QA.


Thanks,

Sergio Pena

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-04-04 Thread Sergio Pena via Review Board 


> On April 4, 2018, 12:34 a.m., Vihang Karajgaonkar wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 408 (patched)
> > 
> >
> > why do we need !success check here?
> 
> Sergio Pena wrote:
> Ah? Thanks, I missed this. I removed the first condition and leave this 
> one instead.
> 
> Vihang Karajgaonkar wrote:
> I thought we need the first condition and not the second condition for 
> !success. Are we not generating the events only when the transaction is 
> successful? Am I missing something?

There is a comment I left to answer that question. I've seen in other 
operations that an event is generated even on failed events. I don't know why 
this is needed. I left the event generation just to be consistent with those 
operations, however, if the event failed, then at least just an ALTER_TABLE 
Event should be generated, and not the DROP/CREATE/ADD partitions events.


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/#review200405
---


On April 4, 2018, 4:06 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65985/
> ---
> 
> (Updated April 4, 2018, 4:06 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Bugs: HIVE-18783
> https://issues.apache.org/jira/browse/HIVE-18783
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
> post-commit listeners for all DDL operations, but it didn't add it to the 
> ALTER TABLE event. This patch in review adds the same behavior for ALTER 
> TABLE events.
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
>  5459554fec911b253583df1f528e5abf134f055b 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
>  ed1b8c5cc2a4cb974dd37419c6b5f0601a035232 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  8539fea42fa2743381833ab3137579caeac64672 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
>  f59f40bc33367cff7a8d0d24d0e200b16c2c30e5 
> 
> 
> Diff: https://reviews.apache.org/r/65985/diff/3/
> 
> 
> Testing
> ---
> 
> All tests passed.
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-04-04 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/
---

(Updated April 4, 2018, 4:06 p.m.)


Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
Karajgaonkar.


Bugs: HIVE-18783
https://issues.apache.org/jira/browse/HIVE-18783


Repository: hive-git


Description
---

HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
post-commit listeners for all DDL operations, but it didn't add it to the ALTER 
TABLE event. This patch in review adds the same behavior for ALTER TABLE events.


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 5459554fec911b253583df1f528e5abf134f055b 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
 ed1b8c5cc2a4cb974dd37419c6b5f0601a035232 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 8539fea42fa2743381833ab3137579caeac64672 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
 f59f40bc33367cff7a8d0d24d0e200b16c2c30e5 


Diff: https://reviews.apache.org/r/65985/diff/3/

Changes: https://reviews.apache.org/r/65985/diff/2-3/


Testing
---

All tests passed.


Thanks,

Sergio Pena

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-04-04 Thread Sergio Pena via Review Board 


> On April 4, 2018, 12:34 a.m., Vihang Karajgaonkar wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 408 (patched)
> > 
> >
> > why do we need !success check here?

Ah? Thanks, I missed this. I removed the first condition and leave this one 
instead.


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/#review200405
---


On March 15, 2018, 5:45 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65985/
> ---
> 
> (Updated March 15, 2018, 5:45 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Bugs: HIVE-18783
> https://issues.apache.org/jira/browse/HIVE-18783
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
> post-commit listeners for all DDL operations, but it didn't add it to the 
> ALTER TABLE event. This patch in review adds the same behavior for ALTER 
> TABLE events.
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
>  e0e29652da94bbdaca515a17955d1409824c1742 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
>  0dd3eb101709969a77998e1488e1c97214426cd3 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  66353e769b2d633ad9dfad2bcae25e8ad90f61d1 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
>  e6de0013bc1be12b2772e2e97102ed476cf5 
> 
> 
> Diff: https://reviews.apache.org/r/65985/diff/2/
> 
> 
> Testing
> ---
> 
> All tests passed.
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-03-15 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/
---

(Updated March 15, 2018, 5:45 p.m.)


Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
Karajgaonkar.


Bugs: HIVE-18783
https://issues.apache.org/jira/browse/HIVE-18783


Repository: hive-git


Description
---

HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
post-commit listeners for all DDL operations, but it didn't add it to the ALTER 
TABLE event. This patch in review adds the same behavior for ALTER TABLE events.


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 e0e29652da94bbdaca515a17955d1409824c1742 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
 0dd3eb101709969a77998e1488e1c97214426cd3 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 66353e769b2d633ad9dfad2bcae25e8ad90f61d1 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
 e6de0013bc1be12b2772e2e97102ed476cf5 


Diff: https://reviews.apache.org/r/65985/diff/2/

Changes: https://reviews.apache.org/r/65985/diff/1-2/


Testing
---

All tests passed.


Thanks,

Sergio Pena

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-03-15 Thread Sergio Pena via Review Board 


> On March 12, 2018, 4:45 p.m., Na Li wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 372 (patched)
> > 
> >
> > why when db name is changed at alter table,  the 
> > transactionalListenerResponses is not past into 
> > MetaStoreListenerNotifier.notifyEvent()? And the command won't be able to 
> > block until sentry gets the notification of alter table.

Good catch. I added other variables for each transaction response.


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/#review199027
---


On March 8, 2018, 4:11 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65985/
> ---
> 
> (Updated March 8, 2018, 4:11 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Bugs: HIVE-18783
> https://issues.apache.org/jira/browse/HIVE-18783
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
> post-commit listeners for all DDL operations, but it didn't add it to the 
> ALTER TABLE event. This patch in review adds the same behavior for ALTER 
> TABLE events.
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
>  e0e29652da94bbdaca515a17955d1409824c1742 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
>  89354a2d34249903a9ff13c4ed913a68de93057e 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  662de9a66767f27f31998f14c68f854e59993ab6 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
>  e6de0013bc1be12b2772e2e97102ed476cf5 
> 
> 
> Diff: https://reviews.apache.org/r/65985/diff/1/
> 
> 
> Testing
> ---
> 
> All tests passed.
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-03-15 Thread Sergio Pena via Review Board 


> On March 13, 2018, 3:35 a.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 362 (patched)
> > 
> >
> > Would it make sense to move this out to its own try/catch block? This 
> > would avoid confusion with transaction try/catch block.

Yes, I moved it.


> On March 13, 2018, 3:35 a.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 365 (patched)
> > 
> >
> > This may cause weird situation where operation succeeds and is 
> > commitetd but throws an exception.

I moved it to its own try/catch to avoid such weird situation.


> On March 13, 2018, 3:35 a.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 366 (patched)
> > 
> >
> > It may be easier to read if you use 
> > 
> > if (listeners.isEmpty()) {
> >   return;
> > }

I'd like to keep the if (!listeners.isEmpty()) in case more code is added in 
the future that should be executed after the finally or this if statement and 
avoid the return if listeners is empty.


> On March 13, 2018, 3:35 a.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
> > Lines 372 (patched)
> > 
> >
> > Why are we creating dropTable/createTable notifications here? I 
> > understand that this mimics the code above, but it looks suspicious.

I left a comment there. The reason is if a rename happens between databases, 
then a drop/create table events between both databases should happen. Honestly, 
I don't think that is necessary and an ALTER event should be enough, but this 
code is already there and I don't know would affect other clients using 
notifications. Perhaps this is something to consider on the new HMS v2 design.


> On March 13, 2018, 3:35 a.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
> > Lines 99 (patched)
> > 
> >
> > There is only a single consumer of this interface - it seems odd to 
> > have an interface which is used in some special case. Should all access to 
> > listeners be converted to the interface?

I don't know. All listeners are used in the HiveMetaStore class which contains 
the HMSHandler subclass. Being listeners and transctionalListeners a variable 
of HMSHandler, then the HiveMetaStore can call them directly without need the 
interface. The only consumer is the HiveAlterHandler which should be re-design 
in my opinion as all DDL operations are inside the HiveMetaStore and only the 
alter is in its own class. Could this also be considered in the HMS v2 design?


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/#review199063
---


On March 8, 2018, 4:11 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65985/
> ---
> 
> (Updated March 8, 2018, 4:11 p.m.)
> 
> 
> Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Bugs: HIVE-18783
> https://issues.apache.org/jira/browse/HIVE-18783
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
> post-commit listeners for all DDL operations, but it didn't add it to the 
> ALTER TABLE event. This patch in review adds the same behavior for ALTER 
> TABLE events.
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
>  e0e29652da94bbdaca515a17955d1409824c1742 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
>  89354a2d34249903a9ff13c4ed913a68de93057e 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  662de9a66767f27f31998f14c68f854e59993ab6 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
>  e6de0013bc1be12b2772e2e97102ed476cf5 
> 
> 
> Diff: https://reviews.apache.org/r/65985/diff/1/
> 
> 
> Testing
> ---
> 
> All tests passed.
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Review Request 65985: HIVE-18783: ALTER TABLE post-commit listener does not include the transactional listener responses

2018-03-08 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65985/
---

Review request for hive, Alexander Kolbasov, Sahil Takiar, and Vihang 
Karajgaonkar.


Bugs: HIVE-18783
https://issues.apache.org/jira/browse/HIVE-18783


Repository: hive-git


Description
---

HIVE-16164 introduced a mechanism to pass HMS notification events ID to the 
post-commit listeners for all DDL operations, but it didn't add it to the ALTER 
TABLE event. This patch in review adds the same behavior for ALTER TABLE events.


Diffs
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 e0e29652da94bbdaca515a17955d1409824c1742 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
 89354a2d34249903a9ff13c4ed913a68de93057e 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 662de9a66767f27f31998f14c68f854e59993ab6 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/IHMSHandler.java
 e6de0013bc1be12b2772e2e97102ed476cf5 


Diff: https://reviews.apache.org/r/65985/diff/1/


Testing
---

All tests passed.


Thanks,

Sergio Pena

Re: Review Request 62816: HIVE-17729 Add Database & Explain related blobstore tests

2017-11-01 Thread Sergio Pena via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62816/#review189798
---


Ship it!




Ship It!

- Sergio Pena


On Oct. 6, 2017, 7:13 p.m., Rentao Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62816/
> ---
> 
> (Updated Oct. 6, 2017, 7:13 p.m.)
> 
> 
> Review request for hive and Sergio Pena.
> 
> 
> Bugs: HIVE-17729
> https://issues.apache.org/jira/browse/HIVE-17729
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-17729 Add Database & Explain related blobstore tests
> 
> 
> This patch introduces the following regression tests into the hive-blobstore 
> qtest module:
> create_database.q -> tests tables with location inherited from database
> multiple_db.q -> tests query spanning multiple databases
> explain.q -> tests EXPLAIN INSERT OVERWRITE command
> 
> 
> Diffs
> -
> 
>   data/files/single_int.txt PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_database.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/explain.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/multiple_db.q 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/create_database.q.out 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/explain.q.out 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/results/clientpositive/multiple_db.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62816/diff/1/
> 
> 
> Testing
> ---
> 
> under /itests/hive-blobstore/
> 
> $ mvn clean test -Dtest=TestBlobstoreCliDriver 
> -Dqfile=create_database.q,explain.q,multiple_db.q
> 
> ---
>  T E S T S
> ---
> Running org.apache.hadoop.hive.cli.TestBlobstoreCliDriver
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.4 sec - in 
> org.apache.hadoop.hive.cli.TestBlobstoreCliDriver
> 
> Results :
> 
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 01:12 min
> [INFO] Finished at: 2017-10-06T11:29:41-07:00
> [INFO] Final Memory: 88M/1003M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Rentao Wu
> 
>