Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-22 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review175684
---


Ship it!




Ship It!

- Daniel Dai


On May 22, 2017, 7:06 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 22, 2017, 7:06 p.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> b28983f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
> fcf6f27 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 1cc838f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
>  da6cd46 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
>  PRE-CREATION 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/19/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-22 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 22, 2017, 7:06 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/19/

Changes: https://reviews.apache.org/r/59205/diff/18-19/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-22 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 22, 2017, 9:04 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7dedd23 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/18/

Changes: https://reviews.apache.org/r/59205/diff/17-18/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-21 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 21, 2017, 7:12 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Changes
---

Patch v16 from jira


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/17/

Changes: https://reviews.apache.org/r/59205/diff/16-17/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-19 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 19, 2017, 8:41 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/16/

Changes: https://reviews.apache.org/r/59205/diff/15-16/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-18 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 19, 2017, 2:38 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/15/

Changes: https://reviews.apache.org/r/59205/diff/14-15/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-18 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 18, 2017, 7:14 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/14/

Changes: https://reviews.apache.org/r/59205/diff/13-14/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-18 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review175354
---




metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Lines 827 (patched)


There is no need to alter partition/partition stats/table stats here. 
HiveAlterHandler will invoke alterPartition, updateTableColumnStatistics and 
updatePartitionColumnStatistics respectively.


- Daniel Dai


On May 17, 2017, 6:47 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 17, 2017, 6:47 p.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> b28983f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
> fcf6f27 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 1cc838f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
>  da6cd46 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
>  PRE-CREATION 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/13/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-17 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 17, 2017, 6:47 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/13/

Changes: https://reviews.apache.org/r/59205/diff/12-13/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-17 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 17, 2017, 8:24 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/12/

Changes: https://reviews.apache.org/r/59205/diff/11-12/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-17 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 17, 2017, 7:49 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/11/

Changes: https://reviews.apache.org/r/59205/diff/10-11/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-16 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 16, 2017, 5:59 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/10/

Changes: https://reviews.apache.org/r/59205/diff/9-10/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-16 Thread Thejas Nair 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review175116
---




metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
Line 266 (original)


don't we need this removePartitionColStatsFromCache call ?


- Thejas Nair


On May 16, 2017, 8:10 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 16, 2017, 8:10 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> b28983f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
> fcf6f27 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 1cc838f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
>  da6cd46 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
>  PRE-CREATION 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/9/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-16 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 16, 2017, 8:10 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1c37b6e 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d296851 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java b28983f 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
1cc838f 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
 da6cd46 
  
metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
 PRE-CREATION 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/9/

Changes: https://reviews.apache.org/r/59205/diff/8-9/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 16, 2017, 3:27 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Changes
---

Please ignore v8 on review board - generated incorrectly.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6068f0d 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d408321 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/8/

Changes: https://reviews.apache.org/r/59205/diff/7-8/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 16, 2017, 3:25 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6068f0d 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 6ff48ee 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/src/test/resources/testconfiguration.properties a378a5d 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
  itests/util/src/main/java/org/apache/hive/beeline/QFile.java 3d9ca99 
  itests/util/src/main/java/org/apache/hive/beeline/QFileBeeLineClient.java 
7c50e18 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
fcf6f27 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java f01efa5 
  ql/src/java/org/apache/hadoop/hive/ql/QueryState.java 6dfaa9f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java cf575de 
  ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 
77bce97 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
 b121eea 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsAutoGatherContext.java 
3b719af 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java c7266bc 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java 
c734988 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBCompact.java 201622e 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBJoinTreeApplyPredicate.java 
e607f10 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBSubQuery.java 2674835 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestReplicationSemanticAnalyzer.java
 80865bd 
  ql/src/test/org/apache/hadoop/hive/ql/parse/TestSemanticAnalyzerFactory.java 
5849950 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestUpdateDeleteSemanticAnalyzer.java
 a573808 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/authorization/TestHiveAuthorizationTaskFactory.java
 58cb4b4 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/authorization/TestPrivilegesV1.java 
5d01080 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/authorization/TestPrivilegesV2.java 
c552ba7 
  
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
 PRE-CREATION 
  service/src/java/org/apache/hive/service/cli/operation/Operation.java 0b27608 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
0b51591 


Diff: https://reviews.apache.org/r/59205/diff/7/

Changes: https://reviews.apache.org/r/59205/diff/6-7/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 15, 2017, 10:27 p.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6068f0d 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java d408321 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/6/

Changes: https://reviews.apache.org/r/59205/diff/5-6/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Thejas Nair 


> On May 15, 2017, 8:19 a.m., Thejas Nair wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
> > Line 281 (original), 284 (patched)
> > 
> >
> > This needs to be TimeUnit.MILLISECONDS, as you want it converted to 
> > MILLISECONDS

the signature is - getTimeVar(Configuration conf, ConfVars var, TimeUnit 
outUnit)


- Thejas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174936
---


On May 15, 2017, 7:59 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 15, 2017, 7:59 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Thejas Nair 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174936
---




metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 281 (original), 284 (patched)


This needs to be TimeUnit.MILLISECONDS, as you want it converted to 
MILLISECONDS



metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 512 (original), 526 (patched)


isDatabaseCacheDirty.set(true); - this needs to be set after obtaining the 
lock

Otherwise, you can have case where it gets reset by background thread 
before actual update.

(Same with this call in other places)


- Thejas Nair


On May 15, 2017, 7:59 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 15, 2017, 7:59 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-15 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 15, 2017, 7:59 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/5/

Changes: https://reviews.apache.org/r/59205/diff/4-5/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Daniel Dai 


> On May 14, 2017, 7:38 a.m., Daniel Dai wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
> > Lines 528 (patched)
> > 
> >
> > If using dirty flag, databaseCacheLock can be removed, right?
> 
> Thejas Nair wrote:
> We still need some locking/synchronization to prevent things like this -
> 
> 
> 1. If (isDirtyFlag == false) {
> 2.  // if db gets updated in another thread while this thread is here, we 
> lose the update.
> 3.updateDBCache()
> 2. }
> 
> A read/write lock similar to what is used here appropriate for that. The 
> background thread is the one that holds the write lock.
> However, not sure if we need to busy-wait with trylock though. (I figure, 
> the intent is to give this one higher priority). The update DbCache part 
> should not take that long. (The busy-wait can be dealt with later also).

A simpler solution is move dirtyFlag to SharedCache. 
SharedCache.refreshDatabases will check the flag and 
SharedCache.addDatabaseToCache/alterDatabaseInCache/removeDatabaseFromCache 
will set the flag. There is no synchronization issue in SharedCache since every 
method is synchronized.


- Daniel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174907
---


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Thejas Nair 


> On May 14, 2017, 7:38 a.m., Daniel Dai wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
> > Line 381 (original), 526 (patched)
> > 
> >
> > I think we only need a dirty flag, it will be reset by the next 
> > refresh, not end of individual call.

I agree, this activeDatabaseUpdateCalls params are not going to prevent the 
race-condition. Its better to use dirty flag instead.


- Thejas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174907
---


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Thejas Nair 


> On May 14, 2017, 7:38 a.m., Daniel Dai wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
> > Lines 528 (patched)
> > 
> >
> > If using dirty flag, databaseCacheLock can be removed, right?

We still need some locking/synchronization to prevent things like this -


1. If (isDirtyFlag == false) {
2.  // if db gets updated in another thread while this thread is here, we lose 
the update.
3.updateDBCache()
2. }

A read/write lock similar to what is used here appropriate for that. The 
background thread is the one that holds the write lock.
However, not sure if we need to busy-wait with trylock though. (I figure, the 
intent is to give this one higher priority). The update DbCache part should not 
take that long. (The busy-wait can be dealt with later also).


- Thejas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174907
---


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Thejas Nair 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174906
---




metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Lines 341 (patched)


this call has already been made in updateTables. Should we just get it from 
the cache to reduce the rawStore call ? This can be done as part of follow up 
jira if necessary.



metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 300 (original), 374 (patched)


This still doesn't prevent the race condition I mentioned (in offline 
discussion).
If another thread calls a method that updates a database object, when this 
thread is here (let say that calls starts and finishes when this thread is at 
this line), it would still go ahead and update the cache. That would result in 
local changes getting lost.

This is not a regression in this patch, we can address that in a follow up 
patch.
To address that, we need a dirty-cache check after acquring the lock.



metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 1131 (original), 1433 (patched)


this would be just like getPartition() . We get it from cache, if its there 
in cache. Otherwise, assume its not there (yet). This can be done as part of 
follow up jira if necessary.


- Thejas Nair


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Daniel Dai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174907
---




metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Line 381 (original), 526 (patched)


I think we only need a dirty flag, it will be reset by the next refresh, 
not end of individual call.



metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
Lines 528 (patched)


If using dirty flag, databaseCacheLock can be removed, right?


- Daniel Dai


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-14 Thread Thejas Nair 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174905
---




metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
Lines 1387 (patched)


There is no order-by in the SQL query above. Is it guaranteed that the list 
is ordered ?
Should we add a "order by" to above SQL query ?


- Thejas Nair


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 5a187d8 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> 0c7d8bb 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  f613c30 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  1720e37 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-13 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 14, 2017, 5:19 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/4/

Changes: https://reviews.apache.org/r/59205/diff/3-4/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-13 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 14, 2017, 2:21 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/3/

Changes: https://reviews.apache.org/r/59205/diff/2-3/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-13 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/
---

(Updated May 14, 2017, 1:28 a.m.)


Review request for hive, Daniel Dai and Thejas Nair.


Bugs: HIVE-16579
https://issues.apache.org/jira/browse/HIVE-16579


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-16579


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
 91a3a38 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b96c27e 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
870896c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
668499b 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
5a187d8 
  metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
7beee42 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
0c7d8bb 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 f613c30 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 1720e37 
  
metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 
0ab20d6 


Diff: https://reviews.apache.org/r/59205/diff/2/

Changes: https://reviews.apache.org/r/59205/diff/1-2/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

2017-05-12 Thread Thejas Nair 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174841
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Line 906 (original), 906 (patched)


Lets keep default time in seconds, just as before.


- Thejas Nair


On May 11, 2017, 9:07 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> ---
> 
> (Updated May 11, 2017, 9:07 p.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 73e0290 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
>  88b9faf 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> b96c27e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> a83e12e 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c22a1db 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 
> b438479 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 
> 39b1676 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 
> 7beee42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
> f6420f5 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
>  3e3fd20 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
>  91d8c2a 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java
>  0ab20d6 
> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>