Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-08 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44248/
---

(Updated March 9, 2016, 2:32 a.m.)


Review request for hive and Gopal V.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5098851 
  
llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java
 d33f724 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/Cache.java cee23a9 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCache.java 
17d9fdf 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
1132171 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/NoopCache.java d0461e8 
  llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
d2c1907 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/EncodedDataConsumer.java
 b81e97d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcColumnVectorProducer.java
 18191da 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 2597848 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
 b36cf64 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java
 2e4e0c5 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
 43c8fb3 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
 8479d22 
  orc/src/java/org/apache/orc/FileMetadata.java d63bdcc 
  ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8 
  ql/src/java/org/apache/hadoop/hive/ql/io/SyntheticFileId.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4a27ee7 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
6cec80e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java 
da673a5a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcCacheKey.java 9a0158e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 246ead6 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/ReaderImpl.java b0ac503 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/StreamUtils.java 9ac53af 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
4fafe8c 
  storage-api/src/java/org/apache/hadoop/hive/common/io/DataCache.java 9046589 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 3ef7abe 

Diff: https://reviews.apache.org/r/44248/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-08 Thread Gopal V 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44248/#review122634
---




ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java (line 48)


equals() need to change to handle the long to Long.


- Gopal V


On March 3, 2016, 11:20 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44248/
> ---
> 
> (Updated March 3, 2016, 11:20 p.m.)
> 
> 
> Review request for hive and Gopal V.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java
>  d33f724 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/Cache.java cee23a9 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCache.java 
> 17d9fdf 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
> 1132171 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/NoopCache.java 
> d0461e8 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
> d2c1907 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/EncodedDataConsumer.java
>  b81e97d 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcColumnVectorProducer.java
>  18191da 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  2597848 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
>  b36cf64 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java
>  2e4e0c5 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
>  43c8fb3 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
>  8479d22 
>   orc/src/java/org/apache/orc/FileMetadata.java d63bdcc 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8 
>   ql/src/java/org/apache/hadoop/hive/ql/io/SyntheticFileId.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4a27ee7 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
> 6cec80e 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java 
> da673a5a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcCacheKey.java 
> 9a0158e 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 246ead6 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/ReaderImpl.java 
> b0ac503 
>   ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
> 4fafe8c 
>   storage-api/src/java/org/apache/hadoop/hive/common/io/DataCache.java 
> 9046589 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  3ef7abe 
> 
> Diff: https://reviews.apache.org/r/44248/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-07 Thread Gopal V 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44248/#review122464
---


Fix it, then Ship it!




Comments mostly about mixed use of fileId fileKey - standardize to fileKey 
everywhere?


orc/src/java/org/apache/orc/FileMetadata.java (line 47)


Change to fileKey?



ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
(line 98)


fileKey?


- Gopal V


On March 3, 2016, 11:20 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44248/
> ---
> 
> (Updated March 3, 2016, 11:20 p.m.)
> 
> 
> Review request for hive and Gopal V.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java
>  d33f724 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/Cache.java cee23a9 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCache.java 
> 17d9fdf 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
> 1132171 
>   llap-server/src/java/org/apache/hadoop/hive/llap/cache/NoopCache.java 
> d0461e8 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
> d2c1907 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/EncodedDataConsumer.java
>  b81e97d 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcColumnVectorProducer.java
>  18191da 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  2597848 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
>  b36cf64 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java
>  2e4e0c5 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
>  43c8fb3 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
>  8479d22 
>   orc/src/java/org/apache/orc/FileMetadata.java d63bdcc 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8 
>   ql/src/java/org/apache/hadoop/hive/ql/io/SyntheticFileId.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4a27ee7 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
> 6cec80e 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java 
> da673a5a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcCacheKey.java 
> 9a0158e 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 246ead6 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/ReaderImpl.java 
> b0ac503 
>   ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
> 4fafe8c 
>   storage-api/src/java/org/apache/hadoop/hive/common/io/DataCache.java 
> 9046589 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  3ef7abe 
> 
> Diff: https://reviews.apache.org/r/44248/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-03 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44248/
---

(Updated March 3, 2016, 11:20 p.m.)


Review request for hive and Gopal V.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad 
  
llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java
 d33f724 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/Cache.java cee23a9 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCache.java 
17d9fdf 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
1132171 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/NoopCache.java d0461e8 
  llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
d2c1907 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/EncodedDataConsumer.java
 b81e97d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcColumnVectorProducer.java
 18191da 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 2597848 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
 b36cf64 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java
 2e4e0c5 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
 43c8fb3 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcStripeMetadata.java
 8479d22 
  orc/src/java/org/apache/orc/FileMetadata.java d63bdcc 
  ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8 
  ql/src/java/org/apache/hadoop/hive/ql/io/SyntheticFileId.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4a27ee7 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
6cec80e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java 
da673a5a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcCacheKey.java 9a0158e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 246ead6 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/ReaderImpl.java b0ac503 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
4fafe8c 
  storage-api/src/java/org/apache/hadoop/hive/common/io/DataCache.java 9046589 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 3ef7abe 

Diff: https://reviews.apache.org/r/44248/diff/


Testing
---


Thanks,

Sergey Shelukhin

Review Request 44248: HIVE-12995 LLAP: Synthetic file ids need collision checks

2016-03-01 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44248/
---

Review request for hive and Gopal V.


Repository: hive-git


Description
---

see jira


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/Cache.java cee23a9 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCache.java 
17d9fdf 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheImpl.java 
1132171 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/NoopCache.java d0461e8 
  llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
d2c1907 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/EncodedDataConsumer.java
 b81e97d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcColumnVectorProducer.java
 18191da 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 2597848 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
 b36cf64 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java
 2e4e0c5 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcMetadataCache.java
 43c8fb3 
  orc/src/java/org/apache/orc/FileMetadata.java d63bdcc 
  ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8 
  ql/src/java/org/apache/hadoop/hive/ql/io/SyntheticFileId.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4a27ee7 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java 
6cec80e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcBatchKey.java 
da673a5a 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/OrcCacheKey.java 9a0158e 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 246ead6 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/ReaderImpl.java b0ac503 
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 
4fafe8c 
  storage-api/src/java/org/apache/hadoop/hive/common/io/DataCache.java 9046589 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 3ef7abe 

Diff: https://reviews.apache.org/r/44248/diff/


Testing
---


Thanks,

Sergey Shelukhin