Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Matt McCline 


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > 
> >
> > You need braces around your control block for break.
> > 
> > Actually, this is an error to have an include vector that is shorter 
> > that the reader type and should cause a thrown exception. Although it 
> > should be a single check that include.length == readerSchema.getMaxId() + 1.
> 
> Sergey Shelukhin wrote:
> as far as I remember, I've seen vectorization(?) use trimmed includes 
> somewhere
> 
> Owen O'Malley wrote:
> Nope. The include vector has to exactly match the reader schema, although 
> it doesn't include the ACID columns. In fact, the current version of 
> SchemaEvolution in ORC will throw in such cases.
> 
> Sergey Shelukhin wrote:
> See e.g. VectorMapOperator - determineDataColumnsToIncludeTruncated
> 
> Owen O'Malley wrote:
> If that boolean array is passed down to an ORC reader (or directly to 
> SchemaEvolution), it will fail by design.

dataColumnsToIncludeTruncated is designed to be used with Text 
(LazySimpleDeserializeRead class) to minimized the number of text fields that 
have to be parsed.  From a recommendation by Gopal who observed a bunch of text 
fields being parsed that didn't need to be.

Schema Evolution does work with vectorized text because the Vectorizer class 
currently insists the partition names match the table schema (a restricted 
positional requirement).

dataColumnsToIncludeTruncated is not used with ORC.


- Matt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---


On Feb. 8, 2017, 1:10 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 8, 2017, 1:10 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f3b01b2 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Feb. 8, 2017, 1:10 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f3b01b2 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 4295c1c 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 8d86d17 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 63a3be2 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 a434763 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 bbd9ca6 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Owen O'Malley 


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > 
> >
> > You need braces around your control block for break.
> > 
> > Actually, this is an error to have an include vector that is shorter 
> > that the reader type and should cause a thrown exception. Although it 
> > should be a single check that include.length == readerSchema.getMaxId() + 1.
> 
> Sergey Shelukhin wrote:
> as far as I remember, I've seen vectorization(?) use trimmed includes 
> somewhere
> 
> Owen O'Malley wrote:
> Nope. The include vector has to exactly match the reader schema, although 
> it doesn't include the ACID columns. In fact, the current version of 
> SchemaEvolution in ORC will throw in such cases.
> 
> Sergey Shelukhin wrote:
> See e.g. VectorMapOperator - determineDataColumnsToIncludeTruncated

If that boolean array is passed down to an ORC reader (or directly to 
SchemaEvolution), it will fail by design.


- Owen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---


On Feb. 7, 2017, 6:35 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 7, 2017, 6:35 p.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Sergey Shelukhin 


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > 
> >
> > You need braces around your control block for break.
> > 
> > Actually, this is an error to have an include vector that is shorter 
> > that the reader type and should cause a thrown exception. Although it 
> > should be a single check that include.length == readerSchema.getMaxId() + 1.
> 
> Sergey Shelukhin wrote:
> as far as I remember, I've seen vectorization(?) use trimmed includes 
> somewhere
> 
> Owen O'Malley wrote:
> Nope. The include vector has to exactly match the reader schema, although 
> it doesn't include the ACID columns. In fact, the current version of 
> SchemaEvolution in ORC will throw in such cases.

See e.g. VectorMapOperator - determineDataColumnsToIncludeTruncated


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---


On Feb. 7, 2017, 6:35 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 7, 2017, 6:35 p.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Owen O'Malley 


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > 
> >
> > You need braces around your control block for break.
> > 
> > Actually, this is an error to have an include vector that is shorter 
> > that the reader type and should cause a thrown exception. Although it 
> > should be a single check that include.length == readerSchema.getMaxId() + 1.
> 
> Sergey Shelukhin wrote:
> as far as I remember, I've seen vectorization(?) use trimmed includes 
> somewhere

Nope. The include vector has to exactly match the reader schema, although it 
doesn't include the ACID columns. In fact, the current version of 
SchemaEvolution in ORC will throw in such cases.


- Owen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---


On Feb. 7, 2017, 6:35 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 7, 2017, 6:35 p.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Feb. 7, 2017, 6:35 p.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cb27cd6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 4295c1c 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 8d86d17 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 63a3be2 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 a434763 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 bbd9ca6 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-07 Thread Sergey Shelukhin 


> On Feb. 7, 2017, 2:31 a.m., Owen O'Malley wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 360
> > 
> >
> > You need braces around your control block for break.
> > 
> > Actually, this is an error to have an include vector that is shorter 
> > that the reader type and should cause a thrown exception. Although it 
> > should be a single check that include.length == readerSchema.getMaxId() + 1.

as far as I remember, I've seen vectorization(?) use trimmed includes somewhere


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---


On Feb. 7, 2017, 12:03 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 7, 2017, 12:03 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d62e527 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-06 Thread Owen O'Malley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review164437
---




ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 352)


You need a javadoc describing what this function does, especially given 
that it is a public function in a public class.



ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 359)


Please break your declarations to one per a line. You're violating the 
style guide.



ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 360)


You need braces around your control block for break.

Actually, this is an error to have an include vector that is shorter that 
the reader type and should cause a thrown exception. Although it should be a 
single check that include.length == readerSchema.getMaxId() + 1.


- Owen O'Malley


On Feb. 7, 2017, 12:03 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 7, 2017, 12:03 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d62e527 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  4295c1c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  8d86d17 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  63a3be2 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  a434763 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  bbd9ca6 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-06 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Feb. 7, 2017, 12:03 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d62e527 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 4295c1c 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 8d86d17 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 63a3be2 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 99cc506 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 a434763 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 bbd9ca6 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-02 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Feb. 3, 2017, 2:52 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 53b9b0c 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 d0e70d1 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 9ab26e6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 98fc9df 
  orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
  orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 0dba1a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 bbd9ca6 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-02 Thread Sergey Shelukhin 


> On Feb. 3, 2017, 1:16 a.m., Prasanth_J wrote:
> > orc/src/java/org/apache/orc/impl/WriterImpl.java, line 2323
> > 
> >
> > seems not required. where is the dereference happening?
> 
> Sergey Shelukhin wrote:
> I will make sure it will work after ORC upgrade by passing in a dummy MM, 
> but I think this change is ok to keep for now. No compat problems when 
> switching. 
> I'll file an ORC jira, I bet it will be ORC-141

Scratch that, setting MM is not accessible in ORC.


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review163695
---


On Feb. 1, 2017, 3:34 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 1, 2017, 3:34 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a2ec1f0 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  d0e70d1 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  9ab26e6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  98fc9df 
>   orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
>   orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  0dba1a0 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  552982c 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-02 Thread Sergey Shelukhin 


> On Feb. 3, 2017, 1:16 a.m., Prasanth_J wrote:
> > llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java,
> >  line 175
> > 
> >
> > nit: typo VertorDeserializeOrcWriter
> > Is the renaming happening elsewhere?

If I renamed it would be impossible to review. Will rename separately (filing a 
jira)


> On Feb. 3, 2017, 1:16 a.m., Prasanth_J wrote:
> > orc/src/java/org/apache/orc/impl/TreeReaderFactory.java, line 169
> > 
> >
> > create ORC jira?

ORC-140


> On Feb. 3, 2017, 1:16 a.m., Prasanth_J wrote:
> > orc/src/java/org/apache/orc/impl/WriterImpl.java, line 2323
> > 
> >
> > seems not required. where is the dereference happening?

I will make sure it will work after ORC upgrade by passing in a dummy MM, but I 
think this change is ok to keep for now. No compat problems when switching. 
I'll file an ORC jira, I bet it will be ORC-141


- Sergey


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review163695
---


On Feb. 1, 2017, 3:34 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 1, 2017, 3:34 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a2ec1f0 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  d0e70d1 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  9ab26e6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  98fc9df 
>   orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
>   orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  0dba1a0 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  552982c 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-02-02 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/#review163695
---




llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 (line 175)


nit: typo VertorDeserializeOrcWriter
Is the renaming happening elsewhere?



llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 (line 1489)


nit: typo



orc/src/java/org/apache/orc/impl/TreeReaderFactory.java (line 169)


create ORC jira?



orc/src/java/org/apache/orc/impl/WriterImpl.java (line 2323)


seems not required. where is the dereference happening?


- Prasanth_J


On Feb. 1, 2017, 3:34 a.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55977/
> ---
> 
> (Updated Feb. 1, 2017, 3:34 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a2ec1f0 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
>  85fae9a 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
>  d0e70d1 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
>  9ab26e6 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
>  98fc9df 
>   orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
>   orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
>  0dba1a0 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
>  b894c11e 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
>  552982c 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
> 6f090a1 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
>  e4f8d82 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
>  bd421f4 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
>  c4a6c0f 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
>  80d4731 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
>  1aeff83 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
>  cf07bca 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
>  28997a0 
>   
> storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
>  0c61243 
> 
> Diff: https://reviews.apache.org/r/55977/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-01-31 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Feb. 1, 2017, 3:34 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a2ec1f0 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 d0e70d1 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 9ab26e6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 98fc9df 
  orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
  orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 0dba1a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 552982c 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-01-30 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Jan. 31, 2017, 1:17 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d19d2ea 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 d0e70d1 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 9ab26e6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 98fc9df 
  orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
  orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 0dba1a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 552982c 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-01-26 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

(Updated Jan. 27, 2017, 3:17 a.m.)


Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 291ca7d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 d0e70d1 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 9ab26e6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 98fc9df 
  orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
  orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 0dba1a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 552982c 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin

Review Request 55977: HIVE-15672 LLAP text cache: improve first query perf II

2017-01-25 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55977/
---

Review request for hive, Gopal V and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 291ca7d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/SerDeLowLevelCacheImpl.java
 85fae9a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java
 d0e70d1 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/SerDeEncodedDataReader.java
 9ab26e6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/VertorDeserializeOrcWriter.java
 98fc9df 
  orc/src/java/org/apache/orc/impl/TreeReaderFactory.java 8b097ba 
  orc/src/java/org/apache/orc/impl/WriterImpl.java b1f3cfb 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 6281edd 
  
ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java
 0dba1a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/Reader.java 1c5f0e6 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/EncodedColumnBatch.java
 b894c11e 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java
 552982c 
  storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java 
6f090a1 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.java
 e4f8d82 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java
 bd421f4 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/IntervalDayTimeColumnVector.java
 c4a6c0f 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java
 80d4731 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/MultiValuedColumnVector.java
 1aeff83 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.java
 cf07bca 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.java
 28997a0 
  
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.java
 0c61243 

Diff: https://reviews.apache.org/r/55977/diff/


Testing
---


Thanks,

Sergey Shelukhin