[jira] [Created] (HIVE-17418) Hive Query fails with ClassCast Exception when Table and partitions are having different fileformat

[Raghuraman Murugaiyan (JIRA)]

Raghuraman Murugaiyan created HIVE-17418:


 Summary: Hive Query fails with ClassCast Exception when Table and 
partitions are having different fileformat
 Key: HIVE-17418
 URL: https://issues.apache.org/jira/browse/HIVE-17418
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 2.2.0, 2.1.1
Reporter: Raghuraman Murugaiyan


We have a partitioned table in RCFILE format, and it has a partition in ORC 
format. When we query the data from that partition, it gives the following 
runtime error:


2017-08-30 20:16:41,035 FATAL [IPC Server handler 3 on 39129] 
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: 
attempt_1503513879002_0178_m_00_0 - exited : java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row [Error getting row data with exception 
java.lang.ClassCastException: org.apache.hadoop.hive.ql.io.orc.OrcStruct cannot 
be cast to org.apache.hadoop.hive.serde2.columnar.ColumnarStructBase
at 
org.apache.hadoop.hive.serde2.objectinspector.ColumnarStructObjectInspector.getStructFieldData(ColumnarStructObjectInspector.java:61)
at 
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldData(UnionStructObjectInspector.java:157)
at 
org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:364)
at 
org.apache.hadoop.hive.serde2.SerDeUtils.getJSONString(SerDeUtils.java:200)


How to reproduce:

Queries to generate related tables:
CREATE EXTERNAL TABLE test_table (
key INT,
value STRING
)
PARTITIONED BY (
data_date STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS RCFILE;

CREATE TABLE data_table (
key INT,
value STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
STORED AS ORC;

2. Queries to generate partition:
ALTER TABLE test_table ADD PARTITION (data_date=20170829);
ALTER TABLE test_table PARTITION (data_date=20170829) SET FILEFORMAT ORC;

3. Query to load data:
INSERT INTO TABLE test_table PARTITION (data_date=20170829)
SELECT key, value FROM data_table
WHERE data_date=20170829;

4. Query to reproduce the error:
SELECT key, value FROM test_table
WHERE data_date=20170829;


I have run this query in Hive 2.1.1 and Hive 2.2.1 where it got failed and ran 
fine in the Hive 1.2.1 built. Please advise us, whether we can create a Table 
in one file format and the new partitions in a different file format? 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17417) Lazy Timestamp and Date serialization is very expensive

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-17417:


 Summary: Lazy Timestamp and Date serialization is very expensive
 Key: HIVE-17417
 URL: https://issues.apache.org/jira/browse/HIVE-17417
 Project: Hive
  Issue Type: Bug
  Components: Serializers/Deserializers
Affects Versions: 3.0.0, 2.4.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran
Priority: Critical


In a specific case where a schema contains array with timestamp and 
date fields (array size >1). Any access to this column very very expensive 
in terms of CPU as most of the time is serialization of timestamp and date. 
Refer attached profiles. >70% time spent in serialization + tostring 
conversions. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17416) Hive Distinct changes column value

[Manoj Durisheti (JIRA)]

Manoj Durisheti created HIVE-17416:
--

 Summary: Hive Distinct changes column value
 Key: HIVE-17416
 URL: https://issues.apache.org/jira/browse/HIVE-17416
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.2.1
Reporter: Manoj Durisheti


Hive 1.2.1000.2.6.1.0-129

Below query with distinct is expected to just dedupe the resultant data. But it 
alters the data.

*Query without Distinct:*
select
REGEXP_EXTRACT(UPPER(field_name), '([A-Z]_[0-9]*[A-Z]?)\\??.*', 1) r_field_name,
REGEXP_EXTRACT(UPPER(field_name), '([A-Z]_[0-9]*[a-z]?)\\??.*', 1) w_field_name
from alpha.table_name
where
datestamp = 20170805
and
field_name = 
'https://www.abcd.com/details/123-main-st-abcde-xx-84004-5434484-e_2300a'
;
Result:
e_2300a e_2300
e_2300a e_2300
e_2300a e_2300
e_2300a e_2300
e_2300a e_2300

*Query with Distinct:*
select distinct
REGEXP_EXTRACT(UPPER(field_name), '([A-Z]_[0-9]*[A-Z]?)\\??.*', 1) r_field_name,
REGEXP_EXTRACT(UPPER(field_name), '([A-Z]_[0-9]*[a-z]?)\\??.*', 1) w_field_name
from alpha.table_name
where
datestamp = 20170805
and
field_name = 
'https://www.abcd.com/details/123-main-st-abcde-xx-84004-5434484-e_2300a'
;
Result:
e_2300 e_2300

*Expected Result with Distinct is: *
e_2300a e_2300





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 62004: HIVE-17415

[Deepak Jaiswal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62004/
---

Review request for hive and Ashutosh Chauhan.


Bugs: hive-17415
https://issues.apache.org/jira/browse/hive-17415


Repository: hive-git


Description
---

Hit error "SemanticException View xxx is corresponding to LIMIT, rather than a 
SelectOperator." in Hive queries when a user creates a view with limits


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e8acabefa5 
  ql/src/test/queries/clientpositive/authorization_view_8.q PRE-CREATION 
  ql/src/test/results/clientpositive/llap/authorization_view_8.q.out 
PRE-CREATION 


Diff: https://reviews.apache.org/r/62004/diff/1/


Testing
---


Thanks,

Deepak Jaiswal

Re: Review Request 61164: HIVE-17006 LLAP: Parquet caching

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61164/
---

(Updated Aug. 30, 2017, 10:21 p.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/FileUtils.java e784797f62 
  itests/src/test/resources/testconfiguration.properties fa6a2aaea0 
  llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapIo.java 
42129b7511 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/EvictionDispatcher.java 
0cbc8f6f4c 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LlapAllocatorBuffer.java 
52144c2432 
  llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
35b9d1f942 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/ParquetMetadataCacheImpl.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java fa7f59d7dc 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 442c921a84 
  
ql/src/java/org/apache/hadoop/hive/ql/io/LlapCacheOnlyInputFormatInterface.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java 
f4fadbb61b 
  
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java
 322178a2f7 
  
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetFooterInputFromCache.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
 6a7a219dfe 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java 8b99ae07ac 
  ql/src/test/queries/clientpositive/parquet_ppd_decimal.q dfca486241 
  ql/src/test/queries/clientpositive/parquet_predicate_pushdown.q a38cdbe007 
  ql/src/test/queries/clientpositive/parquet_types.q db37d2e1b2 
  ql/src/test/queries/clientpositive/parquet_types_vectorization.q bb0e5b258f 
  ql/src/test/queries/clientpositive/vectorized_parquet.q db02ec04a5 
  ql/src/test/queries/clientpositive/vectorized_parquet_types.q 7467cb3cf6 
  ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out 
aecbcfdfe4 
  ql/src/test/results/clientpositive/llap/parquet_types_vectorization.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/vectorized_parquet.q.out 8a84d3dc22 
  ql/src/test/results/clientpositive/llap/vectorized_parquet_types.q.out 
cdf6b3d225 
  storage-api/src/java/org/apache/hadoop/hive/common/io/FileMetadataCache.java 
PRE-CREATION 
  
storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/MemoryBufferOrBuffers.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/61164/diff/3/

Changes: https://reviews.apache.org/r/61164/diff/2-3/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Created] (HIVE-17414) HoS DPP + Vectorization generates invalid explain plan due to CombineEquivalentWorkResolver

[Sahil Takiar (JIRA)]

Sahil Takiar created HIVE-17414:
---

 Summary: HoS DPP + Vectorization generates invalid explain plan 
due to CombineEquivalentWorkResolver
 Key: HIVE-17414
 URL: https://issues.apache.org/jira/browse/HIVE-17414
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Sahil Takiar


Similar to HIVE-16948, the following query generates an invalid explain plan 
when HoS DPP is enabled + vectorization:

{code:sql}
select ds from (select distinct(ds) as ds from srcpart union all select 
distinct(ds) as ds from srcpart) s where s.ds in (select max(srcpart.ds) from 
srcpart union all select min(srcpart.ds) from srcpart)
{code}

Explain Plan:

{code}
STAGE DEPENDENCIES:
  Stage-2 is a root stage
  Stage-1 depends on stages: Stage-2
  Stage-0 depends on stages: Stage-1

STAGE PLANS:
  Stage: Stage-2
Spark
  Edges:
Reducer 11 <- Map 10 (GROUP, 1)
Reducer 13 <- Map 12 (GROUP, 1)
 A masked pattern was here 
  Vertices:
Map 10
Map Operator Tree:
TableScan
  alias: srcpart
  Statistics: Num rows: 2000 Data size: 21248 Basic stats: 
COMPLETE Column stats: NONE
  Select Operator
expressions: ds (type: string)
outputColumnNames: ds
Statistics: Num rows: 2000 Data size: 21248 Basic stats: 
COMPLETE Column stats: NONE
Group By Operator
  aggregations: max(ds)
  mode: hash
  outputColumnNames: _col0
  Statistics: Num rows: 1 Data size: 184 Basic stats: 
COMPLETE Column stats: NONE
  Reduce Output Operator
sort order:
Statistics: Num rows: 1 Data size: 184 Basic stats: 
COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Execution mode: vectorized
Map 12
Map Operator Tree:
TableScan
  alias: srcpart
  Statistics: Num rows: 2000 Data size: 21248 Basic stats: 
COMPLETE Column stats: NONE
  Select Operator
expressions: ds (type: string)
outputColumnNames: ds
Statistics: Num rows: 2000 Data size: 21248 Basic stats: 
COMPLETE Column stats: NONE
Group By Operator
  aggregations: min(ds)
  mode: hash
  outputColumnNames: _col0
  Statistics: Num rows: 1 Data size: 184 Basic stats: 
COMPLETE Column stats: NONE
  Reduce Output Operator
sort order:
Statistics: Num rows: 1 Data size: 184 Basic stats: 
COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Execution mode: vectorized
Reducer 11
Execution mode: vectorized
Reduce Operator Tree:
  Group By Operator
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE 
Column stats: NONE
Filter Operator
  predicate: _col0 is not null (type: boolean)
  Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE 
Column stats: NONE
  Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 368 Basic stats: 
COMPLETE Column stats: NONE
Select Operator
  expressions: _col0 (type: string)
  outputColumnNames: _col0
  Statistics: Num rows: 2 Data size: 368 Basic stats: 
COMPLETE Column stats: NONE
  Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 368 Basic stats: 
COMPLETE Column stats: NONE
Spark Partition Pruning Sink Operator
  Target column: ds (string)
  partition key expr: ds
  Statistics: Num rows: 2 Data size: 368 Basic stats: 
COMPLETE Column stats: NONE
  target work: Map 1
Select Operator
  expressions: _col0 (type: string)
  outputColumnNames: _col0
  Statistics: Num rows: 2 Data size: 368 Basic stats: 
COMPLETE Column stats: NONE
  Group By Operator
   

[jira] [Created] (HIVE-17415) Hit error "SemanticException View xxx is corresponding to LIMIT, rather than a SelectOperator." in Hive queries

[Deepak Jaiswal (JIRA)]

Deepak Jaiswal created HIVE-17415:
-

 Summary: Hit error "SemanticException View xxx is corresponding to 
LIMIT, rather than a SelectOperator." in Hive queries
 Key: HIVE-17415
 URL: https://issues.apache.org/jira/browse/HIVE-17415
 Project: Hive
  Issue Type: Bug
Reporter: Deepak Jaiswal
Assignee: Deepak Jaiswal


Hit error "SemanticException View xxx is corresponding to LIMIT, rather than a 
SelectOperator." in Hive queries when a user creates a view with limits


set 
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider;

create table my_passwd (
username string,
uid int);

insert into my_passwd values
("Dev1", 1),
("Dev2", 2),
("Dev3", 3),
("Dev4", 4),
("Dev5", 5),
("Dev6", 6);


create view my_passwd_vw as select * from my_passwd limit 3;


set hive.security.authorization.enabled=true;
grant select on table my_passwd to user hive_test_user;
grant select on table my_passwd_vw to user hive_test_user;

select * from my_passwd_vw;



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] hive pull request #235: HIVE-17307 Change the metastore to not use the metri...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/hive/pull/235


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #217: HIVE-16895: Multi-threaded execution of bootstrap du...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/217


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #215: HIVE-17144 : export of temporary tables not working ...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/215


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #196: HIVE-16892 : Move creation of _files from ReplCopyTa...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/196


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #214: Hive 16896: move replication load related work in se...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/214


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #201: HIVE-16893 : move replication dump related work in s...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/201


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #189: HIVE-16591 : DR for function Binaries on HDFS

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/189


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #193: HIVE-16866 : existing available UDF is used in TestR...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/193


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #182: HIVE-16269 : enable incremental function dump to be ...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/182


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #179: HIVE-16267 : Enable bootstrap function metadata to b...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/179


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #178: HIVE-16268: enable incremental repl dump to handle f...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/178


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #153: HIVE-16045: Print progress bar along with operation ...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/153


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #159: HIVE-16219 : metastore notification_log contains ser...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/159


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #165: HIVe-16254: metadata for values temporary tables for...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/165


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #155: HIVE-16115: Stop printing progress info from operati...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/155


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #151: Hive 15847: In Progress update refreshes seem slow

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/151


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #138: HIVE-15713: add ldap authentication related configur...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/138


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #146: HIVE-15906 : hrift code regeneration to include new ...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/146


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #137: HIVE-15712: new HiveConf in SQLOperation.getSerDe() ...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/137


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #136: HIVE-15711 : Flaky TestEmbeddedThriftBinaryCLIServic...

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/136


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #130: HIVE-15550 fix arglist logging in schematool

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/130


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #129: HIVE-15473: Progress Bar on Beeline client

[anishek]

Github user anishek closed the pull request at:

https://github.com/apache/hive/pull/129


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] hive pull request #240: HIVE-17410 : repl load task during subsequent DAG ge...

[anishek]

GitHub user anishek opened a pull request:

https://github.com/apache/hive/pull/240

HIVE-17410 : repl load task during subsequent DAG generation does notstart 
from the last partition processed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/anishek/hive HIVE-17410

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/240.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #240


commit f9072a7f76484222f0f78398fec6138d0d0847a3
Author: Anishek Agarwal 
Date:   2017-08-30T00:03:39Z

HIVE-17410 : repl load task during subsequent DAG generation does not start 
from the last partition processed




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 61164: HIVE-17006 LLAP: Parquet caching

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61164/#review184177
---




ql/src/test/results/clientpositive/llap/parquet_types_vectorization.q.out
Lines 1 (patched)


verify that this runs thru the cache if possible


- Sergey Shelukhin


On July 27, 2017, 10:27 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61164/
> ---
> 
> (Updated July 27, 2017, 10:27 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java e8a3a7a49e 
>   itests/src/test/resources/testconfiguration.properties f66e19be3e 
>   llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapIo.java 
> 42129b7511 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/EvictionDispatcher.java
>  0cbc8f6f4c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
> 35b9d1f942 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/ParquetMetadataCacheImpl.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java 9b8b76102a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 21394c6aab 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/LlapCacheOnlyInputFormatInterface.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
>  f4fadbb61b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java
>  322178a2f7 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetFooterInputFromCache.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
>  6a7a219dfe 
>   ql/src/test/queries/clientpositive/parquet_ppd_decimal.q dfca486241 
>   ql/src/test/queries/clientpositive/parquet_predicate_pushdown.q a38cdbe007 
>   ql/src/test/queries/clientpositive/parquet_types.q db37d2e1b2 
>   ql/src/test/queries/clientpositive/parquet_types_vectorization.q bb0e5b258f 
>   ql/src/test/queries/clientpositive/vectorized_parquet.q e6ebdaac62 
>   ql/src/test/queries/clientpositive/vectorized_parquet_types.q 7467cb3cf6 
>   ql/src/test/results/clientpositive/llap/parquet_types_vectorization.q.out 
> PRE-CREATION 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/FileMetadataCache.java 
> PRE-CREATION 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/MemoryBufferOrBuffers.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61164/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 61164: HIVE-17006 LLAP: Parquet caching

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61164/#review184175
---




llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/ParquetMetadataCacheImpl.java
Lines 75 (patched)


comment about not needing to discard



ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java
Lines 60 (patched)


revise comment



ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java
Lines 223 (patched)


should this use "off"?



ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java
Lines 265 (patched)


.



ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java
Lines 307 (patched)


needs to be done?



ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
Lines 198 (patched)


follow up



ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
Lines 327 (patched)


consider reading into cache, then doing the same thing as above


- Sergey Shelukhin


On July 27, 2017, 10:27 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61164/
> ---
> 
> (Updated July 27, 2017, 10:27 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java e8a3a7a49e 
>   itests/src/test/resources/testconfiguration.properties f66e19be3e 
>   llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapIo.java 
> 42129b7511 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/cache/EvictionDispatcher.java
>  0cbc8f6f4c 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java 
> 35b9d1f942 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/ParquetMetadataCacheImpl.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/llap/LlapCacheAwareFs.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java 9b8b76102a 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 21394c6aab 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/LlapCacheOnlyInputFormatInterface.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
>  f4fadbb61b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java
>  322178a2f7 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetFooterInputFromCache.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java
>  6a7a219dfe 
>   ql/src/test/queries/clientpositive/parquet_ppd_decimal.q dfca486241 
>   ql/src/test/queries/clientpositive/parquet_predicate_pushdown.q a38cdbe007 
>   ql/src/test/queries/clientpositive/parquet_types.q db37d2e1b2 
>   ql/src/test/queries/clientpositive/parquet_types_vectorization.q bb0e5b258f 
>   ql/src/test/queries/clientpositive/vectorized_parquet.q e6ebdaac62 
>   ql/src/test/queries/clientpositive/vectorized_parquet_types.q 7467cb3cf6 
>   ql/src/test/results/clientpositive/llap/parquet_types_vectorization.q.out 
> PRE-CREATION 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/FileMetadataCache.java 
> PRE-CREATION 
>   
> storage-api/src/java/org/apache/hadoop/hive/common/io/encoded/MemoryBufferOrBuffers.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61164/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

[jira] [Created] (HIVE-17413) predicate involving CAST affects value returned by the SELECT statement

[Jim Hopper (JIRA)]

Jim Hopper created HIVE-17413:
-

 Summary: predicate involving CAST affects value returned by the 
SELECT statement
 Key: HIVE-17413
 URL: https://issues.apache.org/jira/browse/HIVE-17413
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.2.1
Reporter: Jim Hopper
Priority: Critical


steps to reproduce:

{code}
create table t stored as orc as
select cast('2017-08-29 00:01:26' as timestamp) as ts;
{code}

{code}
select ts from t;
{code}

{code}
ts
2017-08-29 00:01:26
{code}

{code}
select ts from t where cast(ts as date) = '2017-08-29';
{code}

{code}
ts
2017-08-29 00:00:00
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61985: HIVE-17399

[Deepak Jaiswal]

> On Aug. 30, 2017, 3:35 a.m., Gopal V wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java
> > Lines 30 (patched)
> > 
> >
> > I prefer a Boolean object, which goes from null -> false/true, so that 
> > you can encode (not-set, true, false) in one field.
> 
> Deepak Jaiswal wrote:
> Will do that.

In this case, we have very straightforward states so keeping it as it is.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61985/#review184120
---


On Aug. 30, 2017, 5:58 a.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61985/
> ---
> 
> (Updated Aug. 30, 2017, 5:58 a.m.)
> 
> 
> Review request for hive, Gopal V and Jason Dere.
> 
> 
> Bugs: HIVE-17399
> https://issues.apache.org/jira/browse/HIVE-17399
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Do not remove semijoin branch if it feeds to TS->DPP_EVENT
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java 
> 5d7b9e5c6d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 3bd35bf2d8 
> 
> 
> Diff: https://reviews.apache.org/r/61985/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>