[jira] [Created] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming

[Vaibhav Gumashta (JIRA)]

Vaibhav Gumashta created HIVE-18056:
---

 Summary: CachedStore: Have a whitelist/blacklist config to allow 
selective caching of tables/partitions and allow read while prewarming
 Key: HIVE-18056
 URL: https://issues.apache.org/jira/browse/HIVE-18056
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 3.0.0
Reporter: Vaibhav Gumashta






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-18055) cache like pattern object using map object in like function

[wan kun (JIRA)]

wan kun created HIVE-18055:
--

 Summary: cache like pattern object using map object in like 
function
 Key: HIVE-18055
 URL: https://issues.apache.org/jira/browse/HIVE-18055
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: wan kun
Assignee: wan kun
Priority: Minor
 Fix For: 1.2.3


Now, only one pattern object was cached in the like function. If the like 
function is working on one column, the pattern object will be generated 
continuously for the regular expression matching. It's very inefficient. So 
should we use LRU MAP to cache a batch of objects ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 63782: HIVE-18002 add group support for pool mappings

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63782/
---

Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java 3bb4f585ce 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java 
50cf4daeed 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
a73a24aef3 


Diff: https://reviews.apache.org/r/63782/diff/1/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 63533: HIVE-17906 use kill query mechanics to kill queries in WM

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63533/
---

(Updated Nov. 14, 2017, 1 a.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java 03a068257e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WmTezSession.java 6cf2aad34d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
bdbcce5666 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
a73a24aef3 


Diff: https://reviews.apache.org/r/63533/diff/5/

Changes: https://reviews.apache.org/r/63533/diff/4-5/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 63774: HIVE-15436

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63774/#review190912
---




ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
Lines 1418-1439 (patched)


Lets not add public methods which are not used. Unnecessarily increases api 
surface area.


- Ashutosh Chauhan


On Nov. 13, 2017, 11:09 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63774/
> ---
> 
> (Updated Nov. 13, 2017, 11:09 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-15436
> https://issues.apache.org/jira/browse/HIVE-15436
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-15436
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
> cceea019c2c6408a3bfe74868112a132afc44018 
>   
> ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
>  ba46ff76e7b0ce724bc3a3ded8f476f0056f57bd 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
>  d9155c4c35acf01958424ea8daaa593e146c810c 
>   
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
>  575ba05a4d6b09c71182d7e5fedbd9ff72b3480b 
> 
> 
> Diff: https://reviews.apache.org/r/63774/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

Re: Review Request 63533: HIVE-17906 use kill query mechanics to kill queries in WM

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63533/#review190909
---


Ship it!




Ship It!

- Prasanth_J


On Nov. 9, 2017, 10:33 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63533/
> ---
> 
> (Updated Nov. 9, 2017, 10:33 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java 
> 03a068257e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WmTezSession.java 0dd1433a6c 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
> 16f5dce10a 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
> 84a35cc78c 
> 
> 
> Diff: https://reviews.apache.org/r/63533/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: adding a label that would trigger HiveQA?

[Thejas Nair]

+1
It would be great if label could be added automatically and it gets removed
after HiveQA run


On Fri, Nov 10, 2017 at 5:05 PM, Sergey Shelukhin 
wrote:

> Resubmitting the same patch for HiveQA as patches are constantly getting
> dropped is getting old.
> I wonder if we should have a label that would trigger HiveQA and only be
> removed at the end, when posting results to the JIRA?
> We could either add it in addition to the current filter or trigger
> mechanism or, if JIRA allows that, make it the only selection criteria and
> add it automatically when clicking submit patch (+remove it on cancel
> patch and when HiveQA has finished).
>
> Comments/suggestions?
> I can try to modify the scripts some day.
>
>

Review Request 63774: HIVE-15436

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63774/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-15436
https://issues.apache.org/jira/browse/HIVE-15436


Repository: hive-git


Description
---

HIVE-15436


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
cceea019c2c6408a3bfe74868112a132afc44018 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
 ba46ff76e7b0ce724bc3a3ded8f476f0056f57bd 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
 d9155c4c35acf01958424ea8daaa593e146c810c 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 575ba05a4d6b09c71182d7e5fedbd9ff72b3480b 


Diff: https://reviews.apache.org/r/63774/diff/1/


Testing
---


Thanks,

Jesús Camacho Rodríguez

[RESULT][VOTE] Hive 2.3.2 bugfix release

[Sahil Takiar]

With three +1s the vote for the Hive 2.3.2 bugfix release has passed.

I will finish up the release process and send another email with everything
is done.

Re: [VOTE] Apache Hive 2.3.2 Release Candidate 0

[Yongzhi Chen]

+1
I verified the release by
. Checked the gpg signature
. Checked the md5 files.
And install the hive 2.3.2 and test commands:
show tables;
create table;
select * from table;

The release works fine.

On Mon, Nov 13, 2017 at 11:43 AM, Sergio Pena 
wrote:

> +1
>
> I verified the release by doing the following:
> * checked the gpg signature
> * checked the md5 files
> * installed hive 2.3.2 in my local machine with hadoop 2.7.2 and run a few
> commands:
>   > show databases
>   > show tables
>   > insert into table values()
>   > select * from table
>   > select count(*) from table
> * checked the maven artifacts are correctly pulled by other components and
> run unit tests
> * checked that storage-api-2.4.0 is pulled
> * checked the release tag
> * checked the RELEASE_NOTES, NOTICE, LICENSE are correct
>
> The release is working correctly.
>
> Thanks Sahil for making this release.
> - Sergio
>
> On Thu, Nov 9, 2017 at 5:37 PM, Sahil Takiar 
> wrote:
>
> > Apache Hive 2.3.2 Release Candidate 0 is available here:
> > http://people.apache.org/~stakiar/hive-2.3.2/
> >
> > Maven artifacts are available here:
> > https://repository.apache.org/content/repositories/orgapachehive-1082/
> >
> > Source tag for RCN is at:https://github.com/apache/
> hive/tree/release-2.3.2
> >
> > Voting will conclude in 72 hours.
> >
> > Hive PMC Members: Please test and vote.
> >
> > Thanks.
> >
>

Re: [VOTE] Apache Hive 2.3.2 Release Candidate 0

[Aihua Xu]

I built from the sources and ran a few tests. Checked gpg signatures.
Checked md5sum. Looks good. +1.

Thanks Sahil to put the release together.

Aihua

On Mon, Nov 13, 2017 at 8:43 AM, Sergio Pena 
wrote:

> +1
>
> I verified the release by doing the following:
> * checked the gpg signature
> * checked the md5 files
> * installed hive 2.3.2 in my local machine with hadoop 2.7.2 and run a few
> commands:
>   > show databases
>   > show tables
>   > insert into table values()
>   > select * from table
>   > select count(*) from table
> * checked the maven artifacts are correctly pulled by other components and
> run unit tests
> * checked that storage-api-2.4.0 is pulled
> * checked the release tag
> * checked the RELEASE_NOTES, NOTICE, LICENSE are correct
>
> The release is working correctly.
>
> Thanks Sahil for making this release.
> - Sergio
>
> On Thu, Nov 9, 2017 at 5:37 PM, Sahil Takiar 
> wrote:
>
> > Apache Hive 2.3.2 Release Candidate 0 is available here:
> > http://people.apache.org/~stakiar/hive-2.3.2/
> >
> > Maven artifacts are available here:
> > https://repository.apache.org/content/repositories/orgapachehive-1082/
> >
> > Source tag for RCN is at:https://github.com/apache/h
> ive/tree/release-2.3.2
> >
> > Voting will conclude in 72 hours.
> >
> > Hive PMC Members: Please test and vote.
> >
> > Thanks.
> >
>



-- 
Thanks
Aihua

[jira] [Created] (HIVE-18054) Make Lineage work with concurrent queries on a Session

[Andrew Sherman (JIRA)]

Andrew Sherman created HIVE-18054:
-

 Summary:  Make Lineage work with concurrent queries on a Session
 Key: HIVE-18054
 URL: https://issues.apache.org/jira/browse/HIVE-18054
 Project: Hive
  Issue Type: Bug
Reporter: Andrew Sherman
Assignee: Andrew Sherman


A Hive Session can contain multiple concurrent sql Operations.
Lineage is currently tracked in SessionState and is cleared when a query 
completes. This results in Lineage for other running queries being lost.
To fix this, move LineageState from SessionState to QueryState.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-18053) Support different table types for MVs

[Jesus Camacho Rodriguez (JIRA)]

Jesus Camacho Rodriguez created HIVE-18053:
--

 Summary: Support different table types for MVs
 Key: HIVE-18053
 URL: https://issues.apache.org/jira/browse/HIVE-18053
 Project: Hive
  Issue Type: Sub-task
  Components: Materialized views
Affects Versions: 3.0.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Priority: Critical


MVs backed by MM tables, managed tables, external tables. This might work 
already, but we need to add tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-18052) Run p-tests on mm tables

[Steve Yeom (JIRA)]

Steve Yeom created HIVE-18052:
-

 Summary: Run p-tests on mm tables
 Key: HIVE-18052
 URL: https://issues.apache.org/jira/browse/HIVE-18052
 Project: Hive
  Issue Type: Task
Reporter: Steve Yeom






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] Apache Hive 2.3.2 Release Candidate 0

[Sergio Pena]

+1

I verified the release by doing the following:
* checked the gpg signature
* checked the md5 files
* installed hive 2.3.2 in my local machine with hadoop 2.7.2 and run a few
commands:
  > show databases
  > show tables
  > insert into table values()
  > select * from table
  > select count(*) from table
* checked the maven artifacts are correctly pulled by other components and
run unit tests
* checked that storage-api-2.4.0 is pulled
* checked the release tag
* checked the RELEASE_NOTES, NOTICE, LICENSE are correct

The release is working correctly.

Thanks Sahil for making this release.
- Sergio

On Thu, Nov 9, 2017 at 5:37 PM, Sahil Takiar  wrote:

> Apache Hive 2.3.2 Release Candidate 0 is available here:
> http://people.apache.org/~stakiar/hive-2.3.2/
>
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-1082/
>
> Source tag for RCN is at:https://github.com/apache/hive/tree/release-2.3.2
>
> Voting will conclude in 72 hours.
>
> Hive PMC Members: Please test and vote.
>
> Thanks.
>

Re: Review Request 63442: HIVE-17934 Merging Statistics are promoted to COMPLETE (most of the time)

[Ashutosh Chauhan]

> On Nov. 9, 2017, 7:51 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
> > Line 160 (original), 160 (patched)
> > 
> >
> > bucket_small has no stats gathered. This should be NONE.
> 
> Zoltan Haindrich wrote:
> `hive.stats.autogather` is enabled by default from `HiveConf`
> 
> Ashutosh Chauhan wrote:
> Those are load statements, not inserts. We don't gather stats with load 
> statements only with insets.
> 
> Zoltan Haindrich wrote:
> sorry, you are right: basic stats are not gathered in this case in any 
> way.
> 
> But the stat state is complete; because: there is logic which scans the 
> file sizes - to calculate the datasizes; and from there HIVE-16811 can guess 
> some row counts
> 
> 
> https://github.com/kgyrtkirk/hive/blob/9f67a878512117eb5c251794adc1a91bae62fea7/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L386-L393
> 
> Firts I would like to make the standalone table/partitioned table's 
> calculation-s are a bit more similar to eachother
> 
> I've tried to come up with some definitions for NONE/PARTIAL/COMPLETE; 
> currently I would say the following:
> 
> * NONE: not known
> * on table: no information (afaik currently this can't happen)
> * estimation tree: all nodes in the estimation tree were NONE
> * PARTIAL:
> * on table: the current information is estimated from data size
> * estimation tree: contains at least one NONE/PARTIAL
> * COMPLETE:
> * current information is correct (calculated by statstask-s)
> * estimation tree: the whole subtree has COMPLETE status
> 
> If I use these definitions; then I would say that the filesystem size 
> based estimation should be considered PARTIAL.

Definitions sounds good. Lets use them to make sure our state calculation logic 
is built on it.
Can you also add this in code comments.


- Ashutosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63442/#review190633
---


On Nov. 9, 2017, 5:39 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63442/
> ---
> 
> (Updated Nov. 9, 2017, 5:39 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-17934
> https://issues.apache.org/jira/browse/HIVE-17934
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * remove the reactive stat state guessing method
> * make the guessing only work when a new object is created
> * change the way stat objects are merged
> 
> this patch will most probably break almost all qtest outputs
> 
> 
> Diffs
> -
> 
>   accumulo-handler/src/test/results/positive/accumulo_queries.q.out 
> b3adf4e504 
>   hbase-handler/src/test/results/positive/hbase_queries.q.out b2eda12e95 
>   hbase-handler/src/test/results/positive/hbasestats.q.out 29eefd43a9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java
>  7a3fae65e8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  a4f60accce 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 8ffb4ce44b 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java ce7c96c639 
>   ql/src/test/queries/clientpositive/lateral_view_onview2.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/stats_empty_partition2.q PRE-CREATION 
>   ql/src/test/results/clientpositive/acid_table_stats.q.out 351ff0da0a 
>   ql/src/test/results/clientpositive/alterColumnStatsPart.q.out 858e16fe22 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 3a94a6a4e3 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 7875e9693a 
>   ql/src/test/results/clientpositive/cbo_const.q.out e9f885b363 
>   ql/src/test/results/clientpositive/cbo_input26.q.out 77fc194829 
>   ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out 414b715b7a 
>   ql/src/test/results/clientpositive/columnstats_quoting.q.out 683c1e274f 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out a2c6ead293 
>   ql/src/test/results/clientpositive/constGby.q.out c633624935 
>   ql/src/test/results/clientpositive/constant_prop_3.q.out cba4744866 
>   ql/src/test/results/clientpositive/constprog3.q.out f54168d0ee 
>   ql/src/test/results/clientpositive/correlationoptimizer10.q.out a03acd38a7 
>   ql/src/test/results/clientpositive/correlationoptimizer11.q.out cf2250790a 
>   ql/src/test/results/clientpositive/correlationoptimizer13.q.out 6d4f931213 
>   ql/src/test/results/clientpositive/correlationoptimizer14.q.out 149f33

[jira] [Created] (HIVE-18051) qfiles: dataset support

[Zoltan Haindrich (JIRA)]

Zoltan Haindrich created HIVE-18051:
---

 Summary: qfiles: dataset support
 Key: HIVE-18051
 URL: https://issues.apache.org/jira/browse/HIVE-18051
 Project: Hive
  Issue Type: Improvement
  Components: Testing Infrastructure
Reporter: Zoltan Haindrich


it would be great to have some kind of test dataset support; currently there is 
the {{q_test_init.sql}} which is quite large; and I'm often override it with an 
invalid string; because I write independent qtests most of the time - and the 
load of {{src}} and other tables are just a waste of time for me ; not to 
mention that the loading of those tables may also trigger breakpoints - which 
is a bit annoying.

Most of the tests are "only" using the {{src}} table and possibly 2 others; 
however the main init script contains a bunch of tables - meanwhile there are 
quite few other tests which could possibly also benefit from a more general 
feature; for example the creation of {{bucket_small}} is present in 20 q files.

the proposal would be to enable the qfiles to be annotated with metadata like 
datasets:
{code}
--! qt:dataset:src,bucket_small
{code}

proposal for storing a dataset:

* the loader script would be at: {{data/datasets/__NAME__/load.hive.sql}}
* the table data could be stored under that location


a draft about this; and other qfiles related ideas:
https://docs.google.com/document/d/1KtcIx8ggL9LxDintFuJo8NQuvNWkmtvv_ekbWrTLNGc/edit?usp=sharing




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63442: HIVE-17934 Merging Statistics are promoted to COMPLETE (most of the time)

[Zoltan Haindrich]

> On Nov. 9, 2017, 7:51 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
> > Line 160 (original), 160 (patched)
> > 
> >
> > bucket_small has no stats gathered. This should be NONE.
> 
> Zoltan Haindrich wrote:
> `hive.stats.autogather` is enabled by default from `HiveConf`
> 
> Ashutosh Chauhan wrote:
> Those are load statements, not inserts. We don't gather stats with load 
> statements only with insets.

sorry, you are right: basic stats are not gathered in this case in any way.

But the stat state is complete; because: there is logic which scans the file 
sizes - to calculate the datasizes; and from there HIVE-16811 can guess some 
row counts

https://github.com/kgyrtkirk/hive/blob/9f67a878512117eb5c251794adc1a91bae62fea7/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L386-L393

Firts I would like to make the standalone table/partitioned table's 
calculation-s are a bit more similar to eachother

I've tried to come up with some definitions for NONE/PARTIAL/COMPLETE; 
currently I would say the following:

* NONE: not known
* on table: no information (afaik currently this can't happen)
* estimation tree: all nodes in the estimation tree were NONE
* PARTIAL:
* on table: the current information is estimated from data size
* estimation tree: contains at least one NONE/PARTIAL
* COMPLETE:
* current information is correct (calculated by statstask-s)
* estimation tree: the whole subtree has COMPLETE status

If I use these definitions; then I would say that the filesystem size based 
estimation should be considered PARTIAL.


- Zoltan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63442/#review190633
---


On Nov. 9, 2017, 5:39 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63442/
> ---
> 
> (Updated Nov. 9, 2017, 5:39 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-17934
> https://issues.apache.org/jira/browse/HIVE-17934
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * remove the reactive stat state guessing method
> * make the guessing only work when a new object is created
> * change the way stat objects are merged
> 
> this patch will most probably break almost all qtest outputs
> 
> 
> Diffs
> -
> 
>   accumulo-handler/src/test/results/positive/accumulo_queries.q.out 
> b3adf4e504 
>   hbase-handler/src/test/results/positive/hbase_queries.q.out b2eda12e95 
>   hbase-handler/src/test/results/positive/hbasestats.q.out 29eefd43a9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java
>  7a3fae65e8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  a4f60accce 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 8ffb4ce44b 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java ce7c96c639 
>   ql/src/test/queries/clientpositive/lateral_view_onview2.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/stats_empty_partition2.q PRE-CREATION 
>   ql/src/test/results/clientpositive/acid_table_stats.q.out 351ff0da0a 
>   ql/src/test/results/clientpositive/alterColumnStatsPart.q.out 858e16fe22 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 3a94a6a4e3 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 7875e9693a 
>   ql/src/test/results/clientpositive/cbo_const.q.out e9f885b363 
>   ql/src/test/results/clientpositive/cbo_input26.q.out 77fc194829 
>   ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out 414b715b7a 
>   ql/src/test/results/clientpositive/columnstats_quoting.q.out 683c1e274f 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out a2c6ead293 
>   ql/src/test/results/clientpositive/constGby.q.out c633624935 
>   ql/src/test/results/clientpositive/constant_prop_3.q.out cba4744866 
>   ql/src/test/results/clientpositive/constprog3.q.out f54168d0ee 
>   ql/src/test/results/clientpositive/correlationoptimizer10.q.out a03acd38a7 
>   ql/src/test/results/clientpositive/correlationoptimizer11.q.out cf2250790a 
>   ql/src/test/results/clientpositive/correlationoptimizer13.q.out 6d4f931213 
>   ql/src/test/results/clientpositive/correlationoptimizer14.q.out 149f33fee8 
>   ql/src/test/results/clientpositive/correlationoptimizer15.q.out 2d813b239f 
>   ql/src/test/results/clientpositive/correlationoptimizer5.q.out 68d6a54862 
>   ql/src/test/results/clientpositive/correlationoptimizer7.q.out 82fecab594 
>   ql/src/test/results/clientpositive/correlationoptimizer8.q.out

[GitHub] hive pull request #271: HIVE-18050: LlapServiceDriver shoud split HIVE_AUX_J...

[Aegeaner]

GitHub user Aegeaner opened a pull request:

https://github.com/apache/hive/pull/271

HIVE-18050: LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' i…

LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ','

https://issues.apache.org/jira/browse/HIVE-18050


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Aegeaner/hive HIVE-18050

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #271


commit 468be0a68cfc7cb5fdc4357ecd8f0001bfe7ec7c
Author: Aegeaner 
Date:   2017-11-13T10:49:23Z

HIVE-18050: LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead 
of ','




---

[jira] [Created] (HIVE-18050) LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ','

[Aegeaner (JIRA)]

Aegeaner created HIVE-18050:
---

 Summary: LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' 
instead of ','
 Key: HIVE-18050
 URL: https://issues.apache.org/jira/browse/HIVE-18050
 Project: Hive
  Issue Type: Bug
  Components: CLI, Clients
Affects Versions: 2.3.0
Reporter: Aegeaner
Assignee: Aegeaner


LlapServiceDriver shoud split HIVE_AUX_JARS_PATH by ':' instead of ',' , since 
in hive script the environment variable has been replaced:

{code:java}
elif [ "${HIVE_AUX_JARS_PATH}" != "" ]; then 
  HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/,/:/g'`
  if $cygwin; then
  HIVE_AUX_JARS_PATH=`cygpath -p -w "$HIVE_AUX_JARS_PATH"`
  HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/;/,/g'`
  fi
  AUX_CLASSPATH=${AUX_CLASSPATH}:${HIVE_AUX_JARS_PATH}
  AUX_PARAM="file://$(echo ${HIVE_AUX_JARS_PATH} | sed 's/:/,file:\/\//g')"
fi
{code}

But in the LLAP Service Driver, it's processed as :

{code:java}
 private void addAuxJarsToSet(HashSet auxJarSet, String auxJars) {
  if (auxJars != null && !auxJars.isEmpty()) {
// TODO: transitive dependencies warning?
String[] jarPaths = auxJars.split(",");
for (String jarPath : jarPaths) {
  if (!jarPath.isEmpty()) {
auxJarSet.add(jarPath);
  }
}
  }
}
  };
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)