date:20180905

Re: Review Request 68525: HIVE-20296 Improve HivePointLookupOptimizerRule to be able to extract from more sophisticated contexts

2018-09-05 Thread Ashutosh Chauhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68525/#review208392
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
Lines 278 (patched)


Does this class also exist in Calcite? If so, can you please leave a note 
here to remove from Hive once these are made public in Calcite?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
Lines 307 (patched)


LOG.debug



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
Lines 319 (patched)


Error msg : Unable to find constraint which was earlier added.


- Ashutosh Chauhan


On Aug. 27, 2018, 4:01 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68525/
> ---
> 
> (Updated Aug. 27, 2018, 4:01 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20296
> https://issues.apache.org/jira/browse/HIVE-20296
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> generalized rule to extract INs from more complex filter conditions as well.
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePointLookupOptimizerRule.java
>  eff9a312aaa09a889f0c6a045bd813d1fb633956 
>   
> ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/rules/TestHivePointLookupOptimizerRule.java
>  PRE-CREATION 
>   ql/src/test/results/clientpositive/druid/druidmini_test_ts.q.out 
> a8e6894a9786318ed4362ee37d918a3699f074a0 
>   ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 
> 260ba1cbddee7f0946f0cdec1070359ab2a1d2aa 
>   ql/src/test/results/clientpositive/perf/spark/query15.q.out 
> 67684f6b0bc44c0cae6107be94d131a083eca0e1 
>   ql/src/test/results/clientpositive/perf/spark/query47.q.out 
> 690b1054c12f7d588015afc301802d9d2da2d0b9 
>   ql/src/test/results/clientpositive/perf/spark/query57.q.out 
> 51e644a87bf4befbe4368cdf03b5eaab6d4f2049 
>   ql/src/test/results/clientpositive/perf/tez/query15.q.out 
> e1eca99d95e13070f901b75a406391435b2b4f1d 
>   ql/src/test/results/clientpositive/perf/tez/query47.q.out 
> d034ea9433a3b1c54c545edf526c215bf79388e1 
>   ql/src/test/results/clientpositive/perf/tez/query57.q.out 
> 42cbbdc2a4d8bc469c9d91867faf21fc94a057ea 
> 
> 
> Diff: https://reviews.apache.org/r/68525/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zoltan Haindrich
> 
>

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

2018-09-05 Thread Vineet Garg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66370/#review208385
---


Ship it!




Ship It!

- Vineet Garg


On Sept. 5, 2018, 8 p.m., Igor Kryvenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66370/
> ---
> 
> (Updated Sept. 5, 2018, 8 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Vineet Garg.
> 
> 
> Bugs: HIVE-18725
> https://issues.apache.org/jira/browse/HIVE-18725
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> If there is a column reference within subquery which doesn't exist Hive 
> throws misleading error message.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java df40a2878d 
>   ql/src/test/queries/clientnegative/subquery_non_exisiting_column.q 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_corr_from.q.out 3af1a8a710 
>   ql/src/test/results/clientnegative/subquery_corr_grandparent.q.out 
> 6f06fc71b0 
>   ql/src/test/results/clientnegative/subquery_corr_select.q.out d95b9397cf 
>   ql/src/test/results/clientnegative/subquery_non_exisiting_column.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_scalar_multi_columns.q.out 
> ee1e44e1d2 
> 
> 
> Diff: https://reviews.apache.org/r/66370/diff/6/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Igor Kryvenko
> 
>

[jira] [Created] (HIVE-20509) Plan: fix wasted memory in plans with large partition counts

2018-09-05 Thread Gopal V (JIRA)

Gopal V created HIVE-20509:
--

 Summary: Plan: fix wasted memory in plans with large partition 
counts
 Key: HIVE-20509
 URL: https://issues.apache.org/jira/browse/HIVE-20509
 Project: Hive
  Issue Type: Bug
  Components: Query Planning
Reporter: Gopal V


{code}
  public void addPathToAlias(Path path, String newAlias){
ArrayList aliases = pathToAliases.get(path);
if (aliases == null) {
  aliases = new ArrayList<>();
  StringInternUtils.internUriStringsInPath(path);
  pathToAliases.put(path, aliases);
}
aliases.add(newAlias.intern());
  }
{code}

ArrayList::DEFAULT_CAPACITY is 10, so this wastes 500 bytes of memory due to 
the {{new ArrayList<>();}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68633: [HIVE-20481] Add record key column

2018-09-05 Thread Slim Bouguerra


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68633/
---

(Updated Sept. 5, 2018, 9:15 p.m.)


Review request for hive, Gopal V and Vineet Garg.


Bugs: [HIVE-20481]
https://issues.apache.org/jira/browse/[HIVE-20481]


Repository: hive-git


Description
---

This Patch adds the kafka record key as an extra metadata column.


Diffs
-

  
itests/qtest-druid/src/main/java/org/apache/hive/kafka/SingleNodeKafkaCluster.java
 c9339b565e 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/GenericKafkaSerDe.java 
e7ea53f4bc 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaJsonSerDe.java 
f383190083 
  
kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaPullerRecordReader.java
 908ee5e29f 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaRecordWritable.java 
c6924ea480 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaScanTrimmer.java 
76415151ec 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaStorageHandler.java 
5847df5e7e 
  
kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaStorageHandlerInfo.java
 PRE-CREATION 
  kafka-handler/src/java/org/apache/hadoop/hive/kafka/KafkaStreamingUtils.java 
d2d0ebc192 
  
kafka-handler/src/test/org/apache/hadoop/hive/kafka/KafkaPullerInputSplitTest.java
 be26986818 
  
kafka-handler/src/test/org/apache/hadoop/hive/kafka/KafkaRecordIteratorTest.java
 5de51cd00a 
  
kafka-handler/src/test/org/apache/hadoop/hive/kafka/KafkaRecordWritableTest.java
 8f9df548f7 
  kafka-handler/src/test/org/apache/hadoop/hive/kafka/KafkaScanTrimmerTest.java 
289dafde36 
  
kafka-handler/src/test/org/apache/hadoop/hive/kafka/KafkaStreamingUtilsTest.java
 8d68ec27c8 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/StorageHandlerInfo.java 
dbc44a6621 
  ql/src/test/queries/clientpositive/kafka_storage_handler.q 8daa3e3bc0 
  ql/src/test/results/clientpositive/druid/kafka_storage_handler.q.out 
3dec33d790 
  testutils/ptest2/conf/deployed/master-mr2.properties 90a654cf8a 


Diff: https://reviews.apache.org/r/68633/diff/1/


Testing
---

Via Qfiles


Thanks,

Slim Bouguerra

[jira] [Created] (HIVE-20508) Hive does not support user names of type "user@realm"

2018-09-05 Thread Deepak Jaiswal (JIRA)

Deepak Jaiswal created HIVE-20508:
-

 Summary: Hive does not support user names of type "user@realm"
 Key: HIVE-20508
 URL: https://issues.apache.org/jira/browse/HIVE-20508
 Project: Hive
  Issue Type: Bug
Reporter: Deepak Jaiswal
Assignee: Deepak Jaiswal


Hive does not support user names of type "user@realm". This causes 
authentication problem for user names containing email ids in Kerberos 
environment.

 

cc [~thejas]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68310: HIVE-17040

2018-09-05 Thread Jesús Camacho Rodríguez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68310/
---

(Updated Sept. 5, 2018, 9:11 p.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-17040
https://issues.apache.org/jira/browse/HIVE-17040


Repository: hive-git


Description
---

HIVE-17040


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
40ea3ac0c5cd0943a4e9dbe2b0e8b952070a8a67 
  itests/src/test/resources/testconfiguration.properties 
a3a70ecd498bbfc6146cfbfbeb1f265032f440ef 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinConstraintsRule.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectJoinTransposeRule.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectMergeRule.java
 07518df9ec1cea1c331846bbe636cf1a039e762f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
df40a2878d328b76f7de74e2c2db0a029ba4610f 
  ql/src/test/queries/clientpositive/join_constraints_optimization.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_1.q 
ee4844277e5dd1d9ea3911ad2e33a9c3f2481344 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_3.q 
4aadd5fb0a4bc74c5bc4088b3d4e62c81bf9cf8b 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_4.q 
dc20b68ba9a9e0bbcb9c414b10b00f3228fe63fe 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_5.q 
0e4fdf49ac04935f6f14c2ceaa0969008b34f926 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_6.q 
4f05f76330cb74be50187e3b175d4675f5ea8763 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_7.q 
59ed5757569a8dde70fe04eb9ec5e8c91b5931bf 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_no_join_opt.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_no_join_opt_2.q 
PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_rewrite_part_1.q 
5a2e74c8a005ae8422c00a998fa3c07183749176 
  ql/src/test/results/clientpositive/ambiguitycheck.q.out 
80c9582fec9754fe56400064ab3f88e3e9ea2da7 
  
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
 7813aac29465b5193789464fcd32771741a98071 
  ql/src/test/results/clientpositive/druid/druidmini_mv.q.out 
806262d72e687bbdd09b47380eed77c14764c2a5 
  ql/src/test/results/clientpositive/list_bucket_dml_2.q.out 
bd8e215c2207c48ce2e446fcc10333ec0fb4648c 
  ql/src/test/results/clientpositive/list_bucket_dml_4.q.out 
520d48e3d9fa67301852efc9ed1c92494fd528a0 
  ql/src/test/results/clientpositive/list_bucket_dml_9.q.out 
fbd4fde1bd8d2f7379585afe07c018139b7d64e8 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out 
e324cab738c22e31089f437d6e7d8a65160dc5b9 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 
ec1e54060cb7d380bda4d965fff540d002bd456a 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out 
889f23c6da7e3c7b955b83d44afd1bd048468b49 
  ql/src/test/results/clientpositive/list_bucket_query_oneskew_1.q.out 
dcff8a50370b4ebb07300d4a7d9c34aa84f4ae17 
  ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 
268051e2acbbabd02206c5a21690fb563a3dcd2b 
  ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 
3951b71227d93469347264362492e64554efee32 
  ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 
260ba1cbddee7f0946f0cdec1070359ab2a1d2aa 
  ql/src/test/results/clientpositive/llap/current_date_timestamp.q.out 
6831fb2573788033393544b835f1a56d69fb1712 
  ql/src/test/results/clientpositive/llap/join_constraints_optimization.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out 
71adebb2acad1545c48d45835cd5876434b141b5 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_dummy.q.out
 ce1c281bea0bd2a96e67057ee990fa5b45850905 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
 98f74379f6275aed90273929c65144e4bbd51a97 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_1.q.out 
4d8fa52aa94a77faa9d1d1872afcb016e5a3c438 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_2.q.out 
8e54abe61d48ebd3439ee215e88c2186601a4cd3 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_3.q.out 
d7536e408798d38970a30226f7248d69a10b0903 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_4.q.out 
3fd4c59ee68c3b0df56d070df900fbea31e3f6aa 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_5.q.out 
9992409f6aa36f6be18f6b07a6de380b2be18a9a 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_6.q.out 
544c395c0125608b3954c7cc9d51ee088d2bdb51 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_7.q.out

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

2018-09-05 Thread Igor Kryvenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66370/
---

(Updated Сен. 5, 2018, 8 п.п.)


Review request for hive, Ashutosh Chauhan and Vineet Garg.


Bugs: HIVE-18725
https://issues.apache.org/jira/browse/HIVE-18725


Repository: hive-git


Description
---

If there is a column reference within subquery which doesn't exist Hive throws 
misleading error message.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java df40a2878d 
  ql/src/test/queries/clientnegative/subquery_non_exisiting_column.q 
PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_corr_from.q.out 3af1a8a710 
  ql/src/test/results/clientnegative/subquery_corr_grandparent.q.out 6f06fc71b0 
  ql/src/test/results/clientnegative/subquery_corr_select.q.out d95b9397cf 
  ql/src/test/results/clientnegative/subquery_non_exisiting_column.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_scalar_multi_columns.q.out 
ee1e44e1d2 


Diff: https://reviews.apache.org/r/66370/diff/6/

Changes: https://reviews.apache.org/r/66370/diff/5-6/


Testing
---


Thanks,

Igor Kryvenko

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

2018-09-05 Thread Vineet Garg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66370/#review208367
---




ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 528 (patched)


Why should missing stats cause AST to reanalyze? This doesn't look correct.



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 539 (patched)


Same as above. This doesn't look correct.



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 544 (patched)


So basically if CBO fails due to some reason beside unsupported feature we 
will end up throwing semantic exception. This will cause the code to not try 
non-cbo in case of cbo failure. I don't think we want to do that.


- Vineet Garg


On July 1, 2018, 8:55 a.m., Igor Kryvenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66370/
> ---
> 
> (Updated July 1, 2018, 8:55 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Vineet Garg.
> 
> 
> Bugs: HIVE-18725
> https://issues.apache.org/jira/browse/HIVE-18725
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> If there is a column reference within subquery which doesn't exist Hive 
> throws misleading error message.
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSubquerySemanticException.java
>  4321a5c789 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteViewSemanticException.java
>  c2a4e94a03 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java e091f38bc6 
>   ql/src/test/queries/clientnegative/subquery_non_exisiting_column.q 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_non_exisiting_column.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66370/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Igor Kryvenko
> 
>

[jira] [Created] (HIVE-20507) Beeline: Add a utility command to retrieve all uris from beeline-site.xml

2018-09-05 Thread Vaibhav Gumashta (JIRA)

Vaibhav Gumashta created HIVE-20507:
---

 Summary: Beeline: Add a utility command to retrieve all uris from 
beeline-site.xml
 Key: HIVE-20507
 URL: https://issues.apache.org/jira/browse/HIVE-20507
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 3.1.0
Reporter: Vaibhav Gumashta


It will be useful for some clients to get the url list when beeline-site is 
present. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

2018-09-05 Thread Brock Noland (JIRA)

Brock Noland created HIVE-20506:
---

 Summary: HOS times out when cluster is full while Hive-on-MR waits
 Key: HIVE-20506
 URL: https://issues.apache.org/jira/browse/HIVE-20506
 Project: Hive
  Issue Type: Improvement
Reporter: Brock Noland


My understanding is as follows:

Hive-on-MR when the cluster is full will wait for resources to be available 
before submitting a job. This is because the hadoop jar command is the primary 
mechanism Hive uses to know if a job is complete.

 

Hive-on-Spark will timeout after {{SPARK_RPC_CLIENT_CONNECT_TIMEOUT}} because 
the RPC client in the AppMaster doesn't connect back to the RPC Server in HS2. 

This is a behavior difference it'd be great to close.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] hive pull request #390: HIVE-20080: TxnHandler checkLock direct sql fail wit...

2018-09-05 Thread rajkrrsingh

Github user rajkrrsingh closed the pull request at:

https://github.com/apache/hive/pull/390


---

Review Request 68630: HIVE-20420: Provide a fallback authorizer when no other authorizer is in use

2018-09-05 Thread Daniel Dai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68630/
---

Review request for hive.


Repository: hive-git


Description
---

See HIVE-20420


Diffs
-

  ql/pom.xml a55cbe3 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/SettableConfigUpdater.java
 12be41c 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/fallback/FallbackHiveAuthorizer.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/fallback/FallbackHiveAuthorizerFactory.java
 PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_addjar.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_compile.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_create_func1.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_create_func2.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_dfs.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_disallow_transform.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_load.q PRE-CREATION 
  ql/src/test/queries/clientnegative/fallbackauth_set_invalidconf.q 
PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_addjar.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_compile.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_create_func1.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_create_func2.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_dfs.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_disallow_transform.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_load.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/fallbackauth_set_invalidconf.q.out 
PRE-CREATION 


Diff: https://reviews.apache.org/r/68630/diff/1/


Testing
---


Thanks,

Daniel Dai

[jira] [Created] (HIVE-20505) upgrade org.openjdk.jmh:jmh-core to 1.21

2018-09-05 Thread Laszlo Pinter (JIRA)

Laszlo Pinter created HIVE-20505:


 Summary: upgrade org.openjdk.jmh:jmh-core to 1.21
 Key: HIVE-20505
 URL: https://issues.apache.org/jira/browse/HIVE-20505
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 4.0.0
Reporter: Laszlo Pinter
Assignee: Laszlo Pinter
 Fix For: 4.0.0


This ticket tracks the following CVE(s) that were found in the direct 
dependency *org.openjdk.jmh:jmh-core:1.19*: 
[CVE-2009-1896|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-1896], 
[CVE-2009-2689|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-2689], 
[CVE-2009-3879|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3879], 
[CVE-2009-0733|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-0733], 
[CVE-2009-2475|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-2475], 
[CVE-2009-3883|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3883], 
[CVE-2009-2476|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-2476], 
[CVE-2009-3884|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3884], 
[CVE-2013-0169|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-0169], 
[CVE-2012-5373|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-5373], 
[CVE-2009-3880|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3880], 
[CVE-2009-3881|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3881], 
[CVE-2009-3882|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3882], 
[CVE-2009-0581|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-0581], 
[CVE-2009-2690|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-2690], 
[CVE-2012-2739|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-2739], 
[CVE-2009-0723|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-0723], 
[CVE-2009-3728|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3728] 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-20504) Give simple MJ bigger priority than bucketized ones

2018-09-05 Thread Zoltan Haindrich (JIRA)

Zoltan Haindrich created HIVE-20504:
---

 Summary: Give simple MJ bigger priority than bucketized ones
 Key: HIVE-20504
 URL: https://issues.apache.org/jira/browse/HIVE-20504
 Project: Hive
  Issue Type: Improvement
  Components: Statistics
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


from the code it seems "standard" mapjoin is one of the last one tried; in case 
the table estimated to be bucketed in to 2 - but it's small ; Hive willl do a 
bucketmapjoin or  dphj...even thru a simple mapjoin could have been an 
alternative...
https://github.com/apache/hive/blob/154ca3e3b5eb78cd49a4b3650c750ca731fba7da/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java#L157



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-20503) Use datastructure aware estimations during mapjoin selection

2018-09-05 Thread Zoltan Haindrich (JIRA)

Zoltan Haindrich created HIVE-20503:
---

 Summary: Use datastructure aware estimations during mapjoin 
selection
 Key: HIVE-20503
 URL: https://issues.apache.org/jira/browse/HIVE-20503
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] hive pull request #424: HIVE-20476: CopyUtils used by REPL LOAD and EXPORT/I...

2018-09-05 Thread sankarh

Github user sankarh closed the pull request at:

https://github.com/apache/hive/pull/424


---

[jira] [Created] (HIVE-20502) Fix NPE while running skewjoin_mapjoin10.q when column stats is used.

2018-09-05 Thread Zoltan Haindrich (JIRA)

Zoltan Haindrich created HIVE-20502:
---

 Summary: Fix NPE while running skewjoin_mapjoin10.q when column 
stats is used.
 Key: HIVE-20502
 URL: https://issues.apache.org/jira/browse/HIVE-20502
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


Enabling {{hive.stats.fetch.column.stats}} makes this test fail during:

{code}
EXPLAIN
SELECT a.*, b.* FROM T1_n151 a RIGHT OUTER JOIN T2_n88 b ON a.key = b.key
{code}

Seems like joinKeys is null at [this 
point|https://github.com/apache/hive/blob/48f92c31dee3983f573f2e66baaa213a0196f1ba/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2169]

Exception:

{code}
2018-09-04T23:47:02,398 DEBUG [fef236ce-e62e-4c20-b0c0-3b15d2b336f7 main] 
annotation.StatsRulesProcFactory: STATS-JOIN[15]: detects none/multiple PK 
parents.
2018-09-04T23:47:02,409 ERROR [fef236ce-e62e-4c20-b0c0-3b15d2b336f7 main] 
ql.Driver: FAILED: NullPointerException null
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.isJoinKey(StatsRulesProcFactory.java:2169)
at 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.updateNumNulls(StatsRulesProcFactory.java:2210)
at 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.updateColStats(StatsRulesProcFactory.java:2276)
at 
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.process(StatsRulesProcFactory.java:1785)
at 
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
at 
org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
{code}





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68525: HIVE-20296 Improve HivePointLookupOptimizerRule to be able to extract from more sophisticated contexts

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

[jira] [Created] (HIVE-20509) Plan: fix wasted memory in plans with large partition counts

Re: Review Request 68633: [HIVE-20481] Add record key column

[jira] [Created] (HIVE-20508) Hive does not support user names of type "user@realm"

Re: Review Request 68310: HIVE-17040

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

Re: Review Request 66370: HIVE-18725: Improve error handling for subqueries if there is wrong column reference

[jira] [Created] (HIVE-20507) Beeline: Add a utility command to retrieve all uris from beeline-site.xml

[jira] [Created] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

[GitHub] hive pull request #390: HIVE-20080: TxnHandler checkLock direct sql fail wit...

Review Request 68630: HIVE-20420: Provide a fallback authorizer when no other authorizer is in use

[jira] [Created] (HIVE-20505) upgrade org.openjdk.jmh:jmh-core to 1.21

[jira] [Created] (HIVE-20504) Give simple MJ bigger priority than bucketized ones

[jira] [Created] (HIVE-20503) Use datastructure aware estimations during mapjoin selection

[GitHub] hive pull request #424: HIVE-20476: CopyUtils used by REPL LOAD and EXPORT/I...

[jira] [Created] (HIVE-20502) Fix NPE while running skewjoin_mapjoin10.q when column stats is used.

17 matches

Site Navigation

Mail list logo

Footer information