[jira] [Commented] (HIVE-20531) Repl load on cloud storage file system can skip redundant move or add partition tasks.
[
https://issues.apache.org/jira/browse/HIVE-20531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633844#comment-16633844
]
Hive QA commented on HIVE-20531:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12941913/HIVE-20531.07.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14154/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14154/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14154/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-01 10:48:16.881
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14154/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-01 10:48:16.884
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4570807 HIVE-20618: During join selection BucketMapJoin might be
choosen for non bucketed tables (Zoltan Haindrich reviewed by Ashutosh Chauhan,
Deepak Jaiswal)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4570807 HIVE-20618: During join selection BucketMapJoin might be
choosen for non bucketed tables (Zoltan Haindrich reviewed by Ashutosh Chauhan,
Deepak Jaiswal)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-01 10:48:22.833
+ rm -rf ../yetus_PreCommit-HIVE-Build-14154
+ mkdir ../yetus_PreCommit-HIVE-Build-14154
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14154
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14154/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java:
does not exist in index
error:
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadTable.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java: does
not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java: does
not exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc5044337894651540410.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc5044337894651540410.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
[ERROR] Failed to execute goal
[jira] [Commented] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633842#comment-16633842
]
Hive QA commented on HIVE-20660:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12941911/HIVE-20660.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 114 failed/errored test(s), 15008 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_annotate_stats_groupby]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[topnkey] (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_topnkey]
(batchId=45)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[gby_stats_estimate]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_all]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_distinct]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_merge]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_stats]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llapdecider]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge10]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge1]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parallel_colstats]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[reduce_deduplicate_distinct]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_groupby]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats_orc]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[constprog_semijoin]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer1]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer6]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[count_dist_rewrite]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization2]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_4]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[filter_union]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_resolution]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[having]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown3]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_partitioned]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_dyn_part3]
(batchId=160)
[jira] [Updated] (HIVE-20535) Add new configuration to set the size of the global compile lock
[ https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] denys kuzmenko updated HIVE-20535: -- Attachment: HIVE-20535.21.patch > Add new configuration to set the size of the global compile lock > > > Key: HIVE-20535 > URL: https://issues.apache.org/jira/browse/HIVE-20535 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: denys kuzmenko >Assignee: denys kuzmenko >Priority: Major > Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch, > HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch, > HIVE-20535.14.patch, HIVE-20535.15.patch, HIVE-20535.16.patch, > HIVE-20535.17.patch, HIVE-20535.18.patch, HIVE-20535.19.patch, > HIVE-20535.2.patch, HIVE-20535.20.patch, HIVE-20535.21.patch, > HIVE-20535.3.patch, HIVE-20535.4.patch, HIVE-20535.5.patch, > HIVE-20535.6.patch, HIVE-20535.8.patch, HIVE-20535.9.patch > > > When removing the compile lock, it is quite risky to remove it entirely. > It would be good to provide a pool size for the concurrent compilation, so > the administrator can limit the load -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20535) Add new configuration to set the size of the global compile lock
[ https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] denys kuzmenko updated HIVE-20535: -- Attachment: (was: HIVE-20535.21.patch) > Add new configuration to set the size of the global compile lock > > > Key: HIVE-20535 > URL: https://issues.apache.org/jira/browse/HIVE-20535 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: denys kuzmenko >Assignee: denys kuzmenko >Priority: Major > Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch, > HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch, > HIVE-20535.14.patch, HIVE-20535.15.patch, HIVE-20535.16.patch, > HIVE-20535.17.patch, HIVE-20535.18.patch, HIVE-20535.19.patch, > HIVE-20535.2.patch, HIVE-20535.20.patch, HIVE-20535.3.patch, > HIVE-20535.4.patch, HIVE-20535.5.patch, HIVE-20535.6.patch, > HIVE-20535.8.patch, HIVE-20535.9.patch > > > When removing the compile lock, it is quite risky to remove it entirely. > It would be good to provide a pool size for the concurrent compilation, so > the administrator can limit the load -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20535) Add new configuration to set the size of the global compile lock
[ https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] denys kuzmenko updated HIVE-20535: -- Attachment: HIVE-20535.21.patch > Add new configuration to set the size of the global compile lock > > > Key: HIVE-20535 > URL: https://issues.apache.org/jira/browse/HIVE-20535 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: denys kuzmenko >Assignee: denys kuzmenko >Priority: Major > Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch, > HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch, > HIVE-20535.14.patch, HIVE-20535.15.patch, HIVE-20535.16.patch, > HIVE-20535.17.patch, HIVE-20535.18.patch, HIVE-20535.19.patch, > HIVE-20535.2.patch, HIVE-20535.20.patch, HIVE-20535.21.patch, > HIVE-20535.3.patch, HIVE-20535.4.patch, HIVE-20535.5.patch, > HIVE-20535.6.patch, HIVE-20535.8.patch, HIVE-20535.9.patch > > > When removing the compile lock, it is quite risky to remove it entirely. > It would be good to provide a pool size for the concurrent compilation, so > the administrator can limit the load -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20660) Group by statistics estimation could be improved by bounding the total number of rows to source table
[
https://issues.apache.org/jira/browse/HIVE-20660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633803#comment-16633803
]
Hive QA commented on HIVE-20660:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 8 new + 38 unchanged - 0 fixed
= 46 total (was 38) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14153/dev-support/hive-personality.sh
|
| git revision | master / 4570807 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14153/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14153/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14153/yetus/whitespace-tabs.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14153/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Group by statistics estimation could be improved by bounding the total number
> of rows to source table
> -
>
> Key: HIVE-20660
> URL: https://issues.apache.org/jira/browse/HIVE-20660
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20660.1.patch
>
>
> Currently the stats for group by is estimated by taking product of NDVs of
> all the keys and bounding it by the number of rows of its input. This bound
> could be improved by using the source table instead of immediate input, the
> insight in this case
[jira] [Commented] (HIVE-20644) Avoid exposing sensitive infomation through a Hive Runtime exception
[
https://issues.apache.org/jira/browse/HIVE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633772#comment-16633772
]
Hive QA commented on HIVE-20644:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12941915/HIVE-20644.02
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15007 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_dyn_part3]
(batchId=160)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14152/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14152/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14152/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12941915 - PreCommit-HIVE-Build
> Avoid exposing sensitive infomation through a Hive Runtime exception
>
>
> Key: HIVE-20644
> URL: https://issues.apache.org/jira/browse/HIVE-20644
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-20644.01, HIVE-20644.02
>
>
> The HiveException raised from the following methods is exposing the datarow
> the caused the run time exception.
> # ReduceRecordSource::GroupIterator::next() - around line 372
> # MapOperator::process() - around line 567
> # ExecReducer::reduce() - around line 243
> In all the cases, a string representation of the row is constructed on the
> fly and is included in
> the error message.
> VectorMapOperator::process() - around line 973 raises the same exception but
> it's not exposing the row since the row contents are not included in the
> error message.
> While trying to reproduce above error, I also found that the arguments to a
> UDF get exposed in log messages from FunctionRegistry::invoke() around line
> 1114. This too can cause sensitive information to be leaked through error
> message.
> This way some sensitive information is leaked to a user through exception
> message. That information may not be available to the user otherwise. Hence
> it's a kind of security breach or violation of access control.
> The contents of the row or the arguments to a function may be useful for
> debugging and hence it's worth to add those to logs. Hence proposal here to
> log a separate message with log level DEBUG or INFO containing the string
> representation of the row. Users can configure their logging so that
> DEBUG/INFO messages do not go to the client but at the same time are
> available in the hive server logs for debugging. The actual exception message
> will not contain any sensitive data like row data or argument data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20575) Fix flaky connection metric tests
[ https://issues.apache.org/jira/browse/HIVE-20575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633745#comment-16633745 ] Peter Vary commented on HIVE-20575: --- +1 > Fix flaky connection metric tests > - > > Key: HIVE-20575 > URL: https://issues.apache.org/jira/browse/HIVE-20575 > Project: Hive > Issue Type: Test > Components: Hive, Test >Affects Versions: 4.0.0 >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-20575.01.patch, HIVE-20575.02.patch, > HIVE-20575.03.patch, HIVE-20575.04.patch, HIVE-20575.05.patch, > HIVE-20575.06.patch, HIVE-20575.07.patch, HIVE-20575.08.patch > > > TestHs2ConnectionMetricsHttp.testOpenConnectionMetrics() is flaky. We need to > fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20644) Avoid exposing sensitive infomation through a Hive Runtime exception
[
https://issues.apache.org/jira/browse/HIVE-20644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633730#comment-16633730
]
Hive QA commented on HIVE-20644:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14152/dev-support/hive-personality.sh
|
| git revision | master / 4570807 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14152/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Avoid exposing sensitive infomation through a Hive Runtime exception
>
>
> Key: HIVE-20644
> URL: https://issues.apache.org/jira/browse/HIVE-20644
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-20644.01, HIVE-20644.02
>
>
> The HiveException raised from the following methods is exposing the datarow
> the caused the run time exception.
> # ReduceRecordSource::GroupIterator::next() - around line 372
> # MapOperator::process() - around line 567
> # ExecReducer::reduce() - around line 243
> In all the cases, a string representation of the row is constructed on the
> fly and is included in
> the error message.
> VectorMapOperator::process() - around line 973 raises the same exception but
> it's not exposing the row since the row contents are not included in the
> error message.
> While trying to reproduce above error, I also found that the arguments to a
> UDF get exposed in log messages from FunctionRegistry::invoke() around line
> 1114. This too can cause sensitive information to be leaked through error
> message.
> This way some sensitive information is leaked to a user through exception
> message. That information may not be available to the user otherwise. Hence
> it's a kind of security breach or violation of access control.
> The contents of the row or the arguments to a function may be useful
[jira] [Updated] (HIVE-20646) Partition filter condition is not pushed down to metastore query if it has IS NOT NULL.
[ https://issues.apache.org/jira/browse/HIVE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20646: Status: Patch Available (was: Open) > Partition filter condition is not pushed down to metastore query if it has IS > NOT NULL. > --- > > Key: HIVE-20646 > URL: https://issues.apache.org/jira/browse/HIVE-20646 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Metastore, Performance, pull-request-available > Attachments: HIVE-20646.01.patch, HIVE-20646.02.patch > > > If the partition filter condition has "is not null" then the filter query > isn't getting pushed to the SQL query in RDMBS. > This slows down metastore api calls for getting list of partitions with > filter condition. > This condition gets added by optimizer in many cases so this is affecting > many queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20646) Partition filter condition is not pushed down to metastore query if it has IS NOT NULL.
[ https://issues.apache.org/jira/browse/HIVE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20646: Attachment: (was: HIVE-20646.02.patch) > Partition filter condition is not pushed down to metastore query if it has IS > NOT NULL. > --- > > Key: HIVE-20646 > URL: https://issues.apache.org/jira/browse/HIVE-20646 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Metastore, Performance, pull-request-available > Attachments: HIVE-20646.01.patch, HIVE-20646.02.patch > > > If the partition filter condition has "is not null" then the filter query > isn't getting pushed to the SQL query in RDMBS. > This slows down metastore api calls for getting list of partitions with > filter condition. > This condition gets added by optimizer in many cases so this is affecting > many queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20646) Partition filter condition is not pushed down to metastore query if it has IS NOT NULL.
[ https://issues.apache.org/jira/browse/HIVE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20646: Attachment: HIVE-20646.02.patch > Partition filter condition is not pushed down to metastore query if it has IS > NOT NULL. > --- > > Key: HIVE-20646 > URL: https://issues.apache.org/jira/browse/HIVE-20646 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Metastore, Performance, pull-request-available > Attachments: HIVE-20646.01.patch, HIVE-20646.02.patch > > > If the partition filter condition has "is not null" then the filter query > isn't getting pushed to the SQL query in RDMBS. > This slows down metastore api calls for getting list of partitions with > filter condition. > This condition gets added by optimizer in many cases so this is affecting > many queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20646) Partition filter condition is not pushed down to metastore query if it has IS NOT NULL.
[ https://issues.apache.org/jira/browse/HIVE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20646: Status: Open (was: Patch Available) > Partition filter condition is not pushed down to metastore query if it has IS > NOT NULL. > --- > > Key: HIVE-20646 > URL: https://issues.apache.org/jira/browse/HIVE-20646 > Project: Hive > Issue Type: Improvement > Components: HiveServer2, Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Metastore, Performance, pull-request-available > Attachments: HIVE-20646.01.patch, HIVE-20646.02.patch > > > If the partition filter condition has "is not null" then the filter query > isn't getting pushed to the SQL query in RDMBS. > This slows down metastore api calls for getting list of partitions with > filter condition. > This condition gets added by optimizer in many cases so this is affecting > many queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20535) Add new configuration to set the size of the global compile lock
[ https://issues.apache.org/jira/browse/HIVE-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] denys kuzmenko updated HIVE-20535: -- Attachment: HIVE-20535.20.patch > Add new configuration to set the size of the global compile lock > > > Key: HIVE-20535 > URL: https://issues.apache.org/jira/browse/HIVE-20535 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: denys kuzmenko >Assignee: denys kuzmenko >Priority: Major > Attachments: HIVE-20535.1.patch, HIVE-20535.10.patch, > HIVE-20535.11.patch, HIVE-20535.12.patch, HIVE-20535.13.patch, > HIVE-20535.14.patch, HIVE-20535.15.patch, HIVE-20535.16.patch, > HIVE-20535.17.patch, HIVE-20535.18.patch, HIVE-20535.19.patch, > HIVE-20535.2.patch, HIVE-20535.20.patch, HIVE-20535.3.patch, > HIVE-20535.4.patch, HIVE-20535.5.patch, HIVE-20535.6.patch, > HIVE-20535.8.patch, HIVE-20535.9.patch > > > When removing the compile lock, it is quite risky to remove it entirely. > It would be good to provide a pool size for the concurrent compilation, so > the administrator can limit the load -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20623) Shared work: Extend sharing of map-join cache entries in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633702#comment-16633702
]
Hive QA commented on HIVE-20623:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12941912/HIVE-20623.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15007 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_dyn_part3]
(batchId=160)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14151/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14151/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14151/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12941912 - PreCommit-HIVE-Build
> Shared work: Extend sharing of map-join cache entries in LLAP
> -
>
> Key: HIVE-20623
> URL: https://issues.apache.org/jira/browse/HIVE-20623
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logical Optimizer
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20623.01.patch, HIVE-20623.02.patch,
> HIVE-20623.02.patch, HIVE-20623.02.patch, HIVE-20623.03.patch,
> HIVE-20623.03.patch, HIVE-20623.03.patch, HIVE-20623.patch,
> hash-shared-work.json.txt, hash-shared-work.svg
>
>
> For a query like this
> {code}
> with all_sales as (
> select ss_customer_sk as customer_sk, ss_ext_list_price-ss_ext_discount_amt
> as ext_price from store_sales
> UNION ALL
> select ws_bill_customer_sk as customer_sk,
> ws_ext_list_price-ws_ext_discount_amt as ext_price from web_sales
> UNION ALL
> select cs_bill_customer_sk as customer_sk, cs_ext_sales_price -
> cs_ext_discount_amt as ext_price from catalog_sales)
> select sum(ext_price) total_price, c_customer_id from all_sales, customer
> where customer_sk = c_customer_sk
> group by c_customer_id
> order by total_price desc
> limit 100;
> {code}
> The hashtable used for all 3 joins are identical, which is loaded 3x times in
> the same LLAP instance because they are named.
> {code}
> cacheKey = "HASH_MAP_" + this.getOperatorId() + "_container";
> {code}
> in the cache.
> If those are identical in nature (i.e vectorization, hashtable type etc),
> then the duplication is just wasted CPU, memory and network - using the cache
> name for hashtables which will be identical in layout would be extremely
> useful.
> In cases where the join is pushed through a UNION, those are identical.
> This optimization can only be done without concern for accidental delays when
> the same upstream task is generating all of these hashtables, which is what
> is achieved by the shared scan optimizer already.
> In case the shared work is not present, this has potential downsides - in
> case two customer broadcasts were sourced from "Map 1" and "Map 2", the Map 1
> builder will block the other task from reading from Map 2, even though Map 2
> might have started after, but finished ahead of Map 1.
> So this specific optimization can always be considered for cases where the
> shared work unifies the operator tree and the parents of all the RS entries
> involved are same (& the RS layout is the same).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20544) TOpenSessionReq logs password and username
[
https://issues.apache.org/jira/browse/HIVE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-20544:
-
Attachment: HIVE-20544.4.patch
Status: Patch Available (was: Open)
> TOpenSessionReq logs password and username
> --
>
> Key: HIVE-20544
> URL: https://issues.apache.org/jira/browse/HIVE-20544
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: beginner, patch, security
> Attachments: HIVE-20544.1.patch, HIVE-20544.2.patch,
> HIVE-20544.3.patch, HIVE-20544.3.patch, HIVE-20544.4.patch,
> HIVE-20544.4.patch, HIVE-20544.4.patch, HIVE-20544.4.patch,
> HIVE-20544.4.patch, HIVE-20544.patch, non-solution.patch,
> working-solution.patch
>
>
> In
> service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TOpenSessionReq,
> if client protocol is unset, validate() and toString() prints both username
> and password to logs.
> Logging a password is a security risk. We should hide the ***.
> =Edit= (no longer relevant, see comments)
> This issue is tricky since it is caused in a fully generated class. I've been
> playing around and have found one working solution, butI'd truly appreciate
> ideas for a more elegant solution or input.
> The problem:
> TCLIService.thrift is the template for generating all classes in
> service-rpc. Struct TOpenSessionReq is OpenSession()'s one parameter and is
> defined thus:
> {noformat}
> struct TOpenSessionReq {
> 1: required TProtocolVersion client_protocol =
> TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10
> 2: optional string username
> 3: optional string password
> 4: optional map configuration
> }
> {noformat}
> In the generated class TOpenSessionReq.java, client_protocol is checked by a
> validate() method, which is called quite a few times; if client_protocol is
> not set, it throws a TProtocolException, passing along a toString(). This
> toString() gets the names and values of all fields, including username and
> password.
> Working solution:
> * Create a separate struct containing only the username and password, and
> pass it to OpenSession() as a second parameter. Since all fields in the new
> struct are "optional", the generated validate() is empty – toString() is
> never used. This involves changing core classes and breaks the "Each function
> should take exactly one parameter" coding convention (detailed at
> service-rpc/if/TCLIService.thrift:27).
> See working-solution.patch.
> What doesn't work:
> * Making client_protocol optional instead of required. Apparently this will
> break everything.
> * Overwriting toString() – TOpenSessionReq is a struct.
> * Creating two Thrift structs, one struct for required (TRequiredReq) and
> one for optional (TOptionalReq) fields, and nesting them in struct
> TOpenSessionReq. This doesn't work because validate() in TOpenSessionReq can
> call TOptionalReq.toString(), which prints the password to logs. This will
> happen if TRequiredReq.client_protocol isn't set.
> See non-solution.patch
> * Asking Thrift devs to change their code. I wrote them an email but have no
> expectations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20544) TOpenSessionReq logs password and username
[
https://issues.apache.org/jira/browse/HIVE-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-20544:
-
Status: Open (was: Patch Available)
> TOpenSessionReq logs password and username
> --
>
> Key: HIVE-20544
> URL: https://issues.apache.org/jira/browse/HIVE-20544
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: beginner, patch, security
> Attachments: HIVE-20544.1.patch, HIVE-20544.2.patch,
> HIVE-20544.3.patch, HIVE-20544.3.patch, HIVE-20544.4.patch,
> HIVE-20544.4.patch, HIVE-20544.4.patch, HIVE-20544.4.patch,
> HIVE-20544.4.patch, HIVE-20544.patch, non-solution.patch,
> working-solution.patch
>
>
> In
> service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TOpenSessionReq,
> if client protocol is unset, validate() and toString() prints both username
> and password to logs.
> Logging a password is a security risk. We should hide the ***.
> =Edit= (no longer relevant, see comments)
> This issue is tricky since it is caused in a fully generated class. I've been
> playing around and have found one working solution, butI'd truly appreciate
> ideas for a more elegant solution or input.
> The problem:
> TCLIService.thrift is the template for generating all classes in
> service-rpc. Struct TOpenSessionReq is OpenSession()'s one parameter and is
> defined thus:
> {noformat}
> struct TOpenSessionReq {
> 1: required TProtocolVersion client_protocol =
> TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10
> 2: optional string username
> 3: optional string password
> 4: optional map configuration
> }
> {noformat}
> In the generated class TOpenSessionReq.java, client_protocol is checked by a
> validate() method, which is called quite a few times; if client_protocol is
> not set, it throws a TProtocolException, passing along a toString(). This
> toString() gets the names and values of all fields, including username and
> password.
> Working solution:
> * Create a separate struct containing only the username and password, and
> pass it to OpenSession() as a second parameter. Since all fields in the new
> struct are "optional", the generated validate() is empty – toString() is
> never used. This involves changing core classes and breaks the "Each function
> should take exactly one parameter" coding convention (detailed at
> service-rpc/if/TCLIService.thrift:27).
> See working-solution.patch.
> What doesn't work:
> * Making client_protocol optional instead of required. Apparently this will
> break everything.
> * Overwriting toString() – TOpenSessionReq is a struct.
> * Creating two Thrift structs, one struct for required (TRequiredReq) and
> one for optional (TOptionalReq) fields, and nesting them in struct
> TOpenSessionReq. This doesn't work because validate() in TOpenSessionReq can
> call TOptionalReq.toString(), which prints the password to logs. This will
> happen if TRequiredReq.client_protocol isn't set.
> See non-solution.patch
> * Asking Thrift devs to change their code. I wrote them an email but have no
> expectations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20623) Shared work: Extend sharing of map-join cache entries in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633681#comment-16633681
]
Hive QA commented on HIVE-20623:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14151/dev-support/hive-personality.sh
|
| git revision | master / 4570807 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14151/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shared work: Extend sharing of map-join cache entries in LLAP
> -
>
> Key: HIVE-20623
> URL: https://issues.apache.org/jira/browse/HIVE-20623
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logical Optimizer
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20623.01.patch, HIVE-20623.02.patch,
> HIVE-20623.02.patch, HIVE-20623.02.patch, HIVE-20623.03.patch,
> HIVE-20623.03.patch, HIVE-20623.03.patch, HIVE-20623.patch,
> hash-shared-work.json.txt, hash-shared-work.svg
>
>
> For a query like this
> {code}
> with all_sales as (
> select ss_customer_sk as customer_sk, ss_ext_list_price-ss_ext_discount_amt
> as ext_price from store_sales
> UNION ALL
> select ws_bill_customer_sk as customer_sk,
> ws_ext_list_price-ws_ext_discount_amt as ext_price from web_sales
> UNION ALL
> select cs_bill_customer_sk as customer_sk, cs_ext_sales_price -
> cs_ext_discount_amt as ext_price from catalog_sales)
> select sum(ext_price) total_price, c_customer_id from all_sales, customer
> where customer_sk = c_customer_sk
> group by c_customer_id
[jira] [Commented] (HIVE-20545) Ability to exclude potentially large parameters in HMS Notifications
[ https://issues.apache.org/jira/browse/HIVE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633658#comment-16633658 ] Bharathkrishna Guruvayoor Murali commented on HIVE-20545: - Added new version of patch based on review comments. Updated the Jira description to be more informative. > Ability to exclude potentially large parameters in HMS Notifications > > > Key: HIVE-20545 > URL: https://issues.apache.org/jira/browse/HIVE-20545 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20545.1.patch, HIVE-20545.2.patch, > HIVE-20545.3.branch-3.patch, HIVE-20545.3.patch, HIVE-20545.4.patch, > HIVE-20545.6.patch > > > Clients can add large-sized parameters in Table/Partition objects. So we need > to enable adding regex patterns through HiveConf to match parameters to be > filtered from table and partition objects before serialization in HMS > notifications. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20545) Ability to exclude potentially large parameters in HMS Notifications
[ https://issues.apache.org/jira/browse/HIVE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20545: Attachment: HIVE-20545.6.patch > Ability to exclude potentially large parameters in HMS Notifications > > > Key: HIVE-20545 > URL: https://issues.apache.org/jira/browse/HIVE-20545 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20545.1.patch, HIVE-20545.2.patch, > HIVE-20545.3.branch-3.patch, HIVE-20545.3.patch, HIVE-20545.4.patch, > HIVE-20545.6.patch > > > Clients can add large-sized parameters in Table/Partition objects. So we need > to enable adding regex patterns through HiveConf to match parameters to be > filtered from table and partition objects before serialization in HMS > notifications. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20618) During join selection BucketMapJoin might be choosen for non bucketed tables
[ https://issues.apache.org/jira/browse/HIVE-20618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20618: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you Ashutosh and Deepak for reviewing the changes! > During join selection BucketMapJoin might be choosen for non bucketed tables > > > Key: HIVE-20618 > URL: https://issues.apache.org/jira/browse/HIVE-20618 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20618.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
