[jira] [Commented] (HIVE-23495) AcidUtils.getAcidState cleanup
[
https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110905#comment-17110905
]
Hive QA commented on HIVE-23495:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003318/HIVE-23495.1.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 17268 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_exim]
(batchId=124)
org.apache.hadoop.hive.ql.TestTxnCommands.testMmExim (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
(batchId=297)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.updateDeletePartitioned
(batchId=297)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=247)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteWithUnionAll
(batchId=247)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=271)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteWithUnionAll
(batchId=271)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMmExim
(batchId=277)
org.apache.hadoop.hive.ql.TestTxnExIm.testMM (batchId=278)
org.apache.hadoop.hive.ql.TestTxnExIm.testMMCreate (batchId=278)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testDumpAcidTableWithPartitionDirMissing
(batchId=202)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testDumpAcidTableWithTableDirMissing
(batchId=202)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testMultiDBTxn
(batchId=202)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testDumpAcidTableWithPartitionDirMissing
(batchId=204)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testDumpAcidTableWithTableDirMissing
(batchId=204)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testMultiDBTxn
(batchId=204)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testAcidTableIncrementalReplication
(batchId=205)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22451/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22451/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22451/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003318 - PreCommit-HIVE-Build
> AcidUtils.getAcidState cleanup
> --
>
> Key: HIVE-23495
> URL: https://issues.apache.org/jira/browse/HIVE-23495
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Varga
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23495.1.patch
>
>
> since HIVE-21225 there are two redundant implementation of the
> AcidUtils.getAcidState.
> The previous implementation (without the recursive listing) can be removed.
> Also the performance can be improved, by removing unnecessary fileStatus
> calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23495) AcidUtils.getAcidState cleanup
[
https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110897#comment-17110897
]
Hive QA commented on HIVE-23495:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
25s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} ql: The patch generated 0 new + 863 unchanged - 5
fixed = 863 total (was 868) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} hcatalog/streaming: The patch generated 0 new + 228
unchanged - 1 fixed = 228 total (was 229) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} streaming: The patch generated 0 new + 453 unchanged
- 3 fixed = 453 total (was 456) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
9s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100 total
(was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22451/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22451/yetus/whitespace-eol.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22451/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql hcatalog/streaming streaming U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22451/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> AcidUtils.getAcidState cleanup
> --
>
> Key: HIVE-23495
> URL: https://issues.apache.org/jira/browse/HIVE-23495
> Project: Hive
> Issue Type: Improvement
>
[jira] [Updated] (HIVE-22971) Eliminate file rename in insert-only compactor
[ https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-22971: - Attachment: HIVE-22971.06.patch Status: Patch Available (was: Open) > Eliminate file rename in insert-only compactor > -- > > Key: HIVE-22971 > URL: https://issues.apache.org/jira/browse/HIVE-22971 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: ACID, compaction > Attachments: HIVE-22971.01.patch, HIVE-22971.02.patch, > HIVE-22971.03.patch, HIVE-22971.04.patch, HIVE-22971.05.patch, > HIVE-22971.06.patch, HIVE-22971.06.patch > > > File rename is expensive for object stores, so MM (insert-only) compaction > should skip that step when committing and write directly to base_x_cZ or > delta_x_y_cZ. > This also fixes the issue that for MM QB compaction the temp tables were > stored under the table directory, and these temp dirs were never cleaned up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22971) Eliminate file rename in insert-only compactor
[ https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-22971: - Status: Open (was: Patch Available) > Eliminate file rename in insert-only compactor > -- > > Key: HIVE-22971 > URL: https://issues.apache.org/jira/browse/HIVE-22971 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: ACID, compaction > Attachments: HIVE-22971.01.patch, HIVE-22971.02.patch, > HIVE-22971.03.patch, HIVE-22971.04.patch, HIVE-22971.05.patch, > HIVE-22971.06.patch > > > File rename is expensive for object stores, so MM (insert-only) compaction > should skip that step when committing and write directly to base_x_cZ or > delta_x_y_cZ. > This also fixes the issue that for MM QB compaction the temp tables were > stored under the table directory, and these temp dirs were never cleaned up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110871#comment-17110871
]
Hive QA commented on HIVE-22066:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003316/HIVE-22066.10.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22450/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22450/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22450/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-19 05:47:03.907
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22450/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-19 05:47:03.910
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 61c9b2e HIVE-23354: Remove file size sanity checking from
compareTempOrDuplicateFiles (John Sherman, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 61c9b2e HIVE-23354: Remove file size sanity checking from
compareTempOrDuplicateFiles (John Sherman, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-19 05:47:05.348
+ rm -rf ../yetus_PreCommit-HIVE-Build-22450
+ mkdir ../yetus_PreCommit-HIVE-Build-22450
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22450
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22450/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
fatal: git diff header lacks filename information when removing 0 leading
pathname components (line 1631)
Trying to apply the patch with -p1
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.eot'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.eot'
without full index line
error:
llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.eot:
patch does not apply
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.ttf'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.ttf'
without full index line
error:
llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.ttf:
patch does not apply
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.woff'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.woff'
without full index line
error:
llap-server/src/main/resources/hive-webapps/llap/fonts/glyphicons-halflings-regular.woff:
patch does not apply
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/images/hive_logo.jpeg'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'llap-server/src/main/resources/hive-webapps/llap/images/hive_logo.jpeg'
without full index line
error: lla
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[
https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110870#comment-17110870
]
Hive QA commented on HIVE-22675:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003314/HIVE-22675.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17269 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testExternalTablesReplLoadBootstrapIncr
(batchId=208)
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1
(batchId=215)
org.apache.hive.jdbc.TestXSRFFilter.testFilterEnabledWithInjection[0:
tranportMode=all] (batchId=215)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22449/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22449/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22449/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003314 - PreCommit-HIVE-Build
> Replace Base64 in hive-standalone-metastore Package
> ---
>
> Key: HIVE-22675
> URL: https://issues.apache.org/jira/browse/HIVE-22675
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[
https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110854#comment-17110854
]
Hive QA commented on HIVE-22675:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 31 unchanged - 1 fixed = 32 total (was 32) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22449/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22449/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22449/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22449/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22449/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replace Base64 in hive-standalone-metastore Package
> ---
>
> Key: HIVE-22675
> URL: https://issues.apache.org/jira/browse/HIVE-22675
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23491) Move ParseDriver to parser module
[ https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23491: -- Status: Patch Available (was: Open) > Move ParseDriver to parser module > - > > Key: HIVE-23491 > URL: https://issues.apache.org/jira/browse/HIVE-23491 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23491.2.patch, HIVE-23491.3.patch > > > Move *ParseDriver* class and syntax Parsing related unit tests to the parse > module -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23491) Move ParseDriver to parser module
[ https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23491: -- Attachment: HIVE-23491.3.patch > Move ParseDriver to parser module > - > > Key: HIVE-23491 > URL: https://issues.apache.org/jira/browse/HIVE-23491 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23491.2.patch, HIVE-23491.3.patch > > > Move *ParseDriver* class and syntax Parsing related unit tests to the parse > module -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23491) Move ParseDriver to parser module
[ https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23491: -- Status: Open (was: Patch Available) > Move ParseDriver to parser module > - > > Key: HIVE-23491 > URL: https://issues.apache.org/jira/browse/HIVE-23491 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23491.2.patch, HIVE-23491.3.patch > > > Move *ParseDriver* class and syntax Parsing related unit tests to the parse > module -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23353) Atlas metadata replication scheduling
[ https://issues.apache.org/jira/browse/HIVE-23353?focusedWorklogId=434788&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434788 ] ASF GitHub Bot logged work on HIVE-23353: - Author: ASF GitHub Bot Created on: 19/May/20 04:39 Start Date: 19/May/20 04:39 Worklog Time Spent: 10m Work Description: pkumarsinha commented on a change in pull request #1021: URL: https://github.com/apache/hive/pull/1021#discussion_r427025689 ## File path: pom.xml ## @@ -127,6 +127,7 @@ 1.5.7 0.10.0 +2.0.0 Review comment: yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 434788) Time Spent: 2h 20m (was: 2h 10m) > Atlas metadata replication scheduling > - > > Key: HIVE-23353 > URL: https://issues.apache.org/jira/browse/HIVE-23353 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23353.01.patch, HIVE-23353.02.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110845#comment-17110845
]
Hive QA commented on HIVE-23017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003313/HIVE-23017.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17269 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=247)
org.apache.hive.beeline.TestBeeLineWithArgs.testEmbeddedBeelineOutputs
(batchId=211)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithExecutionMode
(batchId=214)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithPerformanceMode
(batchId=214)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22448/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22448/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22448/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003313 - PreCommit-HIVE-Build
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch, HIVE-23017.2.patch,
> HIVE-23017.3.patch, HIVE-23017.4.patch, HIVE-23017.5.patch,
> HIVE-23017.6.patch, HIVE-23017.6.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23353) Atlas metadata replication scheduling
[ https://issues.apache.org/jira/browse/HIVE-23353?focusedWorklogId=434786&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434786 ] ASF GitHub Bot logged work on HIVE-23353: - Author: ASF GitHub Bot Created on: 19/May/20 04:30 Start Date: 19/May/20 04:30 Worklog Time Spent: 10m Work Description: aasha commented on a change in pull request #1021: URL: https://github.com/apache/hive/pull/1021#discussion_r427023450 ## File path: pom.xml ## @@ -127,6 +127,7 @@ 1.5.7 0.10.0 +2.0.0 Review comment: Is this the latest version? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 434786) Time Spent: 2h 10m (was: 2h) > Atlas metadata replication scheduling > - > > Key: HIVE-23353 > URL: https://issues.apache.org/jira/browse/HIVE-23353 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23353.01.patch, HIVE-23353.02.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[ https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23433: --- Status: In Progress (was: Patch Available) > Add Deny Policy on Target Database After Ranger Replication to avoid writes > --- > > Key: HIVE-23433 > URL: https://issues.apache.org/jira/browse/HIVE-23433 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch, > HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch, > HIVE-23433.06.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[ https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23433: --- Attachment: HIVE-23433.06.patch Status: Patch Available (was: In Progress) > Add Deny Policy on Target Database After Ranger Replication to avoid writes > --- > > Key: HIVE-23433 > URL: https://issues.apache.org/jira/browse/HIVE-23433 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch, > HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch, > HIVE-23433.06.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110832#comment-17110832
]
Hive QA commented on HIVE-23017:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
21s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
58s{color} | {color:red} ql: The patch generated 7 new + 258 unchanged - 19
fixed = 265 total (was 277) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} ql generated 0 new + 1521 unchanged - 6 fixed = 1521
total (was 1527) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22448/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22448/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22448/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22448/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch, HIVE-23017.2.patch,
> HIVE-23017.3.patch, HIVE-23017.4.patch, HIVE-23017.5.patch,
> HIVE-23017.6.patch, HIVE-23017.6.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22971) Eliminate file rename in insert-only compactor
[
https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110817#comment-17110817
]
Hive QA commented on HIVE-22971:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003310/HIVE-22971.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17273 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=77)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=216)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22447/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22447/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22447/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003310 - PreCommit-HIVE-Build
> Eliminate file rename in insert-only compactor
> --
>
> Key: HIVE-22971
> URL: https://issues.apache.org/jira/browse/HIVE-22971
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: ACID, compaction
> Attachments: HIVE-22971.01.patch, HIVE-22971.02.patch,
> HIVE-22971.03.patch, HIVE-22971.04.patch, HIVE-22971.05.patch,
> HIVE-22971.06.patch
>
>
> File rename is expensive for object stores, so MM (insert-only) compaction
> should skip that step when committing and write directly to base_x_cZ or
> delta_x_y_cZ.
> This also fixes the issue that for MM QB compaction the temp tables were
> stored under the table directory, and these temp dirs were never cleaned up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22971) Eliminate file rename in insert-only compactor
[
https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110806#comment-17110806
]
Hive QA commented on HIVE-22971:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 134
unchanged - 5 fixed = 134 total (was 139) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22447/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22447/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Eliminate file rename in insert-only compactor
> --
>
> Key: HIVE-22971
> URL: https://issues.apache.org/jira/browse/HIVE-22971
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: ACID, compaction
> Attachments: HIVE-22971.01.patch, HIVE-22971.02.patch,
> HIVE-22971.03.patch, HIVE-22971.04.patch, HIVE-22971.05.patch,
> HIVE-22971.06.patch
>
>
> File rename is expensive for object stores, so MM (insert-only) compaction
> should skip that step when commit
[jira] [Commented] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110764#comment-17110764
]
Hive QA commented on HIVE-23281:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} metastore-server in master failed. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 5 new + 586 unchanged - 1 fixed = 591 total (was 587) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22446/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22446/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22446/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22446/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22446/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to
> DB for ACID tables
> --
>
> Key: HIVE-23281
> URL: https://issues.apache.org/jira/browse/HIVE-23281
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch,
> HIVE-23281.3.patch, image-2020-04-23-13-56-17-210.png
>
>
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/
[jira] [Commented] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[
https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110750#comment-17110750
]
Hive QA commented on HIVE-23433:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003301/HIVE-23433.05.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17271 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=228)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22445/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22445/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22445/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003301 - PreCommit-HIVE-Build
> Add Deny Policy on Target Database After Ranger Replication to avoid writes
> ---
>
> Key: HIVE-23433
> URL: https://issues.apache.org/jira/browse/HIVE-23433
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch,
> HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-23498: -- Summary: Disable HTTP Trace method on ThriftHttpCliService (was: Disable HTTP Trace method on ThriftServer) > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-23498: -- Status: Open (was: Patch Available) > Disable HTTP Trace method on ThriftServer > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-23498: -- Attachment: HIVE-23498.patch Status: Patch Available (was: Open) > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-23498: -- Status: Patch Available (was: Open) > Disable HTTP Trace method on ThriftServer > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23498) Disable HTTP Trace method on ThriftServer
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh reassigned HIVE-23498: - > Disable HTTP Trace method on ThriftServer > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[
https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110731#comment-17110731
]
Hive QA commented on HIVE-23433:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22445/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22445/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add Deny Policy on Target Database After Ranger Replication to avoid writes
> ---
>
> Key: HIVE-23433
> URL: https://issues.apache.org/jira/browse/HIVE-23433
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch,
> HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[
https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110723#comment-17110723
]
Hive QA commented on HIVE-23492:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003309/HIVE-23492.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17269 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.io.TestAcidUtils.testBaseDeltas (batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testBaseWithDeleteDeltas
(batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testObsoleteOriginals (batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDelta (batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDelta2 (batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testOverlapingDeltaAndDeleteDelta
(batchId=260)
org.apache.hadoop.hive.ql.io.TestAcidUtils.testRecursiveDirListingIsNotReusedWhenSnapshotFalse
(batchId=260)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22444/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22444/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22444/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003309 - PreCommit-HIVE-Build
> Remove unnecessary FileSystem#exists calls from ql module
> -
>
> Key: HIVE-23492
> URL: https://issues.apache.org/jira/browse/HIVE-23492
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23492.01.patch, HIVE-23492.02.patch,
> HIVE-23492.03.patch
>
>
> Wherever there is an exists() call before open() or delete(), remove it and
> infer from the FileNotFoundException raised in open/delete that the file does
> not exist. Exists() just checks for a FileNotFoundException so it's a waste
> of time, especially on clunkier FSes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23497) Change RexNodeExprFactory so it doesn't instantiate a factory every call
[ https://issues.apache.org/jira/browse/HIVE-23497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin reassigned HIVE-23497: --- Assignee: Steve Carlin > Change RexNodeExprFactory so it doesn't instantiate a factory every call > > > Key: HIVE-23497 > URL: https://issues.apache.org/jira/browse/HIVE-23497 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Minor > > Minor improvement: Change code in RexNodeTypeCheck to pass in the > RexNodeExprFactory rather than create it within the class. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[
https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110712#comment-17110712
]
Hive QA commented on HIVE-23492:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
7s{color} | {color:red} ql: The patch generated 1 new + 510 unchanged - 0 fixed
= 511 total (was 510) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22444/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22444/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22444/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unnecessary FileSystem#exists calls from ql module
> -
>
> Key: HIVE-23492
> URL: https://issues.apache.org/jira/browse/HIVE-23492
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23492.01.patch, HIVE-23492.02.patch,
> HIVE-23492.03.patch
>
>
> Wherever there is an exists() call before open() or delete(), remove it and
> infer from the FileNotFoundException raised in open/delete that the file does
> not exist. Exists() just checks for a FileNotFoundException so it's a waste
> of time, especially on clunkier FSes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[ https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-23466: - Attachment: HIVE-23466.1.patch > ZK registry base should remove only specific instance instead of host > - > > Key: HIVE-23466 > URL: https://issues.apache.org/jira/browse/HIVE-23466 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-23466.1.patch > > > When ZKRegistryBase detects new ZK nodes it maintains path based cache and > host based cache. The host based cached already handles multiple instances > running in same host. But even if single instance is removed all instances > belonging to the host are removed. > Another issue is that, if single host has multiple instances it returns a Set > with no ordering. Ideally, we want the newest instance to be top of the set > (use TreeSet maybe?). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[ https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110708#comment-17110708 ] Prasanth Jayachandran commented on HIVE-23466: -- This patch requires TEZ-4179 and new tez release to make use of ExtendedNodeId API. > ZK registry base should remove only specific instance instead of host > - > > Key: HIVE-23466 > URL: https://issues.apache.org/jira/browse/HIVE-23466 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-23466.1.patch > > > When ZKRegistryBase detects new ZK nodes it maintains path based cache and > host based cache. The host based cached already handles multiple instances > running in same host. But even if single instance is removed all instances > belonging to the host are removed. > Another issue is that, if single host has multiple instances it returns a Set > with no ordering. Ideally, we want the newest instance to be top of the set > (use TreeSet maybe?). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23353) Atlas metadata replication scheduling
[ https://issues.apache.org/jira/browse/HIVE-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA updated HIVE-23353: -- Attachment: HIVE-23353.02.patch > Atlas metadata replication scheduling > - > > Key: HIVE-23353 > URL: https://issues.apache.org/jira/browse/HIVE-23353 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23353.01.patch, HIVE-23353.02.patch > > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110689#comment-17110689
]
Hive QA commented on HIVE-6998:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003296/HIVE-6998.04.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 17270 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorLazyBinaryDeserializeRow
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorLazyBinarySerializeRow
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClipped
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClippedExact
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsExact
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsClipped
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsExact
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClipped1
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClipped2
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact1
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact2
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact3
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact1
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact2
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact3
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClipped
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClippedExact
(batchId=290)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsExact
(batchId=290)
org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinaryFast.testLazyBinaryFastComplexDepthFour
(batchId=309)
org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinaryFast.testLazyBinaryFastComplexDepthOne
(batchId=309)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22443/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22443/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22443/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003296 - PreCommit-HIVE-Build
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch,
> HIVE-6998.03.patch, HIVE-6998.04.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise,
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1), count(distinct c2), count(distinct c3),
> count(distinct c4), count(distinct c5), count(distinct c6), ,
> count(distinct c128), count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while
> closing operators
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
> operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.ja
[jira] [Comment Edited] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110385#comment-17110385
]
Panagiotis Garefalakis edited comment on HIVE-23480 at 5/18/20, 10:24 PM:
--
Hey [~contextshuffling] thanks for reporting this!
Instead of using an alternative library, I believe we could just use the
_JsonPropertyOrder_ annotation to ensure the ordering of the serialized
properties.
Creating a PR --please let me know what you think.
was (Author: pgaref):
Hey [~contextshuffling] thanks for reporting this!
Instead of using an alternative library, I believe we could also use the
_JsonPropertyOrder_ annotation to ensure the ordering of the serialized
properties.
Creating a PR --please let me know what you think.
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110657#comment-17110657
]
Hive QA commented on HIVE-6998:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} itests/custom-serde in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} serde: The patch generated 21 new + 2141 unchanged -
35 fixed = 2162 total (was 2176) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} ql: The patch generated 11 new + 2539 unchanged - 11
fixed = 2550 total (was 2550) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22443/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22443/yetus/diff-checkstyle-serde.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22443/yetus/diff-checkstyle-ql.txt
|
| modules | C: serde ql itests itests/custom-serde U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22443/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch,
> HIVE-6998.03.patch, HIV
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23214: -- Attachment: HIVE-23214.02.patch > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch, HIVE-23214.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23214: -- Attachment: TEZ-4183.02.patch > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch, HIVE-23214.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23214: -- Attachment: (was: TEZ-4183.02.patch) > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch, HIVE-23214.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[
https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110635#comment-17110635
]
Hive QA commented on HIVE-23214:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003293/HIVE-23214.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17266 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.server.TestHS2HttpServerPam.org.apache.hive.service.server.TestHS2HttpServerPam
(batchId=171)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22442/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22442/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22442/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003293 - PreCommit-HIVE-Build
> Remove skipCorrupt from OrcEncodedDataConsumer
> --
>
> Key: HIVE-23214
> URL: https://issues.apache.org/jira/browse/HIVE-23214
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23214.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SkipCorrupt is always the default (false) so there is no reason to pass it
> around.
> [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86]
> If we want to change the default behaviour we could set
> "orc.skip.corrupt.data" as part of the configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[
https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110603#comment-17110603
]
Hive QA commented on HIVE-23214:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
54s{color} | {color:blue} llap-server in master has 88 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} llap-server: The patch generated 1 new + 158 unchanged
- 6 fixed = 159 total (was 164) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22442/dev-support/hive-personality.sh
|
| git revision | master / 61c9b2e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22442/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22442/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove skipCorrupt from OrcEncodedDataConsumer
> --
>
> Key: HIVE-23214
> URL: https://issues.apache.org/jira/browse/HIVE-23214
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23214.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SkipCorrupt is always the default (false) so there is no reason to pass it
> around.
> [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86]
> If we want to change the default behaviour we could set
> "orc.skip.corrupt.data" as part of the configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110589#comment-17110589
]
Hive QA commented on HIVE-23480:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003283/HIVE-23480.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17269 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22441/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22441/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22441/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003283 - PreCommit-HIVE-Build
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?focusedWorklogId=434599&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434599 ] ASF GitHub Bot logged work on HIVE-23354: - Author: ASF GitHub Bot Created on: 18/May/20 20:20 Start Date: 18/May/20 20:20 Worklog Time Spent: 10m Work Description: asfgit closed pull request #1022: URL: https://github.com/apache/hive/pull/1022 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 434599) Time Spent: 20m (was: 10m) > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch, HIVE-23354.4.patch, HIVE-23354.5.patch, > HIVE-23354.6.patch, HIVE-23354.7.patch > > Time Spent: 20m > Remaining Estimate: 0h > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23354: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks for your contribution [~jfs]! > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch, HIVE-23354.4.patch, HIVE-23354.5.patch, > HIVE-23354.6.patch, HIVE-23354.7.patch > > Time Spent: 10m > Remaining Estimate: 0h > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110582#comment-17110582 ] Jesus Camacho Rodriguez commented on HIVE-23354: +1 > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch, HIVE-23354.4.patch, HIVE-23354.5.patch, > HIVE-23354.6.patch, HIVE-23354.7.patch > > Time Spent: 10m > Remaining Estimate: 0h > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23495) AcidUtils.getAcidState cleanup
[ https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23495: --- Attachment: HIVE-23495.1.patch > AcidUtils.getAcidState cleanup > -- > > Key: HIVE-23495 > URL: https://issues.apache.org/jira/browse/HIVE-23495 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > Attachments: HIVE-23495.1.patch > > > since HIVE-21225 there are two redundant implementation of the > AcidUtils.getAcidState. > The previous implementation (without the recursive listing) can be removed. > Also the performance can be improved, by removing unnecessary fileStatus > calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23495) AcidUtils.getAcidState cleanup
[ https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23495: --- Status: Patch Available (was: In Progress) > AcidUtils.getAcidState cleanup > -- > > Key: HIVE-23495 > URL: https://issues.apache.org/jira/browse/HIVE-23495 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > Attachments: HIVE-23495.1.patch > > > since HIVE-21225 there are two redundant implementation of the > AcidUtils.getAcidState. > The previous implementation (without the recursive listing) can be removed. > Also the performance can be improved, by removing unnecessary fileStatus > calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-23495) AcidUtils.getAcidState cleanup
[ https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-23495 started by Peter Varga. -- > AcidUtils.getAcidState cleanup > -- > > Key: HIVE-23495 > URL: https://issues.apache.org/jira/browse/HIVE-23495 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > > since HIVE-21225 there are two redundant implementation of the > AcidUtils.getAcidState. > The previous implementation (without the recursive listing) can be removed. > Also the performance can be improved, by removing unnecessary fileStatus > calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23495) AcidUtils.getAcidState cleanup
[ https://issues.apache.org/jira/browse/HIVE-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga reassigned HIVE-23495: -- > AcidUtils.getAcidState cleanup > -- > > Key: HIVE-23495 > URL: https://issues.apache.org/jira/browse/HIVE-23495 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > > since HIVE-21225 there are two redundant implementation of the > AcidUtils.getAcidState. > The previous implementation (without the recursive listing) can be removed. > Also the performance can be improved, by removing unnecessary fileStatus > calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110562#comment-17110562
]
Hive QA commented on HIVE-23480:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-common in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} metastore-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22441/dev-support/hive-personality.sh
|
| git revision | master / ece43ea |
| Default Java | 1.8.0_111 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22441/yetus/branch-findbugs-standalone-metastore_metastore-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22441/yetus/patch-findbugs-standalone-metastore_metastore-common.txt
|
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22441/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates
[jira] [Commented] (HIVE-23491) Move ParseDriver to parser module
[
https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110546#comment-17110546
]
Hive QA commented on HIVE-23491:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003284/HIVE-23491.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22440/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22440/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22440/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/Filter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/FilterChain.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/FilterConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/ServletException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims$WebHCatJTSh
[jira] [Commented] (HIVE-23475) Track MJ HashTable mem usage
[
https://issues.apache.org/jira/browse/HIVE-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110539#comment-17110539
]
Hive QA commented on HIVE-23475:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003277/HIVE-23475.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17269 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22439/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22439/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22439/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003277 - PreCommit-HIVE-Build
> Track MJ HashTable mem usage
>
>
> Key: HIVE-23475
> URL: https://issues.apache.org/jira/browse/HIVE-23475
> Project: Hive
> Issue Type: Improvement
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23475.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23494) Upgrade Apache parent POM to version 23
[ https://issues.apache.org/jira/browse/HIVE-23494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23494: - > Upgrade Apache parent POM to version 23 > --- > > Key: HIVE-23494 > URL: https://issues.apache.org/jira/browse/HIVE-23494 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Attachment: HIVE-22066.10.patch > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.10.patch, HIVE-22066.2.patch, > HIVE-22066.3.patch, HIVE-22066.4.patch, HIVE-22066.5.patch, > HIVE-22066.6.patch, HIVE-22066.6.patch, HIVE-22066.7.patch, > HIVE-22066.8.patch, HIVE-22066.8.patch, HIVE-22066.9.patch, > HIVE-22066.999.patch, HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23475) Track MJ HashTable mem usage
[
https://issues.apache.org/jira/browse/HIVE-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110515#comment-17110515
]
Hive QA commented on HIVE-23475:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
29s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22439/dev-support/hive-personality.sh
|
| git revision | master / ece43ea |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22439/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Track MJ HashTable mem usage
>
>
> Key: HIVE-23475
> URL: https://issues.apache.org/jira/browse/HIVE-23475
> Project: Hive
> Issue Type: Improvement
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23475.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23493) Rewrite plan to join back tables with many projected columns joined multiple times
[
https://issues.apache.org/jira/browse/HIVE-23493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-23493:
--
Description:
Queries with a pattern where one or more tables joins with a fact table in a
CTE. Many columns are projected out those tables and then grouped in the CTE.
The main query joins multiple instances of the CTE and may project a subset of
these.
The optimization is to rewrite the CTE to include only key (PK, non null Unique
Key) columns and join the tables back to the resultset of the main query to
fetch the rest of the wide columns. This reduces the datasize of the joined
back tables that is broadcast/shuffled throughout the DAG processing.
Example query, tpc-ds query4
{code}
with year_total as (
select c_customer_id customer_id
,c_first_name customer_first_name
,c_last_name customer_last_name
,c_preferred_cust_flag customer_preferred_cust_flag
,c_birth_country customer_birth_country
,c_login customer_login
,c_email_address customer_email_address
,d_year dyear
,sum(((ss_ext_list_price-ss_ext_wholesale_cost-ss_ext_discount_amt)+ss_ext_sales_price)/2)
year_total
,'s' sale_type
from customer
,store_sales
,date_dim
where c_customer_sk = ss_customer_sk
and ss_sold_date_sk = d_date_sk
group by c_customer_id
,c_first_name
,c_last_name
,c_preferred_cust_flag
,c_birth_country
,c_login
,c_email_address
,d_year
union all
select c_customer_id customer_id
,c_first_name customer_first_name
,c_last_name customer_last_name
,c_preferred_cust_flag customer_preferred_cust_flag
,c_birth_country customer_birth_country
,c_login customer_login
,c_email_address customer_email_address
,d_year dyear
,sumcs_ext_list_price-cs_ext_wholesale_cost-cs_ext_discount_amt)+cs_ext_sales_price)/2)
) year_total
,'c' sale_type
from customer
,catalog_sales
,date_dim
where c_customer_sk = cs_bill_customer_sk
and cs_sold_date_sk = d_date_sk
group by c_customer_id
,c_first_name
,c_last_name
,c_preferred_cust_flag
,c_birth_country
,c_login
,c_email_address
,d_year
union all
select c_customer_id customer_id
,c_first_name customer_first_name
,c_last_name customer_last_name
,c_preferred_cust_flag customer_preferred_cust_flag
,c_birth_country customer_birth_country
,c_login customer_login
,c_email_address customer_email_address
,d_year dyear
,sumws_ext_list_price-ws_ext_wholesale_cost-ws_ext_discount_amt)+ws_ext_sales_price)/2)
) year_total
,'w' sale_type
from customer
,web_sales
,date_dim
where c_customer_sk = ws_bill_customer_sk
and ws_sold_date_sk = d_date_sk
group by c_customer_id
,c_first_name
,c_last_name
,c_preferred_cust_flag
,c_birth_country
,c_login
,c_email_address
,d_year
)
select
t_s_secyear.customer_id
,t_s_secyear.customer_first_name
,t_s_secyear.customer_last_name
,t_s_secyear.customer_birth_country
from year_total t_s_firstyear
,year_total t_s_secyear
,year_total t_c_firstyear
,year_total t_c_secyear
,year_total t_w_firstyear
,year_total t_w_secyear
where t_s_secyear.customer_id = t_s_firstyear.customer_id
and t_s_firstyear.customer_id = t_c_secyear.customer_id
and t_s_firstyear.customer_id = t_c_firstyear.customer_id
and t_s_firstyear.customer_id = t_w_firstyear.customer_id
and t_s_firstyear.customer_id = t_w_secyear.customer_id
and t_s_firstyear.sale_type = 's'
and t_c_firstyear.sale_type = 'c'
and t_w_firstyear.sale_type = 'w'
and t_s_secyear.sale_type = 's'
and t_c_secyear.sale_type = 'c'
and t_w_secyear.sale_type = 'w'
and t_s_firstyear.dyear = 1999
and t_s_secyear.dyear = 1999+1
and t_c_firstyear.dyear = 1999
and t_c_secyear.dyear = 1999+1
and t_w_firstyear.dyear = 1999
and t_w_secyear.dyear = 1999+1
and t_s_firstyear.year_total > 0
and t_c_firstyear.year_total > 0
and t_w_firstyear.year_total > 0
and case when t_c_firstyear.year_total > 0 then t_c_secyear.year_total /
t_c_firstyear.year_total else null end
> case when t_s_firstyear.year_total > 0 then t_s_secyear.year_total
/ t_s_firstyear.year_total else null end
and case when t_c_firstyear.year_total > 0 then t_c_secyear.year_total /
t_c_firstyear.year_total else null end
> case when t_w_firstyear.year_total > 0 then t_w_secyear.year_total
/ t_w_firstyear.year_total else null end
order by t_s_secyear.customer_id
,t_s_secyear.customer_first_name
,t_s_secyear.customer_last_name
,t_s_secyear.customer_bir
[jira] [Assigned] (HIVE-23493) Rewrite plan to join back tables with many projected columns joined multiple times
[ https://issues.apache.org/jira/browse/HIVE-23493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa reassigned HIVE-23493: - > Rewrite plan to join back tables with many projected columns joined multiple > times > -- > > Key: HIVE-23493 > URL: https://issues.apache.org/jira/browse/HIVE-23493 > Project: Hive > Issue Type: New Feature > Components: CBO >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[ https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22675: -- Attachment: HIVE-22675.3.patch > Replace Base64 in hive-standalone-metastore Package > --- > > Key: HIVE-22675 > URL: https://issues.apache.org/jira/browse/HIVE-22675 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch, > HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23017:
--
Attachment: HIVE-23017.6.patch
> Use Inherited Logger in Tasks
> -
>
> Key: HIVE-23017
> URL: https://issues.apache.org/jira/browse/HIVE-23017
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23017.1.patch, HIVE-23017.2.patch,
> HIVE-23017.3.patch, HIVE-23017.4.patch, HIVE-23017.5.patch,
> HIVE-23017.6.patch, HIVE-23017.6.patch
>
>
> {code:java|title=FetchTask}
> public class FetchTask extends Task implements Serializable {
> private static final long serialVersionUID = 1L;
> private int maxRows = 100;
> private FetchOperator fetch;
> private ListSinkOperator sink;
> private int totalRows;
> private static transient final Logger LOG =
> LoggerFactory.getLogger(FetchTask.class);
> JobConf job = null;
> {code}
> The Tasks class is base class for many other. Cleaner to simply make the
> {{Logger}} {{protected}} and usable by other classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23473) Handle NPE when ObjectCache is null while getting DynamicValue during ORC split generation
[
https://issues.apache.org/jira/browse/HIVE-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110498#comment-17110498
]
Hive QA commented on HIVE-23473:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003275/HIVE-23473.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17265 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=187)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22438/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22438/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22438/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003275 - PreCommit-HIVE-Build
> Handle NPE when ObjectCache is null while getting DynamicValue during ORC
> split generation
> --
>
> Key: HIVE-23473
> URL: https://issues.apache.org/jira/browse/HIVE-23473
> Project: Hive
> Issue Type: Bug
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-23473.1.patch
>
>
> NullPointerException is thrown in the following flow.
>
> {code:java}
> java.lang.RuntimeException: ORC split generation failed with exception:
> java.lang.NullPointerException
> Caused by: java.lang.NullPointerException
> at
> org.apache.orc.impl.RecordReaderImpl.compareToRange(RecordReaderImpl.java:312)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:559)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2214)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2190)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2182)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:186)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1477)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1265)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1446)
> .
> .
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1809)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1895)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:526)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:649)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:206)
> {code}
>
> Shouldn't we just throw NoDynamicValuesException when [ObjectCache|#L119]] is
> null instead of returning it similar to how we handled when [conf |#L110]]or
> [DynamicValueRegistry|#L125]] is null while getting dynamic value?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22971) Eliminate file rename in insert-only compactor
[ https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-22971: - Attachment: HIVE-22971.06.patch > Eliminate file rename in insert-only compactor > -- > > Key: HIVE-22971 > URL: https://issues.apache.org/jira/browse/HIVE-22971 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: ACID, compaction > Attachments: HIVE-22971.01.patch, HIVE-22971.02.patch, > HIVE-22971.03.patch, HIVE-22971.04.patch, HIVE-22971.05.patch, > HIVE-22971.06.patch > > > File rename is expensive for object stores, so MM (insert-only) compaction > should skip that step when committing and write directly to base_x_cZ or > delta_x_y_cZ. > This also fixes the issue that for MM QB compaction the temp tables were > stored under the table directory, and these temp dirs were never cleaned up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[ https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23492: - Attachment: HIVE-23492.03.patch > Remove unnecessary FileSystem#exists calls from ql module > - > > Key: HIVE-23492 > URL: https://issues.apache.org/jira/browse/HIVE-23492 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23492.01.patch, HIVE-23492.02.patch, > HIVE-23492.03.patch > > > Wherever there is an exists() call before open() or delete(), remove it and > infer from the FileNotFoundException raised in open/delete that the file does > not exist. Exists() just checks for a FileNotFoundException so it's a waste > of time, especially on clunkier FSes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[ https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23492: - Attachment: HIVE-23492.02.patch > Remove unnecessary FileSystem#exists calls from ql module > - > > Key: HIVE-23492 > URL: https://issues.apache.org/jira/browse/HIVE-23492 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23492.01.patch, HIVE-23492.02.patch > > > Wherever there is an exists() call before open() or delete(), remove it and > infer from the FileNotFoundException raised in open/delete that the file does > not exist. Exists() just checks for a FileNotFoundException so it's a waste > of time, especially on clunkier FSes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[ https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23281: Attachment: (was: HIVE-23281.3.patch) > ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to > DB for ACID tables > -- > > Key: HIVE-23281 > URL: https://issues.apache.org/jira/browse/HIVE-23281 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch, > HIVE-23281.3.patch, image-2020-04-23-13-56-17-210.png > > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1980] > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1982] > > SkewInfo, bucketCols, ordering etc are not needed for ACID tables. It may be > good to check for transactional tables and get rid these calls in table > lookups. > > This should help in reducing DB calls. > !image-2020-04-23-13-56-17-210.png|width=669,height=485! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[ https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23281: Attachment: HIVE-23281.3.patch Status: Patch Available (was: Open) > ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to > DB for ACID tables > -- > > Key: HIVE-23281 > URL: https://issues.apache.org/jira/browse/HIVE-23281 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch, > HIVE-23281.3.patch, image-2020-04-23-13-56-17-210.png > > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1980] > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1982] > > SkewInfo, bucketCols, ordering etc are not needed for ACID tables. It may be > good to check for transactional tables and get rid these calls in table > lookups. > > This should help in reducing DB calls. > !image-2020-04-23-13-56-17-210.png|width=669,height=485! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23178) Add Tez Total Order Partitioner
[ https://issues.apache.org/jira/browse/HIVE-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110467#comment-17110467 ] Gopal Vijayaraghavan commented on HIVE-23178: - Pushed the whitespace fixes to master. > Add Tez Total Order Partitioner > --- > > Key: HIVE-23178 > URL: https://issues.apache.org/jira/browse/HIVE-23178 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 3.1.1, 3.1.2 >Reporter: Roohi Syeda >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: TODOC > Fix For: 4.0.0 > > Attachments: HIVE-23178.01.patch, HIVE-23178.addendum.patch, > HIVE-23178.addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23178) Add Tez Total Order Partitioner
[ https://issues.apache.org/jira/browse/HIVE-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23178: Resolution: Fixed Status: Resolved (was: Patch Available) > Add Tez Total Order Partitioner > --- > > Key: HIVE-23178 > URL: https://issues.apache.org/jira/browse/HIVE-23178 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 3.1.1, 3.1.2 >Reporter: Roohi Syeda >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: TODOC > Fix For: 4.0.0 > > Attachments: HIVE-23178.01.patch, HIVE-23178.addendum.patch, > HIVE-23178.addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23473) Handle NPE when ObjectCache is null while getting DynamicValue during ORC split generation
[
https://issues.apache.org/jira/browse/HIVE-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110466#comment-17110466
]
Hive QA commented on HIVE-23473:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
21s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22438/dev-support/hive-personality.sh
|
| git revision | master / c46bed6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22438/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Handle NPE when ObjectCache is null while getting DynamicValue during ORC
> split generation
> --
>
> Key: HIVE-23473
> URL: https://issues.apache.org/jira/browse/HIVE-23473
> Project: Hive
> Issue Type: Bug
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-23473.1.patch
>
>
> NullPointerException is thrown in the following flow.
>
> {code:java}
> java.lang.RuntimeException: ORC split generation failed with exception:
> java.lang.NullPointerException
> Caused by: java.lang.NullPointerException
> at
> org.apache.orc.impl.RecordReaderImpl.compareToRange(RecordReaderImpl.java:312)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:559)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463)
> at
> org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2214)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2190)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2182)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:186)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1477)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat
[jira] [Updated] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[ https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23281: Status: Open (was: Patch Available) > ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to > DB for ACID tables > -- > > Key: HIVE-23281 > URL: https://issues.apache.org/jira/browse/HIVE-23281 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch, > HIVE-23281.3.patch, image-2020-04-23-13-56-17-210.png > > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1980] > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1982] > > SkewInfo, bucketCols, ordering etc are not needed for ACID tables. It may be > good to check for transactional tables and get rid these calls in table > lookups. > > This should help in reducing DB calls. > !image-2020-04-23-13-56-17-210.png|width=669,height=485! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[ https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23281: Attachment: HIVE-23281.3.patch Status: Patch Available (was: Open) > ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to > DB for ACID tables > -- > > Key: HIVE-23281 > URL: https://issues.apache.org/jira/browse/HIVE-23281 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch, > HIVE-23281.3.patch, image-2020-04-23-13-56-17-210.png > > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1980] > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1982] > > SkewInfo, bucketCols, ordering etc are not needed for ACID tables. It may be > good to check for transactional tables and get rid these calls in table > lookups. > > This should help in reducing DB calls. > !image-2020-04-23-13-56-17-210.png|width=669,height=485! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[ https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23433: --- Status: In Progress (was: Patch Available) > Add Deny Policy on Target Database After Ranger Replication to avoid writes > --- > > Key: HIVE-23433 > URL: https://issues.apache.org/jira/browse/HIVE-23433 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch, > HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[ https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23433: --- Attachment: HIVE-23433.05.patch Status: Patch Available (was: In Progress) > Add Deny Policy on Target Database After Ranger Replication to avoid writes > --- > > Key: HIVE-23433 > URL: https://issues.apache.org/jira/browse/HIVE-23433 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch, > HIVE-23433.03.patch, HIVE-23433.04.patch, HIVE-23433.05.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[ https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-23281: Status: Open (was: Patch Available) > ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to > DB for ACID tables > -- > > Key: HIVE-23281 > URL: https://issues.apache.org/jira/browse/HIVE-23281 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch, > image-2020-04-23-13-56-17-210.png > > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1980] > > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1982] > > SkewInfo, bucketCols, ordering etc are not needed for ACID tables. It may be > good to check for transactional tables and get rid these calls in table > lookups. > > This should help in reducing DB calls. > !image-2020-04-23-13-56-17-210.png|width=669,height=485! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[ https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23492: - Attachment: HIVE-23492.01.patch Status: Patch Available (was: Open) > Remove unnecessary FileSystem#exists calls from ql module > - > > Key: HIVE-23492 > URL: https://issues.apache.org/jira/browse/HIVE-23492 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23492.01.patch > > > Wherever there is an exists() call before open() or delete(), remove it and > infer from the FileNotFoundException raised in open/delete that the file does > not exist. Exists() just checks for a FileNotFoundException so it's a waste > of time, especially on clunkier FSes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[ https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage reassigned HIVE-23492: > Remove unnecessary FileSystem#exists calls from ql module > - > > Key: HIVE-23492 > URL: https://issues.apache.org/jira/browse/HIVE-23492 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > > Wherever there is an exists() call before open() or delete(), remove it and > infer from the FileNotFoundException raised in open/delete that the file does > not exist. Exists() just checks for a FileNotFoundException so it's a waste > of time, especially on clunkier FSes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[
https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110441#comment-17110441
]
Hive QA commented on HIVE-23433:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003274/HIVE-23433.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17271 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22437/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22437/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22437/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003274 - PreCommit-HIVE-Build
> Add Deny Policy on Target Database After Ranger Replication to avoid writes
> ---
>
> Key: HIVE-23433
> URL: https://issues.apache.org/jira/browse/HIVE-23433
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch,
> HIVE-23433.03.patch, HIVE-23433.04.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-6998:
--
Attachment: HIVE-6998.04.patch
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch,
> HIVE-6998.03.patch, HIVE-6998.04.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise,
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1), count(distinct c2), count(distinct c3),
> count(distinct c4), count(distinct c5), count(distinct c6), ,
> count(distinct c128), count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while
> closing operators
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
> operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
> ... 10 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1099)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1138)
> ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:327)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1082)
> ... 16 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -128
> at java.util.ArrayList.get(ArrayList.java:324)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:838)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:600)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.toHiveKey(ReduceSinkOperator.java:401)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:320)
> ... 19 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-6998:
-
Assignee: Jesus Camacho Rodriguez (was: Eugene Koifman)
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch,
> HIVE-6998.03.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise,
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1), count(distinct c2), count(distinct c3),
> count(distinct c4), count(distinct c5), count(distinct c6), ,
> count(distinct c128), count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while
> closing operators
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
> operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
> ... 10 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1099)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1138)
> ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:327)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1082)
> ... 16 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -128
> at java.util.ArrayList.get(ArrayList.java:324)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:838)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:600)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.toHiveKey(ReduceSinkOperator.java:401)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:320)
> ... 19 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23433) Add Deny Policy on Target Database After Ranger Replication to avoid writes
[
https://issues.apache.org/jira/browse/HIVE-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110416#comment-17110416
]
Hive QA commented on HIVE-23433:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} ql: The patch generated 20 new + 0 unchanged - 0 fixed
= 20 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22437/dev-support/hive-personality.sh
|
| git revision | master / c46bed6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22437/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22437/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add Deny Policy on Target Database After Ranger Replication to avoid writes
> ---
>
> Key: HIVE-23433
> URL: https://issues.apache.org/jira/browse/HIVE-23433
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23433.01.patch, HIVE-23433.02.patch,
> HIVE-23433.03.patch, HIVE-23433.04.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-23426) Show create table for Views giving incomplete view definition
[ https://issues.apache.org/jira/browse/HIVE-23426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely resolved HIVE-23426. --- Resolution: Cannot Reproduce [~sureshkumarscb] If you can send me some creation script that leads to this issue, I'll take a look, but neither can I reproduce it, nor do I see any reason for such issue to happen. > Show create table for Views giving incomplete view definition > - > > Key: HIVE-23426 > URL: https://issues.apache.org/jira/browse/HIVE-23426 > Project: Hive > Issue Type: Bug >Reporter: Sureshkumar >Assignee: Miklos Gergely >Priority: Major > > Show create table for View giving incomplete view definition -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23484) Ensure result order for special_character_in_tabnames_1.q
[ https://issues.apache.org/jira/browse/HIVE-23484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23484: -- Resolution: Duplicate Status: Resolved (was: Patch Available) > Ensure result order for special_character_in_tabnames_1.q > - > > Key: HIVE-23484 > URL: https://issues.apache.org/jira/browse/HIVE-23484 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23484.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23214: -- Status: Patch Available (was: In Progress) > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23214: -- Attachment: HIVE-23214.01.patch > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-23214 started by Panagiotis Garefalakis. - > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23214.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23214: -- Labels: pull-request-available (was: ) > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23214) Remove skipCorrupt from OrcEncodedDataConsumer
[ https://issues.apache.org/jira/browse/HIVE-23214?focusedWorklogId=434459&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434459 ] ASF GitHub Bot logged work on HIVE-23214: - Author: ASF GitHub Bot Created on: 18/May/20 15:52 Start Date: 18/May/20 15:52 Worklog Time Spent: 10m Work Description: pgaref opened a new pull request #1025: URL: https://github.com/apache/hive/pull/1025 Change-Id: Ic1efd6dcffc71adfa1ac3059ceacbd3f30e6ef7e This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 434459) Remaining Estimate: 0h Time Spent: 10m > Remove skipCorrupt from OrcEncodedDataConsumer > -- > > Key: HIVE-23214 > URL: https://issues.apache.org/jira/browse/HIVE-23214 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > SkipCorrupt is always the default (false) so there is no reason to pass it > around. > [https://github.com/apache/hive/blob/3e4f6122c32b1ffa22e1458806ae8ee30e51a41f/llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java#L86] > If we want to change the default behaviour we could set > "orc.skip.corrupt.data" as part of the configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23456) Upgrade Calcite version to 1.23.0
[
https://issues.apache.org/jira/browse/HIVE-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110391#comment-17110391
]
Hive QA commented on HIVE-23456:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
46s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
0s{color} | {color:red} ql: The patch generated 3 new + 263 unchanged - 3 fixed
= 266 total (was 266) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
12s{color} | {color:red} root: The patch generated 3 new + 263 unchanged - 3
fixed = 266 total (was 266) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 6m
23s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 2m
4s{color} | {color:red} ql generated 4 new + 96 unchanged - 4 fixed = 100 total
(was 100) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 14m
3s{color} | {color:red} root generated 4 new + 334 unchanged - 4 fixed = 338
total (was 338) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22436/dev-support/hive-personality.sh
|
| git revision | master / c46bed6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus/diff-checkstyle-root.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus/patch-findbugs-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus/diff-javadoc-javadoc-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus/diff-javadoc-javadoc-root.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22436/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade Calcite version to 1.23.0
> -
>
> Key: HIVE-23456
> URL: https://issues.apache.org/jira/browse/HIVE-23456
> Project: Hive
> Issue Type: Task
>
[jira] [Updated] (HIVE-23491) Move ParseDriver to parser module
[ https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23491: -- Attachment: HIVE-23491.2.patch > Move ParseDriver to parser module > - > > Key: HIVE-23491 > URL: https://issues.apache.org/jira/browse/HIVE-23491 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23491.2.patch > > > Move *ParseDriver* class and syntax Parsing related unit tests to the parse > module -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23491) Move ParseDriver to parser module
[ https://issues.apache.org/jira/browse/HIVE-23491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23491: -- Status: Patch Available (was: Open) > Move ParseDriver to parser module > - > > Key: HIVE-23491 > URL: https://issues.apache.org/jira/browse/HIVE-23491 > Project: Hive > Issue Type: Improvement > Components: Parser >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23491.2.patch > > > Move *ParseDriver* class and syntax Parsing related unit tests to the parse > module -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23480:
--
Attachment: HIVE-23480.01.patch
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-23480:
--
Labels: pull-request-available (was: )
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23480:
--
Labels: (was: pull-request-available)
Status: Patch Available (was: In Progress)
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?focusedWorklogId=434448&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434448
]
ASF GitHub Bot logged work on HIVE-23480:
-
Author: ASF GitHub Bot
Created on: 18/May/20 15:33
Start Date: 18/May/20 15:33
Worklog Time Spent: 10m
Work Description: pgaref opened a new pull request #1024:
URL: https://github.com/apache/hive/pull/1024
use the JsonPropertyOrder annotation to ensure the ordering of serialized
properties.
Change-Id: I2f2b3f1d9eec1e26b5b6e445efe6f0106f4ea15d
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 434448)
Remaining Estimate: 0h
Time Spent: 10m
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Attachments: HIVE-23480.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-23480 started by Panagiotis Garefalakis.
-
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110385#comment-17110385
]
Panagiotis Garefalakis commented on HIVE-23480:
---
Hey [~contextshuffling] thanks for reporting this!
Instead of using an alternative library, I believe we could also use the
_JsonPropertyOrder_ annotation to ensure the ordering of the serialized
properties.
Creating a PR --please let me know what you think.
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23480) Test may fail due to a incorrect usage of a third party library
[
https://issues.apache.org/jira/browse/HIVE-23480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis reassigned HIVE-23480:
-
Assignee: Panagiotis Garefalakis
> Test may fail due to a incorrect usage of a third party library
> ---
>
> Key: HIVE-23480
> URL: https://issues.apache.org/jira/browse/HIVE-23480
> Project: Hive
> Issue Type: Bug
> Components: HBase Metastore
>Reporter: contextshuffling
>Assignee: Panagiotis Garefalakis
>Priority: Minor
>
> Tests
> {{org.apache.hadoop.hive.common.TestStatsSetupConst#testStatColumnEntriesCompat}}
> replies on Jackson to serialize the params to string. However, Jackson
> library uses reflection API {{getDeclaredFields}} but it does not guarantee
> any specific order of returned field so the order of fields in the json
> string might change, and thus, test can fail or pass without any changes to
> the code.
> An example error message:
> {code:java}
> org.junit.ComparisonFailure:
> expected:<{"[BASIC_STATS":"true","COLUMN_STATS":{"Foo":"true"}]}> but
> was:<{"[COLUMN_STATS":{"Foo":"true"},"BASIC_STATS":"true"]}>
> at
> org.apache.hadoop.hive.common.TestStatsSetupConst.testStatColumnEntriesCompat(TestStatsSetupConst.java:76)
> {code}
> Ideally, this test should not reply on the order returned by this API so that
> it generates a deterministic result.
> An potential solution is to use library like
> https://github.com/skyscreamer/JSONassert to compare string in a
> order-agnostic way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23456) Upgrade Calcite version to 1.23.0
[
https://issues.apache.org/jira/browse/HIVE-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110381#comment-17110381
]
Hive QA commented on HIVE-23456:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003273/HIVE-23456.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 669 failed/errored test(s), 17269 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=225)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=233)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=239)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic_standard]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=4)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_custom_key2]
(batchId=13)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_custom_key3]
(batchId=12)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_ppd_key_range]
(batchId=13)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_materialized_view_rewrite_ssb]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timeseries]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_topn]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_semijoin_reduction_all_types]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=228)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=228)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_csv]
(batchId=228)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=228)
org.apache.hadoop.hive.cli.TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]
(batchId=237)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=22)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct]
(batchId=26)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[merge_test_dummy_operator]
(batchId=25)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=25)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=113)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_view_delete]
(batchId=64)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[alter_partition_coltype]
(batchId=57)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ambiguitycheck]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=44)
org.apache.hadoop.hiv
[jira] [Work logged] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23434?focusedWorklogId=434427&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-434427
]
ASF GitHub Bot logged work on HIVE-23434:
-
Author: ASF GitHub Bot
Created on: 18/May/20 15:00
Start Date: 18/May/20 15:00
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #1017:
URL: https://github.com/apache/hive/pull/1017#discussion_r426690605
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRewriteToDataSketchesRule.java
##
@@ -0,0 +1,327 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Optional;
+
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.hep.HepRelVertex;
+import org.apache.calcite.rel.RelCollation;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.core.RelFactories.ProjectFactory;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlAggFunction;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.ImmutableBitSet;
+import org.apache.calcite.util.ImmutableBitSet.Builder;
+import org.apache.hadoop.hive.ql.exec.DataSketchesFunctions;
+import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveAggregate;
+import org.apache.hive.plugin.api.HiveUDFPlugin.UDFDescriptor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+
+/**
+ * This rule could rewrite aggregate calls to be calculated using sketch based
functions.
+ *
+ *
+ * Currently it can rewrite:
+ *
+ * {@code count(distinct(x))} to distinct counting sketches
+ * {@code percentile_cont(0.2) within group (order by id)}
+ *
+ *
+ *
+ * The transformation here works on Aggregate nodes; the operations done are
the following:
+ *
+ *
+ * Identify candidate aggregate calls
+ * A new Project is inserted below the Aggregate; to help with data
pre-processing
+ * A new Aggregate is created in which the aggregation is done by the
sketch function
+ * A new Project is inserted on top of the Aggregate; which unwraps the
resulting
+ *count-distinct estimation from the sketch representation
+ *
+ */
+public final class HiveRewriteToDataSketchesRule extends RelOptRule {
+
+ protected static final Logger LOG =
LoggerFactory.getLogger(HiveRewriteToDataSketchesRule.class);
+ private final Optional countDistinctSketchType;
+ private final Optional percentileContSketchType;
+ private final ProjectFactory projectFactory;
+
+ public HiveRewriteToDataSketchesRule(Optional
countDistinctSketchType,
+ Optional percentileContSketchType) {
+super(operand(HiveAggregate.class, any()));
+this.countDistinctSketchType = countDistinctSketchType;
+this.percentileContSketchType = percentileContSketchType;
+projectFactory = HiveRelFactories.HIVE_PROJECT_FACTORY;
+ }
+
+ @Override
+ public void onMatch(RelOptRuleCall call) {
+final Aggregate aggregate = call.rel(0);
+
+if (aggregate.getGroupSets().size() != 1) {
+ // not yet supported
+ return;
+}
+
+List newAggCalls = new ArrayList();
+
+VBuilder vb = new VBuilder(aggregate);
+
+if (aggregate.getAggCallList().equals(vb.newAggCalls)) {
+ // rule didn't made any changes
+ return;
+}
+
+newAggCalls = vb.newAggCalls;
+List filedNames=new ArrayList();
+for (int i=0;i newAggCalls;
+private List newProjects;
+
[jira] [Commented] (HIVE-23490) REPL LOAD config to bootstrap external tables with purge
[
https://issues.apache.org/jira/browse/HIVE-23490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110364#comment-17110364
]
David Lavati commented on HIVE-23490:
-
Based on HIVE-19981 this can probably be achieved with {{REPL LOAD db FROM dir
WITH ('HIVE_EXTERNALTABLE_PURGE_DEFAULT'='true')}}
Will post my findings here.
> REPL LOAD config to bootstrap external tables with purge
>
>
> Key: HIVE-23490
> URL: https://issues.apache.org/jira/browse/HIVE-23490
> Project: Hive
> Issue Type: New Feature
> Components: repl
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Fix For: 4.0.0
>
>
> When replicating data from e.g. an onprem HDFS cluster to cloud storage, a
> common scenario is to load it as external table. However proper cleanup is
> usually missing here, so when the user deletes a table on the source side,
> it's ideal to purge it on the target side.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23475) Track MJ HashTable mem usage
[ https://issues.apache.org/jira/browse/HIVE-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23475: -- Attachment: HIVE-23475.01.patch > Track MJ HashTable mem usage > > > Key: HIVE-23475 > URL: https://issues.apache.org/jira/browse/HIVE-23475 > Project: Hive > Issue Type: Improvement >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23475.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23475) Track MJ HashTable mem usage
[ https://issues.apache.org/jira/browse/HIVE-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23475: -- Status: Patch Available (was: In Progress) > Track MJ HashTable mem usage > > > Key: HIVE-23475 > URL: https://issues.apache.org/jira/browse/HIVE-23475 > Project: Hive > Issue Type: Improvement >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23475.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
