[
https://issues.apache.org/jira/browse/PHOENIX-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538584#comment-17538584
]
ASF GitHub Bot commented on PHOENIX-6698:
-----------------------------------------
stoty commented on PR #79:
URL:
https://github.com/apache/phoenix-connectors/pull/79#issuecomment-1129600452
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 25s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files
found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any
anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any
@author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include
1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 24s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| +1 :green_heart: | checkstyle | 0m 17s | master passed |
| +1 :green_heart: | javadoc | 0m 25s | master passed |
| +0 :ok: | spotbugs | 2m 12s | phoenix-hive-base in master has 46
extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 23m 10s | the patch passed |
| +1 :green_heart: | compile | 1m 11s | the patch passed |
| +1 :green_heart: | javac | 1m 11s | the patch passed |
| -1 :x: | checkstyle | 0m 17s | phoenix-hive-base: The patch generated
39 new + 52 unchanged - 32 fixed = 91 total (was 84) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace
issues. |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed |
| +1 :green_heart: | spotbugs | 2m 1s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 58m 55s | phoenix-hive-base in the patch failed. |
| -1 :x: | asflicense | 0m 12s | The patch generated 2 ASF License
warnings. |
| | | 135m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/phoenix-connectors/pull/79 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti
checkstyle compile |
| uname | Linux 5bb88adec8bd 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-connectors-personality.sh |
| git revision | master / 9ed127f |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-hive-base.txt
|
| unit |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/artifact/yetus-general-check/output/patch-unit-phoenix-hive-base.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/testReport/
|
| asflicense |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/artifact/yetus-general-check/output/patch-asflicense-problems.txt
|
| Max. process+thread count | 1691 (vs. ulimit of 30000) |
| modules | C: phoenix-hive-base U: phoenix-hive-base |
| Console output |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-79/4/console
|
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> hive-connector will take long time to generate splits for large phoenix
> tables.
> -------------------------------------------------------------------------------
>
> Key: PHOENIX-6698
> URL: https://issues.apache.org/jira/browse/PHOENIX-6698
> Project: Phoenix
> Issue Type: Improvement
> Components: hive-connector
> Affects Versions: 5.1.0
> Reporter: jichen
> Assignee: jichen
> Priority: Minor
> Fix For: connectors-6.0.0
>
> Attachments: PHOENIX-6698.master.v1.patch
>
>
> {{{color:#1d1c1d}In our production environment, hive-phoenix connector will
> take nearly 30-40 minutes to generate splits for large phoenix table, which
> has more than 2048 regions.it is because in class PhoenixInputFormat,
> function 'generateSplits' only uses one thread to generate splits for each
> scan. My proposal is to use multi-thread to generate splits in parallel. the
> proposal has been validated in our production environment.by changing code
> {color}}}{color:#1d1c1d}to generate splits in parallel with 24 threads, the
> time cost is reduced to 2 minutes. {color}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
