[
https://issues.apache.org/jira/browse/PHOENIX-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527995#comment-17527995
]
ASF GitHub Bot commented on PHOENIX-6694:
-----------------------------------------
stoty commented on PR #78:
URL:
https://github.com/apache/phoenix-connectors/pull/78#issuecomment-1109490466
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files
found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any
anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any
@author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 33s | master passed |
| +1 :green_heart: | compile | 1m 42s | master passed |
| +1 :green_heart: | checkstyle | 0m 16s | master passed |
| +1 :green_heart: | javadoc | 0m 23s | master passed |
| +0 :ok: | spotbugs | 2m 51s | phoenix-spark-base in master has 40
extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 21m 10s | the patch passed |
| +1 :green_heart: | compile | 1m 47s | the patch passed |
| +1 :green_heart: | javac | 1m 47s | the patch passed |
| -1 :x: | checkstyle | 0m 18s | phoenix-spark-base: The patch
generated 17 new + 127 unchanged - 10 fixed = 144 total (was 137) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace
issues. |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed |
| -1 :x: | spotbugs | 2m 35s | phoenix-spark-base generated 1 new + 38
unchanged - 2 fixed = 39 total (was 40) |
||| _ Other Tests _ |
| -1 :x: | unit | 5m 17s | phoenix-spark-base in the patch failed. |
| -1 :x: | asflicense | 0m 15s | The patch generated 2 ASF License
warnings. |
| | | 77m 43s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:phoenix-spark-base |
| | Class
org.apache.phoenix.spark.datasource.v2.reader.PhoenixDataSourceReadOptions
defines non-transient non-serializable instance field queryPlan In
PhoenixDataSourceReadOptions.java:instance field queryPlan In
PhoenixDataSourceReadOptions.java |
| Failed junit tests | phoenix.spark.SaltedTableIT |
| | phoenix.spark.OrderByIT |
| | phoenix.spark.AggregateIT |
| | phoenix.spark.SaltedTableIT |
| | phoenix.spark.OrderByIT |
| | phoenix.spark.AggregateIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/phoenix-connectors/pull/78 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti
checkstyle compile |
| uname | Linux fa9db5b2156a 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-connectors-personality.sh |
| git revision | master / 9ed127f |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-spark-base.txt
|
| spotbugs |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/artifact/yetus-general-check/output/new-spotbugs-phoenix-spark-base.html
|
| unit |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/artifact/yetus-general-check/output/patch-unit-phoenix-spark-base.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/testReport/
|
| asflicense |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt
|
| Max. process+thread count | 1466 (vs. ulimit of 30000) |
| modules | C: phoenix-spark-base U: phoenix-spark-base |
| Console output |
https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-Connectors-PreCommit-GitHub-PR/job/PR-78/1/console
|
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Share the query plan generated in data source reader in partition readers to
> avoid the unnecessary touch basing system tables and meta table in all the
> workers.
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-6694
> URL: https://issues.apache.org/jira/browse/PHOENIX-6694
> Project: Phoenix
> Issue Type: Bug
> Components: spark-connector
> Reporter: Rajeshbabu Chintaguntla
> Assignee: Rajeshbabu Chintaguntla
> Priority: Major
>
> Currently we are preparing the query plan in both data source and partition
> readers which is creating new connection in each worker and job
> initialisation which unnecessarily touch basing all both system catalog
> table, system stats table as well as meta. When there are jobs with millions
> of parallel workers hotspot the region servers holding the meta and system
> catalog as well system stats table. So if we share the same query plan
> between the workers which can avoid the hotspot.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
