[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427939#comment-16427939
]
stack commented on HBASE-16638:
---
hbase-spark module reverted from branch-2/2.0.0 by HBASE-18817
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Improvement
> Components: spark
>Reporter: Ted Yu
>Assignee: Weiqing Yang
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_ADDENDUM_v2.patch,
> HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576312#comment-15576312
]
Hudson commented on HBASE-16638:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1786 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1786/])
HBASE-16638 Reduce the number of Connection's created in classes of (tedyu: rev
07086036a5295b79c11fd27e0bc7f6ecfd5129b5)
* (edit)
hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCacheSuite.scala
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_ADDENDUM_v2.patch,
> HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575711#comment-15575711
]
Ted Yu commented on HBASE-16638:
>From https://builds.apache.org/job/PreCommit-HBASE-Build/4005/console :
+1 overall
| Vote | Subsystem | Runtime | Comment
| 0 | reexec | 0m 14s| Docker mode activated.
| +1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns.
| +1 |@author | 0m 0s | The patch does not contain any @author
| | || tags.
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or
| | || modified test files.
| +1 | mvninstall | 3m 45s| master passed
| +1 |compile | 1m 2s | master passed
| +1 | scaladoc | 0m 29s| master passed
| +1 |compile | 1m 4s | the patch passed
| +1 | scalac | 1m 4s | the patch passed
| +1 | whitespace | 0m 0s | The patch has no whitespace issues.
| +1 |hadoopcheck | 27m 34s | Patch does not cause any errors with
| | || Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2
| | || 2.6.1 2.6.2 2.6.3 2.7.1.
| +1 |hbaseprotoc | 0m 31s| the patch passed
| +1 | scaladoc | 0m 22s| the patch passed
| +1 | unit | 0m 50s| hbase-spark in the patch passed.
| +1 | asflicense | 0m 8s | The patch does not generate ASF License
| | || warnings.
| | | 36m 12s |
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12833291/HBASE-16638_ADDENDUM_v2.patch
|
| JIRA Issue | HBASE-16638 |
| Optional Tests | asflicense scalac scaladoc unit compile |
| uname | Linux 9aa7db47abeb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 91a7bbd |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4005/testReport/ |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4005/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_ADDENDUM_v2.patch,
> HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574163#comment-15574163
]
Hadoop QA commented on HBASE-16638:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 50s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12833291/HBASE-16638_ADDENDUM_v2.patch
|
| JIRA Issue | HBASE-16638 |
| Optional Tests | asflicense scalac scaladoc unit compile |
| uname | Linux 9aa7db47abeb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 91a7bbd |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4005/testReport/ |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4005/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_ADDENDUM_v2.patch,
> HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574089#comment-15574089
]
Weiqing Yang commented on HBASE-16638:
--
Waiting for the test results.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_ADDENDUM_v2.patch,
> HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573773#comment-15573773
]
Weiqing Yang commented on HBASE-16638:
--
Actually, the test HBaseConnectionCacheSuite can pass in my local machine. I
guess, in this QA testing, HBaseConnectionCacheSuite ran slower than the
previous two QA tests (and my local machine). I am going to submit a new patch
to update time parameter passed to "HBaseConnectionCache.setTimeout".
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573720#comment-15573720
]
Weiqing Yang commented on HBASE-16638:
--
Looking into this
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573697#comment-15573697
]
Appy commented on HBASE-16638:
--
HBaseConnectionCacheSuite is failing
(https://builds.apache.org/view/All/job/HBase-Trunk_matrix/jdk=JDK%201.8%20(latest),label=yahoo-not-h2/1776/consoleFull).
Just found out that pre-commit is not running hbase-spark tests, that's why it
wasn't caught in QA.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566388#comment-15566388
]
Hudson commented on HBASE-16638:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1768 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1768/])
HBASE-16638 Reduce the number of Connection's created in classes of (tedyu: rev
9d304d3b2dffdbe4fe17fa64cea1fcd3b3d52e2c)
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala
* (add)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCache.scala
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala
* (add)
hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCacheSuit.scala
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseResources.scala
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseSparkConf.scala
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala
HBASE-16638 Reduce the number of Connection's created in classes of (tedyu: rev
ee6f0ddef67e7840302640f7237d8308fc3eeadb)
* (delete)
hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCacheSuit.scala
* (edit)
hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCache.scala
* (add)
hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseConnectionCacheSuite.scala
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565987#comment-15565987
]
Weiqing Yang commented on HBASE-16638:
--
Thanks a lot, Ted. I just attached a ADDENDUM_v1 patch. Build successfully and
passed related tests.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Fix For: 2.0.0
>
> Attachments: HBASE-16638.ADDENDUM.patch,
> HBASE-16638_ADDENDUM_v1.patch, HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565632#comment-15565632
]
Ted Yu commented on HBASE-16638:
+1
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549822#comment-15549822
]
Hadoop QA commented on HBASE-16638:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 19s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831800/HBASE-16638_v2.patch |
| JIRA Issue | HBASE-16638 |
| Optional Tests | asflicense scalac scaladoc unit compile |
| uname | Linux 1021e8986222 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 06758bf |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3842/testReport/ |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3842/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549724#comment-15549724
]
Weiqing Yang commented on HBASE-16638:
--
Thanks, [~te...@apache.org][~enis]. I have submitted the second version and
linked its review board. Could you please review it?
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch, HBASE-16638_v2.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547307#comment-15547307
]
Ted Yu commented on HBASE-16638:
Mind putting patch on review board ?
Thanks
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547272#comment-15547272
]
Enis Soztutar commented on HBASE-16638:
---
I've started working on HBASE-16713, but it will take some time. Plus, we are
not sure which branches it will go as of now. Let's continue with this patch.
We can revisit this area once we have HBASE-16713 in all applicable code bases.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547258#comment-15547258
]
Ted Yu commented on HBASE-16638:
I think we should continue with this patch.
It is not clear when HBASE-16713 would be implemented.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547238#comment-15547238
]
Hadoop QA commented on HBASE-16638:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 43s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 23s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831649/HBASE-16638_v1.patch |
| JIRA Issue | HBASE-16638 |
| Optional Tests | asflicense scalac scaladoc unit compile |
| uname | Linux cdb6082543eb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0daeb63 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3824/testReport/ |
| modules | C: hbase-spark U: hbase-spark |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3824/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547129#comment-15547129
]
Weiqing Yang commented on HBASE-16638:
--
Hi, [~ted_yu] I have attached the patch. But there is another jira
[HBASE-16713|https://issues.apache.org/jira/browse/HBASE-16713] which will
implement the connection caching on HBase side. If HBASE-16713 is fixed, this
patch would be not necessary. So I am not sure how to deal with this? Could you
please give some direction? Thanks. Adding [~enis]
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
> Attachments: HBASE-16638_v1.patch
>
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16638) Reduce the number of Connection's created in classes of hbase-spark module
[
https://issues.apache.org/jira/browse/HBASE-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494598#comment-15494598
]
Weiqing Yang commented on HBASE-16638:
--
I will work on this.
> Reduce the number of Connection's created in classes of hbase-spark module
> --
>
> Key: HBASE-16638
> URL: https://issues.apache.org/jira/browse/HBASE-16638
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Weiqing Yang
>
> In HBaseResources.scala :
> {code}
> case class RegionResource(relation: HBaseRelation) extends ReferencedResource
> {
> var connection: Connection = _
> var rl: RegionLocator = _
> ...
> override def init(): Unit = {
> connection = ConnectionFactory.createConnection(relation.hbaseConf)
> {code}
> Connection is created per RegionResource.
> This is not necessary.
> Caching of Connection's should be provided to reduce the number of concurrent
> Connection's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
