[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625500#comment-16625500
]
ASF subversion and git services commented on KYLIN-3543:
Commit 6dcd79566e3a8a65650dfde3a3962bf51d52c4b6 in kylin's branch
refs/heads/update_travis_maven from [~yichen.zhou]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6dcd795 ]
KYLIN-3543 Export HBase configuration only for Spark engine
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.0
>Reporter: Ted Yu
>Assignee: Yichen Zhou
>Priority: Minor
> Fix For: v2.5.1
>
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620132#comment-16620132
]
ASF subversion and git services commented on KYLIN-3543:
Commit f3da67746e730b9f69a8bf128854c5c9d2a207a0 in kylin's branch
refs/heads/2.5.x from [~yichen.zhou]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=f3da677 ]
KYLIN-3543 Export HBase configuration only for Spark engine
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.5.0
>Reporter: Ted Yu
>Assignee: Yichen Zhou
>Priority: Minor
> Fix For: v2.5.1
>
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619965#comment-16619965
]
ASF subversion and git services commented on KYLIN-3543:
Commit 6dcd79566e3a8a65650dfde3a3962bf51d52c4b6 in kylin's branch
refs/heads/master from [~yichen.zhou]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6dcd795 ]
KYLIN-3543 Export HBase configuration only for Spark engine
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619964#comment-16619964
]
ASF GitHub Bot commented on KYLIN-3543:
---
shaofengshi closed pull request #237: KYLIN-3543 Export HBase configuration
only for Spark engine
URL: https://github.com/apache/kylin/pull/237
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java
b/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java
index 93c12960de..7205802388 100644
---
a/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java
+++
b/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java
@@ -53,6 +53,7 @@
import org.apache.kylin.engine.mr.common.CubeStatsReader;
import org.apache.kylin.engine.mr.common.CuboidShardUtil;
import org.apache.kylin.job.exception.ExecuteException;
+import org.apache.kylin.metadata.model.IEngineAware;
import org.apache.kylin.storage.hbase.HBaseConnection;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -119,7 +120,11 @@ public int run(String[] args) throws Exception {
partitionFilePath.getParent());
CubeHTableUtil.createHTable(cubeSegment, splitKeys);
-exportHBaseConfiguration(cubeSegment.getStorageLocationIdentifier());
+
+// export configuration in advance to avoid connecting to hbase from
spark
+if (cubeDesc.getEngineType()== IEngineAware.ID_SPARK){
+
exportHBaseConfiguration(cubeSegment.getStorageLocationIdentifier());
+}
return 0;
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618932#comment-16618932
]
ASF GitHub Bot commented on KYLIN-3543:
---
asfgit commented on issue #237: KYLIN-3543 Export HBase configuration only for
Spark engine
URL: https://github.com/apache/kylin/pull/237#issuecomment-422351011
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618933#comment-16618933
]
ASF GitHub Bot commented on KYLIN-3543:
---
asfgit commented on issue #237: KYLIN-3543 Export HBase configuration only for
Spark engine
URL: https://github.com/apache/kylin/pull/237#issuecomment-422351010
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618931#comment-16618931
]
ASF GitHub Bot commented on KYLIN-3543:
---
ZhouYC627 opened a new pull request #237: KYLIN-3543 Export HBase configuration
only for Spark engine
URL: https://github.com/apache/kylin/pull/237
https://issues.apache.org/jira/browse/KYLIN-3543
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616840#comment-16616840
]
Ted Yu commented on KYLIN-3543:
---
Is it possible to do something for the building cube with MR case ?
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3543) Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
[
https://issues.apache.org/jira/browse/KYLIN-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606656#comment-16606656
]
Yichen Zhou commented on KYLIN-3543:
Thank you Ted. The job configuration is saved in HDFS for later use. Please
refer to this: https://issues.apache.org/jira/browse/KYLIN-3446
Since we could not connect HBase in spark, this job should remain open until
spark use it in "convert to HFlie" step. However, when building cube with MR,
this job does become unless. But I think it's ok for the little cost.
> Unclosed Job instance in CreateHTableJob#exportHBaseConfiguration
> -
>
> Key: KYLIN-3543
> URL: https://issues.apache.org/jira/browse/KYLIN-3543
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> out = fs.create(new Path(hbaseConfPath));
> job.getConfiguration().writeXml(out);
> {code}
> The job instance should be closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
