[jira] [Updated] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-5229:
Description:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{{}PropertiesDelegate introduces performance downgrade because it add lock to
KylinConfigBase#getOptional{}}}, impact query concurrency badly, from Yanghong
Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
KylinConfigBase
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
PropertiesDelegate
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step 4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties()); // Step 5
}
propertiesView.putAll(this.properties);
return propertiesView;
}
Hashtable
public synchronized void putAll(Map t) {
for (Map.Entry e : t.entrySet())
put(e.getKey(), e.getValue()); // Step 7
}
public synchronized V put(K key, V value) { // Step 8
// Make sure the value is not null
if (value == null) {
throw new NullPointerException();
}
// Makes sure the key is not already in the hashtable.
Entry tab[] = table;
int hash = key.hashCode();
int index = (hash & 0x7FFF) % tab.length;
@SuppressWarnings("unchecked")
Entry entry = (Entry)tab[index];
for(; entry != null ; entry = entry.next) {
if ((entry.hash == hash) && entry.key.equals(key)) {
V old = entry.value;
entry.value = value;
return old;
}
}
addEntry(hash, key, value, index);
return null;
}
{code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-17-40-751.png!
was:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{{}PropertiesDelegate introduces performance downgrade because it add lock to
KylinConfigBase#getOptional{}}}, impact query concurrency badly, from Yanghong
Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-17-40-751.png!
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kylin 5.0 dev team plans to add new feature which supports reading config
> entry from external Config Server(such as Nacos) other than local
> kylin.properties. So we add {{PropertiesDelegate}} and
> {{IExternalConfigLoader}} to support this request.
> h2. Issue
>
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580580#comment-17580580 ] ASF subversion and git services commented on KYLIN-5221: Commit 04f81da701827a8d26aa185b323e0d299d09bb41 in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=04f81da701 ] KYLIN-5221 add search feature > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580579#comment-17580579 ] ASF subversion and git services commented on KYLIN-5221: Commit 5221ec3de31e39c43183af24d5809f704295114b in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=5221ec3de3 ] KYLIN-5221 add operations doc > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580577#comment-17580577 ] ASF subversion and git services commented on KYLIN-5221: Commit 02e31db7f0b14469d39efa49fbbd242512ccede5 in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=02e31db7f0 ] KYLIN-5221 add operations docs > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580575#comment-17580575 ] ASF subversion and git services commented on KYLIN-5221: Commit d321b51c9da2a9de5ab58b8457ef731ae033463c in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d321b51c9d ] KYLIN-5221 add installation docs > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580578#comment-17580578 ] ASF subversion and git services commented on KYLIN-5221: Commit b9cfe7db020fe7f2224ab82dcb4bd3288c4fb160 in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b9cfe7db02 ] KYLIN-5221 minor adapt for sidebars and date in doc metadata > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580581#comment-17580581 ] ASF subversion and git services commented on KYLIN-5221: Commit 830a52f0b258a9163782e8dda5258cea7e5d4cd7 in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=830a52f0b2 ] KYLIN-5221 minor fix conflict > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580576#comment-17580576 ] ASF subversion and git services commented on KYLIN-5221: Commit 69ee8500bb27e6585f82b2da0f6921e5e949 in kylin's branch refs/heads/doc5.0 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=69ee8500bb ] KYLIN-5221 add configuration docs > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kylin] hit-lacus merged pull request #1938: Add doc 0816
hit-lacus merged PR #1938: URL: https://github.com/apache/kylin/pull/1938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-5229:
Description:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{{}PropertiesDelegate introduces performance downgrade because it add lock to
KylinConfigBase#getOptional{}}}, impact query concurrency badly, from Yanghong
Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-17-40-751.png!
was:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{{}PropertiesDelegate }}introduces performance downgrade because it add lock
to \{{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
Yanghong Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-17-40-751.png!
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kylin 5.0 dev team plans to add new feature which supports reading config
> entry from external Config Server(such as Nacos) other than local
> kylin.properties. So we add {{PropertiesDelegate}} and
> {{IExternalConfigLoader}} to support this request.
> h2. Issue
> {{{}PropertiesDelegate introduces performance downgrade because it add lock
> to KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
> Yanghong Team's load testing result, query concurrency change from 60 to 3.
> h2. Root Cause
>
> {code:java}
> protected String getOptional(String prop, String dft) {
> final String property = System.getProperty(prop);
> return property != null ? getSubstitutor().replace(property)
> : getSubstitutor().replace(properties.getProperty(prop, dft)); //
> Step 1
> }
>
> protected StrSubstitutor getSubstitutor() {
> // overrides > env > properties
> final Map all = Maps.newHashMap();
> all.putAll((Map) properties); // Step 2
> all.putAll(STATIC_SYSTEM_ENV);
> all.putAll(overrides);
> return new StrSubstitutor(all);
> }
> @Override
> public Set> entrySet() {
> return
[jira] [Updated] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-5229:
Attachment: image-2022-08-17-13-17-40-751.png
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kylin 5.0 dev team plans to add new feature which supports reading config
> entry from external Config Server(such as Nacos) other than local
> kylin.properties. So we add {{PropertiesDelegate}} and
> {{IExternalConfigLoader}} to support this request.
> h2. Issue
> {{PropertiesDelegate }}introduces performance downgrade because it add lock
> to {{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
> Yanghong Team's load testing result, query concurrency change from 60 to 3.
> h2. Root Cause
>
> {code:java}
> protected String getOptional(String prop, String dft) {
> final String property = System.getProperty(prop);
> return property != null ? getSubstitutor().replace(property)
> : getSubstitutor().replace(properties.getProperty(prop, dft)); //
> Step 1
> }
>
> protected StrSubstitutor getSubstitutor() {
> // overrides > env > properties
> final Map all = Maps.newHashMap();
> all.putAll((Map) properties); // Step 2
> all.putAll(STATIC_SYSTEM_ENV);
> all.putAll(overrides);
> return new StrSubstitutor(all);
> }
> @Override
> public Set> entrySet() {
> return getAllProperties().entrySet(); // Step 3
> }
> private synchronized Properties getAllProperties() { // Step4, synchronized
>
> Properties propertiesView = new Properties();
> if (this.configLoader != null) {
> propertiesView.putAll(this.configLoader.getProperties());
> }
> propertiesView.putAll(this.properties);
> return propertiesView;
> } {code}
>
> h2. Screenshots
> Java Stack of Query Thread:
> !image-2022-08-17-13-16-25-721.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-5229:
Description:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{{}PropertiesDelegate }}introduces performance downgrade because it add lock
to \{{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
Yanghong Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-17-40-751.png!
was:
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{PropertiesDelegate }}introduces performance downgrade because it add lock to
{{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
Yanghong Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-16-25-721.png!
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kylin 5.0 dev team plans to add new feature which supports reading config
> entry from external Config Server(such as Nacos) other than local
> kylin.properties. So we add {{PropertiesDelegate}} and
> {{IExternalConfigLoader}} to support this request.
> h2. Issue
> {{{}PropertiesDelegate }}introduces performance downgrade because it add lock
> to \{{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
> Yanghong Team's load testing result, query concurrency change from 60 to 3.
> h2. Root Cause
>
> {code:java}
> protected String getOptional(String prop, String dft) {
> final String property = System.getProperty(prop);
> return property != null ? getSubstitutor().replace(property)
> : getSubstitutor().replace(properties.getProperty(prop, dft)); //
> Step 1
> }
>
> protected StrSubstitutor getSubstitutor() {
> // overrides > env > properties
> final Map all = Maps.newHashMap();
> all.putAll((Map) properties); // Step 2
> all.putAll(STATIC_SYSTEM_ENV);
> all.putAll(overrides);
> return new StrSubstitutor(all);
> }
> @Override
> public Set> entrySet() {
> return
[jira] [Created] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
Xiaoxiang Yu created KYLIN-5229:
---
Summary: PropertiesDelegate introduce lock to KylinConfig
Key: KYLIN-5229
URL: https://issues.apache.org/jira/browse/KYLIN-5229
Project: Kylin
Issue Type: Sub-task
Components: Metadata
Reporter: Xiaoxiang Yu
Fix For: 5.0-alpha
Attachments: image-2022-08-17-13-17-40-751.png
h2. Background
Kylin 5.0 dev team plans to add new feature which supports reading config entry
from external Config Server(such as Nacos) other than local kylin.properties.
So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support this
request.
h2. Issue
{{PropertiesDelegate }}introduces performance downgrade because it add lock to
{{{}KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
Yanghong Team's load testing result, query concurrency change from 60 to 3.
h2. Root Cause
{code:java}
protected String getOptional(String prop, String dft) {
final String property = System.getProperty(prop);
return property != null ? getSubstitutor().replace(property)
: getSubstitutor().replace(properties.getProperty(prop, dft)); //
Step 1
}
protected StrSubstitutor getSubstitutor() {
// overrides > env > properties
final Map all = Maps.newHashMap();
all.putAll((Map) properties); // Step 2
all.putAll(STATIC_SYSTEM_ENV);
all.putAll(overrides);
return new StrSubstitutor(all);
}
@Override
public Set> entrySet() {
return getAllProperties().entrySet(); // Step 3
}
private synchronized Properties getAllProperties() { // Step4, synchronized
Properties propertiesView = new Properties();
if (this.configLoader != null) {
propertiesView.putAll(this.configLoader.getProperties());
}
propertiesView.putAll(this.properties);
return propertiesView;
} {code}
h2. Screenshots
Java Stack of Query Thread:
!image-2022-08-17-13-16-25-721.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1938: Add doc 0816
lgtm-com[bot] commented on PR #1938: URL: https://github.com/apache/kylin/pull/1938#issuecomment-1217387130 This pull request **introduces 1 alert** when merging d803bdcc6b9fb129f89765c2a4aad6d391c6ec27 into a47c606feeefe06feedc1ff6672e4125ab52db3d - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-edb8604ab1ee34a90c84d074b99ba419ad284899) **new alerts:** * 1 for Overwritten property -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang updated KYLIN-5114:
--
Labels: Question (was: )
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Task
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Labels: Question
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang updated KYLIN-5114:
--
Issue Type: Task (was: Bug)
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Task
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580400#comment-17580400
]
liyang edited comment on KYLIN-5114 at 8/16/22 4:41 PM:
这类是环境问题，假如不是一模一样的环境（centos7，Hive 3.1.2，hadoop 3.1.3，spark 2.4.5）即便是同样的报错解法也不一样。
只有解决思路是通用的：
# 看日志先确认，是哪个进程报出的错。可能是 kylin 进程，也可能是远端的 spark 进程，解法会不同。
# 根据具体 java 进程，再看它的 classpath 怎么组装来的。比如 [How can I view the classpath and jvm
args of an executing java program in windows - Stack
Overflow|https://stackoverflow.com/questions/17151058/how-can-i-view-the-classpath-and-jvm-args-of-an-executing-java-program-in-window]
# 定位到有问题的 guava 包，再看怎么替换或者升级
was (Author: liyang.g...@gmail.com):
这类是环境问题，假如不是一模一样的环境（centos7，Hive 3.1.2，hadoop 3.1.3，spark
2.4.5）即便是同样的报错解法也不一样。解决思路通常是这样：
# 看日志先确认，是哪个进程报出的错。可能是 kylin 进程，也可能是远端的 spark 进程，解法会不同。
# 根据具体 java 进程，再看它的 classpath 怎么组装来的。比如 [How can I view the classpath and jvm
args of an executing java program in windows - Stack
Overflow|https://stackoverflow.com/questions/17151058/how-can-i-view-the-classpath-and-jvm-args-of-an-executing-java-program-in-window]
# 定位到有问题的 guava 包，再看怎么替换或者升级
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580400#comment-17580400
]
liyang commented on KYLIN-5114:
---
这类是环境问题，假如不是一模一样的环境（centos7，Hive 3.1.2，hadoop 3.1.3，spark
2.4.5）即便是同样的报错解法也不一样。解决思路通常是这样：
# 看日志先确认，是哪个进程报出的错。可能是 kylin 进程，也可能是远端的 spark 进程，解法会不同。
# 根据具体 java 进程，再看它的 classpath 怎么组装来的。比如 [How can I view the classpath and jvm
args of an executing java program in windows - Stack
Overflow|https://stackoverflow.com/questions/17151058/how-can-i-view-the-classpath-and-jvm-args-of-an-executing-java-program-in-window]
# 定位到有问题的 guava 包，再看怎么替换或者升级
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580295#comment-17580295
]
dingpeng commented on KYLIN-5114:
-
一模一样的错误，请问解决了吗
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [kylin] Mukvin opened a new pull request, #1938: Add doc 0816
Mukvin opened a new pull request, #1938: URL: https://github.com/apache/kylin/pull/1938 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Github Branch As most of the development works are on Kylin 4, we need to switch it as main branch. Apache Kylin community changes the branch settings on Github since 2021-08-04 : 1. The default branch _main_ is for **Kylin 4.x** (Parquet storage); 2. The original branch _master_ for **Kylin 3.x** (HBase storage) has been renamed to **kylin3** ; Please check [Intro to Kylin 4 architecture](https://kylin.apache.org/blog/2021/07/02/Apache-Kylin4-A-new-storage-and-compute-architecture/) and [INFRA-22166](https://issues.apache.org/jira/browse/INFRA-22166) if you are interested. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
