[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378673889
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
Then you could provide a filter func as the parameter to do the filter
works. This indicates this logic doesn't belong to this class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378636611
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
server-core module depends on library-client module, so `Const#LINE` in
server-core can not used in library-client module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378636611
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
server-core module depends on server-client module, so `Const#LINE` in
server-core can not used in server-client module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378636611
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
server-core depends on server-client, so `Const#LINE` can not used in
server-client module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378636611
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
server-core module depends on server-client module, so `Const#LINE` can not
used in server-client module.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng merged pull request #4356: Fill some comments for core.
wu-sheng merged pull request #4356: Fill some comments for core. URL: https://github.com/apache/skywalking/pull/4356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch master updated: Fill some comments for core. (#4356)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new 0300048 Fill some comments for core. (#4356)
0300048 is described below
commit 0300048d9281ae52477d7dc6286b140b22c551d7
Author: 吴晟 Wu Sheng
AuthorDate: Thu Feb 13 11:27:17 2020 +0800
Fill some comments for core. (#4356)
* Fill some comments for core.
* Fix a comment issue.
Co-authored-by: kezhenxu94
---
.../src/main/resources/official_analysis.oal | 2 +-
.../oap/server/core/analysis/data/Window.java | 8 +++
.../oap/server/core/analysis/metrics/Metrics.java | 46 ++-
.../oap/server/core/analysis/record/Record.java| 8 +++
.../analysis/worker/MetricsPersistentWorker.java | 12 ++--
.../core/analysis/worker/PersistenceWorker.java| 32 ++
.../oap/server/core/register/RegisterSource.java | 3 +
.../oap/server/core/source/DefaultScopeDefine.java | 69 ++
.../oap/server/core/source/ScopeDeclaration.java | 13 +++-
.../oap/server/core/storage/StorageData.java | 6 ++
10 files changed, 180 insertions(+), 19 deletions(-)
diff --git
a/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
b/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
index ad66bac..afee68b 100755
--- a/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
+++ b/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
@@ -95,7 +95,7 @@ envoy_heap_memory_max_used =
from(EnvoyInstanceMetric.value).filter(metricName =
envoy_total_connections_used =
from(EnvoyInstanceMetric.value).filter(metricName ==
"server.total_connections").maxDouble();
envoy_parent_connections_used =
from(EnvoyInstanceMetric.value).filter(metricName ==
"server.parent_connections").maxDouble();
-// Disable unnecessary hard core sources
+// Disable unnecessary hard core stream, targeting @Stream#name
/
// disable(segment);
// disable(endpoint_relation_server_side);
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
index 4236a1f..907da94 100644
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
@@ -20,6 +20,14 @@ package org.apache.skywalking.oap.server.core.analysis.data;
import java.util.concurrent.atomic.AtomicInteger;
+/**
+ * Data cache window. Window holds two data collections(A and B). They are
switchable based on outside command. In any
+ * time, one collection is accepting the input data, and the other is
immutable.
+ *
+ * This window makes sure there is not concurrency read-write situation.
+ *
+ * @param type in the Window and internal collection.
+ */
public abstract class Window {
private AtomicInteger windowSwitch = new AtomicInteger(0);
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
index 065a98c..12dbdb3 100644
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
@@ -26,31 +26,71 @@ import
org.apache.skywalking.oap.server.core.storage.annotation.Column;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
+/**
+ * Metrics represents the statistic data, which analysis by OAL script or hard
code. It has the lifecycle controlled by
+ * TTL(time to live).
+ */
public abstract class Metrics extends StreamData implements StorageData {
public static final String TIME_BUCKET = "time_bucket";
public static final String ENTITY_ID = "entity_id";
+/**
+ * Time attribute
+ */
@Getter
@Setter
@Column(columnName = TIME_BUCKET)
private long timeBucket;
+
+/**
+ * Time in the cache, only work when
MetricsPersistentWorker#enableDatabaseSession == true.
+ */
@Getter
-@Setter
private long survivalTime = 0L;
-public abstract String id();
-
+/**
+ * Merge the given metrics instance, these two must be the same metrics
type.
+ *
+ * @param metrics to be merged
+ */
public abstract void combine(Metrics metrics);
+/**
+ * Calculate the metrics final value when required.
+ */
public abstract void calculate();
+/**
+ * Downsampling the metrics to hour precision.
+ *
+ *
[skywalking-data-collect-protocol] branch master updated: add firstReportedError field (#19)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository
https://gitbox.apache.org/repos/asf/skywalking-data-collect-protocol.git
The following commit(s) were added to refs/heads/master by this push:
new 3711072 add firstReportedError field (#19)
3711072 is described below
commit 37110723d59fdb166c5b1ff82fb3ba27c73c95cc
Author: zhang-wei
AuthorDate: Thu Feb 13 11:13:06 2020 +0800
add firstReportedError field (#19)
---
browser/BrowserPerf.proto | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/browser/BrowserPerf.proto b/browser/BrowserPerf.proto
index 2898b6c..a29e293 100644
--- a/browser/BrowserPerf.proto
+++ b/browser/BrowserPerf.proto
@@ -45,15 +45,17 @@ message BrowserPerfData {
// Page path in the browser, mostly it is URI, without parameter
string pagePath = 4;
int32 pagePathId = 5;
+// Then the PV with error is only calculated when firstReportedError is
true.
+bool firstReportedError = 6;
// Unit of all time related field should be `ms`.
-int32 redirectTime = 6;
-int32 dnsTime = 7;
-int32 reqTime = 8;
+int32 redirectTime = 7;
+int32 dnsTime = 8;
+int32 reqTime = 9;
// analysis dom tree time
-int32 domAnalysisTime = 9;
-int32 domReadyTime = 10;
+int32 domAnalysisTime = 10;
+int32 domReadyTime = 11;
// page blank time
-int32 blankTime = 11;
+int32 blankTime = 12;
}
message BrowserErrorLog {
[GitHub] [skywalking-data-collect-protocol] wu-sheng merged pull request #19: Add firstReportedError field
wu-sheng merged pull request #19: Add firstReportedError field URL: https://github.com/apache/skywalking-data-collect-protocol/pull/19 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch comments updated (f0b1b9c -> 4f9d966)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch comments in repository https://gitbox.apache.org/repos/asf/skywalking.git. from f0b1b9c Fix a comment issue. add c761f30 Some code style and refactor (#4355) add 4f9d966 Merge branch 'master' into comments No new revisions were added by this update. Summary of changes: .../core/plugin/match/ClassAnnotationMatch.java| 5 +--- .../agent/core/plugin/match/HierarchyMatch.java| 5 +--- .../TransportClientHandlerInterceptor.java | 5 +--- .../v1/TransportClientHandlerInterceptor.java | 5 +--- .../EndpointRelationServerSideMetrics.java | 5 +--- .../service/ServiceRelationClientSideMetrics.java | 5 +--- .../service/ServiceRelationServerSideMetrics.java | 5 +--- .../server/core/register/EndpointInventory.java| 5 +--- .../core/register/NetworkAddressInventory.java | 5 +--- .../core/register/ServiceInstanceInventory.java| 5 +--- .../oap/server/core/register/ServiceInventory.java | 5 +--- .../listener/segment/TraceSegmentSampler.java | 4 +-- .../plugin/jdbc/h2/dao/H2MetadataQueryDAO.java | 29 ++ 13 files changed, 19 insertions(+), 69 deletions(-)
[GitHub] [skywalking] kezhenxu94 merged pull request #4355: some code style and refactor
kezhenxu94 merged pull request #4355: some code style and refactor URL: https://github.com/apache/skywalking/pull/4355 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch master updated (7c6016b -> c761f30)
This is an automated email from the ASF dual-hosted git repository. kezhenxu94 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/skywalking.git. from 7c6016b Make Profile entities could be disable by OAL (#4354) add c761f30 Some code style and refactor (#4355) No new revisions were added by this update. Summary of changes: .../core/plugin/match/ClassAnnotationMatch.java| 5 +--- .../agent/core/plugin/match/HierarchyMatch.java| 5 +--- .../TransportClientHandlerInterceptor.java | 5 +--- .../v1/TransportClientHandlerInterceptor.java | 5 +--- .../EndpointRelationServerSideMetrics.java | 5 +--- .../service/ServiceRelationClientSideMetrics.java | 5 +--- .../service/ServiceRelationServerSideMetrics.java | 5 +--- .../server/core/register/EndpointInventory.java| 5 +--- .../core/register/NetworkAddressInventory.java | 5 +--- .../core/register/ServiceInstanceInventory.java| 5 +--- .../oap/server/core/register/ServiceInventory.java | 5 +--- .../listener/segment/TraceSegmentSampler.java | 4 +-- .../plugin/jdbc/h2/dao/H2MetadataQueryDAO.java | 29 ++ 13 files changed, 19 insertions(+), 69 deletions(-)
[GitHub] [skywalking] kezhenxu94 edited a comment on issue #4355: some code style and refactor
kezhenxu94 edited a comment on issue #4355: some code style and refactor URL: https://github.com/apache/skywalking/pull/4355#issuecomment-585519526 All plugin tests have passed except for one, PlayFramework, just because of network issue, and I'm sure it's not related to the changes here, I'll merge it now > Could not transfer artifact com.typesafe.play:twirl-parser_2.12:pom:1.3.15 from/to central (https://repo.maven.apache.org/maven2): Failed to transfer file https://repo.maven.apache.org/maven2/com/typesafe/play/twirl-parser_2.12/1.3.15/twirl-parser_2.12-1.3.15.pom with status code 500 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] kezhenxu94 commented on issue #4355: some code style and refactor
kezhenxu94 commented on issue #4355: some code style and refactor URL: https://github.com/apache/skywalking/pull/4355#issuecomment-585519526 All plugin tests have passed except for one, PlayFramework, just because of network issue, and I'm sure it's not related to the changes here, I'll merge it now This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch comments updated (7e844fd -> f0b1b9c)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch comments in repository https://gitbox.apache.org/repos/asf/skywalking.git. from 7e844fd Fill some comments for core. add f0b1b9c Fix a comment issue. No new revisions were added by this update. Summary of changes: .../org/apache/skywalking/oap/server/core/analysis/data/Window.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[GitHub] [skywalking] wu-sheng commented on issue #4356: Fill some comments for core.
wu-sheng commented on issue #4356: Fill some comments for core. URL: https://github.com/apache/skywalking/pull/4356#issuecomment-585509387 Checking whether there is format issue now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] JaredTan95 commented on issue #4356: Fill some comments for core.
JaredTan95 commented on issue #4356: Fill some comments for core. URL: https://github.com/apache/skywalking/pull/4356#issuecomment-585508991 seems CI is not stable~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng opened a new pull request #4356: Fill some comments for core.
wu-sheng opened a new pull request #4356: Fill some comments for core. URL: https://github.com/apache/skywalking/pull/4356 Spend 1 hour to fill more comments in the core, should be helpful for people to read the source codes. FYI @apache/skywalking-committers Feel free to add more in the parts you lead. Within the whole 6.x and 7.x process, the core is stable. I think it is reasonable to fill more comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] 01/01: Fill some comments for core.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch comments
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 7e844fd91e3ff8e6fde73339c38d7281de9ab557
Author: Wu Sheng
AuthorDate: Thu Feb 13 09:40:44 2020 +0800
Fill some comments for core.
---
.../src/main/resources/official_analysis.oal | 2 +-
.../oap/server/core/analysis/data/Window.java | 8 +++
.../oap/server/core/analysis/metrics/Metrics.java | 46 ++-
.../oap/server/core/analysis/record/Record.java| 8 +++
.../analysis/worker/MetricsPersistentWorker.java | 12 ++--
.../core/analysis/worker/PersistenceWorker.java| 32 ++
.../oap/server/core/register/RegisterSource.java | 3 +
.../oap/server/core/source/DefaultScopeDefine.java | 69 ++
.../oap/server/core/source/ScopeDeclaration.java | 13 +++-
.../oap/server/core/storage/StorageData.java | 6 ++
10 files changed, 180 insertions(+), 19 deletions(-)
diff --git
a/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
b/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
index ad66bac..afee68b 100755
--- a/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
+++ b/oap-server/server-bootstrap/src/main/resources/official_analysis.oal
@@ -95,7 +95,7 @@ envoy_heap_memory_max_used =
from(EnvoyInstanceMetric.value).filter(metricName =
envoy_total_connections_used =
from(EnvoyInstanceMetric.value).filter(metricName ==
"server.total_connections").maxDouble();
envoy_parent_connections_used =
from(EnvoyInstanceMetric.value).filter(metricName ==
"server.parent_connections").maxDouble();
-// Disable unnecessary hard core sources
+// Disable unnecessary hard core stream, targeting @Stream#name
/
// disable(segment);
// disable(endpoint_relation_server_side);
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
index 4236a1f..6aa3f6a 100644
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/data/Window.java
@@ -20,6 +20,14 @@ package org.apache.skywalking.oap.server.core.analysis.data;
import java.util.concurrent.atomic.AtomicInteger;
+/**
+ * Data cache window. Window holds two data collections(A and B). They are
switchable based on outside command.
+ * In any time, one collection is accepting the input data, and the other is
immutable.
+ *
+ * This window makes sure there is not concurrency read-write situation.
+ *
+ * @param
+ */
public abstract class Window {
private AtomicInteger windowSwitch = new AtomicInteger(0);
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
index 065a98c..12dbdb3 100644
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/Metrics.java
@@ -26,31 +26,71 @@ import
org.apache.skywalking.oap.server.core.storage.annotation.Column;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
+/**
+ * Metrics represents the statistic data, which analysis by OAL script or hard
code. It has the lifecycle controlled by
+ * TTL(time to live).
+ */
public abstract class Metrics extends StreamData implements StorageData {
public static final String TIME_BUCKET = "time_bucket";
public static final String ENTITY_ID = "entity_id";
+/**
+ * Time attribute
+ */
@Getter
@Setter
@Column(columnName = TIME_BUCKET)
private long timeBucket;
+
+/**
+ * Time in the cache, only work when
MetricsPersistentWorker#enableDatabaseSession == true.
+ */
@Getter
-@Setter
private long survivalTime = 0L;
-public abstract String id();
-
+/**
+ * Merge the given metrics instance, these two must be the same metrics
type.
+ *
+ * @param metrics to be merged
+ */
public abstract void combine(Metrics metrics);
+/**
+ * Calculate the metrics final value when required.
+ */
public abstract void calculate();
+/**
+ * Downsampling the metrics to hour precision.
+ *
+ * @return the metrics in hour precision in the clone mode.
+ */
public abstract Metrics toHour();
+/**
+ * Downsampling the metrics to day precision.
+ *
+ * @return the metrics in day precision in the clone mode.
+ */
public abstract Metrics toDay();
+/**
+ *
[skywalking] branch comments created (now 7e844fd)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch comments in repository https://gitbox.apache.org/repos/asf/skywalking.git. at 7e844fd Fill some comments for core. This branch includes the following new commits: new 7e844fd Fill some comments for core. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[GitHub] [skywalking] cui-liqiang opened a new pull request #4355: some code style and refactor
cui-liqiang opened a new pull request #4355: some code style and refactor URL: https://github.com/apache/skywalking/pull/4355 Please answer these questions before submitting pull request - Why submit this pull request? - [ ] Bug fix - [ ] New feature provided - [ ] Improve performance None of above, just some refactoring and code style changes mainly in two patterns: 1. Change "if(xxx) return true; return false" to "return xxx;". 2. Change "while(rs.next()) return xxx" to "if() return xxx", since rs.next() invocation will not return true once it returns false. I only changed one occurrence, and I will change the rest if it's a correct change. - Related issues ___ ### Bug fix - Bug description. - How to fix? ___ ### New feature or improvement - Describe the details and related test reports. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng merged pull request #4354: Make Profile entities could be disable by OAL
wu-sheng merged pull request #4354: Make Profile entities could be disable by OAL URL: https://github.com/apache/skywalking/pull/4354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch master updated (7ff9a0c -> 7c6016b)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git.
from 7ff9a0c supprt gRPC alarm hook. (#4344)
add 7c6016b Make Profile entities could be disable by OAL (#4354)
No new revisions were added by this update.
Summary of changes:
.../apache/skywalking/oal/rt/grammar/OALLexer.g4 | 6 ++-
.../apache/skywalking/oal/rt/grammar/OALParser.g4 | 3 +-
.../src/main/resources/official_analysis.oal | 3 ++
.../core/profile/ProfileTaskMutationService.java | 31 +
...eTaskNoneStream.java => ProfileTaskRecord.java} | 12 ++---
.../elasticsearch/query/ProfileTaskQueryEsDAO.java | 42 +-
.../plugin/jdbc/h2/dao/H2ProfileTaskQueryDAO.java | 51 +++---
7 files changed, 85 insertions(+), 63 deletions(-)
rename
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/{ProfileTaskNoneStream.java
=> ProfileTaskRecord.java} (90%)
[GitHub] [skywalking-data-collect-protocol] wu-sheng commented on a change in pull request #19: Add firstReportedError field
wu-sheng commented on a change in pull request #19: Add firstReportedError field
URL:
https://github.com/apache/skywalking-data-collect-protocol/pull/19#discussion_r378298953
##
File path: browser/BrowserPerf.proto
##
@@ -45,15 +45,17 @@ message BrowserPerfData {
// Page path in the browser, mostly it is URI, without parameter
string pagePath = 4;
int32 pagePathId = 5;
+// Then the PV with error is only calculated when firstReportedError is
true.
+bool firstReportedError = 6;
Review comment:
After release, such as 7.0.0, we only add field at the last seq for keeping
the compatible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[skywalking] branch disable_profile_idx updated (da71795 -> 512e48f)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch disable_profile_idx in repository https://gitbox.apache.org/repos/asf/skywalking.git. from da71795 Make name better rather than duplicated. add 512e48f Format some files. No new revisions were added by this update. Summary of changes: .../server/core/profile/ProfileTaskLogRecord.java | 10 ++--- .../core/profile/ProfileTaskMutationService.java | 45 ++ .../elasticsearch/query/ProfileTaskQueryEsDAO.java | 33 .../plugin/jdbc/h2/dao/H2ProfileTaskQueryDAO.java | 28 +++--- 4 files changed, 66 insertions(+), 50 deletions(-)
[GitHub] [skywalking] codecov-io commented on issue #4354: Make Profile entities could be disable by OAL
codecov-io commented on issue #4354: Make Profile entities could be disable by OAL URL: https://github.com/apache/skywalking/pull/4354#issuecomment-585228386 # [Codecov](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=h1) Report > Merging [#4354](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=desc) into [master](https://codecov.io/gh/apache/skywalking/commit/7ff9a0cfd95c0ee376607b5294740a458fdf0e8e?src=pr=desc) will **decrease** coverage by `0.17%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4354 +/- ## == - Coverage 26.71% 26.53% -0.18% == Files1181 1181 Lines 2681126811 Branches 3711 3711 == - Hits 7162 7114 -48 - Misses 1901019066 +56 + Partials 639 631 -8 ``` | [Impacted Files](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../oap/server/core/profile/ProfileTaskLogRecord.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-b2FwLXNlcnZlci9zZXJ2ZXItY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2t5d2Fsa2luZy9vYXAvc2VydmVyL2NvcmUvcHJvZmlsZS9Qcm9maWxlVGFza0xvZ1JlY29yZC5qYXZh) | `0% <ø> (ø)` | :arrow_up: | | [...gin/elasticsearch/query/ProfileTaskQueryEsDAO.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-b2FwLXNlcnZlci9zZXJ2ZXItc3RvcmFnZS1wbHVnaW4vc3RvcmFnZS1lbGFzdGljc2VhcmNoLXBsdWdpbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2t5d2Fsa2luZy9vYXAvc2VydmVyL3N0b3JhZ2UvcGx1Z2luL2VsYXN0aWNzZWFyY2gvcXVlcnkvUHJvZmlsZVRhc2tRdWVyeUVzREFPLmphdmE=) | `0% <0%> (ø)` | :arrow_up: | | [...kywalking/oap/server/core/profile/ProfileTask.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-b2FwLXNlcnZlci9zZXJ2ZXItY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2t5d2Fsa2luZy9vYXAvc2VydmVyL2NvcmUvcHJvZmlsZS9Qcm9maWxlVGFzay5qYXZh) | `0% <0%> (ø)` | | | [...erver/core/profile/ProfileTaskMutationService.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-b2FwLXNlcnZlci9zZXJ2ZXItY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2t5d2Fsa2luZy9vYXAvc2VydmVyL2NvcmUvcHJvZmlsZS9Qcm9maWxlVGFza011dGF0aW9uU2VydmljZS5qYXZh) | `0% <0%> (ø)` | :arrow_up: | | [...rage/plugin/jdbc/h2/dao/H2ProfileTaskQueryDAO.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-b2FwLXNlcnZlci9zZXJ2ZXItc3RvcmFnZS1wbHVnaW4vc3RvcmFnZS1qZGJjLWhpa2FyaWNwLXBsdWdpbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2t5d2Fsa2luZy9vYXAvc2VydmVyL3N0b3JhZ2UvcGx1Z2luL2pkYmMvaDIvZGFvL0gyUHJvZmlsZVRhc2tRdWVyeURBTy5qYXZh) | `0% <0%> (ø)` | :arrow_up: | | [...rg/apache/skywalking/apm/agent/core/os/OSUtil.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvb3MvT1NVdGlsLmphdmE=) | `0% <0%> (-76%)` | :arrow_down: | | [.../core/remote/ServiceAndEndpointRegisterClient.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvcmVtb3RlL1NlcnZpY2VBbmRFbmRwb2ludFJlZ2lzdGVyQ2xpZW50LmphdmE=) | `45.91% <0%> (-12.25%)` | :arrow_down: | | [...ache/skywalking/apm/agent/core/jvm/JVMService.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvanZtL0pWTVNlcnZpY2UuamF2YQ==) | `80% <0%> (-3.34%)` | :arrow_down: | | [...king/apm/agent/core/remote/GRPCChannelManager.java](https://codecov.io/gh/apache/skywalking/pull/4354/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvcmVtb3RlL0dSUENDaGFubmVsTWFuYWdlci5qYXZh) | `75.64% <0%> (+5.12%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=footer). Last update [7ff9a0c...da71795](https://codecov.io/gh/apache/skywalking/pull/4354?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[skywalking] branch disable_profile_idx updated (ed0ceb1 -> da71795)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch disable_profile_idx
in repository https://gitbox.apache.org/repos/asf/skywalking.git.
from ed0ceb1 Fill commented OAL scripts.
add da71795 Make name better rather than duplicated.
No new revisions were added by this update.
Summary of changes:
.../server/core/profile/ProfileTaskLogRecord.java | 12 ++---
.../core/profile/ProfileTaskMutationService.java | 21
.../{ProfileTask.java => ProfileTaskRecord.java} | 12 ++---
.../elasticsearch/query/ProfileTaskQueryEsDAO.java | 62 +++---
.../plugin/jdbc/h2/dao/H2ProfileTaskQueryDAO.java | 56 +--
5 files changed, 82 insertions(+), 81 deletions(-)
rename
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/{ProfileTask.java
=> ProfileTaskRecord.java} (91%)
[skywalking] branch disable_profile_idx updated (d1e865f -> ed0ceb1)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch disable_profile_idx in repository https://gitbox.apache.org/repos/asf/skywalking.git. from d1e865f Rename a comment to avoid misguide. add ed0ceb1 Fill commented OAL scripts. No new revisions were added by this update. Summary of changes: oap-server/server-bootstrap/src/main/resources/official_analysis.oal | 3 +++ 1 file changed, 3 insertions(+)
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378257166
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
ACK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378255741
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
> Did you answer the question in the wrong context? I am not saying
`filterNotExistIndex` is useless.
Oh, Let me perfect it. Convert at last minitue.
> But based on the things you talked, I think you should establish a cache
to `filterNotExistIndex` and cache the result for at least a day, considering
the index could only be deleted day by day, right? Set up a guava cache with
`#expireAfterWrite`?
Good advice, i will try.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378255741
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
> You covert to array at the first place, but then you covert to List again
in `#filterNotExistIndex`. Seems unnecessary?
Oh, indeed i will fix it. Convert at last minitue.
> But based on the things you talked, I think you should establish a cache
to `filterNotExistIndex` and cache the result for at least a day, considering
the index could only be deleted day by day, right? Set up a guava cache with
`#expireAfterWrite`?
Good advice, i will try.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378255741
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
> You covert to array at the first place, but then you covert to List again
in `#filterNotExistIndex`. Seems unnecessary?
Oh, indeed i will fix it. Convert at last minitue.
> But based on the things you talked, I think you should establish a cache
to `filterNotExistIndex` and cache the result for at least a day, considering
the index could only be deleted day by day, right? Set up a guava cache with
`#expireAfterWrite`?
Good advice, i will try.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng opened a new pull request #4354: Make Profile entities could be disable by OAL
wu-sheng opened a new pull request #4354: Make Profile entities could be disable by OAL URL: https://github.com/apache/skywalking/pull/4354 We missed the supports of `disable` for these three new entities of profile. We provide the `disable` capability to end-users for every single one entity. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch disable_profile_idx updated: Rename a comment to avoid misguide.
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch disable_profile_idx in repository https://gitbox.apache.org/repos/asf/skywalking.git The following commit(s) were added to refs/heads/disable_profile_idx by this push: new d1e865f Rename a comment to avoid misguide. d1e865f is described below commit d1e865f503b8c632f7dc87ebf1fd5c8dfa5a3793 Author: Wu Sheng AuthorDate: Wed Feb 12 21:39:37 2020 +0800 Rename a comment to avoid misguide. --- .../src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4 b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4 index 72bdf72..03ff22a 100644 --- a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4 +++ b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4 @@ -44,7 +44,7 @@ SRC_SERVICE_INSTANCE_CLR_GC: 'ServiceInstanceCLRGC'; SRC_SERVICE_INSTANCE_CLR_THREAD: 'ServiceInstanceCLRThread'; SRC_ENVOY_INSTANCE_METRIC: 'EnvoyInstanceMetric'; -//hard code sources, only used when need to be deactived. +//hard code entities, only used when need to be deactived. //Disable is targeting @Stream#name SRC_SEGMENT: 'segment'; SRC_TOP_N_DB_STATEMENT: 'top_n_database_statement';
[skywalking] branch disable_profile_idx created (now 3f65a73)
This is an automated email from the ASF dual-hosted git repository. wusheng pushed a change to branch disable_profile_idx in repository https://gitbox.apache.org/repos/asf/skywalking.git. at 3f65a73 Make Profile entities could be disable by OAL This branch includes the following new commits: new 3f65a73 Make Profile entities could be disable by OAL The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[skywalking] 01/01: Make Profile entities could be disable by OAL
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch disable_profile_idx
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 3f65a732d01e1b14db4a8298ac3a45c414ae6b95
Author: Wu Sheng
AuthorDate: Wed Feb 12 21:38:09 2020 +0800
Make Profile entities could be disable by OAL
---
.../apache/skywalking/oal/rt/grammar/OALLexer.g4 | 4 ++
.../apache/skywalking/oal/rt/grammar/OALParser.g4 | 3 +-
...ProfileTaskNoneStream.java => ProfileTask.java} | 12 +++---
.../server/core/profile/ProfileTaskLogRecord.java | 2 +-
.../core/profile/ProfileTaskMutationService.java | 5 +--
.../elasticsearch/query/ProfileTaskQueryEsDAO.java | 43 +++---
.../plugin/jdbc/h2/dao/H2ProfileTaskQueryDAO.java | 43 +++---
7 files changed, 57 insertions(+), 55 deletions(-)
diff --git
a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4
b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4
index 7ab88fc..72bdf72 100644
---
a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4
+++
b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4
@@ -45,6 +45,7 @@ SRC_SERVICE_INSTANCE_CLR_THREAD: 'ServiceInstanceCLRThread';
SRC_ENVOY_INSTANCE_METRIC: 'EnvoyInstanceMetric';
//hard code sources, only used when need to be deactived.
+//Disable is targeting @Stream#name
SRC_SEGMENT: 'segment';
SRC_TOP_N_DB_STATEMENT: 'top_n_database_statement';
SRC_ENDPOINT_RELATION_SERVER_SIDE: 'endpoint_relation_server_side';
@@ -55,6 +56,9 @@ SRC_HTTP_ACCESS_LOG: 'http_access_log';
SRC_ZIPKIN_SPAN: 'zipkin_span';
SRC_JAEGER_SPAN: 'jaeger_span';
+SRC_PROFILE_TASK: 'profile_task';
+SRC_PROFILE_TASK_LOG: 'profile_task_log';
+SRC_PROFILE_THREAD_SHANPSHOT: 'profile_task_segment_snapshot';
// Literals
diff --git
a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALParser.g4
b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALParser.g4
index d574f78..75a2a2d 100644
---
a/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALParser.g4
+++
b/oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALParser.g4
@@ -59,7 +59,8 @@ source
disableSource
: SRC_SEGMENT | SRC_TOP_N_DB_STATEMENT | SRC_ENDPOINT_RELATION_SERVER_SIDE
| SRC_SERVICE_RELATION_SERVER_SIDE |
- SRC_SERVICE_RELATION_CLIENT_SIDE | SRC_ALARM_RECORD |
SRC_HTTP_ACCESS_LOG | SRC_ZIPKIN_SPAN | SRC_JAEGER_SPAN
+ SRC_SERVICE_RELATION_CLIENT_SIDE | SRC_ALARM_RECORD |
SRC_HTTP_ACCESS_LOG | SRC_ZIPKIN_SPAN | SRC_JAEGER_SPAN |
+ SRC_PROFILE_TASK | SRC_PROFILE_TASK_LOG | SRC_PROFILE_THREAD_SHANPSHOT
;
sourceAttribute
diff --git
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTaskNoneStream.java
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTask.java
similarity index 90%
rename from
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTaskNoneStream.java
rename to
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTask.java
index fb92f23..4fd930a 100644
---
a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTaskNoneStream.java
+++
b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/profile/ProfileTask.java
@@ -38,8 +38,8 @@ import static
org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.PR
@Getter
@Setter
@ScopeDeclaration(id = PROFILE_TASK, name = "ProfileTask")
-@Stream(name = ProfileTaskNoneStream.INDEX_NAME, scopeId = PROFILE_TASK,
builder = ProfileTaskNoneStream.Builder.class, processor =
NoneStreamingProcessor.class)
-public class ProfileTaskNoneStream extends NoneStream {
+@Stream(name = ProfileTask.INDEX_NAME, scopeId = PROFILE_TASK, builder =
ProfileTask.Builder.class, processor = NoneStreamingProcessor.class)
+public class ProfileTask extends NoneStream {
public static final String INDEX_NAME = "profile_task";
public static final String SERVICE_ID = "service_id";
@@ -73,11 +73,11 @@ public class ProfileTaskNoneStream extends NoneStream {
@Column(columnName = MAX_SAMPLING_COUNT)
private int maxSamplingCount;
-public static class Builder implements
StorageBuilder {
+public static class Builder implements StorageBuilder {
@Override
-public ProfileTaskNoneStream map2Data(Map dbMap) {
-final ProfileTaskNoneStream record = new ProfileTaskNoneStream();
+public ProfileTask map2Data(Map dbMap) {
+final ProfileTask record = new ProfileTask();
record.setServiceId(((Number) dbMap.get(SERVICE_ID)).intValue());
record.setEndpointName((String)
[GitHub] [skywalking-data-collect-protocol] arugal opened a new pull request #19: Add firstReportedError field
arugal opened a new pull request #19: Add firstReportedError field URL: https://github.com/apache/skywalking-data-collect-protocol/pull/19 Concern at https://github.com/apache/skywalking/pull/4228#issuecomment-584992794. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378225818
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
Did you answer the question in the wrong context? I am not saying
`filterNotExistIndex` is useless.
But based on the things you talked, I think you should establish a cache to
`filterNotExistIndex` and cache the result for at least a day, considering the
index could only be deleted day by day, right? Set up a guava cache with
`#expireAfterWrite`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378223460
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
Use `Const#LINE`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on issue #4228: Support Browser protocol at OAP
wu-sheng commented on issue #4228: Support Browser protocol at OAP URL: https://github.com/apache/skywalking/pull/4228#issuecomment-585183984 I mean, `jsErrorPv` is not clear for its purpose. So I suggest changing it named as `firstReportedError`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378198260
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
Review comment:
good idea
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378197175
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378197273
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378179210
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
for example `segment alarm_record`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378153034
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
If select a date with a long span, which dates are not in the middle, or the
system has been stopped for a few days, there may be no index data for several
days. If you query date contain those days directly, searching the index will
appear `index not found exception`. so, Did a filter that the date index does
not exist
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking-client-js] Fine0830 commented on a change in pull request #6: Feat: add performance data
Fine0830 commented on a change in pull request #6: Feat: add performance data
URL: https://github.com/apache/skywalking-client-js/pull/6#discussion_r378150699
##
File path: src/performance/index.ts
##
@@ -0,0 +1,55 @@
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { CustomPerfOptionsType } from '../types';
+import Report from '../services/report';
+import pagePerf from './perf';
+
+class TracePerf {
+ private isPerf: boolean = true;
+ private perfConfig = {
+resources: [],
+perfDetail: {},
+ } as any;
+
+ public recordPerf(options: CustomPerfOptionsType) {
+if (this.isPerf) {
+ this.perfConfig.performance = pagePerf.getPerfTiming();
+}
+const perfInfo = {
+ perfDetail: this.perfConfig.perfDetail,
+ resources: this.perfConfig.resources,
+ ...options,
+};
+new Report(options.reportUrl).sendByXhr(perfInfo);
+this.clearPerf();
Review comment:
I update the PR, and I think we need service names to show in analysis UI.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] Fine0830 commented on issue #4228: Support Browser protocol at OAP
Fine0830 commented on issue #4228: Support Browser protocol at OAP URL: https://github.com/apache/skywalking/pull/4228#issuecomment-585127170 > @Fine0830 I think (isFirstPageErrorReport) should be a better name in the protocol? Hmm... isFirstPageJSErrorReport ? We will have other error rate. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng commented on issue #4228: Support Browser protocol at OAP
wu-sheng commented on issue #4228: Support Browser protocol at OAP URL: https://github.com/apache/skywalking/pull/4228#issuecomment-585124578 @Fine0830 I think (isFirstPageErrorReport) should be a better name in the protocol? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] Fine0830 edited a comment on issue #4228: Support Browser protocol at OAP
Fine0830 edited a comment on issue #4228: Support Browser protocol at OAP URL: https://github.com/apache/skywalking/pull/4228#issuecomment-585121955 I add `jsErrorPv` in error log protocol to mark. When `jsErrorPv` equal true in one error log, pv with JS error will be plus 1. @arugal You need to update the protocol as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] Fine0830 commented on issue #4228: Support Browser protocol at OAP
Fine0830 commented on issue #4228: Support Browser protocol at OAP URL: https://github.com/apache/skywalking/pull/4228#issuecomment-585121955 I add `jsErrorPv` in error log protocol to mark. @arugal You need update the protocol as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4344: supprt gRPC alarm hook.
wu-sheng commented on a change in pull request #4344: supprt gRPC alarm hook. URL: https://github.com/apache/skywalking/pull/4344#discussion_r378111507 ## File path: oap-server/server-alarm-plugin/src/main/java/org/apache/skywalking/oap/server/core/alarm/provider/AlarmRulesWatcher.java ## @@ -18,23 +18,22 @@ package org.apache.skywalking.oap.server.core.alarm.provider; +import java.io.StringReader; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.apache.skywalking.oap.server.configuration.api.ConfigChangeWatcher; import org.apache.skywalking.oap.server.core.Const; import org.apache.skywalking.oap.server.core.alarm.AlarmModule; +import org.apache.skywalking.oap.server.core.alarm.provider.grpc.GRPCAlarmSetting; Review comment: Got it. I was expected the format check will do in a strict way, it will make the changelog more clear for everyone. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378093441
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
Review comment:
Add comments for these methods.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378094991
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
+{
+add(EndpointInventory.INDEX_NAME);
+add(NetworkAddressInventory.INDEX_NAME);
+add(ServiceInventory.INDEX_NAME);
+add(ServiceInstanceInventory.INDEX_NAME);
+}
+};
+
+if (whiteIndexList.contains(modelName)) {
+return new String[] { modelName };
+}
+
+DateTime startDT =
DurationUtils.INSTANCE.startTimeBucket2DateTime(downsampling, startTB);
+DateTime endDT =
DurationUtils.INSTANCE.endTimeBucket2DateTime(downsampling, endTB);
+if (endDT.isAfter(startDT)) {
+switch (downsampling) {
+case Month:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MM.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusMonths(1);
+}
+break;
+case Day:
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Hour:
+//current hour index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Minute:
+//current minute index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+case Second:
+//current second index is also suffix with MMDD
+while (endDT.isAfter(startDT)) {
+String indexName = build(downsampling, modelName) +
"-" + MMDD.print(startDT);
+indexNameList.add(indexName);
+startDT = startDT.plusDays(1);
+}
+break;
+default:
+indexNameList.add(modelName);
+}
+}
+return indexNameList.toArray(new String[0]);
Review comment:
You covert to array at the first place, but then you covert to List again in
`#filterNotExistIndex`. Seems unnecessary?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378094387
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
+String[] fullIndexNames = formatIndexNames(indexNames);
+SearchRequest searchRequest = new SearchRequest(fullIndexNames);
+searchRequest.types(TYPE);
+searchRequest.source(searchSourceBuilder);
+return client.search(searchRequest);
+}
+
+public String[] filterNotExistIndex(String[] fullIndexNames, String
indName) throws IOException {
+// if no wrap, it is impossible to remove elements
+List indexNameList = new
ArrayList<>(Arrays.asList(fullIndexNames));
+if (fullIndexNames.length > 0) {
+List existIndex = retrievalIndexByAliases(indName);
+indexNameList.removeIf(indexName -> {
+//only filter index name with -
+if (indexName.contains("-")) {
Review comment:
Which one doesn't contain `-`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378091521
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
+List indexNameList = new ArrayList<>();
+
+List whiteIndexList = new ArrayList() {
Review comment:
This should be a static list, rather than create everytime.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378091398
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/ModelName.java
##
@@ -30,10 +43,73 @@ public static String build(Downsampling downsampling,
String modelName) {
return modelName + Const.ID_SPLIT + Downsampling.Day.getName();
case Hour:
return modelName + Const.ID_SPLIT +
Downsampling.Hour.getName();
-//case Second:
-//return modelName + Const.ID_SPLIT +
Downsampling.Second.getName();
default:
return modelName;
}
}
+
+public static String[] build(Downsampling downsampling, String modelName,
long startTB, long endTB) {
Review comment:
This build method is for ES storage only, please move inside the plugin.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378110934
##
File path:
oap-server/server-library/library-client/src/main/java/org/apache/skywalking/oap/server/library/client/elasticsearch/ElasticSearchClient.java
##
@@ -291,6 +292,30 @@ public SearchResponse search(String indexName,
SearchSourceBuilder searchSourceB
return client.search(searchRequest);
}
+public SearchResponse search(String[] indexNames, SearchSourceBuilder
searchSourceBuilder) throws IOException {
Review comment:
Also relate to `String[] indexNames`, should we keep all in List and covert
to array at last minute?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4344: supprt gRPC alarm hook.
kezhenxu94 commented on a change in pull request #4344: supprt gRPC alarm hook. URL: https://github.com/apache/skywalking/pull/4344#discussion_r378103923 ## File path: oap-server/server-alarm-plugin/src/main/java/org/apache/skywalking/oap/server/core/alarm/provider/AlarmRulesWatcher.java ## @@ -18,23 +18,22 @@ package org.apache.skywalking.oap.server.core.alarm.provider; +import java.io.StringReader; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.apache.skywalking.oap.server.configuration.api.ConfigChangeWatcher; import org.apache.skywalking.oap.server.core.Const; import org.apache.skywalking.oap.server.core.alarm.AlarmModule; +import org.apache.skywalking.oap.server.core.alarm.provider.grpc.GRPCAlarmSetting; Review comment: > FYI @kezhenxu94 By following your style, do we have a format way to make sure all these imports are in the same order rather than different people using different orders. Following the `codeStyle.xml`, the `import` order is deterministic, these changes are expected because I didn't format them before This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] JaredTan95 commented on issue #3752: Add gRPCHook to alarm-settings.yml
JaredTan95 commented on issue #3752: Add gRPCHook to alarm-settings.yml URL: https://github.com/apache/skywalking/issues/3752#issuecomment-585086836 provided. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] JaredTan95 closed issue #3752: Add gRPCHook to alarm-settings.yml
JaredTan95 closed issue #3752: Add gRPCHook to alarm-settings.yml URL: https://github.com/apache/skywalking/issues/3752 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[skywalking] branch master updated (e5366c0 -> 7ff9a0c)
This is an automated email from the ASF dual-hosted git repository.
tanjian pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git.
from e5366c0 Tag annotation supports returned expression (#4327)
add 7ff9a0c supprt gRPC alarm hook. (#4344)
No new revisions were added by this update.
Summary of changes:
docs/en/setup/backend/backend-alarm.md | 19 +++
.../exporter/provider/grpc/GRPCExporter.java | 88 +--
.../exporter/provider/grpc/GRPCExporterTest.java | 13 +-
oap-server/server-alarm-plugin/pom.xml | 34 +
.../core/alarm/provider/AlarmRulesWatcher.java | 21 +--
.../server/core/alarm/provider/NotifyHandler.java | 2 +
.../oap/server/core/alarm/provider/Rules.java | 2 +
.../server/core/alarm/provider/RulesReader.java| 17 +++
.../alarm/provider/grpc/GRPCAlarmSetting.java} | 14 +-
.../core/alarm/provider/grpc/GRPCCallback.java | 163 +
.../src/main/proto/alarm-hook.proto| 26 ++--
.../core/alarm/provider/AlarmRulesWatcherTest.java | 4 +
.../alarm/provider/grpc/AlarmMockReceiver.java}| 49 +++
.../alarm/provider/grpc/GRPChookCallbackTest.java | 79 ++
.../src/test/resources/alarm-settings.yml | 4 +
.../src/main/resources/alarm-settings.yml | 4 +
.../{ExporterModule.java => ExportData.java} | 22 +--
.../util/{Address.java => GRPCStreamStatus.java} | 14 +-
18 files changed, 450 insertions(+), 125 deletions(-)
copy
oap-server/{exporter/src/main/java/org/apache/skywalking/oap/server/exporter/provider/grpc/GRPCExporterSetting.java
=>
server-alarm-plugin/src/main/java/org/apache/skywalking/oap/server/core/alarm/provider/grpc/GRPCAlarmSetting.java}
(77%)
create mode 100644
oap-server/server-alarm-plugin/src/main/java/org/apache/skywalking/oap/server/core/alarm/provider/grpc/GRPCCallback.java
copy
test/plugin/mock-collector/src/main/proto/language-agent-v2/JVMMetric.proto =>
oap-server/server-alarm-plugin/src/main/proto/alarm-hook.proto (69%)
copy
oap-server/{exporter/src/test/java/org/apache/skywalking/oap/server/exporter/provider/grpc/ExporterMockReceiver.java
=>
server-alarm-plugin/src/test/java/org/apache/skywalking/oap/server/core/alarm/provider/grpc/AlarmMockReceiver.java}
(52%)
create mode 100644
oap-server/server-alarm-plugin/src/test/java/org/apache/skywalking/oap/server/core/alarm/provider/grpc/GRPChookCallbackTest.java
copy
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/exporter/{ExporterModule.java
=> ExportData.java} (69%)
copy
oap-server/server-library/library-util/src/main/java/org/apache/skywalking/oap/server/library/util/{Address.java
=> GRPCStreamStatus.java} (87%)
[GitHub] [skywalking] JaredTan95 merged pull request #4344: supprt gRPC alarm hook.
JaredTan95 merged pull request #4344: supprt gRPC alarm hook. URL: https://github.com/apache/skywalking/pull/4344 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] codecov-io edited a comment on issue #4239: Provide influxdb as a new storage plugin
codecov-io edited a comment on issue #4239: Provide influxdb as a new storage plugin URL: https://github.com/apache/skywalking/pull/4239#issuecomment-575965825 # [Codecov](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=h1) Report > Merging [#4239](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=desc) into [master](https://codecov.io/gh/apache/skywalking/commit/918673d7bec7906c4fef7f304cc86eca1534df99?src=pr=desc) will **increase** coverage by `0.27%`. > The diff coverage is `52.72%`. [](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4239 +/- ## == + Coverage 25.38% 25.65% +0.27% == Files1196 1197 +1 Lines 2747427518 +44 Branches 3787 3802 +15 == + Hits 6974 7061 +87 + Misses 1988419833 -51 - Partials 616 624 +8 ``` | [Impacted Files](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=tree) | Coverage Δ | | |---|---|---| | [...lking/apm/agent/core/util/CustomizeExpression.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvdXRpbC9DdXN0b21pemVFeHByZXNzaW9uLmphdmE=) | `0% <0%> (ø)` | :arrow_up: | | [...vation/trace/TraceAnnotationMethodInterceptor.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLXRvb2xraXQtYWN0aXZhdGlvbi9hcG0tdG9vbGtpdC10cmFjZS1hY3RpdmF0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9za3l3YWxraW5nL2FwbS90b29sa2l0L2FjdGl2YXRpb24vdHJhY2UvVHJhY2VBbm5vdGF0aW9uTWV0aG9kSW50ZXJjZXB0b3IuamF2YQ==) | `62.5% <61.11%> (+12.5%)` | :arrow_up: | | [...tivation/trace/TagAnnotationMethodInterceptor.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLXRvb2xraXQtYWN0aXZhdGlvbi9hcG0tdG9vbGtpdC10cmFjZS1hY3RpdmF0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9za3l3YWxraW5nL2FwbS90b29sa2l0L2FjdGl2YXRpb24vdHJhY2UvVGFnQW5ub3RhdGlvbk1ldGhvZEludGVyY2VwdG9yLmphdmE=) | `68.96% <81.25%> (+68.96%)` | :arrow_up: | | [...kywalking/apm/toolkit/activation/util/TagUtil.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLXRvb2xraXQtYWN0aXZhdGlvbi9hcG0tdG9vbGtpdC10cmFjZS1hY3RpdmF0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9za3l3YWxraW5nL2FwbS90b29sa2l0L2FjdGl2YXRpb24vdXRpbC9UYWdVdGlsLmphdmE=) | `83.33% <83.33%> (ø)` | | | [...king/apm/agent/core/remote/GRPCChannelManager.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvcmVtb3RlL0dSUENDaGFubmVsTWFuYWdlci5qYXZh) | `70.51% <0%> (+1.28%)` | :arrow_up: | | [.../agent/core/context/trace/AbstractTracingSpan.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvY29udGV4dC90cmFjZS9BYnN0cmFjdFRyYWNpbmdTcGFuLmphdmE=) | `64.28% <0%> (+3.57%)` | :arrow_up: | | [.../skywalking/apm/agent/core/remote/GRPCChannel.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvcmVtb3RlL0dSUENDaGFubmVsLmphdmE=) | `87.5% <0%> (+6.25%)` | :arrow_up: | | [.../agent/core/profile/ProfileTaskChannelService.java](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree#diff-YXBtLXNuaWZmZXIvYXBtLWFnZW50LWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NreXdhbGtpbmcvYXBtL2FnZW50L2NvcmUvcHJvZmlsZS9Qcm9maWxlVGFza0NoYW5uZWxTZXJ2aWNlLmphdmE=) | `45.16% <0%> (+10.75%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/skywalking/pull/4239/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=footer). Last update [918673d...767c85a](https://codecov.io/gh/apache/skywalking/pull/4239?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378090251
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/DurationUtils.java
##
@@ -213,4 +213,36 @@ private DateTime parseToDateTime(Downsampling
downsampling, long time) {
}
throw new UnexpectedException("Unexpected downsampling: " +
downsampling.name());
}
+
+public DateTime startTimeBucket2DateTime(Downsampling downsampling, long
startTB) {
Review comment:
OK, got it, make sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] aderm commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
aderm commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378089483
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/DurationUtils.java
##
@@ -213,4 +213,36 @@ private DateTime parseToDateTime(Downsampling
downsampling, long time) {
}
throw new UnexpectedException("Unexpected downsampling: " +
downsampling.name());
}
+
+public DateTime startTimeBucket2DateTime(Downsampling downsampling, long
startTB) {
Review comment:
When crossing months, you need to consider the big and small months. In
addition, you need to consider some leap years. like `20200227-20200305`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4353: Optimizing performance reduces es index queries scope by timebucket
wu-sheng commented on a change in pull request #4353: Optimizing performance
reduces es index queries scope by timebucket
URL: https://github.com/apache/skywalking/pull/4353#discussion_r378088298
##
File path:
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/DurationUtils.java
##
@@ -213,4 +213,36 @@ private DateTime parseToDateTime(Downsampling
downsampling, long time) {
}
throw new UnexpectedException("Unexpected downsampling: " +
downsampling.name());
}
+
+public DateTime startTimeBucket2DateTime(Downsampling downsampling, long
startTB) {
Review comment:
Why do you need to add? The current prefix should be enough.
`2020021015`-`2020021315` to days is 20200210`-`20200213`
Do I miss anything?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
