[GitHub] [skywalking] wu-sheng commented on a change in pull request #4292: Improve ES query performance
wu-sheng commented on a change in pull request #4292: Improve ES query
performance
URL: https://github.com/apache/skywalking/pull/4292#discussion_r372845626
##
File path:
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/query/AggregationQueryEsDAO.java
##
@@ -96,7 +96,7 @@ public AggregationQueryEsDAO(ElasticSearchClient client) {
SearchSourceBuilder sourceBuilder = SearchSourceBuilder.searchSource();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
-sourceBuilder.query(boolQueryBuilder);
+
sourceBuilder.query(QueryBuilders.boolQuery().filter(boolQueryBuilder));
boolQueryBuilder.must().add(QueryBuilders.rangeQuery(Metrics.TIME_BUCKET).lte(endTB).gte(startTB));
boolQueryBuilder.must().add(QueryBuilders.termQuery(EndpointInventory.SERVICE_ID,
serviceId));
Review comment:
> However, it should be sure at present that most of the ES queries in SW
should hit yes or no.
Yes. The score is not important for us. We use ES like normal storage,
rather than a search engine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4292: Improve ES query performance
wu-sheng commented on a change in pull request #4292: Improve ES query
performance
URL: https://github.com/apache/skywalking/pull/4292#discussion_r372794606
##
File path:
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/query/AggregationQueryEsDAO.java
##
@@ -96,7 +96,7 @@ public AggregationQueryEsDAO(ElasticSearchClient client) {
SearchSourceBuilder sourceBuilder = SearchSourceBuilder.searchSource();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
-sourceBuilder.query(boolQueryBuilder);
+
sourceBuilder.query(QueryBuilders.boolQuery().filter(boolQueryBuilder));
boolQueryBuilder.must().add(QueryBuilders.rangeQuery(Metrics.TIME_BUCKET).lte(endTB).gte(startTB));
boolQueryBuilder.must().add(QueryBuilders.termQuery(EndpointInventory.SERVICE_ID,
serviceId));
Review comment:
Thanks for providing this. Still, the document about `as well as cache
filters for faster subsequent performance` is not very clear. My guess, at our
implementation, you moved all the conditions into the filter, then this part of
optimization should be none(no subsequent), except for aggregation top N query.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4292: Improve ES query performance
wu-sheng commented on a change in pull request #4292: Improve ES query performance URL: https://github.com/apache/skywalking/pull/4292#discussion_r372794082 ## File path: oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/base/EsDAO.java ## @@ -53,7 +53,7 @@ protected final void queryBuild(SearchSourceBuilder sourceBuilder, Where where, boolQuery.must().add(QueryBuilders.termQuery(keyValues.getKey(), keyValues.getValues().get(0))); } }); -sourceBuilder.query(boolQuery); + sourceBuilder.query(QueryBuilders.boolQuery().filter(QueryBuilders.boolQuery().filter(boolQuery))); Review comment: Filter twice too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4292: Improve ES query performance
wu-sheng commented on a change in pull request #4292: Improve ES query
performance
URL: https://github.com/apache/skywalking/pull/4292#discussion_r372793949
##
File path:
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/base/EsDAO.java
##
@@ -41,7 +41,7 @@ public EsDAO(ElasticSearchClient client) {
protected final void queryBuild(SearchSourceBuilder sourceBuilder, Where
where, long startTB, long endTB) {
RangeQueryBuilder rangeQueryBuilder =
QueryBuilders.rangeQuery(Metrics.TIME_BUCKET).gte(startTB).lte(endTB);
if (where.getKeyValues().isEmpty()) {
-sourceBuilder.query(rangeQueryBuilder);
+
sourceBuilder.query(QueryBuilders.boolQuery().filter(QueryBuilders.boolQuery().filter(rangeQueryBuilder)));
Review comment:
Why filter twice?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [skywalking] wu-sheng commented on a change in pull request #4292: Improve ES query performance
wu-sheng commented on a change in pull request #4292: Improve ES query
performance
URL: https://github.com/apache/skywalking/pull/4292#discussion_r372736264
##
File path:
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/query/AggregationQueryEsDAO.java
##
@@ -96,7 +96,7 @@ public AggregationQueryEsDAO(ElasticSearchClient client) {
SearchSourceBuilder sourceBuilder = SearchSourceBuilder.searchSource();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
-sourceBuilder.query(boolQueryBuilder);
+
sourceBuilder.query(QueryBuilders.boolQuery().filter(boolQueryBuilder));
boolQueryBuilder.must().add(QueryBuilders.rangeQuery(Metrics.TIME_BUCKET).lte(endTB).gte(startTB));
boolQueryBuilder.must().add(QueryBuilders.termQuery(EndpointInventory.SERVICE_ID,
serviceId));
Review comment:
I am wondering, what is the meaning of cache? The data is updated every
several seconds, even for the metrics. What is the scope of cache? @dmsolr
@aderm Do you have any documentation to explain this? The official one is
better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
