aderm opened a new issue #4291: Improve ES query performance URL: https://github.com/apache/skywalking/issues/4291 Please answer these questions before submitting your issue. - Why do you submit this issue? - [ ] Question or discussion - [ ] Bug - [ ] Requirement - [x] Feature or performance improvement ___ ### Question - What do you want to know? ___ ### Bug - Which version of SkyWalking, OS and JRE? - Which company or project? - What happen? If possible, provide a way for reproducing the error. e.g. demo application, component version. ___ ### Requirement or improvement - Please describe about your requirements or improvement suggestions. We can use the 'qool bool filter' statement to replace the normal query of 'qool bool'. There are two benefits: 1.No scoring calculation is needed to speed up the calculation; 2. Filter query can make full use of the cache and improve the time of repeated sub-query Take the query interface `MetadataQueryEsDAO#getAllServices` as an example： - ES query dsl： `{"size":5000,"query":{"bool":{"must":[{"bool":{"should":[{"bool":{"must":[{"range":{"heartbeat_time":{"from":1580292720000,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}},{"range":{"register_time":{"from":null,"to":1580292720000,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},{"bool":{"must":[{"range":{"register_time":{"from":null,"to":1580292720000,"include_lower":true,"include_upper":true,"boost":1.0}}},{"range":{"heartbeat_time":{"from":1580291760000,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}},{"term":{"is_address":{"value":0,"boost":1.0}}},{"term":{"node_type":{"value":0,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}}` - and result： `{"took":0,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},**"max_score":6.0**,"hits":[{"_index":"service_inventory","_type":"_doc","_id":"Your_ApplicationName_0_0",**"_score":6.0**,"_source":{"sequence":2,"last_update_time":1580287478790,"heartbeat_time":1580292752350,"node_type":0,"name":"Your_ApplicationName","address_id":0,"is_address":0,"mapping_service_id":0,"register_time":1580287478790,"properties":"{}"}}]}}` In the results, we see that the search engine scored：`max_score:6.0`. When improvements are made: - ES query dsl： `{"size":5000,"query":{"bool":{"**filter**":[{"bool":{"must":[{"bool":{"should":[{"bool":{"must":[{"range":{"heartbeat_time":{"from":1580293080000,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}},{"range":{"register_time":{"from":null,"to":1580293080000,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},{"bool":{"must":[{"range":{"register_time":{"from":null,"to":1580293080000,"include_lower":true,"include_upper":true,"boost":1.0}}},{"range":{"heartbeat_time":{"from":1580292120000,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}},{"term":{"is_address":{"value":0,"boost":1.0}}},{"term":{"node_type":{"value":0,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}}` - and result： `{"took":0,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},**"max_score":0.0**,"hits":[{"_index":"service_inventory","_type":"_doc","_id":"Your_ApplicationName_0_0",**"_score":0.0**,"_source":{"sequence":2,"last_update_time":1580287478790,"heartbeat_time":1580293072125,"node_type":0,"name":"Your_ApplicationName","address_id":0,"is_address":0,"mapping_service_id":0,"register_time":1580287478790,"properties":"{}"}}]}}` As you can see from the results, no score is calculated, this is what we need. **max_score":0.0**
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
