[
https://issues.apache.org/jira/browse/SOLR-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-18571:
--------------------------------
Attachment: query-insights-design.pdf
> Query Insights: opt-in top-N expensive queries per node
> -------------------------------------------------------
>
> Key: SOLR-18571
> URL: https://issues.apache.org/jira/browse/SOLR-18571
> Project: Solr
> Issue Type: New Feature
> Components: metrics, search, v2 API
> Reporter: Amrit Sarkar
> Priority: Major
> Attachments: query-insights-design.pdf
>
>
> Umbrella for Query Insights: an opt-in module that keeps the most expensive
> searches on each node in memory and returns them through one v2 API across
> the cluster. It implements the idea in SOLR-11994.
> Discussed on dev@:
> [thread|https://lists.apache.org/thread/t7mft9dbntgtp8pjfh3zvomxh8zdobtd]. No
> objections came in on the thread.
> *Shape*
> * Off by default. Turned on by adding {{query-insights}} to {{SOLR_MODULES}}
> and setting {{SOLR_QUERYINSIGHTS_ENABLED=true}}. Without both, nothing loads
> and each search pays one {{isEmpty()}} check.
> * Records only on the coordinator. Top-N by latency and by CPU over
> clock-aligned windows; the current and previous window are returned.
> Defaults: N=10 (max 100), 5 minute windows.
> * Each record has collection, handler, params (redactable with
> {{params.hidden}}, capped at 8 KB), hits, shard count, user, trace id,
> latency, CPU, and the costly features the query used (re-ranking, LTR, knn,
> grouping, faceting, paramsets).
> * {{GET /api/node/query-insights/top-queries?type=latency|cpu&nodes=all}},
> needs {{metrics-read}}. Optional {{collection}} and {{traceId}} filters.
> Nodes that don't answer are listed in {{failedNodes}}.
> * Defaults from system properties; runtime changes through
> {{ext.queryInsights.*}} cluster properties.
> *Plan* (one PR each, each reviewable and mergeable on its own)
> # SOLR-11994: {{SearchRequestListener}} hook in core. No behavior change when
> nothing is registered.
> # The {{query-insights}} module: per-node top-N and the node-local API.
> # Cluster-wide view: {{nodes=all}} merge with {{failedNodes}}.
> # {{collection}} and {{traceId}} filters, runtime settings through cluster
> properties.
> # Ref guide page.
> *Defaults for the open questions from the thread* (happy to change any of
> these in review)
> * Module plus a small core listener, rather than all in core.
> * {{metrics-read}} permission, with {{params.hidden}} redaction.
> * {{SearchRequestListener}} marked {{@lucene.experimental}}.
> * {{/export}} is recorded; {{/get}} is not.
> *Later, separate sub-tasks*: stored history as a Time Routed Alias with
> {{autoDeleteAge}} (to be lined up with [UBI PR
> #2452|https://github.com/apache/solr/pull/2452]), grouping similar queries by
> shape, memory as a metric, an Admin UI page.
> Full design is attached as {{query-insights-design.pdf}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
