[
https://issues.apache.org/jira/browse/SOLR-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18130654#comment-18130654
]
Eric Pugh commented on SOLR-18571:
----------------------------------
in my world, I'd love to see it on by default... if it's too memory intensive
or otherwise intrusive to be on by default, well, then that kind of defeats the
prupose. I think it's great to be able to disable it, but we don't awant to
make everythign opt in because maybe someone doesn't know they need it till
well, they need it!
> Query Insights: opt-in top-N expensive queries per node
> -------------------------------------------------------
>
> Key: SOLR-18571
> URL: https://issues.apache.org/jira/browse/SOLR-18571
> Project: Solr
> Issue Type: New Feature
> Components: metrics, search, v2 API
> Reporter: Amrit Sarkar
> Assignee: Eric Pugh
> Priority: Major
> Attachments: query-insights-design.pdf
>
>
> Umbrella for Query Insights: an opt-in module that keeps the most expensive
> searches on each node in memory and returns them through one v2 API across
> the cluster. It implements the idea in SOLR-11994.
> Discussed on dev@:
> [thread|https://lists.apache.org/thread/t7mft9dbntgtp8pjfh3zvomxh8zdobtd]. No
> objections came in on the thread.
> *Shape*
> * Off by default. Turned on by adding {{query-insights}} to {{SOLR_MODULES}}
> and setting {{SOLR_QUERYINSIGHTS_ENABLED=true}}. Without both, nothing loads
> and each search pays one {{isEmpty()}} check.
> * Records only on the coordinator. Top-N by latency and by CPU over
> clock-aligned windows; the current and previous window are returned.
> Defaults: N=10 (max 100), 5 minute windows.
> * Each record has collection, handler, params (redactable with
> {{params.hidden}}, capped at 8 KB), hits, shard count, user, trace id,
> latency, CPU, and the costly features the query used (re-ranking, LTR, knn,
> grouping, faceting, paramsets).
> * {{GET /api/node/query-insights/top-queries?type=latency|cpu&nodes=all}},
> needs {{metrics-read}}. Optional {{collection}} and {{traceId}} filters.
> Nodes that don't answer are listed in {{failedNodes}}.
> * Defaults from system properties; runtime changes through
> {{ext.queryInsights.*}} cluster properties.
> *Plan* (one PR each, each reviewable and mergeable on its own)
> # SOLR-11994: {{SearchRequestListener}} hook in core. No behavior change when
> nothing is registered.
> # The {{query-insights}} module: per-node top-N and the node-local API.
> # Cluster-wide view: {{nodes=all}} merge with {{failedNodes}}.
> # {{collection}} and {{traceId}} filters, runtime settings through cluster
> properties.
> # Ref guide page.
> *Defaults for the open questions from the thread* (happy to change any of
> these in review)
> * Module plus a small core listener, rather than all in core.
> * {{metrics-read}} permission, with {{params.hidden}} redaction.
> * {{SearchRequestListener}} marked {{@lucene.experimental}}.
> * {{/export}} is recorded; {{/get}} is not.
> *Later, separate sub-tasks*: stored history as a Time Routed Alias with
> {{autoDeleteAge}} (to be lined up with [UBI PR
> #2452|https://github.com/apache/solr/pull/2452]), grouping similar queries by
> shape, memory as a metric, an Admin UI page.
> Full design is attached as {{query-insights-design.pdf}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
