liangkaiwen commented on code in PR #3682:
URL: https://github.com/apache/solr/pull/3682#discussion_r2377190887
##########
solr/core/src/java/org/apache/solr/update/SolrIndexWriter.java:
##########
@@ -387,12 +239,192 @@ public Map<String, Object> getRunningMerges() {
@Override
protected void doAfterFlush() throws IOException {
- if (flushes != null) { // this is null when writer is used only for
snapshot cleanup
- flushes.inc(); // or if mergeTotals == false
+ if (flushesCounter != null) { // this is null when writer is used only for
snapshot cleanup
+ flushesCounter.inc(); // or if mergeTotals == false
}
super.doAfterFlush();
}
+ private void initMetrics(final SolrCore core) {
+ if (solrMetricsContext == null) {
+ solrMetricsContext = core.getSolrMetricsContext().getChildContext(this);
+ }
+ var coreName = core.getName();
+ var baseAttributesBuilder =
+ Attributes.builder()
+ .put(CATEGORY_ATTR, SolrInfoBean.Category.INDEX.toString())
+ .put(CORE_ATTR, coreName);
+ if (core.getCoreContainer().isZooKeeperAware()) {
+ String collectionName = core.getCoreDescriptor().getCollectionName();
+ baseAttributesBuilder
+ .put(COLLECTION_ATTR, collectionName)
+ .put(SHARD_ATTR,
core.getCoreDescriptor().getCloudDescriptor().getShardId())
+ .put(REPLICA_ATTR, Utils.parseMetricsReplicaName(collectionName,
coreName));
+ }
+ var baseAttributes = baseAttributesBuilder.build();
+
+ var mergeTimerBaseMetric =
+ solrMetricsContext.longHistogram(
+ "solr_indexwriter_merge_time", "Time spent merging segments",
OtelUnit.MILLISECONDS);
+
+ majorMergeTimer =
+ new AttributedLongTimer(
+ mergeTimerBaseMetric,
baseAttributes.toBuilder().put(MERGE_TYPE_ATTR, "major").build());
+ minorMergeTimer =
+ new AttributedLongTimer(
+ mergeTimerBaseMetric,
baseAttributes.toBuilder().put(MERGE_TYPE_ATTR, "minor").build());
+
+ mergeErrorsCounter =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_indexwriter_merge_errors", "Number of merge errors"),
+ baseAttributes);
+
+ flushesCounter =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_indexwriter_flushes", "Number of times documents have
been flushed to disk"),
+ baseAttributes);
+
+ var mergesCountBaseMetric =
+ solrMetricsContext.longCounter("solr_indexwriter_merges", "Number of
merge operations");
+ var docsMergedCountBaseMetric =
+ solrMetricsContext.longCounter(
+ "solr_indexwriter_docs_merged", "Number of documents involved in
merge");
+ var docsDeletedCountBasedMetric =
+ solrMetricsContext.longCounter(
+ "solr_indexwriter_docs_deleted", "Number of documents deleted in
merge");
Review Comment:
@dsmiley I hear what you're saying, but I think this change is primarily to
understand the behavior in the context of merges. Solr already has document
count metrics, so create / delete are already discernable (to some degree). I
think it would be a little murky to include document "creation" as now we need
to consider document being created in-memory by soft-commit?
I'm not hard opposed to restructuring some of these metrics, but would like
to better nail down the scope before doing so. The way I see it is this change
is closing a gap on visibility into 1) when are merges happening (as they are
expensive operations) and 2) what are the characteristics of those operations
Would love to chat with you and pick your brain further before I go and
re-write these metrics
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
