gerlowskija commented on code in PR #4057:
URL: https://github.com/apache/solr/pull/4057#discussion_r2708080791
##########
solr/core/src/java/org/apache/solr/jersey/JerseyApplications.java:
##########
@@ -48,6 +48,9 @@ public CoreContainerApp() {
register(MessageBodyWriters.XmlMessageBodyWriter.class, 5);
register(MessageBodyWriters.CsvMessageBodyWriter.class, 5);
register(MessageBodyWriters.RawMessageBodyWriter.class, 5);
+ // Not sure if required. Ref.
org.apache.solr.handler.admin.api.GetMetrics
Review Comment:
[+1] This is definitely correct and needed; feel free to remove this comment
and the others like it in MessageBodyWriters.java
##########
solr/core/src/java/org/apache/solr/metrics/MetricsUtil.java:
##########
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.metrics;
+
+import io.prometheus.metrics.model.snapshots.CounterSnapshot;
+import io.prometheus.metrics.model.snapshots.GaugeSnapshot;
+import io.prometheus.metrics.model.snapshots.HistogramSnapshot;
+import io.prometheus.metrics.model.snapshots.InfoSnapshot;
+import io.prometheus.metrics.model.snapshots.MetricSnapshot;
+import io.prometheus.metrics.model.snapshots.MetricSnapshots;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.StrUtils;
+
+/** Utility methods for Metrics */
+public class MetricsUtil {
Review Comment:
[Q] I'm doing a good bit of skimming over this class in my review, as
(afaict) it's mostly code that preexists your PR and that is only being moved
from MetricsHandler.
Is that correct? Are there particular changes or sections of this file that
I _should_ pay closer attention to?
##########
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler.admin.api;
+
+import jakarta.ws.rs.core.StreamingOutput;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.util.Map;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.ModifiableSolrParams;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.metrics.MetricsUtil;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.request.SolrQueryRequestBase;
+import org.apache.solr.response.SolrQueryResponse;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class GetMetricsTest extends SolrTestCaseJ4 {
+
+ private static CoreContainer cc;
+ private static Path outputPath;
+
+ @BeforeClass
+ public static void beforeClass() throws Exception {
+ initCore("solrconfig-minimal.xml", "schema.xml");
+ h.getCoreContainer().waitForLoadingCoresToFinish(30000);
+ cc = h.getCoreContainer();
+ outputPath = createTempDir();
+ }
+
+ @AfterClass
+ public static void afterClass() throws Exception {
+ SolrTestCaseJ4.teardownTestCases();
+ }
+
+ @Test
+ public void testGetMetricsDefault() throws IOException {
+ String expectedHelp = "# HELP solr_core_disk_space_megabytes";
+ String expectedType = "# TYPE solr_core_disk_space_megabytes";
+ String expectedDiskSpaceMetric =
+ """
+
solr_core_disk_space_megabytes{category="CORE",core="collection1",otel_scope_name="org.apache.solr",type="total_space"}
+ """;
+ SolrQueryRequest req = new SolrQueryRequestBase(h.getCore(), new
ModifiableSolrParams()) {};
+ SolrQueryResponse resp = new SolrQueryResponse();
+
+ GetMetrics get = new GetMetrics(cc, req, resp);
+ StreamingOutput output = get.getMetrics();
+ Assert.assertNotNull(output);
+
+ Path tmpFile = Files.createTempFile(outputPath, "test-",
"-GetMetricsDefault");
+ try (OutputStream tmpOut =
+ Files.newOutputStream(
+ tmpFile,
+ StandardOpenOption.CREATE,
+ StandardOpenOption.TRUNCATE_EXISTING,
+ StandardOpenOption.WRITE)) {
+ output.write(tmpOut);
+ }
+ try (InputStream tmpIn = Files.newInputStream(tmpFile,
StandardOpenOption.READ)) {
Review Comment:
[-1] Ah, it looks like your main goal in creating this temp file is as a
temporary buffer for getting a String version of the StreamingOutput?
If that's the case, then you should be able to avoid the temp file
altogether:
```
StreamingOutput output = stringifyStreamingResponse(get.getMetrics());
...
private String stringifyStreamingResponse(StreamingOutput output) {
final var baos = new ByteArrayOutputStream();
output.write(baos);
return new String(baos.toByteArray(), StandardCharsets.UTF_8);
}
```
##########
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc:
##########
@@ -256,26 +261,26 @@ The replica type to filter on. Valid values are NRT,
TLOG, or PULL. This attribu
Request only metrics from the `foobar` collection:
[source,text]
-http://localhost:8983/solr/admin/metrics?collection=foobar
+http://localhost:8983/api/metrics?collection=foobar
Review Comment:
[-0] Hmm, a lot of these little examples reference query parameters that I
didn't notice when reviewing `GetMetrics` above: "collection", "category",
"core", etc.
Assuming I just overlooked those and they **are** valid inputs to the v2
API, then they should probably each get a method parameter in the
`MetricsApi.getMetrics(...)` signature. See [this comment
above](https://github.com/apache/solr/pull/4057/files#r2705583523) where I
described this in a bit more detail for the metricNames param
##########
solr/solrj/src/java/org/apache/solr/client/solrj/request/MetricsRequest.java:
##########
@@ -17,15 +17,15 @@
package org.apache.solr.client.solrj.request;
import org.apache.solr.client.solrj.SolrRequest;
-import org.apache.solr.client.solrj.SolrResponse;
-import org.apache.solr.client.solrj.response.SolrResponseBase;
+import org.apache.solr.client.solrj.response.InputStreamResponse;
+import org.apache.solr.client.solrj.response.InputStreamResponseParser;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.common.params.SolrParams;
import org.apache.solr.common.util.NamedList;
/** Request to "/admin/metrics" */
-public class MetricsRequest extends SolrRequest<SolrResponse> {
+public class MetricsRequest extends SolrRequest<InputStreamResponse> {
Review Comment:
[Q] Why change the SolrJ class that corresponds to the v1 metrics API? Your
PR only modifies the v2 API, right?
##########
solr/core/src/java/org/apache/solr/response/PrometheusResponseWriter.java:
##########
@@ -64,6 +62,12 @@ public void write(
throw new IOException("No metrics found in response");
}
MetricSnapshots snapshots = (MetricSnapshots) metrics;
+ writeMetricSnapshots(out, request, snapshots);
+ }
+
+ /** Write MetricSnapshots in Prometheus or OpenMertics format */
Review Comment:
[0] Typo: "OpenMertics" -> "OpenMetrics"
##########
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc:
##########
@@ -132,7 +132,12 @@ Metrics collection for index merges can be configured in
the `<metrics>` section
== Metrics API
-The `/admin/metrics` endpoint natively provides access to all metrics in
Prometheus format by default. You can also specify `wt=prometheus` as a
parameter for Prometheus format or `wt=openmetrics` for OpenMetrics format.
More information on the data models is provided in the sections below.
+The `/metrics` endpoint natively provides access to all metrics in Prometheus
format by default. You can also specify `wt=prometheus` as a parameter for
Prometheus format or `wt=openmetrics` for OpenMetrics format. More information
on the data models is provided in the sections below.
+
+[NOTE]
+====
+The V1 `/admin/metrics` endpoint is deprecated and will be removed in a future
version.
Review Comment:
[-1] I wish : (
I hope to change this soon but for now our v2 API is still "experimental".
We can't deprecate any of the v1 functionality until v2 has graduated from that
designation. I'd drop this note in the docs.
##########
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc:
##########
@@ -148,14 +153,14 @@ The `prometheus-config.yml` file needs to be configured
for a Prometheus server
----
scrape_configs:
- job_name: 'solr'
- metrics_path: "/solr/admin/metrics"
+ metrics_path: "/api/metrics"
static_configs:
- targets: ['localhost:8983', 'localhost:7574']
----
=== OpenMetrics
-OpenMetrics format is available from the `/admin/metrics` endpoint by
providing the `wt=openmetrics` parameter or by passing the Accept header
`application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of
the Prometheus format that adds additional metadata and exemplars.
+OpenMetrics format is available from the `/metrics` endpoint by providing the
`wt=openmetrics` parameter or by passing the Accept header
`application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of
the Prometheus format that adds additional metadata and exemplars.
Review Comment:
[Q] Huh - I'm surprised that these docs mention the "Accept" header even
before you added a v2 endpoint in this PR. I didn't think anything on the v1
side looked at "Accept". I guess that's a good thing, I'm just surprised is
all...
##########
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler.admin.api;
+
+import jakarta.ws.rs.core.StreamingOutput;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.util.Map;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.ModifiableSolrParams;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.metrics.MetricsUtil;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.request.SolrQueryRequestBase;
+import org.apache.solr.response.SolrQueryResponse;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class GetMetricsTest extends SolrTestCaseJ4 {
+
+ private static CoreContainer cc;
+ private static Path outputPath;
+
+ @BeforeClass
+ public static void beforeClass() throws Exception {
+ initCore("solrconfig-minimal.xml", "schema.xml");
+ h.getCoreContainer().waitForLoadingCoresToFinish(30000);
+ cc = h.getCoreContainer();
+ outputPath = createTempDir();
+ }
+
+ @AfterClass
+ public static void afterClass() throws Exception {
+ SolrTestCaseJ4.teardownTestCases();
+ }
+
+ @Test
+ public void testGetMetricsDefault() throws IOException {
+ String expectedHelp = "# HELP solr_core_disk_space_megabytes";
+ String expectedType = "# TYPE solr_core_disk_space_megabytes";
+ String expectedDiskSpaceMetric =
+ """
+
solr_core_disk_space_megabytes{category="CORE",core="collection1",otel_scope_name="org.apache.solr",type="total_space"}
+ """;
+ SolrQueryRequest req = new SolrQueryRequestBase(h.getCore(), new
ModifiableSolrParams()) {};
+ SolrQueryResponse resp = new SolrQueryResponse();
+
+ GetMetrics get = new GetMetrics(cc, req, resp);
+ StreamingOutput output = get.getMetrics();
+ Assert.assertNotNull(output);
+
+ Path tmpFile = Files.createTempFile(outputPath, "test-",
"-GetMetricsDefault");
Review Comment:
[-1] SolrTestCaseJ4 inherits "createTempDir" and "createTempFile" methods
that will track the files you've created in your test and clean them up for you
at the end. Would you mind using those for file creation instead? They tend
to be a little safer as they'll make sure the file is still cleaned up even if
an exception is thrown in your test method, an assertion fails, etc.
##########
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler.admin.api;
+
+import jakarta.ws.rs.core.StreamingOutput;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardOpenOption;
+import java.util.Map;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.ModifiableSolrParams;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.metrics.MetricsUtil;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.request.SolrQueryRequestBase;
+import org.apache.solr.response.SolrQueryResponse;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class GetMetricsTest extends SolrTestCaseJ4 {
+
+ private static CoreContainer cc;
+ private static Path outputPath;
+
+ @BeforeClass
+ public static void beforeClass() throws Exception {
+ initCore("solrconfig-minimal.xml", "schema.xml");
+ h.getCoreContainer().waitForLoadingCoresToFinish(30000);
+ cc = h.getCoreContainer();
+ outputPath = createTempDir();
+ }
+
+ @AfterClass
+ public static void afterClass() throws Exception {
+ SolrTestCaseJ4.teardownTestCases();
+ }
+
+ @Test
+ public void testGetMetricsDefault() throws IOException {
+ String expectedHelp = "# HELP solr_core_disk_space_megabytes";
+ String expectedType = "# TYPE solr_core_disk_space_megabytes";
+ String expectedDiskSpaceMetric =
+ """
+
solr_core_disk_space_megabytes{category="CORE",core="collection1",otel_scope_name="org.apache.solr",type="total_space"}
+ """;
+ SolrQueryRequest req = new SolrQueryRequestBase(h.getCore(), new
ModifiableSolrParams()) {};
+ SolrQueryResponse resp = new SolrQueryResponse();
+
+ GetMetrics get = new GetMetrics(cc, req, resp);
Review Comment:
[-0] Is there a reason you had this test instantiate the "GetMetrics" class
and invoke it directly, as opposed to, say, having the test start a Solr node
that we can send an HTTP request to?
Most of our tests take that latter approach. It's not always the right one,
but it has the benefit of testing a wider chunk of functionality. For example,
as-is this test doesn't really validate that the new v2 API looks the way we'd
expect it to (available at the expected HTTP method and path, takes the
query-params that we expect, etc.). It doesn't validate any the SolrJ classes
generated from the annotations in `MetricsApi`, etc.
I don't think we have to switch necessarily, but if we're going the less
common route there should probably be a reason or some other plan for testing
those other aspects.
##########
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc:
##########
@@ -148,14 +153,14 @@ The `prometheus-config.yml` file needs to be configured
for a Prometheus server
----
scrape_configs:
- job_name: 'solr'
- metrics_path: "/solr/admin/metrics"
+ metrics_path: "/api/metrics"
Review Comment:
[0] I'm fine with switching the "path" in these examples to "/api/metrics"
as you've done here, but it might be nice to have a little `[NOTE]` or some
similar blurb that explains that the two paths "/api/metrics" and
"/solr/admin/metrics" are synonymous, in case any users are confused?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
