This is an automated email from the ASF dual-hosted git repository. HTHou pushed a commit to branch codex/prometheus in repository https://gitbox.apache.org/repos/asf/iotdb.git
commit f4d6d374582869578052a7cb27dc491234c771aa Author: HTHou <[email protected]> AuthorDate: Mon Jun 15 18:14:00 2026 +0800 Add metric scrape service readme --- external-service-impl/metric-scrape/README.md | 99 ++++++++++++++++++++++++ external-service-impl/metric-scrape/README_ZH.md | 99 ++++++++++++++++++++++++ 2 files changed, 198 insertions(+) diff --git a/external-service-impl/metric-scrape/README.md b/external-service-impl/metric-scrape/README.md new file mode 100644 index 00000000000..ea0f2f7ce9e --- /dev/null +++ b/external-service-impl/metric-scrape/README.md @@ -0,0 +1,99 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> + +# Metric Scrape Service + +The Metric Scrape Service actively scrapes Prometheus text exposition endpoints and writes the samples into IoTDB through the table model. It is a pull-based scrape service inside IoTDB, not a Prometheus remote write receiver. + +## Configuration + +Configure the service in `iotdb-system.properties` and restart IoTDB for the changes to take effect. + +```properties +enable_metric_scrape_service=true +metric_scrape_targets=http://127.0.0.1:9091/metrics,http://127.0.0.1:9092/metrics +metric_scrape_database=confignode,datanode +metric_scrape_interval_seconds=15 +metric_scrape_http_timeout_ms=10000 +``` + +| Property | Default | Description | +| --- | --- | --- | +| `enable_metric_scrape_service` | `false` | Whether to enable the metric scrape service. | +| `metric_scrape_targets` | empty | Comma-separated HTTP or HTTPS target URLs that expose Prometheus text metrics. | +| `metric_scrape_database` | `metrics` | Comma-separated table model database names. The count and order must match `metric_scrape_targets`. | +| `metric_scrape_interval_seconds` | `15` | Scrape interval for each target, in seconds. | +| `metric_scrape_http_timeout_ms` | `10000` | HTTP connect and read timeout, in milliseconds. | + +When multiple targets are configured, `metric_scrape_database` works like Prometheus job separation. For example, the first target above is written to database `confignode`, and the second target is written to database `datanode`. + +## Data Model + +The service uses IoTDB table model writes and lets IoTDB create schemas automatically. + +| Prometheus text item | IoTDB table model mapping | +| --- | --- | +| Target URL | The database configured at the same position in `metric_scrape_database`. | +| Metric family name from `# HELP` | Table name. | +| Sample name | Field column name. | +| Sample value | Field column value, stored as `DOUBLE`. | +| Sample labels | Tag columns, stored as `STRING`. | +| Sample timestamp | Row time. If the sample has no timestamp, the current IoTDB time is used. | + +For histogram, summary, and other suffixed samples, the service chooses the longest matching `# HELP` metric family name as the table name. For example, `request_duration_seconds_sum` and `request_duration_seconds_count` are written to table `request_duration_seconds` when that family appears in a `# HELP` line. + +## Example + +Prometheus text: + +```text +# HELP request_duration_seconds Request duration in seconds. +# TYPE request_duration_seconds summary +request_duration_seconds_sum{method="query",status="ok"} 10.5 +request_duration_seconds_count{method="query",status="ok"} 3 +``` + +With this configuration: + +```properties +enable_metric_scrape_service=true +metric_scrape_targets=http://127.0.0.1:9091/metrics +metric_scrape_database=metrics +``` + +IoTDB writes rows into database `metrics`, table `request_duration_seconds`. The labels `method` and `status` are tag columns, and `request_duration_seconds_sum` and `request_duration_seconds_count` are field columns. + +Query the scraped data: + +```sql +USE metrics; + +SELECT time, method, status, request_duration_seconds_sum, request_duration_seconds_count +FROM request_duration_seconds +ORDER BY time DESC +LIMIT 10; +``` + +## Notes + +- The target response must use the Prometheus text exposition sample format. +- Each target is scraped periodically by IoTDB. Prometheus does not need to push data to IoTDB. +- `metric_scrape_database` must contain exactly one database name for each configured target when `metric_scrape_targets` is not empty. diff --git a/external-service-impl/metric-scrape/README_ZH.md b/external-service-impl/metric-scrape/README_ZH.md new file mode 100644 index 00000000000..8daf385f34c --- /dev/null +++ b/external-service-impl/metric-scrape/README_ZH.md @@ -0,0 +1,99 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> + +# Metric Scrape Service + +Metric Scrape Service 会主动拉取 Prometheus text exposition 端点，并通过 IoTDB 表模型写入样本数据。它是 IoTDB 内部的 pull scrape 服务，不是 Prometheus remote write 接收端。 + +## 配置 + +在 `iotdb-system.properties` 中配置该服务，修改后需要重启 IoTDB 才能生效。 + +```properties +enable_metric_scrape_service=true +metric_scrape_targets=http://127.0.0.1:9091/metrics,http://127.0.0.1:9092/metrics +metric_scrape_database=confignode,datanode +metric_scrape_interval_seconds=15 +metric_scrape_http_timeout_ms=10000 +``` + +| 参数 | 默认值 | 说明 | +| --- | --- | --- | +| `enable_metric_scrape_service` | `false` | 是否启用 metric scrape 服务。 | +| `metric_scrape_targets` | 空 | 以逗号分隔的 HTTP 或 HTTPS 目标地址，目标地址需要暴露 Prometheus text metrics。 | +| `metric_scrape_database` | `metrics` | 以逗号分隔的表模型数据库名。数量和顺序必须与 `metric_scrape_targets` 一致。 | +| `metric_scrape_interval_seconds` | `15` | 每个目标的拉取间隔，单位为秒。 | +| `metric_scrape_http_timeout_ms` | `10000` | HTTP 连接和读取超时时间，单位为毫秒。 | + +配置多个目标时，`metric_scrape_database` 的作用类似 Prometheus 中按 job 区分数据。例如上面的第一个目标会写入数据库 `confignode`，第二个目标会写入数据库 `datanode`。 + +## 数据建模 + +该服务使用 IoTDB 表模型写入，并由 IoTDB 自动创建 schema。 + +| Prometheus text 内容 | IoTDB 表模型映射 | +| --- | --- | +| 目标地址 | `metric_scrape_database` 中相同位置配置的数据库。 | +| `# HELP` 中的 metric family 名称 | 表名。 | +| Sample 名称 | Field 列名。 | +| Sample 值 | Field 列值，类型为 `DOUBLE`。 | +| Sample labels | Tag 列，类型为 `STRING`。 | +| Sample timestamp | 行时间。如果 sample 中没有 timestamp，则使用当前 IoTDB 时间。 | + +对于 histogram、summary 等带后缀的 sample，服务会选择最长匹配的 `# HELP` metric family 名称作为表名。例如存在 `# HELP request_duration_seconds ...` 时，`request_duration_seconds_sum` 和 `request_duration_seconds_count` 都会写入表 `request_duration_seconds`。 + +## 示例 + +Prometheus text: + +```text +# HELP request_duration_seconds Request duration in seconds. +# TYPE request_duration_seconds summary +request_duration_seconds_sum{method="query",status="ok"} 10.5 +request_duration_seconds_count{method="query",status="ok"} 3 +``` + +使用如下配置： + +```properties +enable_metric_scrape_service=true +metric_scrape_targets=http://127.0.0.1:9091/metrics +metric_scrape_database=metrics +``` + +IoTDB 会将数据写入数据库 `metrics`、表 `request_duration_seconds`。其中 labels `method` 和 `status` 是 tag 列，`request_duration_seconds_sum` 和 `request_duration_seconds_count` 是 field 列。 + +查询拉取到的数据： + +```sql +USE metrics; + +SELECT time, method, status, request_duration_seconds_sum, request_duration_seconds_count +FROM request_duration_seconds +ORDER BY time DESC +LIMIT 10; +``` + +## 注意事项 + +- 目标端点返回内容需要符合 Prometheus text exposition sample 格式。 +- IoTDB 会周期性主动拉取目标端点，Prometheus 不需要向 IoTDB 推送数据。 +- 当 `metric_scrape_targets` 非空时，`metric_scrape_database` 必须为每个目标配置一个对应数据库名。
