This is an automated email from the ASF dual-hosted git repository.
CritasWang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb-extras.git
The following commit(s) were added to refs/heads/master by this push:
new de991c4 Add the user and migration guides and correct two stale API
descriptions (#119)
de991c4 is described below
commit de991c4ec8e7230ccece2fc15c033dc541adfda1
Author: ZIHAN DAI <[email protected]>
AuthorDate: Mon Aug 3 12:02:43 2026 +1000
Add the user and migration guides and correct two stale API descriptions
(#119)
Add docs/user-guide.md and docs/migration-guide.md for the IoTDB Table Mode
ThingsBoard backend: how each of the three storage paths is activated, the
configuration reference, schema and bootstrap behaviour, retention, and what
a migration from an existing ThingsBoard backend does and does not carry
over.
Writing them surfaced two descriptions that no longer match the code.
The experimentalRawOnly javadoc still said time-bucketed aggregation was
outside the module's scope. Aggregation has been implemented since the
date_bin and calendar-bucket read paths landed, so the flag now describes
what
it actually gates, why the name predates that support, and why it is still
marked experimental.
The user guide's unsupported-operations section said the batch latest read
throws UnsupportedOperationException. It does not: findLatestByEntityIds and
its async twin return empty, deliberately, because they back the
includeSamples
branch of the entity-query keys endpoint where a synchronous throw would
surface as an HTTP 500 -- the same choice CassandraBaseTimeseriesLatestDao
makes. findNextBatch is the only method in the module that throws, and that
half of the sentence was correct.
Also drops a comment reference to a document outside this repository from
IoTDBTableTimeseriesDao.
Signed-off-by: Zihan Dai <[email protected]>
---
iotdb-thingsboard-table/docs/migration-guide.md | 392 +++++++++++++++++++++
iotdb-thingsboard-table/docs/user-guide.md | 315 +++++++++++++++++
.../extras/thingsboard/table/IoTDBTableConfig.java | 10 +-
.../thingsboard/table/IoTDBTableTimeseriesDao.java | 2 +-
4 files changed, 715 insertions(+), 4 deletions(-)
diff --git a/iotdb-thingsboard-table/docs/migration-guide.md
b/iotdb-thingsboard-table/docs/migration-guide.md
new file mode 100644
index 0000000..344ed91
--- /dev/null
+++ b/iotdb-thingsboard-table/docs/migration-guide.md
@@ -0,0 +1,392 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# Migration Guide: ThingsBoard storage on IoTDB Table Mode
+
+This guide is for an operator who wants to move a ThingsBoard deployment's
+telemetry, latest-value, and (optionally) entity-attribute storage onto Apache
+IoTDB 2.x Table Mode using the `iotdb-thingsboard-table` module. It covers
+version pairing, step-by-step enablement, the ThingsBoard-to-IoTDB data-model
+mapping, coexistence with the existing storage backend, rollback, and the
+limitations to plan for.
+
+For configuration reference and the full limitation list see the module
+[`README.md`](../README.md); for day-to-day operation see
+[`docs/user-guide.md`](user-guide.md). This guide focuses on the one-time
+cut-over.
+
+## Audience and scope
+
+The module is a ThingsBoard DAO backend built on IoTDB's table-session API. It
+adds three independently activated routes:
+
+- **historical telemetry** (`TimeseriesDao`) → the `telemetry` table;
+- **latest value** (`TimeseriesLatestDao`) → derived from `telemetry` plus a
+ per-key `telemetry_latest` overlay table;
+- **entity attributes** (`AttributesDao`, opt-in) → the `entity_attributes`
+ table.
+
+**The module is additive and default-inert.** Putting the jar on the
ThingsBoard
+classpath changes nothing on its own: every route is gated behind an explicit
+activation selector, and with no selector set the activation conditions all
+return `false`, no IoTDB bean or session pool is created, and ThingsBoard keeps
+using its existing storage backend. Enabling a route is a deliberate
+configuration step, described below.
+
+The timeseries route implements time-bucketed aggregation over the IoTDB
+`telemetry` table — both the fixed-width millisecond `date_bin` path and the
+timezone-aware calendar buckets (`WEEK` / `WEEK_ISO` / `MONTH` / `QUARTER`) —
in
+addition to raw read, write, and delete. Aggregation is therefore **not** a
+reason to keep the host backend for the telemetry route.
+
+## Version pairing
+
+Pin these versions so the module, the IoTDB client, and the IoTDB server line
+up. The module overrides **no** shared reactor versions; the IoTDB client and
+TsFile versions are inherited from the `iotdb-extras` parent reactor.
+
+| Component | Coordinate / image | Version | Where it is fixed |
+| --- | --- | --- | --- |
+| Module | `org.apache.iotdb:iotdb-thingsboard-table` | `2.0.4-SNAPSHOT`
(parent version) | module `pom.xml` `<parent>` |
+| Parent reactor | `org.apache.iotdb:iotdb-extras-parent` | `2.0.4-SNAPSHOT` |
module `pom.xml` `<parent>` |
+| IoTDB session client | `org.apache.iotdb:iotdb-session` | `2.0.5` |
inherited from parent `iotdb.version`; module declares the dependency with no
`<version>` |
+| TsFile | `org.apache.tsfile:tsfile` | `2.1.1` | inherited from parent
`tsfile.version` (transitive of `iotdb-session`) |
+| Guava | `com.google.guava:guava` | `32.1.2-jre` | inherited from parent
`guava.version`; `provided` scope |
+| Bean-validation API | `jakarta.validation:jakarta.validation-api` | `3.0.2`
| module override (`jakarta.* ` namespace, `provided` scope) to match the
Spring Boot 3 runtime host |
+| ThingsBoard host | (ThingsBoard distribution) | `4.3.1.2` | module `pom.xml`
`thingsboard.version`; SPI surface verified against this tag |
+| IoTDB server (tested) | `apache/iotdb` standalone image | `2.0.8` |
integration tests run against `apache/iotdb:2.0.8-standalone` |
+
+Notes:
+
+- The module compiles against the **2.0.5** table-session client; its
+ integration tests exercise the real write path against an **IoTDB 2.0.8**
+ standalone server, so the 2.0.5-client / 2.0.8-server RPC path is the
+ validated pairing. Any IoTDB 2.x server that speaks the same table-session
RPC
+ is a candidate, but 2.0.8 is the version the module is tested against.
+- The build parent itself targets an older Spring line; the module deploys into
+ ThingsBoard 4.3.x, which runs Spring Boot 3.5.x / Spring 6 / JDK 17 (the
+ `jakarta.*` namespace). That is why `jakarta.validation-api` is overridden to
+ `3.0.2` and why the module requires **JDK 17+** to build and run. The
+ Hibernate Validator provider is supplied by the ThingsBoard runtime
classpath,
+ not bundled by the module.
+- If you cannot determine a version from this table, read it from the module
+ `pom.xml` and the `iotdb-extras` parent `pom.xml` rather than assuming a
value.
+
+## Before you begin
+
+- A reachable IoTDB 2.x server in **Table Mode** (the tested server is IoTDB
+ 2.0.8). Have its host, port, and credentials ready.
+- A ThingsBoard 4.3.1.2 build you control (the module is consumed as a
+ compile/runtime dependency of the ThingsBoard application; it is not a
+ drop-in for a pre-built binary distribution).
+- JDK 17+ for the ThingsBoard build that includes this module.
+- Decide which routes you are enabling: telemetry only, telemetry + latest, and
+ whether to opt in to the stretch attributes route.
+
+## Activation selectors
+
+Each route is controlled by its own selector. Until a route's selector is set,
+that route stays inert.
+
+| Route | Required properties | Default state |
+| --- | --- | --- |
+| Historical telemetry (raw read/write/delete) |
`database.ts.type=iotdb-table` **and** `iotdb.ts.experimental-raw-only=true` |
inert |
+| Latest value | `database.ts.type=iotdb-table` **and**
`database.ts_latest.type=iotdb-table` **and**
`iotdb.ts.experimental-raw-only=true` | inert |
+| Entity attributes (stretch) | `database.attributes.type=iotdb-table`
(independent of the timeseries selectors); also requires
`iotdb.attributes.cluster_mode` set to `sticky-routing` or `disabled` | inert |
+
+The latest route deliberately also requires the timeseries selector: the latest
+value is derived from the `telemetry` table that only the IoTDB writer
+populates, so activating latest without the IoTDB write path would read from a
+table the module never fills. The attribute selector is fully independent of
the
+timeseries selectors.
+
+## Step-by-step enablement
+
+### 1. Add the module to the ThingsBoard build
+
+Build and install `org.apache.iotdb:iotdb-thingsboard-table` and add it as a
+dependency of the ThingsBoard application so its classes are on the runtime
+classpath. The module is a Spring Boot auto-configuration
+(`IoTDBTableConfiguration`) registered through
+`META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`,
+so it activates without the host application having to component-scan
+`org.apache.iotdb.extras`.
+
+In the `iotdb-extras` reactor the module lives behind a named opt-in profile
and
+requires JDK 17+:
+
+```bash
+# from the iotdb-extras repository root
+mvn -pl iotdb-thingsboard-table -am -P with-thingsboard clean install
+```
+
+At this point the jar is present but **inert** — no selector is set yet, so
+ThingsBoard still uses its existing backend.
+
+### 2. Bootstrap the schema
+
+On startup the module runs an idempotent bootstrap (gated by
+`iotdb.schema.bootstrap`, default `true`) that reads `schema-iotdb-table.sql`
+from the classpath and creates the target database plus the `telemetry` and
+`entity_attributes` tables on a fresh IoTDB before the first write. When the
+latest route is active, a **second** bootstrap reads
+`schema-iotdb-table-latest.sql` to create the `telemetry_latest` overlay table.
+Both bootstraps use `CREATE ... IF NOT EXISTS`, so they are safe to re-run.
+
+If you manage the schema out of band, set `iotdb.schema.bootstrap=false` and
+apply `schema-iotdb-table.sql` (and `schema-iotdb-table-latest.sql` when the
+latest route is enabled) yourself before first start.
+
+### 3. Set the activation properties for the routes you want
+
+Add the selectors from the table above to the ThingsBoard configuration. For
+example, to route both historical telemetry and latest values to IoTDB:
+
+```properties
+database.ts.type=iotdb-table
+database.ts_latest.type=iotdb-table
+iotdb.ts.experimental-raw-only=true
+```
+
+To additionally opt in to the stretch attributes route (only on a ThingsBoard
+build that exposes the attribute selector — no shipped release does yet, see
+*Limitations*):
+
+```properties
+database.attributes.type=iotdb-table
+iotdb.attributes.cluster_mode=sticky-routing
+```
+
+`iotdb.attributes.cluster_mode` is mandatory when the attribute route is active
+and must be `sticky-routing` (per-identity writes pinned to one node) or
+`disabled` (single-node / acknowledged best-effort); any other value, including
+the empty default, fails construction fast, because the attribute write path
+converges only within a single JVM.
+
+### 4. Point the module at IoTDB and size the pools
+
+Set the connection and pool properties (bound from `iotdb.*`):
+
+```properties
+iotdb.host=<iotdb-host>
+iotdb.port=6667
+iotdb.username=<iotdb-user>
+iotdb.password=<iotdb-password>
+iotdb.database=thingsboard
+iotdb.session-pool-size=8
+```
+
+The defaults are `127.0.0.1:6667`, credentials `root`/`root`, database
+`thingsboard`, and a session pool of `8`. `iotdb.database` must be a valid
IoTDB
+identifier (letter or underscore, then letters/digits/underscores). The write
+path and read path have their own pools, tunable under `iotdb.ts.save.*` and
+`iotdb.ts.read.*`:
+
+- Write path: `iotdb.ts.save.batch-size` (default `500`),
+ `iotdb.ts.save.max-linger-ms` (`20`), `iotdb.ts.save.queue-capacity`
+ (`50000`), and retry/backoff settings. The writer uses a single dedicated
+ flush worker, so `iotdb.ts.save.flush-threads` is fixed at `1` (any other
+ value fails binding).
+- Read path: `iotdb.ts.read.threads` (default `4`) and
+ `iotdb.ts.read.queue-capacity` (`10000`).
+
+### 5. Set retention (optional, table-level TTL)
+
+Physical retention in IoTDB Table Mode is a **table-level** property expressed
in
+**milliseconds** as a bare long literal, or the keyword `INF` (never expire) /
+`DEFAULT` (inherit the database default, which is `INF` on a fresh node). The
+shipped `telemetry` table is created `WITH (TTL=DEFAULT)`. To enable a concrete
+retention, either edit the schema before bootstrap, e.g. 7 days:
+
+```sql
+CREATE TABLE telemetry (...) WITH (TTL=604800000);
+```
+
+or change it at runtime on the live table:
+
+```sql
+ALTER TABLE telemetry SET PROPERTIES TTL=604800000; -- 7 days, in ms
+ALTER TABLE telemetry SET PROPERTIES TTL=DEFAULT; -- back to the db
default
+```
+
+Quoted numbers (`'604800000'`) and duration forms (`'7d'`) are rejected by
IoTDB
+2.0.8. The `telemetry_latest` overlay table is created `WITH (TTL='INF')` and
is
+**exempt** from the `telemetry` TTL: setting a retention window on `telemetry`
+does not evict overlay rows, so bound the overlay separately if needed (see
+*Limitations*).
+
+## Data-model mapping
+
+ThingsBoard's relational `ts_kv` stores one row per `(entity_id, key, ts)` with
+the value in one typed column (`bool_v` / `long_v` / `dbl_v` / `str_v` /
+`json_v`) and `key` dictionary-encoded. The IoTDB `telemetry` table keys each
row
+by a TAG tuple plus the built-in `time` column, with the value in the matching
+FIELD column. Exactly one value column is non-null per row, preserving the
+source shape.
+
+### `ts_kv` → `telemetry`
+
+| ThingsBoard `ts_kv` | IoTDB `telemetry` | IoTDB column kind |
+| --- | --- | --- |
+| `entity_id` | `entity_id` | TAG |
+| *(entity type)* | `entity_type` | TAG |
+| *(tenant)* | `tenant_id` | TAG |
+| `key` (dictionary id → string) | `key` (resolved to the string key) | TAG |
+| `ts` | `time` (built-in) | TIME |
+| `bool_v` | `bool_v` (`BOOLEAN`) | FIELD |
+| `long_v` | `long_v` (`INT64`) | FIELD |
+| `dbl_v` | `double_v` (`DOUBLE`) | FIELD |
+| `str_v` | `str_v` (`STRING`) | FIELD |
+| `json_v` | `json_v` (`TEXT`) | FIELD |
+
+### `ts_kv_latest` → `telemetry_latest`
+
+Do **not** migrate `ts_kv_latest` as a separate store. The latest value is read
+from **both** the historical `telemetry` table (derived, newest row per key)
and
+a minimal per-key `telemetry_latest` overlay, merged by the maximum timestamp
+per key. Once history is in `telemetry`, the derived latest follows
+automatically; the overlay is written by the module on every `saveLatest` going
+forward. The overlay table has the same TAG tuple
+(`entity_type`, `tenant_id`, `key`, `entity_id`) and FIELD columns as
+`telemetry`, plus an explicit `time TIMESTAMP TIME` column.
+
+### `attribute_kv` → `entity_attributes` (stretch route only)
+
+Applies only if the attribute route is activated. Each identity tuple
+`(tenant_id, entity_type, entity_id, attribute_scope, key)` holds exactly one
+current row at `time = lastUpdateTs`, with one typed FIELD set.
+
+| ThingsBoard attributes | IoTDB `entity_attributes` | IoTDB column kind |
+| --- | --- | --- |
+| `entity_id` | `entity_id` | TAG |
+| *(entity type)* | `entity_type` | TAG |
+| *(tenant)* | `tenant_id` | TAG |
+| attribute scope | `attribute_scope` (`CLIENT_SCOPE` / `SERVER_SCOPE` /
`SHARED_SCOPE`) | TAG |
+| attribute key | `key` | TAG |
+| `last_update_ts` | `time` | TIME |
+| boolean / long / double / string / json value | `bool_v` / `long_v` /
`double_v` / `str_v` / `json_v` | FIELD |
+
+## Migrating existing data (backfill)
+
+The module supplies the write/read DAO surface but not an ETL tool; the
backfill
+is operator-driven.
+
+1. **Stand up IoTDB** in Table Mode and bootstrap the schema (Step 2). Set a
+ concrete `telemetry` TTL now (Step 5) if you want retention, before
+ backfilling.
+2. **Backfill** historical `ts_kv` into `telemetry`: read from the source
+ backend, resolve the `key` dictionary id to the string key, supply
+ `tenant_id` / `entity_type` for each `entity_id`, and write via batched
+ `Tablet` inserts using the value-column mapping above. The module's own
write
+ path batches 500 rows per flush, a reasonable starting batch size.
+3. **Verify** a sample: spot-check historical reads and latest values for a set
+ of `(entity, key)` pairs against the source before the cut-over.
+4. **Switch** ThingsBoard to the IoTDB routes by setting the selectors (Step 3)
+ and restart. New writes flow into `telemetry`; the derived latest is
+ immediately correct because it reads the same table, and the overlay is
+ written from the first new `saveLatest` onward.
+
+Idempotency: re-running the backfill for the same `(tags, time)` overwrites the
+typed column in place (IoTDB same-timestamp overwrite), so re-runs converge
+rather than duplicate. A value whose **type** changed at the same timestamp is
+subject to the same-timestamp limitation below.
+
+For a zero-downtime cut-over, a dual-write or read-old / write-new bridge
during
+backfill is an operator concern beyond this guide.
+
+## Coexistence and rollback
+
+Each route is independently activated and guarded:
+
+- **Independent activation.** Enabling telemetry does not enable attributes,
and
+ vice versa. You can route telemetry + latest to IoTDB while attributes stay
in
+ the host entity database (the default Phase-1 posture).
+- **Fail-fast conflict guard.** When a route is enabled but a conflicting
+ non-IoTDB host DAO bean of the same SPI type is also present, startup fails
+ fast with a clear message rather than silently shadowing one DAO with
another.
+ The historical (`TimeseriesDao`), latest (`TimeseriesLatestDao`), and
attribute
+ (`AttributesDao`) routes each have their own guard. Make sure the host
backend
+ for a route is removed/disabled when you point that route at IoTDB.
+
+### Rollback
+
+To roll a route back, **unset its selector(s)** and restart. The route's
+activation condition then returns `false`, the IoTDB DAO and its session pool
are
+not created, the conflict guard does not fire, and the host backend's DAO
+resumes serving that route.
+
+What is and isn't reversible:
+
+- **Configuration is fully reversible.** Unsetting the selectors returns
+ ThingsBoard to the host backend with no schema changes required on the IoTDB
+ side; the IoTDB tables simply stop receiving new writes.
+- **Data written to IoTDB stays in IoTDB.** Rolling back the configuration does
+ not copy IoTDB-resident telemetry/latest/attributes back into the host
+ backend. If the host backend was serving the route during the IoTDB window,
+ those two stores will have diverged for that period. Plan a reverse backfill
+ (or a dual-write window) if you need the host backend to be current after a
+ rollback.
+
+## Limitations to plan for
+
+These are the documented current-scope limitations (see the DAO Javadoc and the
+module `README.md` for the authoritative list):
+
+- **`version` is always `null`** on the latest and attribute routes. IoTDB has
no
+ SQL sequence (same as the Cassandra backend); this is type-correct and
+ contract-legal, but ThingsBoard notifications that key off a non-null version
+ are not driven in Phase-1.
+- **Same-timestamp cross-store type change.** A single
+ `(tenant, entity, key, timestamp)` point whose value *type* changes between
two
+ separate flushes can land two non-null typed columns. The behavior is
+ **fail-fast, not silent**: a raw read of that one point throws an
+ `IllegalStateException` rather than returning a wrong value; every other
point
+ is unaffected. The latest overlay wins an exact-timestamp tie, continuing the
+ same limitation on the latest route.
+- **Overlay TTL=`INF` growth.** `telemetry_latest` never physically expires and
+ has no entity-level cleanup, so under unbounded key cardinality it grows
+ without bound (one row per identity; bounded for normal key sets). Bound it
+ separately with a TTL on `telemetry_latest` if needed.
+- **Per-save `ttl` is not honored as physical retention.** The
+ `TimeseriesDao.save(..., long ttl)` per-data-point TTL cannot be reconciled
+ with table-wide IoTDB TTL; the module uses it only for ThingsBoard's
+ storage-accounting, never as a physical-retention directive. Set physical
+ retention on the table (Step 5).
+- **The attributes route is a stretch / Phase-2 opt-in.** No shipped
ThingsBoard
+ release exposes a `database.attributes.type` selector yet (open question,
+ tracked upstream), so in a real Phase-1 deployment the selector is unset and
+ attributes stay in the host entity database. When activated, `save` is a
+ non-atomic tag-only delete-then-insert under a per-identity in-JVM lock that
+ converges only within one JVM; `findNextBatch` is unsupported
+ (`UnsupportedOperationException`), and `findAllKeysByDeviceProfileId` with a
+ non-null profile returns an empty list (matching the non-relational backend).
+
+## Verifying the cut-over
+
+After switching a route to IoTDB:
+
+- Confirm the target tables exist and carry data: `SHOW TABLES` in the IoTDB
+ database, and spot-check rows in `telemetry` (and `telemetry_latest` when the
+ latest route is on).
+- Confirm the effective TTL via `information_schema.tables` (the `ttl(ms)`
+ column) or `SHOW TABLES` (the `TTL(ms)` column) if you set retention.
+- Read back a few `(entity, key)` latest values through ThingsBoard and compare
+ against the pre-cut-over source sample.
diff --git a/iotdb-thingsboard-table/docs/user-guide.md
b/iotdb-thingsboard-table/docs/user-guide.md
new file mode 100644
index 0000000..2c19743
--- /dev/null
+++ b/iotdb-thingsboard-table/docs/user-guide.md
@@ -0,0 +1,315 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# IoTDB ThingsBoard Table — User Guide
+
+This guide explains how to configure, activate, and operate the IoTDB Table
Mode
+storage backend for ThingsBoard. For the architecture overview, the
+compile-only ThingsBoard SPI surface, build/test instructions, and the design
+rationale, see the module [`README.md`](../README.md).
+
+## 1. What the module is
+
+`iotdb-thingsboard-table` is a storage backend that lets a ThingsBoard
+deployment persist telemetry, latest values, and (optionally) entity attributes
+in Apache IoTDB 2.x relational **Table Mode** instead of the default
+Cassandra / SQL backends. It implements ThingsBoard's `TimeseriesDao`,
+`TimeseriesLatestDao`, and `AttributesDao` SPIs and ships as a Spring Boot
+auto-configuration, so it activates inside a real ThingsBoard deployment
without
+the host application having to component-scan the module's package.
+
+The module is built against the reactor's IoTDB 2.0.5 table-session client and
+its integration tests exercise the real write path against an
+`apache/iotdb:2.0.8-standalone` server. It targets ThingsBoard `4.3.1.2` and
+requires JDK 17+.
+
+**It is default-inert.** None of the three DAOs activate unless the operator
+sets the explicit selector properties below. With no selector set, the module
+contributes no beans and ThingsBoard keeps using its configured backends.
+
+## 2. Prerequisites
+
+- A running Apache IoTDB 2.x node with relational Table Mode, reachable from
the
+ ThingsBoard host over the IoTDB client RPC port.
+- The target IoTDB database (default `thingsboard`) and its tables. On a fresh
+ node the module's idempotent startup bootstrap creates them for you (see
+ [§5](#5-schema-and-bootstrap)); operators who manage the schema out-of-band
+ can disable the bootstrap and pre-create the tables from the shipped SQL.
+- IoTDB credentials with permission to create the database/tables (when the
+ bootstrap is enabled) and to read/write the tables.
+
+## 3. Activation
+
+Each path is selected independently and is inert until its selector is present.
+Selector values are matched case-insensitively (trimmed).
+
+| Path | Required properties | Default state |
+| --- | --- | --- |
+| Historical timeseries (`TimeseriesDao`) | `database.ts.type=iotdb-table`
**and** `iotdb.ts.experimental-raw-only=true` | inert |
+| Latest telemetry (`TimeseriesLatestDao`) | all of
`database.ts.type=iotdb-table`, `iotdb.ts.experimental-raw-only=true`, **and**
`database.ts_latest.type=iotdb-table` | inert |
+| Entity attributes (`AttributesDao`) | `database.attributes.type=iotdb-table`
(independent of the timeseries selectors) | inert (opt-in / stretch) |
+
+Notes:
+
+- **Timeseries** requires both `database.ts.type=iotdb-table` and the explicit
+ experimental opt-in `iotdb.ts.experimental-raw-only=true`; setting only one
of
+ them leaves the path inert.
+- **Latest** additionally requires `database.ts_latest.type=iotdb-table`. The
+ timeseries selectors are required as well because the latest value is derived
+ from the `telemetry` table that only the IoTDB writer populates — the latest
+ path can never activate without that writer.
+- **Attributes** is an opt-in stretch feature. No shipped
+ ThingsBoard release exposes a `database.attributes.type` selector yet (open
+ question Q6 / ThingsBoard Discussion #15296), so a real deployment normally
+ leaves it unset and attributes keep flowing to the host entity database. When
+ it is activated, `iotdb.attributes.cluster-mode` must also be set or the DAO
+ fails fast at startup (see [§6](#6-configuration-reference)).
+
+When activated, the DAOs share a single module-owned IoTDB table session pool.
+
+### Conflict guards
+
+When a selector is on, the module fails startup fast if a conflicting non-IoTDB
+DAO bean of the same SPI type is also present (for example, another
+`TimeseriesDao` while `database.ts.type=iotdb-table`). This is deliberate: it
+prevents the module from silently shadowing, or being shadowed by, a different
+backend. Remove the conflicting backend or unset the IoTDB selector.
+
+## 4. The three DAOs
+
+### 4.1 Timeseries DAO (`TimeseriesDao`)
+
+- **Write** (`save`): telemetry points are accepted onto a bounded in-memory
+ queue and flushed by a single writer thread as multi-row `Tablet` inserts
into
+ the `telemetry` table. Writes are asynchronous and batched; the call returns
a
+ future. Duplicate `(tenant, entity, key, timestamp)` saves within one flush
are
+ collapsed so the last write wins.
+- **Raw read** (`findAllAsync` with no aggregation, or an interval below 1):
+ returns the stored points for the query window, ordered and limited as
+ requested.
+- **Aggregated read** (`findAllAsync` with a positive interval): time-bucketed
+ aggregation matching ThingsBoard's contract, in two forms:
+ - **Fixed-width millisecond buckets** anchored at the query start timestamp,
+ using IoTDB's `date_bin` primitive (supports `AVG`, `SUM`, `COUNT`, `MIN`,
+ `MAX`).
+ - **Timezone-aware calendar buckets** for `WEEK`, `WEEK_ISO`, `MONTH`, and
+ `QUARTER` intervals, where each calendar boundary is computed in the
query's
+ timezone and one bounded aggregate query is issued per bucket.
+- **Delete** (`remove`): deletes points for an identity within the query's
+ half-open time window.
+- **Retention hooks**: `cleanup` is a no-op and `savePartition` does nothing,
+ because IoTDB Table Mode keeps no per-partition bookkeeping and physical
+ retention is a table-level property (see [§7](#7-retention--ttl)).
+
+### 4.2 Latest telemetry DAO (`TimeseriesLatestDao`)
+
+Active only when all three timeseries+latest selectors are set.
+
+- **Read** (`findLatest`, `findLatestOpt`, `findAllLatest`): the latest value
per
+ `(tenant, entity, key)` is read from **both** the historical `telemetry`
table
+ (derived via `ORDER BY time DESC LIMIT 1` / `LAST_BY`, engine-accelerated by
+ IoTDB's last cache) **and** the `telemetry_latest` overlay table, then merged
+ by the maximum timestamp per key (the overlay wins an exact tie). The merge
is
+ not additive, so a key present in both stores is never double-counted.
+- **Write** (`saveLatest`): writes the per-key overlay row as a
delete-then-insert
+ under a per-identity lock. It is **max-timestamp-wins**: a backdated
+ `saveLatest` whose timestamp is not newer than the stored latest is skipped,
so
+ an out-of-order write never regresses the latest. The overlay is written on
+ every (non-backdated) call so a latest-only write — one with no paired
+ historical `save`, e.g. the EntityView telemetry-copy path — is not lost.
+- **Delete** (`removeLatest`): snapshots the merged latest under the
per-identity
+ lock; when that latest falls inside the half-open `[startTs, endTs)` window
it
+ deletes the overlay row, and when `rewriteLatestIfDeleted` is set it
resurrects
+ the next-older historical value (from `telemetry`) back into the overlay.
+- **Key discovery** (`findAllKeysByEntityIds` / `…Async`): returns the distinct
+ telemetry keys for the entity set, unioned across `telemetry` and the
+ `telemetry_latest` overlay, so a latest-only key is still discoverable.
+ `findAllKeysByDeviceProfileId` returns the tenant-wide distinct keys for a
null
+ (all-profiles) device profile and an empty list for a specific profile (the
+ tables carry no device-profile membership).
+
+### 4.3 Entity attributes DAO (`AttributesDao`) — opt-in / stretch
+
+Inert by default; activated only by `database.attributes.type=iotdb-table`.
+
+- **Write** (`save`): each identity tuple
+ `(tenant_id, entity_type, entity_id, attribute_scope, key)` holds exactly one
+ current row. `save` is a tag-only `DELETE` (no time predicate) followed by an
+ `INSERT` at `time = lastUpdateTs` with one typed value column set, both
under a
+ per-identity lock so concurrent same-identity writes converge to a single
row.
+- **Read** (`find`, `findAll`, and the by-keys / by-entity-ids variants):
+ synchronous `SELECT`s over the `entity_attributes` table.
+- **Delete** (`removeAll`, `removeAllWithVersions`): one future per key, each a
+ tag-only `DELETE`. `removeAllByEntityId` is a best-effort select-then-delete
+ under an entity-level write lock.
+
+## 5. Schema and bootstrap
+
+The module uses one table per path. All three share the same five typed value
+columns — `bool_v BOOLEAN`, `long_v INT64`, `double_v DOUBLE`, `str_v STRING`,
+`json_v TEXT` — of which **exactly one is non-null per row**.
+
+| Table | Purpose | TAG columns (in order) | TTL |
+| --- | --- | --- | --- |
+| `telemetry` | historical telemetry | `entity_type`, `tenant_id`, `key`,
`entity_id` | `DEFAULT` |
+| `telemetry_latest` | per-key latest overlay | `entity_type`, `tenant_id`,
`key`, `entity_id` | `INF` |
+| `entity_attributes` | entity attributes | `attribute_scope`, `entity_type`,
`tenant_id`, `key`, `entity_id` | `INF` |
+
+The shipped DDL lives in
+[`schema-iotdb-table.sql`](../src/main/resources/schema-iotdb-table.sql)
+(`telemetry` + `entity_attributes`) and
+[`schema-iotdb-table-latest.sql`](../src/main/resources/schema-iotdb-table-latest.sql)
+(`telemetry_latest`).
+
+**Bootstrap.** When `iotdb.schema.bootstrap` is `true` (the default), the
module
+runs an idempotent startup bootstrap that reads the schema SQL from the
classpath
+and creates the database and tables before the first write. Every statement
uses
+`CREATE … IF NOT EXISTS`, so re-runs are harmless. The base schema is created
+whenever the timeseries or attributes path is active; the `telemetry_latest`
+overlay is created by a second bootstrap only when the latest path is active.
+Set `iotdb.schema.bootstrap=false` to manage the schema out-of-band.
+
+## 6. Configuration reference
+
+All module knobs are bound from the `iotdb.*` prefix; the three `database.*`
+selectors and `iotdb.ts.experimental-raw-only` are the activation switches from
+[§3](#3-activation). Defaults are shown.
+
+### Connection
+
+| Property | Default | Notes |
+| --- | --- | --- |
+| `iotdb.host` | `127.0.0.1` | IoTDB node host. |
+| `iotdb.port` | `6667` | IoTDB client RPC port (1–65535). |
+| `iotdb.database` | `thingsboard` | Target Table Mode database. Must be a
valid IoTDB identifier (a letter or underscore, then
letters/digits/underscores). |
+| `iotdb.username` | `root` | Set a real credential in production. |
+| `iotdb.password` | `root` | Set a real credential in production. |
+| `iotdb.session-pool-size` | `8` | Table session pool size (1–1024). Keep it
at least `iotdb.ts.read.threads + iotdb.ts.save.flush-threads`, or the module
logs a warning that reads/flushes may wait for a session. |
+| `iotdb.connection-timeout-ms` | `5000` | Connection timeout (minimum 100). |
+| `iotdb.enable-compression` | `false` | RPC compression. |
+| `iotdb.default-ttl-ms` | `-1` | ThingsBoard storage data-point accounting
only; **not** an IoTDB physical-retention setting. `-1` disables the default. |
+| `iotdb.schema.bootstrap` | `true` | Run the idempotent startup schema
bootstrap. Set `false` to manage the schema yourself. |
+
+### Write path (`iotdb.ts.save.*`)
+
+| Property | Default | Notes |
+| --- | --- | --- |
+| `batch-size` | `500` | Rows per `Tablet` insert. |
+| `max-linger-ms` | `20` | Maximum time a partial batch waits to fill before
flushing. |
+| `queue-capacity` | `50000` | Bounded accept queue; over-capacity saves are
rejected (back-pressure), not blocked. |
+| `flush-threads` | `1` | Fixed at 1 (single-writer ordering). Any other value
fails binding with a clear error. |
+| `shutdown-drain-timeout-ms` | `5000` | Graceful-drain window on shutdown.
Keep `≥ max-linger-ms`. |
+| `retry-max-attempts` | `3` | Per-batch attempts on a transient IoTDB error. |
+| `retry-initial-backoff-ms` | `50` | Initial retry backoff. |
+| `retry-max-backoff-ms` | `1000` | Maximum retry backoff (exponential between
the two). |
+
+### Read path (`iotdb.ts.read.*`)
+
+These also size the bounded executor used by the latest and attribute DAOs.
+
+| Property | Default | Notes |
+| --- | --- | --- |
+| `threads` | `4` | Bounded read/IO executor size. |
+| `queue-capacity` | `10000` | Read/IO task queue bound; an over-capacity task
is rejected with a queue-full error. |
+
+### Attributes (`iotdb.attributes.*`) — only when the attribute DAO is active
+
+| Property | Default | Notes |
+| --- | --- | --- |
+| `iotdb.attributes.cluster-mode` | *(empty)* | Required when
`database.attributes.type=iotdb-table`. Must be `sticky-routing` (writes for an
identity pinned to one node) or `disabled` (single-node / acknowledged
best-effort). Any other value, including the empty default, fails fast at
startup, because the per-identity write lock converges writes only within a
single JVM. (The module also accepts the equivalent
`iotdb.attributes.cluster_mode` spelling.) |
+
+## 7. Retention / TTL
+
+Physical retention is a **table-level** IoTDB property, expressed in
+milliseconds, set by the operator on the schema — not a per-data-point setting.
+The shipped `telemetry` table is declared `WITH (TTL=DEFAULT)` (inherit the
+database default, which is `INF` on a fresh node). To enable a concrete
+retention, edit the schema before bootstrap or run
+`ALTER TABLE telemetry SET PROPERTIES TTL=<milliseconds>` on the live table.
+
+Two consequences to be aware of:
+
+- The per-save `ttl` argument carried by `TimeseriesDao.save(..., ttl)` is
**not**
+ applied as physical retention per row — IoTDB Table Mode TTL is table-wide.
The
+ module uses the per-save `ttl` (and `iotdb.default-ttl-ms`) only for
+ ThingsBoard's storage data-point accounting.
+- The `telemetry_latest` overlay is declared `WITH (TTL='INF')` and is exempt
+ from the `telemetry` TTL, so a retention window on `telemetry` does not evict
+ overlay rows. Operators who need to bound the overlay must set a TTL on
+ `telemetry_latest` separately.
+
+See the README's *Retention / TTL* section for the full mechanism and the
+syntax IoTDB accepts.
+
+## 8. Operational notes
+
+- **Back-pressure.** The write accept queue (`iotdb.ts.save.queue-capacity`) is
+ bounded. When it is full, a `save` is rejected with a queue-full error rather
+ than blocking the caller — size the queue for your ingest rate.
+- **Retries.** A batch that hits a transient IoTDB condition (connection error,
+ or a server-side overload / timeout / dispatch status) is retried up to
+ `retry-max-attempts` with exponential backoff. Permanent errors (parse, type,
+ schema) fail fast without consuming the retry budget.
+- **Graceful shutdown.** On shutdown the writer stops accepting new saves and
+ drains already-accepted writes within
`iotdb.ts.save.shutdown-drain-timeout-ms`.
+ If the window expires, the remaining pending writes are failed. Shutdown is
+ at-least-once: if the timeout races an insert that has already started, the
+ write may still commit after its future has failed, so tolerate a duplicate
or
+ uncertain final batch.
+- **Session pool sizing.** Keep `iotdb.session-pool-size` at least as large as
+ the sum of read threads and flush threads (see
[§6](#6-configuration-reference)).
+
+## 9. Known limitations (Phase-1 residuals)
+
+These are documented behaviors of the current module, not defects to work
around
+silently:
+
+- **Latest/attribute `version` is always `null`.** IoTDB has no SQL sequence,
so
+ `saveLatest`, attribute `save`, and `removeAllWithVersions` return a `null`
+ version (type-correct, matching the Cassandra backend). ThingsBoard
+ notifications that key off a non-null version are therefore not driven.
+- **Same-timestamp cross-store type change.** The writer collapses duplicate
+ `(tenant, entity, key, timestamp)` saves within a single flush, but it does
not
+ defend against a same-timestamp save whose value *type* changes across two
+ separate flushes. Because each typed value lands in its own column, that
single
+ point can end up with two non-null typed columns. The behavior is fail-fast,
+ not silent: a read of that one point throws rather than returning a wrong
value;
+ every other point is unaffected. In the latest path, the overlay wins an
exact
+ timestamp tie.
+- **Overlay growth.** `telemetry_latest` is `TTL='INF'` with no entity-level
+ cleanup, so under unbounded key cardinality it grows without bound (one row
per
+ identity; bounded for normal key sets).
+- **Latest delete eventual consistency.** A purely telemetry-derived
(full-save)
+ latest can transiently still be read from `telemetry` until the separate
+ historical delete commits.
+- **Single-JVM write convergence.** The per-identity locks that serialize the
+ latest-overlay and attribute writes are in-JVM, so cross-node single-writer
+ safety is the operator's responsibility. The attribute path makes this
explicit
+ via the required `iotdb.attributes.cluster-mode` acknowledgement.
+- **Unsupported SPI operations.** The attribute relational-migration helper
+ `findNextBatch` throws `UnsupportedOperationException`: it is a relational
+ keyset-pagination helper with no IoTDB equivalent, and it is the only method
in
+ the module that throws. The batch latest read (`findLatestByEntityIds` /
+ `…Async`, new in ThingsBoard 4.3.1.2) instead returns empty, matching
+ `CassandraBaseTimeseriesLatestDao` — it backs the `includeSamples` branch of
+ `POST /api/entitiesQuery/find/keys`, where a synchronous throw would surface
as
+ an HTTP 500. A specific (non-null) device-profile key lookup likewise returns
+ an empty list.
diff --git
a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java
b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java
index 2155f0c..915d352 100644
---
a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java
+++
b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfig.java
@@ -84,9 +84,13 @@ public class IoTDBTableConfig {
@Data
public static class Ts {
/**
- * Explicit opt-in for the raw-only backend. This backend currently
implements write, raw read,
- * and delete only; time-bucketed aggregation is outside the current
scope, so it is not
- * production-complete and must be enabled deliberately.
+ * Explicit opt-in for this backend. It implements write, raw read,
time-bucketed aggregation
+ * (both the fixed-width millisecond {@code date_bin} path and the
timezone-aware calendar
+ * buckets) and delete. The property name predates the aggregation support
and is kept for
+ * compatibility. It remains experimental because of a known Phase-1
relaxation: a single {@code
+ * (tenant, entity, key, timestamp)} point whose value type changes across
two separate flushes
+ * can leave two non-null typed columns, and a raw read of that one point
then fails fast — so
+ * the backend must still be enabled deliberately.
*/
private boolean experimentalRawOnly = false;
diff --git
a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java
b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java
index 37c244c..96387db 100644
---
a/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java
+++
b/iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java
@@ -80,7 +80,7 @@ public class IoTDBTableTimeseriesDao extends IoTDBTableBaseDao
private static final long SECONDS_PER_DAY = 86400L;
private static final String TABLE_NAME =
IoTDBTableTimeseriesWriter.TABLE_NAME;
- // Result-set column aliases for the time-bucketed aggregation read path
(design doc §3.3).
+ // Result-set column aliases for the time-bucketed aggregation read path.
private static final String BUCKET_TS_COLUMN = "bucket_ts";
private static final String AGG_NUM_COLUMN = "agg_num";
private static final String AGG_STR_COLUMN = "agg_str";
