This is an automated email from the ASF dual-hosted git repository.
jackietien pushed a change to branch new-compaction
in repository https://gitbox.apache.org/repos/asf/iotdb.git.
from 8862cfa Merge remote-tracking branch 'origin/master' into
new-compaction
add 8944470 [IOTDB-2320] MemoryLeak cause by wal Scheduled trim task
thread (#4804)
add 302e9b9 [IOTDB-2323] Fix remove lock file in syncClient error (#4803)
add b981551 [IOTDB-2384]Enhanced data type check on creating schema
template (#4800)
add 9b1c52a fix a logical bug in processPlanLocally (#4811)
add e3827ea [IOTDB-2367] Linear fill should log warning for unsupported
data type (#4809)
add 495d122 [IOTDB-2358] Declaring UDF does not support use in aligned
time series (#4812)
add 6deaac8 [IOTDB-2373][Trigger] MQTTSink: there should be a timeout
rule (#4814)
add 02b64d2 [IOTDB-2313] GC overhead limit when use "select * from
root.**" (#4778)
add d0f8e67 [IOTDB-2393] Distinguish client version via version nubmer in
TSOpenSessionReq (#4813)
add 2e5c9e6 [IOTDB-2386] Add aggregate query and query cannot appear in
SQL user guide at the same time (#4817)
add da91b60 [IOTDB-2395] Thread Name Conflict (#4816)
add 32ff938 [IOTDB-2393][PY] Distinguish client version via version
nubmer in TSOpenSessionReq (#4820)
add bcd7a0b [IOTDB-2344]Fix docs as issue-2344 requested (#4823)
add 2f49925 [IOTDB-2276] Optimize aligned query with value filter to
accelerate it and reduce its memory footprint (#4768)
add 001490b [IOTDB-2315] Modify some problems of metric (#4782)
add c31903b [IOTDB-2353] CQ: support specifying first execution time
(BOUNDARY) when creating CQ (#4831)
add 53f48d2 [IOTDB-2400] Fix series reader bug (#4819)
add 46ee27b [IOTDB-1892] Add TsfileSelfCheckTool in server (#4674)
add 186c12f [IOTDB-2363] LEVEL can be specified at non-last level in
metadata queries (#4838)
add 6c24a24 [IOTDB-2392] Memory control of raft log in cluster (#4825)
add 23c35f3 [IOTDB-2379]Fix code of pulsar example (#4840)
add de01258 [IOTDB-2328] Missing system config in user manual (#4841)
add b959fb3 [IOTDB-2403] The parameter "ascending" is not set correctly
when getting QueryDataSource (#4830)
add 3c94818 [IOTDB-2383] Make simplified SQL of creating timeseries and
schema template more flexible (#4832)
add cca356e Fix timed flush memtable and timed close tsfile test (#4839)
add 76fe523 [IOTDB-2321] Add schema template APIs for C++ client (#4796)
add 81154e6 [IOTDB-2394] Fixed temporary network unavailability caused by
frequent closed connections (#4815)
add ff60ebc [IOTDB-2375][IOTDB-2376] CQ: Get a wrong result data set
(#4842)
add d84856e [IOTDB-2407] When there is an xxx.tsfile.mods, using load
tsfile will lose all data (#4843)
add aebe6f1 [IOTDB-2310][IOTDB-2311][IOTDB-2372] Add query type check for
REST query API (#4844)
add fdf79b8 Close the tsfile once the memtable is flushed into (#4821)
add 138d14c [IOTDB-2313] REST: GC overhead limit when select * from
root.** (#4845)
add 2bf83a5 [IOTDB-2287] Separate CLI history by different hosts, ports
and usernames (#4826)
add 707c2b4 [IOTDB-2181] Fix cluster throw exception when queried tag
does not exist (#4613)
add 3115b09 [IOTDB-2326] add doc for compaction param (#4822)
add 478b9bd [IOTDB-2191] Use IoTDB itself to collect runtime metrics
(#4634)
new a87ec25 Merge remote-tracking branch 'origin/master' into
new-compaction
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/iotdb/db/qp/sql/IoTDBSqlLexer.g4 | 4 +
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 4 +-
cli/src/main/java/org/apache/iotdb/cli/Cli.java | 18 +-
.../org/apache/iotdb/cli/utils/JlineUtils.java | 15 +-
.../main/java/org/apache/iotdb/tool/ExportCsv.java | 2 +-
client-cpp/src/main/Session.cpp | 426 +++++++++++++++--
client-cpp/src/main/Session.h | 529 +++++++++++++++------
client-cpp/src/test/cpp/sessionIT.cpp | 469 +++++++++---------
client-py/iotdb/Session.py | 1 +
.../resources/conf/iotdb-cluster.properties | 21 +-
.../org/apache/iotdb/cluster/ClusterIoTDB.java | 26 +-
.../iotdb/cluster/client/ClientPoolFactory.java | 3 +
.../apache/iotdb/cluster/config/ClusterConfig.java | 51 ++
.../iotdb/cluster/config/ClusterDescriptor.java | 26 +-
.../cluster/query/ClusterUDTFQueryExecutor.java | 21 +-
.../iotdb/cluster/query/LocalQueryExecutor.java | 3 +-
.../cluster/query/reader/ClusterReaderFactory.java | 2 +-
.../org/apache/iotdb/cluster/server/Response.java | 3 +-
.../server/handlers/caller/LogCatchUpHandler.java | 11 +-
.../iotdb/cluster/server/member/RaftMember.java | 158 ++++--
.../cluster/client/ClientPoolFactoryTest.java | 31 ++
docs/SystemDesign/SchemaManager/SchemaManager.md | 8 +-
docs/UserGuide/API/Programming-Cpp-Native-API.md | 422 ++++++++++------
.../Advanced-Features/Continuous-Query.md | 97 +++-
docs/UserGuide/Advanced-Features/Select-Into.md | 3 +-
.../Advanced-Features/UDF-User-Defined-Function.md | 6 +
docs/UserGuide/Appendix/Config-Manual.md | 391 +++++++++++++--
docs/UserGuide/Appendix/SQL-Reference.md | 2 +-
docs/UserGuide/Data-Concept/Schema-Template.md | 36 +-
.../DDL-Data-Definition-Language.md | 15 +-
.../DML-Data-Manipulation-Language.md | 14 +
docs/UserGuide/System-Tools/Metric-Tool.md | 85 ++--
.../UserGuide/System-Tools/TsFileSelfCheck-Tool.md | 42 ++
.../zh/SystemDesign/SchemaManager/SchemaManager.md | 2 +-
.../zh/UserGuide/API/Programming-Cpp-Native-API.md | 417 +++++++++++-----
.../UserGuide/API/Programming-Java-Native-API.md | 37 +-
.../Advanced-Features/Continuous-Query.md | 114 ++++-
docs/zh/UserGuide/Advanced-Features/Select-Into.md | 3 +-
.../Advanced-Features/UDF-User-Defined-Function.md | 6 +-
docs/zh/UserGuide/Appendix/Config-Manual.md | 310 ++++++++----
docs/zh/UserGuide/Appendix/SQL-Reference.md | 2 +-
.../Collaboration-of-Edge-and-Cloud/Sync-Tool.md | 2 +-
.../Communication-Service-Protocol/RestService.md | 45 +-
.../Data-Concept/Data-Model-and-Terminology.md | 6 +-
docs/zh/UserGuide/Data-Concept/Schema-Template.md | 81 +++-
.../DDL-Data-Definition-Language.md | 12 +-
.../DML-Data-Manipulation-Language.md | 14 +
docs/zh/UserGuide/Library-UDF/Data-Quality.md | 6 +-
docs/zh/UserGuide/Library-UDF/Data-Repair.md | 4 +-
docs/zh/UserGuide/QuickStart/Files.md | 6 +-
docs/zh/UserGuide/System-Tools/Metric-Tool.md | 90 ++--
.../UserGuide/System-Tools/TsFileSelfCheck-Tool.md | 42 ++
.../src/AlignedTimeseriesSessionExample.cpp | 52 +-
example/client-cpp-example/src/SessionExample.cpp | 47 +-
.../java/org/apache/iotdb/mqtt/MQTTClient.java | 2 +
.../apache/iotdb/pulsar/PulsarConsumerThread.java | 1 +
.../db/integration/IoTDBContinuousQueryIT.java | 191 +++++++-
.../db/integration/IoTDBLoadExternalTsfileIT.java | 204 ++++++++
.../iotdb/db/integration/IoTDBMetadataFetchIT.java | 2 +-
.../db/integration/IoTDBNewTsFileCompactionIT.java | 4 +
.../iotdb/db/integration/IoTDBTagAlterIT.java | 7 +-
.../apache/iotdb/db/integration/IoTDBTagIT.java | 15 +-
.../integration/IoTDBUDTFAlignByTimeQueryIT.java | 25 +
.../aligned/IOTDBInsertAlignedValuesIT.java | 3 +
.../aligned/IoTDBLoadExternalAlignedTsFileIT.java | 211 +++++++-
...ava => IoTDBRawQueryWithoutValueFilter3IT.java} | 8 +-
.../main/java/org/apache/iotdb/jdbc/Config.java | 3 +
.../main/java/org/apache/iotdb/jdbc/Constant.java | 6 +
.../org/apache/iotdb/jdbc/IoTDBConnection.java | 1 +
.../apache/iotdb/jdbc/IoTDBConnectionParams.java | 11 +
.../src/main/java/org/apache/iotdb/jdbc/Utils.java | 3 +
metrics/ReadMe.md | 73 ++-
.../dropwizard/DropwizardMetricManager.java | 29 +-
.../reporter/prometheus/PrometheusReporter.java | 4 +-
.../reporter/prometheus/PushGateway.java | 13 +-
.../dropwizard/DropwizardMetricManagerTest.java | 22 +-
.../metrics/dropwizard/DropwizardMetricTest.java | 181 -------
.../dropwizard/DropwizardMetricTestPlan.java | 62 ---
.../metrics/dropwizard/PrometheusRunTest.java | 42 --
.../src/test/resources/iotdb-metric.yml | 42 --
.../main/assembly/resources/conf/iotdb-metric.yml | 12 +-
.../apache/iotdb/metrics/CompositeReporter.java | 18 +-
...ingCounter.java => DoNothingMetricService.java} | 15 +-
.../org/apache/iotdb/metrics/MetricManager.java | 12 -
.../org/apache/iotdb/metrics/MetricService.java | 88 ++--
.../apache/iotdb/metrics/config/MetricConfig.java | 46 ++
.../metrics/config/MetricConfigDescriptor.java | 43 +-
.../ReloadLevel.java} | 11 +-
.../iotdb/metrics/impl/DoNothingMetricManager.java | 20 -
.../iotdb/metrics/utils/PredefinedMetric.java | 4 +-
.../apache/iotdb/metrics/utils/ReporterType.java | 2 +-
.../iotdb/metrics/config/MetricConfigTest.java | 13 +-
.../src/{main => }/test/resources/iotdb-metric.yml | 3 +-
.../micrometer/MicrometerMetricManager.java | 66 +--
.../micrometer/reporter/MicrometerJmxReporter.java | 9 +-
.../reporter/MicrometerPrometheusReporter.java | 21 +-
.../org.apache.iotdb.metrics.MetricManager | 2 +-
.../micrometer/MicrometerMetricManagerTest.java | 15 +-
.../metrics/micrometer/MicrometerMetricTest.java | 181 -------
.../micrometer/MicrometerMetricTestPlan.java | 62 ---
.../src/test/resources/iotdb-metric.yml | 42 --
openapi/src/main/openapi3/iotdb-rest.yaml | 2 +
server/pom.xml | 5 +
.../resources/conf/iotdb-engine.properties | 13 +-
.../assembly/resources/conf/iotdb-rest.properties | 5 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 24 +-
.../org/apache/iotdb/db/conf/IoTDBConstant.java | 7 +
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 15 +-
.../iotdb/db/conf/rest/IoTDBRestServiceConfig.java | 10 +
.../db/conf/rest/IoTDBRestServiceDescriptor.java | 5 +
.../cross/inplace/manage/MergeManager.java | 3 +-
.../engine/cq/ContinuousQuerySchemaCheckTask.java | 99 ++++
.../db/{ => engine}/cq/ContinuousQueryService.java | 50 +-
.../db/{ => engine}/cq/ContinuousQueryTask.java | 177 ++-----
.../cq/ContinuousQueryTaskPoolManager.java | 2 +-
.../selectinto/InsertTabletPlansIterator.java | 51 +-
.../storagegroup/VirtualStorageGroupProcessor.java | 45 +-
.../trigger/sink/mqtt/MQTTConfiguration.java | 31 ++
.../db/engine/trigger/sink/mqtt/MQTTHandler.java | 2 +
...java => TsFileTimeseriesMetadataException.java} | 11 +-
.../org/apache/iotdb/db/metadata/MManager.java | 30 +-
.../db/metadata/lastCache/LastCacheManager.java | 4 +-
.../org/apache/iotdb/db/metadata/mtree/MTree.java | 6 +-
.../mtree/traverser/collector/MNodeCollector.java | 30 +-
.../mtree/traverser/counter/MNodeLevelCounter.java | 29 +-
.../apache/iotdb/db/metadata/path/AlignedPath.java | 18 +
.../iotdb/db/metadata/path/MeasurementPath.java | 3 +
.../apache/iotdb/db/metadata/tag/TagManager.java | 13 +-
.../apache/iotdb/db/metadata/utils/MetaUtils.java | 2 +-
.../reporter/DropwizardIoTDBReporter.java | 47 +-
.../metrics/dropwizard/reporter/IoTDBReporter.java | 233 +++++++++
.../org/apache/iotdb/db/metrics/metricsUtils.java | 61 +++
.../micrometer/registry/IoTDBMeterRegistry.java | 150 ++++++
.../micrometer/registry/IoTDBRegistryConfig.java | 12 +-
.../reporter/MicrometerIoTDBReporter.java | 36 +-
.../handler/PhysicalPlanValidationHandler.java | 78 +++
.../protocol/rest/handler/QueryDataSetHandler.java | 23 +-
.../rest/handler/RequestValidationHandler.java | 5 +
.../protocol/rest/impl/GrafanaApiServiceImpl.java | 7 +-
.../db/protocol/rest/impl/RestApiServiceImpl.java | 23 +-
.../main/java/org/apache/iotdb/db/qp/Planner.java | 38 ++
.../apache/iotdb/db/qp/executor/PlanExecutor.java | 6 +-
.../logical/sys/CreateContinuousQueryOperator.java | 8 +-
.../apache/iotdb/db/qp/physical/crud/UDTFPlan.java | 12 +
.../qp/physical/sys/CreateContinuousQueryPlan.java | 24 +-
.../apache/iotdb/db/qp/sql/IoTDBSqlVisitor.java | 6 +-
.../iotdb/db/query/context/QueryContext.java | 9 -
.../db/query/control/QueryResourceManager.java | 4 +-
.../iotdb/db/query/control/SessionManager.java | 10 +-
.../dataset/RawQueryDataSetWithValueFilter.java | 52 +-
.../query/dataset/ShowContinuousQueriesResult.java | 21 +-
.../db/query/dataset/UDTFAlignByTimeDataSet.java | 2 +
.../apache/iotdb/db/query/dataset/UDTFDataSet.java | 2 +
.../db/query/dataset/UDTFNonAlignDataSet.java | 2 +
.../query/dataset/groupby/GroupByFillDataSet.java | 20 +-
.../groupby/GroupByWithValueFilterDataSet.java | 3 +-
.../groupby/LocalAlignedGroupByExecutor.java | 2 +-
.../dataset/groupby/LocalGroupByExecutor.java | 2 +-
.../db/query/executor/AggregationExecutor.java | 20 +-
.../iotdb/db/query/executor/FillQueryExecutor.java | 18 +-
.../iotdb/db/query/executor/LastQueryExecutor.java | 4 +-
.../iotdb/db/query/executor/QueryRouter.java | 10 +-
.../db/query/executor/RawDataQueryExecutor.java | 83 +++-
.../iotdb/db/query/executor/UDFQueryExecutor.java | 21 +-
.../iotdb/db/query/executor/fill/LinearFill.java | 8 +-
.../iotdb/db/query/executor/fill/PreviousFill.java | 5 +-
.../iotdb/db/query/expression/ResultColumn.java | 5 +
.../iotdb/db/query/reader/series/SeriesReader.java | 19 +-
.../query/timegenerator/ServerTimeGenerator.java | 3 +-
.../query/udf/core/layer/RawQueryInputLayer.java | 4 +-
.../java/org/apache/iotdb/db/service/IoTDB.java | 8 +-
.../org/apache/iotdb/db/service/SettleService.java | 1 -
.../iotdb/db/service/basic/ServiceProvider.java | 18 +-
.../iotdb/db/service/metrics/MetricsService.java | 164 ++-----
.../db/service/thrift/impl/TSServiceImpl.java | 23 +-
.../iotdb/db/sync/sender/transfer/SyncClient.java | 2 +
.../apache/iotdb/db/tools/TsFileSelfCheckTool.java | 190 ++++++++
.../apache/iotdb/db/tools/TsFileSketchTool.java | 1 +
.../apache/iotdb/db/utils/EnvironmentUtils.java | 2 +-
.../org/apache/iotdb/db/utils/FileLoaderUtils.java | 3 -
.../services/org.apache.iotdb.metrics.Reporter | 4 +-
.../SizeTieredCompactionRecoverTest.java | 11 +-
.../engine/modification/DeletionFileNodeTest.java | 4 +-
.../storagegroup/StorageGroupProcessorTest.java | 19 +-
.../iotdb/db/metadata/MManagerBasicTest.java | 75 +++
.../org/apache/iotdb/db/metadata/MTreeTest.java | 13 +-
.../iotdb/db/tools/TsFileSelfCheckToolTest.java | 239 ++++++++++
server/src/test/resources/iotdb-rest.properties | 5 +-
.../main/java/org/apache/iotdb/session/Config.java | 4 +
.../java/org/apache/iotdb/session/Session.java | 66 ++-
.../apache/iotdb/session/SessionConnection.java | 1 +
.../org/apache/iotdb/session/util/Version.java | 8 +-
.../apache/iotdb/session/SessionCacheLeaderUT.java | 3 +-
.../java/org/apache/iotdb/session/SessionTest.java | 4 +
session/src/test/resources/iotdb-rest.properties | 5 +-
.../test/java/org/apache/iotdb/db/sql/Cases.java | 12 +-
.../TsFileStatisticsMistakesException.java | 7 +-
.../file/metadata/statistics/BinaryStatistics.java | 15 +
.../metadata/statistics/BooleanStatistics.java | 17 +
.../file/metadata/statistics/DoubleStatistics.java | 20 +
.../file/metadata/statistics/FloatStatistics.java | 20 +
.../metadata/statistics/IntegerStatistics.java | 19 +
.../file/metadata/statistics/LongStatistics.java | 19 +
.../iotdb/tsfile/read/TsFileCheckStatus.java | 7 +-
.../iotdb/tsfile/read/TsFileSequenceReader.java | 135 ++++++
.../apache/iotdb/tsfile/read/common/RowRecord.java | 18 +
206 files changed, 6401 insertions(+), 2644 deletions(-)
create mode 100644 docs/UserGuide/System-Tools/TsFileSelfCheck-Tool.md
create mode 100644 docs/zh/UserGuide/System-Tools/TsFileSelfCheck-Tool.md
copy
integration/src/test/java/org/apache/iotdb/db/integration/aligned/{IoTDBLastQuery2IT.java
=> IoTDBRawQueryWithoutValueFilter3IT.java} (92%)
delete mode 100644
metrics/dropwizard-metrics/src/test/java/org/apache/iotdb/metrics/dropwizard/DropwizardMetricTest.java
delete mode 100644
metrics/dropwizard-metrics/src/test/java/org/apache/iotdb/metrics/dropwizard/DropwizardMetricTestPlan.java
delete mode 100644
metrics/dropwizard-metrics/src/test/java/org/apache/iotdb/metrics/dropwizard/PrometheusRunTest.java
delete mode 100644
metrics/dropwizard-metrics/src/test/resources/iotdb-metric.yml
copy
metrics/interface/src/main/java/org/apache/iotdb/metrics/{impl/DoNothingCounter.java
=> DoNothingMetricService.java} (77%)
copy
metrics/interface/src/main/java/org/apache/iotdb/metrics/{utils/PredefinedMetric.java
=> config/ReloadLevel.java} (84%)
rename metrics/interface/src/{main/test =>
test/java}/org/apache/iotdb/metrics/config/MetricConfigTest.java (78%)
rename metrics/interface/src/{main => }/test/resources/iotdb-metric.yml (92%)
delete mode 100644
metrics/micrometer-metrics/src/test/java/org/apache/iotdb/metrics/micrometer/MicrometerMetricTest.java
delete mode 100644
metrics/micrometer-metrics/src/test/java/org/apache/iotdb/metrics/micrometer/MicrometerMetricTestPlan.java
delete mode 100644
metrics/micrometer-metrics/src/test/resources/iotdb-metric.yml
create mode 100644
server/src/main/java/org/apache/iotdb/db/engine/cq/ContinuousQuerySchemaCheckTask.java
rename server/src/main/java/org/apache/iotdb/db/{ =>
engine}/cq/ContinuousQueryService.java (83%)
rename server/src/main/java/org/apache/iotdb/db/{ =>
engine}/cq/ContinuousQueryTask.java (52%)
rename server/src/main/java/org/apache/iotdb/db/{ =>
engine}/cq/ContinuousQueryTaskPoolManager.java (98%)
copy
server/src/main/java/org/apache/iotdb/db/exception/{WriteLockFailedException.java
=> TsFileTimeseriesMetadataException.java} (81%)
copy
metrics/dropwizard-metrics/src/main/java/org/apache/iotdb/metrics/dropwizard/reporter/DropwizardJmxReporter.java
=>
server/src/main/java/org/apache/iotdb/db/metrics/dropwizard/reporter/DropwizardIoTDBReporter.java
(58%)
create mode 100644
server/src/main/java/org/apache/iotdb/db/metrics/dropwizard/reporter/IoTDBReporter.java
create mode 100644
server/src/main/java/org/apache/iotdb/db/metrics/metricsUtils.java
create mode 100644
server/src/main/java/org/apache/iotdb/db/metrics/micrometer/registry/IoTDBMeterRegistry.java
copy
metrics/micrometer-metrics/src/main/java/org/apache/iotdb/metrics/micrometer/reporter/IoTDBJmxConfig.java
=>
server/src/main/java/org/apache/iotdb/db/metrics/micrometer/registry/IoTDBRegistryConfig.java
(75%)
copy
metrics/micrometer-metrics/src/main/java/org/apache/iotdb/metrics/micrometer/reporter/MicrometerJmxReporter.java
=>
server/src/main/java/org/apache/iotdb/db/metrics/micrometer/reporter/MicrometerIoTDBReporter.java
(62%)
create mode 100644
server/src/main/java/org/apache/iotdb/db/protocol/rest/handler/PhysicalPlanValidationHandler.java
create mode 100644
server/src/main/java/org/apache/iotdb/db/tools/TsFileSelfCheckTool.java
copy {metrics/dropwizard-metrics =>
server}/src/main/resources/META-INF/services/org.apache.iotdb.metrics.Reporter
(84%)
create mode 100644
server/src/test/java/org/apache/iotdb/db/tools/TsFileSelfCheckToolTest.java
copy tsfile/src/main/java/org/apache/iotdb/tsfile/fileSystem/FSType.java =>
session/src/main/java/org/apache/iotdb/session/util/Version.java (90%)
copy
server/src/main/java/org/apache/iotdb/db/exception/WriteLockFailedException.java
=>
tsfile/src/main/java/org/apache/iotdb/tsfile/exception/TsFileStatisticsMistakesException.java
(81%)
