This is an automated email from the ASF dual-hosted git repository.
jackietien pushed a change to branch auth
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from 031495a6301 Pipe: refactor connector packages and rename v1/v2 to
sync/async (#10840)
add da7e66db404 fix parameter schemaRegionPerNode load error (#10841)
add 80427a00c5f Enhance the event notification mechanism of
StatisticsService (#10829)
add 59a0dcf11a1 Refactoring DeleteOutdatedFileTask in WalNode (#10765)
add ccd73310fac fix inaccurate statistics for monitoring the number of
unseq Tsfile on cross compaction (#10651)
add 47a14313621 add memory estimator on inner space compaction (#10688)
add 7dd7cf78956 Fix a concurrent bug when recover last flush time and
compact (#10839)
add 3dc1490cfc8 [IOTDB-6108] Fix AlignedTVList memory calculation is
imprecise (#10827)
add 11af323b79d Pipe: Remove parameters from iotdb-common.properties that
users will not modify (#10851)
add 459a394a86b [IOTDB-6113] Load: Fix setDatabase does not print logs and
set failure except DATABASE_ALREADY_EXISTS (#10854)
add 98128a4f02e Update the description of schemaRegionPerNode in
common.properties (#10857)
add 0133c139538 [IOTDB-6111] Fix Region creation bugs (#10844)
add f7ca23cbd00 Fix SchemaFileSketchTool is not found
add cdcb4a691d3 [IOTDB-6112] Fix Limit & Offset push down doesn't take
effect while there exist time filter
add 8e0256658e0 Remove useless dependency bugsnag (#10861)
add b0bd6065a42 Replace dependency commons-lang to commons-lang3
add 621b66a871e Remove useless dependency h2-mvstore
add 44d6bec0f39 Pipe IT: Changed IoTDBPipeDemoIT timeout to 10 mins to
allow pipe-internal retries when unstable cases occuring (#10867)
add 16a9f3fe760 Bump
com.googlecode.maven-download-plugin:download-maven-plugin (#10747)
add 89645c6889f Delete useless airline, concurrent-200 and jaxb-runtime
dependencies (#10866)
add 3e9ddc77c86 Pipe IT: fix the usage of Awaitility.awit (#10869)
add b445540475a [IOTDB-6116] Disassociate the IoTConsensus retry logic
from the forkjoinPool (#10872)
add 679c495b043 [RatisConsensus] Unify read timeout to Thrift connection
timeout (#10876)
add 9acb3594a18 Fix Region migrate bug (#10862)
add 80f9a80e50f [IOTDB-6118] Pipe: Support transfer data through air gap
(IoTDBAirGapConnector) (#10853)
add 0a90c45486d Pipe IT: Stablelized pipe IT by prohibiting exceptions
from assertEqual function (#10881)
add 0b28f31379d fix dataMigrationExample error (#10886)
add 6bef0b0b232 fix: Hopefully fixed the build of the cpp-example on
windows systems (#10891)
add 709e7ef8694 Upgrade guava to avoid vulnerability issue (#10892)
add 052ff28cf89 Override existing jar instead of deleting it when
registering UDF
add 97e083efcec [IOTDB-6115] Fix Limit & Offset push down doesn't take
effect while there exist null value
add a45248d4959 Rename iotdb-thrift module to iotdb-thrift-datanode
(#10884)
add 29e038d6a96 Fix memory leak when deleting database in PB_Tree mode
(#10885)
add 38b36006b2a [IOTDB-6117] Pipe: Optimize RPC requests from CN to DN. CN
send exactly one pipeMeta to each DN upon create/start/stop/drop pipe (#10875)
add 5f6db2e308a Fix some ITs of UDFManagement
add 2c2f40bdb45 Implement intersect with prefix pattern for PartialPath
and PathPatternTree
add af250e0b9ca [IOTDB-6122] Pipe: Progress Index markers an incorrect
value when load a tsfile with tsfileResource (#10908)
add 30a006118d0 reduce wal log output level (#10873)
add 144e5428fbb [IOTDB-6114] Pipe: Support multi-cluster data sync (#10868)
add 8d79a5e4e7c Add hot load compaction configs (#10758)
add 0cd30225af5 Move SchemaConstant to common package
add 40617cd3940 Revert removed dependencies to avoid write performance
declining (#10921)
add d614139f71f Optimize client cpp package (#10923)
add 51e6d17b821 [IOTDB-6120] Push down limit/offset in query with group by
time
add 23340404c99 [IOTDB-6080] Pipe: Introduce batch mode for tablet
transferring (#10912)
add 1dce787c39e [Metric] Fix flush point statistics (#10915)
add 6357533a25c [IOTDB-6125] Fix DataPartition allocation bug when insert
big batch data (#10924)
add e891a5d1f31 [IOTDB-6121] Consensus layer interface and exception
handling refactoring (#10896)
add fe7b3a4dff2 [IOTDB-6125] Fix DataPartition allocation bug when insert
big batch data (#10938)
add 332521a3241 [IOTDB-6083] Pipe: Fix subscrption running with the
pattern option causing OOM & make PipeRawTabletInsertionEvent able to report
progress to avoid losing data (#10865)
add 6f7751bc1c1 [IOTDB-6082] Improve disk space metrics (#10925)
add 55bea920618 Pipe: Swapped the start order of realtimeExtractor and
historicalExtractor to avoid losing data (#10927)
add 4341c152a77 [IOTDB-6103] Adding count_time aggregation feature (#10756)
add e9fe0b1082d Feature/simplified tsfile examples (#10906)
add 97a5ab693b4 Feature/test externalized iotdb tools thrift (#10893)
add 810ec62f824 [IOTDB-6124] Add scope filter parameter for tree visitor
framework
No new revisions were added by this update.
Summary of changes:
.gitattributes | 17 +
.github/workflows/client-cpp.yml | 8 +-
CONTRIBUTING.md | 2 +-
LICENSE-binary | 4 +-
distribution/pom.xml | 22 +-
distribution/src/assembly/client-cpp.xml | 2 +-
example/client-cpp-example/pom.xml | 4 +-
example/client-cpp-example/src/CMakeLists.txt | 10 +-
.../src/main/java/org/apache/iotdb/SSLClient.java | 2 +-
.../org/apache/iotdb/DataMigrationExample.java | 3 +-
.../tsfile/TsFileWriteAlignedWithTSRecord.java | 8 +-
.../iotdb/tsfile/TsFileWriteAlignedWithTablet.java | 6 +-
.../iotdb/tsfile/TsFileWriteWithTSRecord.java | 8 +-
.../apache/iotdb/tsfile/TsFileWriteWithTablet.java | 6 +-
.../it/env/cluster/config/MppCommonConfig.java | 6 +
.../env/cluster/config/MppSharedCommonConfig.java | 7 +
.../it/env/remote/config/RemoteCommonConfig.java | 4 +
.../org/apache/iotdb/itbase/env/CommonConfig.java | 2 +
.../partition/IoTDBAutoRegionGroupExtensionIT.java | 17 +-
.../IoTDBAutoRegionGroupExtensionIT2.java | 160 +++++++
.../it/partition/IoTDBPartitionCreationIT.java | 127 +++---
.../it/partition/IoTDBPartitionDurableIT.java | 6 +-
.../aggregation/IoTDBCountTimeAlignedDeviceIT.java | 304 +++++++++++++
.../iotdb/db/it/aggregation/IoTDBCountTimeIT.java | 339 +++++++++++++++
.../iotdb/db/it/udf/IoTDBUDFManagementIT.java | 50 ++-
.../org/apache/iotdb/db/it/utils/TestUtils.java | 84 ++--
...TDBPipeDemoIT.java => IoTDBPipeDataSyncIT.java} | 35 +-
...leDemoIT.java => IoTDBPipeSingleEnvDemoIT.java} | 2 +-
iotdb-client/client-cpp/pom.xml | 8 +-
iotdb-client/client-py/pom.xml | 2 +-
.../confignode/client/DataNodeRequestType.java | 3 +-
.../client/async/AsyncDataNodeClientPool.java | 9 +-
.../client/async/handlers/AsyncClientHandler.java | 3 +-
.../confignode/conf/ConfigNodeDescriptor.java | 6 +-
.../confignode/manager/ClusterQuotaManager.java | 52 +--
.../iotdb/confignode/manager/ConfigManager.java | 18 +-
.../iotdb/confignode/manager/ModelManager.java | 75 ++--
.../confignode/manager/PermissionManager.java | 39 +-
.../iotdb/confignode/manager/TriggerManager.java | 56 +--
.../iotdb/confignode/manager/UDFManager.java | 28 +-
.../manager/consensus/ConsensusManager.java | 95 +++--
.../iotdb/confignode/manager/cq/CQManager.java | 42 +-
.../confignode/manager/cq/CQScheduleTask.java | 26 +-
.../iotdb/confignode/manager/load/LoadManager.java | 8 +-
.../manager/load/balancer/PartitionBalancer.java | 21 +-
.../manager/load/balancer/RegionBalancer.java | 17 +-
.../manager/load/balancer/RouteBalancer.java | 4 +-
.../partition/DataPartitionPolicyTable.java | 27 ++
.../region/GreedyRegionGroupAllocator.java | 41 +-
.../load/cache/node/DataNodeHeartbeatCache.java | 4 +-
.../manager/load/cache/region/RegionCache.java | 5 +-
.../load/cache/region/RegionHeartbeatSample.java | 5 +
.../manager/load/service/StatisticsService.java | 16 +-
.../iotdb/confignode/manager/node/NodeManager.java | 73 +++-
.../manager/partition/PartitionManager.java | 227 ++++++----
.../manager/pipe/plugin/PipePluginCoordinator.java | 23 +-
.../pipe/runtime/PipeLeaderChangeHandler.java | 4 +-
.../manager/pipe/task/PipeTaskCoordinator.java | 14 +-
.../manager/schema/ClusterSchemaManager.java | 284 ++++++++++---
.../partition/DatabasePartitionTable.java | 4 +-
.../persistence/partition/RegionGroup.java | 11 +
.../confignode/persistence/pipe/PipeTaskInfo.java | 9 +
.../persistence/schema/ClusterSchemaInfo.java | 6 +-
.../confignode/persistence/schema/ConfigMTree.java | 10 +-
.../schema/mnode/info/ConfigMNodeInfo.java | 2 +-
.../procedure/env/ConfigNodeProcedureEnv.java | 60 ++-
.../procedure/env/DataNodeRemoveHandler.java | 14 +-
.../procedure/impl/cq/CreateCQProcedure.java | 113 +++--
.../procedure/impl/model/CreateModelProcedure.java | 18 +-
.../procedure/impl/model/DropModelProcedure.java | 7 +-
.../impl/pipe/AbstractOperatePipeProcedureV2.java | 31 +-
.../pipe/plugin/CreatePipePluginProcedure.java | 27 +-
.../impl/pipe/plugin/DropPipePluginProcedure.java | 13 +-
.../runtime/PipeHandleLeaderChangeProcedure.java | 19 +-
.../runtime/PipeHandleMetaChangeProcedure.java | 23 +-
.../impl/pipe/task/CreatePipeProcedureV2.java | 55 ++-
.../impl/pipe/task/DropPipeProcedureV2.java | 23 +-
.../impl/pipe/task/StartPipeProcedureV2.java | 45 +-
.../impl/pipe/task/StopPipeProcedureV2.java | 45 +-
.../impl/schema/DeleteDatabaseProcedure.java | 3 +-
.../impl/schema/SetTemplateProcedure.java | 77 +++-
.../statemachine/CreateRegionGroupsProcedure.java | 7 +-
.../impl/trigger/CreateTriggerProcedure.java | 22 +-
.../procedure/store/ConfigProcedureStore.java | 13 +-
.../iotdb/confignode/service/ConfigNode.java | 3 +-
.../thrift/ConfigNodeRPCServiceProcessor.java | 12 +-
iotdb-core/consensus/pom.xml | 2 +-
.../org/apache/iotdb/consensus/IConsensus.java | 166 +++++---
.../iotdb/consensus/common/ConsensusGroup.java | 1 -
.../org/apache/iotdb/consensus/common/Peer.java | 1 -
.../org/apache/iotdb/consensus/common/Utils.java | 13 +-
.../common/request/ByteBufferConsensusRequest.java | 12 +-
.../common/response/ConsensusGenericResponse.java | 64 ---
.../common/response/ConsensusReadResponse.java | 69 ---
.../common/response/ConsensusWriteResponse.java | 82 ----
.../iotdb/consensus/config/ConsensusConfig.java | 18 +-
.../PeerNotInConsensusGroupException.java | 6 +-
.../exception/RatisUnderRecoveryException.java | 3 +-
.../apache/iotdb/consensus/iot/IoTConsensus.java | 237 +++++------
.../consensus/iot/IoTConsensusServerImpl.java | 8 +
.../consensus/iot/client/DispatchLogHandler.java | 52 ++-
.../consensus/iot/logdispatcher/LogDispatcher.java | 4 +
.../iotdb/consensus/ratis/RatisConsensus.java | 261 +++++-------
.../iotdb/consensus/ratis/SnapshotStorage.java | 2 +-
.../iotdb/consensus/simple/SimpleConsensus.java | 142 +++----
...verImpl.java => SimpleConsensusServerImpl.java} | 4 +-
.../apache/iotdb/consensus/iot/ReplicateTest.java | 18 +-
.../apache/iotdb/consensus/iot/StabilityTest.java | 106 ++++-
.../iotdb/consensus/ratis/RatisConsensusTest.java | 132 +++---
.../iotdb/consensus/ratis/RecoverReadTest.java | 49 ++-
.../apache/iotdb/consensus/ratis/TestUtils.java | 18 +-
.../iotdb/consensus/simple/RecoveryTest.java | 22 +-
.../consensus/simple/SimpleConsensusTest.java | 228 +++++-----
iotdb-core/datanode/pom.xml | 12 +-
.../resources/conf/iotdb-datanode.properties | 2 +-
.../resources/tools/schema/print-pb-tree-file.sh | 2 +-
.../org/apache/iotdb/db/audit/AuditLogger.java | 1 +
.../org/apache/iotdb/db/auth/AuthorityChecker.java | 1 +
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 40 +-
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 95 ++++-
.../db/consensus/DataRegionConsensusImpl.java | 291 +++++++------
.../db/consensus/SchemaRegionConsensusImpl.java | 208 ++++-----
.../dataregion/DataExecutionVisitor.java | 23 +
.../metadata/MNodeTypeMismatchException.java | 2 +-
.../db/pipe/agent/receiver/PipeReceiverAgent.java | 70 +--
.../db/pipe/agent/runtime/PipeRuntimeAgent.java | 14 +-
.../iotdb/db/pipe/agent/task/PipeTaskAgent.java | 131 ++++--
.../config/constant/PipeConnectorConstant.java | 15 +
.../config/constant/PipeExtractorConstant.java | 4 +
.../payload/airgap/AirGapOneByteResponse.java} | 13 +-
.../airgap/AirGapPseudoTPipeTransferRequest.java} | 9 +-
.../payload/evolvable/PipeRequestType.java | 2 +
...oTDBThriftAsyncPipeTransferBatchReqBuilder.java | 98 +++++
...IoTDBThriftSyncPipeTransferBatchReqBuilder.java | 89 ++++
.../builder/PipeTransferBatchReqBuilder.java | 64 +++
.../evolvable/request/PipeTransferBatchReq.java | 172 ++++++++
.../request/PipeTransferFilePieceReq.java | 29 +-
.../evolvable/request/PipeTransferFileSealReq.java | 26 +-
.../request/PipeTransferHandshakeReq.java | 22 +-
.../request/PipeTransferInsertNodeReq.java | 26 +-
.../evolvable/request/PipeTransferTabletReq.java | 140 +++---
...TDBThriftConnector.java => IoTDBConnector.java} | 17 +-
...sion.java => IoTDBConnectorRequestVersion.java} | 6 +-
.../protocol/airgap/IoTDBAirGapConnector.java | 344 ++++++++++++++-
.../protocol/legacy/IoTDBLegacyPipeConnector.java | 2 +-
.../thrift/async/IoTDBThriftAsyncConnector.java | 143 +++++--
...eTransferTabletBatchInsertionEventHandler.java} | 63 ++-
.../PipeTransferTabletInsertionEventHandler.java | 2 +-
.../thrift/sync/IoTDBThriftSyncConnector.java | 109 +++--
.../apache/iotdb/db/pipe/event/EnrichedEvent.java | 16 +-
.../db/pipe/event/common/row/PipeRowCollector.java | 18 +-
.../tablet/PipeInsertNodeTabletInsertionEvent.java | 32 +-
.../common/tablet/PipeRawTabletInsertionEvent.java | 102 ++++-
.../tablet/TabletInsertionDataContainer.java | 27 +-
.../common/tsfile/PipeTsFileInsertionEvent.java | 21 +-
.../tsfile/TsFileInsertionDataContainer.java | 34 +-
.../db/pipe/event/realtime/PipeRealtimeEvent.java | 8 +-
.../event/realtime/PipeRealtimeEventFactory.java | 10 +-
.../pipe/extractor/IoTDBDataRegionExtractor.java | 7 +-
.../PipeHistoricalDataRegionTsFileExtractor.java | 2 +
.../realtime/PipeRealtimeDataRegionExtractor.java | 10 +
.../realtime/assigner/PipeDataRegionAssigner.java | 4 +
.../listener/PipeInsertionDataNodeListener.java | 6 +-
.../db/pipe/processor/PipeDoNothingProcessor.java | 49 +--
.../pipe/receiver/airgap/IoTDBAirGapReceiver.java | 180 +++++++-
.../receiver/airgap/IoTDBAirGapReceiverAgent.java | 97 +++++
...iver.java => IoTDBLegacyPipeReceiverAgent.java} | 20 +-
.../pipe/receiver/thrift/IoTDBThriftReceiver.java | 4 +-
.../thrift/IoTDBThriftReceiverAgent.java} | 38 +-
.../receiver/thrift/IoTDBThriftReceiverV1.java | 121 ++++--
.../pipe/task/connection/PipeEventCollector.java | 31 +-
.../connector/PipeConnectorSubtaskManager.java | 4 +
.../iotdb/db/protocol/client/ConfigNodeClient.java | 2 +-
.../iotdb/db/protocol/session/SessionManager.java | 2 +-
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 16 +-
.../impl/DataNodeInternalRPCServiceImpl.java | 161 ++++---
.../thrift/impl/DataNodeRegionManager.java | 39 +-
.../common/schematree/ClusterSchemaTree.java | 2 +-
.../visitor/SchemaTreeDeviceVisitor.java | 6 +
.../visitor/SchemaTreeMeasurementVisitor.java | 7 +
.../schematree/visitor/SchemaTreeVisitor.java | 36 ++
.../visitor/SchemaTreeVisitorFactory.java | 23 +
.../execution/aggregation/AccumulatorFactory.java | 3 +-
.../execution/aggregation/Aggregator.java | 13 +-
.../execution/aggregation/CountAccumulator.java | 1 -
...tAccumulator.java => CountTimeAccumulator.java} | 21 +-
.../SlidingWindowAggregatorFactory.java | 1 +
.../execution/executor/RegionReadExecutor.java | 18 +-
.../execution/executor/RegionWriteExecutor.java | 111 +++--
.../execution/load/LoadTsFileManager.java | 12 +-
.../operator/process/AbstractIntoOperator.java | 4 -
.../operator/process/DeviceViewIntoOperator.java | 8 +-
.../execution/operator/process/IntoOperator.java | 9 +-
.../operator/schema/source/DeviceSchemaSource.java | 2 +-
.../schema/source/LogicalViewSchemaSource.java | 2 +-
.../schema/source/TimeSeriesSchemaSource.java | 2 +-
.../AbstractSeriesAggregationScanOperator.java | 10 +-
.../operator/source/AlignedSeriesScanOperator.java | 9 +-
.../operator/source/AlignedSeriesScanUtil.java | 85 ++--
.../db/queryengine/plan/analyze/Analysis.java | 26 +-
.../queryengine/plan/analyze/AnalyzeVisitor.java | 468 +++++++++++++--------
.../plan/analyze/ExpressionAnalyzer.java | 2 +
.../plan/analyze/ExpressionTypeAnalyzer.java | 4 +-
.../queryengine/plan/analyze/ExpressionUtils.java | 3 +-
.../queryengine/plan/execution/QueryExecution.java | 4 +-
.../plan/execution/config/ConfigTaskVisitor.java | 10 +-
.../config/executor/ClusterConfigTaskExecutor.java | 19 +-
.../config/executor/IConfigTaskExecutor.java | 10 +-
.../execution/config/sys/pipe/CreatePipeTask.java | 2 +-
.../execution/config/sys/pipe/DropPipeTask.java | 2 +-
.../execution/config/sys/pipe/ShowPipeTask.java | 2 +-
.../execution/config/sys/pipe/StartPipeTask.java | 2 +-
.../execution/config/sys/pipe/StopPipeTask.java | 2 +-
.../plan/expression/ExpressionFactory.java | 7 +
.../plan/expression/binary/BinaryExpression.java | 4 +
.../plan/expression/leaf/TimestampOperand.java | 5 +-
.../plan/expression/multi/FunctionExpression.java | 18 +
...java => CountTimeAggregationAmountVisitor.java} | 33 +-
.../visitor/ExpressionNormalizeVisitor.java | 3 +-
.../visitor/GetMeasurementExpressionVisitor.java | 3 +-
.../visitor/LowercaseNormalizeVisitor.java | 3 +-
.../cartesian/BindSchemaForExpressionVisitor.java | 19 +
.../cartesian/BindSchemaForPredicateVisitor.java | 18 +
...catDeviceAndBindSchemaForExpressionVisitor.java | 23 +-
.../plan/optimization/LimitOffsetPushDown.java | 117 ++++++
.../db/queryengine/plan/parser/ASTVisitor.java | 36 +-
.../plan/planner/LogicalPlanBuilder.java | 72 +++-
.../plan/planner/LogicalPlanVisitor.java | 98 ++++-
.../plan/planner/OperatorTreeGenerator.java | 74 ++--
.../plan/planner/distribution/SourceRewriter.java | 8 +-
.../plan/planner/plan/node/PlanGraphPrinter.java | 1 +
.../plan/planner/plan/node/PlanNodeType.java | 7 +-
.../plan/planner/plan/node/PlanVisitor.java | 5 +
.../planner/plan/node/load/LoadTsFileNode.java | 4 -
.../plan/node/source/AlignedSeriesScanNode.java | 37 +-
.../plan/node/write/InsertMultiTabletsNode.java | 6 +
.../plan/planner/plan/node/write/InsertNode.java | 12 +-
.../planner/plan/node/write/InsertRowsNode.java | 15 +-
.../plan/node/write/InsertRowsOfOneDeviceNode.java | 6 +
.../plan/node/write/PipeEnrichedInsertNode.java | 276 ++++++++++++
.../scheduler/load/LoadTsFileDispatcherImpl.java | 11 +-
.../plan/scheduler/load/LoadTsFileScheduler.java | 11 +-
.../queryengine/plan/statement/StatementType.java | 1 +
.../plan/statement/StatementVisitor.java | 58 ++-
.../plan/statement/component/OrderByComponent.java | 8 +-
.../plan/statement/crud/InsertBaseStatement.java | 3 +-
.../plan/statement/crud/LoadTsFileStatement.java | 11 +
.../crud/PipeEnrichedInsertBaseStatement.java | 224 ++++++++++
.../crud/PipeEnrichedLoadTsFileStatement.java | 132 ++++++
.../plan/statement/crud/QueryStatement.java | 109 ++++-
.../pipe/CreatePipeStatement.java | 2 +-
.../{sys => metadata}/pipe/DropPipeStatement.java | 2 +-
.../{sys => metadata}/pipe/ShowPipesStatement.java | 2 +-
.../{sys => metadata}/pipe/StartPipeStatement.java | 2 +-
.../{sys => metadata}/pipe/StopPipeStatement.java | 2 +-
.../schemaregion/SchemaRegionLoader.java | 2 +-
.../schemaregion/impl/SchemaRegionMemoryImpl.java | 2 +-
.../schemaregion/impl/SchemaRegionPBTreeImpl.java | 2 +-
.../mtree/impl/mem/MTreeBelowSGMemoryImpl.java | 2 +-
.../mtree/impl/mem/mnode/info/DeviceInfo.java | 2 +-
.../mnode/iterator/AbstractTraverserIterator.java | 2 +-
.../impl/mem/snapshot/MemMTreeSnapshotUtil.java | 16 +-
.../mtree/impl/pbtree/CachedMTreeStore.java | 1 +
.../mtree/impl/pbtree/MTreeBelowSGCachedImpl.java | 2 +-
.../mtree/impl/pbtree/cache/CacheManager.java | 1 +
.../impl/pbtree/cache/CacheMemoryManager.java | 4 +
.../mtree/impl/pbtree/schemafile/RecordUtils.java | 2 +-
.../mtree/impl/pbtree/schemafile/SchemaFile.java | 2 +-
.../mtree/loader/MNodeFactoryLoader.java | 2 +-
.../schemaregion/mtree/traverser/Traverser.java | 48 ++-
.../schemaengine/schemaregion/tag/TagManager.java | 2 +-
.../schemaregion/utils/ResourceByPathUtils.java | 30 +-
.../schemaengine/schemaregion/view/ViewPaths.java | 2 +-
.../template/ClusterTemplateManager.java | 3 +-
.../java/org/apache/iotdb/db/service/DataNode.java | 16 +-
.../apache/iotdb/db/service/IoTDBShutdownHook.java | 25 +-
.../iotdb/db/service/RegionMigrateService.java | 101 ++---
.../db/service/metrics/DataNodeMetricsHelper.java | 3 +-
.../metrics/IoTDBInternalLocalReporter.java | 6 +-
.../iotdb/db/storageengine/StorageEngine.java | 5 +-
.../db/storageengine/dataregion/DataRegion.java | 7 +-
.../dataregion/HashLastFlushTimeMap.java | 15 +-
.../exception/CompactionExceptionHandler.java | 11 +-
.../execute/task/AbstractCompactionTask.java | 2 +
.../execute/task/CrossSpaceCompactionTask.java | 24 +-
.../execute/task/InnerSpaceCompactionTask.java | 58 ++-
.../compaction/execute/utils/CompactionUtils.java | 53 +++
.../compaction/schedule/CompactionTaskManager.java | 1 -
.../estimator/AbstractCompactionEstimator.java | 3 +-
.../estimator/AbstractInnerSpaceEstimator.java | 74 +++-
.../estimator/CompactionEstimateUtils.java | 96 +++++
.../FastCompactionInnerCompactionEstimator.java | 52 +++
.../compaction/selector/estimator/FileInfo.java | 48 +++
.../ReadChunkInnerCompactionEstimator.java | 59 +++
.../ReadPointCrossCompactionEstimator.java | 81 +---
.../dataregion/flush/MemTableFlushTask.java | 59 ++-
.../memtable/AlignedReadOnlyMemChunk.java | 12 +-
.../dataregion/memtable/TsFileProcessor.java | 149 ++++---
.../read/reader/chunk/DiskAlignedChunkLoader.java | 13 +-
.../read/reader/chunk/MemAlignedChunkLoader.java | 11 +-
.../read/reader/chunk/MemAlignedChunkReader.java | 8 +-
.../read/reader/chunk/MemAlignedPageReader.java | 41 +-
.../metadata/DiskAlignedChunkMetadataLoader.java | 16 +-
.../metadata/MemAlignedChunkMetadataLoader.java | 19 +-
.../dataregion/tsfile/TsFileManager.java | 17 +
.../dataregion/tsfile/TsFileResource.java | 25 +-
.../wal/checkpoint/CheckpointManager.java | 2 +-
.../storageengine/dataregion/wal/node/WALNode.java | 245 +++++++----
.../file/UnsealedTsFileRecoverPerformer.java | 2 +-
.../quotas/DataNodeThrottleQuotaManager.java | 1 +
.../rescon/quotas/DefaultOperationQuota.java | 14 +-
.../db/trigger/executor/TriggerFireVisitor.java | 20 +
.../org/apache/iotdb/db/utils/FileLoaderUtils.java | 63 ++-
.../org/apache/iotdb/db/utils/SchemaUtils.java | 3 +
.../apache/iotdb/db/utils/TypeInferenceUtils.java | 6 +-
.../iotdb/db/utils/constant/SqlConstant.java | 3 +
.../db/utils/datastructure/AlignedTVList.java | 46 ++
.../mtree/schemafile/SchemaFileLogTest.java | 2 +-
.../schemaRegion/SchemaRegionManagementTest.java | 2 +-
.../db/pipe/extractor/PipeRealtimeExtractTest.java | 3 +-
.../common/schematree/ClusterSchemaTreeTest.java | 178 ++++++++
.../ClusterSchemaTreeTestWithRelease.java | 16 +
.../MockSchemaTreeMeasurementVisitor.java | 6 +
.../execution/aggregation/AccumulatorTest.java | 44 ++
.../execution/executor/RegionReadExecutorTest.java | 31 +-
.../executor/RegionWriteExecutorTest.java | 24 +-
.../operator/AlignedSeriesScanOperatorTest.java | 15 +-
.../execution/operator/OperatorMemoryTest.java | 4 +-
.../queryengine/plan/analyze/AnalyzeFailTest.java | 72 ++++
.../plan/optimization/LimitOffsetPushDownTest.java | 271 ++++++++++++
.../plan/plan/QueryLogicalPlanUtil.java | 5 +-
.../plan/statement/sys/pipe/PipeStatementTest.java | 10 +-
.../scheduler/load/LoadTsFileSchedulerTest.java | 3 +-
.../DataNodeInternalRPCServiceImplTest.java | 8 +-
.../compaction/CompactionSchedulerTest.java | 1 +
.../inner/InnerSpaceCompactionExceptionTest.java | 4 +-
.../utils/CompactionTaskMemCostEstimatorTest.java | 94 +++++
.../dataregion/memtable/TsFileProcessorTest.java | 97 +++++
.../reader/chunk/MemAlignedChunkLoaderTest.java | 2 +-
.../org/apache/iotdb/db/tools/MLogParserTest.java | 2 +-
.../iotdb/db/tools/PBTreeFileSketchTest.java | 2 +-
.../datanode1conf/iotdb-common.properties | 2 +-
.../datanode2conf/iotdb-common.properties | 3 +-
.../datanode3conf/iotdb-common.properties | 2 +-
.../iotdb/metrics/AbstractMetricService.java | 45 +-
.../metrics/metricsets/system/SystemMetrics.java | 69 +--
.../metrics/reporter/iotdb/IoTDBReporter.java | 6 +-
iotdb-core/mlnode/pom.xml | 2 +-
iotdb-core/node-commons/pom.xml | 5 -
.../resources/conf/iotdb-common.properties | 86 +---
.../iotdb/commons/client/ClientPoolFactory.java | 2 +-
.../iotdb/commons/concurrent/ThreadName.java | 11 +-
.../apache/iotdb/commons/conf/CommonConfig.java | 42 +-
.../iotdb/commons/conf/CommonDescriptor.java | 19 +-
.../iotdb/commons/consensus/ConsensusGroupId.java | 38 +-
.../commons/executable/ExecutableManager.java | 11 +-
.../org/apache/iotdb/commons/path/PartialPath.java | 158 +++++--
.../iotdb/commons/path/PathPatternTreeUtils.java | 53 +++
.../apache/iotdb/commons/path/PathPatternUtil.java | 4 +-
.../apache/iotdb/commons/path/fa/FAFactory.java | 14 +-
.../apache/iotdb/commons/path/fa/IPatternFA.java | 17 +-
.../iotdb/commons/path/fa/dfa/PatternDFA.java | 60 +++
.../iotdb/commons/path/fa/dfa/graph/NFAGraph.java | 67 ++-
.../commons/path/fa/match/IStateMatchInfo.java | 2 +
.../commons/path/fa/match/StateMultiMatchInfo.java | 14 +-
.../path/fa/match/StateSingleMatchInfo.java | 10 +-
.../iotdb/commons/pipe/config/PipeConfig.java | 24 +-
.../pipe/plugin/builtin/BuiltinPipePlugin.java | 2 +
.../builtin/connector/IoTDBAirGapConnector.java} | 27 +-
.../connector/IoTDBLegacyPipeConnector.java | 54 +--
.../builtin/connector/IoTDBThriftConnector.java | 54 +--
...ipeConnector.java => PlaceholderConnector.java} | 12 +-
.../commons/pipe/task/meta/PipeMetaKeeper.java | 4 +
.../iotdb/commons/schema}/SchemaConstant.java | 18 +-
.../commons/schema/tree/AbstractTreeVisitor.java | 136 ++++--
.../iotdb/commons/schema/tree/ITreeNode.java | 4 +
.../apache/iotdb/commons/service/ServiceType.java | 1 +
.../iotdb/commons/structure/BalanceTreeMap.java | 4 +
.../udf/builtin/BuiltinAggregationFunction.java | 5 +-
.../apache/iotdb/commons/utils/StatusUtils.java | 2 +-
.../apache/iotdb/commons/path/PartialPathTest.java | 68 +++
.../iotdb/commons/path/PathPatternTreeTest.java | 121 ++++++
.../apache/iotdb/commons/path/PatternDFATest.java | 45 ++
.../file/metadata/AlignedTimeSeriesMetadata.java | 34 +-
.../read/TsFileAlignedSeriesReaderIterator.java | 2 +-
.../iotdb/tsfile/read/TsFileSequenceReader.java | 12 +-
.../read/reader/chunk/AlignedChunkReader.java | 26 +-
.../tsfile/read/reader/page/AlignedPageReader.java | 125 +++---
.../read/reader/series/FileSeriesReader.java | 2 +-
.../org/apache/iotdb/tsfile/utils/BytesUtils.java | 4 +-
.../org/apache/iotdb/tsfile/utils/PublicBAOS.java | 1 +
.../tsfile/write/chunk/AlignedChunkWriterImpl.java | 2 +-
.../tsfile/write/TsFileIntegrityCheckingTool.java | 3 +-
iotdb-protocol/pom.xml | 4 +-
.../thrift-commons/src/main/thrift/common.thrift | 3 +-
.../pom.xml | 4 +-
.../src/main/thrift/iotconsensus.thrift | 0
.../{thrift => thrift-datanode}/README.md | 0
iotdb-protocol/{thrift => thrift-datanode}/pom.xml | 4 +-
.../{thrift => thrift-datanode}/rpc-changelist.md | 0
.../src/main/thrift/client.thrift | 0
.../src/main/thrift/datanode.thrift | 11 +
pom.xml | 27 +-
tools/README.md | 183 --------
tools/pom.xml | 33 --
tools/thrift/pom.xml | 312 --------------
tools/thrift/src/main/assembly/thrift.xml | 64 ---
407 files changed, 10656 insertions(+), 4710 deletions(-)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/confignode/it/partition/IoTDBAutoRegionGroupExtensionIT2.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/aggregation/IoTDBCountTimeAlignedDeviceIT.java
create mode 100644
integration-test/src/test/java/org/apache/iotdb/db/it/aggregation/IoTDBCountTimeIT.java
rename
integration-test/src/test/java/org/apache/iotdb/pipe/it/{IoTDBPipeDemoIT.java
=> IoTDBPipeDataSyncIT.java} (80%)
rename
integration-test/src/test/java/org/apache/iotdb/pipe/it/{IoTDBPipeSingleDemoIT.java
=> IoTDBPipeSingleEnvDemoIT.java} (98%)
delete mode 100644
iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/common/response/ConsensusGenericResponse.java
delete mode 100644
iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/common/response/ConsensusReadResponse.java
delete mode 100644
iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/common/response/ConsensusWriteResponse.java
rename
iotdb-core/consensus/src/main/java/org/apache/iotdb/consensus/simple/{SimpleServerImpl.java
=> SimpleConsensusServerImpl.java} (94%)
rename
iotdb-core/{consensus/src/main/java/org/apache/iotdb/consensus/exception/NodeReadOnlyException.java
=>
datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/airgap/AirGapOneByteResponse.java}
(69%)
copy
iotdb-core/{node-commons/src/main/java/org/apache/iotdb/commons/schema/tree/ITreeNode.java
=>
datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/airgap/AirGapPseudoTPipeTransferRequest.java}
(80%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/builder/IoTDBThriftAsyncPipeTransferBatchReqBuilder.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/builder/IoTDBThriftSyncPipeTransferBatchReqBuilder.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/builder/PipeTransferBatchReqBuilder.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/payload/evolvable/request/PipeTransferBatchReq.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/{thrift/IoTDBThriftConnector.java
=> IoTDBConnector.java} (80%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/{thrift/IoTDBThriftConnectorRequestVersion.java
=> IoTDBConnectorRequestVersion.java} (85%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/thrift/async/handler/{PipeTransferTabletInsertionEventHandler.java
=> PipeTransferTabletBatchInsertionEventHandler.java} (62%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/airgap/IoTDBAirGapReceiverAgent.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/receiver/legacy/{IoTDBLegacyPipeReceiver.java
=> IoTDBLegacyPipeReceiverAgent.java} (96%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/{agent/receiver/PipeReceiverAgent.java
=> receiver/thrift/IoTDBThriftReceiverAgent.java} (68%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/{CountAccumulator.java
=> CountTimeAccumulator.java} (84%)
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/expression/visitor/{LowercaseNormalizeVisitor.java
=> CountTimeAggregationAmountVisitor.java} (50%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/PipeEnrichedInsertNode.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/crud/PipeEnrichedInsertBaseStatement.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/crud/PipeEnrichedLoadTsFileStatement.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys
=> metadata}/pipe/CreatePipeStatement.java (97%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys
=> metadata}/pipe/DropPipeStatement.java (96%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys
=> metadata}/pipe/ShowPipesStatement.java (96%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys
=> metadata}/pipe/StartPipeStatement.java (96%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/{sys
=> metadata}/pipe/StopPipeStatement.java (96%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/estimator/CompactionEstimateUtils.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/estimator/FastCompactionInnerCompactionEstimator.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/estimator/FileInfo.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/selector/estimator/ReadChunkInnerCompactionEstimator.java
create mode 100644
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/utils/CompactionTaskMemCostEstimatorTest.java
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/path/PathPatternTreeUtils.java
rename
iotdb-core/{consensus/src/main/java/org/apache/iotdb/consensus/common/response/ConsensusResponse.java
=>
node-commons/src/main/java/org/apache/iotdb/commons/pipe/plugin/builtin/connector/IoTDBAirGapConnector.java}
(63%)
copy
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/plugin/builtin/connector/{IoTDBLegacyPipeConnector.java
=> PlaceholderConnector.java} (86%)
rename iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/schemaengine =>
node-commons/src/main/java/org/apache/iotdb/commons/schema}/SchemaConstant.java
(88%)
rename iotdb-protocol/{thrift-iot-consensus => thrift-consensus}/pom.xml (95%)
rename iotdb-protocol/{thrift-iot-consensus =>
thrift-consensus}/src/main/thrift/iotconsensus.thrift (100%)
rename iotdb-protocol/{thrift => thrift-datanode}/README.md (100%)
rename iotdb-protocol/{thrift => thrift-datanode}/pom.xml (96%)
rename iotdb-protocol/{thrift => thrift-datanode}/rpc-changelist.md (100%)
rename iotdb-protocol/{thrift =>
thrift-datanode}/src/main/thrift/client.thrift (100%)
rename iotdb-protocol/{thrift =>
thrift-datanode}/src/main/thrift/datanode.thrift (98%)
delete mode 100644 tools/README.md
delete mode 100644 tools/pom.xml
delete mode 100644 tools/thrift/pom.xml
delete mode 100644 tools/thrift/src/main/assembly/thrift.xml
