This is an automated email from the ASF dual-hosted git repository.
spricoder pushed a change to branch feature/memory_transfer
in repository https://gitbox.apache.org/repos/asf/iotdb.git
from d0caaca0cbf add params
add af7ae01d1b5 Directly throw DateTimeParseException instead of wrapping
RuntimeException (#15096)
add d34f1c53c59 Restore rollback for UnsetTemplateProcedure
add f56e06b32c7 Make null comment display "null" instead of "" in details
queries & Updated tsFile version
add 45e3900a111 Fix null timestamp (#15177)
add ee779e2b499 Fix database hint of CLI when change sql_dialect back to
table
add bc078fccc89 Pipe: Fixed Async Connector Repeatedly Putting Events into
RetryQueue (#15179)
add 149756e3a25 Subscription: intro SubscriptionLogManager for global
uniform downsampling of prefetched logs (#15175)
add 598080b47f7 Fixed the show regions with specified database & grant /
revoke with ""
add 33d5b195fb0 Support Training Module of AINode
add 242db7672c7 Update Cli -help information (#15159)
add c520c39ff2f use getter for_encryptParam and change tsfile version
(#15185)
add 5f635bf464e DAL: Remove dal file after drop databases or remove
dataRegions (#15191)
add 299c3df372c Improve ConfigNode removing #15189
add a5c1d117ea5 Load: Added the missing permission check for auto-create
database in loading tsFile (#15190)
add 0f0aaf410da Pipe: Fixed the authority for table model insertion
(#15186)
add 77b7541eb7e Fix incorrect os dirs creation (#15196)
add 91fb32a5c3a Pipe: Added default logic for username / password in tree
sql-dialect pipes & Fixed the NPE for configNode table schema snapshot parsing
need (#15200)
add e86ec17bc95 Pipe: Add metrics for tsfile to tablets invocation call
count and time (#15202)
add c986324b0e6 Pipe: Enable showing WRITE_BACK_SINK (#15207)
add 3805b47e58b remove export tsfile script (#15206)
add e62bbf52833 Pipe: Enable showing WRITE_BACK_SINK (Fix IoTDBDatabaseIT)
(#15209)
add a81c5c6ac70 Load: Support multi-disks awareness for IoTV2 and Pipe
(#15181)
add 7fe614c7cc2 Pipe: Added close for tree session in WriteBackConnector
(#15211)
add 7bfcb4965c1 fix: clone & delete issues (#15153)
add 4678776f5c1 Pipe Plugin: Fix ClassLoader and Visibility not
initialized from CN snapshot after reboot (#15212)
add f66fcd91642 Pipe: Fixed the NPE of OPC UA Sink in table model when
there are null values & Deleted the extra folder of opc ua sink in table model
(#15215)
add ebb1bcf8910 Load: Add default value for multi disks awareness (#15216)
add 3d1cd0ea7f6 Pipe: Fix the problem of missing points in TsFile of
parsing table model (#15217)
add 0ee458e991f Added retry for insertion when fetch table schema failed
due to connection error & Improved the retry mechanism of client rpc
add a6088a28a07 [RTO/RPO] Chore: Fix remaining issues in PR 15014 (#15171)
add c5d0c397d98 Pipe: Added userName / password logic for alter pipe
replace statement (#15219)
add 630a3d255d9 Set safeDeletedSearchIndex for iotv2 to accelerate wal
deletion #15221
add 680026eefea Pipe: Prevent Duplicate Time Usage Reporting on Close in
PipeTsFileToTabletsMetrics (#15220)
add a8c80ecb140 Load: Batched tablet insertion during conversion (#15125)
add 08c29635d38 Pipe Plugin: Only Allow Create Model Compatible Plugins
(#15227)
add c746da2faec Deleted the useless read lock in getDeviceAttribute
add 8889255c0c4 Load: Implement region operations cache for load × region
migration detection (#15210)
add 7e4e01cc632 Load & Pipe: Support Active Load Table Model TsFiles &
Support Async Load in SQL & Support Async Load Strategy in Pipe (#15208)
add bf2eee9ad1e Fix: Null value settings not applying correctly in tablet.
(#15228)
add c9ec0fda2d1 Subscription: distinguish between reference count of ack
and clean in tsfile batch to avoid cleaning before ack (#15229)
add 4256ea48ad1 Fix BigArray NPE of some aggregation functions (first,
last, sum, extreme) when groups are more than 1024 in aggregation query
add 9b406e15353 update description when file_type is csv, sql_dialect is
table and table is not required (#15232)
add 1697f545307 Fixed the NullPointerException caused by non-current table
procedures #15238
add b6fea634d61 Add some supplementary logs during partition allocation
(#15234)
add 4350a56c237 Description add the file name when importing file
exception (#15237)
add 46fc1e0a6fa Pipe: Fixed the tsFile parsing & write-back-sink auto
create db bug (#15240)
add e237f723974 [Python-Client] Use executeV2 interface to query
add 304fbab00bd Unifying Exception Messages for Aggregation Functions
(#15231)
add b1d4fedb33c Pipe IT: Fixed the issue that the Pipe IT test error log
could not be printed (#15242)
add 5f28e331d29 Pipe: Optimized the error log when source/sink's username
is without password in alter (#15241)
add c0a642892f7 Pipe: Added memory control for aligned chunk reader in
TsFileInsertionEventScanParser & TsFileInsertionEventTableParserTabletIterator
(#15222)
add 7c301137057 Add GroupNode and parallelize TableFunctionProcessorNode
add 91d067b161e Pipe: Fix stuck caused by async connector client not
returned after transferring tsfiles & Fix validateTsFile and
shouldMarkAsPipeRequest may not be effective (#15245) (#15249)
add 5ddd4234666 using correct encoding in read chunk compaction (#15250)
add 9132389e4fd Import/Export schema script adapt table model (#15192)
add e6713d4f029 IoTV2: Clean receiver files when dropping consensus pipe &
Improve robustness when cleaning some dirs. (#15252)
add a4d1a508100 Pipe: Applied the login function to config receiver
(#15258)
add e7ce326ca07 [opt](query) Optimize count(1), count(constant expression)
to count(*)
add d3d0687b3be Reduce default value of rpcMaxConcurrentClientNum (#15260)
add 518126ed7ce Enhance remove confignode robustness by increase waiting
time for new leader #15261
add fac2e6334bd Harmonizing the case of reported error messages for
aggregation (#15251)
add 838dbab4fb5 Made memory not enough exception cause by schema fetching
retry-able for insertion & Refactor & Fixed the database name bug when alter
non-exists database
add 72aeed2591f Pipe: Optimized insert node cache hit possibility & Pipe:
Set thread name for pipe receiver (#15263)
add 82be8be5844 Pipe: Added total floating memory usage for mem-ctrl of
linked but deleted tsFile resource and insertNode memory (#15205)
add e4e3c29d274 fix grant all to others.
add b4abdd2bf34 Pipe: Fixed the meta inclusion between SQL dialects
(#15278)
add 961f3581728 Fix the output ordinal of AggTableScanOperator are not
consistent with OutputSymbols of AggTableScanNode when do single input distinct
optimize
add ab6cbaf6f8b rest add support show models (#15287)
add b466bd1c699 fix: remove mqtt with table auth check (#15288)
add 7a4c1273151 IoTV2: Fix closing receiver corner case regarding to
concurrently closing and receiving file. (#15271)
add eccbcc2a012 fix mods file metrics (#15284)
add ef8dc52e89f perf: add encodeBatch interface to accelerate flushing for
sequential insert (#15243)
add 20aef48a87a Merge branch 'master' into feature/memory_transfer
No new revisions were added by this update.
Summary of changes:
.github/workflows/multi-language-client.yml | 10 +-
.github/workflows/pipe-it.yml | 54 +-
.../org/apache/iotdb/db/it/IoTDBRestartIT.java | 3 +-
.../iotdb/db/it/auth/IoTDBRelationalAuthIT.java | 43 ++
.../it/schema/IoTDBDeleteAlignedTimeseriesIT.java | 42 ++
.../pipe/it/dual/tablemodel/TableModelUtils.java | 8 +-
.../manual/basic/IoTDBPipePermissionIT.java | 90 ++-
.../manual/enhanced/IoTDBPipeMetaIT.java | 52 ++
.../pipe/it/single/IoTDBPipePermissionIT.java | 43 ++
.../it/db/it/IoTDBAuthenticationTableIT.java | 90 +--
.../relational/it/db/it/IoTDBInsertTableIT.java | 39 ++
.../db/it/IoTDBMultiTAGsWithAttributesTableIT.java | 122 +++-
.../relational/it/db/it/IoTDBWindowTVFIT.java | 229 ++++++++
.../udf/IoTDBUserDefinedAggregateFunctionIT.java | 1 -
.../it/query/recent/IoTDBTableAggregationIT.java | 64 +++
.../relational/it/schema/IoTDBDatabaseIT.java | 21 +-
.../iotdb/relational/it/schema/IoTDBTableIT.java | 3 +-
.../it/session/IoTDBSessionRelationalIT.java | 53 ++
.../apache/iotdb/tools/it/ExportDataTestIT.java | 13 +-
.../apache/iotdb/tools/it/ExportSchemaTestIT.java | 5 +-
.../apache/iotdb/tools/it/ExportTsFileTestIT.java | 4 +-
.../apache/iotdb/tools/it/ImportSchemaTestIT.java | 8 +-
.../relational/table/TableFunctionAnalysis.java | 28 +-
.../cli/src/assembly/resources/sbin/start-cli.sh | 4 -
.../src/assembly/resources/tools/export-tsfile.bat | 62 ---
.../src/assembly/resources/tools/export-tsfile.sh | 59 --
.../java/org/apache/iotdb/cli/AbstractCli.java | 33 +-
.../org/apache/iotdb/tool/common/Constants.java | 45 +-
.../org/apache/iotdb/tool/common/OptionsUtil.java | 156 +++++-
.../apache/iotdb/tool/data/AbstractImportData.java | 12 +-
.../org/apache/iotdb/tool/data/ExportData.java | 1 -
.../apache/iotdb/tool/data/ExportDataTable.java | 2 +-
.../org/apache/iotdb/tool/data/ExportDataTree.java | 2 +-
.../org/apache/iotdb/tool/data/ImportData.java | 6 +
.../apache/iotdb/tool/data/ImportDataTable.java | 7 +-
.../iotdb/tool/schema/AbstractExportSchema.java} | 14 +-
.../iotdb/tool/schema/AbstractImportSchema.java | 94 ++++
.../iotdb/tool/schema/AbstractSchemaTool.java | 120 ++--
.../org/apache/iotdb/tool/schema/ExportSchema.java | 325 +++--------
.../iotdb/tool/schema/ExportSchemaTable.java | 252 +++++++++
.../apache/iotdb/tool/schema/ExportSchemaTree.java | 116 ++++
.../org/apache/iotdb/tool/schema/ImportSchema.java | 613 +++------------------
.../iotdb/tool/schema/ImportSchemaTable.java | 188 +++++++
.../{ImportSchema.java => ImportSchemaTree.java} | 417 ++++----------
iotdb-client/client-cpp/src/main/Session.cpp | 12 +-
iotdb-client/client-py/iotdb/Session.py | 324 +++++------
.../client-py/iotdb/template/MeasurementNode.py | 2 +-
iotdb-client/client-py/iotdb/template/Template.py | 6 +-
.../{TsFileConstant.py => tsfile_constant.py} | 0
.../tsfile/utils/{DateUtils.py => date_utils.py} | 0
.../iotdb/tsfile/utils/{Pair.py => pair.py} | 0
...{ReadWriteIOUtils.py => read_write_io_utils.py} | 0
.../client-py/iotdb/tsfile/utils/tsblock_serde.py | 266 +++++++++
iotdb-client/client-py/iotdb/utils/Field.py | 6 +-
.../client-py/iotdb/utils/IoTDBRpcDataSet.py | 463 ----------------
iotdb-client/client-py/iotdb/utils/NumpyTablet.py | 2 +-
.../client-py/iotdb/utils/SessionDataSet.py | 48 +-
iotdb-client/client-py/iotdb/utils/Tablet.py | 2 +-
.../{IoTDBConnectionException.py => exception.py} | 20 +
.../client-py/iotdb/utils/iotdb_rpc_dataset.py | 396 +++++++++++++
iotdb-client/client-py/iotdb/utils/rpc_utils.py | 69 +++
iotdb-client/client-py/resources/pyproject.toml | 5 +-
.../tests/integration/test_tablemodel_query.py | 456 +++++++++++++++
.../org/apache/iotdb/jdbc/IoTDBConnection.java | 7 +-
.../java/org/apache/iotdb/jdbc/IoTDBStatement.java | 2 +-
.../java/org/apache/iotdb/rpc/TSStatusCode.java | 1 +
iotdb-core/ainode/.gitignore | 4 +-
iotdb-core/ainode/{iotdb => ainode}/__init__.py | 0
.../{iotdb/ainode => ainode/core}/__init__.py | 0
.../ainode/{iotdb/ainode => ainode/core}/client.py | 42 +-
.../ainode/{iotdb/ainode => ainode/core}/config.py | 22 +-
.../{iotdb/ainode => ainode/core}/constant.py | 0
.../{iotdb/ainode => ainode/core}/exception.py | 2 +-
.../{iotdb/ainode => ainode/core}/handler.py | 19 +-
.../ainode/{iotdb/ainode => ainode/core}/log.py | 4 +-
.../ainode => ainode/core}/manager/__init__.py | 0
.../core}/manager/cluster_manager.py | 4 +-
.../core}/manager/inference_manager.py | 14 +-
.../core}/manager/model_manager.py | 16 +-
.../ainode => ainode/core}/model/__init__.py | 0
.../core}/model/built_in_model_factory.py | 8 +-
.../ainode => ainode/core}/model/model_factory.py | 10 +-
.../ainode => ainode/core}/model/model_storage.py | 15 +-
.../ainode/{iotdb/ainode => ainode/core}/script.py | 16 +-
.../{iotdb/ainode => ainode/core}/service.py | 8 +-
.../{iotdb/ainode => ainode/core}/util/__init__.py | 0
.../ainode => ainode/core}/util/decorator.py | 0
.../{iotdb/ainode => ainode/core}/util/lock.py | 0
.../{iotdb/ainode => ainode/core}/util/serde.py | 6 +-
.../{iotdb/ainode => ainode/core}/util/status.py | 6 +-
iotdb-core/ainode/pom.xml | 8 +-
iotdb-core/ainode/pyproject.toml | 21 +-
.../org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | 13 +
.../antlr4/org/apache/iotdb/db/qp/sql/SqlLexer.g4 | 4 +
.../iotdb/confignode/conf/ConfigNodeConfig.java | 15 +-
.../request/write/model/UpdateModelInfoPlan.java | 4 +
.../iotdb/confignode/manager/ConfigManager.java | 121 +++-
.../iotdb/confignode/manager/ModelManager.java | 51 +-
.../confignode/manager/PermissionManager.java | 2 +
.../iotdb/confignode/manager/ProcedureManager.java | 54 +-
.../manager/load/service/HeartbeatService.java | 12 +
.../manager/load/service/StatisticsService.java | 4 +-
.../manager/load/service/TopologyService.java | 60 +-
.../iotdb/confignode/manager/node/NodeManager.java | 25 +-
.../manager/partition/PartitionManager.java | 67 ++-
.../pipe/extractor/IoTDBConfigRegionExtractor.java | 11 +-
.../receiver/protocol/IoTDBConfigNodeReceiver.java | 12 +-
.../persistence/executor/ConfigPlanExecutor.java | 10 +-
.../confignode/persistence/node/NodeInfo.java | 7 +
.../persistence/pipe/PipePluginInfo.java | 51 +-
.../procedure/env/ConfigNodeProcedureEnv.java | 11 +-
.../impl/region/RegionMigrateProcedure.java | 8 +-
.../impl/schema/DeleteDatabaseProcedure.java | 4 +-
.../impl/schema/UnsetTemplateProcedure.java | 74 ++-
.../thrift/ConfigNodeRPCServiceProcessor.java | 17 +-
.../java/org/apache/iotdb/db/conf/IoTDBConfig.java | 37 +-
.../org/apache/iotdb/db/conf/IoTDBDescriptor.java | 24 +-
.../db/pipe/agent/task/PipeDataNodeTaskAgent.java | 53 +-
.../agent/task/connection/PipeEventCollector.java | 4 +-
.../subtask/processor/PipeProcessorSubtask.java | 15 +-
.../client/IoTDBDataNodeAsyncClientManager.java | 6 +-
.../IoTDBDataNodeCacheLeaderClientManager.java | 4 +-
.../connector/protocol/opcua/OpcUaNameSpace.java | 11 +-
.../async/IoTDBDataRegionAsyncConnector.java | 43 +-
.../async/handler/PipeTransferTsFileHandler.java | 56 +-
.../protocol/writeback/WriteBackConnector.java | 13 +
.../util/builder/PipeTableModeTsFileBuilder.java | 5 +-
.../sorter/PipeTableModelTabletEventSorter.java | 12 +-
.../consensus/ConsensusPipeDataNodeDispatcher.java | 3 -
.../deletion/DeletionResourceManager.java | 15 +
.../common/tsfile/PipeTsFileInsertionEvent.java | 6 +-
.../tsfile/parser/TsFileInsertionEventParser.java | 23 +
.../parser/TsFileInsertionEventParserProvider.java | 63 ++-
.../query/TsFileInsertionEventQueryParser.java | 19 +-
.../scan/TsFileInsertionEventScanParser.java | 25 +-
.../table/TsFileInsertionEventTableParser.java | 16 +-
...ileInsertionEventTableParserTabletIterator.java | 164 +++---
.../dataregion/IoTDBDataRegionExtractor.java | 2 +
.../PipeRealtimeDataRegionHybridExtractor.java | 8 +-
.../iotdb/db/pipe/metric/PipeDataNodeMetrics.java | 3 +
.../pipe/metric/overview/PipeResourceMetrics.java | 2 +-
.../overview/PipeTsFileToTabletsMetrics.java | 170 ++++++
.../pipeconsensus/PipeConsensusReceiver.java | 26 +
.../pipeconsensus/PipeConsensusReceiverAgent.java | 2 +-
.../protocol/thrift/IoTDBDataNodeReceiver.java | 127 ++---
.../visitor/PipeStatementExceptionVisitor.java | 5 +
.../db/pipe/resource/memory/PipeMemoryManager.java | 45 +-
.../resource/tsfile/PipeTsFileResourceManager.java | 2 +-
.../iotdb/db/protocol/client/ConfigNodeClient.java | 14 +
.../iotdb/db/protocol/mqtt/MPPPublishHandler.java | 45 +-
.../rest/v1/handler/ExecuteStatementHandler.java | 2 +
.../rest/v1/handler/QueryDataSetHandler.java | 2 +
.../rest/v2/handler/ExecuteStatementHandler.java | 2 +
.../rest/v2/handler/QueryDataSetHandler.java | 2 +
.../protocol/thrift/impl/ClientRPCServiceImpl.java | 111 ++--
.../impl/DataNodeInternalRPCServiceImpl.java | 14 +-
.../process/function/PartitionRecognizer.java | 66 ++-
.../process/function/TableFunctionOperator.java | 94 ++--
.../{SliceCache.java => PartitionCache.java} | 14 +-
.../operator/process/function/partition/Slice.java | 61 +-
.../relational/aggregation/AvgAccumulator.java | 8 +-
.../relational/aggregation/CountAccumulator.java | 4 +-
.../aggregation/CountAllAccumulator.java | 4 +-
.../relational/aggregation/ExtremeAccumulator.java | 2 +-
.../relational/aggregation/FirstAccumulator.java | 12 +-
.../relational/aggregation/FirstByAccumulator.java | 17 +-
.../relational/aggregation/LastAccumulator.java | 12 +-
.../relational/aggregation/LastByAccumulator.java | 17 +-
.../relational/aggregation/MaxAccumulator.java | 12 +-
.../relational/aggregation/MinAccumulator.java | 12 +-
.../relational/aggregation/SumAccumulator.java | 8 +-
.../aggregation/TableMaxMinByBaseAccumulator.java | 9 +-
.../aggregation/TableVarianceAccumulator.java | 4 +-
.../aggregation/grouped/GroupedAvgAccumulator.java | 8 +-
.../grouped/GroupedExtremeAccumulator.java | 17 +-
.../grouped/GroupedFirstAccumulator.java | 21 +-
.../grouped/GroupedFirstByAccumulator.java | 22 +-
.../grouped/GroupedLastAccumulator.java | 41 +-
.../grouped/GroupedLastByAccumulator.java | 24 +-
.../aggregation/grouped/GroupedMaxAccumulator.java | 16 +-
.../grouped/GroupedMaxMinByBaseAccumulator.java | 34 +-
.../aggregation/grouped/GroupedMinAccumulator.java | 16 +-
.../grouped/GroupedModeAccumulator.java | 6 +-
.../aggregation/grouped/GroupedSumAccumulator.java | 5 +-
.../grouped/GroupedVarianceAccumulator.java | 2 +-
.../grouped/StreamingAggregationOperator.java | 32 +-
.../grouped/StreamingHashAggregationOperator.java | 32 +-
.../iotdb/db/queryengine/plan/Coordinator.java | 4 +
.../plan/analyze/ClusterPartitionFetcher.java | 30 +-
.../analyze/cache/partition/PartitionCache.java | 7 +-
.../plan/analyze/load/LoadTsFileAnalyzer.java | 81 ++-
.../analyze/load/LoadTsFileTableSchemaCache.java | 6 +-
.../analyze/load/LoadTsFileTreeSchemaCache.java | 6 +-
.../analyze/schema/AutoCreateSchemaExecutor.java | 14 +-
.../analyze/schema/ClusterSchemaFetchExecutor.java | 9 +-
.../execution/config/TableConfigTaskVisitor.java | 97 +++-
.../execution/config/TreeConfigTaskVisitor.java | 73 ++-
.../config/executor/ClusterConfigTaskExecutor.java | 144 ++++-
.../config/executor/IConfigTaskExecutor.java | 27 +-
.../config/metadata/CreatePipePluginTask.java | 2 +
.../metadata/{model => ai}/CreateModelTask.java | 2 +-
.../config/metadata/ai/CreateTrainingTask.java | 108 ++++
.../metadata/{model => ai}/DropModelTask.java | 2 +-
.../metadata/{model => ai}/ShowModelsTask.java | 2 +-
.../relational/DescribeTableDetailsTask.java | 16 +-
.../metadata/relational/ShowTablesDetailsTask.java | 13 +-
.../execution/config/sys/pipe/AlterPipeTask.java | 3 +-
.../db/queryengine/plan/parser/ASTVisitor.java | 66 ++-
.../plan/planner/TableOperatorGenerator.java | 17 +-
.../SimpleFragmentParallelPlanner.java | 126 +----
.../exceptions/RootFIPlacementException.java | 2 +-
.../plan/AbstractFragmentParallelPlanner.java | 168 ++++++
.../plan/planner/plan/node/PlanGraphPrinter.java | 20 +
.../plan/planner/plan/node/PlanNodeType.java | 4 +
.../plan/planner/plan/node/PlanVisitor.java | 5 +
.../relational/analyzer/ExpressionAnalyzer.java | 2 +-
.../relational/analyzer/StatementAnalyzer.java | 13 +-
.../TableFunctionInvocationAnalysis.java | 9 +-
.../plan/relational/function/FunctionId.java | 2 +
.../plan/relational/metadata/ResolvedFunction.java | 1 -
.../metadata/fetcher/TableDeviceSchemaFetcher.java | 12 +-
.../fetcher/cache/TableDeviceSchemaCache.java | 11 +-
.../plan/relational/planner/RelationPlanner.java | 3 +-
.../plan/relational/planner/SymbolAllocator.java | 4 +
.../distribute/TableDistributedPlanGenerator.java | 161 +++++-
.../distribute/TableDistributedPlanner.java | 2 +-
.../distribute/TableModelQueryFragmentPlanner.java | 167 ++----
.../TableModelTypeProviderExtractor.java | 7 +
.../plan/relational/planner/ir/IrUtils.java | 7 +
.../rule/ImplementTableFunctionSource.java | 78 +--
.../rule/PruneTableFunctionProcessorColumns.java | 4 +-
.../PruneTableFunctionProcessorSourceColumns.java | 4 +-
.../iterative/rule/SimplifyCountOverConstant.java | 145 +++++
.../rule/SingleDistinctAggregationToGroupBy.java | 3 +-
.../planner/node/AggregationTableScanNode.java | 11 +-
.../node/{StreamSortNode.java => GroupNode.java} | 86 ++-
.../relational/planner/node/TableFunctionNode.java | 20 +-
.../planner/node/TableFunctionProcessorNode.java | 42 +-
.../optimizations/LogicalOptimizeFactory.java | 10 +-
.../planner/optimizations/ParallelizeGrouping.java | 266 +++++++++
.../PushLimitOffsetIntoTableScan.java | 6 +
.../TransformAggregationToStreamable.java | 22 +
...mQuantifiedComparisonApplyToCorrelatedJoin.java | 13 +-
.../optimizations/TransformSortToStreamSort.java | 6 +
.../optimizations/UnaliasSymbolReferences.java | 29 +-
.../relational/planner/optimizations/Util.java | 2 +-
.../relational/security/AccessControlImpl.java | 7 +-
.../plan/relational/sql/ast/AstVisitor.java | 8 +
.../plan/relational/sql/ast/CreateTraining.java | 156 ++++++
.../plan/relational/sql/ast/LoadTsFile.java | 6 +
.../ast/{ShowConfigNodes.java => ShowModels.java} | 34 +-
.../plan/relational/sql/ast/ShowRegions.java | 43 +-
.../plan/relational/sql/parser/AstBuilder.java | 149 +++--
.../plan/relational/sql/util/AstUtil.java | 22 +
.../plan/scheduler/AsyncSendPlanNodeHandler.java | 2 +-
.../plan/scheduler/load/LoadTsFileScheduler.java | 5 +-
.../plan/statement/StatementVisitor.java | 5 +
.../plan/statement/crud/LoadTsFileStatement.java | 10 +
.../statement/metadata/ShowRegionStatement.java | 29 +-
.../metadata/model/CreateTrainingStatement.java | 140 +++++
.../metadata/pipe/AlterPipeStatement.java | 9 +
.../metadata/pipe/CreatePipePluginStatement.java | 10 +
.../schemaengine/table/InformationSchemaUtils.java | 4 +-
.../template/TemplateInternalRPCUpdateType.java | 1 -
.../iotdb/db/service/RegionMigrateService.java | 69 ++-
.../iotdb/db/service/metrics/FileMetrics.java | 10 +
.../db/service/metrics/file/ModsFileMetrics.java | 4 +-
.../iotdb/db/storageengine/StorageEngine.java | 1 +
.../db/storageengine/dataregion/DataRegion.java | 83 ++-
.../ReadChunkAlignedSeriesCompactionExecutor.java | 5 +-
.../dataregion/flush/MemTableFlushTask.java | 15 +-
.../memtable/AbstractWritableMemChunk.java | 4 +-
.../memtable/AlignedWritableMemChunk.java | 328 +++++------
.../dataregion/memtable/IWritableMemChunk.java | 3 +-
.../dataregion/memtable/TsFileProcessor.java | 59 +-
.../dataregion/memtable/WritableMemChunk.java | 93 +---
.../dataregion/tsfile/TsFileResource.java | 1 -
.../dataregion/wal/utils/WALEntryHandler.java | 10 +-
.../dataregion/wal/utils/WALEntryPosition.java | 4 +-
.../dataregion/wal/utils/WALInsertNodeCache.java | 6 +
.../load/active/ActiveLoadTsFileLoader.java | 102 ++--
.../load/config/LoadTsFileConfigurator.java | 20 +
...ertedInsertTabletStatementExceptionVisitor.java | 5 +
...vertedInsertTabletStatementTSStatusVisitor.java | 7 +
...leStatementDataTypeConvertExecutionVisitor.java | 1 +
...eeStatementDataTypeConvertExecutionVisitor.java | 186 +++++--
.../storageengine/load/disk/ILoadDiskSelector.java | 66 +++
.../InheritSystemMultiDisksStrategySelector.java | 58 ++
.../db/storageengine/load/disk/MinIOSelector.java | 112 ++++
...MemoryBlock.java => LoadTsFileMemoryBlock.java} | 13 +-
.../load/memory/LoadTsFileMemoryManager.java | 19 +-
.../load/metrics/LoadTsFileMemMetricSet.java | 6 +-
.../db/storageengine/rescon/disk/TierManager.java | 2 +-
.../agent/SubscriptionBrokerAgent.java | 10 +-
.../db/subscription/broker/SubscriptionBroker.java | 25 +-
.../broker/SubscriptionPrefetchingQueue.java | 11 +-
.../broker/SubscriptionPrefetchingQueueStates.java | 2 +-
.../batch/SubscriptionPipeTsFileEventBatch.java | 5 +-
.../event/cache/SubscriptionPollResponseCache.java | 4 +-
.../pipe/SubscriptionPipeTsFileBatchEvents.java | 16 +-
.../SubscriptionDataNodeResourceManager.java} | 42 +-
.../resource/log/SubscriptionLogManager.java} | 16 +-
.../resource/log/SubscriptionLogStatus.java | 82 +++
.../org/apache/iotdb/db/utils/DateTimeUtils.java | 35 +-
.../apache/iotdb/db/utils/ErrorHandlingUtils.java | 8 +-
.../db/utils/datastructure/AlignedTVList.java | 178 +++++-
...{TimAlignedTVList.java => BatchEncodeInfo.java} | 43 +-
.../db/utils/datastructure/MemPointIterator.java | 3 +
.../MergeSortMultiAlignedTVListIterator.java | 105 ++++
.../MergeSortMultiTVListIterator.java | 77 ++-
.../utils/datastructure/MultiTVListIterator.java | 11 +-
.../OrderedMultiAlignedTVListIterator.java | 15 +
.../datastructure/OrderedMultiTVListIterator.java | 31 +-
.../iotdb/db/utils/datastructure/TVList.java | 124 ++++-
.../process/tvf/TableFunctionOperatorTest.java | 6 +-
.../analyzer/AggregationCornerCaseTest.java | 179 ++++++
.../plan/relational/analyzer/AggregationTest.java | 45 ++
.../relational/analyzer/TableFunctionTest.java | 108 ++--
.../planner/assertions/PlanMatchPattern.java | 5 +
.../compaction/ReadChunkInnerCompactionTest.java | 70 +++
...atchedAlignedSeriesReadChunkCompactionTest.java | 2 +-
.../apache/iotdb/metrics/utils/FileStoreUtils.java | 2 +-
.../conf/iotdb-system.properties.template | 25 +-
.../iotdb/commons/client/ainode/AINodeClient.java | 13 +
.../apache/iotdb/commons/conf/CommonConfig.java | 39 +-
.../iotdb/commons/conf/CommonDescriptor.java | 20 +
.../iotdb/commons/model/ModelInformation.java | 20 +
.../apache/iotdb/commons/model/ModelStatus.java | 1 +
.../agent/plugin/builtin/BuiltinPipePlugin.java | 1 -
.../commons/pipe/agent/task/PipeTaskAgent.java | 13 +
.../iotdb/commons/pipe/config/PipeConfig.java | 10 +
.../options/PipeInclusionOptions.java | 117 ++--
.../commons/pipe/extractor/IoTDBExtractor.java | 15 +-
.../commons/pipe/receiver/IoTDBFileReceiver.java | 35 +-
.../iotdb/commons/service/metric/enums/Metric.java | 2 +
.../subscription/config/SubscriptionConfig.java | 12 +
.../relational/TableBuiltinTableFunction.java | 17 +-
.../relational/tvf/CapacityTableFunction.java | 89 +--
.../builtin/relational/tvf/HOPTableFunction.java | 29 +-
...ableFunction.java => SessionTableFunction.java} | 88 ++-
...leFunction.java => VariationTableFunction.java} | 111 ++--
.../udf/builtin/relational/tvf/WindowTVFUtils.java | 55 ++
.../commons/udf/service/UDFManagementService.java | 3 +
.../org/apache/iotdb/commons/utils/FileUtils.java | 49 ++
.../apache/iotdb/commons/utils/JVMCommonUtils.java | 2 +-
.../apache/iotdb/commons/utils/StatusUtils.java | 6 +
.../db/relational/grammar/sql/RelationalSql.g4 | 60 +-
.../thrift-ainode/src/main/thrift/ainode.thrift | 16 +
.../src/main/thrift/confignode.thrift | 35 ++
.../src/main/thrift/datanode.thrift | 9 +-
pom.xml | 2 +-
351 files changed, 10138 insertions(+), 4485 deletions(-)
create mode 100644
integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBWindowTVFIT.java
delete mode 100644
iotdb-client/cli/src/assembly/resources/tools/export-tsfile.bat
delete mode 100644
iotdb-client/cli/src/assembly/resources/tools/export-tsfile.sh
copy
iotdb-client/{session/src/main/java/org/apache/iotdb/session/InsertConsumer.java
=> cli/src/main/java/org/apache/iotdb/tool/schema/AbstractExportSchema.java}
(71%)
create mode 100644
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/AbstractImportSchema.java
create mode 100644
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ExportSchemaTable.java
create mode 100644
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ExportSchemaTree.java
create mode 100644
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ImportSchemaTable.java
copy
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/{ImportSchema.java
=> ImportSchemaTree.java} (53%)
rename iotdb-client/client-py/iotdb/tsfile/common/constant/{TsFileConstant.py
=> tsfile_constant.py} (100%)
rename iotdb-client/client-py/iotdb/tsfile/utils/{DateUtils.py =>
date_utils.py} (100%)
rename iotdb-client/client-py/iotdb/tsfile/utils/{Pair.py => pair.py} (100%)
rename iotdb-client/client-py/iotdb/tsfile/utils/{ReadWriteIOUtils.py =>
read_write_io_utils.py} (100%)
create mode 100644 iotdb-client/client-py/iotdb/tsfile/utils/tsblock_serde.py
delete mode 100644 iotdb-client/client-py/iotdb/utils/IoTDBRpcDataSet.py
rename iotdb-client/client-py/iotdb/utils/{IoTDBConnectionException.py =>
exception.py} (64%)
create mode 100644 iotdb-client/client-py/iotdb/utils/iotdb_rpc_dataset.py
create mode 100644 iotdb-client/client-py/iotdb/utils/rpc_utils.py
create mode 100644
iotdb-client/client-py/tests/integration/test_tablemodel_query.py
rename iotdb-core/ainode/{iotdb => ainode}/__init__.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/__init__.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/client.py (81%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/config.py (91%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/constant.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/exception.py (98%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/handler.py (69%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/log.py (97%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/manager/__init__.py
(100%)
rename iotdb-core/ainode/{iotdb/ainode =>
ainode/core}/manager/cluster_manager.py (93%)
rename iotdb-core/ainode/{iotdb/ainode =>
ainode/core}/manager/inference_manager.py (95%)
rename iotdb-core/ainode/{iotdb/ainode =>
ainode/core}/manager/model_manager.py (86%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/model/__init__.py (100%)
rename iotdb-core/ainode/{iotdb/ainode =>
ainode/core}/model/built_in_model_factory.py (99%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/model/model_factory.py
(96%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/model/model_storage.py
(92%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/script.py (93%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/service.py (91%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/util/__init__.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/util/decorator.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/util/lock.py (100%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/util/serde.py (98%)
rename iotdb-core/ainode/{iotdb/ainode => ainode/core}/util/status.py (90%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/metric/overview/PipeTsFileToTabletsMetrics.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/partition/{SliceCache.java
=> PartitionCache.java} (86%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{model
=> ai}/CreateModelTask.java (99%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/ai/CreateTrainingTask.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{model
=> ai}/DropModelTask.java (99%)
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/{model
=> ai}/ShowModelsTask.java (99%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/AbstractFragmentParallelPlanner.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/iterative/rule/SimplifyCountOverConstant.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/{StreamSortNode.java
=> GroupNode.java} (51%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/ParallelizeGrouping.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/CreateTraining.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/{ShowConfigNodes.java
=> ShowModels.java} (72%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/model/CreateTrainingStatement.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/ILoadDiskSelector.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/InheritSystemMultiDisksStrategySelector.java
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/disk/MinIOSelector.java
rename
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/memory/{LoadTsFileAnalyzeSchemaMemoryBlock.java
=> LoadTsFileMemoryBlock.java} (88%)
copy
iotdb-core/{metrics/interface/src/main/java/org/apache/iotdb/metrics/utils/SystemType.java
=>
datanode/src/main/java/org/apache/iotdb/db/subscription/resource/SubscriptionDataNodeResourceManager.java}
(55%)
copy
iotdb-core/{node-commons/src/main/java/org/apache/iotdb/commons/pipe/resource/log/PipeLogManager.java
=>
datanode/src/main/java/org/apache/iotdb/db/subscription/resource/log/SubscriptionLogManager.java}
(71%)
create mode 100644
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/resource/log/SubscriptionLogStatus.java
copy
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/{TimAlignedTVList.java
=> BatchEncodeInfo.java} (58%)
copy
integration-test/src/main/java/org/apache/iotdb/db/query/udf/example/relational/MyRepeatWithIndex.java
=>
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/CapacityTableFunction.java
(55%)
copy
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/{HOPTableFunction.java
=> SessionTableFunction.java} (61%)
copy
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/{HOPTableFunction.java
=> VariationTableFunction.java} (54%)
create mode 100644
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/WindowTVFUtils.java
