刘珍 created IOTDB-2741: ------------------------- Summary: [IDTable] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right Key: IOTDB-2741 URL: https://issues.apache.org/jira/browse/IOTDB-2741 Project: Apache IoTDB Issue Type: Bug Components: Core/Schema Manager Affects Versions: 0.13.0-SNAPSHOT Reporter: 刘珍 Attachments: config.properties, iotdb-engine.properties, iotdb-env.sh, log_error.log
测试版本: https://github.com/apache/iotdb/pull/5214 问题描述: 开启id table,benchmark工具写入,写入完成立即kill -u 当前用户,备份iotdb的数据, 重启iotdb恢复,报错: 2022-03-15 13:31:56,249 [pool-9-IoTDB-Recovery-Thread-Pool-5] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,249 [pool-9-IoTDB-Recovery-Thread-Pool-5] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,251 [pool-9-IoTDB-Recovery-Thread-Pool-2] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,252 [pool-9-IoTDB-Recovery-Thread-Pool-2] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,253 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,253 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,254 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,254 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,255 [Thread-2] ERROR o.a.i.d.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread Thread-2-25 org.apache.iotdb.db.exception.runtime.StorageEngineFailureException: StorageEngine failed to recover. at org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:245) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: can't initialize disk schema manager at ./sbin/../data/system/storage_groups/root.test.g_30/SeriesKeyMapping.meta at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:243) ... 1 common frames omitted Caused by: java.lang.IllegalArgumentException: can't initialize disk schema manager at ./sbin/../data/system/storage_groups/root.test.g_30/SeriesKeyMapping.meta at org.apache.iotdb.db.metadata.idtable.AppendOnlyDiskSchemaManager.<init>(AppendOnlyDiskSchemaManager.java:71) at org.apache.iotdb.db.metadata.idtable.IDTableHashmapImpl.<init>(IDTableHashmapImpl.java:82) at org.apache.iotdb.db.metadata.idtable.IDTableManager.lambda$getIDTable$0(IDTableManager.java:90) at java.util.HashMap.computeIfAbsent(HashMap.java:1127) at org.apache.iotdb.db.metadata.idtable.IDTableManager.getIDTable(IDTableManager.java:86) at org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.<init>(VirtualStorageGroupProcessor.java:398) at org.apache.iotdb.db.engine.StorageEngine.buildNewStorageGroupProcessor(StorageEngine.java:552) at org.apache.iotdb.db.engine.storagegroup.virtualSg.StorageGroupManager.lambda$asyncRecover$0(StorageGroupManager.java:208) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 common frames omitted 2022-03-15 13:31:56,256 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,256 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,257 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,258 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,259 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,259 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,260 [pool-9-IoTDB-Recovery-Thread-Pool-6] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,260 [pool-9-IoTDB-Recovery-Thread-Pool-6] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,261 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,262 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,262 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,263 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,264 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,264 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,267 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,267 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 2022-03-15 13:31:56,268 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right 2022-03-15 13:31:56,268 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption 测试过程: 1. 测试机器 私有云172.20.70.4 配置文件见附件 device_id_transformation_method=SHA256 enable_id_table=true enable_id_table_log_file=true 2. 启动iotdb 3. benchmark写入 配置文件见附件。 benchmark执行完之后,立即执行kill -u cluster 4. 备份data (用于bug复现) 5. 清缓存 suod -s<<EOF echo 3 >/proc/sys/vm/drop_caches EOF 6.启动iotdb 报错,error日志见附件。 -- This message was sent by Atlassian Jira (v8.20.1#820001)