[ https://issues.apache.org/jira/browse/IOTDB-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614380#comment-17614380 ]
Jinrui Zhang commented on IOTDB-4521: ------------------------------------- It seems that this is by design. Too many series in one dev with a large batch will definitely exceed the limit of thrift buffer size > ERROR o.a.t.s.AbstractNonblockingServer$FrameBuffer:356 - Read a frame size > of 562223808, which is bigger than the maximum allowable buffer size for ALL > connections > -------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: IOTDB-4521 > URL: https://issues.apache.org/jira/browse/IOTDB-4521 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster > Affects Versions: 0.14.0-SNAPSHOT > Reporter: 刘珍 > Assignee: Jinrui Zhang > Priority: Major > Attachments: image-2022-09-26-10-04-30-709.png, > image-2022-09-26-10-05-57-896.png, ip72_datanode_logs.tar.gz, more_ts.conf > > > m_0922_0d65058 > 3副本3C3D > schemaregion :RatisConsensus > dataregion :MultiLeaderConsensus > DEVICE_NUMBER=600 > SENSOR_NUMBER=50000 > CLIENT_NUMBER=100 > (5万dev,600sensor /dev 全部数据写入成功)600dev,5万序列/dev,写入 datanode error : > 2022-09-24 00:00:02,666 > [pool-25-IoTDB-ClientRPC-Processor-142$20220923_155931_09082_3.1.0] ERROR > {color:#DE350B}*o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - > null*{color} > 2022-09-24 00:00:02,685 [Thread-18] ERROR > {color:#DE350B}*o.a.t.s.AbstractNonblockingServer$FrameBuffer:356 - Read a > frame size of 722859150, which is bigger than the maximum allowable buffer > size for ALL connections.*{color} > 2022-09-24 00:00:02,726 > [pool-25-IoTDB-ClientRPC-Processor-83$20220923_155931_09083_3.1.0] ERROR > o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - null > 2022-09-24 00:00:03,501 > [pool-25-IoTDB-ClientRPC-Processor-23$20220923_155932_09084_3.1.0] ERROR > o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - null > 2022-09-24 00:00:05,059 > [pool-25-IoTDB-ClientRPC-Processor-113$20220923_155934_09085_3.1.0] ERROR > o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - null > 2022-09-24 00:00:05,138 [Thread-18] ERROR > o.a.t.s.AbstractNonblockingServer$FrameBuffer:356 - Read a frame size of > 803176833, which is bigger than the maximum allowable buffer size for ALL > connections. > 2022-09-24 00:00:07,448 > [pool-25-IoTDB-ClientRPC-Processor-47$20220923_155935_09086_3.1.0] ERROR > o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - null > 2022-09-24 00:00:08,220 > [pool-25-IoTDB-ClientRPC-Processor-59$20220923_155937_09087_3.1.0] ERROR > o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:179 - null > 2022-09-24 00:00:09,320 [Thread-18] ERROR > o.a.t.s.AbstractNonblockingServer$FrameBuffer:356 - Read a frame size of > 669314043, which is bigger than the maximum allowable buffer size for ALL > connections. > 2022-09-24 00:00:12,375 [Thread-18] ERROR > o.a.t.s.AbstractNonblockingServer$FrameBuffer:356 - Read a frame size of > 803176825, which is bigger than the maximum allowable buffer size for ALL > connections. > 测试环境 > 1. 192.168.10.72/73/74 48CPU 384GB > bm在71 > ConfigNode > MAX_HEAP_SIZE="8G" > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus > schema_replication_factor=3 > data_replication_factor=3 > connection_timeout_ms=36000000 > DataNode > MAX_HEAP_SIZE="256G" > MAX_DIRECT_MEMORY_SIZE="32G" > connection_timeout_ms=36000000 > max_connection_for_internal_service=300 > enable_timed_flush_seq_memtable=true > seq_memtable_flush_interval_in_ms=3600000 > seq_memtable_flush_check_interval_in_ms=600000 > enable_timed_flush_unseq_memtable=true > unseq_memtable_flush_interval_in_ms=3600000 > unseq_memtable_flush_check_interval_in_ms=600000 > max_waiting_time_when_insert_blocked=3600000 > query_timeout_threshold=36000000 > 2. bm配置见附件 > 600dev,5万序列/dev, bm结果 > !image-2022-09-26-10-04-30-709.png! > 5万dev,600sensor 全部数据写入成功 > !image-2022-09-26-10-05-57-896.png! > 3. ip72 datanode log见附件 -- This message was sent by Atlassian Jira (v8.20.10#820010)