[jira] [Created] (IOTDB-2058) Query is blocked without sub-query-threads exist
Jialin Qiao created IOTDB-2058: -- Summary: Query is blocked without sub-query-threads exist Key: IOTDB-2058 URL: https://issues.apache.org/jira/browse/IOTDB-2058 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.12.3 Reporter: Jialin Qiao Assignee: Alima777 Attachments: image-2021-11-25-14-36-27-927.png, jstack.txt In an IoTDB Instance, the query main thread is blocking. We could see the queryId using show query proceslist. However, we couldn't see the sub query thread of the blocked query in the jstack file. !image-2021-11-25-14-36-27-927.png! This may be due to that the sub-thread is dead because some error（not catched by Exception） -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2057) OutOfMemoryError: Direct buffer memory
Jialin Qiao created IOTDB-2057: -- Summary: OutOfMemoryError: Direct buffer memory Key: IOTDB-2057 URL: https://issues.apache.org/jira/browse/IOTDB-2057 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.12.3 Reporter: Jialin Qiao Attachments: image-2021-11-25-14-31-56-959.png, image-2021-11-25-14-33-39-006.png, image-2021-11-25-14-35-13-448.png In an IoTDB instance, we set: MAX_HEAP_SIZE="50G" HEAP_NEWSIZE="50G" MAX_DIRECT_MEMORY_SIZE="50G" There are read/write and compaction in this IoTDB. However, the direct memory exceeds 50G and OutOfMemory. !image-2021-11-25-14-31-56-959.png! !image-2021-11-25-14-33-39-006.png! The heap memory looks good. !image-2021-11-25-14-35-13-448.png! We need to control the direct memory. E.g., when using FileChannel to read TsFile. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-2056) When the master branch executes mixed reading and writing, it reports Too many open files and read only
[ https://issues.apache.org/jira/browse/IOTDB-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448947#comment-17448947 ] xiaozhihong commented on IOTDB-2056: The positioning problem is: a method of controlling the flashing, the threshold calculation inside is wrong: !image-2021-11-25-13-51-40-360.png! Verify branch https://github.com/apache/iotdb/tree/Fix_flush > When the master branch executes mixed reading and writing, it reports Too > many open files and read only > --- > > Key: IOTDB-2056 > URL: https://issues.apache.org/jira/browse/IOTDB-2056 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Server >Affects Versions: master branch >Reporter: xiaozhihong >Priority: Major > Attachments: image-2021-11-25-13-51-40-360.png, > rw_no_overflow.config.properties > > > In the weekly test version from iotdb_11-22_f8c18ac to iotdb_11-24_1f853f1, > when using the benchmark to mix reading and writing, you can see in the log: > warn prompts read only, error reports Too many open files > step 1：start server > step 2：using the benchmark to mix reading and writing > step 3：see logs > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2056) When the master branch executes mixed reading and writing, it reports Too many open files and read only
xiaozhihong created IOTDB-2056: -- Summary: When the master branch executes mixed reading and writing, it reports Too many open files and read only Key: IOTDB-2056 URL: https://issues.apache.org/jira/browse/IOTDB-2056 Project: Apache IoTDB Issue Type: Bug Components: Core/Server Affects Versions: master branch Reporter: xiaozhihong Attachments: rw_no_overflow.config.properties In the weekly test version from iotdb_11-22_f8c18ac to iotdb_11-24_1f853f1, when using the benchmark to mix reading and writing, you can see in the log: warn prompts read only, error reports Too many open files step 1：start server step 2：using the benchmark to mix reading and writing step 3：see logs -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2055) degrade server side's log level when the parameters in insertRecord() are invalid
Xiangdong Huang created IOTDB-2055:
--
Summary: degrade server side's log level when the parameters in
insertRecord() are invalid
Key: IOTDB-2055
URL: https://issues.apache.org/jira/browse/IOTDB-2055
Project: Apache IoTDB
Issue Type: Improvement
Reporter: Xiangdong Huang
[https://github.com/apache/iotdb/issues/4446] shows an unpredictable behavior
but it has been fixed in v0.12.3.
>From v0.12.3 on, the server side will check whether the parameters are valid,
>and show exception errors on the server-side and return failure to clients.
The client-side:
{panel}
Exception in thread "main" org.apache.iotdb.rpc.StatementExecutionException:
313: failed to insert measurements [TL_JS06.R_50NeckTemp2] caused by
TL_JS06.R_50NeckTemp2 is an illegal measurementId
at org.apache.iotdb.rpc.RpcUtils.verifySuccess(RpcUtils.java:86)
at org.apache.iotdb.rpc.RpcUtils.verifySuccessWithRedirection(RpcUtils.java:92)
at
org.apache.iotdb.session.SessionConnection.insertRecord(SessionConnection.java:465)
at org.apache.iotdb.session.Session.insertRecord(Session.java:742)
at org.apache.iotdb.session.Session.insertRecord(Session.java:902)
at org.apache.iotdb.SessionExample.insertRecord(SessionExample.java:41)
at org.apache.iotdb.SessionExample.main(SessionExample.java:57)
{panel}
The server-side:
{panel}
2021-11-24 21:03:08,614 [pool-14-IoTDB-RPC-Client-2] WARN
o.a.i.d.metadata.MManager:1966 - meet error when check
root.test.SD-T-JS006.TL_JS06.R_50NeckTemp2, message: TL_JS06.R_50NeckTemp2 is
an illegal measurementId
2021-11-24 21:03:08,615 [pool-14-IoTDB-RPC-Client-2] WARN
o.a.i.d.s.TSServiceImpl:1957 - Exception occurred while processing non-query.
org.apache.iotdb.db.exception.query.QueryProcessException:
org.apache.iotdb.db.exception.StorageEngineException: failed to insert
measurements [TL_JS06.R_50NeckTemp2] caused by TL_JS06.R_50NeckTemp2 is an
illegal measurementId
at org.apache.iotdb.db.qp.executor.PlanExecutor.insert(PlanExecutor.java:1392)
at
org.apache.iotdb.db.qp.executor.PlanExecutor.processNonQuery(PlanExecutor.java:246)
at
org.apache.iotdb.db.service.TSServiceImpl.executeNonQuery(TSServiceImpl.java:1150)
at
org.apache.iotdb.db.service.TSServiceImpl.executeNonQueryPlan(TSServiceImpl.java:1888)
at
org.apache.iotdb.db.service.TSServiceImpl.insertStringRecord(TSServiceImpl.java:1496)
at
org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertStringRecord.getResult(TSIService.java:2798)
at
org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertStringRecord.getResult(TSIService.java:2778)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.iotdb.db.exception.StorageEngineException: failed to
insert measurements [TL_JS06.R_50NeckTemp2] caused by TL_JS06.R_50NeckTemp2 is
an illegal measurementId
at
org.apache.iotdb.db.qp.executor.PlanExecutor.checkFailedMeasurments(PlanExecutor.java:1292)
at org.apache.iotdb.db.qp.executor.PlanExecutor.insert(PlanExecutor.java:1386)
... 12 common frames omitted
{panel}
The info in the client-side is ok, but the server-side info has improper log
level: it is just caused by an incorrect client input, just a{color:#FF}
debug level {color}message is fine.
Otherwise, if someone sends a lot of such incorrect requests, the server will
be attracted by too many logs.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (IOTDB-2033) Survey Buffer-based Hashmap
[ https://issues.apache.org/jira/browse/IOTDB-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448441#comment-17448441 ] Kaifeng Xue commented on IOTDB-2033: Survey doc: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195727895 > Survey Buffer-based Hashmap > --- > > Key: IOTDB-2033 > URL: https://issues.apache.org/jira/browse/IOTDB-2033 > Project: Apache IoTDB > Issue Type: Task >Reporter: Jialin Qiao >Assignee: Kaifeng Xue >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > We need a memory-friendly HashMap implementation for IDTable in the new > SchemaManager. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-1895) Cache leader optimization for batch write interfaces on multiple devices
[ https://issues.apache.org/jira/browse/IOTDB-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448443#comment-17448443 ] 刘珍 commented on IOTDB-1895: --- PASS. [^SessionExample.java] rc2 session： real0m20.598s rc1 session： real0m37.477s > Cache leader optimization for batch write interfaces on multiple devices > > > Key: IOTDB-1895 > URL: https://issues.apache.org/jira/browse/IOTDB-1895 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Xinyu Tan >Assignee: Quan Siyi >Priority: Major > Labels: pull-request-available > Attachments: SessionExample.java > > Original Estimate: 24h > Remaining Estimate: 24h > > At present, the cache leader performance optimization for distributed write > performance is critical ([the design > document|https://shimo.im/docs/dYwjwtrTPXPWcqxj]) . Although the current > implementation in the session module for all equipment cache handling, On the > server side, however, only writes to a single device are passed clues, > causing the partition cache of the batch interface to be effectively > invalidated. As the number of users of multi-device interfaces increases, we > need to do this optimization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
