0x55aa opened a new issue, #8871:
URL: https://github.com/apache/iotdb/issues/8871
**environment:**
version: 1.0.0
deployment: all in one, one confignode, one datanode
host: cpu: 16core, mem:16G, disk: ssd, os: ubuntu 22.04
client sdk: python, use normal tablet
**background:**
the measurements number of a batch of data is huge, in my usage scenario,
sometimes it is 200,000.
**experiment:**
I tested the insertion performance.
Insertion takes 2.5s-4s when all measurement points are inserted in one
database and the measurements number of a batch is 200,000.
Insertion takes 0.3s when all measurement points are inserted in 10
databases of one instance and 10 threads, one thread insert 20,000 measurements
once.
I looked at the usage performance of host, cpu only used under 200%, memory
is still remain, disk io is 8MB/s.
None of this is a bottleneck. so i strace the process:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ------------------
79.04 52.355396 1545 33867 4478 futex
8.88 5.880482 2816 2088 epoll_pwait2
5.25 3.480728 102374 34 15 restart_syscall
4.60 3.049973 7682 397 epoll_wait
1.84 1.217815 1363 893 recvfrom
0.19 0.124981 17 7105 writev
0.10 0.065069 10 6045 read
0.03 0.023011 442 52 pwrite64
0.02 0.013909 5 2750 write
0.01 0.005350 137 39 fdatasync
```
in strace log, many lock timeout:
```
335339 16:00:27.993273 futex(0x7f1b240f6028, FUTEX_WAKE_PRIVATE, 1) = 0
<0.000011>
335339 16:00:27.993317 futex(0x7f1b240f6078, FUTEX_WAIT_BITSET_PRIVATE, 0,
{tv_sec=359621, tv_nsec=748782361}, FUTEX_BITSET_MATCH_ANY <unfinished ...>
335381 16:00:27.995534 <... futex resumed>) = -1 ETIMEDOUT (Connection timed
out) <0.200098>
335381 16:00:27.995648 futex(0x7f19e0025718, FUTEX_WAKE_PRIVATE, 1) = 0
<0.000100>
335381 16:00:27.995913 futex(0x7f19e0025768, FUTEX_WAIT_BITSET_PRIVATE, 0,
{tv_sec=359621, tv_nsec=901319853}, FUTEX_BITSET_MATCH_ANY <unfinished ...>
335454 16:00:27.996630 <... futex resumed>) = -1 ETIMEDOUT (Connection timed
out) <0.100112>
335454 16:00:27.996656 futex(0x7f19e00b70d8, FUTEX_WAKE_PRIVATE, 1) = 0
<0.000009>
335454 16:00:27.996762 futex(0x7f19e00b7128, FUTEX_WAIT_BITSET_PRIVATE, 0,
{tv_sec=359621, tv_nsec=802178374}, FUTEX_BITSET_MATCH_ANY <unfinished ...>
```
so i traceing java process use jstack, but I can't read the contents, but
there is code related to locks.
**question:**
1. In cases where there are so many measurement points, How do I need to
optimize it? I look at the documentation and there are too many configuration
parameters.
https://iotdb.apache.org/UserGuide/Master/Reference/Common-Config-Manual.html
2. Can it improve performance, if i use cluster mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@iotdb.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
