[GitHub] [incubator-hugegraph-toolchain] wuchaojing opened a new issue, #449: [Bug] 边shard不均导致backup很慢

[via GitHub]

wuchaojing opened a new issue, #449:
URL: https://github.com/apache/incubator-hugegraph-toolchain/issues/449

   ### Bug Type (问题类型)
   
   performance (性能下降)
   
   ### Before submit
   
   - [X] 我已经确认现有的 [Issues](https://github.com/apache/hugegraph/issues) 与 
[FAQ](https://hugegraph.apache.org/docs/guides/faq/) 中没有相同 / 重复问题 (I have 
confirmed and searched that there are no similar problems in the historical 
issue and documents)
   
   ### Environment (环境信息)
   
   - Server Version: 1.0.0 (Apache Release Version)
   - Backend: RocksDB x nodes, HDD or SSD 
   - OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x 
   - Data Size:  xx vertices, xx edges <!-- (like 1000W 点, 9000W 边) -->
   
   
   ### Expected & Actual behavior (期望与实际表现)
   
   执行边的备份，发现非常慢（点备份很快），命令为：
   `bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t edge 
--directory ./backup_test`
   
   
通过源码发现问题应该是边shard不均匀，导致大部分的线程执行完各自的小分片之后退出，最后只会有1～2个线程在执行较大的分片。虽然配置了多线程，但最后只有一个线程在执行备份，其他线程均退出。
   
   <img width="504" alt="image" 
src="https://user-images.githubusercontent.com/37741352/230309159-ae1229f4-a3ee-4916-8d54-e8f671d53126.png";>
   
   也打印了线程池的活跃情况，一开始全部活跃，到最后只有2个活跃。（忽略线程数与备份生成的文件数不一致的情况，修改过线程数）
   <img width="1186" alt="image" 
src="https://user-images.githubusercontent.com/37741352/230309830-85661d90-4d4c-4936-9e42-26fe3fa24fc5.png";>
   
   <img width="1083" alt="image" 
src="https://user-images.githubusercontent.com/37741352/230309512-4c330b8c-b726-4110-b5de-b866d8e1ad50.png";>
   
   
   ### Vertex/Edge example (问题点 / 边数据举例)
   
   _No response_
   
   ### Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@hugegraph.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org