Joe McDonnell created IMPALA-12669: -------------------------------------- Summary: DiskIoMgr should not allocate threads for loop devices Key: IMPALA-12669 URL: https://issues.apache.org/jira/browse/IMPALA-12669 Project: IMPALA Issue Type: Improvement Components: Backend Affects Versions: Impala 4.4.0 Reporter: Joe McDonnell
On my development machine, Ubuntu's snap package system has various /snap directories mounted as loop devices: {noformat} /dev/loop3 83328 83328 0 100% /snap/gtk-common-themes/1534 /dev/loop2 508928 508928 0 100% /snap/gnome-42-2204/132 /dev/loop5 41856 41856 0 100% /snap/snapd/20290 /dev/loop4 47104 47104 0 100% /snap/snap-store/638 /dev/loop7 508928 508928 0 100% /snap/gnome-42-2204/141 /dev/loop8 75776 75776 0 100% /snap/core22/864 /dev/loop9 65024 65024 0 100% /snap/core20/2015 /dev/loop10 358144 358144 0 100% /snap/gnome-3-38-2004/140 /dev/loop12 12672 12672 0 100% /snap/snap-store/959 {noformat} DiskIoMgr is allocating 8 threads for each of these devices, so my system has 344 disk-io-mgr threads of which 152 are for loop devices. This also causes there to be an equivalent number of hdfs-monitor threads. This makes minidumps larger and take longer to process. It also slows down pstack and other tools. These loop devices are not real devices and they shouldn't matter in a production cluster. DiskIoMgr should not maintain a queue and spawn threads for them. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org