Looks like I could archive about 1.3TB, of a 1.64TB snapshot, before things started to go bad, again:
[110079.681102] INFO: task z_wr_iss_h:2171 blocked for more than 120 seconds. [110079.688123] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.695167] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.703260] z_wr_iss_h D 0 2171 2 0x00000028 [110079.703276] Call trace: [110079.703297] __switch_to+0x104/0x170 [110079.703308] __schedule+0x30c/0x7c0 [110079.703317] schedule+0x3c/0xb8 [110079.703325] io_schedule+0x20/0x58 [110079.703338] rq_qos_wait+0x100/0x178 [110079.703347] wbt_wait+0xb4/0xf0 [110079.703357] __rq_qos_throttle+0x38/0x50 [110079.703370] blk_mq_make_request+0x128/0x610 [110079.703382] generic_make_request+0xb4/0x2d8 [110079.703392] submit_bio+0x48/0x218 [110079.703629] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.703850] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.704068] zio_nowait+0xd4/0x170 [zfs] [110079.704285] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110079.704503] zio_vdev_io_start+0x248/0x2b8 [zfs] [110079.704720] zio_execute+0xac/0x110 [zfs] [110079.704760] taskq_thread+0x2f8/0x570 [spl] [110079.704773] kthread+0xfc/0x128 [110079.704784] ret_from_fork+0x10/0x1c [110079.704795] INFO: task z_wr_int:2173 blocked for more than 120 seconds. [110079.711650] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.718668] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.726745] z_wr_int D 0 2173 2 0x00000028 [110079.726760] Call trace: [110079.726780] __switch_to+0x104/0x170 [110079.726790] __schedule+0x30c/0x7c0 [110079.726803] schedule+0x3c/0xb8 [110079.726811] io_schedule+0x20/0x58 [110079.726823] rq_qos_wait+0x100/0x178 [110079.726832] wbt_wait+0xb4/0xf0 [110079.726842] __rq_qos_throttle+0x38/0x50 [110079.726854] blk_mq_make_request+0x128/0x610 [110079.726866] generic_make_request+0xb4/0x2d8 [110079.726877] submit_bio+0x48/0x218 [110079.727100] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.727320] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.727549] zio_nowait+0xd4/0x170 [zfs] [110079.727766] vdev_queue_io_done+0x1ec/0x2a0 [zfs] [110079.727991] zio_vdev_io_done+0xec/0x220 [zfs] [110079.728209] zio_execute+0xac/0x110 [zfs] [110079.728255] taskq_thread+0x2f8/0x570 [spl] [110079.728268] kthread+0xfc/0x128 [110079.728279] ret_from_fork+0x10/0x1c [110079.728309] INFO: task txg_sync:2388 blocked for more than 120 seconds. [110079.735157] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.742175] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.750306] txg_sync D 0 2388 2 0x00000028 [110079.750321] Call trace: [110079.750337] __switch_to+0x104/0x170 [110079.750348] __schedule+0x30c/0x7c0 [110079.750356] schedule+0x3c/0xb8 [110079.750366] schedule_timeout+0x9c/0x190 [110079.750374] io_schedule_timeout+0x28/0x48 [110079.750412] __cv_timedwait_common+0x1a8/0x1f8 [spl] [110079.750447] __cv_timedwait_io+0x3c/0x50 [spl] [110079.750670] zio_wait+0x130/0x2a0 [zfs] [110079.750890] dsl_pool_sync+0x3fc/0x498 [zfs] [110079.751109] spa_sync+0x538/0xe68 [zfs] [110079.751326] txg_sync_thread+0x2c0/0x468 [zfs] [110079.751364] thread_generic_wrapper+0x74/0xa0 [spl] [110079.751377] kthread+0xfc/0x128 [110079.751388] ret_from_fork+0x10/0x1c [110079.751459] INFO: task borg:1063669 blocked for more than 120 seconds. [110079.758224] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.765237] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.773310] borg D 0 1063669 1063661 0x00000000 [110079.773324] Call trace: [110079.773344] __switch_to+0x104/0x170 [110079.773355] __schedule+0x30c/0x7c0 [110079.773366] schedule+0x3c/0xb8 [110079.773406] cv_wait_common+0x188/0x1b0 [spl] [110079.773441] __cv_wait+0x30/0x40 [spl] [110079.773664] zil_commit_impl+0x234/0xd30 [zfs] [110079.773882] zil_commit+0x48/0x70 [zfs] [110079.774112] zfs_write+0xa3c/0xb90 [zfs] [110079.774331] zpl_write_common_iovec+0xac/0x120 [zfs] [110079.774557] zpl_iter_write+0xe4/0x150 [zfs] [110079.774569] new_sync_write+0x100/0x1a8 [110079.774578] __vfs_write+0x74/0x90 [110079.774593] vfs_write+0xe4/0x1c8 [110079.774603] ksys_write+0x78/0x100 [110079.774612] __arm64_sys_write+0x24/0x30 [110079.774622] el0_svc_common.constprop.0+0x80/0x218 [110079.774631] el0_svc_handler+0x34/0xa0 [110079.774640] el0_svc+0x10/0x2cc [110079.774665] INFO: task z_wr_iss_h:3853507 blocked for more than 120 seconds. [110079.781953] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.788973] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.797052] z_wr_iss_h D 0 3853507 2 0x00000028 [110079.797067] Call trace: [110079.797083] __switch_to+0x104/0x170 [110079.797094] __schedule+0x30c/0x7c0 [110079.797102] schedule+0x3c/0xb8 [110079.797110] io_schedule+0x20/0x58 [110079.797123] rq_qos_wait+0x100/0x178 [110079.797132] wbt_wait+0xb4/0xf0 [110079.797142] __rq_qos_throttle+0x38/0x50 [110079.797154] blk_mq_make_request+0x128/0x610 [110079.797165] generic_make_request+0xb4/0x2d8 [110079.797176] submit_bio+0x48/0x218 [110079.797398] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.797616] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.797834] zio_nowait+0xd4/0x170 [zfs] [110079.798050] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110079.798267] zio_vdev_io_start+0x248/0x2b8 [zfs] [110079.798484] zio_execute+0xac/0x110 [zfs] [110079.798522] taskq_thread+0x2f8/0x570 [spl] [110079.798535] kthread+0xfc/0x128 [110079.798545] ret_from_fork+0x10/0x1c [110079.798674] INFO: task z_wr_iss_h:3853521 blocked for more than 120 seconds. [110079.805950] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.812940] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.820986] z_wr_iss_h D 0 3853521 2 0x00000028 [110079.820998] Call trace: [110079.821007] __switch_to+0x104/0x170 [110079.821014] __schedule+0x30c/0x7c0 [110079.821017] schedule+0x3c/0xb8 [110079.821020] io_schedule+0x20/0x58 [110079.821026] rq_qos_wait+0x100/0x178 [110079.821031] wbt_wait+0xb4/0xf0 [110079.821035] __rq_qos_throttle+0x38/0x50 [110079.821040] blk_mq_make_request+0x128/0x610 [110079.821045] generic_make_request+0xb4/0x2d8 [110079.821050] submit_bio+0x48/0x218 [110079.821157] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.821248] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.821343] zio_nowait+0xd4/0x170 [zfs] [110079.821430] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110079.821521] zio_vdev_io_start+0x248/0x2b8 [zfs] [110079.821608] zio_execute+0xac/0x110 [zfs] [110079.821627] taskq_thread+0x2f8/0x570 [spl] [110079.821633] kthread+0xfc/0x128 [110079.821638] ret_from_fork+0x10/0x1c [110079.821643] INFO: task z_wr_iss_h:3853701 blocked for more than 120 seconds. [110079.828906] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.835893] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.843938] z_wr_iss_h D 0 3853701 2 0x00000028 [110079.843944] Call trace: [110079.843951] __switch_to+0x104/0x170 [110079.843956] __schedule+0x30c/0x7c0 [110079.843959] schedule+0x3c/0xb8 [110079.843962] io_schedule+0x20/0x58 [110079.843967] rq_qos_wait+0x100/0x178 [110079.843971] wbt_wait+0xb4/0xf0 [110079.843975] __rq_qos_throttle+0x38/0x50 [110079.843980] blk_mq_make_request+0x128/0x610 [110079.843985] generic_make_request+0xb4/0x2d8 [110079.843989] submit_bio+0x48/0x218 [110079.844083] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.844170] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.844257] zio_nowait+0xd4/0x170 [zfs] [110079.844343] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110079.844430] zio_vdev_io_start+0x248/0x2b8 [zfs] [110079.844517] zio_execute+0xac/0x110 [zfs] [110079.844533] taskq_thread+0x2f8/0x570 [spl] [110079.844539] kthread+0xfc/0x128 [110079.844543] ret_from_fork+0x10/0x1c [110079.844547] INFO: task z_wr_iss_h:3853747 blocked for more than 120 seconds. [110079.851801] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.858786] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.866829] z_wr_iss_h D 0 3853747 2 0x00000028 [110079.866835] Call trace: [110079.866844] __switch_to+0x104/0x170 [110079.866849] __schedule+0x30c/0x7c0 [110079.866853] schedule+0x3c/0xb8 [110079.866857] io_schedule+0x20/0x58 [110079.866862] rq_qos_wait+0x100/0x178 [110079.866866] wbt_wait+0xb4/0xf0 [110079.866870] __rq_qos_throttle+0x38/0x50 [110079.866875] blk_mq_make_request+0x128/0x610 [110079.866880] generic_make_request+0xb4/0x2d8 [110079.866884] submit_bio+0x48/0x218 [110079.866980] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.867071] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.867166] zio_nowait+0xd4/0x170 [zfs] [110079.867253] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110079.867344] zio_vdev_io_start+0x248/0x2b8 [zfs] [110079.867431] zio_execute+0xac/0x110 [zfs] [110079.867451] taskq_thread+0x2f8/0x570 [spl] [110079.867457] kthread+0xfc/0x128 [110079.867462] ret_from_fork+0x10/0x1c [110079.867466] INFO: task z_wr_int:3853774 blocked for more than 121 seconds. [110079.874546] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110079.881532] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110079.889574] z_wr_int D 0 3853774 2 0x00000028 [110079.889580] Call trace: [110079.889588] __switch_to+0x104/0x170 [110079.889592] __schedule+0x30c/0x7c0 [110079.889595] schedule+0x3c/0xb8 [110079.889599] io_schedule+0x20/0x58 [110079.889604] rq_qos_wait+0x100/0x178 [110079.889607] wbt_wait+0xb4/0xf0 [110079.889611] __rq_qos_throttle+0x38/0x50 [110079.889616] blk_mq_make_request+0x128/0x610 [110079.889621] generic_make_request+0xb4/0x2d8 [110079.889625] submit_bio+0x48/0x218 [110079.889721] vdev_disk_io_start+0x670/0x9f8 [zfs] [110079.889809] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110079.889896] zio_nowait+0xd4/0x170 [zfs] [110079.889982] vdev_queue_io_done+0x1ec/0x2a0 [zfs] [110079.890069] zio_vdev_io_done+0xec/0x220 [zfs] [110079.890156] zio_execute+0xac/0x110 [zfs] [110079.890172] taskq_thread+0x2f8/0x570 [spl] [110079.890177] kthread+0xfc/0x128 [110079.890181] ret_from_fork+0x10/0x1c [110200.513965] INFO: task z_wr_iss_h:2171 blocked for more than 241 seconds. [110200.520988] Tainted: P C OE 5.4.0-1018-raspi #20-Ubuntu [110200.528031] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [110200.536134] z_wr_iss_h D 0 2171 2 0x00000028 [110200.536150] Call trace: [110200.536170] __switch_to+0x104/0x170 [110200.536182] __schedule+0x30c/0x7c0 [110200.536191] schedule+0x3c/0xb8 [110200.536199] io_schedule+0x20/0x58 [110200.536212] rq_qos_wait+0x100/0x178 [110200.536222] wbt_wait+0xb4/0xf0 [110200.536232] __rq_qos_throttle+0x38/0x50 [110200.536244] blk_mq_make_request+0x128/0x610 [110200.536256] generic_make_request+0xb4/0x2d8 [110200.536266] submit_bio+0x48/0x218 [110200.536505] vdev_disk_io_start+0x670/0x9f8 [zfs] [110200.536726] zio_vdev_io_start+0xdc/0x2b8 [zfs] [110200.536945] zio_nowait+0xd4/0x170 [zfs] [110200.537161] vdev_mirror_io_start+0xa8/0x1b0 [zfs] [110200.537380] zio_vdev_io_start+0x248/0x2b8 [zfs] [110200.537597] zio_execute+0xac/0x110 [zfs] [110200.537638] taskq_thread+0x2f8/0x570 [spl] [110200.537651] kthread+0xfc/0x128 [110200.537662] ret_from_fork+0x10/0x1c -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899249 Title: OpenZFS writing stalls, under load To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1899249/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
