Re: No space left on device (28)
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Am 25.03.2013 21:14, schrieb Josef Bacik:
On Fri, Mar 22, 2013 at 02:55:07PM -0600, Stefan Priebe wrote:
Hi Jsoef,
thanks!
Ok I don't think the thing I just fixed will make any difference for you so
here's another debug patch, just apply it on top of what I've already sent you
and re-run and give me the dmesg again. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index aabaea6..bf6433f 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4162,7 +4162,11 @@ out:
}
if (flushing) {
if (ret == -ENOSPC) {
- printk(KERN_ERR returning enospc, dumping space
info\n);
+ spin_lock(block_rsv-lock);
+ printk(KERN_ERR returning enospc, space_info %u, size %Lu
reserved %Lu,
+ flush %d, flush_state %d dumping space info\n,
block_rsv-type,
+ block_rsv-size, block_rsv-reserved, flush,
flush_state);
+ spin_unlock(block_rsv-lock);
dump_space_info(space_info, 0, 0);
}
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 13:53, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: Hi, output here: [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.548280] space_info 4 has 6439292928 free, is full [ 590.548283] space_info total=25748307968, used=19308916736, pinned=0, reserved=32768, may_use=6438354944, readonly=65536 [ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.552264] space_info 4 has 6439284736 free, is full [ 590.552267] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.556258] space_info 4 has 6439284736 free, is full [ 590.556261] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 591.147373] space_info 4 has 6439235584 free, is full [ 591.147375] space_info total=25748307968, used=19308916736, pinned=0, reserved=90112, may_use=6438354944, readonly=65536 [ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 595.562390] space_info 4 has 6439120896 free, is full [ 595.562393] space_info total=25748307968, used=19309047808, pinned=0, reserved=73728, may_use=6438297600, readonly=65536 Weird, we have all the flushing stuff set and yet there is still a whole lot of outstanding reservations. Do you have compression enabled? Thanks, no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote:
Hi Josef,
Am 26.03.2013 13:53, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote:
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Weird, we have all the flushing stuff set and yet there is still a whole
lot of
outstanding reservations. Do you have compression enabled? Thanks,
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it fixes
it? Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index bf6433f..84e8909 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3803,6 +3803,19 @@ static int can_overcommit(struct btrfs_root *root,
return 0;
}
+static int btrfs_try_writeback(struct super_block *sb, unsigned long nr_pages,
+ enum wb_reason reason)
+{
+ if (!writeback_in_progress(sb-s_bdi)
+ down_read_trylock(sb-s_umount)) {
+ writeback_inodes_sb_nr(sb, nr_pages, reason);
+ up_read(sb-s_umount);
+ return 1;
+ }
+
+ return 0;
+}
+
void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
unsigned long nr_pages)
{
@@ -3810,8 +3823,7 @@ void btrfs_writeback_inodes_sb_nr(struct btrfs_root *root,
int started;
/* If we can not start writeback, just sync all the delalloc file. */
- started = try_to_writeback_inodes_sb_nr(sb, nr_pages,
- WB_REASON_FS_FREE_SPACE);
+ started = btrfs_try_writeback(sb, nr_pages, WB_REASON_FS_FREE_SPACE);
if (!started) {
/*
* We needn't worry the filesystem going from r/w to r/o though
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 14:30, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote: Hi Josef, Am 26.03.2013 13:53, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: Hi, output here: [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.548280] space_info 4 has 6439292928 free, is full [ 590.548283] space_info total=25748307968, used=19308916736, pinned=0, reserved=32768, may_use=6438354944, readonly=65536 [ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.552264] space_info 4 has 6439284736 free, is full [ 590.552267] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.556258] space_info 4 has 6439284736 free, is full [ 590.556261] space_info total=25748307968, used=19308916736, pinned=0, reserved=40960, may_use=6438354944, readonly=65536 [ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 591.147373] space_info 4 has 6439235584 free, is full [ 591.147375] space_info total=25748307968, used=19308916736, pinned=0, reserved=90112, may_use=6438354944, readonly=65536 [ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 595.562390] space_info 4 has 6439120896 free, is full [ 595.562393] space_info total=25748307968, used=19309047808, pinned=0, reserved=73728, may_use=6438297600, readonly=65536 Weird, we have all the flushing stuff set and yet there is still a whole lot of outstanding reservations. Do you have compression enabled? Thanks, no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 07:49:16AM -0600, Stefan Priebe - Profihost AG wrote:
Hi Josef,
Am 26.03.2013 14:30, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG
wrote:
Hi Josef,
Am 26.03.2013 13:53, schrieb Josef Bacik:
On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote:
Hi,
output here:
[ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.548280] space_info 4 has 6439292928 free, is full
[ 590.548283] space_info total=25748307968, used=19308916736, pinned=0,
reserved=32768, may_use=6438354944, readonly=65536
[ 590.550147] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.552264] space_info 4 has 6439284736 free, is full
[ 590.552267] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 590.554141] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 590.556258] space_info 4 has 6439284736 free, is full
[ 590.556261] space_info total=25748307968, used=19308916736, pinned=0,
reserved=40960, may_use=6438354944, readonly=65536
[ 591.145255] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 591.147373] space_info 4 has 6439235584 free, is full
[ 591.147375] space_info total=25748307968, used=19308916736, pinned=0,
reserved=90112, may_use=6438354944, readonly=65536
[ 595.560257] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 595.562390] space_info 4 has 6439120896 free, is full
[ 595.562393] space_info total=25748307968, used=19309047808, pinned=0,
reserved=73728, may_use=6438297600, readonly=65536
Weird, we have all the flushing stuff set and yet there is still a whole
lot of
outstanding reservations. Do you have compression enabled? Thanks,
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it
fixes
it? Thanks,
It still does not fix the problem.
The rsync output looks like this so it does not work for file a but then
continues on c d e, ...
sync -av --progress /backup/ /mnt/
sending incremental file list
.etc_openvpn/ipp.txt
229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196)
.etc_openvpn/openvpn-status.log
360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196)
rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX -
.etc_openvpn/ipp.txt: No space left on device (28)
.log/
.log/UcliEvt.log
104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700)
.log/auth.log
15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700)
.log/auth.log.1
19431424 61%7.35MB/s0:00:01
the dmesg output looks like this:
[ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 551.323694] space_info 4 has 6439526400 free, is full
[ 551.323696] space_info total=25748307968, used=19308666880, pinned=0,
reserved=49152, may_use=6438453248, readonly=65536
Ok so then this is probably it, let me know if it helps. Thanks,
Josef
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index dc08d77..005c45d 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -557,6 +557,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
INIT_LIST_HEAD(splice);
INIT_LIST_HEAD(works);
+ mutex_lock(root-fs_info-ordered_operations_mutex);
spin_lock(root-fs_info-ordered_extent_lock);
list_splice_init(root-fs_info-ordered_extents, splice);
while (!list_empty(splice)) {
@@ -600,6 +601,7 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root,
int delay_iput)
cond_resched();
}
+ mutex_unlock(root-fs_info-ordered_operations_mutex);
}
/*
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Then this again: rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.vhost_net.mod.GrhWet - .software/kernel/linux-3.9-rc3/.tmp_versions/vhost_net.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.via-rhine.mod.X2Ofhz - .software/kernel/linux-3.9-rc3/.tmp_versions/via-rhine.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.via-rng.mod.RAyxkF - .software/kernel/linux-3.9-rc3/.tmp_versions/via-rng.mod: No space left on device (28) rsync: rename /mnt/.software/kernel/linux-3.9-rc3/.tmp_versions/.virtio.mod.w7INoL - .software/kernel/linux-3.9-rc3/.tmp_versions/virtio.mod: No space left on device (28)^C [ 5012.468988] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.472981] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.472982] space_info 4 has 6439399424 free, is full [ 5012.472982] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.476974] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.476975] space_info 4 has 6439399424 free, is full [ 5012.476976] space_info total=25748307968, used=19308773376, pinned=0, reserved=69632, may_use=6438354944, readonly=65536 [ 5012.480968] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 5012.480969] space_info 4 has 6439399424 free, is full Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote:
Hi,
Am 26.03.2013 15:44, schrieb Josef Bacik:
Am 26.03.2013 13:53, schrieb Josef Bacik:
no - it's just mounted with mount -o noatime
:~# cat /proc/mounts | grep btrfs
/dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
Ok I think I see what's going on. Can you try this patch and see if it
fixes
it? Thanks,
It still does not fix the problem.
The rsync output looks like this so it does not work for file a but then
continues on c d e, ...
sync -av --progress /backup/ /mnt/
sending incremental file list
.etc_openvpn/ipp.txt
229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196)
.etc_openvpn/openvpn-status.log
360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196)
rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX -
.etc_openvpn/ipp.txt: No space left on device (28)
.log/
.log/UcliEvt.log
104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700)
.log/auth.log
15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700)
.log/auth.log.1
19431424 61%7.35MB/s0:00:01
the dmesg output looks like this:
[ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
2, flush_state 7 dumping space info
[ 551.323694] space_info 4 has 6439526400 free, is full
[ 551.323696] space_info total=25748307968, used=19308666880, pinned=0,
reserved=49152, may_use=6438453248, readonly=65536
Ok so then this is probably it, let me know if it helps. Thanks,
OK it now has copied a lot of files (170) without an error all were very
small.
Welp progress is good. Throw this into the mix and go again, it's just adding
some more debugging so I can make sure I'm going down the right rabbit hole.
Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 84e8909..1cf810a 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4026,6 +4026,15 @@ static int flush_space(struct btrfs_root *root,
return ret;
}
+
+static void dump_block_rsv(struct btrfs_block_rsv *block_rsv)
+{
+ spin_lock(block_rsv-lock);
+ printk(KERN_ERR dumping block rsv %u, size %Lu reserved %Lu\n,
+ block_rsv-type, block_rsv-size, block_rsv-reserved);
+ spin_unlock(block_rsv-lock);
+}
+
/**
* reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
* @root - the root we're allocating for
@@ -4179,6 +4188,9 @@ out:
flush %d, flush_state %d dumping space
info\n, block_rsv-type,
block_rsv-size, block_rsv-reserved, flush,
flush_state);
spin_unlock(block_rsv-lock);
+ dump_block_rsv(root-fs_info-delalloc_block_rsv);
+ dump_block_rsv(root-fs_info-delayed_block_rsv);
+ dump_block_rsv(root-fs_info-global_block_rsv);
dump_space_info(space_info, 0, 0);
}
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
HI, Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 10:19:19AM -0600, Stefan Priebe wrote: HI, Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Josef diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1cf810a..ac415cf7 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4471,7 +4471,12 @@ static void update_global_block_rsv(struct btrfs_fs_info *fs_info) spin_lock(sinfo-lock); spin_lock(block_rsv-lock); - block_rsv-size = num_bytes; + /* +* Limit the global block rsv to 512mb, we have infrastructure in place +* to throttle reservations if we start getting low on global block rsv +* space. +*/ + block_rsv-size = min_t(u64, num_bytes, 512 * 1024 * 1024); num_bytes = sinfo-bytes_used + sinfo-bytes_pinned + sinfo-bytes_reserved + sinfo-bytes_readonly + -- 1.7.7.6 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs raid0? Will the fix be part of 3.9-rc5? Thanks and greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 01:05:36PM -0600, Stefan Priebe wrote: Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs raid0? Will the fix be part of 3.9-rc5? So I'll put together all of the patches that actually need to go up for this and post them, but basically its the mutex patch, the last patch I sent you and the one that adjusts the reservations for rename and delete. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [20368.895140] rsync D 8160f580 0 14911 1 0x [20368.895148] 8801ca63fc78 0086 8800c28f8198 88022394f800 [20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8 00012c40 [20368.895163] 81a11440 8801c9d36340 8801ca63fc88 8801cefce130 [20368.895169] Call Trace: [20368.895180] [8151a774] schedule+0x24/0x70 [20368.895207] [a0158c75] wait_current_trans.isra.32+0x95/0x100 [btrfs] [20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60 [20368.895236] [a015a45d] start_transaction.part.33+0x13d/0x4d0 [btrfs] [20368.895252] [811420f3] ? inode_permission+0x13/0x50 [20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs] [20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20 [btrfs] [20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs] [20368.895307] [8150ee3e] ? check_acl+0x5a/0x122 [20368.895312] [81055ff0] ? ns_capable+0x30/0x60 [20368.895317] [811413bd] ? generic_permission+0xbd/0x110 [20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs] [20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0 [20368.895345] [81143ad7] vfs_unlink+0x37/0x50 [20368.895349] [81143c8b] do_unlinkat+0x19b/0x240 [20368.895354] [81146171] sys_unlink+0x11/0x20 [20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b Speed is just 100kb/s instead of 100MB/s. Stefan Am 26.03.2013 20:16, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:05:36PM -0600, Stefan Priebe wrote: Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 Ok I think I see what's going on. Can you try this patch and see if it fixes it? Thanks, It still does not fix the problem. The rsync output looks like this so it does not work for file a but then continues on c d e, ... sync -av --progress /backup/ /mnt/ sending incremental file list .etc_openvpn/ipp.txt 229 100%3.99kB/s0:00:00 (xfer#2, to-check=1009/1196) .etc_openvpn/openvpn-status.log 360 100%6.28kB/s0:00:00 (xfer#3, to-check=1007/1196) rsync: rename /mnt/.etc_openvpn/.ipp.txt.t9lucX - .etc_openvpn/ipp.txt: No space left on device (28) .log/ .log/UcliEvt.log 104188 100% 147.67kB/s0:00:00 (xfer#4, to-check=1131/2700) .log/auth.log 15211522 100%2.97MB/s0:00:04 (xfer#5, to-check=1105/2700) .log/auth.log.1 19431424 61%7.35MB/s0:00:01 the dmesg output looks like this: [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 551.323694] space_info 4 has 6439526400 free, is full [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, reserved=49152, may_use=6438453248, readonly=65536 Ok so then this is probably it, let me know if it helps. Thanks, OK it now has copied a lot of files (170) without an error all were very small. Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Output is now: [ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.527392] dumping block rsv 2, size 0 reserved 0 [ 9587.567871] dumping block rsv 5, size 196608 reserved 196608 [ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.646958] space_info 4 has 6439428096 free, is full [ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438453248, readonly=65536 [ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 9587.727000] dumping block rsv 2, size 0 reserved 0 [ 9587.765284] dumping block rsv 5, size 98304 reserved 98304 [ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640 [ 9587.839935] space_info 4 has 6439428096 free, is full [ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, reserved=45056, may_use=6438354944, readonly=65536 Well then that looks like I was going down the wrong rabbit hole. This should fix you up, for real this time ;). Thanks, Yes - this works now. Which of the patches can i drop? Do i just need the last one? Is it safe to add another 18TB raid via converting it to btrfs
Re: No space left on device (28)
On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote:
Hi,
but when i transfer big files i see now this one:
[20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds.
[20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs
disables this message.
[20368.895140] rsync D 8160f580 0 14911 1
0x
[20368.895148] 8801ca63fc78 0086 8800c28f8198
88022394f800
[20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8
00012c40
[20368.895163] 81a11440 8801c9d36340 8801ca63fc88
8801cefce130
[20368.895169] Call Trace:
[20368.895180] [8151a774] schedule+0x24/0x70
[20368.895207] [a0158c75]
wait_current_trans.isra.32+0x95/0x100 [btrfs]
[20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60
[20368.895236] [a015a45d]
start_transaction.part.33+0x13d/0x4d0 [btrfs]
[20368.895252] [811420f3] ? inode_permission+0x13/0x50
[20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs]
[20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20
[btrfs]
[20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs]
[20368.895307] [8150ee3e] ? check_acl+0x5a/0x122
[20368.895312] [81055ff0] ? ns_capable+0x30/0x60
[20368.895317] [811413bd] ? generic_permission+0xbd/0x110
[20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs]
[20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0
[20368.895345] [81143ad7] vfs_unlink+0x37/0x50
[20368.895349] [81143c8b] do_unlinkat+0x19b/0x240
[20368.895354] [81146171] sys_unlink+0x11/0x20
[20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b
Speed is just 100kb/s instead of 100MB/s.
Hrm I wonder if 512 is too small for your case, can you add this patch to the
pile and see what dmesg says when you are having these problems? Thanks,
Josef
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 50767bb..d19c9f6 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -31,6 +31,7 @@
#include inode-map.h
#include volumes.h
#include dev-replace.h
+#include math.h
#define BTRFS_ROOT_TRANS_TAG 0
@@ -576,10 +577,19 @@ void btrfs_throttle(struct btrfs_root *root)
static int should_end_transaction(struct btrfs_trans_handle *trans,
struct btrfs_root *root)
{
- int ret;
+ struct btrfs_block_rsv *block_rsv = root-fs_info-global_block_rsv;
+ u64 num_bytes = 0;
+ int ret = 1;
- ret = btrfs_block_rsv_check(root, root-fs_info-global_block_rsv, 5);
- return ret ? 1 : 0;
+ spin_lock(block_rsv-lock);
+ num_bytes = div_factor(block_rsv-size, 5);
+ if (block_rsv-reserved = num_bytes)
+ ret = 0;
+ else
+ printk(KERN_ERR we're pretty low, setting blocked, reserved
%Lu, size %Lu, num %Lu\n,
+ block_rsv-reserved, block_rsv-size, num_bytes);
+ spin_unlock(block_rsv-lock);
+ return ret;
}
int btrfs_should_end_transaction(struct btrfs_trans_handle *trans,
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
HI, Am 26.03.2013 20:38, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote: Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [20368.895140] rsync D 8160f580 0 14911 1 0x [20368.895148] 8801ca63fc78 0086 8800c28f8198 88022394f800 [20368.895158] 8801ca63ffd8 8801ca63ffd8 8801ca63ffd8 00012c40 [20368.895163] 81a11440 8801c9d36340 8801ca63fc88 8801cefce130 [20368.895169] Call Trace: [20368.895180] [8151a774] schedule+0x24/0x70 [20368.895207] [a0158c75] wait_current_trans.isra.32+0x95/0x100 [btrfs] [20368.895214] [8106d4f0] ? add_wait_queue+0x60/0x60 [20368.895236] [a015a45d] start_transaction.part.33+0x13d/0x4d0 [btrfs] [20368.895252] [811420f3] ? inode_permission+0x13/0x50 [20368.895271] [a015a814] start_transaction+0x24/0x30 [btrfs] [20368.895287] [a015aae3] btrfs_start_transaction+0x13/0x20 [btrfs] [20368.895302] [a015b2f0] __unlink_start_trans+0x70/0x460 [btrfs] [20368.895307] [8150ee3e] ? check_acl+0x5a/0x122 [20368.895312] [81055ff0] ? ns_capable+0x30/0x60 [20368.895317] [811413bd] ? generic_permission+0xbd/0x110 [20368.895336] [a0163f92] btrfs_unlink+0x32/0xc0 [btrfs] [20368.895341] [8114186d] vfs_unlink.part.61+0x6d/0xd0 [20368.895345] [81143ad7] vfs_unlink+0x37/0x50 [20368.895349] [81143c8b] do_unlinkat+0x19b/0x240 [20368.895354] [81146171] sys_unlink+0x11/0x20 [20368.895359] [8151c2e9] system_call_fastpath+0x16/0x1b Speed is just 100kb/s instead of 100MB/s. Hrm I wonder if 512 is too small for your case, can you add this patch to the pile and see what dmesg says when you are having these problems? Thanks, No idea why but i can't reproduce... ;-( Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 02:55:07PM -0600, Stefan Priebe wrote:
Hi Jsoef,
thanks!
Ok I don't think the thing I just fixed will make any difference for you so
here's another debug patch, just apply it on top of what I've already sent you
and re-run and give me the dmesg again. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index aabaea6..bf6433f 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4162,7 +4162,11 @@ out:
}
if (flushing) {
if (ret == -ENOSPC) {
- printk(KERN_ERR returning enospc, dumping space
info\n);
+ spin_lock(block_rsv-lock);
+ printk(KERN_ERR returning enospc, space_info %u, size
%Lu reserved %Lu,
+ flush %d, flush_state %d dumping space
info\n, block_rsv-type,
+ block_rsv-size, block_rsv-reserved, flush,
flush_state);
+ spin_unlock(block_rsv-lock);
dump_space_info(space_info, 0, 0);
}
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Should i try to downgrade? Greets, Stefan Am 21.03.2013 um 20:42 schrieb Stefan Priebe s.pri...@profihost.ag: Yes, i've now upgraded to 3.9-rc3 same result. rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/.c2_ae.h.WEhLGP - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: No space left on device (28) # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.98GB Metadata: total=8.00MB, used=0.00 Stefan Am 21.03.2013 20:02, schrieb Chris Mason: Quoting Stefan Priebe - Profihost AG (2013-03-21 14:35:59) Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Thu, 21 Mar 2013 20:42:28 +0100 Stefan Priebe s.pri...@profihost.ag wrote: I might be wrong here, but doesn't this rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/.c2_ae.h.WEhLGP - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: ...try to move a file from /mnt/.software/ to .software/ (relative to current dir)?? Maybe you end up trying to rsync your array to a small root partition or something? No space left on device (28) # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.98GB Metadata: total=8.00MB, used=0.00 Stefan Am 21.03.2013 20:02, schrieb Chris Mason: Quoting Stefan Priebe - Profihost AG (2013-03-21 14:35:59) Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- With respect, Roman signature.asc Description: PGP signature
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 12:13 AM, Roman Mamedov r...@romanrm.ru wrote: On Thu, 21 Mar 2013 20:42:28 +0100 Stefan Priebe s.pri...@profihost.ag wrote: I might be wrong here, but doesn't this rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/ - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: ...try to move a file from /mnt/.software/ to .software/ (relative to current dir)?? No; that's rsync giving the full path, and then the target path relative to the command it was given. The filename itself (.c2_ae.h.WEhLGP) is a semi-random filename rsync uses to write to temporarily, so it can mv it over the original in an atomic fashion... Stefan: ...which means that the actual copy succeeded, which suggests that this is more of a metadata enospc thing. You might try btrfs balance start -musage=5 (instead of -dusage), and if that doesn't report any chunks balanced, try a high number until it does. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
No this is fine it's rsync style same happens with cp. Am 22.03.2013 um 07:13 schrieb Roman Mamedov r...@romanrm.ru: On Thu, 21 Mar 2013 20:42:28 +0100 Stefan Priebe s.pri...@profihost.ag wrote: I might be wrong here, but doesn't this rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/.c2_ae.h.WEhLGP - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: ...try to move a file from /mnt/.software/ to .software/ (relative to current dir)?? Maybe you end up trying to rsync your array to a small root partition or something? No space left on device (28) # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.98GB Metadata: total=8.00MB, used=0.00 Stefan Am 21.03.2013 20:02, schrieb Chris Mason: Quoting Stefan Priebe - Profihost AG (2013-03-21 14:35:59) Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- With respect, Roman -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Already tried with value 5 did not help ;-( and it also happens with plain cp copying a 15gb file and aborts at about 80% Am 22.03.2013 um 07:24 schrieb cwillu cwi...@cwillu.com: On Fri, Mar 22, 2013 at 12:13 AM, Roman Mamedov r...@romanrm.ru wrote: On Thu, 21 Mar 2013 20:42:28 +0100 Stefan Priebe s.pri...@profihost.ag wrote: I might be wrong here, but doesn't this rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/ - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: ...try to move a file from /mnt/.software/ to .software/ (relative to current dir)?? No; that's rsync giving the full path, and then the target path relative to the command it was given. The filename itself (.c2_ae.h.WEhLGP) is a semi-random filename rsync uses to write to temporarily, so it can mv it over the original in an atomic fashion... Stefan: ...which means that the actual copy succeeded, which suggests that this is more of a metadata enospc thing. You might try btrfs balance start -musage=5 (instead of -dusage), and if that doesn't report any chunks balanced, try a high number until it does. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 12:39 AM, Stefan Priebe - Profihost AG s.pri...@profihost.ag wrote: Already tried with value 5 did not help ;-( and it also happens with plain cp copying a 15gb file and aborts at about 80% You tried -musage=5? Your original email said -dusage=5. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi, Am 22.03.2013 07:41, schrieb cwillu: On Fri, Mar 22, 2013 at 12:39 AM, Stefan Priebe - Profihost AG s.pri...@profihost.ag wrote: Already tried with value 5 did not help ;-( and it also happens with plain cp copying a 15gb file and aborts at about 80% You tried -musage=5? Your original email said -dusage=5. sorry i missed the difference - but it does not work: ~# btrfs fi balance start -musage=5 /mnt/ ERROR: error during balancing '/mnt/' - No space left on device There may be more info in syslog - try dmesg | tail ~# dmesg|tail [ 1183.019367] device fsid fc23c4a8-a351-4c91-96a2-ee6abeffe59a devid 1 transid 6224 /dev/mapper/raid54tb1 [ 1183.019860] btrfs: disk space caching is enabled [42719.915781] btrfs: relocating block group 4194304 flags 4 [42727.916141] btrfs: 5 enospc errors during balance Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG wrote: Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Sorry Stefan I'm almost done with what I'm working on and then I'll work up a patch for you to run so I can narrow down what's going on. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 22.03.2013 14:53, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG wrote: Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Sorry Stefan I'm almost done with what I'm working on and then I'll work up a patch for you to run so I can narrow down what's going on. Thanks, Great! Thanks - just wanted to know that it's not my fault. I'm happy to test the patch and provide feedback. Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 07:56:41AM -0600, Stefan Priebe - Profihost AG wrote:
Hi Josef,
Am 22.03.2013 14:53, schrieb Josef Bacik:
On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG
wrote:
Hi Chris,
Which kernel are you running?
-chris
vanilla 3.8.3.
Ok, with the 3.9 merge window Josef changed how we do the reservations.
Are you able to try a slightly more experimental kernel?
any ideas what i can check? 3.9-rc3 gives me same results.
Sorry Stefan I'm almost done with what I'm working on and then I'll work up
a
patch for you to run so I can narrow down what's going on. Thanks,
Great!
Thanks - just wanted to know that it's not my fault. I'm happy to test
the patch and provide feedback.
Ok I think we are way over-reserving for rename, can you give this patch a whirl
and see what happens? If it still fails can you capture dmesg and reply with
that so I can see what's going on. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 9ac2eca..aabaea6 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4161,6 +4161,11 @@ out:
ret = 0;
}
if (flushing) {
+ if (ret == -ENOSPC) {
+ printk(KERN_ERR returning enospc, dumping space
info\n);
+ dump_space_info(space_info, 0, 0);
+ }
+
spin_lock(space_info-lock);
space_info-flush = 0;
wake_up_all(space_info-wait);
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index ca1b767..3980ae7 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -3679,11 +3679,9 @@ static struct btrfs_trans_handle
*__unlink_start_trans(struct inode *dir,
* 1 for the dir item
* 1 for the dir index
* 1 for the inode ref
-* 1 for the inode ref in the tree log
-* 2 for the dir entries in the log
* 1 for the inode
*/
- trans = btrfs_start_transaction(root, 8);
+ trans = btrfs_start_transaction(root, 5);
if (!IS_ERR(trans) || PTR_ERR(trans) != -ENOSPC)
return trans;
@@ -8127,7 +8125,7 @@ static int btrfs_rename(struct inode *old_dir, struct
dentry *old_dentry,
* inodes. So 5 * 2 is 10, plus 1 for the new link, so 11 total items
* should cover the worst case number of items we'll modify.
*/
- trans = btrfs_start_transaction(root, 20);
+ trans = btrfs_start_transaction(root, 11);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out_notrans;
--
1.7.7.6
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Josef, Am 22.03.2013 16:54, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 07:56:41AM -0600, Stefan Priebe - Profihost AG wrote: Hi Josef, Am 22.03.2013 14:53, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG wrote: Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Sorry Stefan I'm almost done with what I'm working on and then I'll work up a patch for you to run so I can narrow down what's going on. Thanks, Great! Thanks - just wanted to know that it's not my fault. I'm happy to test the patch and provide feedback. Ok I think we are way over-reserving for rename, can you give this patch a whirl and see what happens? If it still fails can you capture dmesg and reply with that so I can see what's going on. Thanks, Thanks for the patch. I was able to copy some files and i'm still able put it copy some then fails on some then copies some then fails on some. Rsync output: .software/kernel/linux-3.9-rc3/drivers/rtc/rtc-wm8350.c 12156 100% 11.17kB/s0:00:01 (xfer#21395, to-check=1008/52625) .software/kernel/linux-3.9-rc3/drivers/rtc/rtc-x1205.c 16460 100% 15.05kB/s0:00:01 (xfer#21396, to-check=1007/52625) .software/kernel/linux-3.9-rc3/drivers/rtc/systohc.c 1223 100%1.12kB/s0:00:01 (xfer#21397, to-check=1006/52625) .software/kernel/linux-3.9-rc3/drivers/s390/ .software/kernel/linux-3.9-rc3/drivers/s390/Makefile 144 100%0.13kB/s0:00:01 (xfer#21398, to-check=1052/52672) .software/kernel/linux-3.9-rc3/drivers/s390/block/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/block failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/s390/char/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/char failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/s390/cio/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/cio failed: No space left on device (28) rsync: mkstemp /mnt/.software/kernel/linux-3.9-rc3/drivers/regulator/.88pm8607.c.YBi0Rz failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/s390/crypto/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/crypto failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/s390/kvm/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/kvm failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/s390/net/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/net failed: No space left on device (28) rsync: mkstemp /mnt/.software/kernel/linux-3.9-rc3/drivers/regulator/.Kconfig.0lsfuE failed: No space left on device (28) *** Skipping any contents from this failed directory *** rsync: mkstemp /mnt/.software/kernel/linux-3.9-rc3/drivers/regulator/.Makefile.C9wI7I failed: No space left on device (28) .software/kernel/linux-3.9-rc3/drivers/s390/scsi/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/s390/scsi failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/sbus/ .software/kernel/linux-3.9-rc3/drivers/sbus/Makefile 70 100%0.06kB/s0:00:01 (xfer#21399, to-check=1003/52824) .software/kernel/linux-3.9-rc3/drivers/sbus/char/ rsync: recv_generator: mkdir /mnt/.software/kernel/linux-3.9-rc3/drivers/sbus/char failed: No space left on device (28) rsync: mkstemp /mnt/.software/kernel/linux-3.9-rc3/drivers/regulator/.aat2870-regulator.c.BGDwPN failed: No space left on device (28) *** Skipping any contents from this failed directory *** .software/kernel/linux-3.9-rc3/drivers/scsi/ .software/kernel/linux-3.9-rc3/drivers/scsi/.3w-9xxx.ko.cmd 208 100%0.16kB/s0:00:01 (xfer#21400, to-check=1407/53242) .software/kernel/linux-3.9-rc3/drivers/scsi/.3w-9xxx.mod.o.cmd 27987 100% 21.54kB/s0:00:01 (xfer#21401, to-check=1406/53242) .software/kernel/linux-3.9-rc3/drivers/scsi/.3w-9xxx.o.cmd 43340 100% 32.63kB/s0:00:01 (xfer#21402, to-check=1405/53242) .software/kernel/linux-3.9-rc3/drivers/scsi/.3w-sas.ko.cmd 204 100% 15.32kB/s0:00:00 (xfer#21403, to-check=1404/53242) .software/kernel/linux-3.9-rc3/drivers/scsi/.3w-sas.mod.o.cmd 27975 100%1.91MB/s0:00:00 (xfer#21404, to-check=1403/53242)
Re: No space left on device (28)
On Fri, Mar 22, 2013 at 01:10:05PM -0600, Stefan Priebe wrote: Hi Josef, Am 22.03.2013 16:54, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 07:56:41AM -0600, Stefan Priebe - Profihost AG wrote: Hi Josef, Am 22.03.2013 14:53, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG wrote: Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Sorry Stefan I'm almost done with what I'm working on and then I'll work up a patch for you to run so I can narrow down what's going on. Thanks, Great! Thanks - just wanted to know that it's not my fault. I'm happy to test the patch and provide feedback. Ok I think we are way over-reserving for rename, can you give this patch a whirl and see what happens? If it still fails can you capture dmesg and reply with that so I can see what's going on. Thanks, Thanks for the patch. I was able to copy some files and i'm still able put it copy some then fails on some then copies some then fails on some. Ok so I've been working on another ENOSPC related problem that I think is affecting you as well. So I'm going to nail that down and see if that helps you too, if not we'll poke around your problem some more and try and work out what's happening. I'll get back to this fresh Monday morning. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Jsoef, thanks! Am 22.03.2013 21:49, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 01:10:05PM -0600, Stefan Priebe wrote: Hi Josef, Am 22.03.2013 16:54, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 07:56:41AM -0600, Stefan Priebe - Profihost AG wrote: Hi Josef, Am 22.03.2013 14:53, schrieb Josef Bacik: On Fri, Mar 22, 2013 at 06:11:56AM -0600, Stefan Priebe - Profihost AG wrote: Hi Chris, Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? any ideas what i can check? 3.9-rc3 gives me same results. Sorry Stefan I'm almost done with what I'm working on and then I'll work up a patch for you to run so I can narrow down what's going on. Thanks, Great! Thanks - just wanted to know that it's not my fault. I'm happy to test the patch and provide feedback. Ok I think we are way over-reserving for rename, can you give this patch a whirl and see what happens? If it still fails can you capture dmesg and reply with that so I can see what's going on. Thanks, Thanks for the patch. I was able to copy some files and i'm still able put it copy some then fails on some then copies some then fails on some. Ok so I've been working on another ENOSPC related problem that I think is affecting you as well. So I'm going to nail that down and see if that helps you too, if not we'll poke around your problem some more and try and work out what's happening. I'll get back to this fresh Monday morning. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Stefan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Quoting Stefan Priebe - Profihost AG (2013-03-21 14:35:59) Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: No space left on device (28)
Yes, i've now upgraded to 3.9-rc3 same result. rsync: rename /mnt/.software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/.c2_ae.h.WEhLGP - .software/kernel/linux-3.9-rc3/drivers/infiniband/hw/amso1100/c2_ae.h: No space left on device (28) # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.98GB Metadata: total=8.00MB, used=0.00 Stefan Am 21.03.2013 20:02, schrieb Chris Mason: Quoting Stefan Priebe - Profihost AG (2013-03-21 14:35:59) Hi Chris, Am 21.03.2013 um 19:00 schrieb Chris Mason chris.ma...@fusionio.com: Quoting Stefan Priebe - Profihost AG (2013-03-21 04:03:04) Hi, while copying a 15GB file to my btrfs volume i'm getting No space left on device. Some information: # btrfs filesystem df /mnt/ Data: total=18.14TB, used=15.05TB System, DUP: total=8.00MB, used=1.94MB System: total=4.00MB, used=0.00 Metadata, DUP: total=23.98GB, used=17.97GB Metadata: total=8.00MB, used=0.00 # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/raid54tb1 19T 16T 3,1T 84% /mnt # btrfs fi show Label: none uuid: fc23c4a8-a351-4c91-96a2-ee6abeffe59a Total devices 1 FS bytes used 15.07TB devid1 size 18.19TB used 18.19TB path /dev/dm-3 # btrfs fi balance start -dusage=5 /mnt/ Done, had to relocate 0 out of 18604 chunks What wrong here? What is about the last 3TB? Which kernel are you running? -chris vanilla 3.8.3. Ok, with the 3.9 merge window Josef changed how we do the reservations. Are you able to try a slightly more experimental kernel? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
