Re: btrfs filesystem keeps allocating new chunks for no apparent reason

2016-06-09 Thread Hans van Kranenburg
On 06/09/2016 10:52 AM, Marc Haber wrote: On Thu, Jun 09, 2016 at 01:10:46AM +0200, Hans van Kranenburg wrote: So, instead of being the cause, apt-get update causing a new chunk to be allocated might as well be the result of existing ones already filled up with too many fragments. The next

Re: btrfs filesystem keeps allocating new chunks for no apparent reason

2016-06-08 Thread Hans van Kranenburg
Hi list, On 05/31/2016 03:36 AM, Qu Wenruo wrote: Hans van Kranenburg wrote on 2016/05/06 23:28 +0200: Hi, I've got a mostly inactive btrfs filesystem inside a virtual machine somewhere that shows interesting behaviour: while no interesting disk activity is going on, btrfs keeps allocating

Re: [PATCH] Btrfs-progs: add check-only option for balance

2016-06-12 Thread Hans van Kranenburg
Hi! On 06/12/2016 08:41 PM, Goffredo Baroncelli wrote: Hi All, On 2016-06-10 22:47, Hans van Kranenburg wrote: +if (sk->min_objectid < sk->max_objectid) + sk->min_objectid += 1; ...and now it's (289406977 168 19193856), which means you're continuing your search *afte

Re: [PATCH] Btrfs-progs: add check-only option for balance

2016-06-10 Thread Hans van Kranenburg
first look at the chunk tree, and for every chunk listed in there look up 1 exact match in the extent tree, and then read the used value of it. Ironically, this is exact the thing that I was looking for when that IRC conversation referenced above happened, resulting in a very similar thing, playing a

Re: Allocator behaviour during device delete

2016-06-10 Thread Hans van Kranenburg
previous empty one is removed, bumping up the start vaddr of the new chunk with 1GB each time. -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrf

Re: Cannot balance FS (No space left on device)

2016-06-10 Thread Hans van Kranenburg
u can always stop it when you think it moved around enough data with btrfs balance cancel. Moo, -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs&q

Re: Allocator behaviour during device delete

2016-06-10 Thread Hans van Kranenburg
On 06/10/2016 09:58 PM, Hans van Kranenburg wrote: On 06/10/2016 09:26 PM, Henk Slager wrote: On Thu, Jun 9, 2016 at 3:54 PM, Brendan Hide <bren...@swiftspirit.co.za> wrote: On 06/09/2016 03:07 PM, Austin S. Hemmelgarn wrote: OK, I'm pretty sure I know what was going on in this case.

Re: Cannot balance FS (No space left on device)

2016-06-10 Thread Hans van Kranenburg
On 06/11/2016 12:10 AM, ojab // wrote: On Fri, Jun 10, 2016 at 9:56 PM, Hans van Kranenburg <hans.van.kranenb...@mendix.com> wrote: You can work around it by either adding two disks (like Henk said), or by temporarily converting some chunks to single. Just enough to get some free

Re: Recommended why to use btrfs for production?

2016-06-03 Thread Hans van Kranenburg
like these: http://www.spinics.net/lists/linux-btrfs/msg55642.html o/ Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: btrfs filesystem keeps allocating new chunks for no apparent reason

2016-06-11 Thread Hans van Kranenburg
On 06/10/2016 07:07 PM, Henk Slager wrote: On Thu, Jun 9, 2016 at 5:41 PM, Duncan <1i5t5.dun...@cox.net> wrote: Hans van Kranenburg posted on Thu, 09 Jun 2016 01:10:46 +0200 as excerpted: The next question is what files these extents belong to. To find out, I need to open up the extent

Re: btrfs filesystem keeps allocating new chunks for no apparent reason

2016-05-30 Thread Hans van Kranenburg
balance every week to get them away again. Hans On 05/06/2016 11:28 PM, Hans van Kranenburg wrote: Hi, I've got a mostly inactive btrfs filesystem inside a virtual machine somewhere that shows interesting behaviour: while no interesting disk activity is going on, btrfs keeps allocating new chunks

Re: btrfs filesystem keeps allocating new chunks for no apparent reason

2016-05-30 Thread Hans van Kranenburg
On 05/30/2016 09:55 PM, Duncan wrote: Hans van Kranenburg posted on Mon, 30 May 2016 13:07:26 +0200 as excerpted: [Please don't post "upside down". Reply in context under the quoted point, here the whole post, you're replying to. It makes further replies in context far easier.

Hello world, python-btrfs

2016-06-17 Thread Hans van Kranenburg
The ioctl search returns a generator, the search key is a nice object you can just increment etc... * This is a work in progress. I can only add support for parts of btrfs that I understand myself first. Moo! Hans van Kranenburg -- Hans van Kranenburg - System / Network Engineer T +31 (0)

Re: btrfs: page allocation failure: order:1, mode:0x2204020

2016-06-22 Thread Hans van Kranenburg
On 06/20/2016 02:49 PM, David Sterba wrote: On Sat, Jun 18, 2016 at 08:47:55PM +0200, Hans van Kranenburg wrote: Last night, one of my btrfs filesystems went read-only after a memory allocation failure (logging attached). According to the logs, the allocation itself happens out of btrfs so we

Re: [RFC][PATCH] btrfs-progs: inspect: new subcommand to dump chunks

2016-06-22 Thread Hans van Kranenburg
>max_transid = (u64)-1; + + sk->nr_items = 4096; Or set to 1 because we already know there can only be one result. -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the

Re: [RFC][PATCH] btrfs-progs: inspect: new subcommand to dump chunks

2016-06-22 Thread Hans van Kranenburg
commit/be85c2a0e2774909b532326e99e0c23ab8467263 Sorry, couldn't resist /:D\ -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a m

Re: [RFC][PATCH] btrfs-progs: inspect: new subcommand to dump chunks

2016-06-23 Thread Hans van Kranenburg
On 06/23/2016 03:13 PM, David Sterba wrote: On Thu, Jun 23, 2016 at 12:20:38AM +0200, Hans van Kranenburg wrote: Printing 'usage' is not default as it's quite slow, it uses the search ioctl and probably not in the best way, or there's some other issue in the implementation. Interesting. So

Re: Confusing output from fi us/df

2016-06-20 Thread Hans van Kranenburg
Hi, On 06/21/2016 01:30 AM, Marc Grondin wrote: I have a btrfs filesystem ontop of a 4x1tb mdraid raid5 array and I've been getting confusing output on metadata usage. Seems that even tho both data and metadata are in single profile metadata is reporting double the space(as if it was in dupe

btrfs: page allocation failure: order:1, mode:0x2204020

2016-06-18 Thread Hans van Kranenburg
64: errno=-12 Out of memory -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More maj

Re: Hello world, python-btrfs

2016-06-23 Thread Hans van Kranenburg
Hi, On 06/23/2016 04:26 PM, David Sterba wrote: On Sat, Jun 18, 2016 at 12:01:53AM +0200, Hans van Kranenburg wrote: proof-of-concept level scripts to be able to debug my btrfs file systems, the inevitable happened: https://github.com/knorrie/python-btrfs/ Currently, the primary goal

Re: Btrfs full balance command fails due to ENOSPC (bug 121071)

2016-06-27 Thread Hans van Kranenburg
urself, it gets easier every day. :D Have fun, -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger

btrfs filesystem keeps allocating new chunks for no apparent reason

2016-05-06 Thread Hans van Kranenburg
, and still, btrfs keeps allocating new data chunks like a hungry beast. Why would this happen? Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: Balance and subvolume delete causes deadlock and metadata corruption

2016-07-30 Thread Hans van Kranenburg
On 07/31/2016 01:18 AM, Hans van Kranenburg wrote: > blahblahblahblahblalahba > > Doing a btrfs check now on the block device, will fup with the output. > Output so far: -# btrfs check /dev/xvdc 2>&1 | tee btrfs-check-xvdc checking extents ref mismatch on [3649516437504 1

Balance and subvolume delete causes deadlock and metadata corruption

2016-07-30 Thread Hans van Kranenburg
d a subvolume delete at the same time, which is something that btrfs definitely needs to be able to handle. Doing a btrfs check now on the block device, will fup with the output. -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com --

Re: Balance and subvolume delete causes deadlock and metadata corruption

2016-07-30 Thread Hans van Kranenburg
On 07/31/2016 02:13 AM, Hans van Kranenburg wrote: > On 07/31/2016 01:18 AM, Hans van Kranenburg wrote: >> blahblahblahblahblalahba >> >> Doing a btrfs check now on the block device, will fup with the output. >> > > Output so far: > > -# btrfs chec

Re: Cannot balance FS (No space left on device)

2016-07-02 Thread Hans van Kranenburg
On 06/13/2016 02:33 PM, Austin S. Hemmelgarn wrote: On 2016-06-10 18:39, Hans van Kranenburg wrote: On 06/11/2016 12:10 AM, ojab // wrote: On Fri, Jun 10, 2016 at 9:56 PM, Hans van Kranenburg <hans.van.kranenb...@mendix.com> wrote: You can work around it by either adding two disks (lik

Filesystem locks up, also with older kernel on any action after booting into 4.7-rc4 once

2016-07-02 Thread Hans van Kranenburg
I try to do anything with it... -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More

Re: Filesystem locks up, also with older kernel on any action after booting into 4.7-rc4 once

2016-07-02 Thread Hans van Kranenburg
On 07/02/2016 09:18 PM, Chris Murphy wrote: On Sat, Jul 2, 2016 at 11:34 AM, Hans van Kranenburg <hans.van.kranenb...@mendix.com> wrote: On 07/02/2016 07:14 PM, Hans van Kranenburg wrote: I just rebooted a VM into a 4.7 kernel. The joy didn't last long. After 177 seconds the btrf

Re: Filesystem locks up, also with older kernel on any action after booting into 4.7-rc4 once

2016-07-02 Thread Hans van Kranenburg
On 07/02/2016 07:14 PM, Hans van Kranenburg wrote: I just rebooted a VM into a 4.7 kernel. The joy didn't last long. After 177 seconds the btrfs data partition (root is on ext4) locked up. Worse, it keeps locking up on any action performed even when rebooting it with older kernels again. D

Re: Filesystem locks up, also with older kernel on any action after booting into 4.7-rc4 once

2016-07-02 Thread Hans van Kranenburg
On 07/02/2016 09:40 PM, Hans van Kranenburg wrote: On 07/02/2016 09:18 PM, Chris Murphy wrote: On Sat, Jul 2, 2016 at 11:34 AM, Hans van Kranenburg <hans.van.kranenb...@mendix.com> wrote: On 07/02/2016 07:14 PM, Hans van Kranenburg wrote: I just rebooted a VM into a 4.7 kernel. T

[PATCH] Btrfs: use the correct struct for BTRFS_IOC_LOGICAL_INO

2016-07-03 Thread Hans van Kranenburg
not change. But, it makes the code less confusing for the reader. Signed-off-by: Hans van Kranenburg <hans.van.kranenb...@mendix.com> --- include/uapi/linux/btrfs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index 2

[PATCH] btrfs-progs: use the correct struct for BTRFS_IOC_LOGICAL_INO

2016-07-03 Thread Hans van Kranenburg
not change. But, it makes the code less confusing for the reader. Signed-off-by: Hans van Kranenburg <hans.van.kranenb...@mendix.com> --- ioctl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ioctl.h b/ioctl.h index 5f18bcb..620dd3d 100644 --- a/ioctl.h +++ b/ioctl.h @@

Re: [RFC] python-btrfs: getting arch dependent ioctl handling right

2017-02-04 Thread Hans van Kranenburg
On 02/05/2017 12:56 AM, Hugo Mills wrote: > On Sun, Feb 05, 2017 at 12:37:18AM +0100, Hans van Kranenburg wrote: >> Hi, >> >> since I'd like to have my python btrfs library work correctly for users >> with other systems than amd64, I have been looking at IOCTL number &

[RFC] python-btrfs: getting arch dependent ioctl handling right

2017-02-04 Thread Hans van Kranenburg
32 bit userland with a 64 bit kernel. - What is the exact technical reason for this? - What's the general opinion about this usecase? (needs to work, or, "doctor, if I push here it hurts; well, don't push there"...) Thanks, -- Hans van Kranenburg -- To unsubscribe from this list: sen

Re: BTRFS for OLTP Databases

2017-02-07 Thread Hans van Kranenburg
esystems (like a backup server with concurrent rsync data streaming in, also doing snapshotting) from having incoming traffic drop to the ground every time there was a transaction commit. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body o

Re: dup vs raid1 in single disk

2017-02-07 Thread Hans van Kranenburg
SSD firmware deduplicating writes, converting the DUP into single again, giving a false idea of it being DUP. This is one that can be solved by e.g. using disk encryption, which causes same writes to show up as different data on disk. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: BTRFS for OLTP Databases

2017-02-07 Thread Hans van Kranenburg
apshot is like a power loss - even tho there is no power > loss. So the database has to be properly configured. It is simply short > sighted if you don't think about this fact. The documentation should > really point that fact out. I'd almost say that it would be short sighted to assume a btrfs sna

Bug or feature? logical_to_ino skips inodes with offset mismatch

2017-02-05 Thread Hans van Kranenburg
extent compression(none) -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Is it possible to have metadata-only device with no data?

2017-02-05 Thread Hans van Kranenburg
On 02/05/2017 10:42 PM, Alexander Tomokhov wrote: > Is it possible, having two drives to do raid1 for metadata but keep data on a > single drive only? Nope. Would be a really nice feature though... Putting metadata on SSD and bulk data on HDD... -- Hans van Kranenburg -- To unsubscrib

Re: Bug or feature? logical_to_ino skips inodes with offset mismatch

2017-02-05 Thread Hans van Kranenburg
On 02/05/2017 10:54 PM, Hans van Kranenburg wrote: > I was playing around with logical_to_ino and also implemented calling > ino_lookup today to create a program that will print all data extents in > a block group, and their related inodes and at least one filename per inode. &

Re: Btrfs Heatmap - v4 ... colors!!

2017-02-03 Thread Hans van Kranenburg
KGBUILD?h=python-btrfs ? It's a package, not a single file, so maybe that compile command doesn't do anything? I'm not familiar with arch, so correct me if I'm wrong. . ├── btrfs │ ├── crc32c.py │ ├── ctree.py │ ├── __init__.py │ ├── ioctl.py │ ├── utils.py -- Hans van Kranenburg -- To un

Re: Btrfs Heatmap - v4 ... colors!!

2017-02-03 Thread Hans van Kranenburg
gt; PIP_CONFIG_FILE=/dev/null pip install --isolated --root="$pkgdir" > --ignore-installed --no-deps btrfs > Even better :-) Thanks, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Btrfs Heatmap - v4 ... colors!!

2017-02-03 Thread Hans van Kranenburg
On 02/03/2017 03:18 PM, Timofey Titovets wrote: > 2017-02-03 15:57 GMT+03:00 Hans van Kranenburg > <hans.van.kranenb...@mendix.com>: >> On 02/03/2017 12:25 PM, Timofey Titovets wrote: >>> Thank you for your great work: >>> JFYI Packaged in AUR: >>> ht

Re: "not a btrfs filesystem"

2017-01-21 Thread Hans van Kranenburg
mh/banana.sdcard 0 > [26/3442]mh@fan:~ $ > > Can a btrfs be so broken that btrfs balance doesn't recognize it any > more? What is going on with this file system? -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Hard crash on 4.9.5

2017-01-23 Thread Hans van Kranenburg
ve info). I can also find it in Oct 2016 in my IRC logs, but without any info on kernel version. Anyway, it seems to point to something that's going wrong with changes that are *not* on disk *yet*, and the crash is preventing . -- Hans van Kranenburg -- To unsubscribe from this list: send the

Re: Hard crash on 4.9.5

2017-01-23 Thread Hans van Kranenburg
On 01/23/2017 09:27 PM, Hans van Kranenburg wrote: > [... press send without rereading ...] > > Anyway, it seems to point to something that's going wrong with changes > that are *not* on disk *yet*, and the crash is preventing ... ... whatever incorrect data this situation might re

Re: Fresh Raid-1 setup, dump-tree shows invalid owner id

2017-01-30 Thread Hans van Kranenburg
but then unsigned 64 bits. >>> -9 + 2**64 18446744073709551607L So the result is a number that's close to the max or 64 bits. You can find those numbers in the kernel source in include/uapi/linux/btrfs_tree.h e.g.: #define BTRFS_DATA_RELOC_TREE_OBJECTID -9ULL -- Hans van Krane

Re: btrfs recovery

2017-01-30 Thread Hans van Kranenburg
parts that do not connect with each other any more. And that's exactly what you see in all the errors. E.g. "parent transid verify failed on 32869482496 wanted 550112 found 550121" <- a part of a tree points to another part, but suddenly something else is found which should not be

Re: btrfs recovery

2017-01-30 Thread Hans van Kranenburg
ritten somewhere at the end of the disk which also relates to something that gets written at the beginning of the disk, while your dd copy is doing its thing somewhere in between... -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a

Re: btrfs recovery

2017-01-28 Thread Hans van Kranenburg
(or pipe it through base64 and pastebin it), so we can help a bit more efficiently. After getting the bytelevel stuff right again, the block needs a new checksum, and then you have to carefully dd it back in both of the places which are listed in the stripe lines. If everything goes right... bam

Re: btrfs recovery

2017-01-29 Thread Hans van Kranenburg
On 01/29/2017 08:52 PM, Oliver Freyermuth wrote: > Am 29.01.2017 um 20:28 schrieb Hans van Kranenburg: >> On 01/29/2017 08:09 PM, Oliver Freyermuth wrote: >>>> [..whaaa.. text.. see previous message..] >>> Wow - this nice python toolset really makes it easy, bi

Re: btrfs recovery

2017-01-29 Thread Hans van Kranenburg
lock=35028992, root=1, slot=243 > So does this tell me that also item 242 was corrupted? No, I was just going too fast. A nice extra excercise is to look up the block at 596459520, which this item points to, and then see which object is the first one in the part of the tree stored in that page. It s

Re: btrfs recovery

2017-01-29 Thread Hans van Kranenburg
On 01/29/2017 03:02 AM, Oliver Freyermuth wrote: > Am 28.01.2017 um 23:27 schrieb Hans van Kranenburg: >> On 01/28/2017 10:04 PM, Oliver Freyermuth wrote: >>> Am 26.01.2017 um 12:01 schrieb Oliver Freyermuth: >>>> Am 26.01.2017 um 11:00 schrieb Hugo Mills: >>>

Btrfs Heatmap - v5 ... snake, linear ... the virtual address space

2017-02-18 Thread Hans van Kranenburg
or misbehaving. :) -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Btrfs Heatmap - v5 ... snake, linear ... the virtual address space

2017-02-19 Thread Hans van Kranenburg
On 02/19/2017 11:25 AM, Adam Borowski wrote: > On Sun, Feb 19, 2017 at 01:01:29AM +0100, Hans van Kranenburg wrote: >> I just tagged v5 of the Btrfs Heatmap utility, which visualizes the >> usage of your btrfs filesystem > >> Since I got tired of cloning and updating the

Re: BTRFS critical: corrupt leaf, slot offset bad; then read-only

2017-02-23 Thread Hans van Kranenburg
On 02/24/2017 12:47 AM, Lukas Tribus wrote: > Hello Hans, > > > Am 22.02.2017 um 20:40 schrieb Hans van Kranenburg: >> >> Question here is... is it easier for you to nuke the filesystem and >> restore the files from somewhere else, or do you want to figure out &

Re: Subvolume corruption after restart on Raid1 array

2017-02-23 Thread Hans van Kranenburg
"not caring", but more like "I really don't know" and just like not mailing a whole list with a "me too!" post, people won't mail "I don't know, dude, let's go bowling" too much. Or, it might be possible, but only realistically done when travelling to you, ge

Re: BTRFS hybrid system setup issues

2017-02-25 Thread Hans van Kranenburg
is ? I downloaded the git master branch and it > too seems to suffer from this issue. Maybe this is relevant here: https://www.spinics.net/lists/linux-btrfs/msg62692.html -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a

Re: BTRFS critical: corrupt leaf, slot offset bad; then read-only

2017-02-22 Thread Hans van Kranenburg
in it with a shotgun), then there's no automated other tool that can do it now. Since it's block 5242107641856 all the time, it might be worthwhile to have a look at it. Either it's that block, or there's a bigger mess hidden behind it. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Root volume (ID 5) in deleting state

2017-02-13 Thread Hans van Kranenburg
r disk space is allocated (200GiB of 200GiB). Even while there's still 70GiB of free space scattered around inside, this might lead to out-of-space issues, depending on how badly fragmented that free space is. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe

Re: Root volume (ID 5) in deleting state

2017-02-13 Thread Hans van Kranenburg
Hi, On 02/13/2017 09:50 PM, Martin Mlynář wrote: > On 13.2.2017 21:03, Hans van Kranenburg wrote: >> On 02/13/2017 12:26 PM, Martin Mlynář wrote: >>> I've currently run into strange problem with BTRFS. I'm using it as my >>> daily driver as root FS. Nothing compl

Re: Btrfs Heatmap - v4 ... colors!!

2017-01-18 Thread Hans van Kranenburg
up boundaries are too different from pixel boundaries. But, being able to play around with it is fun and serves a purpose, so I'll put this back in, and write a nice subpage of the README with some pictures of this kind to explain. o/ -- Hans van Kranenburg

Re: compress=lzo safe to use?

2016-09-11 Thread Hans van Kranenburg
u certainly want to have these commits in the kernel you're running right now. And when the bugs caused corruption, using a fixed kernel with not retroactively fix the corrupt data. Hint: "this was fixed in 4.x.y, so run that version or later" is not always the only answer here, because yo

Re: btrfstune -x -> extent-tree.c:2688: btrfs_reserve_extent: Assertion `ret` failed.

2016-09-12 Thread Hans van Kranenburg
On 09/09/2016 11:37 PM, Hans van Kranenburg wrote: > > While trying to enable skinny metadata on a filesystem, I got this error > (after minutes of reading from disk by the program): > > -# btrfstune -x /dev/xvdb > extent-tree.c:2688: btrfs_reserve_extent: Assertion `ret` f

Re: btrfstune -x -> extent-tree.c:2688: btrfs_reserve_extent: Assertion `ret` failed.

2016-09-12 Thread Hans van Kranenburg
On 09/12/2016 02:39 PM, David Sterba wrote: > On Fri, Sep 09, 2016 at 11:37:21PM +0200, Hans van Kranenburg wrote: >> Hi, >> >> While trying to enable skinny metadata on a filesystem, I got this error >> (after minutes of reading from disk by the program): >> >&

Re: btrfstune -x -> extent-tree.c:2688: btrfs_reserve_extent: Assertion `ret` failed.

2016-09-12 Thread Hans van Kranenburg
On 09/12/2016 02:46 PM, Hans van Kranenburg wrote: > On 09/12/2016 02:39 PM, David Sterba wrote: >> On Fri, Sep 09, 2016 at 11:37:21PM +0200, Hans van Kranenburg wrote: >>> Hi, >>> >>> While trying to enable skinny metadata on a filesystem, I got this error >

btrfstune -x -> extent-tree.c:2688: btrfs_reserve_extent: Assertion `ret` failed.

2016-09-09 Thread Hans van Kranenburg
level, on iSCSI storage) meant to be used for upgrade-testing and performance testing, so if anything goes wrong in whatever way, there will be no panicing involved. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message

Re: stability matrix

2016-09-15 Thread Hans van Kranenburg
f Filipe in the commits mentioned above. This helps setting up and maintaining the bug page, and helps advanced users to decide if they're hitting the edge case or not with their usage pattern. I'd like to help creating/maintaining this bug overview. A good start would be to just crawl through all stable

Re: Is stability a joke?

2016-09-19 Thread Hans van Kranenburg
t still needs > some other way to actually make it appear on wiki. Edit with courage! Oh, right there at the end, I expected: Join #btrfs on freenode IRC! :-D -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a messa

Re: [BUG] kernel BUG at fs/btrfs/extent_io.c:2062 (v4.2.0-rc8)

2016-08-25 Thread Hans van Kranenburg
more info: btrfs-progs 4.7, check reports many "incorrect local backref count" messages -- Hans van Kranenburg - System / Network Engineer T +31 (0)10 2760434 | hans.van.kranenb...@mendix.com | www.mendix.com -- To unsubscribe from this list: send the line "unsubscribe l

Re: What are "parent" and "parent_uuid" in "btrfs sub list"?

2016-10-17 Thread Hans van Kranenburg
ent 5 top level 5 parent_uuid 8de7ab74-4654-e542-a29b-169848ee73b3 path bar-snap and there's the parent_uuid... -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Btrfs Heatmap - v2 - block group internals!

2016-11-16 Thread Hans van Kranenburg
a fairly recent python-btrfs for the 'skinny' METADATA_ITEM_KEY to be present. Latest python-btrfs release is v0.3, created yesterday. Yay, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kerne

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-18 Thread Hans van Kranenburg
f course not perfect (you can't tell what device each segment of >> empty space corresponds to), but would probably cover most use cases. >> (for example, with such a scheme, you could look at an image and tell >> whether the data is relatively well distributed across all the de

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-18 Thread Hans van Kranenburg
Ha, On 11/18/2016 01:36 PM, Austin S. Hemmelgarn wrote: > On 2016-11-17 16:08, Hans van Kranenburg wrote: >> On 11/17/2016 08:27 PM, Austin S. Hemmelgarn wrote: >>> On 2016-11-17 13:51, Hans van Kranenburg wrote: >> But, the fun with visualizations of data is that you le

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-18 Thread Hans van Kranenburg
; people. It's in there, but hidden :) --curve linear -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-18 Thread Hans van Kranenburg
Hi, On 11/19/2016 01:57 AM, Qu Wenruo wrote: > On 11/18/2016 11:08 PM, Hans van Kranenburg wrote: >> On 11/18/2016 03:08 AM, Qu Wenruo wrote: >>>>> I don't see what displaying a blockgroup-level aggregate usage number >>>>> has to do with multi-device,

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Hans van Kranenburg
On 11/17/2016 08:27 PM, Austin S. Hemmelgarn wrote: > On 2016-11-17 13:51, Hans van Kranenburg wrote: >> >> When generating a picture of a file system with multiple devices, >> boundaries between the separate devices are not visible now. >> >> If someone has

Re: Btrfs Heatmap - v2 - block group internals!

2016-11-17 Thread Hans van Kranenburg
Hey, On 11/17/2016 02:27 AM, Qu Wenruo wrote: > > At 11/17/2016 04:30 AM, Hans van Kranenburg wrote: >> In the last two days I've added the --blockgroup option to btrfs heatmap >> to let it create pictures of block group internals. >> >> Examples and mor

Btrfs Heatmap - visualize your filesystem

2016-11-12 Thread Hans van Kranenburg
Hi, It seems I never mailed the list yet about a fun little tool to visualize the space usage of a btrfs filesystem: https://github.com/knorrie/btrfs-heatmap It needs python-btrfs to query a btrfs filesystem for block group usage and pypng to generate an image showing disk space usage as some

Re: Send/receive snapshot from/between backup (also restore example)

2016-11-02 Thread Hans van Kranenburg
On 11/02/2016 04:14 PM, René Bühlmann wrote: > On 11/02/2016 03:49 PM, Hans van Kranenburg wrote: >> On 11/02/2016 03:23 PM, René Bühlmann wrote: >>> I have a system running on btrfs which is backed up to two (one local >>> USB and one through SSH) external drives (also

Re: Snapper & apt-btrfs-snapshot on Ubuntu

2016-11-01 Thread Hans van Kranenburg
f btrfs, it's because those tools probably are designed to ignore anything they didn't cause to happen themselves. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Send/receive snapshot from/between backup

2016-11-02 Thread Hans van Kranenburg
same geographical location at the same moment, I have the same problem. What I did is just setting expiry the snapshots on the origin manually, and keep the meta-administration in my head. I don't do it that often anyway. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "

Re: speed up cp --reflink=always

2016-10-15 Thread Hans van Kranenburg
Hi, On 10/15/2016 10:49 PM, Stefan Priebe - Profihost AG wrote: > > cp --reflink=always takes sometimes very long. (i.e. 25-35 minutes) > > An example: > > source file: > # ls -la vm-279-disk-1.img > -rw-r--r-- 1 root root 204010946560 Oct 14 12:15 vm-279-disk-1.img > > target file after

Re: speed up cp --reflink=always

2016-10-16 Thread Hans van Kranenburg
On 10/16/2016 09:48 PM, Hans van Kranenburg wrote: > On 10/16/2016 08:54 PM, Stefan Priebe - Profihost AG wrote: >> Am 16.10.2016 um 00:37 schrieb Hans van Kranenburg: >>> On 10/15/2016 10:49 PM, Stefan Priebe - Profihost AG wrote: >>>> >>>> cp --reflink=

Re: speed up cp --reflink=always

2016-10-16 Thread Hans van Kranenburg
On 10/16/2016 08:54 PM, Stefan Priebe - Profihost AG wrote: > Am 16.10.2016 um 00:37 schrieb Hans van Kranenburg: >> On 10/15/2016 10:49 PM, Stefan Priebe - Profihost AG wrote: >>> >>> cp --reflink=always takes sometimes very long. (i.e. 25-35 minutes) >>>

Re: Incremental send robustness question

2016-10-12 Thread Hans van Kranenburg
eferenced file is > there but contains different data? Are there checks for this sort of > thing, or is it always assumed that the parent subvols are identical and > if they're not, you're in undefined behavior land? btrfs send/receive is not rsync :-) -- Hans van Kranenburg -- To

Re: Incremental send robustness question

2016-10-12 Thread Hans van Kranenburg
On 10/13/2016 01:47 AM, Sean Greenslade wrote: > On Thu, Oct 13, 2016 at 01:14:51AM +0200, Hans van Kranenburg wrote: >> On 10/13/2016 12:29 AM, Sean Greenslade wrote: >>> And while we're at it, what are the failure modes for incremental sends? >>> Will it throw an error

Re: Rust library for BTRFS

2016-10-15 Thread Hans van Kranenburg
#btrfs on freenode is a good place to hang out. Have fun, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

python-btrfs v5, yay

2017-01-14 Thread Hans van Kranenburg
ter writing back the metadata page, the filesystem was usable again! :-) Have fun, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Btrfs Heatmap - v4 ... colors!!

2017-01-14 Thread Hans van Kranenburg
of beautiful pictures and timelapses of your filesystem! Have fun! P.S. Here's where it all started... :D http://logs.tvrrug.org.uk/logs/%23btrfs/2015-12-09.html#2015-12-09T20:54:04 -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of

Re: Metadata balance fails ENOSPC

2016-12-01 Thread Hans van Kranenburg
re whether it was merged. http://www.spinics.net/lists/linux-btrfs/msg56772.html >>> Try remounting with enoscp_debug, and then trigger the problem again, >>> and post the resulting kernel messages. >> >> With enospc debug it says: >> [39193.425682] BTRFS warnin

Re: [markfasheh/duperemove] Why blocksize is limit to 1MB?

2017-01-03 Thread Hans van Kranenburg
On 01/04/2017 12:12 AM, Peter Becker wrote: > Good hint, this would be an option and i will try this. > > Regardless of this the curiosity has packed me and I will try to > figure out where the problem with the low transfer rate is. > > 2017-01-04 0:07 GMT+01:00 H

Re: [markfasheh/duperemove] Why blocksize is limit to 1MB?

2017-01-03 Thread Hans van Kranenburg
or bsd systems and > snapshoted after copy. XY problem? Why not use rsync --inplace in combination with btrfs snapshots? Even if the remote does not support rsync and you need to pull the full file first, you could again use rsync locally. -- Hans van Kranenburg -- To unsubscribe from thi

Btrfs Heatmap - v3 - scriptable .. more flexible

2016-12-18 Thread Hans van Kranenburg
31afcf0e3ef076d7ba3c5ec7966 Oh, and I removed the code for the linear output mode, because my opinion is that is really does not lead to usable images. And yes, the next task is getting some colors involved! \o/ -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscr

python-btrfs v4, yay

2016-12-18 Thread Hans van Kranenburg
one is the documentation. Have fun, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Btrfs Heatmap - v3 - scriptable .. more flexible

2016-12-19 Thread Hans van Kranenburg
Rereading this... On 12/19/2016 12:53 AM, Hans van Kranenburg wrote: > [...] > > Qu Wenruo pointed out that the greyscale value used for dev_extent (the > usage for the block group is used here) does not necessarily have to be > correct: "And for 50%/50% assumption for

Re: uncorrectable error, but no file there

2016-12-16 Thread Hans van Kranenburg
e than what was written before. With some other filesystems you would be working with silently corrupted data right now. -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Qgroups are not applied when snapshotting a subvol?

2017-03-27 Thread Hans van Kranenburg
o be deleted by a non-root user. Use with > caution. > What are actual use cases for creating subvolumes by 'normal' users? Does someone have an example? Why is it possible at all, by default? -- Hans van Kranenburg -- To unsubscribe from this list: send the line &q

Cosmetics bug: remounting ssd does not clear nossd

2017-03-30 Thread Hans van Kranenburg
in the future, when anything would test for NOSSD explicitely and make a decision based on the result. Moo, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo inf

Bug: misleading free space tree messages when mounting ro

2017-03-30 Thread Hans van Kranenburg
rse not displaying the option as active afterwards. Moo, -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

  1   2   3   4   >