Re: btrfs recovery

2017-01-29 Thread Oliver Freyermuth
Am 29.01.2017 um 20:28 schrieb Hans van Kranenburg: > On 01/29/2017 08:09 PM, Oliver Freyermuth wrote: >>> [..whaaa.. text.. see previous message..] >> Wow - this nice python toolset really makes it easy, bigmomma holding your >> hands ;-) . >> >> Indeed, I

Re: btrfs recovery

2017-01-29 Thread Oliver Freyermuth
Am 29.01.2017 um 17:44 schrieb Hans van Kranenburg: > On 01/29/2017 03:02 AM, Oliver Freyermuth wrote: >> Am 28.01.2017 um 23:27 schrieb Hans van Kranenburg: >>> On 01/28/2017 10:04 PM, Oliver Freyermuth wrote: >>>> Am 26.01.2017 um 12:01 schrieb Oliver Freyermuth

Re: btrfs recovery

2017-01-28 Thread Oliver Freyermuth
Am 28.01.2017 um 23:27 schrieb Hans van Kranenburg: > On 01/28/2017 10:04 PM, Oliver Freyermuth wrote: >> Am 26.01.2017 um 12:01 schrieb Oliver Freyermuth: >>> Am 26.01.2017 um 11:00 schrieb Hugo Mills: >>>>We can probably talk you through fixing this by hand

Re: btrfs recovery

2017-01-28 Thread Oliver Freyermuth
Am 26.01.2017 um 12:01 schrieb Oliver Freyermuth: >Am 26.01.2017 um 11:00 schrieb Hugo Mills: >>We can probably talk you through fixing this by hand with a decent >> hex editor. I've done it before... >> > That would be nice! Is it fine via the mailing list? > Po

Re: btrfs recovery

2017-01-28 Thread Oliver Freyermuth
Hi Duncan, thanks for your extensive reply! Am 28.01.2017 um 06:00 schrieb Duncan: > All three options apparently default to 64K (as that's what I see here > and I don't believe I've changed them), but can be changed. See the > kernel options help and where it points for more. > Indeed, I

Re: btrfs recovery

2017-01-28 Thread Oliver Freyermuth
Am 28.01.2017 um 13:37 schrieb Janos Toth F.: > I usually compile my kernels with CONFIG_X86_RESERVE_LOW=640 and > CONFIG_X86_CHECK_BIOS_CORRUPTION=N because 640 kilobyte seems like a > very cheap price to pay in order to avoid worrying about this (and > skip the associated checking + monitoring).

Re: btrfs recovery

2017-01-27 Thread Oliver Freyermuth
> I'm also running 'memtester 12G' right now, which at least tests 2/3 of the > memory. I'll leave that running for a day or so, but of course it will not > provide a clear answer... A small update: while the online memtester is without any errors still, I checked old syslogs from the machine

Re: btrfs recovery

2017-01-26 Thread Oliver Freyermuth
>It's on line 248 of the paste: > > 246. key (5547032576 EXTENT_ITEM 204800) block 596426752 (36403) gen 20441 > 247. key (5561905152 EXTENT_ITEM 184320) block 596443136 (36404) gen 20441 > 248. key (15606380089319694336 UNKNOWN.76 303104) block 596459520 (36405) > gen 20441 > 249.

Re: btrfs recovery

2017-01-26 Thread Oliver Freyermuth
Hi and thanks for the quick reply! Am 26.01.2017 um 10:25 schrieb Hugo Mills: >Can you post the output of "btrfs-debug-tree -b 35028992 > /dev/sdb1", specifically the 5 or so entries around item 243. It is > quite likely that you have bad RAM, and the output will help confirm > that. >

btrfs recovery

2017-01-26 Thread Oliver Freyermuth
Hi, I have just encountered on mount of one of my filesystems (after a clean reboot...): [ 495.303313] BTRFS critical (device sdb1): corrupt node, bad key order: block=35028992, root=1, slot=243 [ 495.315642] BTRFS critical (device sdb1): corrupt node, bad key order: block=35028992,

Re: [bug] btrfs-progs-v4.8.4: test of 'btrfs receive' in xfstests fails

2016-11-29 Thread Oliver Freyermuth
Hi, just to chime in on this: This issue also affects me as a "downstream" user, so it also breaks real-life usecases. I use btrbk for backup, and when performing a regular incremental backup, after switching to btrfs-progs 4.8.4, I get the same "short read from stream" problem.

Re: btrfs send extremely slow (almost stuck)

2016-09-06 Thread Oliver Freyermuth
Duncan wrote on Mon, 05 Sep 2016 19:14:30 -0700: > I had something very similar happen here a few weeks ago, except with my > firefox profile dir (I don't run thunderbird, preferring claws-mail, but > I do run firefox as my browser). Indeed, I also note Firefox doing a lot of IO especially if

Re: btrfs send extremely slow (almost stuck)

2016-09-06 Thread Oliver Freyermuth
Am 06.09.2016 um 04:46 schrieb Qu Wenruo: > But your idea to locate the inode seems good enough for debugging though. Based on this I even had another idea which seems to have worked well - and I am now also able to provide any additional debug output you may need. Since my procedure may be

Re: btrfs send extremely slow (almost stuck)

2016-09-05 Thread Oliver Freyermuth
Am 05.09.2016 um 07:21 schrieb Qu Wenruo: > Did you get the half way send stream? Luckily, yes! > If the send stream has something, please use "--no-data" option to send the > subvolume again to get the metadata only dump, and upload it for debug. Also the metadata-only dump fails with the

Re: btrfs send extremely slow (almost stuck)

2016-09-04 Thread Oliver Freyermuth
Am 30.08.2016 um 02:48 schrieb Qu Wenruo: > Yes. > And more specifically, it doesn't even affect delta backup. > > For shared extents caused by reflink/dedupe(out-of-band or even incoming > in-band), it will be send as individual files. > > For contents, they are all the same, just more space

Re: btrfstune settings

2016-09-01 Thread Oliver Freyermuth
Hi, Am 01.09.2016 um 19:14 schrieb David Sterba: > all your questions should be now answered and documentation cross > referenced among the relevant manual pages (will be synced to wiki at > release time: > > https://github.com/kdave/btrfs-progs/blob/devel/Documentation/mkfs.btrfs.asciidoc >

Re: btrfs send extremely slow (almost stuck)

2016-08-29 Thread Oliver Freyermuth
Am 29.08.2016 um 04:11 schrieb Qu Wenruo: > Unknown bug, while unfortunately no good idea to solve yet. > > I sent a RFC patch to completely disable shared extent detection, while > got strong objection. > > I also submitted some other ideas on fixing it, while still got strong > objection.

Re: btrfstune settings

2016-08-28 Thread Oliver Freyermuth
> Try borgbackup, I'm using it very successfully. It is very fast, > supports very impressive deduplication and compression, retention > policies, and remote backups - and it is available as a single binary > version so you can more easily use it for disaster recovery. One > downside: while it

Re: btrfstune settings

2016-08-28 Thread Oliver Freyermuth
(sorry if my Message-ID header is missing, I am not subscribed to the mailing list, so I reply using mail-archive) > Try btrfs-show-super . The incompat_flags section enumerates the > flags that are on (at least with a reasonably new btrfs-progs). Thanks a lot for this hint - I totally

Re: btrfs send extremely slow (almost stuck)

2016-08-28 Thread Oliver Freyermuth
(sorry if my Message-ID header is missing, I am not subscribed to the mailing list, so I reply using mail-archive) > So a workaround would be reducing your duperemove usage and possibly > rewriting (for instance via defrag) the deduped files to kill the > multiple reflinks. Or simply delete

btrfstune settings

2016-08-27 Thread Oliver Freyermuth
Dear btrfs experts, I hope this is the correct place to ask, the wiki and manpages did not help me on these questions. BTRFS has gained extended inode refs, skinny metadata and no-holes quite a while ago and these are now the defaults for new mkfs.btrfs. btrfstune can activate those

btrfs send extremely slow (almost stuck)

2016-08-27 Thread Oliver Freyermuth
Dear btrfs experts, I just tried to make use of btrfs send / receive for incremental backups (using btrbk to simplify the process). It seems that on my two machines, btrfs send gets stuck after transferring some GiB - it's not fully halted, but instead of making full use of the available