Re: "Unable to find ref byte nr ...." (4.11 somehow fishy?)

] BTRFS info (device dm-1): forced readonly [11972.530866] BTRFS: error (device dm-1) in btrfs_run_delayed_refs:2971: errno=-2 No such entry 2017-07-31 21:13 GMT+02:00 Ivan Sizov <sivan...@gmail.com>: > 2017-07-11 21:57 GMT+03:00 Clemens Eisserer <linuxhi...@gmail.com>: >>

"Unable to find ref byte nr ...." (4.11 somehow fishy?)

Hi, My external drive (single metadata, inline) caused an Ooops recently and went read-only. I wonder, has this something to do with the btrfs issues some have with 4.11? Best regards, Clemens [ 619.850316] perf: interrupt took too long (3958 > 3941), lowering kernel.perf_event_max_sample_rate

Re: [PATCH v2 3/4] btrfs: Add zstd support

> So, I don't see any problem making the level configurable. +1 - configureable compression level would be very appreciated from my side. Can't wait until zstd support is mainline :) Thanks and br, Clemens -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a

Error during backup to zlib-compressed btrfs-volume (kernel stack frame pointer at ffff9f1f433ebf50 in kworker/u8:7:6702 has bad value (null))

Hi, During a backup to an external USB3 hdd with the following mount options "rw,nosuid,nodev,relatime,compress-force=zlib,space_cache,subvolid=5,subvol=/,user" runnning linux-4.10.11-200.fc25.x86_64 I got the backtrace at the end of the email. Best regards, Clemens [ 3723.511132] WARNING:

Will dupremove work for compressed btrfs volumnes?

Hi, Sorry if this is off-topic, but the dupremove project doesn't seem to have a mailing list - and it is actually only generating extent-same ioctls handled by btrfs anyway. I recently gave dupremove a try on my btrfs-volume (mounted with compress-force=lzo), specifically on a folder where I

Re: btrfs ate my data in just two days, after a fresh install. ram and disk are ok. it still mounts, but I cannot repair

nel-3.14 or so. Br, Clemens Eisserer 2016-05-07 17:45 GMT+02:00 Niccolò Belli <darkba...@linuxsystems.it>: > btrfs + dmcrypt + compress=lzo + autodefrag = corruption at first boot > So discard is not the culprit. Will try to remove compress=lzo and > autodefrag and see i

Re: Bad performance with near-full FS

Hi, Thanks for you reply, If I really understand, it's always a good idea to keep more than 1% of free space. Right ? Usually the rule of thumb is 10-20%. Br, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

Re: Deadlock with 3.15.10

Hi Liu, I've recently run into a deadlock on 3.15.10, don't know if the kernel stack-trace is useful: http://pastebin.com/guQNDAMX FYI, this's been fixed by https://patchwork.kernel.org/patch/4727711/ Thanks for letting me know. - Clemens -- To unsubscribe from this list: send the line

Deadlock with 3.15.10

Hi, I've recently run into a deadlock on 3.15.10, don't know if the kernel stack-trace is useful: http://pastebin.com/guQNDAMX Br, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: ENOSPC on mostly empty file system

Hi Arnd, Ok, one more data point: Why don't you provide the data point you were specifically asked for, btrfs fi df ;) Regards, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: Large files, nodatacow and fragmentation

Hi Richard, It is interesting that for me the number of extents before and after bcache are essentially the same. The lesson here for me there is that the fragmentation of a btrfs nodatacow file is not mitigated by bcache. There seems to be nothing I can do to prevent that fragmentation,

Re: Announcement: buttersink - like rsync for btrfs snapshots

Hi Jim, To any core btrfs devs who are listening and care - the unreliability of btrfs send/receive is IMO the single biggest roadblock to adoption of btrfs as a serious next-gen FS. For you it are send/receive deficiencies, however there are many other feature/enhancement requests having top

Re: ENOSPC with mkdir and rename

Hi Russel, The Debian installer has BTRFS in a list of filesystems to choose with no special notice about it. I'm thinking of filing a Debian bug requesting that they put a warning against it. As long as it is not selected as the default filesystem, I think it is fine. Other distributions

Re: ENOSPC with mkdir and rename

Hi Peter, All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn't realise there was an un-holstered footgun on my lap at this point. How can it be made clear how to avoid the ENOSPC problem to myself and other sysadmins? Or

Re: ENOSPC with mkdir and rename

Hi Chris, Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. Which doesn't protect the *average* user from running into issues like this. Just because it has been discussed, doesn't mean nothing can/should be done about it ;) However, as I am only a

Re: ENOSPC with mkdir and rename

Hi Hugo, On the 3.15+ kernels, the block reserve is split out of metadata and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now. One thing I didn't find any information about is the overhead introduced by

What do the wait_current_trans messages mean I see on my raspberry pi?

Hi, Over the weekend I tried to copy one external usb drive (on ext4) to another one formatted with btrfs. Now I came back, and 48h later only ~300GB were copied and I found messages like the following on syslog: May 16 18:33:13 raspberrypi kernel: [ 8161.213142] btrfs-endio-wri D c0420f24 0

Re: Rebalance makes BTRFS 10x slower

Hi Ducan, But as I said, I'd sure like to get to the bottom of this one, since I do believe it has other potential implications in terms of bugs, etc. In theory, a balance should either not affect performance or should improve it, so getting to the bottom of why it's having such a bad

Re: Rebalance makes BTRFS 10x slower

Hi, I also found this to be the case as I rebalanced the root filesystem of my Debian installation on this ThinkPad T520 on an Intel SSD 320. Same here, I ran balance on kernel-3.12 some time ago and after balancing performance dropped noticeable and stayed there. When booting natively I can

Re: Is there any way to determine fragmentation for compressed btrfs volumes?

will handle workload like mine more gracefully with these issues fixed. Thanks best regards, Clemens 2014-04-08 21:41 GMT+02:00 Clemens Eisserer linuxhi...@gmail.com: Hi Ducan, You mention trying scrub and defragging the entire volume, but you don't mention balance. Balance by default rewrites all

Is there any way to determine fragmentation for compressed btrfs volumes?

in 500ms). What I wonder is how can I diagnose whats happening and why the FS is rather slow? The only target value I know is the extend count, which doesn't seem to be meaningful when using fragmentation. Thank you in advance, Clemens Eisserer -- To unsubscribe from this list: send the line

Re: BTRFS setup advice for laptop performance ?

Hi, This is because every other filesystem (except ZFS) doesn't use COW semantics. Nilfs2 also is COW based. Regards, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: Is there any way to determine fragmentation for compressed btrfs volumes?

Hi Ducan, You mention trying scrub and defragging the entire volume, but you don't mention balance. Balance by default rewrites all chunks (tho you can add filters to rewrite only say data chunks, not metadata, if you like), so that's what I'd say to try, as it should defrag in the process.

Filesystem extremly fragmented after scrub

Hi, I am using btrfs on a single device with crompress-force=lzo and after I scrubbed the device once (kernel-3.12), the filesystem is extremly fragmented - running defrag on all file didn't improve the situation unfourtunately. Because I have a fast SSD this is barely noticeable when booting the

Re: Filesystem extremly fragmented after scrub

Hi, Some general notes .. VM images ... While NOCOW will prevent COW ... As mentioned the VM is configured to access the partition where linux is installed directly. So btrfs is not used as filesystem on the host machine, but rather on the guest which writes directly to disk (so no other

How does btrfs handle bad blocks in raid1?

Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be horribly unreliable. A 32GB microSDHC card reported bad blocks after 4 days, while a usb pen drive returns bogus data without any warning at all. So I wonder, how would btrfs

Re: How does btrfs handle bad blocks in raid1?

Hi George, I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures accumulate, SMART will warn in an obvious way that the drive in

Re: rootfs crash

Hi Jogi, When I tried to recovered a broken btrfs-parition, I encountered the same errors you did - btrfs restore seems to be a bit broken. What helped was to: a. comment out the free()-calls which led to crashes b. re-run btrfs restore to account for the crashes which happen elsewhere,

Re: Btrfsck Assertion btrfs_find_last_root !(path-slots[0] == 0) - any hope left?

of this broken FS ... Would it make sense to simply disable the transid checks? Thank you in advance, Clemens 2013/9/7 Clemens Eisserer linuxhi...@gmail.com: Hi, Because of being clumsy, I recently overwrote my btrfs-partition. Worse, the only recent backup I own is a dd-copy which was created

Btrfsck Assertion btrfs_find_last_root !(path-slots[0] == 0) - any hope left?

Hi, Because of being clumsy, I recently overwrote my btrfs-partition. Worse, the only recent backup I own is a dd-copy which was created while the system was running. Both, btrfsck (from btrfsprogs-next) and mount -o recovery failed - so I wonder, are there any steps left I could try?

Re: snapshot space available

You are out of metadata, not normal space. However, the good question is why 0.5GB of metadata are unused and btrfs reports no space left. I have seen similar behaviour on a machine of mine, with exactly 0.5GB of metadata unused. Regards 2013/8/26 Russell Coker russ...@coker.com.au: Linux xev

Re: Samba strict allocate = yes stops btrfs compression working

Hi, The speed improvement for dumping large databases through samba with strict allocate = yes to BTRFS was amazing. It reduced a 1 hour dump down to 20 minutes. What you want btrfs to do is to allocate a file of fixed-size on disk in advance, without knowing how large the file will be after

Re: Why does btrfs benchmark so badly in this case?

What is going on here? Why is btrfs doing so poorly? Funny thing, I was thinking exactly the same when reading the article ;) Regards -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: abysmal rm performance?

So... read up on the wiki a bit, then come back with questions you have that aren't answered there. (I certainly had some I didn't see directly answered there when I first started with btrfs.) I guess the original email was more ment as a bug-report than a question, as the question was more

Re: unclean shutdown and space cache rebuild

Hi, I suspect this is, at least in part, related to severe fragmentation in /home. In his cause those issues are only present after an unclean shutdown - whereas fragmentation would show its effect after every reboot. Regards, Clemens -- To unsubscribe from this list: send the line

Re: hang on 3.9, 3.10-rc5

Hi Jon, Is this what you are looking for? After this, the CPU gets stuck and I have to reboot. This issue has already been reported: https://bugzilla.kernel.org/show_bug.cgi?id=59451 Regards, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a

Two identical copies of an image mounted result in changes to both images if only one is modified

Hi, I've observed a rather strange behaviour while trying to mount two identical copies of the same image to different mount points. Each modification to one image is also performed in the second one. Example: dd if=/dev/sda? of=image1 bs=1M cp image1 image2 mount -o loop image1 m1 mount -o loop

Re: Btrfsck errors (fs tree 565 refs 125 not found) - how serious

Hi, You are running on an unmounted fs right? Also please make sure you are running the git version Yes, the fs was properly unmounted. git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git I still get those errors, I'll try to file a bug-report with the tools/data

Re: Btrfsck errors (fs tree 565 refs 125 not found) - how serious

Hi again, I am able to induce the btrfsck errors I experienced using a synthetic workload on a fresh filesystem with linux-3.10.0.rc2. However as filing the bug-report would take quite some time (uploading 512mb trace-files, writing a short read-me, ...) I wonder whether this is an issue woth of

Re: systemd journalctl is 1.89sec on ext4, 1.49 min on btrfs

Hi, Journalctl on btrfs was always this slow, some btrfsck were made on the file system too, but I don't think it was corrupted. On just the first run it's sluggish, after it's fast as the ext4 one. Most likely the file is heavily fragmented due to COW. Have you tried to manually defrag the

Btrfsck errors (fs tree 565 refs 125 not found) - how serious

Hi, For about 3 weeks I've been using btrfs on my desktop machine, using the snapshot-functionality quite a lot. Today I ran btrfsck and got about ~10MB of output which looks like: fs tree 565 refs 125 not found unresolved ref root 807 dir 813347 index 277 namelen 39 name

Re: btrfs (general) raid for other filesystems?

Hi Martin, So, an interesting variation could be to have filesystem level raid operating on ext4 or nilfs or whatever... Would that be a sensible idea? Thats already supported by using LVM. What do you think you would gain from layering in top of btrfs? - Clemens -- To unsubscribe from this

unlinked 10 orphans - something to worry about?

Hi, I frequently get messages like unlinked 10 orphans in syslog (running linux 3.9.1), although I have never had a power outage nor a kernel crash. Is this something to worry about, or just a usual clean-up information? Thank you in advance, Clemens -- To unsubscribe from this list: send the

Can btrfs cope with a high number of snapshpts (continuous snapshotting)?

Hi, I am looking for a file-system which is able to provide continuous snapshotting like some log-structured file-systems do. The btrfs snapshot-mechanism seems to be what I am looking for, in combination with a daemon monitoring the free space it should be possible to create a snapshot every

Re: No space left on device although df reports only 55% in use

Hi again, I wonder if this is the intended behaviour or some known issue? Otherwise I could provide a tool written by a friend of mine which can trigger this issue within a few minutes on a a fresh btrfs partition. Its basically a file-system aging tester, which replays some real-world logs taken

Re: No space left on device although df reports only 55% in use

Hi Hugo, # btrfs filesystem df . Data: total=28.22GB, used=14.25GB System, DUP: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=1.50GB, used=1.16GB Your metadata is close to full -- we need quite a lot of working space to CoW into. You (probably) have

No space left on device although df reports only 55% in use

Hi, I am using a btrfs loopback mounted file with lzo-compression on Linux-3.7.9, and I ran into No space left on device messages, although df reports only 55% of space is used: # touch testfile touch: cannot touch `testfile': No space left on device # df Filesystem 1K-blocks Used

Re: SSD format/mount parameters questions

I would not buy anything else than intel. I have about 26 of them for years now (both in servers and workstations, several series), and never had an issue. Two of my colleagues have OCZ, and both had to RMA them. I guess it boils down wether you want intel also to rule the SSD market in the

Can btrfs silently repair read-error in raid1

Hi, I have a quite unreliable SSD here which develops some bad blocks from time to time which result in read-errors. Once the block is written to again, its remapped internally and everything is fine again for that block. Would it be possible to create 2 btrfs partitions on that drive and use it

Re: kernel 3.3.4 damages filesystem (?)

Hi Helmut, But where's the gain? If a disk fails I have a lot of tools for repairing an ext2/3/4 system. Nope, when a disk in your ext4 raid0 array fails, you are just as doomed. But when I use ext2/3/4 I neither need RAID0 nor do I need LVM. You can use btrfs, without using its raid

Re: brtfs on top of dmcrypt with SSD. No corruption iff write cache off?

Hi, -o discard is supported, but can have some negative consequences on performance on some SSDs or at least whether it adds worthwhile performance is up for debate The problem is, that TRIM can't be benchmarked in a traditional way - by running one run with TRIM and another one without

Re: Honest timeline for btrfsck

Any update on the state of btrfschk? Thanks, Clemens 2011/10/31 David Summers bt...@summers5913.freeserve.co.uk: On 05/10/11 07:16, Chris Mason wrote: So over the next two weeks I'm juggling the merge window and the fsck release.  My goal is to demo fsck at linuxcon europe.  Thanks again

Re: Honest timeline for btrfsck

+1 2011/9/27 Jeff Putney jeffrey.put...@gmail.com: On Thu, Aug 18, 2011 at 3:50 PM, Chris Mason chris.ma...@oracle.com wrote:  I had hoped to get #1 out the door before I left on vacation and I still might post it tonight. -chris I don't think this is the honest time line that people were

Re: BTRFS fsck tool

Well, the missing file-system checker is the main reason I don't use btrfs in production environments. The other issue are servere performance problems in corner cases (e.g. when deleting 15GB data takes 100% cpu and hours) - Clemens 2011/3/8 Peter Stuge pe...@stuge.se: Hi, Alexey A Nikitin

Summer of Code project idea

, in the long term this could be even more profitable for btrfs - without any porting effort the latest btrfs-features could be used by the Windows-driver too, without time consuming code-sync. If you find the project-idea cool and/or would like to mentor me, please let me know. Thanks, Clemens Eisserer

Re: Summer of Code project idea

Hi Chris, I think there are a ton of good summer of code ideas for Btrfs, but the windows driver option isn't quite at the top of my list.  It isn't that I have something against windows, but there's a ton of work to be done on the Linux side that I'd rather concentrate on first ;) Hmm,

null pointer dereference in iov_iter_copy_from_user_atomic while updating rpm packages

Hi, While updating my fedora rawhide installation, I got the Ooops listed at the end of the Email. Is this a known bug (I didn't find anything specific), or should I file a bug? Thank you in advance, Clemens Feb 10 10:59:45 testbox kernel: [ 524.495751] BUG: unable to handle kernel NULL

Re: slow deletion of files

Forgot to mention, I filed already a report for this problem: https://bugzilla.kernel.org/show_bug.cgi?id=16117 However, like the other btrfs bug I filed, it was never looked at - so I decided it was a waste of time to file bugs against it. - Clemens -- To unsubscribe from this list: send the

Music-files suddenly not recognized anymore, empty files - Btrfschk reports errors

Hi, Recently Amarok started to reject some of music files. Some files seem to be simply corrupted (mplayer is still able to play them), other have zero length. Btrfsck reports: root 5 inode 1371 errors 400 found 42498768896 bytes used err is 1 total csum bytes: 41015664 total tree bytes:

Re: Music-files suddenly not recognized anymore, empty files - Btrfschk reports errors

Hi, have you taken any snapshots?  you could pull them from there maybe. No snapshots unfourtunatly. I wonder what could have caused those filesystem errors. The only unusual thing I did was to mount it with compress, but other than that - i didn't even stress it a lot. btrfsck is not able to