Compressed Filesystem
I have read on the mailing list that there has been some interest in implementing transparent compression on btrfs and I too am thinking about trying to implement it. Before I start from scratch I am wondering if anyone else has started to work on this and if so how far along have they gotten? I would be happy to work on this alone or with someone else but currently I am doing some preliminary research. Thanks, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
On Tue, Oct 28, 2008 at 11:47:27AM -0400, Chris Mason wrote: > On Mon, 2008-10-27 at 10:54 -0400, Lee Trager wrote: > > I have read on the mailing list that there has been some interest in > > implementing transparent compression on btrfs and I too am thinking about > > trying to implement it. Before I start from scratch I am wondering if > > anyone else has started to work on this and if so how far along have > > they gotten? I would be happy to work on this alone or with someone else > > but currently I am doing some preliminary research. > > Compression is working on my machine, I'm just running some long tests > before I push it out to the unstable repo. The current code uses the > in-kernel zlib implementation. > > -chris > Thats great I am eager to try it. How long will these tests take? I would love to look at the code. Is compression done for every file by default or does a user space program have to set a compression flag on the file? Thanks, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Kernel oops when running bonnie++ on btrfs
I wanted to see how btrfs compares to other filesystems so I have been running bonnie++ on it. While the results are good(much faster then ext2) every once in awhile I get a kernel oops. I am testing on xubuntu 8.10 with the 2.6.27-7-686 kernel using the latest git sources. Most of the time the oops happens within 20min of running bonnie++ but sometimes it takes a few hours. This happens with and without compression. To reproduce this bug you can run while true; do bonnie++ -s 5120 -n 4:524288:0:512; done Nov 12 22:39:12 Intrepid-btrfs kernel: [ 2531.781731] allocation failed flags 1, wanted 4096^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.781758] space_info has 3822407680 free, is full^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814789] block group 12582912 has 8388608 bytes, 8388608 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814837] 0 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814869] block group 424542208 has 790429696 bytes, 790429696 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814890] 1 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814893] block group 1214971904 has 790429696 bytes, 790347776 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814896] 1 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814898] block group 2005401600 has 790429696 bytes, 790421504 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814901] 1 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814903] block group 2795831296 has 790429696 bytes, 4096 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814906] 1 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814909] block group 3586260992 has 790429696 bytes, 1867776 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814911] 1 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814950] block group 4376690688 has 790429696 bytes, 0 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814952] 0 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814955] block group 5167120384 has 790429696 bytes, 0 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814958] 0 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814960] block group 5957550080 has 790429696 bytes, 127959040 used 0 pinned 0 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814963] 0 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814965] block group 6747979776 has 752943104 bytes, 728870912 used 0 pinned 24072192 reserved^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.814968] 0 blocks of free space at or bigger than bytes is^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.815588] [ cut here ]^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.815960] kernel BUG at /home/ltrager/btrfs/btrfs-unstable-standalone/extent-tree.c:2437!^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] invalid opcode: [#1] SMP ^M Nov 12 22:39:13 Intrepid-btrfsNov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] ^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] Pid: 5193, comm: btrfs-delalloc- Not tainted (2.6.27-7-generic #1)^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] EIP: 0060:[] EFLAGS: 00010257 CPU: 0^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] EIP is at __btrfs_reserve_extent+0x3c2/0x480 [btrfs]^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] EAX: df72d284 EBX: dd27c180 ECX: EDX: 0001^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] ESI: dd27c1ac EDI: df72d278 EBP: d80e7de8 ESP: d80e7d74^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] DS: 007b ES: 007b FS: 00d8 GS: SS: 0068^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] Process btrfs-delalloc- (pid: 5193, ti=d80e6000 task=d80bcb60 task.ti=d80e6000)^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] Stack: e090d750 9236 0001 2ce1 2b71b000 ^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] 016f5000 0001 ^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] 1000 dee450a0 df61c000 ^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] Call Trace:^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] [] ? btrfs_reserve_extent+0x77/0xb0 [btrfs]^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 2531.816012] [] ? cow_file_range+0x227/0x4d0 [btrfs]^M Nov 12 22:39:13 Intrepid-btrfs kernel: [ 25
Re: Kernel oops when running bonnie++ on btrfs
Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] 00189000 0001 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] 1000 dee44190 df6aa000 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] Call Trace: Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? btrfs_reserve_extent+0x77/0xb0 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? cow_file_range+0x227/0x4d0 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? submit_compressed_extents+0x458/0x4d0 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? finish_task_switch+0x2b/0xe0 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? async_cow_submit+0x8b/0xa0 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? run_ordered_completions+0x74/0xd0 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? worker_loop+0x98/0x180 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? worker_loop+0x0/0x180 [btrfs] Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? kthread+0x41/0x80 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? kthread+0x0/0x80 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] [] ? kernel_thread_helper+0x7/0x10 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] === Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] Code: ec aa df 8d 4b 44 89 c8 89 4d f0 e8 39 02 ab df 8b 7b 38 8b 07 0f 18 00 90 83 c3 38 39 fb 89 5d ec 75 33 8b 45 f0 e8 fe e2 87 df <0f> 0b eb fe 66 90 8b 90 90 18 00 00 8b 88 78 18 00 00 8b b0 8c Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.136203] EIP: [] __btrfs_reserve_extent+0x3c2/0x480 [btrfs] SS:ESP 0068:d82dbd74 Nov 14 16:23:46 Intrepid-btrfsc kernel: [ 1299.159252] ---[ end trace ae4786bfdd8753ad ]---? On Thu, Nov 13, 2008 at 02:20:05PM -0500, Josef Bacik wrote: > On Thu, Nov 13, 2008 at 01:49:07PM -0500, Lee Trager wrote: > > I wanted to see how btrfs compares to other filesystems so I have been > > running bonnie++ on it. While the results are good(much faster then > > ext2) every once in awhile I get a kernel oops. I am testing on xubuntu > > 8.10 with the 2.6.27-7-686 kernel using the latest git sources. Most of the > > time the oops happens within 20min of running bonnie++ but sometimes it > > takes a few hours. This happens with and without compression. > > > > To reproduce this bug you can run > > > > while true; do bonnie++ -s 5120 -n 4:524288:0:512; done > > > > Thanks, still tracking down random places where we ENOSPC too soon. Have you > updated today? There was a fix that went in earlier today that may help you. > I > wll run with your command and see if I can replicate here and fix it. > > Josef > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel oops when running bonnie++ on btrfs
7539M Lee On Fri, Nov 14, 2008 at 03:26:37PM -0500, Josef Bacik wrote: > How big is your fs? Thanks, > > Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fix free space leak
I to am getting the same error when running bonnie++. Lee Content-Description: Forwarded message - Re: [PATCH] fix free space leak > From: "Mitch Harder (aka DontPanic)" <[EMAIL PROTECTED]> > To: "linux-btrfs@vger.kernel.org" > Subject: Re: [PATCH] fix free space leak > Accept-Language: en-US > X-Auto-Response-Suppress: All > list-id: > x-mailing-list: linux-btrfs@vger.kernel.org > > I tested this patch by compiling OpenOffice on a 3.5 GB partition > using compression. > > I am still getting an allocation error at the very end of the build > process when trying to delete the work files once the compilation > completed and installed successfully. > > The FETCH_HEAD that I applied the patch to was: > $ cat .git/FETCH_HEAD > a350bf67481b0a0cf52bc0be9171f27e442871a5branch > 'master' of > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable-standalone > > In the dmesg output, please note that the typesconfig segfault is a > documented issue with the open office build process, but doesn't > prevent the build process from successfully completing. > > $ dmesg > [ 2318.204899] Btrfs loaded > [ 2378.233013] device fsid 2848aff4aec48eec-803ffacdada2ea82 devid 1 > transid 9 /dev/sdc2 > [ 2378.233816] btrfs: use compression > [ 4624.807941] typesconfig[14668]: segfault at 0 ip 08048b50 sp > bfb9def0 error 6 in typesconfig[8048000+2000] > [16452.073382] space info full 36 > [17087.464215] space info full 1 > [39182.628998] allocation failed flags 36, wanted 4096 > [39182.629028] space_info has 122880 free, is full > [39182.629339] block group 29360128 has 179699712 bytes, 135962624 > used 43737088 pinned 0 reserved > [39182.629343] 0 blocks of free space at or bigger than bytes is > [39182.629346] block group 927858688 has 179699712 bytes, 98021376 > used 81678336 pinned 0 reserved > [39182.629349] 0 blocks of free space at or bigger than bytes is > [39182.629351] block group 1826357248 has 179699712 bytes, 64278528 > used 115421184 pinned 0 reserved > [39182.629354] 1 blocks of free space at or bigger than bytes is > [39182.629356] block group 2006056960 has 179699712 bytes, 129122304 > used 50454528 pinned 0 reserved > [39182.629359] 0 blocks of free space at or bigger than bytes is > [39182.629410] [ cut here ] > [39182.629413] kernel BUG at > /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/extent-tree.c:3085! > [39182.629415] invalid opcode: [#1] > [39182.629417] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss > snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device > ipv6 ppdev pcspkr nvidia(P) snd_intel8x0 snd_ac97_codec ac97_bus > snd_pcm snd_timer forcedeth snd snd_page_alloc ohci_hcd i2c_nforce2 > ssb pcmcia i2c_core parport_pc parport nvidia_agp sr_mod > scsi_wait_scan sl811_hcd uhci_hcd ehci_hcd > [39182.629434] > [39182.629438] Pid: 29182, comm: rm Tainted: P > (2.6.27-sabayon-r10 #1) > [39182.629441] EIP: 0060:[] EFLAGS: 00210257 CPU: 0 > [39182.629477] EIP is at __btrfs_reserve_extent+0x2be/0x430 [btrfs] > [39182.629480] EAX: d5e40520 EBX: f64d2780 ECX: e426b4d0 EDX: 0001 > [39182.629483] ESI: d5e40518 EDI: d5e40518 EBP: d5e40520 ESP: ee40f92c > [39182.629485] DS: 007b ES: 007b FS: GS: 0033 SS: 0068 > [39182.629488] Process rm (pid: 29182, ti=ee40e000 task=e426b4d0 > task.ti=ee40e000) > [39182.629490] Stack: f8e6d894 7792 0ab6 > 07b24000 0301e000 > [39182.629495] > 0024 > [39182.629499] e2c4b600 e4978028 f4c8 > 0002 e2c4b600 > [39182.629504] Call Trace: > [39182.629507] [] btrfs_alloc_extent+0x9f/0x140 [btrfs] > [39182.629527] [] btrfs_alloc_free_block+0xce/0x110 [btrfs] > [39182.629545] [] __btrfs_cow_block+0x219/0x870 [btrfs] > [39182.629564] [] verify_parent_transid+0x62/0x1a0 [btrfs] > [39182.629582] [] btrfs_cow_block+0x13c/0x1e0 [btrfs] > [39182.629600] [] btrfs_search_slot+0x1c0/0x7f0 [btrfs] > [39182.629618] [] finish_current_insert+0x523/0x630 [btrfs] > [39182.629636] [] lookup_extent_backref+0x57/0x120 [btrfs] > [39182.629654] [] __btrfs_update_extent_ref+0x1f2/0x3a0 > [btrfs] > [39182.629673] [] btrfs_update_ref+0x2b3/0x3c0 [btrfs] > [39182.629692] [] copy_extent_buffer+0xa9/0x120 [btrfs] > [39182.629714] [] __btrfs_cow_block+0x5c4/0x870 [btrfs] > [39182.629732] [] verify_parent_transid+0x62/0x1a0 [btrfs] > [39182.629750] [] btrfs_cow_block+0x13c/0x1e0 [btrfs] > [39182.629768] [] btrfs_search_slot+0x1c0/0x7f0 [btrfs] > [39182.629785] [] set_extent_buffer_dirty+0xe4/0x160 > [btrfs] > [39182.629806] [] btrfs_del_inode_ref+0x70/0x190 [btrfs] > [39182.629825] [] btrfs_unlink_inode+0xeb/0x2c0 [btrfs] > [39182.629845] [] d_instantiate+0x32/0x50 > [39182.629852] [] start_transaction+0xd6/0x100 [btrfs] > [39182.629871] [] btrfs_unlink+0xb1/0xf0 [btrfs] > [39182.629891] [] vfs_unlink+0x156/0x1f0 > [39182.629897] [] __lookup_hash+0xd5/0x1
Re: [DEBUG PATCH] for anybody who gets a panic due to ENOSPC
I still get a kernel panic with both of your patches installed. When I checked with df the file system is about 65% full. But even if it was full it shouldn't cause a kernel panic. Lee Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234036] we were searching for 24576 bytes, num_bytes 24576, loop 2, allowed alloc 1 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234040] we were searching for 12288 bytes, num_bytes 12288, loop 2, allowed alloc 1 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234045] we were searching for 4096 bytes, num_bytes 4096, loop 2, allowed alloc 1 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234080] allocation failed flags 1, wanted 4096 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234098] space_info has 1580007424 free, is full Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234134] block group 12582912 has 8388608 bytes, 8380416 used 0 pinned 8192 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234172] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234201] block group 424542208 has 790429696 bytes, 790429696 used 0 pinned 0 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234204] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234206] block group 1214971904 has 790429696 bytes, 790421504 used 0 pinned 8192 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234227] 1 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234230] block group 2005401600 has 790429696 bytes, 0 used 0 pinned 0 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234233] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234235] block group 2795831296 has 790429696 bytes, 851968 used 0 pinned 0 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234238] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234240] block group 3586260992 has 790429696 bytes, 789479424 used 0 pinned 950272 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234243] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234284] block group 4376690688 has 790429696 bytes, 786604032 used 0 pinned 3825664 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234287] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234290] block group 5167120384 has 790429696 bytes, 788373504 used 0 pinned 2056192 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234292] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234295] block group 5957550080 has 790429696 bytes, 789020672 used 0 pinned 1409024 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234297] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234302] block group 6747979776 has 752943104 bytes, 721526784 used 0 pinned 31416320 reserved Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234305] 0 blocks of free space at or bigger than bytes is Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234737] [ cut here ] Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234760] kernel BUG at /home/ltrager/btrfs/btrfs-unstable-standalone/extent-tree.c:3088! Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234796] invalid opcode: [#1] SMP Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.234861] Modules linked in: btrfs zlib_deflate libcrc32c ipv6 af_packet bridge stp bnep rfcomm sco l2cap bluetooth ppdev cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_ondemand freq_table cpufreq_conservative wmi video output sbs sbshc pci_slot battery iptable_filter ip_tables x_tables lp evdev psmouse serio_raw pcspkr snd_ens1371 gameport snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi parport_pc snd_seq_midi_event parport snd_seq snd_timer snd_seq_device snd soundcore i2c_piix4 snd_page_alloc container ac button i2c_core shpchp pci_hotplug intel_agp agpgart btrfs mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi sg ata_piix ehci_hcd uhci_hcd pcnet32 mii ata_generic usbcore mptspi mptscsih mptbase scsi_transport_spi libata scsi_mod dock thermal processor fan fbcon tileblit font bitblit softcursor fuse Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235013] Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235084] Pid: 6150, comm: bonnie++ Not tainted (2.6.27-7-generic #1) Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235097] EIP: 0060:[] EFLAGS: 00010257 CPU: 0 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235365] EIP is at __btrfs_reserve_extent+0x3c2/0x480 [btrfs] Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235383] EAX: de255464 EBX: dd4b0300 ECX: EDX: 0001 Nov 17 21:16:50 Intrepid-btrfs kernel: [ 1201.235393] ESI: dd4b032c EDI: de255458 EBP: ddb03a78 ESP: ddb03a04 Nov 17 21:16:50 Intr
Re: [DEBUG PATCH] for anybody who gets a panic due to ENOSPC
Sorry it took so long to reply it took awhile to run this test. It didn't cause a kernel panic for about 8 hours. df says only 623M of the available 7539M are used. Anyway this is with the latest checkin by Chris Mason with the patch you sent me. Also since you said you cann't reproduce this I thought it might be helpful to know the environment I'm doing testing on. I have been running bonnie++ on xubuntu 8.10 fully updated on vmware 6.5. No vmware tools are instead except for the mouse and video xorg drivers which come by default with ubuntu. I have two 8G prealloced virtual drives. The first one is where the xubuntu install is using ext2 for /boot and / and has a 512M swap partition. The second one is where I do all of the btrfs testing. I have it partitioned because I was experimenting with booting with btrfs as root so the partition btrfs is on has 7539M. The VM has 512M of RAM dedicated to it and is using one core of my Intel Core 2 Duo T7500 running at 2.2GHz. Lee ... [ 7133.346381] block_group 5167120384 didn't have what we needed. 790429696 total, 383414272 used, 407011328 pinned, 4096 reserved. Dumping free space [ 7133.346385] 0 blocks of free space at or bigger than bytes is [ 7133.346388] block_group 5957550080 didn't have what we needed. 790429696 total, 1011712 used, 789413888 pinned, 4096 reserved. Dumping free space [ 7133.346391] 0 blocks of free space at or bigger than bytes is [ 7133.346394] block_group 6747979776 didn't have what we needed. 752943104 total, 125444096 used, 627499008 pinned, 0 reserved. Dumping free space [ 7133.346397] 0 blocks of free space at or bigger than bytes is [ 7133.346400] we were searching for 4096 bytes, num_bytes 4096, loop 2, allowed_alloc 1 [ 7133.346403] allocation failed flags 1, wanted 4096 [ 7133.346405] space_info has 1064009728 free, is full [ 7133.346408] block group 12582912 has 8388608 bytes, 516096 used 7872512 pinned 0 reserved [ 7133.346410] 0 blocks of free space at or bigger than bytes is [ 7133.346413] block group 424542208 has 790429696 bytes, 2105344 used 788324352 pinned 0 reserved [ 7133.346416] offset=1214971904, bytes=274759680 [ 7133.346417] 1 blocks of free space at or bigger than bytes is [ 7133.346420] block group 1214971904 has 790429696 bytes, 3964928 used 511705088 pinned 0 reserved [ 7133.346423] 0 blocks of free space at or bigger than bytes is [ 7133.346425] block group 2005401600 has 790429696 bytes, 1490944 used 788938752 pinned 0 reserved [ 7133.346428] offset=2795831296, bytes=789250048 [ 7133.346430] 1 blocks of free space at or bigger than bytes is [ 7133.346432] block group 2795831296 has 790429696 bytes, 1179648 used 0 pinned 0 reserved [ 7133.346435] 0 blocks of free space at or bigger than bytes is [ 7133.346438] block group 3586260992 has 790429696 bytes, 1392640 used 789037056 pinned 0 reserved [ 7133.346440] 0 blocks of free space at or bigger than bytes is [ 7133.346443] block group 4376690688 has 790429696 bytes, 128237568 used 662192128 pinned 0 reserved [ 7133.346445] 0 blocks of free space at or bigger than bytes is [ 7133.346448] block group 5167120384 has 790429696 bytes, 383414272 used 407011328 pinned 4096 reserved [ 7133.346451] 0 blocks of free space at or bigger than bytes is [ 7133.346453] block group 5957550080 has 790429696 bytes, 1011712 used 789413888 pinned 4096 reserved [ 7133.346456] 0 blocks of free space at or bigger than bytes is [ 7133.346458] block group 6747979776 has 752943104 bytes, 125444096 used 627499008 pinned 0 reserved [ 7133.346461] 0 blocks of free space at or bigger than bytes is [ 7133.346490] [ cut here ] [ 7133.346493] kernel BUG at /home/ltrager/btrfs/btrfs-unstable-standalone/extent-tree.c:3101! [ 7133.346499] invalid opcode: [#309] SMP [ 7133.346502] Modules linked in: btrfs zlib_deflate libcrc32c ipv6 af_packet bridge stp rfcomm bnep sco l2cap bluetooth ppdev cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_ondemand freq_table cpufreq_conservative wmi video output sbs sbshc pci_slot battery iptable_filter ip_tables x_tables lp evdev psmouse serio_raw pcspkr snd_ens1371 gameport snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore snd_page_alloc parport_pc parport i2c_piix4 i2c_core container ac intel_agp button shpchp pci_hotplug agpgart ext2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi sg ata_piix uhci_hcd ehci_hcd usbcore ata_generic pcnet32 mii mptspi mptscsih mptbase scsi_transport_spi libata scsi_mod dock thermal processor fan fbcon tileblit font bitblit softcursor fuse [ 7133.346570] [ 7133.346573] Pid: 6575, comm: bonnie++ Tainted: G D (2.6.27-7-generic #1) [ 7133.346576] EIP: 0060:[] EFLAGS: 00210257 CPU: 0 [ 7133.346591] EIP is at __btrfs_reserve_extent+0x36a/0x420 [btrfs] [ 7133.346594] EAX: ddce128c EBX: ddce3cc0 ECX: EDX: 0001 [ 7133.346597]
btrfs thinks its full
Because the last bug I dealt with had so much to do with the disk being full I decided to test and see what happens when I fill up the disk. Unfortunatly the disk thinks its full before it actually is. I have a 7539M btrfs partition and tried to fill it by doing dd if=/dev/zero of=/mnt/btrfs/fill btrfs reports its full after 6408M have been used. dd, ls, and df all confirm this. This uses only 85% of the disk but I can not put one more bit onto the partition. The only thing that shows up in my system logs is "space info full 1." I am using the latest git sources. Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs thinks its full
Is that just finishing btrfs_check_free_space? What would that involve? I haven't done much kernel work but I could give it a try. Lee Josef Bacik wrote: > On Wed, Nov 19, 2008 at 05:24:34PM -0500, Lee Trager wrote: > >> Because the last bug I dealt with had so much to do with the disk being >> full I decided to test and see what happens when I fill up the disk. >> Unfortunatly the disk thinks its full before it actually is. I have a >> 7539M btrfs partition and tried to fill it by doing >> >> dd if=/dev/zero of=/mnt/btrfs/fill >> >> btrfs reports its full after 6408M have been used. dd, ls, and df all >> confirm this. This uses only 85% of the disk but I can not put one more >> bit onto the partition. The only thing that shows up in my system logs >> is "space info full 1." I am using the latest git sources. >> >> > > Yup thats by design :). Theres a 85% full short-circuit in there to keep the > panic unpleasentness from happening since there isn't proper ENOSPC handling. > This will be tossed when the ENOSPC handling is done. Thanks, > > Josef > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Supported Kernels
According to the FAQ at http://btrfs.wiki.kernel.org/index.php/FAQ all kernels downto 2.6.18 should be supported. I have tried to compile against 2.6.18(Debian Kernel), 2.6.24(Debian Kernel), 2.6.26(Debian Kernel), and 2.6.27(Ubuntu Kernel) and I have only been able to compile successfully against 2.6.27. Is btrfs still trying to maintain backwards compatibility with older version or is the focus on finilizing the on disk format? Thanks, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Selective Compression/Encryption
On Tue, Dec 09, 2008 at 05:22:18PM +0100, Christian Hesse wrote: > On Tuesday 09 December 2008, Miguel Figueiredo Mascarenhas Sousa Filipe wrote: > > Hi there, > > > > On Tue, Dec 9, 2008 at 2:59 PM, Lee Trager <[EMAIL PROTECTED]> wrote: > > > Currently compression and I assume if encryption is implemented it is > > > turned on or off during mount. There are however many times when a user > > > may want to select which files/directories they want to compress or > > > encrypt. This will also be helpful when implementing btrfs support in > > > grub for example. We can say the disk can be compressed/encrypted except > > > for /boot so compression/encryption doesn't have to be implemented in > > > grub. > > You could just use an additional partition for /boot that has compression an > encryption disabled... > Yes you could but many distributions are moving away from that such as ubuntu. Also while this works well on desktop and server environments where space is not an issue on embedded systems where free space is measured in kb or mb it starts becomming a huge issue. > > > I was thinking of adding this functionality to the userspace application > > > btrfstune. The way I was thinking of doing this is when btrfstune +c is > > > applied to a directory or file the directory(and all its contents) or > > > file will always be compressed reguardless of how the filesystem is > > > mounted. The opposite would happen when btrfstune -c is used. > > > > > > Would this be a reasonable thing to implement? Any suggestions before I > > > start doing this? > > > > Things like compression or encription should be used at the "volume" level. > > That was what I said some time ago when I asked why encryption support for > btrfs is planned. On a volume level you can use dm-crypt and the fs can > ignore that part completely. > > The answer was that different users on a home partition could use their own > encryption key. That sounds like volume level is out of bet. ;) > It does seem that doing it with volumes would limit user control and add lots of complexity to such a simple task. Miguel, what is your reason for suggesting this be done at the volume level? > > So.. if a user wants a specific set of files or dirs ..they should > > create a mount-point/volume like: > > > > private_vol > > bigarchives_vol > > > > and set those volumes as compressed or encripted volumes > > > > Regarding usability, the best would be for the sub-volume creation > > tool to optionally allow passing encription/compression arguments. > > > > > > and then: > > should mount those volumes somewhere like: ~/Confidential or ~/Archives. > > > > Basically, do it at the directory level (which in btrfs is at the > > sub-volume level). > > File-level granularity is totally unmanageable in the long term. > > > > Kind regards, > > -- > Regards, > Chris Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Selective Compression/Encryption
Would you suggest using the existing chattr/lsattr commands from e2fsprogs for userspace control and just add support at the kernel level? Miguel suggested this be done at the volume level. Do you have any thoughts on that? Thanks, Lee On Tue, Dec 09, 2008 at 11:35:16AM -0500, Chris Mason wrote: > On Tue, 2008-12-09 at 09:59 -0500, Lee Trager wrote: > > Currently compression and I assume if encryption is implemented it is > > turned on or off during mount. There are however many times when a user may > > want to select which files/directories they want to compress or encrypt. > > This will also be helpful when implementing btrfs support in grub for > > example. We can say the disk can be compressed/encrypted except for /boot > > so > > compression/encryption doesn't have to be implemented in grub. > > > > I was thinking of adding this functionality to the userspace application > > btrfstune. The way I was thinking of doing this is when btrfstune +c is > > applied to a directory or file the directory(and all its contents) or > > file will always be compressed reguardless of how the filesystem is > > mounted. The opposite would happen when btrfstune -c is used. > > > > This was my plan, but btrfstune probably isn't the best program to do it > (the ext2 tune program is mostly aimed at the super block level things). > > I think it would be better to make a setattr style program to call the > ioctls. There is already a per file compression flag, and the code > should already be checking it. > > -chris > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Selective Compression/Encryption
On Tue, Dec 09, 2008 at 03:22:29PM -0500, jim owens wrote: > Joshua J. Berry wrote: >> On Tuesday 09 December 2008 08:35:16 Chris Mason wrote: >>> On Tue, 2008-12-09 at 09:59 -0500, Lee Trager wrote: >>>> Currently compression and I assume if encryption is implemented it is >>>> turned on or off during mount. There are however many times when a user >>>> may want to select which files/directories they want to compress or >>>> encrypt. This will also be helpful when implementing btrfs support in >>>> grub for example. We can say the disk can be compressed/encrypted except >>>> for /boot so compression/encryption doesn't have to be implemented in >>>> grub. >>>> >>>> I was thinking of adding this functionality to the userspace application >>>> btrfstune. The way I was thinking of doing this is when btrfstune +c is >>>> applied to a directory or file the directory(and all its contents) or >>>> file will always be compressed reguardless of how the filesystem is >>>> mounted. The opposite would happen when btrfstune -c is used. >>> This was my plan, but btrfstune probably isn't the best program to do it >>> (the ext2 tune program is mostly aimed at the super block level things). >>> >>> I think it would be better to make a setattr style program to call the >>> ioctls. There is already a per file compression flag, and the code >>> should already be checking it. >> >> Is there some reason this can't be done with the existing extended >> attribute facilities? >> >> It seems like xattrs would be preferable to some btrfs-specific >> tunable, as programs like rsync or backup tools would be able to >> preserve (and restore) these bits with no extra work required. > > I had the same thought... that many btrfs per-file tunables would > be better implemented in xattrs (I've read christoph's earlier > objections and hope to get around those issues). I agree that implementing it in the xattr would be best but what userspace tools should be used to control it? lsattr/chattr or our own? lsattr/chattr does seem to do what we want for compression but it really has nothing for encryption. Should we also support attribute(a), no modification(i), etc support in xattrs? > > I have been working on changing the xattr code with the first > step getting it functioning properly when selinux is enabled > so we can see just how costly btrfs xattrs are in actual use. Could you send me to xattr code you are working on or will you be submitting patches soon? Thanks, Lee > > jim -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
If multiple compression schemes are implemented how should the user go about choosing which one they want? Should it be done at kernel time? Or with the userland tools on a per file basis(maybe zlib is the default but a user could say I want this directory to be bzip)? On Mon, Dec 15, 2008 at 11:14:01PM +0100, devz...@web.de wrote: > fantastic feature! > > i`m curious: can btrfs support more than one compression scheme at the same > time, i.e. is compression "pluggable" ? If you look at compression.c, compression.h, and ctree.h you can clearly see that support for multiple compression scheme was in mind. Implmented a new one shouldn't be to hard but you probably want to make the current system a little bit more pluggable and move all the zlib stuff into zlib.c. > > lzo compression coming to my mind, as this is giving real-time compession and > may even speed up disk access. > > compression ratio isn`t too bad, but speed is awesome and doesn`t need as > much cpu as gzip. > In some tests I've run zlib is actually faster then nocompression because of the lesser amount of data that has to transfer to and from the disk. It would be instresting to see how bzip works with this to. > experimental lzo compression in zfs-fuse showed that it could compress tarred > kernel-source with 2.99x compressratio (where gzip gave 3.41x), so maybe lzo > is a better algorithm for realtime filesystem compression... > > regards > roland > > > > From: Chris Mason oracle.com> > Subject: Re: Compressed Filesystem > Newsgroups: gmane.comp.file-systems.btrfs > Date: 2008-10-29 20:08:42 GMT (6 weeks, 5 days, 1 hour and 53 minutes ago) > > On Wed, 2008-10-29 at 12:14 -0600, Anthony Roberts wrote: > > Hi, I have a few questions about this: > > > > > Compression is optional and off by default (mount -o compress to enable > > > it). When enabled, every file is compressed. > > > > Do you know what the CPU load is like with this enabled? > > Now that I've finally pushed the code out, you can try it ;) One part > of the implementation I need to revisit is the place in the code where I > do compression means that most of the time the single threaded pdflush > is the one compressing. > > This doesn't spread the load very well across the cpus. It can be > fixed, but I wanted to get the code out there. > > The decompression does spread across cpus, and I've gotten about 800MB/s > doing decompress and checksumming on a zero filled compressed file. At > the time, the disk was reading 14MB/s. > > > > > Do you know whether data can be compressed at a sufficient rate to still > > saturate the disk on recent-ish AMD/Intel CPUs? > > My recentish intel cpu can compress and checksum at about 120MB/s. > > > > If no, is the effective pre-compression I/O rate still comparable to the > > disk without compression? > > > > It depends on your disks... > > > I'm pretty sure that won't even matter in many cases (eg you're seeking > > too much to care, or you're on a VM with lots of cores but congested > > disks, or you're dealing with media files that it doesn't bother > > compressing, etc), but I'm curious what sort of overhead this adds. :) > > > > Mostly it seems like a good tradeoff, it trades plentiful cores for scarce > > disk resources. > > This varies quite a bit from workload to workload, in some places it'll > make a big difference, but many workloads are seek bound and not > bandwidth bound. > > -chris > > > > Pt! Schon vom neuen WEB.DE MultiMessenger geh?rt? > Der kann`s mit allen: http://www.produkte.web.de/messenger/?did=3123 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
While I agree that the command you send should be possible it wasn't exactly what I was thinking. Currently I am working on a way for the user to individually set which files/directories they want compressed or not. What I was saying is that assuming you are in a mounted btrfs directory you could do something like chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip chattr +c lzo file1 Compress fil1 with lzo chattr -c file2 Uncompress file2 chattr +c none dir3 Uncompress dir3 but leave contents as is If the user did something like mount -o compress,cscheme=zlib /dev/xyz /mntpoint and then chattr +c /mntpoint/dir /mntpoint/dir would default to zlib as would anything else written to the disk. Lee On Tue, Dec 16, 2008 at 12:19:13AM +0100, devz...@web.de wrote: > > If multiple compression schemes are implemented how should the user go > > about choosing which one they want? Should it be done at kernel time? Or > > with the userland tools on a per file basis(maybe zlib is the default > > but a user could say I want this directory to be bzip)? > > yes, why not... > > doing that at mounttime like > > mount -o compress,cscheme=myzip /dev/xyz /mntpoint > > would be a good start > > > > > -Urspr?ngliche Nachricht- > > Von: "Lee Trager" > > Gesendet: 16.12.08 00:07:32 > > An: devz...@web.de > > CC: linux-btrfs@vger.kernel.org > > Betreff: Re: Compressed Filesystem > > > > If multiple compression schemes are implemented how should the user go > > about choosing which one they want? Should it be done at kernel time? Or > > with the userland tools on a per file basis(maybe zlib is the default > > but a user could say I want this directory to be bzip)? > > > > On Mon, Dec 15, 2008 at 11:14:01PM +0100, devz...@web.de wrote: > > > fantastic feature! > > > > > > i`m curious: can btrfs support more than one compression scheme at the > > > same time, i.e. is compression "pluggable" ? > > If you look at compression.c, compression.h, and ctree.h you can clearly > > see that support for multiple compression scheme was in mind. Implmented > > a new one shouldn't be to hard but you probably want to make the current > > system a little bit more pluggable and move all the zlib stuff into > > zlib.c. > > > > > > lzo compression coming to my mind, as this is giving real-time compession > > > and may even speed up disk access. > > > > > > compression ratio isn`t too bad, but speed is awesome and doesn`t need as > > > much cpu as gzip. > > > > > In some tests I've run zlib is actually faster then nocompression > > because of the lesser amount of data that has to transfer to and from > > the disk. It would be instresting to see how bzip works with this to. > > > experimental lzo compression in zfs-fuse showed that it could compress > > > tarred kernel-source with 2.99x compressratio (where gzip gave 3.41x), so > > > maybe lzo is a better algorithm for realtime filesystem compression... > > > > > > regards > > > roland > > > > > > > > > > > > From: Chris Mason oracle.com> > > > Subject: Re: Compressed Filesystem > > > Newsgroups: gmane.comp.file-systems.btrfs > > > Date: 2008-10-29 20:08:42 GMT (6 weeks, 5 days, 1 hour and 53 minutes ago) > > > > > > On Wed, 2008-10-29 at 12:14 -0600, Anthony Roberts wrote: > > > > Hi, I have a few questions about this: > > > > > > > > > Compression is optional and off by default (mount -o compress to > > > > > enable > > > > > it). When enabled, every file is compressed. > > > > > > > > Do you know what the CPU load is like with this enabled? > > > > > > Now that I've finally pushed the code out, you can try it ;) One part > > > of the implementation I need to revisit is the place in the code where I > > > do compression means that most of the time the single threaded pdflush > > > is the one compressing. > > > > > > This doesn't spread the load very well across the cpus. It can be > > > fixed, but I wanted to get the code out there. > > > > > > The decompression does spread across cpus, and I've gotten about 800MB/s > > > doing decompress and checksumming on a zero filled compressed file. At > > > the time, the disk was reading 14MB/s. > > > > > > > > > &g
Re: Compressed Filesystem
I agree that adding more options will add more complexity but it seems the same amount of work in kernel space will have to be done. If we support mutiple compression schemes somewhere the compression scheme used will have to be stored so we know what to use in the future. If we store it on the super block the user will have to choose when they format at which point they may not see the need to use compression. Or they may choose one compression scheme and later want to change to something else. It doesn't make sence to have to reformat your drive just to change compression scheme. This leaves us with storing what the compression scheme is on each inode. We currently store if compression is used on a per inode basis so storing the type wouldn't be a huge leap. Lee On Tue, Dec 16, 2008 at 10:26:10AM -0500, Chris Mason wrote: > On Tue, 2008-12-16 at 10:20 -0500, Lee Trager wrote: > > While I agree that the command you send should be possible it wasn't > > exactly what I was thinking. Currently I am working on a way for the > > user to individually set which files/directories they want compressed or > > not. What I was saying is that assuming you are in a mounted btrfs > > directory you could do something like > > > > chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib > > chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip > > chattr +c lzo file1 Compress fil1 with lzo > > chattr -c file2 Uncompress file2 > > chattr +c none dir3 Uncompress dir3 but leave contents as is > > > > If the user did something like > > mount -o compress,cscheme=zlib /dev/xyz /mntpoint > > and then > > chattr +c /mntpoint/dir > > /mntpoint/dir would default to zlib as would anything else written to > > the disk. > > > > This is one of those places where more options isn't always better. > Every option adds complexity to the filesystem and the testing matrix. > > I'd much rather have just one compression scheme per FS. If people need > a specific compression scheme for a specific file, they can just > compress it in userland. > > -chris > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: compilation problem on last unstable
On Wed, Dec 17, 2008 at 05:43:50PM +, Michele Petrazzo wrote: > Hi, > I just tried to compile the last unstable version, but: > > CC [M] /home/michele/btrfs-unstable-standalone/inode.o > /home/michele/btrfs-unstable-standalone/inode.c: In function > ???btrfs_new_inode???: > /home/michele/btrfs-unstable-standalone/inode.c:3470: error: implicit > declaration of function ???current_fsuid??? > /home/michele/btrfs-unstable-standalone/inode.c:3471: error: implicit > declaration of function ???current_fsgid??? > /home/michele/btrfs-unstable-standalone/inode.c: In function > ???btrfs_cache_create???: > /home/michele/btrfs-unstable-standalone/inode.c:4527: warning: passing > argument > 5 of ???kmem_cache_create??? from incompatible pointer type > /home/michele/btrfs-unstable-standalone/inode.c: At top level: > /home/michele/btrfs-unstable-standalone/inode.c:4966: warning: initialization > from incompatible pointer type > /home/michele/btrfs-unstable-standalone/inode.c:4970: warning: initialization > from incompatible pointer type > /home/michele/btrfs-unstable-standalone/inode.c:5024: warning: initialization > from incompatible pointer type > /home/michele/btrfs-unstable-standalone/inode.c:5030: warning: initialization > from incompatible pointer type > /home/michele/btrfs-unstable-standalone/inode.c:5040: warning: initialization > from incompatible pointer type > make[2]: *** [/home/michele/btrfs-unstable-standalone/inode.o] Error 1 > make[1]: *** [_module_/home/michele/btrfs-unstable-standalone] Error 2 > make[1]: Leaving directory `/usr/src/linux-headers-2.6.26-1-686' > make: *** [all] Error 2 > michele:~/btrfs-unstable-standalone$ > > > michele:~/btrfs-unstable-standalone$ uname -r > 2.6.26-1-686 > > from debian Currently btrfs only compiles on 2.6.27 and above although support all the way back to 2.6.18 is planned. I'm currently using Ubuntu 8.10 for all btrfs testing. > > Thanks, > Michele > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs-progs: btrfs file system size should be bigger then 256m
This has been bothering me for some time. Why does btrfs need to have a
disk greater then 256M? I could see a much smaller limit, say 16M but
why so much? The file system itself does not need that much space for
its own use.
Thanks,
Lee
Shen Feng wrote:
According to btrfs_prepare_device, btrfs file sysstem size
should be bigger then 256m.
If mkfs.btrfs specifies the file system size samaller then
that, mkfs.btrfs should report error.
Signed-off-by: Shen Feng
---
mkfs.c |4
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/mkfs.c b/mkfs.c
index be93aaa..447e8d7 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -377,6 +377,10 @@ int main(int ac, char **av)
break;
case 'b':
block_count = parse_size(optarg);
+ if (block_count < 256*1024*1024) {
+ fprintf(stderr, "File system size is too
small\n");
+ exit(1);
+ }
zero_end = 0;
break;
default:
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Backporting to 2.6.27 and below
A couple of weeks ago it was mentioned that btrfs in the stand alone tree would be patched to support 2.6.27 with older versions coming after. Has anyone actually done these patches? I'd like to get btrfs running on 2.6.26(Debian Lenny kernel) and if no one has done any code towards it I'd be happy to do it. Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Backport to 2.6.27 and 2.6.26
This patch will allow btrfs-unstable-standalone to compile cleanly
against 2.6.27, 2.6.26, and possibly older(I havn't tested older then
26).
Signed-off-by: Lee Trager
---
compat.h | 24
export.h | 28
extent-tree.c |4
extent_io.c | 12
file.c|4
inode.c | 16 +++-
ioctl.c |4
7 files changed, 91 insertions(+), 1 deletions(-)
diff --git a/compat.h b/compat.h
index 7c4503e..eedf14c 100644
--- a/compat.h
+++ b/compat.h
@@ -4,4 +4,28 @@
#define btrfs_drop_nlink(inode) drop_nlink(inode)
#define btrfs_inc_nlink(inode) inc_nlink(inode)
+#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 27)
+static inline struct dentry *d_obtain_alias(struct inode *inode)
+{
+ struct dentry *d;
+
+ if (!inode)
+ return NULL;
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ d = d_alloc_anon(inode);
+ if (!d)
+ iput(inode);
+ return d;
+}
+#endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 28)
+#define __pagevec_lru_add_file __pagevec_lru_add
+#define open_bdev_exclusive open_bdev_excl
+#define close_bdev_exclusive(bdev, mode) close_bdev_excl(bdev)
+typedef unsigned __bitwise__ fmode_t;
+#endif
+
#endif /* _COMPAT_H_ */
diff --git a/export.h b/export.h
index 074348a..63b6a20 100644
--- a/export.h
+++ b/export.h
@@ -16,4 +16,32 @@ struct btrfs_fid {
u64 parent_root_objectid;
} __attribute__ ((packed));
+// BTRFS needs the btrfs from fid_type which was added in 2.6.27
+// All I did was copy the btrfs defse which was found in
+// linux-2.6.27/include/linux/exportfs.h
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
+enum btrfs_fid_type {
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number.
+*/
+ FILEID_BTRFS_WITHOUT_PARENT = 0x4d,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation.
+*/
+ FILEID_BTRFS_WITH_PARENT = 0x4e,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation,
+* 64 bit parent root object ID.
+*/
+ FILEID_BTRFS_WITH_PARENT_ROOT = 0x4f,
+};
+#endif
+
#endif
diff --git a/extent-tree.c b/extent-tree.c
index 293da65..2113f5c 100644
--- a/extent-tree.c
+++ b/extent-tree.c
@@ -869,7 +869,11 @@ static noinline int remove_extent_backref(struct
btrfs_trans_handle *trans,
static void btrfs_issue_discard(struct block_device *bdev,
u64 start, u64 len)
{
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 28)
blkdev_issue_discard(bdev, start >> 9, len >> 9, GFP_KERNEL);
+#else
+ blkdev_issue_discard(bdev, start >> 9, len >> 9);
+#endif
}
#endif
diff --git a/extent_io.c b/extent_io.c
index e086d40..072165c 100644
--- a/extent_io.c
+++ b/extent_io.c
@@ -3078,13 +3078,21 @@ int clear_extent_buffer_dirty(struct extent_io_tree
*tree,
}
}
clear_page_dirty_for_io(page);
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_lock_irq(&page->mapping->tree_lock);
+#else
+ read_lock_irq(&page->mapping->tree_lock);
+#endif
if (!PageDirty(page)) {
radix_tree_tag_clear(&page->mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_unlock_irq(&page->mapping->tree_lock);
+#else
+ read_unlock_irq(&page->mapping->tree_lock);
+#endif
unlock_page(page);
}
return 0;
@@ -3262,7 +3270,11 @@ int read_extent_buffer_pages(struct extent_io_tree *tree,
for (i = start_i; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!wait) {
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
if (!trylock_page(page))
+#else
+ if(!TestSetPageLocked(page))
+#endif
goto unlock_exit;
} else {
lock_page(page);
diff --git a/file.c b/file.c
index 9026833..f5377b6 100644
--- a/file.c
+++ b/file.c
@@ -1043,7 +1043,11 @@ static ssize_t btrfs_file_write(struct file *file, const
char __user *buf,
if (count == 0)
goto out_nolock;
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
err = file_remove_suid(file);
+#else
+ err = remove_suid(fdentry(file));
+#endif
if (err)
goto out_nolock;
Status of btrfs-unstable-standalone?
I've noticed that the btrfs-unstable-standalone repository hasn't been updated in over three weeks. It seems all active development is taking place on btrfs-unstable. What is happening to the btfs-unstable-standalone repository? Will all future development only be on btrfs-unstable or is there some other reason btrfs-unstable-standalone hasn't been updated? Thanks, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Updated Backport to 2.6.27 and 2.6.26
Since for the past three weeks all new patches have been submitted only
to btrfs-unstable I have updated my backport patch for btrfs-unstable.
This patch only works with 2.6.27 and 2.6.26(thanks Michele for testing
it for me)
Lee
diff -Naur btrfs-old/async-thread.c btrfs/async-thread.c
--- btrfs-old/async-thread.c2009-02-11 16:24:09.0 -0500
+++ btrfs/async-thread.c2009-02-11 15:47:44.0 -0500
@@ -20,7 +20,10 @@
#include
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 29)
#include
+#endif
#include "async-thread.h"
#define WORK_QUEUED_BIT 0
diff -Naur btrfs-old/compat.h btrfs/compat.h
--- btrfs-old/compat.h 2009-02-11 16:24:09.0 -0500
+++ btrfs/compat.h 2009-02-11 15:24:52.0 -0500
@@ -1,7 +1,33 @@
#ifndef _COMPAT_H_
#define _COMPAT_H_
+#include
+
#define btrfs_drop_nlink(inode) drop_nlink(inode)
#define btrfs_inc_nlink(inode) inc_nlink(inode)
+#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 27)
+static inline struct dentry *d_obtain_alias(struct inode *inode)
+{
+ struct dentry *d;
+
+ if (!inode)
+ return NULL;
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ d = d_alloc_anon(inode);
+ if (!d)
+ iput(inode);
+ return d;
+}
+#endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 28)
+#define __pagevec_lru_add_file __pagevec_lru_add
+#define open_bdev_exclusive open_bdev_excl
+#define close_bdev_exclusive(bdev, mode) close_bdev_excl(bdev)
+typedef unsigned __bitwise__ fmode_t;
+#endif
+
#endif /* _COMPAT_H_ */
diff -Naur btrfs-old/export.h btrfs/export.h
--- btrfs-old/export.h 2009-02-11 16:24:11.0 -0500
+++ btrfs/export.h 2009-02-11 15:18:16.0 -0500
@@ -16,4 +16,32 @@
u64 parent_root_objectid;
} __attribute__ ((packed));
+// BTRFS needs the btrfs from fid_type which was added in 2.6.27
+// All I did was copy the btrfs defse which was found in
+// linux-2.6.27/include/linux/exportfs.h
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
+enum btrfs_fid_type {
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number.
+*/
+ FILEID_BTRFS_WITHOUT_PARENT = 0x4d,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation.
+*/
+ FILEID_BTRFS_WITH_PARENT = 0x4e,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation,
+* 64 bit parent root object ID.
+*/
+ FILEID_BTRFS_WITH_PARENT_ROOT = 0x4f,
+};
+#endif
+
#endif
diff -Naur btrfs-old/extent_io.c btrfs/extent_io.c
--- btrfs-old/extent_io.c 2009-02-11 16:24:11.0 -0500
+++ btrfs/extent_io.c 2009-02-11 15:44:02.0 -0500
@@ -2849,6 +2849,7 @@
return sector;
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 29)
int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
__u64 start, __u64 len, get_extent_t *get_extent)
{
@@ -2940,6 +2941,7 @@
GFP_NOFS);
return ret;
}
+#endif
static inline struct page *extent_buffer_page(struct extent_buffer *eb,
unsigned long i)
@@ -3165,13 +3167,21 @@
}
}
clear_page_dirty_for_io(page);
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_lock_irq(&page->mapping->tree_lock);
+#else
+ read_lock_irq(&page->mapping->tree_lock);
+#endif
if (!PageDirty(page)) {
radix_tree_tag_clear(&page->mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_unlock_irq(&page->mapping->tree_lock);
+#else
+ read_unlock_irq(&page->mapping->tree_lock);
+#endif
unlock_page(page);
}
return 0;
@@ -3349,7 +3359,11 @@
for (i = start_i; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!wait) {
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
if (!trylock_page(page))
+#else
+ if(!TestSetPageLocked(page))
+#endif
goto unlock_exit;
} else {
lock_page(page);
diff -Naur btrfs-old/extent_io.h btrfs/extent_io.h
--- btrfs-old/extent_io.h 2009-02-11 16:24:11.0 -0500
+++ btrfs/extent_io.h 2009-02-11 15:44:41.0 -0500
@@ -2,6 +2,7 @@
#define __EXTENTIO__
#include
+#include
/* bits for the extent state */
#define EXTENT_DIRTY 1
@@ -205,8 +206,10 @@
btrfs lockup after mounting for a second time on 2.6.26
While running a few tests with the btrfs sources pulled from btrfs-unstable patched with my patch to compile under 2.6.26 I encountered a very weird problem. Everything works fine the first time I mount the file system (either actual disk or loop back). When I unmounted the file system and mounted it again(I'm not doing mount -o remount ...) btrfs is completely unusable. When I tried to view some of the data which I previously put on the btrfs partition by doing a simple ls /mnt/btrfs in bash nothing happens. ls shows nothing, just hangs, and I am unable to kill ls. The same thing happens when I try to copy a file from my ext3 partition onto the btrfs partition after mounting it for a second time. The rest of the system is completely usable and the only things that are effects are applications which are trying to read/write from the btrfs file system. There are no kernel opps or any other errors messages in any of my logs. This happens if I have compression on or not. I'd be happy to fix this issue but I don't have a clue about where to start looking for what is wrong. Could someone please help me? Thanks, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs lockup after mounting for a second time on 2.6.26
Ok here it is [ 227.637684] SysRq : Show Blocked State [ 227.639369] taskPC stack pid father [ 227.639369] lsD f784b9d4 0 3101 3003 [ 227.639369]f74e4da0 0082 d950 f784b9d4 d950 f74e4f2c c1815fa0 0001 [ 227.639369]f784b9cc 9728 c18031b4 c013604c 00ff [ 227.639369]c1815fa0 0145b000 c18031b4 c015688a c02b8498 f3459bb4 f3459bb4 c01568bd [ 227.639369] Call Trace: [ 227.639369] [] getnstimeofday+0x37/0xbc [ 227.639369] [] sync_page+0x0/0x36 [ 227.639369] [] io_schedule+0x49/0x80 [ 227.639369] [] sync_page+0x33/0x36 [ 227.639369] [] __wait_on_bit_lock+0x2a/0x52 [ 227.639369] [] __lock_page+0x4e/0x54 [ 227.639369] [] wake_bit_function+0x0/0x3c [ 227.639369] [] read_extent_buffer_pages+0x133/0x2f1 [btrfs] [ 227.639369] [] kmap_atomic_prot+0x9d/0xcc [ 227.639369] [] btree_read_extent_buffer_pages+0x39/0x8c [btrfs] [ 227.639369] [] btree_get_extent+0x0/0x173 [btrfs] [ 227.639369] [] read_tree_block+0x29/0x4c [btrfs] [ 227.639369] [] read_node_slot+0xa4/0xb2 [btrfs] [ 227.639369] [] btrfs_next_leaf+0x16f/0x27d [btrfs] [ 227.639369] [] cache_block_group+0xe9/0x2a0 [btrfs] [ 227.639369] [] alloc_inode+0x12e/0x186 [ 227.639369] [] find_free_extent+0x32a/0x7b2 [btrfs] [ 227.639369] [] __btrfs_reserve_extent+0x1d5/0x370 [btrfs] [ 227.639369] [] btrfs_reserve_extent+0x3f/0x61 [btrfs] [ 227.639369] [] btrfs_search_slot+0x257/0x79c [btrfs] [ 227.639369] [] kunmap_atomic+0x67/0x8a [ 227.639369] [] btrfs_lookup_inode+0x27/0x88 [btrfs] [ 227.639369] [] btrfs_update_inode+0x3d/0xae [btrfs] [ 227.639369] [] btrfs_dirty_inode+0x3d/0x4a [btrfs] [ 227.639369] [] __mark_inode_dirty+0x21/0x12a [ 227.639369] [] touch_atime+0xc7/0xd1 [ 227.639369] [] vfs_readdir+0x75/0x8c [ 227.639369] [] filldir64+0x0/0xc5 [ 227.639369] [] sys_getdents64+0x63/0xa5 [ 227.639369] [] sysenter_past_esp+0x78/0xb1 [ 227.639369] === [ 227.639369] Sched Debug Version: v0.07, 2.6.26-1-686 #1 [ 227.639369] now at 210726.291061 msecs [ 227.639369] .sysctl_sched_latency: 40.00 [ 227.639369] .sysctl_sched_min_granularity: 8.00 [ 227.639369] .sysctl_sched_wakeup_granularity : 20.00 [ 227.639369] .sysctl_sched_child_runs_first : 0.01 [ 227.639369] .sysctl_sched_features : 895 [ 227.639369] [ 227.639369] cpu#0, 2200.225 MHz [ 227.639369] .nr_running: 2 [ 227.639369] .load : 2048 [ 227.639369] .nr_switches : 34074 On Thu, Feb 12, 2009 at 09:51:12AM -0500, Chris Mason wrote: > On Thu, 2009-02-12 at 01:31 -0500, Lee Trager wrote: > > While running a few tests with the btrfs sources pulled from > > btrfs-unstable patched with my patch to compile under 2.6.26 I > > encountered a very weird problem. Everything works fine the first time I > > mount the file system (either actual disk or loop back). When I > > unmounted the file system and mounted it again(I'm not doing mount -o > > remount ...) btrfs is completely unusable. When I tried to view some of > > the data which I previously put on the btrfs partition by doing a simple > > ls /mnt/btrfs in bash nothing happens. ls shows nothing, just hangs, and > > I am unable to kill ls. The same thing happens when I try to copy a file > > from my ext3 partition onto the btrfs partition after mounting it for a > > second time. The rest of the system is completely usable and the only > > things that are effects are applications which are trying to read/write > > from the btrfs file system. There are no kernel opps or any other errors > > messages in any of my logs. This happens if I have compression on or > > not. I'd be happy to fix this issue but I don't have a clue about where > > to start looking for what is wrong. > > > > Could someone please help me? > > Thanks for working on this. The first step is to figure out what ls is > doing, and my guess is trying to read block groups. Do a sysrq-w while > it is hung and send the results along. > > -chris > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs lockup after mounting for a second time on 2.6.26
_balance : 4294.945104 [ 227.639369] .curr->pid : 2411 [ 227.639369] .clock : 210728.301926 [ 227.639369] .cpu_load[0] : 2048 [ 227.639369] .cpu_load[1] : 1024 [ 227.639369] .cpu_load[2] : 519 [ 227.639369] .cpu_load[3] : 321 [ 227.639369] .cpu_load[4] : 275 [ 227.639369] [ 227.639369] cfs_rq[1]: [ 227.639369] .exec_clock: 0.00 [ 227.639369] .MIN_vruntime : 34787.608965 [ 227.639369] .min_vruntime : 34826.431444 [ 227.639369] .max_vruntime : 34787.608965 [ 227.639369] .spread: 0.00 [ 227.639369] .spread0 : -14483.365823 [ 227.639369] .nr_running: 2 [ 227.639369] .load : 2048 [ 227.639369] .nr_spread_over: 0 [ 227.639369] [ 227.639369] runnable tasks: [ 227.639369] task PID tree-key switches prio exec-runtime sum-execsum-sleep [ 227.639369] -- [ 227.639369] Rsyslogd 2411 34791.757624 811 120 0 0 0.00 0.00 0.00 / [ 227.639369]klogd 2420 34787.608965 605 120 0 0 0.00 0.00 0.00 / [ 227.639369] On Thu, Feb 12, 2009 at 09:51:12AM -0500, Chris Mason wrote: > On Thu, 2009-02-12 at 01:31 -0500, Lee Trager wrote: > > While running a few tests with the btrfs sources pulled from > > btrfs-unstable patched with my patch to compile under 2.6.26 I > > encountered a very weird problem. Everything works fine the first time I > > mount the file system (either actual disk or loop back). When I > > unmounted the file system and mounted it again(I'm not doing mount -o > > remount ...) btrfs is completely unusable. When I tried to view some of > > the data which I previously put on the btrfs partition by doing a simple > > ls /mnt/btrfs in bash nothing happens. ls shows nothing, just hangs, and > > I am unable to kill ls. The same thing happens when I try to copy a file > > from my ext3 partition onto the btrfs partition after mounting it for a > > second time. The rest of the system is completely usable and the only > > things that are effects are applications which are trying to read/write > > from the btrfs file system. There are no kernel opps or any other errors > > messages in any of my logs. This happens if I have compression on or > > not. I'd be happy to fix this issue but I don't have a clue about where > > to start looking for what is wrong. > > > > Could someone please help me? > > Thanks for working on this. The first step is to figure out what ls is > doing, and my guess is trying to read block groups. Do a sysrq-w while > it is hung and send the results along. > > -chris > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs lockup after mounting for a second time on 2.6.26
I took your suggestions from the phone conference today and tired them out on my system. It seems that even though 2.6.26 can not mount a btrfs partition a second time 2.6.27 has no problem doing this. I created the the btrfs parition on 2.6.26, copied data to it on 2.6.26, I tried to mount the btrfs for a second time on 2.6.26 and I experienced the bug. I then tried mounting the btrfs parition on 2.6.27 and it worked fine. When I run vmstat it seems no data is comming from the btrfs partition on 2.6.26. While I get some burts of data every few seconds it doesn't seem I am getting anything from btrfs. Lee P.S Thanks for helping me get this working and answering my questions! On Thu, Feb 12, 2009 at 09:51:12AM -0500, Chris Mason wrote: > On Thu, 2009-02-12 at 01:31 -0500, Lee Trager wrote: > > While running a few tests with the btrfs sources pulled from > > btrfs-unstable patched with my patch to compile under 2.6.26 I > > encountered a very weird problem. Everything works fine the first time I > > mount the file system (either actual disk or loop back). When I > > unmounted the file system and mounted it again(I'm not doing mount -o > > remount ...) btrfs is completely unusable. When I tried to view some of > > the data which I previously put on the btrfs partition by doing a simple > > ls /mnt/btrfs in bash nothing happens. ls shows nothing, just hangs, and > > I am unable to kill ls. The same thing happens when I try to copy a file > > from my ext3 partition onto the btrfs partition after mounting it for a > > second time. The rest of the system is completely usable and the only > > things that are effects are applications which are trying to read/write > > from the btrfs file system. There are no kernel opps or any other errors > > messages in any of my logs. This happens if I have compression on or > > not. I'd be happy to fix this issue but I don't have a clue about where > > to start looking for what is wrong. > > > > Could someone please help me? > > Thanks for working on this. The first step is to figure out what ls is > doing, and my guess is trying to read block groups. Do a sysrq-w while > it is hung and send the results along. > > -chris > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch
I have updated my backport patch to apply cleanly to the experimental
btrfs branch as well as fix a couple of mistakes I had. I am still
having lockup issues on 2.6.26. My best guess right now is that there is
some sort of race condition going on. I'm trying to track it down but
I'm not having any luck. If someone could please help me that would be
great.
Thanks,
Lee
diff -Naur btrfs-orig/async-thread.c btrfs/async-thread.c
--- btrfs-orig/async-thread.c 2009-02-20 12:10:13.0 -0500
+++ btrfs/async-thread.c2009-02-20 12:13:20.0 -0500
@@ -20,7 +20,10 @@
#include
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
#include
+#endif
#include "async-thread.h"
#define WORK_QUEUED_BIT 0
diff -Naur btrfs-orig/compat.h btrfs/compat.h
--- btrfs-orig/compat.h 2009-02-20 12:10:14.0 -0500
+++ btrfs/compat.h 2009-02-20 12:11:30.0 -0500
@@ -1,7 +1,33 @@
#ifndef _COMPAT_H_
#define _COMPAT_H_
+#include
+
#define btrfs_drop_nlink(inode) drop_nlink(inode)
#define btrfs_inc_nlink(inode) inc_nlink(inode)
+#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 27)
+static inline struct dentry *d_obtain_alias(struct inode *inode)
+{
+ struct dentry *d;
+
+ if (!inode)
+ return NULL;
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ d = d_alloc_anon(inode);
+ if (!d)
+ iput(inode);
+ return d;
+}
+#endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 28)
+#define __pagevec_lru_add_file __pagevec_lru_add
+#define open_bdev_exclusive open_bdev_excl
+#define close_bdev_exclusive(bdev, mode) close_bdev_excl(bdev)
+typedef unsigned __bitwise__ fmode_t;
+#endif
+
#endif /* _COMPAT_H_ */
diff -Naur btrfs-orig/delayed-ref.c btrfs/delayed-ref.c
--- btrfs-orig/delayed-ref.c2009-02-20 12:10:22.0 -0500
+++ btrfs/delayed-ref.c 2009-02-20 12:16:21.0 -0500
@@ -18,7 +18,10 @@
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
#include
+#endif
#include "ctree.h"
#include "delayed-ref.h"
#include "transaction.h"
diff -Naur btrfs-orig/delayed-unlink.c btrfs/delayed-unlink.c
--- btrfs-orig/delayed-unlink.c 2009-02-20 12:10:22.0 -0500
+++ btrfs/delayed-unlink.c 2009-02-20 12:15:01.0 -0500
@@ -18,7 +18,10 @@
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
#include
+#endif
#include "ctree.h"
#include "delayed-unlink.h"
#include "transaction.h"
diff -Naur btrfs-orig/export.h btrfs/export.h
--- btrfs-orig/export.h 2009-02-20 12:10:15.0 -0500
+++ btrfs/export.h 2009-02-20 12:11:30.0 -0500
@@ -16,4 +16,32 @@
u64 parent_root_objectid;
} __attribute__ ((packed));
+// BTRFS needs the btrfs from fid_type which was added in 2.6.27
+// All I did was copy the btrfs defse which was found in
+// linux-2.6.27/include/linux/exportfs.h
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
+enum btrfs_fid_type {
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number.
+*/
+ FILEID_BTRFS_WITHOUT_PARENT = 0x4d,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation.
+*/
+ FILEID_BTRFS_WITH_PARENT = 0x4e,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation,
+* 64 bit parent root object ID.
+*/
+ FILEID_BTRFS_WITH_PARENT_ROOT = 0x4f,
+};
+#endif
+
#endif
diff -Naur btrfs-orig/extent_io.c btrfs/extent_io.c
--- btrfs-orig/extent_io.c 2009-02-20 12:10:20.0 -0500
+++ btrfs/extent_io.c 2009-02-20 12:11:30.0 -0500
@@ -2847,6 +2847,7 @@
return sector;
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 29)
int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
__u64 start, __u64 len, get_extent_t *get_extent)
{
@@ -2938,6 +2939,7 @@
GFP_NOFS);
return ret;
}
+#endif
static inline struct page *extent_buffer_page(struct extent_buffer *eb,
unsigned long i)
@@ -3163,13 +3165,21 @@
}
}
clear_page_dirty_for_io(page);
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_lock_irq(&page->mapping->tree_lock);
+#else
+ read_lock_irq(&page->mapping->tree_lock);
+#endif
if (!PageDirty(page)) {
radix_tree_tag_clear(&page->mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_unlock_irq(&page->mapping-
Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch
On Fri, Feb 20, 2009 at 01:25:19PM -0500, jim owens wrote: > lee, > > A couple of thoughts about your .26 lockup: > > - I assume you are on the same hardware with 27. I am. > > - Are you using a module or builtin btrfs (I build it in). > As a module > - It looks like you are using vmware... when I'm doing kernel > stuff I want to be on the iron, not trusting virtual machine code. > (people who use vmware tell me they have to rebuild vmware stuff >even when they apply kernel patch distro updates) I tried running the same code(on 27 and 26) on a physical machine running Debian Lenny and ran into the same problem. So I don't think its vmware. > > - Compare your .26 and .27 configs, changes to default io sched > or vm etc. can hurt or help you. I used the standard Debian Lenny 2.6.26 kernel for testing. For 2.6.27 I tested against the config for lenny found in the Debian svn. The only differences seems to be the new features. > > jim The more and more I look at this problem the more I tend to think that the issue is because of some change in the way the VFS or something interacts with the file system. Does anyone know of any big changes? Why is the inode being marked dirty? Is there some kind of read error. I'm completly lost in solving this problem. Thanks for all your help, Lee -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch
I ran a few tests that Jim suggested and found that btrfs works fine on 2.6.26 as long as there are only 23 or less files on the file system. Anymore and I experience the lockup. Jim and I will be working to find a solution but if anyone else has any clues that would be greatly appreciated. Lee On Tue, Feb 24, 2009 at 11:24:06AM -0500, jim owens wrote: > Lee Trager wrote: >> The more and more I look at this problem the more I tend to think that >> the issue is because of some change in the way the VFS or something >> interacts with the file system. Does anyone know of any big changes? Why >> is the inode being marked dirty? Is there some kind of read error. I'm >> completly lost in solving this problem. > > Being a filesystem guy, I always try blaming vm or drivers :) > > Until someone with real experience gives us the answer, > I'll work with you off the mailing list to try to narrow > down why this is happening. > > jim -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: questions about GRUB and BTRFS
I'm not sure when we should start developing BTRFS support for GRUB but I do agree that it will be very difficult to support all the features of BTRFS. As far as I know GRUB does not support LVM and only supports RAID1. Doing this shouldn't be that hard to do, in fact it should be easier to do with BTRFS since the filesystem is aware that it has RAID. The big problem with BTRFS GRUB support is all the advanced features BTRFS has to offer. I plan to eventually write a way for BTRFS to force on or off compression/encryption. One of the main reasons for this is so the GRUB BTRFS implementation doesn't have to, at least initially, support compression/encryption. All that would be required is that the data is left raw. Other features like snapshots would be really great to be able to access from the boot loader but currently would be very difficult to implement. It may even be worth skipping GRUB support and just adding support for BTRFS in GRUB2 so we can have all BTRFS's features. If you would like to start working on this I would also talk to the GRUB guys first. Lee Dmitri Nikulin wrote: > On Wed, Feb 25, 2009 at 12:32 PM, Anthony Roberts > wrote: > >> Hi, >> >> A quick googling turns up posts that GRUB support for BTRFS is planned. My >> curiosity is more towards how this will be managed, because the way this is >> currently implemented with software RAID/LVM is quite haphazard. I >> therefore have some questions about GRUB + BTRFS: >> > > IANAGD (I Am Not A GRUB Developer), but I'll post some intuitive respones. > > >> -With GRUB booting, it's easy to think of awkward use cases and limitations >> unless it's capable of discovering BTRFS instances, and can boot by >> specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this >> planned "eventually"? >> > > I don't know how much filesystem code can be crammed into the > pre-/boot parts of GRUB, but I doubt it's enough to support btrfs' > advanced features like object-level striping. > > For comparison with how the two major ZFS operating systems support root on > ZFS: > > *Solaris (Open, Nexenta, etc.) support booting from ZFS using GRUB, > but ONLY plain or mirrored, not striped or raid-z. Not sure about > linear, if the kernel is installed on anything but the first vdev. > > FreeBSD unofficially supports / on ZFS very well, but you still need a > /boot to let the bootloader find the kernel and modules. However the > kernel itself can be given a ZFS pool and path such as > "zfs:pool/freebsd/root" and it will find all of the ZFS metadata it > needs on disk blocks and the small cache in /boot. However in return > for this /boot you get the ability to boot right off RAID-Z or > whatever you like, because it's using the kernel with full driver and > filesystem code instead of very limited bootloader code. > > >> -Might it be possible to tweak the userspace component of GRUB to install >> the bootloader to every member device? This seems necessary for reliable >> booting and rebuilding after a dead disk. >> > > Even if you couldn't tweak grub, device-mapper already has an easy way > to mirror just the boot blocks per disk. However GRUB would get > confused since the virtual device does not map to a BIOS boot device. > Legacy BIOS booting is a pain that way. You may as well just write a > shell script to automatically invoke grub-install for each device > individually. > > >> -64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 + >> 1.5. Might this allow bootable BTRFS without paritions being used at all? >> The space used for partitioning is negligible, however we're on the cusp of >> disks that are too big to partition with MBR, and GPT booting doesn't seem >> well supported yet. >> > > As far as I know, we don't even have a way to boot straight off LVM > (because GRUB doesn't support it, and for a kernel and initrd you need > a supported partition), and btrfs would only be more difficult. > > >> There's obviously no point in getting worked up about this before >> production ready support is available in the first place. :) However, I am >> curious about what sort of implementation is planned. >> > > Well before production ready support is there, people will already > want to test btrfs as their / (which should be automagic like for > FreeBSD ZFS) and /boot (because they're difficult that way). Long > before reiser4 was even proposed for mainline merge, it already had > GRUB support. Enthusiasts will always believe that even /boot should > be fortified with COW, checksums and snapshotting :) > > Especially if btrfs is intended to be the "next default Linux > filesystem" as quoted in many places, it will need /boot support in > some form. I'll personally keep an ext3 /boot for a long time just > because recovery is easier that way. > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo in
Re: btrfs: warn_slowpath in clean_tree_block and others
Mitch, I haven't seen any problems using BTRFS and my patch on 2.6.28 or 2.6.27, what are you doing to cause this error? Are you using the latest sources from btrfs-unstable? Lee Mitch Harder (aka DontPanic) wrote: > I have also been getting similar warnings filling up my logs. > > However, in my case, I have been experimenting with back-porting btrfs > to a 2.6.28 kernel. So I've been waiting for the back-porting efforts > to get a little further along. > > But I thought I'd respond in case this information helps. > > Here's an example of the warnings I've been seeing: > > [80577.151167] [ cut here ] > [80577.151169] WARNING: at > /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/disk-io.c:860 > clean_tree_block+0xa4/0xb0 [btrfs]() > [80577.151172] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss > snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device > ipv6 ppdev snd_intel8x0 snd_ac97_codec parport_pc nvidia(P) ac97_bus > snd_pcm snd_timer ohci_hcd ssb shpchp pci_hotplug pcmcia i2c_nforce2 > snd forcedeth sr_mod pcspkr parport i2c_core snd_page_alloc nvidia_agp > sl811_hcd pcmcia_core uhci_hcd ehci_hcd > [80577.151190] Pid: 11503, comm: cp Tainted: PW 2.6.28-sabayon-r10 #1 > [80577.151192] Call Trace: > [80577.151195] [] warn_on_slowpath+0x5f/0x90 > [80577.151203] [] rb_insert_color+0x77/0xe0 > [80577.151221] [] alloc_extent_buffer+0x1fe/0x300 [btrfs] > [80577.151238] [] clean_tree_block+0xa4/0xb0 [btrfs] > [80577.151253] [] btrfs_init_new_buffer+0x7d/0x130 [btrfs] > [80577.151269] [] btrfs_alloc_free_block+0x104/0x110 [btrfs] > [80577.151285] [] __btrfs_cow_block+0x22a/0x8b0 [btrfs] > [80577.151300] [] generic_bin_search+0x162/0x1c0 [btrfs] > [80577.151315] [] btrfs_cow_block+0x156/0x200 [btrfs] > [80577.151330] [] btrfs_search_slot+0x1a7/0x910 [btrfs] > [80577.151333] [] irq_exit+0x27/0x60 > [80577.151336] [] do_IRQ+0x6b/0x80 > [80577.151354] [] read_extent_buffer+0xd5/0x170 [btrfs] > [80577.151369] [] btrfs_insert_empty_items+0x6d/0x410 [btrfs] > [80577.151385] [] btrfs_find_block_group+0xff/0x1a0 [btrfs] > [80577.151402] [] btrfs_new_inode+0x18d/0x360 [btrfs] > [80577.151420] [] btrfs_create+0x189/0x2a0 [btrfs] > [80577.151423] [] security_capable+0x9/0x10 > [80577.151427] [] vfs_create+0xcd/0x160 > [80577.151430] [] do_filp_open+0x5af/0x7d0 > [80577.151433] [] cp_new_stat64+0xf9/0x110 > [80577.151436] [] do_sys_open+0x4e/0xe0 > [80577.151439] [] sys_open+0x2c/0x40 > [80577.151442] [] sysenter_do_call+0x12/0x21 > [80577.151444] ---[ end trace 79cdc48bc88dedf7 ]--- > > > On Tue, Feb 24, 2009 at 5:02 PM, Hugo Mills wrote: > >> This is essentially a repost of a mail I made last week, to which I >> didn't get a reply. >> >> I'm getting huge numbers of kernel warnings whilst using >> btrfs. They're all "warn_slowpath", and all seem to be in >> fs/btrfs/disk-io.c. I've included one typical example at the end of >> this mail. >> >> Kernel versions are 2.6.29-rc2, -rc4 and -rc6. >> >> If I do lots of writes to my btrfs filesystem (e.g. video >> encoding), I end up with a syslog in the tens-of-megabytes range. This >> makes logcheck an unhappy bunny... >> >> I don't know if this behaviour is expected, and everyone using >> btrfs simply puts up with it for now, or if it's something unusual >> that needs investigating. On the chance that it's the latter, I'm >> reporting it here. >> >> Hugo. >> >> Feb 23 21:45:42 vlad kernel: [ cut here ] >> Feb 23 21:45:42 vlad kernel: WARNING: at fs/btrfs/disk-io.c:815 >> clean_tree_block+0x9d/0xbb [btrfs]() >> Feb 23 21:45:42 vlad kernel: Hardware name: System Product Name >> Feb 23 21:45:42 vlad kernel: Modules linked in: tun ext3 jbd btrfs >> zlib_deflate tcp_diag inet_diag kqemu cpufreq_userspace ipv6 nfsd nfs lockd >> nfs_acl auth_rpcgss sunrpc af_packet bridge stp llc xfs exportfs it87 >> hwmon_vid powernow_k8 sbp2 ieee1394 ide_generic ide_gd_mod ide_cd_mod pcspkr >> evdev k8temp hwmon i2c_viapro i2c_core button dm_mirror dm_region_hash >> dm_log dm_snapshot dm_mod raid1 md_mod usbhid usb_storage libusual sg sr_mod >> cdrom via82cxxx floppy via_rhine mii ehci_hcd uhci_hcd usbcore pata_via >> ide_pci_generic ide_core sd_mod thermal processor fan unix >> Feb 23 21:45:42 vlad kernel: Pid: 27034, comm: hdparm Tainted: GW >> 2.6.29-rc4 #1 >> Feb 23 21:45:42 vlad kernel: Call Trace: >> Feb 23 21:45:42 vlad kernel: [] warn_slowpath+0xd8/0x111 >> Feb 23 21:45:42 vlad kernel: [] >> radix_tree_insert+0xd7/0x19f >> Feb 23 21:45:42 vlad kernel: [] >> add_to_page_cache_locked+0x52/0x9e >> Feb 23 21:45:42 vlad kernel: [] >> add_to_page_cache_lru+0x40/0x58 >> Feb 23 21:45:42 vlad kernel: [] >> find_or_create_page+0x62/0x88 >> Feb 23 21:45:42 vlad kernel: [] >> alloc_extent_buffer+0x268/0x2ec [btrfs] >> Feb 23 21:45:42 vlad kernel: [] clean_tree_block+0x9d/0xbb >> [btrfs] >> Feb 23 21:45:42 vlad kernel: [] >> btrfs_init_new_buffer+0x99/0xf3 [btrf
Re: btrfs: warn_slowpath in clean_tree_block and others
But what are you doing to the filesystem when it crashes? How did you mount it? Lee On Wed, Feb 25, 2009 at 08:03:01AM -0600, Mitch Harder (aka DontPanic) wrote: > I've been creating a local git repository of full btrfs-unstable sources. > > I'll create a new branch off the master branch, and apply the patch > supplied in the Feb. 11 message to the M/L. > > I then create a kernel module based on the results in /fs/btrfs/ > > I have also tried replicating the experimental branch, and merging the > patch into that branch, but I get the same results. > > On Wed, Feb 25, 2009 at 12:26 AM, Lee Trager wrote: > > Mitch, I haven't seen any problems using BTRFS and my patch on 2.6.28 or > > 2.6.27, what are you doing to cause this error? Are you using the latest > > sources from btrfs-unstable? > > > > Lee > > > > Mitch Harder (aka DontPanic) wrote: > >> I have also been getting similar warnings filling up my logs. > >> > >> However, in my case, I have been experimenting with back-porting btrfs > >> to a 2.6.28 kernel. ?So I've been waiting for the back-porting efforts > >> to get a little further along. > >> > >> But I thought I'd respond in case this information helps. > >> > >> Here's an example of the warnings I've been seeing: > >> > >> [80577.151167] [ cut here ] > >> [80577.151169] WARNING: at > >> /var/tmp/portage/sys-fs/btrfs-9998/work/btrfs-9998/disk-io.c:860 > >> clean_tree_block+0xa4/0xb0 [btrfs]() > >> [80577.151172] Modules linked in: btrfs snd_pcm_oss snd_mixer_oss > >> snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device > >> ipv6 ppdev snd_intel8x0 snd_ac97_codec parport_pc nvidia(P) ac97_bus > >> snd_pcm snd_timer ohci_hcd ssb shpchp pci_hotplug pcmcia i2c_nforce2 > >> snd forcedeth sr_mod pcspkr parport i2c_core snd_page_alloc nvidia_agp > >> sl811_hcd pcmcia_core uhci_hcd ehci_hcd > >> [80577.151190] Pid: 11503, comm: cp Tainted: P ? ? ? ?W > >> ?2.6.28-sabayon-r10 #1 > >> [80577.151192] Call Trace: > >> [80577.151195] ?[] warn_on_slowpath+0x5f/0x90 > >> [80577.151203] ?[] rb_insert_color+0x77/0xe0 > >> [80577.151221] ?[] alloc_extent_buffer+0x1fe/0x300 [btrfs] > >> [80577.151238] ?[] clean_tree_block+0xa4/0xb0 [btrfs] > >> [80577.151253] ?[] btrfs_init_new_buffer+0x7d/0x130 [btrfs] > >> [80577.151269] ?[] btrfs_alloc_free_block+0x104/0x110 [btrfs] > >> [80577.151285] ?[] __btrfs_cow_block+0x22a/0x8b0 [btrfs] > >> [80577.151300] ?[] generic_bin_search+0x162/0x1c0 [btrfs] > >> [80577.151315] ?[] btrfs_cow_block+0x156/0x200 [btrfs] > >> [80577.151330] ?[] btrfs_search_slot+0x1a7/0x910 [btrfs] > >> [80577.151333] ?[] irq_exit+0x27/0x60 > >> [80577.151336] ?[] do_IRQ+0x6b/0x80 > >> [80577.151354] ?[] read_extent_buffer+0xd5/0x170 [btrfs] > >> [80577.151369] ?[] btrfs_insert_empty_items+0x6d/0x410 [btrfs] > >> [80577.151385] ?[] btrfs_find_block_group+0xff/0x1a0 [btrfs] > >> [80577.151402] ?[] btrfs_new_inode+0x18d/0x360 [btrfs] > >> [80577.151420] ?[] btrfs_create+0x189/0x2a0 [btrfs] > >> [80577.151423] ?[] security_capable+0x9/0x10 > >> [80577.151427] ?[] vfs_create+0xcd/0x160 > >> [80577.151430] ?[] do_filp_open+0x5af/0x7d0 > >> [80577.151433] ?[] cp_new_stat64+0xf9/0x110 > >> [80577.151436] ?[] do_sys_open+0x4e/0xe0 > >> [80577.151439] ?[] sys_open+0x2c/0x40 > >> [80577.151442] ?[] sysenter_do_call+0x12/0x21 > >> [80577.151444] ---[ end trace 79cdc48bc88dedf7 ]--- > >> > >> > >> On Tue, Feb 24, 2009 at 5:02 PM, Hugo Mills > >> wrote: > >> > >>> ? This is essentially a repost of a mail I made last week, to which I > >>> didn't get a reply. > >>> > >>> ? I'm getting huge numbers of kernel warnings whilst using > >>> btrfs. They're all "warn_slowpath", and all seem to be in > >>> fs/btrfs/disk-io.c. I've included one typical example at the end of > >>> this mail. > >>> > >>> ? Kernel versions are 2.6.29-rc2, -rc4 and -rc6. > >>> > >>> ? If I do lots of writes to my btrfs filesystem (e.g. video > >>> encoding), I end up with a syslog in the tens-of-megabytes range. This > >>> makes logcheck an unhappy bunny... > >>> > >>> ? I don't know if this behaviour is expected, and everyone using > >>> btrfs simply puts up with it for now, or if it's somet
Re: [PATCH] Backport for 2.6.27 and 2.6.26 on the experimental branch
4.393942] .jiffies : 2060701 [ 9824.393942] .next_balance : 2.060674 [ 9824.393942] .curr->pid : 3450 [ 9824.393942] .clock : 8542801.036413 [ 9824.393942] .cpu_load[0] : 0 [ 9824.393942] .cpu_load[1] : 0 [ 9824.393942] .cpu_load[2] : 20 [ 9824.393942] .cpu_load[3] : 56 [ 9824.393942] .cpu_load[4] : 60 [ 9824.393942] [ 9824.393942] cfs_rq[1]: [ 9824.393942] .exec_clock: 0.00 [ 9824.393942] .MIN_vruntime : 40916.410301 [ 9824.393942] .min_vruntime : 40926.274857 [ 9824.393942] .max_vruntime : 40916.410301 [ 9824.393942] .spread: 0.00 [ 9824.393942] .spread0 : -138652.273960 [ 9824.393942] .nr_running: 2 [ 9824.393942] .load : 2048 [ 9824.393942] .nr_spread_over: 0 [ 9824.393942] [ 9824.393942] runnable tasks: [ 9824.393942] task PID tree-key switches prio exec-runtime sum-execsum-sleep [ 9824.393942] -- [ 9824.393942] metacity 3176 40916.410301 2904 120 0 0 0.00 0.00 0.00 / [ 9824.393942] R bash 3450 40886.27488034 120 0 0 0.00 0.00 0.00 / [ 9824.393942] Lee On Tue, Feb 24, 2009 at 03:36:29PM -0500, Lee Trager wrote: > I ran a few tests that Jim suggested and found that btrfs works fine on > 2.6.26 as long as there are only 23 or less files on the file system. > Anymore and I experience the lockup. Jim and I will be working to find a > solution but if anyone else has any clues that would be greatly > appreciated. > > Lee > On Tue, Feb 24, 2009 at 11:24:06AM -0500, jim owens wrote: > > Lee Trager wrote: > >> The more and more I look at this problem the more I tend to think that > >> the issue is because of some change in the way the VFS or something > >> interacts with the file system. Does anyone know of any big changes? Why > >> is the inode being marked dirty? Is there some kind of read error. I'm > >> completly lost in solving this problem. > > > > Being a filesystem guy, I always try blaming vm or drivers :) > > > > Until someone with real experience gives us the answer, > > I'll work with you off the mailing list to try to narrow > > down why this is happening. > > > > jim > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental branch rebased
I have also cleaned up the backport experimental patch so it applies
cleanly on the latest code.
Lee
diff -Naur btrfs-orig/async-thread.c btrfs/async-thread.c
--- btrfs-orig/async-thread.c 2009-02-26 13:07:03.0 -0500
+++ btrfs/async-thread.c2009-02-26 13:09:18.0 -0500
@@ -20,7 +20,10 @@
#include
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
#include
+#endif
#include "async-thread.h"
#define WORK_QUEUED_BIT 0
diff -Naur btrfs-orig/compat.h btrfs/compat.h
--- btrfs-orig/compat.h 2009-02-26 13:07:03.0 -0500
+++ btrfs/compat.h 2009-02-26 13:09:18.0 -0500
@@ -1,7 +1,33 @@
#ifndef _COMPAT_H_
#define _COMPAT_H_
+#include
+
#define btrfs_drop_nlink(inode) drop_nlink(inode)
#define btrfs_inc_nlink(inode) inc_nlink(inode)
+#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 27)
+static inline struct dentry *d_obtain_alias(struct inode *inode)
+{
+ struct dentry *d;
+
+ if (!inode)
+ return NULL;
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ d = d_alloc_anon(inode);
+ if (!d)
+ iput(inode);
+ return d;
+}
+#endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 28)
+#define __pagevec_lru_add_file __pagevec_lru_add
+#define open_bdev_exclusive open_bdev_excl
+#define close_bdev_exclusive(bdev, mode) close_bdev_excl(bdev)
+typedef unsigned __bitwise__ fmode_t;
+#endif
+
#endif /* _COMPAT_H_ */
diff -Naur btrfs-orig/export.h btrfs/export.h
--- btrfs-orig/export.h 2009-02-26 13:07:05.0 -0500
+++ btrfs/export.h 2009-02-26 13:09:56.0 -0500
@@ -16,4 +16,32 @@
u64 parent_root_objectid;
} __attribute__ ((packed));
+// BTRFS needs the btrfs from fid_type which was added in 2.6.27
+// All I did was copy the btrfs defse which was found in
+// linux-2.6.27/include/linux/exportfs.h
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
+enum btrfs_fid_type {
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number.
+*/
+ FILEID_BTRFS_WITHOUT_PARENT = 0x4d,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation.
+*/
+ FILEID_BTRFS_WITH_PARENT = 0x4e,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation,
+* 64 bit parent root object ID.
+*/
+ FILEID_BTRFS_WITH_PARENT_ROOT = 0x4f,
+};
+#endif
+
#endif
diff -Naur btrfs-orig/extent_io.c btrfs/extent_io.c
--- btrfs-orig/extent_io.c 2009-02-26 13:07:06.0 -0500
+++ btrfs/extent_io.c 2009-02-26 13:09:56.0 -0500
@@ -2847,6 +2847,7 @@
return sector;
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 29)
int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
__u64 start, __u64 len, get_extent_t *get_extent)
{
@@ -2938,6 +2939,7 @@
GFP_NOFS);
return ret;
}
+#endif
static inline struct page *extent_buffer_page(struct extent_buffer *eb,
unsigned long i)
@@ -3163,13 +3165,21 @@
}
}
clear_page_dirty_for_io(page);
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_lock_irq(&page->mapping->tree_lock);
+#else
+ read_lock_irq(&page->mapping->tree_lock);
+#endif
if (!PageDirty(page)) {
radix_tree_tag_clear(&page->mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_unlock_irq(&page->mapping->tree_lock);
+#else
+ read_unlock_irq(&page->mapping->tree_lock);
+#endif
unlock_page(page);
}
return 0;
@@ -3347,7 +3357,11 @@
for (i = start_i; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!wait) {
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
if (!trylock_page(page))
+#else
+ if(!TestSetPageLocked(page))
+#endif
goto unlock_exit;
} else {
lock_page(page);
diff -Naur btrfs-orig/extent_io.h btrfs/extent_io.h
--- btrfs-orig/extent_io.h 2009-02-26 13:07:06.0 -0500
+++ btrfs/extent_io.h 2009-02-26 13:09:56.0 -0500
@@ -2,6 +2,7 @@
#define __EXTENTIO__
#include
+#include
/* bits for the extent state */
#define EXTENT_DIRTY 1
@@ -205,8 +206,10 @@
unsigned from, unsigned to);
sector_t extent_bmap(struct address_space *mapping, sector_t iblock,
ge
Re: experimental branch rebased
Sorry it took so long to reply its been a crazy week for me. Anyway
thanks for figuring this out for me. I can't believe I made such a
stupid mistake. Anyway here is the rebased patch.
Lee
diff -Naur btrfs-orig/async-thread.c btrfs/async-thread.c
--- btrfs-orig/async-thread.c 2009-02-26 13:07:03.0 -0500
+++ btrfs/async-thread.c2009-02-26 14:02:05.0 -0500
@@ -20,7 +20,10 @@
#include
#include
#include
+#include
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
#include
+#endif
#include "async-thread.h"
#define WORK_QUEUED_BIT 0
diff -Naur btrfs-orig/compat.h btrfs/compat.h
--- btrfs-orig/compat.h 2009-02-26 13:07:03.0 -0500
+++ btrfs/compat.h 2009-02-26 14:02:05.0 -0500
@@ -1,7 +1,33 @@
#ifndef _COMPAT_H_
#define _COMPAT_H_
+#include
+
#define btrfs_drop_nlink(inode) drop_nlink(inode)
#define btrfs_inc_nlink(inode) inc_nlink(inode)
+#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 27)
+static inline struct dentry *d_obtain_alias(struct inode *inode)
+{
+ struct dentry *d;
+
+ if (!inode)
+ return NULL;
+ if (IS_ERR(inode))
+ return ERR_CAST(inode);
+
+ d = d_alloc_anon(inode);
+ if (!d)
+ iput(inode);
+ return d;
+}
+#endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 28)
+#define __pagevec_lru_add_file __pagevec_lru_add
+#define open_bdev_exclusive open_bdev_excl
+#define close_bdev_exclusive(bdev, mode) close_bdev_excl(bdev)
+typedef unsigned __bitwise__ fmode_t;
+#endif
+
#endif /* _COMPAT_H_ */
diff -Naur btrfs-orig/export.h btrfs/export.h
--- btrfs-orig/export.h 2009-02-26 13:07:05.0 -0500
+++ btrfs/export.h 2009-02-26 14:02:05.0 -0500
@@ -16,4 +16,32 @@
u64 parent_root_objectid;
} __attribute__ ((packed));
+// BTRFS needs the btrfs from fid_type which was added in 2.6.27
+// All I did was copy the btrfs defse which was found in
+// linux-2.6.27/include/linux/exportfs.h
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
+enum btrfs_fid_type {
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number.
+*/
+ FILEID_BTRFS_WITHOUT_PARENT = 0x4d,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation.
+*/
+ FILEID_BTRFS_WITH_PARENT = 0x4e,
+
+ /*
+* 64 bit object ID, 64 bit root object ID,
+* 32 bit generation number,
+* 64 bit parent object ID, 32 bit parent generation,
+* 64 bit parent root object ID.
+*/
+ FILEID_BTRFS_WITH_PARENT_ROOT = 0x4f,
+};
+#endif
+
#endif
diff -Naur btrfs-orig/extent_io.c btrfs/extent_io.c
--- btrfs-orig/extent_io.c 2009-02-26 13:07:06.0 -0500
+++ btrfs/extent_io.c 2009-03-06 23:43:36.0 -0500
@@ -2847,6 +2847,7 @@
return sector;
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 29)
int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
__u64 start, __u64 len, get_extent_t *get_extent)
{
@@ -2938,6 +2939,7 @@
GFP_NOFS);
return ret;
}
+#endif
static inline struct page *extent_buffer_page(struct extent_buffer *eb,
unsigned long i)
@@ -3163,13 +3165,21 @@
}
}
clear_page_dirty_for_io(page);
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_lock_irq(&page->mapping->tree_lock);
+#else
+ read_lock_irq(&page->mapping->tree_lock);
+#endif
if (!PageDirty(page)) {
radix_tree_tag_clear(&page->mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
}
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
spin_unlock_irq(&page->mapping->tree_lock);
+#else
+ read_unlock_irq(&page->mapping->tree_lock);
+#endif
unlock_page(page);
}
return 0;
@@ -3347,7 +3357,11 @@
for (i = start_i; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!wait) {
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 27)
if (!trylock_page(page))
+#else
+ if(TestSetPageLocked(page))
+#endif
goto unlock_exit;
} else {
lock_page(page);
diff -Naur btrfs-orig/extent_io.h btrfs/extent_io.h
--- btrfs-orig/extent_io.h 2009-02-26 13:07:06.0 -0500
+++ btrfs/extent_io.h 2009-02-26 14:02:05.0 -0500
@@ -2,6 +2,7 @@
#define __EXTENTIO__
#include
+#include
/* bits for the extent state */
#define EXTENT_DIRTY 1
@@ -205,8 +206,10 @@
unsigned from, unsigned to);
