Re: Gloves

2015-02-08 Thread Lion Leather
Hi,

We are professional manufacturer of all kinds of leather jackets and leather 
gloves. 

Please visit our website www.lionleather.com 

We realize that you are in the business of gloves so we would like to give you 
our web link so that you can directly access to the category of our gloves.

Gloves Link:
http://lionleather.com/products.php?IDZ=0-0-0-115-35

Are you interested in?

Regards

Hassan Afzal
Proprietor

*
Lion Leather
Sialkot Pakistan
Tel: 0092-52-4593535
Fax: 0092-52-4569377
Mob: 0092-3006122353
www.lionleather.com
has...@lionleather.com
Whats Ap: 00923006122353
Viber:  00923006122353
skype: everfast5
**

To unsubscribe plz send us email at   has...@lionleather.com   and write in the 
subject  “UNSUBSCRIBE”
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Предлагаю высококачественный кофе от швейцарского производителя

2015-02-08 Thread Степанова Злата
Предлагаем высококачественный кофе от швейцарского производителя 
http://bit.ly/1xqBJAo

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
Hello everybody,

I have a Raid-1 btrfs filesystem with 5 devices and I was running
btrfs scrub once a week. Unfortunately, one disk (4TB) failed.
I added two new  (6TB each) disks in the array and now I get:

# btrfs filesystem df /mnt/mountpoint
Data, RAID1: total=6.58TiB, used=6.57TiB
System, RAID1: total=32.00MiB, used=1.02MiB
Metadata, RAID1: total=11.00GiB, used=9.61GiB
GlobalReserve, single: total=512.00MiB, used=0.00B


Label: 'label'  uuid: 1d0d0850-d0bc-4376-96a3-177968ff2431
Total devices 7 FS bytes used 6.58TiB
devid1 size 2.73TiB used 2.68TiB path /dev/sdi1
devid3 size 1.82TiB used 1.68TiB path /dev/sdd1
devid4 size 1.82TiB used 1.60TiB path /dev/sdc1
devid5 size 2.73TiB used 2.49TiB path /dev/sdg1
devid6 size 5.46TiB used 878.03GiB path /dev/sda
devid7 size 5.46TiB used 878.03GiB path /dev/sdb
*** Some devices missing

This was working for some time (the new disks got ~878GB each):
# mount -o degraded /dev/sdi1 /mnt/mountpoint
# btrfs device delete missing /mnt/mountpoint

but it stopped with Input/Output error which appears again after
restarting my system.

# btrfs device delete missing /mnt/mountpoint
ERROR: error removing the device 'missing' - Input/output error

dmesg shows:
dmesg | grep BTRFS
[  117.535217] BTRFS info (device sdc1): relocating block group
10792241987584 flags 17
[  133.386996] BTRFS info (device sdc1): csum failed ino 257 off
541310976 csum 4144645530 expected csum 4144645376
[  133.413795] BTRFS info (device sdc1): csum failed ino 257 off
541310976 csum 4144645530 expected csum 4144645376
[  133.423884] BTRFS info (device sdc1): csum failed ino 257 off
541310976 csum 4144645530 expected csum 4144645376
[  303.627547] BTRFS info (device sdc1): relocating block group
10792241987584 flags 17
[  308.604231] BTRFS info (device sdc1): csum failed ino 258 off
541310976 csum 4144645530 expected csum 4144645376
[  308.631229] BTRFS info (device sdc1): csum failed ino 258 off
541310976 csum 4144645530 expected csum 4144645376
[  308.641205] BTRFS info (device sdc1): csum failed ino 258 off
541310976 csum 4144645530 expected csum 4144645376



What should be my next step?

Should I run a btrfs scrub?
Should I do a btrfsck?
Something else?
Can I somehow ignore or solve these sdc1 errors?
On the proposed solution, should the filesystem be mounted (in
degraded mode?) or not?


Thanks a lot! ;)


PS:
As you can see from the following, I should probably remove /dev/sdc,
too, right? I ignored the errors because they happened in ~7k hours
but at ~11k hours (and before) extended offline test(s) completed
successfully. I considered them to be SATA-cable related errors.

#
  smartctl -a /dev/sdi 
#
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME  FLAG VALUE WORST THRESH TYPE
UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate 0x002f   200   200   051Pre-fail
Always   -   0
  3 Spin_Up_Time0x0027   173   173   021Pre-fail
Always   -   6350
  4 Start_Stop_Count0x0032   100   100   000Old_age
Always   -   236
  5 Reallocated_Sector_Ct   0x0033   200   200   140Pre-fail
Always   -   0
  7 Seek_Error_Rate 0x002e   200   200   000Old_age
Always   -   0
  9 Power_On_Hours  0x0032   091   091   000Old_age
Always   -   6887
 10 Spin_Retry_Count0x0032   100   100   000Old_age
Always   -   0
 11 Calibration_Retry_Count 0x0032   100   100   000Old_age
Always   -   0
 12 Power_Cycle_Count   0x0032   100   100   000Old_age
Always   -   220
192 Power-Off_Retract_Count 0x0032   200   200   000Old_age
Always   -   141
193 Load_Cycle_Count0x0032   189   189   000Old_age
Always   -   33166
194 Temperature_Celsius 0x0022   125   098   000Old_age
Always   -   25
196 Reallocated_Event_Count 0x0032   200   200   000Old_age
Always   -   0
197 Current_Pending_Sector  0x0032   200   200   000Old_age
Always   -   0
198 Offline_Uncorrectable   0x0030   200   200   000Old_age
Offline  -   0
199 UDMA_CRC_Error_Count0x0032   200   200   000Old_age
Always   -   0
200 Multi_Zone_Error_Rate   0x0008   200   200   000Old_age
Offline  -   0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_DescriptionStatus  Remaining
LifeTime(hours)  LBA_of_first_error
# 1  Short offline   Completed without error   00%  6872 -
# 2  Short offline   Completed without error   00%  6850 -
# 3  Short offline   Completed without error   00%  6826 -
# 4  Short offline   Completed without error   00%  6802

Segfault when deleting device

2015-02-08 Thread Robert Mucina
Hello,
I have kubuntu 14.10 with 3.17.1 kernel and 3.17 tools
the setup of data pool was btrfs raid1, 3x6TB + 1x3TB. Time has come
to replace old 3TB with new 3TB.
As usual I added new device and deleted the old one, system started
relocating chunks and I went to bed...
In the morning I was greeted with pool in RO mode and segmentation
fault in dmesg, see here http://pastebin.com/vcr7qagB
After some nail biting and reisub I found data to be ok and pool
worked normally in RW. Pheww.
By output in dmesg and in btrfs fi show I can see the new device has
336GBs used so the problem occurred after some data copying.
I ran 7 hour SMART extensive test on new device and it came out
perfectly ok. Then I tried disconnecting old drive in attempt to do
delete missing. Unfortunately this also led to segmentation fault
almost immediately after firing the command (didn't make a copy of
that one though)
All three 6TB drives seem to be fine by quick SMART checks.

So now I'm running in degraded mode, unable to delete missing device.
Options I see here is to attempt rebalance which I'm rather affraid of
in this state. Or upgrade kernel to 3.18.x in hope the problem's been
already fixed...
Any other thoughts?

Regards,
Robert
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Replicate snapshot to second machine fails

2015-02-08 Thread Thomas Schneider


Hi,
 
I want to replicate a snapshot from PC1 to virtual machine using this command:

user@pc1-gigabyte ~ $ sudo btrfs send 
/home/.snapshots/lmde_home_2015-02-07_02:38:21 | ssh vm1-debian sudo btrfs 
receive /home/.snapshots/
At subvol /home/.snapshots/lmde_home_2015-02-07_02:38:21
sudo: Kein TTY vorhanden und kein »askpass«-Programm angegeben
 
Unfortunately I cannot detect the root cause for the failure.
Any ideas?
 
On both machines I have installed programs ssh-askpass and ssh-askpass-gnome.
 

user@pc1-gigabyte ~ $ uname -a
Linux pc1-gigabyte 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt2-1 (2014-12-08) 
i686 GNU/Linux
user@pc1-gigabyte ~ $ sudo btrfs --version
Btrfs v3.17
user@pc1-gigabyte ~ $ sudo btrfs fi show
Label: none  uuid: 236fe36a-3187-4955-977d-f22cd818c424
    Total devices 1 FS bytes used 103.96GiB
    devid    1 size 147.00GiB used 147.00GiB path /dev/sda5
Btrfs v3.17
user@pc1-gigabyte ~ $ sudo btrfs fi df /home
Data, single: total=142.97GiB, used=103.03GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=2.00GiB, used=957.17MiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=320.00MiB, used=0.00B
 
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replicate snapshot to second machine fails

2015-02-08 Thread cwillu
This isn't a btrfs-send or a btrfs-receive question:

$ echo hi | ssh machine.local sudo echo test
sudo: no tty present and no askpass program specified

How were you planning on providing credentials to sudo?

On Sun, Feb 8, 2015 at 9:17 AM, Thomas Schneider c.mo...@web.de wrote:


 Hi,

 I want to replicate a snapshot from PC1 to virtual machine using this command:

 user@pc1-gigabyte ~ $ sudo btrfs send 
 /home/.snapshots/lmde_home_2015-02-07_02:38:21 | ssh vm1-debian sudo btrfs 
 receive /home/.snapshots/
 At subvol /home/.snapshots/lmde_home_2015-02-07_02:38:21
 sudo: Kein TTY vorhanden und kein »askpass«-Programm angegeben

 Unfortunately I cannot detect the root cause for the failure.
 Any ideas?

 On both machines I have installed programs ssh-askpass and ssh-askpass-gnome.


 user@pc1-gigabyte ~ $ uname -a
 Linux pc1-gigabyte 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt2-1 (2014-12-08) 
 i686 GNU/Linux
 user@pc1-gigabyte ~ $ sudo btrfs --version
 Btrfs v3.17
 user@pc1-gigabyte ~ $ sudo btrfs fi show
 Label: none  uuid: 236fe36a-3187-4955-977d-f22cd818c424
 Total devices 1 FS bytes used 103.96GiB
 devid1 size 147.00GiB used 147.00GiB path /dev/sda5
 Btrfs v3.17
 user@pc1-gigabyte ~ $ sudo btrfs fi df /home
 Data, single: total=142.97GiB, used=103.03GiB
 System, DUP: total=8.00MiB, used=16.00KiB
 System, single: total=4.00MiB, used=0.00B
 Metadata, DUP: total=2.00GiB, used=957.17MiB
 Metadata, single: total=8.00MiB, used=0.00B
 GlobalReserve, single: total=320.00MiB, used=0.00B

 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
What kernel and btrfs-progs version?
Also include the entire dmesg.


Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
I have Arch Linux:

# uname -a
Linux hostname 3.19.0-1-mainline #1 SMP PREEMPT Wed Dec 24 00:27:17
WET 2014 x86_64 GNU/Linux

btrfs-progs 3.17.3-1

dmesg:
[0.00] Initializing cgroup subsys cpuset
[0.00] Initializing cgroup subsys cpu
[0.00] Initializing cgroup subsys cpuacct
[0.00] Linux version 3.19.0-1-mainline (username@censored)
(gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT Wed Dec 24 00:27:17 WET 2014
[0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline
root=UUID=6c6da07e-0da7-475e-84cc-2b968971f18c rw quiet
[0.00] e820: BIOS-provided physical RAM map:
[0.00] BIOS-e820: [mem 0x-0x0009d7ff] usable
[0.00] BIOS-e820: [mem 0x0009d800-0x0009] reserved
[0.00] BIOS-e820: [mem 0x000e-0x000f] reserved
[0.00] BIOS-e820: [mem 0x0010-0x1fff] usable
[0.00] BIOS-e820: [mem 0x2000-0x201f] reserved
[0.00] BIOS-e820: [mem 0x2020-0x40003fff] usable
[0.00] BIOS-e820: [mem 0x40004000-0x40004fff] reserved
[0.00] BIOS-e820: [mem 0x40005000-0x9d0c6fff] usable
[0.00] BIOS-e820: [mem 0x9d0c7000-0x9d871fff] reserved
[0.00] BIOS-e820: [mem 0x9d872000-0x9d8fcfff] usable
[0.00] BIOS-e820: [mem 0x9d8fd000-0x9d99bfff] ACPI NVS
[0.00] BIOS-e820: [mem 0x9d99c000-0x9e16efff] reserved
[0.00] BIOS-e820: [mem 0x9e16f000-0x9e16] usable
[0.00] BIOS-e820: [mem 0x9e17-0x9e1b2fff] ACPI NVS
[0.00] BIOS-e820: [mem 0x9e1b3000-0x9ebfbfff] usable
[0.00] BIOS-e820: [mem 0x9ebfc000-0x9eff1fff] reserved
[0.00] BIOS-e820: [mem 0x9eff2000-0x9eff] usable
[0.00] BIOS-e820: [mem 0xdf80-0xdf9f] reserved
[0.00] BIOS-e820: [mem 0xf800-0xfbff] reserved
[0.00] BIOS-e820: [mem 0xfec0-0xfec00fff] reserved
[0.00] BIOS-e820: [mem 0xfed0-0xfed03fff] reserved
[0.00] BIOS-e820: [mem 0xfed1c000-0xfed1] reserved
[0.00] BIOS-e820: [mem 0xfee0-0xfee00fff] reserved
[0.00] BIOS-e820: [mem 0xff00-0x] reserved
[0.00] BIOS-e820: [mem 0x0001-0x00031f5f] usable
[0.00] NX (Execute Disable) protection: active
[0.00] SMBIOS 2.7 present.
[0.00] DMI: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77
Pro4-M, BIOS P2.00 07/11/2013
[0.00] e820: update [mem 0x-0x0fff] usable == reserved
[0.00] e820: remove [mem 0x000a-0x000f] usable
[0.00] AGP: No AGP bridge found
[0.00] e820: last_pfn = 0x31f600 max_arch_pfn = 0x4
[0.00] MTRR default type: uncachable
[0.00] MTRR fixed ranges enabled:
[0.00]   0-9 write-back
[0.00]   A-B uncachable
[0.00]   C-D3FFF write-protect
[0.00]   D4000-E7FFF uncachable
[0.00]   E8000-F write-protect
[0.00] MTRR variable ranges enabled:
[0.00]   0 base 0 mask E write-back
[0.00]   1 base 2 mask F write-back
[0.00]   2 base 3 mask FE000 write-back
[0.00]   3 base 0C000 mask FC000 uncachable
[0.00]   4 base 0A000 mask FE000 uncachable
[0.00]   5 base 09F80 mask FFF80 uncachable
[0.00]   6 base 31F80 mask FFF80 uncachable
[0.00]   7 base 31F60 mask FFFE0 uncachable
[0.00]   8 disabled
[0.00]   9 disabled
[0.00] PAT configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- UC
[0.00] e820: update [mem 0x9f80-0x] usable == reserved
[0.00] e820: last_pfn = 0x9f000 max_arch_pfn = 0x4
[0.00] found SMP MP-table at [mem 0x000fd810-0x000fd81f]
mapped at [880fd810]
[0.00] Scanning 1 areas for low memory corruption
[0.00] Base memory trampoline at [88097000] 97000 size 24576
[0.00] init_memory_mapping: [mem 0x-0x000f]
[0.00]  [mem 0x-0x000f] page 4k
[0.00] BRK [0x01b31000, 0x01b31fff] PGTABLE
[0.00] BRK [0x01b32000, 0x01b32fff] PGTABLE
[0.00] BRK [0x01b33000, 0x01b33fff] PGTABLE
[0.00] init_memory_mapping: [mem 0x31f40-0x31f5f]
[0.00]  [mem 0x31f40-0x31f5f] page 2M
[0.00] BRK [0x01b34000, 0x01b34fff] PGTABLE
[0.00] init_memory_mapping: [mem 0x31c00-0x31f3f]
[0.00]  [mem 0x31c00-0x31f3f] page 2M
[0.00] init_memory_mapping: [mem 0x3-0x31bff]
[0.00]  [mem 

kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123

2015-02-08 Thread Jeroen Van den Keybus
Hi,


I have a LUKS encrypted raw external (USB) disk mapped to
/dev/mapper/sd.backup. This mapped device was default btrfs formatted.
I can mount the mapped device to /mnt/backup. There used to be a
subvolume in /mnt/backup, which I deleted.

I now seem unable to either add a directory to /mnt/backup or umount
/mnt/backup; the command never finishes and the dmesg log reports:

[17937.939438] [ cut here ]
[17937.939523] kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!
[17937.939602] invalid opcode:  [#1] SMP
[17937.939664] Modules linked in: xts gf128mul rfcomm bluetooth joydev
hid_logitech_dj xt_multiport nfsd auth_rpcgss nfs_acl nfs lockd grace
sunrpc fscache dm_crypt xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4
xt_physdev br_netfilter xt_tcpudp xt_conntrack iptable_filter
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack ip_tables ebtable_filter ebtable_broute bridge stp llc
ebtables x_tables eeepc_wmi asus_wmi sparse_keymap video kvm_amd kvm
pl2303 usbserial serio_raw k10temp snd_usb_audio snd_usbmidi_lib
snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic
snd_hda_intel sp5100_tco snd_hda_controller i2c_piix4 snd_hda_codec
snd_hwdep snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq
snd_seq_device snd_timer snd shpchp soundcore 8250_fintek mac_hid
parport_pc
[17937.940798]  ppdev lp parport nct6775 nls_iso8859_1 hwmon_vid btrfs
raid10 raid1 multipath linear raid0 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx hid_generic usbhid hid xor
raid6_pq psmouse radeon uas usb_storage r8169 i2c_algo_bit ttm mii
drm_kms_helper drm wmi ahci libahci
[17937.941259] CPU: 1 PID: 16331 Comm: btrfs-cleaner Not tainted
3.18.3-031803-generic #201501161810
[17937.941358] Hardware name: System manufacturer System Product
Name/E45M1-M PRO, BIOS 0801 01/10/2012
[17937.941464] task: 88023383a800 ti: 880183868000 task.ti:
880183868000
[17937.941547] RIP: 0010:[c04dd109]  [c04dd109]
btrfs_orphan_add+0x1a9/0x1c0 [btrfs]
[17937.941705] RSP: 0018:88018386bc98  EFLAGS: 00010286
[17937.941769] RAX: ffe4 RBX: 8801ebe6b000 RCX: 
[17937.941849] RDX: 5cb8 RSI: 0004 RDI: 8801b7f85138
[17937.941928] RBP: 88018386bcd8 R08: 88023ed1db40 R09: 88019ab07b40
[17937.942007] R10:  R11: 0010 R12: 880233513630
[17937.942086] R13: 880041414d58 R14: 8801ebe6b458 R15: 0001
[17937.942168] FS:  7f8d60824740() GS:88023ed0()
knlGS:
[17937.942261] CS:  0010 DS:  ES:  CR0: 8005003b
[17937.942327] CR2: 7f70b2ec CR3: 000124627000 CR4: 07e0
[17937.942406] Stack:
[17937.942435]  88018386bcd8 c051bd4f 8801b93b9000
880204bbe200
[17937.942537]  8801b93b9000 88019ab07b40 880233513630
0001
[17937.942640]  88018386bd58 c04c5310 8801b7f85000
0004c04abffa
[17937.942742] Call Trace:
[17937.942819]  [c051bd4f] ?
lookup_free_space_inode+0x4f/0x100 [btrfs]
[17937.942934]  [c04c5310]
btrfs_remove_block_group+0x140/0x490 [btrfs]
[17937.943056]  [c0500065] btrfs_remove_chunk+0x245/0x380 [btrfs]
[17937.943163]  [c04c5896] btrfs_delete_unused_bgs+0x236/0x270 [btrfs]
[17937.943272]  [c04ced6c] cleaner_kthread+0x12c/0x190 [btrfs]
[17937.943374]  [c04cec40] ?
btrfs_destroy_all_delalloc_inodes+0x120/0x120 [btrfs]
[17937.943471]  [85093a49] kthread+0xc9/0xe0
[17937.943531]  [85093980] ? flush_kthread_worker+0x90/0x90
[17937.943608]  [857b3b7c] ret_from_fork+0x7c/0xb0
[17937.943673]  [85093980] ? flush_kthread_worker+0x90/0x90
[17937.943746] Code: e8 4d 9f fc ff 8b 45 c8 e9 6d ff ff ff 0f 1f 44
00 00 f0 41 80 65 80 fd 4c 89 ef 89 45 c8 e8 bf 1e fe ff 8b 45 c8 e9
48 ff ff ff 0f 0b 4c 89 f7 45 31 f6 e8 ea 64 2d c5 e9 f9 fe ff ff 0f
1f 44
[17937.944273] RIP  [c04dd109] btrfs_orphan_add+0x1a9/0x1c0 [btrfs]
[17937.944392]  RSP 88018386bc98
[17937.944503] ---[ end trace cee2bcd2393b84fb ]---

$ uname -a:
Linux zacate 3.18.3-031803-generic #201501161810 SMP Fri Jan 16
18:12:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I also ran '$ sudo btrfs check /dev/mapper/sd.backup':
Checking filesystem on /dev/mapper/sd.backup
UUID: 15628b78-7380-4d57-bc77-426016205aa0
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 35881083 bytes used err is 0
total csum bytes: 0
total tree bytes: 360448
total fs tree bytes: 32768
total extent tree bytes: 81920
btree space waste bytes: 194142
file data blocks allocated: 67371008
 referenced 67371008
Btrfs v3.18.2

$ sudo btrfs fi show
Label: none  uuid: 6815db4b-bbde-4c98-8cb3-4f984f9bc99f
Total devices 1 FS bytes used 4.35GiB
devid1 size 103.81GiB used 30.02GiB path /dev/sdf2
Label: none  uuid: 

Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
On Sun, Feb 8, 2015 at 2:06 PM, constantine costas.magn...@gmail.com wrote:

 [   78.039253] BTRFS info (device sdc1): disk space caching is enabled
 [   78.056020] BTRFS: failed to read chunk tree on sdc1
 [   78.091062] BTRFS: open_ctree failed
 [   84.729944] BTRFS info (device sdc1): allowing degraded mounts
 [   84.729950] BTRFS info (device sdc1): disk space caching is enabled
 [   84.754301] BTRFS warning (device sdc1): devid 2 missing
 [   84.856408] BTRFS: bdev (null) errs: wr 13, rd 0, flush 0, corrupt 63, gen 
 5
 [   84.856415] BTRFS: bdev /dev/sdc1 errs: wr 1176932, rd 99072, flush
 5946, corrupt 2178961, gen 7557
 [   84.856419] BTRFS: bdev /dev/sdd1 errs: wr 0, rd 0, flush 0,
 corrupt 17, gen 0
 [   84.856425] BTRFS: bdev /dev/sdi1 errs: wr 0, rd 0, flush 0,
 corrupt 60, gen 0
 [   84.856428] BTRFS: bdev /dev/sdg1 errs: wr 0, rd 0, flush 0,
 corrupt 57, gen 0

You had problems with sdc for a long time.  It's reporting millions of
corrupt events. This is cumulative, not just on this mount. So likely
Btrfs was trying to fix them before the device failure. If sdc is not
pristine, with a device failure, you basically have a partially lost
array because Btrfs raid1 tolerates a single device failure. With a 2
device failure which is what you have now, there will be some amount
of data loss.

Confusing is that sdd1, sdi1, sdg1 have gen 0 and also have
corruptions reported, just not anywhere near as many as sdc1. So I
don't know what problems you have with your hardware, but they're not
restricted to just one or two drives. Generation 0 makes no sense to
me.


 [  117.535217] BTRFS info (device sdc1): relocating block group
 10792241987584 flags 17
 [  133.386996] BTRFS info (device sdc1): csum failed ino 257 off
 541310976 csum 4144645530 expected csum 4144645376
 [  133.413795] BTRFS info (device sdc1): csum failed ino 257 off
 541310976 csum 4144645530 expected csum 4144645376
 [  133.423884] BTRFS info (device sdc1): csum failed ino 257 off
 541310976 csum 4144645530 expected csum 4144645376

So sdc1 still has problems, despite the scrubs, the problems with it
are persistent. Without historical kernel messages for a scrub prior
to the device failure, we can only speculate whether those scrubs were
repairing things correctly and now the reads are bad (read failures);
or if the original scrubs didn't actually fix the problem on sdc
(write failures).



 [  303.627547] BTRFS info (device sdc1): relocating block group
 10792241987584 flags 17
 [  308.604231] BTRFS info (device sdc1): csum failed ino 258 off
 541310976 csum 4144645530 expected csum 4144645376
 [  308.631229] BTRFS info (device sdc1): csum failed ino 258 off
 541310976 csum 4144645530 expected csum 4144645376
 [  308.641205] BTRFS info (device sdc1): csum failed ino 258 off
 541310976 csum 4144645530 expected csum 4144645376
 [ 1240.379575] BTRFS info (device sdc1): relocating block group
 10792241987584 flags 17
 [ 1247.867399] BTRFS info (device sdc1): csum failed ino 259 off
 541310976 csum 4144645530 expected csum 4144645376
 [ 1247.894211] BTRFS info (device sdc1): csum failed ino 259 off
 541310976 csum 4144645530 expected csum 4144645376
 [ 1247.904300] BTRFS info (device sdc1): csum failed ino 259 off
 541310976 csum 4144645530 expected csum 4144645376

More sdc1 errors.

For each drive, what do you get for:

smartctl -l scterc /dev/sdX
cat /sys/block/sdX/device/timeout

Basically you're in data recovery mode if you don't have a current
backup. If you have a current backup, give up on this volume, get rid
of the bad hardware after requalifying all the hardware you intend to
use in a new volume.

If you don't have a current backup, make one now. Just make sure you
don't overwrite any previous backup data in case you need it. Any
files that don't pass checksum will not be copied, these will be
recorded in dmesg. If you have those files backedup, you're done with
this volume.

If not, first upgrade to btrfs-progs 3.18.2, then do btrfs check
--repair --init-csum-tree to delete and recreate the csum tree, and
then doing another incremental. Be clear with labeling these
incremental backups. Later you can diff them, and if any files don't
match between them, manually inspect to find out which one is the good
one. I'd say 50/50 chance the init-csum-tree won't work because it
looks like sdc1 always produces bad data. It's entirely possible the
repair goes badly, and the filesystem becomes read-only at which point
no more changes will be possible. To get files off that fail csum
(again, they're listed in dmesg), you'll have to use btrfs restore on
the unmount volume to extract them. This may be tedious.




-- 
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
Thank you very much for your help. I do not have any recovery backup
and I need these data :(

Before my problems begun I was running btrfs-scrub in a weekly basis
and I only got 17 uncorrectable errors for this array, concerning
files that I do not care of, so I ignored them. I clearly should not.
I'll probably need your help again, in which case I will let you know.

I get:
# for letter in i d c g a b; do smartctl -l scterc /dev/sd$letter; done
smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
   Read: 70 (7.0 seconds)
  Write: 70 (7.0 seconds)

smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
   Read: 70 (7.0 seconds)
  Write: 70 (7.0 seconds)

- only the last two are WD RED drives.

# for letter in i d c g a b; do cat /sys/block/sd$letter/device/timeout; done
30
30
30
30
30
30



On Sun, Feb 8, 2015 at 10:43 PM, Chris Murphy li...@colorremedies.com wrote:
 If you don't have a current backup, make one now.

The only way I can think of making a backup with my current available
hardware is removing my one WD Red 6TB from the array and copying
every file on this removed disk.
Can I remove the /dev/sdb without letting any of the data enter the
soon-to-fail /dev/sdc1?

 Just make sure you
 don't overwrite any previous backup data in case you need it. Any
 files that don't pass checksum will not be copied, these will be
 recorded in dmesg. If you have those files backedup, you're done with
 this volume.

 If not, first upgrade to btrfs-progs 3.18.2, then do btrfs check
 --repair --init-csum-tree to delete and recreate the csum tree, and
 then doing another incremental. Be clear with labeling these
 incremental backups. Later you can diff them, and if any files don't
 match between them, manually inspect to find out which one is the good
 one. I'd say 50/50 chance the init-csum-tree won't work because it
 looks like sdc1 always produces bad data. It's entirely possible the
 repair goes badly, and the filesystem becomes read-only at which point
 no more changes will be possible. To get files off that fail csum
 (again, they're listed in dmesg), you'll have to use btrfs restore on
 the unmount volume to extract them. This may be tedious.




 --
 Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
By the way, /dev/sdc just completed the extended offline test without
any error... I feel so confused,
constantine


On Sun, Feb 8, 2015 at 11:04 PM, constantine costas.magn...@gmail.com wrote:
 Thank you very much for your help. I do not have any recovery backup
 and I need these data :(

 Before my problems begun I was running btrfs-scrub in a weekly basis
 and I only got 17 uncorrectable errors for this array, concerning
 files that I do not care of, so I ignored them. I clearly should not.
 I'll probably need your help again, in which case I will let you know.

 I get:
 # for letter in i d c g a b; do smartctl -l scterc /dev/sd$letter; done
 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control command not supported

 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control command not supported

 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control command not supported

 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control command not supported

 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control:
Read: 70 (7.0 seconds)
   Write: 70 (7.0 seconds)

 smartctl 6.3 2014-07-26 r3976 [x86_64-linux-3.19.0-1-mainline] (local build)
 Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

 SCT Error Recovery Control:
Read: 70 (7.0 seconds)
   Write: 70 (7.0 seconds)

 - only the last two are WD RED drives.

 # for letter in i d c g a b; do cat /sys/block/sd$letter/device/timeout; done
 30
 30
 30
 30
 30
 30



 On Sun, Feb 8, 2015 at 10:43 PM, Chris Murphy li...@colorremedies.com wrote:
 If you don't have a current backup, make one now.

 The only way I can think of making a backup with my current available
 hardware is removing my one WD Red 6TB from the array and copying
 every file on this removed disk.
 Can I remove the /dev/sdb without letting any of the data enter the
 soon-to-fail /dev/sdc1?

 Just make sure you
 don't overwrite any previous backup data in case you need it. Any
 files that don't pass checksum will not be copied, these will be
 recorded in dmesg. If you have those files backedup, you're done with
 this volume.

 If not, first upgrade to btrfs-progs 3.18.2, then do btrfs check
 --repair --init-csum-tree to delete and recreate the csum tree, and
 then doing another incremental. Be clear with labeling these
 incremental backups. Later you can diff them, and if any files don't
 match between them, manually inspect to find out which one is the good
 one. I'd say 50/50 chance the init-csum-tree won't work because it
 looks like sdc1 always produces bad data. It's entirely possible the
 repair goes badly, and the filesystem becomes read-only at which point
 no more changes will be possible. To get files off that fail csum
 (again, they're listed in dmesg), you'll have to use btrfs restore on
 the unmount volume to extract them. This may be tedious.




 --
 Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Accepting discard to free space from disk images

2015-02-08 Thread Roman Mamedov
On Mon, 09 Feb 2015 00:17:49 -0500
Devon B. devo...@virtualcomplete.com wrote:

 Looking to use btrfs with disk images that contain ext4, xfs, and other
 possible filesystems.  One of my main concerns is reclaiming disk space
 when files are deleted on the disk image's filesystem.  Using fstrim on
 the mount point of the disk image or mounting the disk image (loop) with
 discard doesn't appear to pass the freed blocks to btrfs.
 
 The image file on the btrfs filesystem just keeps growing in size.  When
 hosting images on ext4, using fstrim or discard frees up the space on
 the host filesystem (ext4).
 
 I see discard is pretty well supported for btrfs to the underlying block
 devices but is there any way to properly free space on the btrfs host
 filesystem from overlying filesystems while they are online?

I use KVM (QEMU) with discard pass-through from the VM guest (discard=unmap
option), with the VM images stored on Btrfs. It works just fine, the disk
space used for the image file does shrink when the guest OS issues discards on
its FS. Maybe there is a difference in how KVM and the 'loop' module submit
discards to Btrfs?

-- 
With respect,
Roman
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs: cleanup: remove no-used alloc_chunk in btrfs_check_data_free_space()

2015-02-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com

int alloc_chunk is never used in this function, remove it.

Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com
---
 fs/btrfs/extent-tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index a684086..765e72a 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3679,7 +3679,7 @@ int btrfs_check_data_free_space(struct inode *inode, u64 
bytes)
struct btrfs_root *root = BTRFS_I(inode)-root;
struct btrfs_fs_info *fs_info = root-fs_info;
u64 used;
-   int ret = 0, committed = 0, alloc_chunk = 1;
+   int ret = 0, committed = 0;
 
/* make sure bytes are sectorsize aligned */
bytes = ALIGN(bytes, root-sectorsize);
@@ -3707,7 +3707,7 @@ again:
 * if we don't have enough free bytes in this space then we need
 * to alloc a new chunk.
 */
-   if (!data_sinfo-full  alloc_chunk) {
+   if (!data_sinfo-full) {
u64 alloc_target;
 
data_sinfo-force_alloc = CHUNK_ALLOC_FORCE;
-- 
1.8.5.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/4 v2] test-btrfs-devmgt.sh

2015-02-08 Thread Anand Jain
simple controllable test script to exercise btrfs devices and at the same time 
collect sysfs (optional) to test for regression.

v1-v2: add some clean ups

Anand Jain (4):
  Btrfs-progs: add regression tests for sysfs contents during btrfs
device management
  Btrfs-progs: adds simple to use seed test for test-btrfs-devmgt.sh
  Btrfs-progs: test-btrfs-devmgt.sh support if btrfs is root fs
  Btrfs-progs: test-btrfs-devmgt.sh have a safe defaults

 tests/test-btrfs-devmgt.sh | 935 +
 1 file changed, 935 insertions(+)
 create mode 100755 tests/test-btrfs-devmgt.sh

-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/4] Btrfs-progs: add regression tests for sysfs contents during btrfs device management

2015-02-08 Thread Anand Jain
This contains a series of btrfs device operations and at each operation
the btrfs sysfs contents are logged.  This helps to check if the patch
is affecting any of the btrfs sysfs contents.

OR This script can be used to test the only the device operations.

as of now there are 32 test cases

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 tests/test-btrfs-devmgt.sh | 863 +
 1 file changed, 863 insertions(+)
 create mode 100755 tests/test-btrfs-devmgt.sh

diff --git a/tests/test-btrfs-devmgt.sh b/tests/test-btrfs-devmgt.sh
new file mode 100755
index 000..2f75200
--- /dev/null
+++ b/tests/test-btrfs-devmgt.sh
@@ -0,0 +1,863 @@
+# * GNU General Public License v2. Copyright Oracle 2015
+
+
+# Series of btrfs device operation test cases.
+#
+
+# sysfs: test
+# sysfs contents are taken at reasonable places (but you may disable it).
+# So to compare with the next iteration with your kernel patch. so
+# you can check for the sysfs changes by running diff of TMP_FILE(s)
+
+# Changelog:
+# v1.0 init asj
+
+
+   #When you change something related to device
+   #remember to test on btrfs boot separately
+ #test0: btrfs boot test
+
+   #Replace:
+ #test1: raid1, replace normal
+ #test2: raid1, replace missing
+ #test3: NOP
+ #test4: raid1, replace missing, replace normal
+ #test5: raid1, replace dev2, replace dev1
+
+   #Add sprout, replace:
+ #test6: add sprout, replace seed
+ #test7: raid1 seed, add sprout, replace seed, replace sprout
+ #test8: 3 level nested seeds, add sprout, replace mid level seed
+ #test9: raid1, degraded seed mount, add sprout, replace missing, replace non 
missing seed
+ #test10: add sprout, replace sprout
+ #test11: raid1 seed, add sprout, replace sprout, replace sprout again
+ #test12: 3 level nested seeds, add sprout, replace sprout
+ #test13: degraded raid1 seed, add sprout, replace sprout
+
+   #Mount sprout, replace:
+ #test14: NOP
+ #test15: mount sprout, replace sprout
+ #test16: mount sprout, replace seed
+ #test17: Raid1, mount sprout, replace sprout
+ #test18: Raid1, mount sprout, replace seed
+ #test19: Raid1 degraded, mount sprout, replace sprout
+ #test20: Raid1 degraded, mount sprout, replace missing
+ #test21: 3 level nested seeds, mount sprout, replace mid level seed
+
+   #seed sprout test:
+ #test22: mount sprout, mount seed
+ #test23: clean, mount -o device sprout
+ #test24: raid1, mount sprout
+ #test25: clean, scan, mount sprout
+ #test26: raid1, clean, mount -o device sprout
+ #test26: raid1, clean, scan, mount sprout
+
+   #dev add del test:
+ #test27: dev add
+ #test28: dev del
+
+   #dev scan test:
+ #test29: scan mount
+ #test30: use -o mount
+
+   #subvol mount test:
+ #test31: mount, mount subvol
+
+   #remount test:
+ #test32: mount, remount
+
+
+
+
+# Devices are hard coded. sorry
+
+DEV0=/dev/sdb
+x=0
+if [ $x -eq 1 ]; then
+DEV1=/dev/sdc
+DEV2=/dev/sdd
+DEV3=/dev/sde
+DEV4=/dev/sdf
+DEV5=/dev/sdg
+else
+DEV1=/dev/sdd
+DEV2=/dev/sde
+DEV3=/dev/sdf
+DEV4=/dev/sdg
+DEV5=/dev/sdc
+fi
+
+TEST_FSID=1c52f894-0ead-43d6-847a-d42359f78370
+
+#Enable or disable sysfs data collection by set/unset the below
+#TMP_FILE=''
+TMP_FILE=`mktemp`
+
+ent_cont()
+{
+   echo -n Enter to continue: 
+   #read
+   echo wait for input is disabled, uncomment above to wait.
+}
+
+erase()
+{
+   for i in $DEV0 $DEV1 $DEV2 $DEV3 $DEV4 $DEV5; do wipefs -a $i  
/dev/null; done
+}
+
+clean()
+{
+   modprobe -r btrfs  modprobe btrfs
+}
+
+collect_sysfs()
+{
+   # see above to disable sysfs data collection
+   [[ -z $TMP_FILE ]]  return
+
+   echo  $1 -  $TMP_FILE
+   find /sys/fs/btrfs -type f -exec cat {} \; -print  $TMP_FILE
+}
+
+_mkfs.btrfs()
+{
+   mkfs.btrfs $*  /dev/null
+}
+
+
+test1()
+{
+   TEST=test1
+   erase
+   echo -e \n$TEST
+_mkfs.btrfs -L $TEST -U $TEST_FSID -d raid1 -m raid1 $DEV1 $DEV2 -f
+   collect_sysfs $TEST
+mount $DEV2 /btrfs
+   collect_sysfs $TEST
+btrfs rep start -B $DEV2 $DEV3 /btrfs -f
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+   clean
+   ent_cont
+}
+
+test2()
+{
+   TEST=test2
+   erase
+   echo -e \n$TEST
+_mkfs.btrfs -L $TEST -U $TEST_FSID -d raid1 -m raid1 $DEV1 $DEV2 -f
+   clean
+mount -o degraded $DEV1 /btrfs
+   collect_sysfs $TEST
+btrfs rep start -B 2 $DEV3 /btrfs -f
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+   clean
+   ent_cont
+}
+
+test4()
+{
+   TEST=test4
+   erase
+   echo -e \n$TEST
+_mkfs.btrfs -L $TEST -U $TEST_FSID -d raid1 -m raid1 $DEV1 $DEV2 -f
+   clean
+mount -o degraded $DEV1 /btrfs
+   collect_sysfs $TEST
+btrfs rep start -B 2 $DEV3 /btrfs -f
+   collect_sysfs $TEST
+btrfs rep start -B $DEV1 $DEV4 /btrfs -f
+   collect_sysfs $TEST
+umount /btrfs
+   

[PATCH 2/4] Btrfs-progs: adds simple to use seed test for test-btrfs-devmgt.sh

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 tests/test-btrfs-devmgt.sh | 76 +-
 1 file changed, 75 insertions(+), 1 deletion(-)

diff --git a/tests/test-btrfs-devmgt.sh b/tests/test-btrfs-devmgt.sh
index 2f75200..5facbdb 100755
--- a/tests/test-btrfs-devmgt.sh
+++ b/tests/test-btrfs-devmgt.sh
@@ -66,7 +66,10 @@
#remount test:
  #test32: mount, remount
 
-
+   #add sprout test:
+ #test33: add sprout
+ #test34: raid1 seed, add sprout
+ #test35: add sprout, umount, mount seed, mount sprout
 
 
 # Devices are hard coded. sorry
@@ -814,6 +817,73 @@ clean
ent_cont
 }
 
+test33()
+{
+   TEST=test33  erase  echo -e \n$TEST
+clean
+   collect_sysfs $TEST
+_mkfs.btrfs -L $TEST -U $TEST_FSID $DEV1 -f
+   collect_sysfs $TEST
+btrfstune -S 1 $DEV1
+mount $DEV1 /btrfs
+   collect_sysfs $TEST
+btrfs dev add $DEV2 /btrfs
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+clean
+   collect_sysfs $TEST
+
+   ent_cont
+}
+
+test34()
+{
+   TEST=test34  erase  echo -e \n$TEST
+clean
+   collect_sysfs $TEST
+_mkfs.btrfs -draid1 -mraid1 -L $TEST -U $TEST_FSID $DEV1 $DEV2 -f
+   collect_sysfs $TEST
+btrfstune -S 1 $DEV1
+mount $DEV1 /btrfs
+   collect_sysfs $TEST
+btrfs dev add $DEV3 /btrfs
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+clean
+   collect_sysfs $TEST
+
+   ent_cont
+}
+
+test35()
+{
+   TEST=test35  erase  echo -e \n$TEST
+clean
+   collect_sysfs $TEST
+_mkfs.btrfs -L $TEST -U $TEST_FSID $DEV1 -f
+   collect_sysfs $TEST
+btrfstune -S 1 $DEV1
+mount $DEV1 /btrfs
+   collect_sysfs $TEST
+btrfs dev add $DEV2 /btrfs
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+mount $DEV1 /btrfs
+   collect_sysfs $TEST
+mount $DEV2 /btrfs1
+   collect_sysfs $TEST
+umount /btrfs
+   collect_sysfs $TEST
+umount /btrfs1
+clean
+   collect_sysfs $TEST
+
+   ent_cont
+}
+
 test0()
 {
   echo Have you tested with btrfs boot, you can't do that here\n
@@ -860,4 +930,8 @@ sleep 2; test30
 sleep 2; test31
 sleep 2; test32
 
+sleep 2; test33
+sleep 2; test34
+sleep 2; test35
+
 [[ -z $TMP_FILE ]] || echo -e \nTMP_FILE= $TMP_FILE
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 4/4] Btrfs-progs: test-btrfs-devmgt.sh have a safe defaults

2015-02-08 Thread Anand Jain
After I burnt my fingers by testing on non test disks,
decided to have a safe defaults

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 tests/test-btrfs-devmgt.sh | 36 +++-
 1 file changed, 15 insertions(+), 21 deletions(-)

diff --git a/tests/test-btrfs-devmgt.sh b/tests/test-btrfs-devmgt.sh
index 55bc877..71d9ac4 100755
--- a/tests/test-btrfs-devmgt.sh
+++ b/tests/test-btrfs-devmgt.sh
@@ -9,9 +9,6 @@
 # So to compare with the next iteration with your kernel patch. so
 # you can check for the sysfs changes by running diff of TMP_FILE(s)
 
-# Changelog:
-# v1.0 init asj
-
 
#When you change something related to device
#remember to test on btrfs boot separately
@@ -74,21 +71,15 @@
 
 # Devices are hard coded. sorry
 
-DEV0=/dev/sdb
-x=0
-if [ $x -eq 1 ]; then
-DEV1=/dev/sdc
-DEV2=/dev/sdd
-DEV3=/dev/sde
-DEV4=/dev/sdf
-DEV5=/dev/sdg
-else
-DEV1=/dev/sdd
-DEV2=/dev/sde
-DEV3=/dev/sdf
-DEV4=/dev/sdg
-DEV5=/dev/sdc
-fi
+# Assign per your config, all 5 needed, replace might fail
+# if DEV5  DEV4  DEV3  DEV2  DEV1
+#DEV1=/dev/sdd
+#DEV2=/dev/sde
+#DEV3=/dev/sdf
+#DEV4=/dev/sdg
+#DEV5=/dev/sdc
+
+[[ -z $DEV1 ]] || [[ -z $DEV2 ]] || [[ -z $DEV3 ]] || [[ -z $DEV4 ]] || [[ -z 
$DEV5 ]]  echo Need to initialize DEVx as above here  exit
 
 TEST_FSID=1c52f894-0ead-43d6-847a-d42359f78370
 
@@ -97,7 +88,7 @@ TEST_FSID=1c52f894-0ead-43d6-847a-d42359f78370
 TMP_FILE=`mktemp`
 
 #If the btrfs is root fs as well then set this
-CANT_CLEAN='yes'
+#CANT_CLEAN='yes'
 #CANT_CLEAN=''
 
 ent_cont()
@@ -109,12 +100,15 @@ ent_cont()
 
 erase()
 {
-   for i in $DEV0 $DEV1 $DEV2 $DEV3 $DEV4 $DEV5; do wipefs -a $i  
/dev/null; done
+   for i in $DEV1 $DEV2 $DEV3 $DEV4 $DEV5; do wipefs -a $i  /dev/null; 
done
 }
 
 clean()
 {
-   [[ -z $CANT_CLEAN ]]  modprobe -r btrfs  modprobe btrfs
+   [[ -z $CANT_CLEAN ]]  return
+
+   ! modprobe -r btrfs  echo For btrfs boot set CANT_CLEAN to yes here 
above  exit
+   modprobe btrfs
 }
 
 collect_sysfs()
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/4] Btrfs-progs: test-btrfs-devmgt.sh support if btrfs is root fs

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 tests/test-btrfs-devmgt.sh | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/test-btrfs-devmgt.sh b/tests/test-btrfs-devmgt.sh
index 5facbdb..55bc877 100755
--- a/tests/test-btrfs-devmgt.sh
+++ b/tests/test-btrfs-devmgt.sh
@@ -96,6 +96,10 @@ TEST_FSID=1c52f894-0ead-43d6-847a-d42359f78370
 #TMP_FILE=''
 TMP_FILE=`mktemp`
 
+#If the btrfs is root fs as well then set this
+CANT_CLEAN='yes'
+#CANT_CLEAN=''
+
 ent_cont()
 {
echo -n Enter to continue: 
@@ -110,7 +114,7 @@ erase()
 
 clean()
 {
-   modprobe -r btrfs  modprobe btrfs
+   [[ -z $CANT_CLEAN ]]  modprobe -r btrfs  modprobe btrfs
 }
 
 collect_sysfs()
-- 
1.9.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/1] export symbol kobject_move()

2015-02-08 Thread Anand Jain
drivers/cpufreq/cpufreq.c is already using this function. And now btrfs
needs it as well. export symbol kobject_move().
---
 lib/kobject.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/kobject.c b/lib/kobject.c
index 58751bb..e055c06 100644
--- a/lib/kobject.c
+++ b/lib/kobject.c
@@ -548,6 +548,7 @@ out:
kfree(devpath);
return error;
 }
+EXPORT_SYMBOL_GPL(kobject_move);
 
 /**
  * kobject_del - unlink kobject from hierarchy.
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123

2015-02-08 Thread Qu Wenruo


 Original Message 
Subject: kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123
From: Jeroen Van den Keybus jeroen.vandenkey...@gmail.com
To: linux-btrfs@vger.kernel.org
Date: 2015年02月09日 06:14

Hi,


I have a LUKS encrypted raw external (USB) disk mapped to
/dev/mapper/sd.backup. This mapped device was default btrfs formatted.
I can mount the mapped device to /mnt/backup. There used to be a
subvolume in /mnt/backup, which I deleted.

I now seem unable to either add a directory to /mnt/backup or umount
/mnt/backup; the command never finishes and the dmesg log reports:

[17937.939438] [ cut here ]
[17937.939523] kernel BUG at /home/apw/COD/linux/fs/btrfs/inode.c:3123!
[17937.939602] invalid opcode:  [#1] SMP
[17937.939664] Modules linked in: xts gf128mul rfcomm bluetooth joydev
hid_logitech_dj xt_multiport nfsd auth_rpcgss nfs_acl nfs lockd grace
sunrpc fscache dm_crypt xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4
xt_physdev br_netfilter xt_tcpudp xt_conntrack iptable_filter
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack ip_tables ebtable_filter ebtable_broute bridge stp llc
ebtables x_tables eeepc_wmi asus_wmi sparse_keymap video kvm_amd kvm
pl2303 usbserial serio_raw k10temp snd_usb_audio snd_usbmidi_lib
snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic
snd_hda_intel sp5100_tco snd_hda_controller i2c_piix4 snd_hda_codec
snd_hwdep snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq
snd_seq_device snd_timer snd shpchp soundcore 8250_fintek mac_hid
parport_pc
[17937.940798]  ppdev lp parport nct6775 nls_iso8859_1 hwmon_vid btrfs
raid10 raid1 multipath linear raid0 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx hid_generic usbhid hid xor
raid6_pq psmouse radeon uas usb_storage r8169 i2c_algo_bit ttm mii
drm_kms_helper drm wmi ahci libahci
[17937.941259] CPU: 1 PID: 16331 Comm: btrfs-cleaner Not tainted
3.18.3-031803-generic #201501161810
[17937.941358] Hardware name: System manufacturer System Product
Name/E45M1-M PRO, BIOS 0801 01/10/2012
[17937.941464] task: 88023383a800 ti: 880183868000 task.ti:
880183868000
[17937.941547] RIP: 0010:[c04dd109]  [c04dd109]
btrfs_orphan_add+0x1a9/0x1c0 [btrfs]
[17937.941705] RSP: 0018:88018386bc98  EFLAGS: 00010286
[17937.941769] RAX: ffe4 RBX: 8801ebe6b000 RCX: 
[17937.941849] RDX: 5cb8 RSI: 0004 RDI: 8801b7f85138
[17937.941928] RBP: 88018386bcd8 R08: 88023ed1db40 R09: 88019ab07b40
[17937.942007] R10:  R11: 0010 R12: 880233513630
[17937.942086] R13: 880041414d58 R14: 8801ebe6b458 R15: 0001
[17937.942168] FS:  7f8d60824740() GS:88023ed0()
knlGS:
[17937.942261] CS:  0010 DS:  ES:  CR0: 8005003b
[17937.942327] CR2: 7f70b2ec CR3: 000124627000 CR4: 07e0
[17937.942406] Stack:
[17937.942435]  88018386bcd8 c051bd4f 8801b93b9000
880204bbe200
[17937.942537]  8801b93b9000 88019ab07b40 880233513630
0001
[17937.942640]  88018386bd58 c04c5310 8801b7f85000
0004c04abffa
[17937.942742] Call Trace:
[17937.942819]  [c051bd4f] ?
lookup_free_space_inode+0x4f/0x100 [btrfs]
[17937.942934]  [c04c5310]
btrfs_remove_block_group+0x140/0x490 [btrfs]
[17937.943056]  [c0500065] btrfs_remove_chunk+0x245/0x380 [btrfs]
[17937.943163]  [c04c5896] btrfs_delete_unused_bgs+0x236/0x270 [btrfs]
[17937.943272]  [c04ced6c] cleaner_kthread+0x12c/0x190 [btrfs]
[17937.943374]  [c04cec40] ?
btrfs_destroy_all_delalloc_inodes+0x120/0x120 [btrfs]
[17937.943471]  [85093a49] kthread+0xc9/0xe0
[17937.943531]  [85093980] ? flush_kthread_worker+0x90/0x90
[17937.943608]  [857b3b7c] ret_from_fork+0x7c/0xb0
[17937.943673]  [85093980] ? flush_kthread_worker+0x90/0x90
[17937.943746] Code: e8 4d 9f fc ff 8b 45 c8 e9 6d ff ff ff 0f 1f 44
00 00 f0 41 80 65 80 fd 4c 89 ef 89 45 c8 e8 bf 1e fe ff 8b 45 c8 e9
48 ff ff ff 0f 0b 4c 89 f7 45 31 f6 e8 ea 64 2d c5 e9 f9 fe ff ff 0f
1f 44
[17937.944273] RIP  [c04dd109] btrfs_orphan_add+0x1a9/0x1c0 [btrfs]
[17937.944392]  RSP 88018386bc98
[17937.944503] ---[ end trace cee2bcd2393b84fb ]---
The BUG_ON in btrfs_orphan_add() seems have already been fixed by the 
patch from Forrest Liu.
[PATCH] Btrfs: fix BUG_ON in btrfs_orphan_add() when delete unused block 
group


https://patchwork.kernel.org/patch/5759741/

Thanks,
Qu

$ uname -a:
Linux zacate 3.18.3-031803-generic #201501161810 SMP Fri Jan 16
18:12:22 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I also ran '$ sudo btrfs check /dev/mapper/sd.backup':
Checking filesystem on /dev/mapper/sd.backup
UUID: 15628b78-7380-4d57-bc77-426016205aa0
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 35881083 

Accepting discard to free space from disk images

2015-02-08 Thread Devon B.
Looking to use btrfs with disk images that contain ext4, xfs, and other
possible filesystems.  One of my main concerns is reclaiming disk space
when files are deleted on the disk image's filesystem.  Using fstrim on
the mount point of the disk image or mounting the disk image (loop) with
discard doesn't appear to pass the freed blocks to btrfs.

The image file on the btrfs filesystem just keeps growing in size.  When
hosting images on ext4, using fstrim or discard frees up the space on
the host filesystem (ext4).

I see discard is pretty well supported for btrfs to the underlying block
devices but is there any way to properly free space on the btrfs host
filesystem from overlying filesystems while they are online?

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: [PATCH v2] btrfs: Fix out-of-space bug

2015-02-08 Thread Zhao Lei
Hi, Filipe

  Changelog v1-v2:
   V1 will introduce a new bug when create a metadata bg in space of
   old data bg which was just removed, noticed by:
   Filipe David Manana fdman...@gmail.com
   V2 fix this bug by commit transaction after remove block grops.
 
 Well it's not specific to reusing the space from a deleted metadata
 block group for a new data block group. This is true for any other
 combination, even data - data. When using COW for both metadata and
 data, if a crash happens before the transaction commits, it is
 guaranteed that all data and metadata committed in the previous
 transaction are available on the next mount (and all new data in the
 current transaction if it was fsync'ed) - this applies to any COW
 system, be it a filesystem or a database for example.
 
Yes, wil change above description.

 
  Tested for severial times by above script.
 
  Reported-by: Tsutomu Itoh t-i...@jp.fujitsu.com
  Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com
  ---
   fs/btrfs/extent-tree.c | 4 
   1 file changed, 4 insertions(+)
 
  diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
  index a684086..67c85ff 100644
  --- a/fs/btrfs/extent-tree.c
  +++ b/fs/btrfs/extent-tree.c
  @@ -9653,6 +9653,10 @@ next:
  spin_lock(fs_info-unused_bgs_lock);
  }
  spin_unlock(fs_info-unused_bgs_lock);
  +   trans = btrfs_join_transaction(root);
  +   if (!IS_ERR(trans))
  +   btrfs_commit_transaction(trans, root);
  +
 
 At the very least, before doing the commit, you should verify if any
 block groups were actually deleted - under some conditions we skip
 their deletion in the loop. 
Good suggestion, it can reduce useless committing.

 Plus, it would be good to check if the sum
 of the sizes of the deleted block groups exceeds some minimum
 threshold (lets say 1G or 2G, whatever), so that we don't get the
 overhead of committing a transaction for little or no gains.

Chunks are large enough in normal-size btrfs filesystem(size  10G), or
worthed to be deleted in small-size filesystem, which have chunk
size = 128M but large percent of total size.

And it is easy to trigger above problem in case that we ignore
deleting 128M chunks when fs is empty.

So I think always commit transaction when delete empty chunks
will make logic simple, but ... , see below.

 I would also consider doing the commit only if the fs is really
 running out of free space or close to being out of free space.
 
Maybe the better way is to delay deleting-chunks.

The patch of delete empty bgs is used to fix problem that
no space for allocate metadata chunks but data chunks are free.
It is to say, we only need to free data chunks when we have no space
for metadata(or conversely), and need not always delete free chunks.
Do delete-bg in allow_chunk() will be better.

But it is complex than current method, I want to avoid new bugs in rc,
to give users a stable version in 3.19.

 You can get into many similar cases of space not being released until
 the transaction commits.
 For example, create 10 files with a size of 1Gb each (via fallocate
 for e.g.), and then truncate them all to a size of 1M in the same
 transaction for example. You'll get 999Mb * 10 of space that won't be
 available for use until the current transaction commits - it's exactly
 the same problem you found, it just doesn't imply deleting whole block
 groups/chunks.
 
This bug is more serious than above.

In my test, when NO_SPACE happened, we can not write into filesystem
forever(may be something with space_info-full), so I wish to solve it
before 3.19.

Thanks
Zhaolei

 Thanks
 
   }
 
   int btrfs_init_space_info(struct btrfs_fs_info *fs_info)
  --
  1.8.5.1
 
  --
  To unsubscribe from this list: send the line unsubscribe linux-btrfs in
  the body of a message to majord...@vger.kernel.org
  More majordomo info at  http://vger.kernel.org/majordomo-info.html
 
 
 
 --
 Filipe David Manana,
 
 Reasonable men adapt themselves to the world.
  Unreasonable men adapt the world to themselves.
  That's why all progress depends on unreasonable men.
 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Brendan Hide

On 2015/02/09 01:58, constantine wrote:

Second, SMART is only saying its internal test is good. The errors are
related to data transfer, so that implicates the enclosure (bridge
chipset or electronics), the cable, or the controller interface.
Actually it could also be a flaky controller or RAM on the drive
itself too which I don't think get checked with SMART tests.

Which test should I do from now on (on a weekly basis?) so as to
prevent similar things from happening?
Chris has given some very good info so far. I've also had to learn some 
of this stuff the hard way (failed/unreliable drives, data 
unavailable/lost, etc). The info below will be of help to you in 
following some of the advice already given. Unfortunately, the best 
course of action I see so far is to follow Chris' advice and to purchase 
more disks so you can make a backup ASAP.


I have the following two lines in 
/etc/udev/rules.d/61-persistent-storage.rules for two old 250GB 
spindles. It sets the timeout to 120 seconds because these two disks 
don't support SCT ERC. This may very well apply without modification to 
other distros - but this is only tested in Arch:
ACTION==add, KERNEL==sd*, SUBSYSTEM==block, 
ENV{ID_SERIAL}=ST3250410AS_6RYF5NP7 RUN+=/bin/sh -c 'echo 120  
/sys$devpath/device/timeout'
ACTION==add, KERNEL==sd*, SUBSYSTEM==block, 
ENV{ID_SERIAL}=ST3250820AS_9QE2CQWC RUN+=/bin/sh -c 'echo 120  
/sys$devpath/device/timeout'


I have a smart_scan script* that does a check of all disks using 
smartctl. The meat of the script is in main(). The rest of the script is 
from a template of mine. The script, with no parameters, will do a short 
and then a long test on all drives. It does not give any output - 
however if you have smartd running and configured appropriately, smartd 
will pick up on any issues found and send appropriate alerts 
(email/system log/etc).


It is configured in /etc/cron.d/smart. It runs a short test every 
morning and a long test every Saturday evening:

255***root/usr/local/sbin/smart_scan short
2518**6root/usr/local/sbin/smart_scan long

Then, scrubbing**:
This relatively simple script runs a scrub on all disks and prints the 
results *only* if there were errors.
I've scheduled this in a cron as well to execute *every* morning shortly 
after 2am. Cron is configured to send me an email if there is any output 
- so I only get an email if there's something to look into.


And finally, I have btsync configured to synchronise my Arch desktop's 
system journal to a couple of local and remote servers of mine. A much 
cleaner way to do this would be to use an external syslog server - I 
haven't yet looked into doing that properly, however.


http://swiftspirit.co.za/down/smart_scan
http://swiftspirit.co.za/down/btrfs-scrub-all

--
__
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Duncan
Chris Murphy posted on Sun, 08 Feb 2015 15:43:35 -0700 as excerpted:

 Confusing is that sdd1, sdi1, sdg1 have gen 0 and also have corruptions
 reported, just not anywhere near as many as sdc1. So I don't know what
 problems you have with your hardware, but they're not restricted to just
 one or two drives. Generation 0 makes no sense to me.

There was a btrfs bug along about kernel 3.17 (or was it 3.18), that 
reset the generation count.  Several people posted weird generation 
outputs within a few days, and then there was a fast bug-fix that went 
into the new kernel and stable for the previous kernel (which IIRC was 
the only one affected), almost before we'd figured out that the reports 
were connected and it was a definite bug!

The good news is that the devs are getting MUCH faster at fixing these 
things -- the first widespread bug from a few kernels ago took just over 
a full kernel cycle to trace down and fix, the next one was fixed in the 
same kernel cycle, this one was traced and fixed so fast that even tho we 
had a number of reports, the devs found and fixed the bug at about the 
same time most list users even became aware that there was a bug in the 
first place!

The bad news is that because it was found and fixed so fast, you 
evidently weren't aware of it at all or you'd have seen it in these zero-
gen reports here, and while I was aware of it, I have less details than 
on the other bugs.

Meanwhile, more good news in that there's a fix, and btrfs check from the 
latest btrfs-progs 3.18.x should have no problems fixing it -- on an 
otherwise normal btrfs, anyway.  But the bad news is that I don't know if 
it'll handle it with the one missing device and one failing, as seems to 
be the case here.

So to the problem at hand, and mostly addressing constantine (OP) now...

The lack of a good backup in this case is... unfortunate.  I'd be 
surprised if there's any list regulars that trust the still stabilizing 
btrfs without backups for any data they value, yet.  As any good sysadmin 
(and by that I mean anyone administering systems, professional or not, 
IOW, anyone making this sort of decision is a sysadmin) knows, if the 
data's not backed up (and a good sysadmin knows a backup isn't complete 
until it's tested to whatever level they're comfortable with restoring 
from), by definition, the data is considered of low enough value that 
it's not worth the backup hassle and the admin is prepared to do without 
it as a fair price to pay for avoiding the hassle of backup.  And that's 
the general rule, applying even to stable and mature filesystems.  
Because btrfs is still maturing and isn't yet fully stable, the rule 
applies to btrfs more than ever, so indeed, if it's not backed up, by 
definition, you don't care about losing the data.

Which interpreted strictly, means since there wasn't a backup here, the 
data is by definition of actions of the person responsible, worth less 
than the hassle of backing it up, which by definition means it's loss is 
no big deal, no matter any claims to the contrary.  That being the case, 
in the strict sense, call it a loss and be on to something more important.

But back here dealing in reality... well let's just try to retrieve what 
can be retrieved.

As Chris Murphy has said, that means backing up what you can now.  If at 
all possible leave the existing filesystem as it is when you do that (and 
preferably mount it read-only for now, to minimize the chance of anything 
else going bad on it in the mean time), which means... if you have to get 
out the credit card to get enough storage to create a backup... well, 
it's that or very high risk of losing even more of the data by taking 
another drive out of the array and using it for backup.  This is serious 
rock and hard place time, now.

What fails to backup should show in the log.  Hopefully it's not too much 
and not too valuable.  If you can do without it, great.  Otherwise, again 
as Chris suggested, try btrfs restore on the unmounted filesystem... and 
hope.  Because worked properly, restore can let you recover earlier 
states of the files in question, you may be able to do that for otherwise 
unrecoverable files, even if you can't get the current version.

Personally, I'd try those before trying the checksums reset he 
suggested.  Because that's writing to the damaged filesystem, which of 
course means possibly losing more if things go wrong.  Only if both 
normal backup of what's possible, and restore, can't retrieve critical 
files, would I even bother with the checksum zeroing and further rescue 
efforts.


Meanwhile, when you're done and everything's settled down again, 
regardless of whether you stick with btrfs or revert to a more mature 
filesystem, for the love of your data, PLEASE backup anything you 
consider important.  Even if you don't regularly update those backups, a 
backup of most of it losing the last month or six of work still gives you 
SOMEWHERE to start!  Because if it's not 

[PATCH] btrfs-progs: fsck-test: Add check_sudo to check valid root/sudo privilege

2015-02-08 Thread Qu Wenruo
Although fsck-test/012 uses sudo, it uses 'sudo -n', which won't prompt
user to input password and will return 1 if no valid credential is
found.

And this makes test result quite annoying since it fails to mount and
still continue, which will always fail.

This patch introduced the new check_sudo() to check sudo before calling
$sudo. This function will check sudo -v -n to get the credential.
And if it fails, then the test will not be run.

Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
 tests/common | 18 ++
 tests/fsck-tests/012-leaf-corruption/test.sh |  9 +
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/tests/common b/tests/common
index d7d2e9b..56ae0e6 100644
--- a/tests/common
+++ b/tests/common
@@ -62,3 +62,21 @@ setup_root_helper()
fi
have_root_helper=1
 }
+
+check_sudo()
+{
+   if [ $UID -eq 0 ]; then
+   return 0
+   elif [ $have_root_helper ]; then
+   $sudo -v  /dev/null
+   return $?
+   else
+   return 1
+   fi
+}
+
+not_run_lack_privlege ()
+{
+   echo [NOTRUN] root or valid sudo credential needed
+   exit 0
+}
diff --git a/tests/fsck-tests/012-leaf-corruption/test.sh 
b/tests/fsck-tests/012-leaf-corruption/test.sh
index 896f717..7396b05 100755
--- a/tests/fsck-tests/012-leaf-corruption/test.sh
+++ b/tests/fsck-tests/012-leaf-corruption/test.sh
@@ -55,12 +55,14 @@ check_inode()
name=$5
 
# Check whether the inode exists
+   check_sudo || not_run_lack_privlege
exists=$($sudo find $path -inum $ino)
if [ -z $exists ]; then
_fail inode $ino not recovered correctly
fi
 
# Check inode type
+   check_sudo || not_run_lack_privlege
found_mode=$(printf %o 0x$($sudo stat $exists -c %f))
if [ $found_mode -ne $mode ]; then
echo $found_mode
@@ -68,6 +70,7 @@ check_inode()
fi
 
# Check inode size
+   check_sudo || not_run_lack_privlege
found_size=$($sudo stat $exists -c %s)
if [ $mode -ne 41700 -a $found_size -ne $size ]; then
_fail inode $ino size not recovered correctly
@@ -85,15 +88,12 @@ check_inode()
 check_leaf_corrupt_no_data_ext()
 {
image=$1
-   if [ $have_root_helper -ne 1 ]; then
-   echo  [NOTRUN] root privileges needed to verify recovery
-   exit 0
-   fi
if [ -z $TEST_MNT ]; then
echo \$TEST_MNT not set, use $(pwd)/tmp as fallback
TEST_MNT=$(pwd)/tmp
fi
mkdir -p $TEST_MNT || _fail failed to create mount point
+   check_sudo || not_run_lack_privlege
$sudo mount $image -o ro $TEST_MNT
 
i=0
@@ -106,6 +106,7 @@ check_leaf_corrupt_no_data_ext()
${leaf_no_data_ext_list[i + 4]}
((i+=4))
done
+   check_sudo || not_run_lack_privlege
$sudo umount $TEST_MNT
 }
 
-- 
2.3.0

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs: cleanup: remove no-used alloc_chunk in btrfs_check_data_free_space()

2015-02-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com

int alloc_chunk is never used in this function, remove it.

Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com
---
 fs/btrfs/extent-tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index a684086..765e72a 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3679,7 +3679,7 @@ int btrfs_check_data_free_space(struct inode *inode, u64 
bytes)
struct btrfs_root *root = BTRFS_I(inode)-root;
struct btrfs_fs_info *fs_info = root-fs_info;
u64 used;
-   int ret = 0, committed = 0, alloc_chunk = 1;
+   int ret = 0, committed = 0;
 
/* make sure bytes are sectorsize aligned */
bytes = ALIGN(bytes, root-sectorsize);
@@ -3707,7 +3707,7 @@ again:
 * if we don't have enough free bytes in this space then we need
 * to alloc a new chunk.
 */
-   if (!data_sinfo-full  alloc_chunk) {
+   if (!data_sinfo-full) {
u64 alloc_target;
 
data_sinfo-force_alloc = CHUNK_ALLOC_FORCE;
-- 
1.8.5.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
On Sun, Feb 8, 2015 at 4:58 PM, constantine costas.magn...@gmail.com wrote:

 Which test should I do from now on (on a weekly basis?) so as to
 prevent similar things from happening?

You should check dmesg after each scrub and at each mount. You can also use

btrfs scrub -Bd mountpoint

This means scrub won't run in the background and will report detailed
errors at the end. -R includes a bit more information. You can also
get lifetime (or since last reset) stats per device:

btrfs device stats mountpoint

And use -z to reset.


-- 
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
I understood my mistake on using consumer drives and this is why I
bought the RED versions some days ago. I would have done this earlier
if I had the money.

So to sum up.

I have upgraded my btrfs-progs and I have mounted the filesystem with
# mount -o degraded /dev/sdi1 /mnt/mountpoint


I want to minimize my risk.

Now I should do
# btrfs device delete /dev/sdc1 ?
or
# btrfs check --repair --init-csum-tree ?


constantine


On Sun, Feb 8, 2015 at 11:34 PM, Chris Murphy li...@colorremedies.com wrote:
 On Sun, Feb 8, 2015 at 4:09 PM, constantine costas.magn...@gmail.com wrote:
 By the way, /dev/sdc just completed the extended offline test without
 any error... I feel so confused,

 First, we know from a number of studies, including the famous (and now
 kinda old) Google study that  a huge percent of drive failures come
 with no SMART errors.

 Second, SMART is only saying its internal test is good. The errors are
 related to data transfer, so that implicates the enclosure (bridge
 chipset or electronics), the cable, or the controller interface.
 Actually it could also be a flaky controller or RAM on the drive
 itself too which I don't think get checked with SMART tests.

 --
 Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
On Sun, Feb 8, 2015 at 4:53 PM, constantine costas.magn...@gmail.com wrote:
 I understood my mistake on using consumer drives and this is why I
 bought the RED versions some days ago. I would have done this earlier
 if I had the money.

You need to raise the SCSI command timer value for the drives that
don't support SCT ERC. That way the kernel won't reset the link when
the drive hangs on a read error; this will allow the drive to report
the read error and allow Btrfs to fix the problem. If you don't do
this, long recover problems on these drives simply get worse until
there'd data loss. And as far as I know, Btrfs doesn't create another
copy in such a case.


 So to sum up.

 I have upgraded my btrfs-progs and I have mounted the filesystem with
 # mount -o degraded /dev/sdi1 /mnt/mountpoint


 I want to minimize my risk.

Backup the fking volume first. And whatever files don't backup, you
have to use btrfs restore on them if you ever want to retrieve them.


 Now I should do
 # btrfs device delete /dev/sdc1 ?
 or
 # btrfs check --repair --init-csum-tree ?

Try the first one. Just realize that any data that's on both the
failed drive and sdc1 will be lost, which is why you must have a
backup or use btrfs restore to start out. There is no way to re-add
sdc1 once you remove it.

-- 
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
So I just created a small 5 device btrfs raid1 volume, and added some
data to it so all the devices have been written to. Then unmounted the
volume, and destroyed one of the devices. So now I have a 4 device
raid1 mounted degraded. And I can still device delete another device.
So one device missing and one device removed. Shortly thereafter I get
a kernel panic. This is 3.19rc7.

https://bugzilla.kernel.org/show_bug.cgi?id=92921


Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 02/24] Btrfs: sysfs: fix, fs_info kobject_unregister has init_completion() twice

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

kobject_unregister is to handle the release of the kobject,
its completion init is being called in btrfs_sysfs_add_one(),
so we don't have to do the same in the open_ctree() again.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/disk-io.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 8c63419..0cd6550 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2238,7 +2238,6 @@ int open_ctree(struct super_block *sb,
mutex_init(fs_info-delalloc_root_mutex);
seqlock_init(fs_info-profiles_lock);
 
-   init_completion(fs_info-kobj_unregister);
INIT_LIST_HEAD(fs_info-dirty_cowonly_roots);
INIT_LIST_HEAD(fs_info-space_info);
INIT_LIST_HEAD(fs_info-tree_mod_seq_list);
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 00/24 V2] provide frame work so that sysfs attributs from the fs_devices can be added

2015-02-08 Thread Anand Jain
This patch set will provide a framework and help to create attributes
from the structure btrfs_fs_devices which are available even before
fs_info is created. So by moving the parent kobject super_kobj from
fs_info to btrfs_fs_devices, it will help to create attributes
from the btrfs_fs_devices as well.

Just to note, this does not change any of the existing btrfs sysfs
external kobject names and its attributes and not even the life
cycle of them. Changes are internal only. And to ensure the same,
this path has been tested with various device operations and,
checking and comparing the sysfs kobjects and attributes with
sysfs kobject and attributes with out this patch, and they remain
same. These test cases are added to the progs as test-btrfs-devmgt.sh,
its patch is below as well.

v1-v2:
  . adds fresh set of patches from 12 to 24 to support seed fsids
  on btrfs sysfs layout and the related frame work changes
  . the patches 1 to 12 has no changes from the previous submit

Anand Jain (24):
  Btrfs: sysfs: fix, btrfs_release_super_kobj() should to clean up the
kobject data
  Btrfs: sysfs: fix, fs_info kobject_unregister has init_completion()
twice
  Btrfs: sysfs: fix, undo sysfs device links
  Btrfs: sysfs: fix, kobject pointer clean up needed after kobject
release
  Btrfc: sysfs: fix, check if device_dir_kobj is init before destroy
  Btrfs: sysfs: reorder the kobject creations
  Btrfs: sysfs: rename __btrfs_sysfs_remove_one to
btrfs_sysfs_remove_fsid
  Btrfs: sysfs: introduce function btrfs_sysfs_add_fsid() to create
sysfs fsid
  Btrfs: sysfs: let default_attrs be separate from the kset
  Btrfs: sysfs: separate device kobject and its attribute creation
  Btrfs: sysfs: move super_kobj and device_dir_kobj from fs_info to
btrfs_fs_devices
  Btrfs: sysfs: add pointer to access fs_info from fs_devices
  Btrfs: introduce btrfs_get_fs_uuids
  Btrfs: sysfs: provide framework to remove all fsid kobject
  Btrfs: sysfs btrfs_kobj_add_device() pass fs_devices instead of
fs_info
  Btrfs: sysfs btrfs_kobj_rm_device() pass fs_devices instead of fs_info
  Btrfs: sysfs: make btrfs_sysfs_add_fsid() non static
  Btrfs: sysfs: make btrfs_sysfs_add_device() non static
  Btrfs: sysfs: btrfs_sysfs_remove_fsid() make it non static
  Btrfs: sysfs: separate kobject and attribute creation
  Btrfs: sysfs: add support to add parent for fsid
  Btrfs: sysfs: don't fail seeding for the sake of sysfs kobject issue
  Btrfs: sysfs: support seed devices in the sysfs layout
  Btrfs: sysfs: add check if super kobject is already initialized

 fs/btrfs/ctree.h   |   3 -
 fs/btrfs/dev-replace.c |   4 +-
 fs/btrfs/disk-io.c |  19 +-
 fs/btrfs/sysfs.c   | 180 ++---
 fs/btrfs/sysfs.h   |  12 ++--
 fs/btrfs/volumes.c |  46 +++--
 fs/btrfs/volumes.h |   7 ++
 7 files changed, 214 insertions(+), 57 deletions(-)

-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 01/24] Btrfs: sysfs: fix, btrfs_release_super_kobj() should to clean up the kobject data

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

The following test case fails indicating that, thread tried to init an 
initialized object.

kernel: [232104.016513] kobject (880006c1c980): tried to init an 
initialized object, something is seriously wrong.

btrfs_sysfs_remove_one() self test code:

open_tree()
{
 ::
ret = btrfs_sysfs_add_one(fs_info);
if (ret) {
  pr_err(BTRFS: failed to init sysfs interface: %d\n, ret);
goto fail_block_groups;
}
+   btrfs_sysfs_remove_one(fs_info);
+   ret = btrfs_sysfs_add_one(fs_info);
+   if (ret) {
+   pr_err(BTRFS: failed to init sysfs interface: %d\n, ret);
+   goto fail_block_groups;
+   }

cleaning up the unregistered kobject fixes this.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 92db3f6..68dcd17 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -439,6 +439,8 @@ static struct attribute *btrfs_attrs[] = {
 static void btrfs_release_super_kobj(struct kobject *kobj)
 {
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
+
+   memset(fs_info-super_kobj, 0, sizeof(struct kobject));
complete(fs_info-kobj_unregister);
 }
 
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 04/24] Btrfs: sysfs: fix, kobject pointer clean up needed after kobject release

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

The sysfs clean up self test like in the below code fails, since
fs_info-device_dir_kobject still points to its stale kobject.
Reseting this pointer will help to fix this.

open_ctree()
{

ret = btrfs_sysfs_add_one(fs_info);
::
+   btrfs_sysfs_remove_one(fs_info);
+   ret = btrfs_sysfs_add_one(fs_info);
+   if (ret) {
+   pr_err(BTRFS: failed to init sysfs interface: %d\n, ret);
+   goto fail_block_groups;
+   }

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index adfac3e..15fead2 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -525,6 +525,7 @@ void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
btrfs_kobj_rm_device(fs_info, NULL);
kobject_del(fs_info-device_dir_kobj);
kobject_put(fs_info-device_dir_kobj);
+   fs_info-device_dir_kobj = NULL;
addrm_unknown_feature_attrs(fs_info, false);
sysfs_remove_group(fs_info-super_kobj, btrfs_feature_attr_group);
__btrfs_sysfs_remove_one(fs_info);
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 08/24] Btrfs: sysfs: introduce function btrfs_sysfs_add_fsid() to create sysfs fsid

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

We need it in a seperate function so that it can be called from the
device discovery thread as well.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 15 ++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index c923e8b..f42d8fd 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -690,7 +690,12 @@ static struct dentry *btrfs_debugfs_root_dentry;
 /* Debugging tunables and exported data */
 u64 btrfs_debugfs_test;
 
-int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
+/*
+ * Can be called by the device discovery thread.
+ * And parent can be specified for seed device
+ */
+int btrfs_sysfs_add_fsid(struct btrfs_fs_info *fs_info,
+   struct kobject *parent)
 {
int error;
 
@@ -698,6 +703,14 @@ int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
fs_info-super_kobj.kset = btrfs_kset;
error = kobject_init_and_add(fs_info-super_kobj, btrfs_ktype, NULL,
 %pU, fs_info-fsid);
+   return error;
+}
+
+int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
+{
+   int error;
+
+   error = btrfs_sysfs_add_fsid(fs_info, NULL);
if (error)
return error;
 
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 12/24] Btrfs: sysfs: add pointer to access fs_info from fs_devices

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

adds fs_info pointer with struct btrfs_fs_devices.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c   | 4 
 fs/btrfs/volumes.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index ac15fbb..4b5bac6 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -530,6 +530,8 @@ static void btrfs_sysfs_remove_fsid(struct btrfs_fs_devices 
*fs_devs)
 
 void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
 {
+   fs_info-fs_devices-fs_info = NULL;
+
if (fs_info-space_info_kobj) {
sysfs_remove_files(fs_info-space_info_kobj, allocation_attrs);
kobject_del(fs_info-space_info_kobj);
@@ -729,6 +731,8 @@ int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
struct btrfs_fs_devices *fs_devs = fs_info-fs_devices;
struct kobject *super_kobj = fs_devs-super_kobj;
 
+   fs_devs-fs_info = fs_info;
+
error = btrfs_sysfs_add_fsid(fs_devs, NULL);
if (error)
return error;
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index c2e5bd0..53fd278 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -254,6 +254,7 @@ struct btrfs_fs_devices {
 */
int rotating;
 
+   struct btrfs_fs_info *fs_info;
/* sysfs kobjects */
struct kobject super_kobj;
struct kobject *device_dir_kobj;
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 11/24] Btrfs: sysfs: move super_kobj and device_dir_kobj from fs_info to btrfs_fs_devices

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

This patch will provide a framework and help to create attributes
from the structure btrfs_fs_devices which are available even before
fs_info is created. So by moving the parent kobject super_kobj from
fs_info to btrfs_fs_devices, it will help to create attributes
from the btrfs_fs_devices as well.

Patches on top of this patch now will be able to create the
sys/fs/btrfs/fsid kobject and attributes from btrfs_fs_devices
when devices are scanned and registered to the kernel.

Just to note, this does not change any of the existing btrfs sysfs
external kobject names and its attributes and not even the life
cycle of them. Changes are internal only. And to ensure the same,
this path has been tested with various device operations and,
checking and comparing the sysfs kobjects and attributes with
sysfs kobject and attributes with out this patch, and they remain
same.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/ctree.h   |  3 --
 fs/btrfs/sysfs.c   | 88 ++
 fs/btrfs/volumes.c |  3 +-
 fs/btrfs/volumes.h |  5 
 4 files changed, 56 insertions(+), 43 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 7e60741..9493b91 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1580,10 +1580,7 @@ struct btrfs_fs_info {
struct task_struct *cleaner_kthread;
int thread_pool_size;
 
-   struct kobject super_kobj;
struct kobject *space_info_kobj;
-   struct kobject *device_dir_kobj;
-   struct completion kobj_unregister;
int do_barriers;
int closing;
int log_root_recovering;
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 2cb4c69..ac15fbb 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -33,6 +33,7 @@
 #include volumes.h
 
 static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj);
+static inline struct btrfs_fs_devices *to_fs_devs(struct kobject *kobj);
 
 static u64 get_features(struct btrfs_fs_info *fs_info,
enum btrfs_feature_set set)
@@ -438,10 +439,10 @@ static const struct attribute *btrfs_attrs[] = {
 
 static void btrfs_release_super_kobj(struct kobject *kobj)
 {
-   struct btrfs_fs_info *fs_info = to_fs_info(kobj);
+   struct btrfs_fs_devices *fs_devs = to_fs_devs(kobj);
 
-   memset(fs_info-super_kobj, 0, sizeof(struct kobject));
-   complete(fs_info-kobj_unregister);
+   memset(fs_devs-super_kobj, 0, sizeof(struct kobject));
+   complete(fs_devs-kobj_unregister);
 }
 
 static struct kobj_type btrfs_ktype = {
@@ -449,11 +450,18 @@ static struct kobj_type btrfs_ktype = {
.release= btrfs_release_super_kobj,
 };
 
+static inline struct btrfs_fs_devices *to_fs_devs(struct kobject *kobj)
+{
+   if (kobj-ktype != btrfs_ktype)
+   return NULL;
+   return container_of(kobj, struct btrfs_fs_devices, super_kobj);
+}
+
 static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj)
 {
if (kobj-ktype != btrfs_ktype)
return NULL;
-   return container_of(kobj, struct btrfs_fs_info, super_kobj);
+   return to_fs_devs(kobj)-fs_info;
 }
 
 #define NUM_FEATURE_BITS 64
@@ -494,12 +502,12 @@ static int addrm_unknown_feature_attrs(struct 
btrfs_fs_info *fs_info, bool add)
attrs[0] = fa-kobj_attr.attr;
if (add) {
int ret;
-   ret = sysfs_merge_group(fs_info-super_kobj,
+   ret = 
sysfs_merge_group(fs_info-fs_devices-super_kobj,
agroup);
if (ret)
return ret;
} else
-   sysfs_unmerge_group(fs_info-super_kobj,
+   
sysfs_unmerge_group(fs_info-fs_devices-super_kobj,
agroup);
}
 
@@ -507,18 +515,17 @@ static int addrm_unknown_feature_attrs(struct 
btrfs_fs_info *fs_info, bool add)
return 0;
 }
 
-static void btrfs_sysfs_remove_fsid(struct btrfs_fs_info *fs_info)
+static void btrfs_sysfs_remove_fsid(struct btrfs_fs_devices *fs_devs)
 {
-   if (fs_info-device_dir_kobj) {
-   btrfs_kobj_rm_device(fs_info, NULL);
-   kobject_del(fs_info-device_dir_kobj);
-   kobject_put(fs_info-device_dir_kobj);
-   fs_info-device_dir_kobj = NULL;
+   if (fs_devs-device_dir_kobj) {
+   kobject_del(fs_devs-device_dir_kobj);
+   kobject_put(fs_devs-device_dir_kobj);
+   fs_devs-device_dir_kobj = NULL;
}
 
-   kobject_del(fs_info-super_kobj);
-   kobject_put(fs_info-super_kobj);
-   wait_for_completion(fs_info-kobj_unregister);
+   kobject_del(fs_devs-super_kobj);
+   kobject_put(fs_devs-super_kobj);
+ 

[PATCH 21/24] Btrfs: sysfs: add support to add parent for fsid

2015-02-08 Thread Anand Jain
To support seed sysfs layout and represent seed fsid under
the sprout we need the facility to create fsid under the
specified parent.
---
 fs/btrfs/sysfs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index d0caa32..d89bf4d 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -729,8 +729,8 @@ int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
 
init_completion(fs_devs-kobj_unregister);
fs_devs-super_kobj.kset = btrfs_kset;
-   error = kobject_init_and_add(fs_devs-super_kobj, btrfs_ktype, NULL,
-%pU, fs_devs-fsid);
+   error = kobject_init_and_add(fs_devs-super_kobj,
+   btrfs_ktype, parent, %pU, fs_devs-fsid);
return error;
 }
 
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 15/24] Btrfs: sysfs btrfs_kobj_add_device() pass fs_devices instead of fs_info

2015-02-08 Thread Anand Jain
btrfs_kobj_add_device() does not need fs_info any more.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/dev-replace.c | 2 +-
 fs/btrfs/sysfs.c   | 7 +++
 fs/btrfs/sysfs.h   | 2 +-
 fs/btrfs/volumes.c | 2 +-
 4 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index ca6a3a3..2acc0aa 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -593,7 +593,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info 
*fs_info,
 
/* replace the sysfs entry */
btrfs_kobj_rm_device(fs_info, src_device);
-   btrfs_kobj_add_device(fs_info, tgt_device);
+   btrfs_kobj_add_device(fs_info-fs_devices, tgt_device);
btrfs_rm_dev_replace_free_srcdev(fs_info, src_device);
 
/* write back the superblocks */
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 83d7535..15e4d54 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -682,11 +682,10 @@ int btrfs_sysfs_add_device(struct btrfs_fs_devices 
*fs_devs)
return 0;
 }
 
-int btrfs_kobj_add_device(struct btrfs_fs_info *fs_info,
-   struct btrfs_device *one_device)
+int btrfs_kobj_add_device(struct btrfs_fs_devices *fs_devices,
+   struct btrfs_device *one_device)
 {
int error = 0;
-   struct btrfs_fs_devices *fs_devices = fs_info-fs_devices;
struct btrfs_device *dev;
 
error = btrfs_sysfs_add_device(fs_devices);
@@ -752,7 +751,7 @@ int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
if (error)
return error;
 
-   error = btrfs_kobj_add_device(fs_info, NULL);
+   error = btrfs_kobj_add_device(fs_devs, NULL);
if (error) {
btrfs_sysfs_remove_fsid(fs_devs);
return error;
diff --git a/fs/btrfs/sysfs.h b/fs/btrfs/sysfs.h
index f7dd298..eeb86a8 100644
--- a/fs/btrfs/sysfs.h
+++ b/fs/btrfs/sysfs.h
@@ -70,7 +70,7 @@ char *btrfs_printable_features(enum btrfs_feature_set set, 
u64 flags);
 extern const char * const btrfs_feature_set_names[3];
 extern struct kobj_type space_info_ktype;
 extern struct kobj_type btrfs_raid_ktype;
-int btrfs_kobj_add_device(struct btrfs_fs_info *fs_info,
+int btrfs_kobj_add_device(struct btrfs_fs_devices *fs_devices,
struct btrfs_device *one_device);
 int btrfs_kobj_rm_device(struct btrfs_fs_info *fs_info,
 struct btrfs_device *one_device);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index c1b1038..e567d54 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2206,7 +2206,7 @@ int btrfs_init_new_device(struct btrfs_root *root, char 
*device_path)
tmp + 1);
 
/* add sysfs device entry */
-   btrfs_kobj_add_device(root-fs_info, device);
+   btrfs_kobj_add_device(root-fs_info-fs_devices, device);
 
/*
 * we've got more storage, clear any full flags on the space
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 24/24] Btrfs: sysfs: add check if super kobject is already initialized

2015-02-08 Thread Anand Jain
---
 fs/btrfs/sysfs.c | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index f8358d2..5c555da 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -750,10 +750,14 @@ int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
int error = 0;
 
while (fs_devs) {
-   init_completion(fs_devs-kobj_unregister);
-   fs_devs-super_kobj.kset = btrfs_kset;
-   error = kobject_init_and_add(fs_devs-super_kobj,
+   if (!fs_devs-super_kobj.state_initialized) {
+   init_completion(fs_devs-kobj_unregister);
+   fs_devs-super_kobj.kset = btrfs_kset;
+   error = kobject_init_and_add(fs_devs-super_kobj,
btrfs_ktype, parent, %pU, fs_devs-fsid);
+   } else {
+   error = -EINVAL;
+   }
if (!follow_seed)
return error;
parent = fs_devs-super_kobj;
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 14/24] Btrfs: sysfs: provide framework to remove all fsid kobject

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 4b5bac6..83d7535 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -515,7 +515,7 @@ static int addrm_unknown_feature_attrs(struct btrfs_fs_info 
*fs_info, bool add)
return 0;
 }
 
-static void btrfs_sysfs_remove_fsid(struct btrfs_fs_devices *fs_devs)
+static void __btrfs_sysfs_remove_fsid(struct btrfs_fs_devices *fs_devs)
 {
if (fs_devs-device_dir_kobj) {
kobject_del(fs_devs-device_dir_kobj);
@@ -528,6 +528,21 @@ static void btrfs_sysfs_remove_fsid(struct 
btrfs_fs_devices *fs_devs)
wait_for_completion(fs_devs-kobj_unregister);
 }
 
+/* when fs_devs is NULL it will remove all fsid kobject */
+static void btrfs_sysfs_remove_fsid(struct btrfs_fs_devices *fs_devs)
+{
+   struct list_head *fs_uuids = btrfs_get_fs_uuids();
+
+   if (fs_devs) {
+   __btrfs_sysfs_remove_fsid(fs_devs);
+   return;
+   }
+
+   list_for_each_entry(fs_devs, fs_uuids, list) {
+   __btrfs_sysfs_remove_fsid(fs_devs);
+   }
+}
+
 void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
 {
fs_info-fs_devices-fs_info = NULL;
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 18/24] Btrfs: sysfs: make btrfs_sysfs_add_device() non static

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fs/btrfs/sysfs.h b/fs/btrfs/sysfs.h
index aaff124..ac06b5c 100644
--- a/fs/btrfs/sysfs.h
+++ b/fs/btrfs/sysfs.h
@@ -76,4 +76,5 @@ int btrfs_kobj_rm_device(struct btrfs_fs_devices *fs_devices,
 struct btrfs_device *one_device);
 int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
struct kobject *parent);
+int btrfs_sysfs_add_device(struct btrfs_fs_devices *fs_devs);
 #endif /* _BTRFS_SYSFS_H_ */
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 16/24] Btrfs: sysfs btrfs_kobj_rm_device() pass fs_devices instead of fs_info

2015-02-08 Thread Anand Jain
since btrfs_kobj_rm_device() does nothing with fs_info

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/dev-replace.c |  2 +-
 fs/btrfs/sysfs.c   | 12 ++--
 fs/btrfs/sysfs.h   |  2 +-
 fs/btrfs/volumes.c |  4 ++--
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index 2acc0aa..124b60f 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -592,7 +592,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info 
*fs_info,
mutex_unlock(uuid_mutex);
 
/* replace the sysfs entry */
-   btrfs_kobj_rm_device(fs_info, src_device);
+   btrfs_kobj_rm_device(fs_info-fs_devices, src_device);
btrfs_kobj_add_device(fs_info-fs_devices, tgt_device);
btrfs_rm_dev_replace_free_srcdev(fs_info, src_device);
 
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 15e4d54..4c86e62 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -555,7 +555,7 @@ void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
addrm_unknown_feature_attrs(fs_info, false);
sysfs_remove_group(fs_info-fs_devices-super_kobj, 
btrfs_feature_attr_group);
sysfs_remove_files(fs_info-fs_devices-super_kobj, btrfs_attrs);
-   btrfs_kobj_rm_device(fs_info, NULL);
+   btrfs_kobj_rm_device(fs_info-fs_devices, NULL);
btrfs_sysfs_remove_fsid(fs_info-fs_devices);
 }
 
@@ -636,20 +636,20 @@ static void init_feature_attrs(void)
 
 /* when one_device is NULL, it removes all device links */
 
-int btrfs_kobj_rm_device(struct btrfs_fs_info *fs_info,
+int btrfs_kobj_rm_device(struct btrfs_fs_devices *fs_devices,
struct btrfs_device *one_device)
 {
struct hd_struct *disk;
struct kobject *disk_kobj;
 
-   if (!fs_info-fs_devices-device_dir_kobj)
+   if (!fs_devices-device_dir_kobj)
return -EINVAL;
 
if (one_device  one_device-bdev) {
disk = one_device-bdev-bd_part;
disk_kobj = part_to_dev(disk)-kobj;
 
-   sysfs_remove_link(fs_info-fs_devices-device_dir_kobj,
+   sysfs_remove_link(fs_devices-device_dir_kobj,
disk_kobj-name);
}
 
@@ -657,13 +657,13 @@ int btrfs_kobj_rm_device(struct btrfs_fs_info *fs_info,
return 0;
 
list_for_each_entry(one_device,
-   fs_info-fs_devices-devices, dev_list) {
+   fs_devices-devices, dev_list) {
if (!one_device-bdev)
continue;
disk = one_device-bdev-bd_part;
disk_kobj = part_to_dev(disk)-kobj;
 
-   sysfs_remove_link(fs_info-fs_devices-device_dir_kobj,
+   sysfs_remove_link(fs_devices-device_dir_kobj,
disk_kobj-name);
}
 
diff --git a/fs/btrfs/sysfs.h b/fs/btrfs/sysfs.h
index eeb86a8..3938ac1 100644
--- a/fs/btrfs/sysfs.h
+++ b/fs/btrfs/sysfs.h
@@ -72,6 +72,6 @@ extern struct kobj_type space_info_ktype;
 extern struct kobj_type btrfs_raid_ktype;
 int btrfs_kobj_add_device(struct btrfs_fs_devices *fs_devices,
struct btrfs_device *one_device);
-int btrfs_kobj_rm_device(struct btrfs_fs_info *fs_info,
+int btrfs_kobj_rm_device(struct btrfs_fs_devices *fs_devices,
 struct btrfs_device *one_device);
 #endif /* _BTRFS_SYSFS_H_ */
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index e567d54..51873ec 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1701,7 +1701,7 @@ int btrfs_rm_device(struct btrfs_root *root, char 
*device_path)
if (device-bdev) {
device-fs_devices-open_devices--;
/* remove sysfs entry */
-   btrfs_kobj_rm_device(root-fs_info, device);
+   btrfs_kobj_rm_device(root-fs_info-fs_devices, device);
}
 
call_rcu(device-rcu, free_device);
@@ -2285,7 +2285,7 @@ int btrfs_init_new_device(struct btrfs_root *root, char 
*device_path)
 error_trans:
btrfs_end_transaction(trans, root);
rcu_string_free(device-name);
-   btrfs_kobj_rm_device(root-fs_info, device);
+   btrfs_kobj_rm_device(root-fs_info-fs_devices, device);
kfree(device);
 error:
blkdev_put(bdev, FMODE_EXCL);
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 22/24] Btrfs: sysfs: don't fail seeding for the sake of sysfs kobject issue

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/volumes.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 51873ec..1490723 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2249,7 +2249,8 @@ int btrfs_init_new_device(struct btrfs_root *root, char 
*device_path)
root-fs_info-fsid);
if (kobject_rename(root-fs_info-fs_devices-super_kobj,
fsid_buf))
-   goto error_trans;
+   printk(KERN_WARNING\
+   BTRFS: sysfs: failed to create fsid for sprout\n);
}
 
root-fs_info-num_tolerated_disk_barrier_failures =
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 20/24] Btrfs: sysfs: separate kobject and attribute creation

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/disk-io.c | 18 +-
 fs/btrfs/sysfs.c   | 15 ++-
 2 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 0cd6550..4b7f3b8 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2785,10 +2785,22 @@ retry_root_backup:
 
btrfs_close_extra_devices(fs_info, fs_devices, 1);
 
+   ret = btrfs_sysfs_add_fsid(fs_devices, NULL);
+   if (ret) {
+   pr_err(BTRFS: failed to init sysfs fsid interface: %d\n, ret);
+   goto fail_block_groups;
+   }
+
+   ret = btrfs_sysfs_add_device(fs_devices);
+   if (ret) {
+   pr_err(BTRFS: failed to init sysfs device interface: %d\n, 
ret);
+   goto fail_fsdev_sysfs;
+   }
+
ret = btrfs_sysfs_add_one(fs_info);
if (ret) {
pr_err(BTRFS: failed to init sysfs interface: %d\n, ret);
-   goto fail_block_groups;
+   goto fail_fsdev_sysfs;
}
 
ret = btrfs_init_space_info(fs_info);
@@ -3002,6 +3014,9 @@ fail_cleaner:
 fail_sysfs:
btrfs_sysfs_remove_one(fs_info);
 
+fail_fsdev_sysfs:
+   btrfs_sysfs_remove_fsid(fs_info-fs_devices);
+
 fail_block_groups:
btrfs_put_block_group_cache(fs_info);
btrfs_free_block_groups(fs_info);
@@ -3679,6 +3694,7 @@ void close_ctree(struct btrfs_root *root)
}
 
btrfs_sysfs_remove_one(fs_info);
+   btrfs_sysfs_remove_fsid(fs_info-fs_devices);
 
btrfs_free_fs_roots(fs_info);
 
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index ff9e5f6..d0caa32 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -556,7 +556,6 @@ void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
sysfs_remove_group(fs_info-fs_devices-super_kobj, 
btrfs_feature_attr_group);
sysfs_remove_files(fs_info-fs_devices-super_kobj, btrfs_attrs);
btrfs_kobj_rm_device(fs_info-fs_devices, NULL);
-   btrfs_sysfs_remove_fsid(fs_info-fs_devices);
 }
 
 const char * const btrfs_feature_set_names[3] = {
@@ -688,10 +687,6 @@ int btrfs_kobj_add_device(struct btrfs_fs_devices 
*fs_devices,
int error = 0;
struct btrfs_device *dev;
 
-   error = btrfs_sysfs_add_device(fs_devices);
-   if (error)
-   return error;
-
list_for_each_entry(dev, fs_devices-devices, dev_list) {
struct hd_struct *disk;
struct kobject *disk_kobj;
@@ -747,19 +742,13 @@ int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
 
fs_devs-fs_info = fs_info;
 
-   error = btrfs_sysfs_add_fsid(fs_devs, NULL);
-   if (error)
-   return error;
-
error = btrfs_kobj_add_device(fs_devs, NULL);
-   if (error) {
-   btrfs_sysfs_remove_fsid(fs_devs);
+   if (error)
return error;
-   }
 
error = sysfs_create_files(super_kobj, btrfs_attrs);
if (error) {
-   btrfs_sysfs_remove_fsid(fs_devs);
+   btrfs_kobj_rm_device(fs_devs, NULL);
return error;
}
 
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 17/24] Btrfs: sysfs: make btrfs_sysfs_add_fsid() non static

2015-02-08 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 2 +-
 fs/btrfs/sysfs.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 4c86e62..2dbb064 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -727,7 +727,7 @@ u64 btrfs_debugfs_test;
  * Can be called by the device discovery thread.
  * And parent can be specified for seed device
  */
-static int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
+int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
struct kobject *parent)
 {
int error;
diff --git a/fs/btrfs/sysfs.h b/fs/btrfs/sysfs.h
index 3938ac1..aaff124 100644
--- a/fs/btrfs/sysfs.h
+++ b/fs/btrfs/sysfs.h
@@ -74,4 +74,6 @@ int btrfs_kobj_add_device(struct btrfs_fs_devices *fs_devices,
struct btrfs_device *one_device);
 int btrfs_kobj_rm_device(struct btrfs_fs_devices *fs_devices,
 struct btrfs_device *one_device);
+int btrfs_sysfs_add_fsid(struct btrfs_fs_devices *fs_devs,
+   struct kobject *parent);
 #endif /* _BTRFS_SYSFS_H_ */
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 23/24] Btrfs: sysfs: support seed devices in the sysfs layout

2015-02-08 Thread Anand Jain
This adds an enhancement to show the seed fsid and devices.

The way sprouting handles fs_devices:
  clone seed fs_devices and add to the fs_uuids
  mem copy seed fs_devices and assign to fs_devices-seed (move dev_list)
  evacuate seed fs_devices contents to hold sprout fs devices contents

  So to be inline with this fs_devices changes during seeding,
  represent seed fsid under the sprout fsid, this is achieved
  by using the kobject_move()

eg: showing two levels of seeding.

find /sys/fs/btrfs/ -type d -name devices -exec ls {} \; -print

sde
/sys/fs/btrfs/8c2772d4-6951-43c3-89b6-3ab3c70a13f8/f7ef2904-ce89-4421-bfb0-49fd999e9a0b/devices
sdd
/sys/fs/btrfs/8c2772d4-6951-43c3-89b6-3ab3c70a13f8/f7ef2904-ce89-4421-bfb0-49fd999e9a0b/53ac3265-0c34-4afd-9453-cc0d1a07be64/devices
sdf
/sys/fs/btrfs/8c2772d4-6951-43c3-89b6-3ab3c70a13f8/devices

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/dev-replace.c |  4 +--
 fs/btrfs/disk-io.c |  4 +--
 fs/btrfs/sysfs.c   | 66 +++---
 fs/btrfs/sysfs.h   |  8 +++---
 fs/btrfs/volumes.c | 40 +-
 5 files changed, 89 insertions(+), 33 deletions(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index 124b60f..e72b986 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -592,8 +592,8 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info 
*fs_info,
mutex_unlock(uuid_mutex);
 
/* replace the sysfs entry */
-   btrfs_kobj_rm_device(fs_info-fs_devices, src_device);
-   btrfs_kobj_add_device(fs_info-fs_devices, tgt_device);
+   btrfs_kobj_rm_device(fs_info-fs_devices, src_device, 0);
+   btrfs_kobj_add_device(fs_info-fs_devices, tgt_device, 0);
btrfs_rm_dev_replace_free_srcdev(fs_info, src_device);
 
/* write back the superblocks */
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 4b7f3b8..77372af 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2785,13 +2785,13 @@ retry_root_backup:
 
btrfs_close_extra_devices(fs_info, fs_devices, 1);
 
-   ret = btrfs_sysfs_add_fsid(fs_devices, NULL);
+   ret = btrfs_sysfs_add_fsid(fs_devices, NULL, 1);
if (ret) {
pr_err(BTRFS: failed to init sysfs fsid interface: %d\n, ret);
goto fail_block_groups;
}
 
-   ret = btrfs_sysfs_add_device(fs_devices);
+   ret = btrfs_sysfs_add_device(fs_devices, 1);
if (ret) {
pr_err(BTRFS: failed to init sysfs device interface: %d\n, 
ret);
goto fail_fsdev_sysfs;
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index d89bf4d..f8358d2 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -517,15 +517,20 @@ static int addrm_unknown_feature_attrs(struct 
btrfs_fs_info *fs_info, bool add)
 
 static void __btrfs_sysfs_remove_fsid(struct btrfs_fs_devices *fs_devs)
 {
+   if (fs_devs-seed)
+   __btrfs_sysfs_remove_fsid(fs_devs-seed);
+
if (fs_devs-device_dir_kobj) {
kobject_del(fs_devs-device_dir_kobj);
kobject_put(fs_devs-device_dir_kobj);
fs_devs-device_dir_kobj = NULL;
}
 
-   kobject_del(fs_devs-super_kobj);
-   kobject_put(fs_devs-super_kobj);
-   wait_for_completion(fs_devs-kobj_unregister);
+   if (fs_devs-super_kobj.state_initialized) {
+   kobject_del(fs_devs-super_kobj);
+   kobject_put(fs_devs-super_kobj);
+   wait_for_completion(fs_devs-kobj_unregister);
+   }
 }
 
 /* when fs_devs is NULL it will remove all fsid kobject */
@@ -555,7 +560,7 @@ void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
addrm_unknown_feature_attrs(fs_info, false);
sysfs_remove_group(fs_info-fs_devices-super_kobj, 
btrfs_feature_attr_group);
sysfs_remove_files(fs_info-fs_devices-super_kobj, btrfs_attrs);
-   btrfs_kobj_rm_device(fs_info-fs_devices, NULL);
+   btrfs_kobj_rm_device(fs_info-fs_devices, NULL, 1);
 }
 
 const char * const btrfs_feature_set_names[3] = {
@@ -636,7 +641,7 @@ static void init_feature_attrs(void)
 /* when one_device is NULL, it removes all device links */
 
 int btrfs_kobj_rm_device(struct btrfs_fs_devices *fs_devices,
-   struct btrfs_device *one_device)
+   struct btrfs_device *one_device, int follow_seed)
 {
struct hd_struct *disk;
struct kobject *disk_kobj;
@@ -666,27 +671,39 @@ int btrfs_kobj_rm_device(struct btrfs_fs_devices 
*fs_devices,
disk_kobj-name);
}
 
+   if (follow_seed  fs_devices-seed)
+   btrfs_kobj_rm_device(fs_devices-seed, NULL, follow_seed);
+
return 0;
 }
 
-int btrfs_sysfs_add_device(struct btrfs_fs_devices *fs_devs)
+int btrfs_sysfs_add_device(struct btrfs_fs_devices *fs_devs, int follow_seed)
 {
-   if (!fs_devs-device_dir_kobj)
-   

[PATCH 09/24] Btrfs: sysfs: let default_attrs be separate from the kset

2015-02-08 Thread Anand Jain
From: Anand Jain anand.j...@oracle.com

As of now btrfs_attrs are provided using the default_attrs through
the kset. Separate them and create the default_attrs using the
sysfs_create_files instead. By doing this we will have the
flexibility that device discovery thread could create fsid
kobject.

Signed-off-by: Anand Jain anand.j...@oracle.com
---
 fs/btrfs/sysfs.c | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index f42d8fd..5208a49 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -428,7 +428,7 @@ static ssize_t btrfs_clone_alignment_show(struct kobject 
*kobj,
 
 BTRFS_ATTR(clone_alignment, btrfs_clone_alignment_show);
 
-static struct attribute *btrfs_attrs[] = {
+static const struct attribute *btrfs_attrs[] = {
BTRFS_ATTR_PTR(label),
BTRFS_ATTR_PTR(nodesize),
BTRFS_ATTR_PTR(sectorsize),
@@ -447,7 +447,6 @@ static void btrfs_release_super_kobj(struct kobject *kobj)
 static struct kobj_type btrfs_ktype = {
.sysfs_ops  = kobj_sysfs_ops,
.release= btrfs_release_super_kobj,
-   .default_attrs  = btrfs_attrs,
 };
 
 static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj)
@@ -531,6 +530,7 @@ void btrfs_sysfs_remove_one(struct btrfs_fs_info *fs_info)
}
addrm_unknown_feature_attrs(fs_info, false);
sysfs_remove_group(fs_info-super_kobj, btrfs_feature_attr_group);
+   sysfs_remove_files(fs_info-super_kobj, btrfs_attrs);
btrfs_sysfs_remove_fsid(fs_info);
 }
 
@@ -720,13 +720,17 @@ int btrfs_sysfs_add_one(struct btrfs_fs_info *fs_info)
return error;
}
 
-   error = sysfs_create_group(fs_info-super_kobj,
-  btrfs_feature_attr_group);
+   error = sysfs_create_files(fs_info-super_kobj, btrfs_attrs);
if (error) {
btrfs_sysfs_remove_fsid(fs_info);
return error;
}
 
+   error = sysfs_create_group(fs_info-super_kobj,
+  btrfs_feature_attr_group);
+   if (error)
+   goto failure;
+
error = addrm_unknown_feature_attrs(fs_info, true);
if (error)
goto failure;
-- 
2.0.0.153.g79d

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread constantine
 Second, SMART is only saying its internal test is good. The errors are
 related to data transfer, so that implicates the enclosure (bridge
 chipset or electronics), the cable, or the controller interface.
 Actually it could also be a flaky controller or RAM on the drive
 itself too which I don't think get checked with SMART tests.

Which test should I do from now on (on a weekly basis?) so as to
prevent similar things from happening?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Replacing a (or two?) failed drive(s) in RAID-1 btrfs filesystem

2015-02-08 Thread Chris Murphy
On Sun, Feb 8, 2015 at 4:09 PM, constantine costas.magn...@gmail.com wrote:
 By the way, /dev/sdc just completed the extended offline test without
 any error... I feel so confused,

First, we know from a number of studies, including the famous (and now
kinda old) Google study that  a huge percent of drive failures come
with no SMART errors.

Second, SMART is only saying its internal test is good. The errors are
related to data transfer, so that implicates the enclosure (bridge
chipset or electronics), the cable, or the controller interface.
Actually it could also be a flaky controller or RAM on the drive
itself too which I don't think get checked with SMART tests.

-- 
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html