Re: [PATCH 2/2] Btrfs: qgroup: Introduce a may_use to account space_info-bytes_may_use.

2014-12-10 Thread Dongsheng Yang
On 12/09/2014 11:55 PM, Josef Bacik wrote: On 12/09/2014 06:27 AM, Dongsheng Yang wrote: Currently, for pre_alloc or delay_alloc, the bytes will be accounted in space_info by the three guys. space_info-bytes_may_use --- space_info-reserved --- space_info-used. But on the other hand, in qgroup,

Re: [PATCH 1/2] Btrfs: qgroup: free reserved in exceeding quota.

2014-12-10 Thread Dongsheng Yang
On 12/09/2014 11:42 PM, Josef Bacik wrote: On 12/09/2014 06:27 AM, Dongsheng Yang wrote: When we exceed quota limit in writing, we will free some reserved extent when we need to drop but not free account in qgroup. It means, each time we exceed quota in writing, there will be some remain space

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Robert White
On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote: When function btrfs_statfs() calculate the tatol size of fs, it is calculating the total size of disks and then dividing it by a factor.

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Robert White
On 12/09/2014 11:19 PM, Patrik Lundquist wrote: On 10 December 2014 at 00:13, Robert White rwh...@pobox.com wrote: On 12/09/2014 02:29 PM, Patrik Lundquist wrote: Label: none uuid: 770fe01d-6a45-42b9-912e-e8f8b413f6a4 Total devices 1 FS bytes used 1.35TiB devid1 size 2.73TiB

Re: [PATCH v4 00/13] btrfs-progs:fsck: Add inode nlink mismatch and

2014-12-10 Thread David Sterba
On Tue, Dec 09, 2014 at 04:27:19PM +0800, Qu Wenruo wrote: The patchset introduce two new repair function and some helpers to archive a huge goal: Repair btrfs whose fs tree's non-root leaf/node is corrupted when no duplication is valid. The two new repair functions are:

Re: [PATCH v4 08/13] btrfs-progs: Add count_digit() function to help calculate filename len.

2014-12-10 Thread David Sterba
On Tue, Dec 09, 2014 at 04:27:27PM +0800, Qu Wenruo wrote: +static inline int count_digit(u64 num) FYI, I've renamed it to count_digits, and updated all callers. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Duncan
Robert White posted on Tue, 09 Dec 2014 16:01:02 -0800 as excerpted: On 12/09/2014 03:48 PM, Robert White wrote: On 12/09/2014 02:29 PM, Patrik Lundquist wrote: (stuff depicting a nearly full file system). Having taken another look at it all, I'd bet (there is not sufficient information to

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Duncan
Robert White posted on Wed, 10 Dec 2014 04:17:50 -0800 as excerpted: BTRFS info (device sdc1): relocating block group 1821099687936 flags 1 BTRFS error (device sdc1): allocation failed flags 1, wanted 2013265920 BTRFS: space_info 1 has 4773171200 free, is not full BTRFS: space_info

[RFC PATCH V10 08/19] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks.

2014-12-10 Thread Chandan Rajendra
Checksums are applicable to sectorsize units. The current code uses bio-bv_len units to compute and look up checksums. This works on machines where sectorsize == PAGE_CACHE_SIZE. This patch makes the checksum computation and look up code to work with sectorsize units. Signed-off-by: Chandan

[RFC PATCH V10 13/19] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize scenario, extent allocations for only some of the dirty blocks of a page can succeed, while allocation for rest of the blocks can fail. This patch allows I/O against such partially allocated ordered extents to be submitted. Signed-off-by: Chandan Rajendra

[RFC PATCH V10 05/19] Btrfs: subpagesize-blocksize: Read tree blocks whose size is PAGE_CACHE_SIZE.

2014-12-10 Thread Chandan Rajendra
In the case of subpagesize-blocksize, this patch makes it possible to read only a single metadata block from the disk instead of all the metadata blocks that map into a page. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 45 --

[RFC PATCH V10 09/19] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page.

2014-12-10 Thread Chandan Rajendra
The code now loops across 'ordered extents' instead of 'extent maps' to figure out the dirty blocks of the page to be submitted for a write operation. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 74 1

[RFC PATCH V10 12/19] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize scenario it is not sufficient to search using the first byte of the page to make sure that there are no ordered extents present across the page. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 3 ++- fs/btrfs/inode.c |

[RFC PATCH V10 06/19] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page

2014-12-10 Thread Chandan Rajendra
For the subpagesize-blocksize scenario, This patch adds the ability to write a single extent buffer to the disk. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 20 ++-- fs/btrfs/extent_io.c | 300 --- 2 files

[RFC PATCH V10 19/19] Btrfs: subpagesize-blocksize: btrfs_submit_direct_hook: Handle map_length bio vector length

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize scenario, map_length can be less than the length of a bio vector. Such a condition may cause btrfs_submit_direct_hook() to submit a zero length bio. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 23 --- 1

[RFC PATCH V10 11/19] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in sectorsized units.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize scenario, if i_size occurs in a block which is not the last block in the page, then the space to be reserved should be calculated appropriately. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 33 ++--- 1 file

[RFC PATCH V10 18/19] Btrfs: subpagesize-blocksize: Use (eb-start, seq) as search key for tree modification log.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize a page can map multiple extent buffers and hence using (page index, seq) as the search key is incorrect. For example, searching through tree modification log tree can return an entry associated with the first extent buffer mapped by the page (if such an entry exists), when

[RFC PATCH V10 07/19] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE

2014-12-10 Thread Chandan Rajendra
From: Chandra Seetharaman sekha...@us.ibm.com This patch allows mounting filesystems with blocksize smaller than the PAGE_SIZE. Signed-off-by: Chandra Seetharaman sekha...@us.ibm.com Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 6 -- 1 file changed, 6

[RFC PATCH V10 17/19] Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set.

2014-12-10 Thread Chandan Rajendra
In non-subpagesize-blocksize scenario, BTRFS_HEADER_FLAG_WRITTEN flag prevents Btrfs code from writing into an extent buffer whose pages are under writeback. This facility isn't sufficient for achieving the same in subpagesize-blocksize scenario, since we have more than one extent buffer mapped to

[RFC PATCH V10 02/19] Btrfs: subpagesize-blocksize: Get rid of whole page writes.

2014-12-10 Thread Chandan Rajendra
This commit brings back functions that set/clear EXTENT_WRITEBACK bits. These are required to reliably clear PG_writeback page flag. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 47 +++ fs/btrfs/inode.c | 40

[RFC PATCH V10 04/19] Btrfs: subpagesize-blocksize: Define extent_buffer_head.

2014-12-10 Thread Chandan Rajendra
From: Chandra Seetharaman sekha...@us.ibm.com In order to handle multiple extent buffers per page, first we need to create a way to handle all the extent buffers that are attached to a page. This patch creates a new data structure 'struct extent_buffer_head', and moves fields that are common to

[RFC PATCH V10 14/19] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize scenario a page can have more than one block. So in addition to PagePrivate2 flag, we would have to track the I/O status of each block of a page to reliably mark the ordered extent as complete. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com ---

[RFC PATCH V10 16/19] Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted for write I/O.

2014-12-10 Thread Chandan Rajendra
In the subpagesize-blocksize scenario, the following command (with 4k as the PAGE_SIZE and 2k as the block size) can cause false accounting of blocks of an ordered extent that is written to disk: $ xfs_io -f -c pwrite 0 10240 \ -c sync_range 0 4096 \ -c sync_range 8192 2048 \ -c pwrite 10240 2048

[RFC PATCH V10 01/19] Btrfs: subpagesize-blocksize: Get rid of whole page reads.

2014-12-10 Thread Chandan Rajendra
Based on original patch from Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com For the subpagesize-blocksize scenario, a page can contain multiple blocks. This patch handles this case. This patch adds the new EXTENT_READ_IO extent state bit to reliably unlock pages in readpage's end bio function.

[RFC PATCH V10 03/19] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size.

2014-12-10 Thread Chandan Rajendra
Currently, the code reserves/releases extents in multiples of PAGE_CACHE_SIZE units. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/file.c | 32 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/file.c

[RFC PATCH V10 15/19] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43.

2014-12-10 Thread Chandan Rajendra
In subpagesize-blocksize, we have multiple blocks in a page. Checking for existence of a page in the page cache isn't a sufficient check, since we could be truncating a subset of the blocks mapped by the page. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/btrfs_inode.h

[RFC PATCH V10 10/19] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units.

2014-12-10 Thread Chandan Rajendra
While at it, this commit changes btrfs_truncate_page() to truncate sectorsized blocks instead of pages. Hence the function has been renamed to btrfs_truncate_block(). Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/ctree.h | 2 +- fs/btrfs/file.c | 41

[RFC PATCH V10 00/19] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O.

2014-12-10 Thread Chandan Rajendra
This patchset continues with the work posted earlier at https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg38862.html. Changes from V9: 1. Earlier, In read_extent_buffer_pages(), we used to check for extent buffer pages' PG_uptodate flag immediately after the page was unlocked by the

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Duncan
Robert White posted on Wed, 10 Dec 2014 02:53:40 -0800 as excerpted: On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote: When function btrfs_statfs() calculate the tatol size of fs, it is

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Patrik Lundquist
On 10 December 2014 at 13:17, Robert White rwh...@pobox.com wrote: On 12/09/2014 11:19 PM, Patrik Lundquist wrote: BUT FIRST UNDERSTAND: you do _not_ need to balance a newly converted filesystem. That is, the recommended balance (and recursive defrag) is _not_ a useability issue, its an

[PATCH] btrfs-progs: basic support for TREE_SEARCH_V2 ioctl

2014-12-10 Thread David Sterba
Add the interface and helper that checks if the v2 ioctl is supported. Signed-off-by: David Sterba dste...@suse.cz --- ioctl.h | 14 ++ utils.c | 40 utils.h | 2 ++ 3 files changed, 56 insertions(+) diff --git a/ioctl.h b/ioctl.h index

[PATCH] btrfs-progs: mkfs: make skinny-metadata default

2014-12-10 Thread David Sterba
According to public poll, this is desired and deemed to be safe. Feature introduced in kernel 3.10 (Jun 2013). Signed-off-by: David Sterba dste...@suse.cz --- mkfs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mkfs.c b/mkfs.c index e10e62d2f2e3..f930a5353f75 100644 ---

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Shriramana Sharma
On Tue, Dec 9, 2014 at 4:50 PM, Dongsheng Yang yangds.f...@cn.fujitsu.com wrote: # df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/vdf1 3.0G 1018M 1.3G 45% /mnt LOL -- not being a user of RAID I can't comment on the patch, but I was somewhat wondering what the

Re: systemd.setenv and a mount.unit

2014-12-10 Thread David Sterba
On Thu, Nov 20, 2014 at 11:39:19AM -0700, Chris Murphy wrote: On Thu, Nov 20, 2014 at 4:14 AM, Goffredo Baroncelli kreij...@inwind.it wrote: Supposing to have the following four subvolumes /root/ /root/etc /root/usr /root/var When you need to snapshot, you should: # btrfs

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Dongsheng Yang
On Wed, Dec 10, 2014 at 6:53 PM, Robert White rwh...@pobox.com wrote: On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote: When function btrfs_statfs() calculate the tatol size of fs, it

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Dongsheng Yang
On Wed, Dec 10, 2014 at 9:59 PM, Shriramana Sharma samj...@gmail.com wrote: On Tue, Dec 9, 2014 at 4:50 PM, Dongsheng Yang yangds.f...@cn.fujitsu.com wrote: # df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/vdf1 3.0G 1018M 1.3G 45% /mnt LOL -- not being a

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Dongsheng Yang
On Wed, Dec 10, 2014 at 9:21 PM, Duncan 1i5t5.dun...@cox.net wrote: Robert White posted on Wed, 10 Dec 2014 02:53:40 -0800 as excerpted: On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote:

[PATCH 3/3] btrfs-progs: subvol delete: rename variable to match the option name

2014-12-10 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz --- cmds-subvolume.c | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index b14f86e06cb4..15d4b975a916 100644 --- a/cmds-subvolume.c +++ b/cmds-subvolume.c @@ -218,10 +218,10

[PATCH 1/3] btrfs-progs: let subvol delete print commit mode inline

2014-12-10 Thread David Sterba
There are options to specify if the subvolume deletion should wait for commit after each subvol or at the end. This is reported at the beginning and considered as a noise. We'd like to report the mode for each subvolume instead.

[PATCH 2/3] btrfs-progs: subvol delete: add verbosity option

2014-12-10 Thread David Sterba
Add an the option -v and use it for the transaction commit mode message. Signed-off-by: David Sterba dste...@suse.cz --- cmds-subvolume.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index 4e452f4f4eb7..b14f86e06cb4

[PATCH 0/3] Btrfs-progs: subvolume deletion commit mode update

2014-12-10 Thread David Sterba
Minor change in the output of 'subvolume delete' command, the commit mode is printed inline with the subvolume and the global message is moved under the newly added 'verbose' option. David Sterba (3): btrfs-progs: let subvol delete print commit mode inline btrfs-progs: subvol delete: add

Re: PROBLEM: #89121 BTRFS mixes up mounted devices with their snapshots

2014-12-10 Thread Phillip Susi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/9/2014 10:10 PM, Anand Jain wrote: In the test case provided earlier who is triggering the scan ? grub-probe ? The scan is initiated by udev. grub-probe only comes into it because it is looking to /proc/mounts to find out what device is

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Goffredo Baroncelli
On 12/10/2014 11:53 AM, Robert White wrote: On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote: When function btrfs_statfs() calculate the tatol size of fs, it is calculating the total size

Re: [PATCH V2][BTRFS-PROGS] Don't use LVM snapshot device

2014-12-10 Thread Goffredo Baroncelli
On 12/10/2014 08:52 AM, Anand Jain wrote: This patch allows btrfs to skip LVM snapshot during the device scan phase. Its better to generalize the problem and fix it. The fix here is very specific to LVM use case. This does not work in cases where device is cloned using dd (device is

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Patrik Lundquist
On 10 December 2014 at 14:11, Duncan 1i5t5.dun...@cox.net wrote: From there... I've never used it but I /think/ btrfs inspect-internal logical-resolve should let you map the 182109... address to a filename. From there, moving that file out of the filesystem and back in should eliminate that

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Goffredo Baroncelli
On 12/10/2014 04:02 PM, Dongsheng Yang wrote: On Wed, Dec 10, 2014 at 9:21 PM, Duncan 1i5t5.dun...@cox.net wrote: Robert White posted on Wed, 10 Dec 2014 02:53:40 -0800 as excerpted: [...] And in the example, the mkfs was supplied with two devices, so there's no dup metadata remaining from a

Re: btrfs scrub status misreports as interrupted

2014-12-10 Thread Marc Joliet
Am Wed, 10 Dec 2014 10:51:15 +0800 schrieb Anand Jain anand.j...@oracle.com: Is there any relevant log in the dmegs ? Not in my case; at least, nothing that made it into the syslog. -- Marc Joliet -- People who think they know everything really annoy those of us who know we don't - Bjarne

Re: btrfs scrub status misreports as interrupted

2014-12-10 Thread Konstantinos Skarlatos
On 10/12/2014 9:28 μμ, Marc Joliet wrote: Am Wed, 10 Dec 2014 10:51:15 +0800 schrieb Anand Jain anand.j...@oracle.com: Is there any relevant log in the dmegs ? Not in my case; at least, nothing that made it into the syslog. Same with me, no messages at all -- To unsubscribe from this

Re: Possibility to have a transient snapshot?

2014-12-10 Thread James West
I was just looking into using overlayfs, and although it has some promise, I think it's biggest drawback is the upperdir will have to be some sort of storage backed filesystem. From my limited understanding of tmpfs, it's not supposed to be the greatest with many large files (and my system in

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Patrik Lundquist
On 10 December 2014 at 13:47, Duncan 1i5t5.dun...@cox.net wrote: The recursive btrfs defrag after deleting the saved ext* subvolume _should_ have split up any such 1 GiB extents so balance could deal with them, but either it failed for some reason on at least one such file, or there's some

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Robert White
On 12/10/2014 05:21 AM, Duncan wrote: Robert White posted on Wed, 10 Dec 2014 02:53:40 -0800 as excerpted: On 12/09/2014 05:08 PM, Dongsheng Yang wrote: On 12/10/2014 02:47 AM, Goffredo Baroncelli wrote: Hi Dongsheng On 12/09/2014 12:20 PM, Dongsheng Yang wrote: When function btrfs_statfs()

A story of btrfs corruption and recovery

2014-12-10 Thread Martin Wilck
In April 2014, I reported a btrfs corruption on the linux-btrfs mailing list (http://www.spinics.net/lists/linux-btrfs/msg33318.html). 8 months later, I am happy to be able to say I've been able to recover the data with a combination of persistence and luck. I want to share some of my insight with

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Goffredo Baroncelli
On 12/10/2014 09:36 PM, Robert White wrote: [...] I tested it and sure enough, it's RAID1... I also noticed that the default for data goes from single to RAID0 in a two slice build. I generally don't expect defaults to change in undocumented ways. Particularly since that makes

[PATCH 02/18] btrfs-progs: btrfs-debug-tree: add option -B (backup root)

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Option -B causes btrfs-debug-tree to dump the tree rooted at the backup root number given instead of the real root. Signed-off-by: Martin Wilck mwi...@arcor.de --- btrfs-debug-tree.c | 39 ++- 1 files changed, 38

[PATCH 04/18] btrfs-progs: btrfs-debug-tree: handle corruption more gracefully

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de This patch fixes the same thing in two different places. First, the first of the two BUG() tests is just a special case of the second one and can therefore be omitted. Second, instead of bailing out with BUG(), just print a reasonable error message and check the

[PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de make btrfs_inode_[amc]time work like the other btrfs_inode_xxx functions. The current definition appears broken to me; it never returns valid pointer unless an extent buffer address is added. Signed-off-by: Martin Wilck mwi...@arcor.de --- ctree.h | 15

[PATCH 07/18] btrfs restore: better output readability

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Don't print whole path for files, which will mangle output for long path names. Rather distinguish between directories and files. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff

[PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for block only

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de btrfs-debug-tree prints only the given block. It is sometimes useful to be able to print the subtree under this block. This patch enables this behavior with the option -f. Signed-off-by: Martin Wilck mwi...@arcor.de --- btrfs-debug-tree.c | 10 -- 1

[PATCH 00/18] Patch series related to my btrfs recovery

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de This patch series contains all changes I made to the btrfs tools in the course of analyzing and repairing the corruption I described in my other mail to linux-btrfs titled A story of btrfs corruption and recovery. The bottom line of this patch set is: 1) have

[PATCH 06/18] btrfs restore: set uid/gid/mode/times

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de current btrfs restore will discard file attributes. This patch sets them regular files and directories, as found in the meta data. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c | 116 --- 1

[PATCH 17/18] btrfs-progs: ctree.c: make bin_search non-static

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de I need it in btrfs-search-metadata Signed-off-by: Martin Wilck mwi...@arcor.de --- ctree.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ctree.c b/ctree.c index 23399e2..1137312 100644 --- a/ctree.c +++ b/ctree.c @@ -602,8 +602,8

[PATCH 18/18] btrfs-progs: documentation for btrfs-raw and btrfs-search-metadata

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Update documentation for btrfs-debug-tree, and add pages for btrfs-search-metadata and btrfs-raw. Signed-off-by: Martin Wilck mwi...@arcor.de --- Documentation/Makefile |2 + Documentation/btrfs-debug-tree.txt | 10 +

[PATCH 08/18] btrfs restore: track number of bytes restored

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Track the number of bytes read from extents and restored. This is useful for detecting errors and corruptions. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c | 16 1 files changed, 12 insertions(+), 4 deletions(-) diff --git

[PATCH 16/18] btrfs-progs: NEW: brtfs-search-metadata

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de A new tool for dumping all meta data (also unlinked nodes and leaves) and searching nodes or leaves with certain properties. Signed-off-by: Martin Wilck mwi...@arcor.de --- Makefile|2 +- btrfs-search-metadata.c | 224

[PATCH 15/18] btrfs-progs: NEW: btrfs-raw

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de This program can be used to dump a meta data block, fix it e.g. using a hex editor, and write it back to disk, adapting the check sum. Signed-off-by: Martin Wilck mwi...@arcor.de --- Makefile|2 +- btrfs-raw.c | 143

[PATCH 12/18] btrfs restore: check progress of file restoration

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de extents should be ordered by file offset. Expect no overlaps, and report holes. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/cmds-restore.c b/cmds-restore.c index

[PATCH 09/18] btrfs restore: more graceful error handling in copy_file

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Setting size and attributes of a file makes sense even if some errors have occured during revovery. Also, do something useful with the number of bytes written. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c | 27 ++-

[PATCH 14/18] btrfs restore: report mismatch in file size

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de A mismatch between the file size stored in the inode and the number of bytes restored may indicate a problem. restore reads data in 4k chunks, so it's normal that files are truncated. Only emit the warning in unusual cases. Signed-off-by: Martin Wilck

[PATCH 10/18] btrfs restore: hide offset is X messages

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Almost everyone who cares about her data will run btrfs restore with -v. The offset is messages displayed will irritate users because they reveal only btrfs internals. Users will think that offset refers to a file offset and suspect severe corruption. Limit

[PATCH 03/18] btrfs-progs: btrfs-debug-tree: fix usage message

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de Adapt usage message to the additional options introduced. Signed-off-by: Martin Wilck mwi...@arcor.de --- btrfs-debug-tree.c | 13 +++-- 1 files changed, 7 insertions(+), 6 deletions(-) diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c index

[PATCH 13/18] btrfs restore: improve user-asking logic for files with many extents

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de The logic to ask after 1024 extents is broken. It unnecessarily confuses users if big files are being restored, making them think somthing is going wrong. Change it to two cases: 1) no or little progress restoring, 2) writing beyond the file size.

[PATCH 11/18] btrfs restore: print progress marks for big files

2014-12-10 Thread mwilck
From: Martin Wilck mwi...@arcor.de print a '+' for every 64k restored. This gives people more confidence in long-running restore processes. Signed-off-by: Martin Wilck mwi...@arcor.de --- cmds-restore.c |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-12-10 Thread Zygo Blaxell
On Thu, Dec 04, 2014 at 02:56:55PM +0800, Qu Wenruo wrote: The main memory usage in btrfsck is extent record, which we can't free them until we read them all in and checked, so even we mmap/unmap, it can only help with the extent_buffer(which is already freed if not used according to refs).

Balance scrub defrag

2014-12-10 Thread sys.syphus
I am working on a script that i can run daily that will do maintenance on my btrfs mountpoints. is there any reason not to concurrently do all of the above? possibly including discards as well. also, is there anything existing currently that will do maintenance on btrfs so i don't have to

mkfs.btrfs limits odd [and maybe a failed phantom device?]

2014-12-10 Thread Robert White
So I started looking at the mkfs.btrfs manual page with an eye towards documenting some of the tidbits like metadata automatically switching from dup to raid1 when more than one device is used. In experimenting I ended up with some questions... (1) why is the dup profile for data restricted

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Robert White
On 12/10/2014 10:56 AM, Patrik Lundquist wrote: On 10 December 2014 at 14:11, Duncan 1i5t5.dun...@cox.net wrote: Assuming no snapshots still contain the file, of course, and that the ext* saved subvolume has already been deleted. Got no snapshots or subvolumes. Keeping it simple for now.

out of space warning?

2014-12-10 Thread sys.syphus
I would like to avoid running out of space. is there a way to know that I am getting close? i'd like to make a script that runs as part of my bash prompt and lets me know when i am getting close. i know there are several ways you can run out of space and I'd like to avoid all of them. -- To

Re: [PATCH v4 00/13] btrfs-progs:fsck: Add inode nlink mismatch and

2014-12-10 Thread Qu Wenruo
Original Message Subject: Re: [PATCH v4 00/13] btrfs-progs:fsck: Add inode nlink mismatch and From: David Sterba dste...@suse.cz To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2014年12月10日 20:37 On Tue, Dec 09, 2014 at 04:27:19PM +0800, Qu Wenruo wrote: The patchset introduce two

Re: Possibility to have a transient snapshot?

2014-12-10 Thread Robert White
On 12/10/2014 11:52 AM, James West wrote: I was just looking into using overlayfs, and although it has some promise, I think it's biggest drawback is the upperdir will have to be some sort of storage backed filesystem. From my limited understanding of tmpfs, it's not supposed to be the greatest

Re: Balance scrub defrag

2014-12-10 Thread Robert White
On 12/10/2014 02:15 PM, sys.syphus wrote: I am working on a script that i can run daily that will do maintenance on my btrfs mountpoints. is there any reason not to concurrently do all of the above? possibly including discards as well. also, is there anything existing currently that will do

Re: out of space warning?

2014-12-10 Thread Robert White
On 12/10/2014 02:54 PM, sys.syphus wrote: I would like to avoid running out of space. is there a way to know that I am getting close? i'd like to make a script that runs as part of my bash prompt and lets me know when i am getting close. i know there are several ways you can run out of space and

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-12-10 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Zygo Blaxell zblax...@furryterror.org To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2014年12月11日 05:57 On Thu, Dec 04, 2014 at 02:56:55PM +0800, Qu Wenruo wrote: The main memory

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-12-10 Thread Zygo Blaxell
On Thu, Dec 11, 2014 at 10:05:20AM +0800, Qu Wenruo wrote: Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Zygo Blaxell zblax...@furryterror.org To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2014年12月11日 05:57 On Thu, Dec

Re: out of space warning?

2014-12-10 Thread Qu Wenruo
Original Message Subject: Re: out of space warning? From: Robert White rwh...@pobox.com To: sys.syphus syssyp...@gmail.com, linux-btrfs@vger.kernel.org Date: 2014年12月11日 09:29 On 12/10/2014 02:54 PM, sys.syphus wrote: I would like to avoid running out of space. is there a way

Re: [PATCH] Btrfs: get more accurate output in fd command.

2014-12-10 Thread Duncan
Dongsheng Yang posted on Wed, 10 Dec 2014 23:02:15 +0800 as excerpted: And in the example, the mkfs was supplied with two devices, so there's no dup metadata remaining from a formerly single-device filesystem, either. (Tho there will be the small single-mode stubs, empty, remaining from the

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Duncan
Patrik Lundquist posted on Wed, 10 Dec 2014 21:11:52 +0100 as excerpted: Is btrfs-debug-tree -e useful in finding problematic files? Since you were replying directly to me, my answer... ENOTENOUGHINFO I don't know enough about it to honestly say, as I've never used it myself and haven't seen

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Duncan
Robert White posted on Wed, 10 Dec 2014 14:28:10 -0800 as excerpted: On 12/10/2014 10:56 AM, Patrik Lundquist wrote: On 10 December 2014 at 14:11, Duncan 1i5t5.dun...@cox.net wrote: Assuming no snapshots still contain the file, of course, and that the ext* saved subvolume has already been

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Duncan
Patrik Lundquist posted on Wed, 10 Dec 2014 21:11:52 +0100 as excerpted: Patrik, assuming no btrfs snapshots yet, can you do a du --all --block- size=1M | sort -n (or similar), then take a look at all results over 1024 (1 GiB since the du specified 1 MiB blocks), and see if it's reasonable to

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-10 Thread Patrik Lundquist
On 10 December 2014 at 23:28, Robert White rwh...@pobox.com wrote: On 12/10/2014 10:56 AM, Patrik Lundquist wrote: On 10 December 2014 at 14:11, Duncan 1i5t5.dun...@cox.net wrote: Assuming no snapshots still contain the file, of course, and that the ext* saved subvolume has already been

Re: mkfs.btrfs limits odd [and maybe a failed phantom device?]

2014-12-10 Thread Duncan
Robert White posted on Wed, 10 Dec 2014 14:18:55 -0800 as excerpted: So I started looking at the mkfs.btrfs manual page with an eye towards documenting some of the tidbits like metadata automatically switching from dup to raid1 when more than one device is used. In experimenting I ended up

Re: [PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for block only

2014-12-10 Thread Qu Wenruo
Original Message Subject: [PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for block only From: mwi...@arcor.de To: linux-btrfs@vger.kernel.org Date: 2014年12月11日 04:51 From: Martin Wilck mwi...@arcor.de btrfs-debug-tree prints only the given block. It is sometimes