On 17.03.21 г. 0:46 ч., Victor Erminpour wrote:
> Calling cc-option will use KBUILD_CFLAGS, which when lazy setting
> subdir-ccflags-y produces the following build error:
>
> scripts/Makefile.lib:10: *** Recursive variable `KBUILD_CFLAGS' \
> references itself (eventually). Stop.
>
> Us
On Tue, Mar 16, 2021 at 7:39 PM Qu Wenruo wrote:
> > Using that restore I was able to restore approx. 7 TB of the
> > originally stored 22 TB under that directory.
> > Unfortunately nearly all the files are damaged. Small text files are
> > still OK. But every larger binary file is useless.
> > Is
On 17/03/2021 06:46, Victor Erminpour wrote:
Calling cc-option will use KBUILD_CFLAGS, which when lazy setting
subdir-ccflags-y produces the following build error:
scripts/Makefile.lib:10: *** Recursive variable `KBUILD_CFLAGS' \
references itself (eventually). Stop.
Use := assignment
otrd., 2021. g. 23. febr., plkst. 17:51 — lietotājs Sebastian Roller
() rakstīja:
>
[...]
>
> root@hikitty:~$ install/btrfs-progs-5.9/btrfs check --readonly /dev/sdi1
> Opening filesystem to check...
> checksum verify failed on 99593231630336 found 00B6 wanted
> checksum verify failed
On 2021/3/16 下午9:27, Sidong Yang wrote:
When user assign qgroup with qgroup id that is too big to exceeds
range and invade level value, and it works without any error. but
this action would be make undefined error. this code make sure that
qgroup id doesn't exceed range(0 ~ 2^48-1).
Signed-of
On 2021/2/23 下午11:45, Sebastian Roller wrote:
Hello all.
Sorry for asking here directly, but I'm in a desperate situation and
out of options.
I have a 72 TB btrfs filesystem which functions as a backup drive.
After a recent controller hardware failure while the backup was
running, both origina
trešd., 2021. g. 17. marts, plkst. 03:18 — lietotājs Dāvis Mosāns
() rakstīja:
>
> Currently if there's any corruption at all in extent tree
> (eg. even single bit) then mounting will fail with:
> "failed to read block groups: -5" (-EIO)
> It happens because we immediately abort on first error when
Currently if there's any corruption at all in extent tree
(eg. even single bit) then mounting will fail with:
"failed to read block groups: -5" (-EIO)
It happens because we immediately abort on first error when
searching in extent tree for block groups.
Now with this patch if `ignorebadroots` opti
Calling cc-option will use KBUILD_CFLAGS, which when lazy setting
subdir-ccflags-y produces the following build error:
scripts/Makefile.lib:10: *** Recursive variable `KBUILD_CFLAGS' \
references itself (eventually). Stop.
Use := assignment to subdir-ccflags-y when referencing cc-option.
From: Boris Burkov
Adapt the existing send/receive tests by passing '-o --force-compress'
to the mount commands in a new test. After writing a few files in the
various compression formats, send/receive them with and without
--force-decompress to test both the encoded_write path and the
fallback t
From: Boris Burkov
Send stream v2 can emit fallocate commands, so receive must support them
as well. The implementation simply passes along the arguments to the
syscall. Note that mode is encoded as a u32 in send stream but fallocate
takes an int, so there is a unsigned->signed conversion there.
From: Boris Burkov
To make the btrfs send ioctl use the stream v2 format requires passing
BTRFS_SEND_FLAG_STREAM_V2 in flags. Further, to cause the ioctl to emit
encoded_write commands for encoded extents, we must set that flag as
well as BTRFS_SEND_FLAG_COMPRESSED. Finally, we bump up the versio
From: Boris Burkov
In send stream v2, send can emit a command for setting inode flags via
the setflags ioctl. Pass the flags attribute through to the ioctl call
in receive.
Signed-off-by: Boris Burkov
---
cmds/receive-dump.c | 6 ++
cmds/receive.c | 24
com
From: Boris Burkov
Add a new btrfs_send_op and support for both dumping and proper receive
processing which does actual encoded writes.
Encoded writes are only allowed on a file descriptor opened with an
extra flag that allows encoded writes, so we also add support for this
flag when opening or
From: Boris Burkov
An encoded_write can fail if the file system it is being applied to does
not support encoded writes or if it can't find enough contiguous space
to accommodate the encoded extent. In those cases, we can likely still
process an encoded_write by explicitly decoding the data and do
From: Omar Sandoval
Now that the new support is implemented, allow the ioctl to accept the
flags and update the version in sysfs.
Signed-off-by: Omar Sandoval
---
fs/btrfs/send.c| 10 +-
fs/btrfs/send.h| 2 +-
include/uapi/linux/btrfs.h | 4 +++-
3 files chang
From: Boris Burkov
Encoded writes in receive will use pwritev2. It is possible that the
system libc does not export this function, so we stub it out and detect
whether to build the stub code with autoconf.
This syscall has special semantics in x32 (no hi lo, just takes loff_t)
so we have to dete
From: Boris Burkov
Send stream v2 adds three commands and several attributes associated to
those commands. Before we implement processing them, add all the
commands and attributes. This avoids leaving the enums in an
intermediate state that doesn't correspond to any version of send
stream.
Signe
From: Omar Sandoval
The length field of the send stream TLV header is 16 bits. This means
that the maximum amount of data that can be sent for one write is 64k
minus one. However, encoded writes must be able to send the maximum
compressed extent (128k) in one command. To support this, send stream
From: Boris Burkov
The new format privileges the BTRFS_SEND_A_DATA attribute by
guaranteeing it will always be the last attribute in any command that
needs it, and by implicitly encoding the data length as the difference
between the total command length in the command header and the sizes of
the
From: Omar Sandoval
Now that all of the pieces are in place, we can use the ENCODED_WRITE
command to send compressed extents when appropriate.
Signed-off-by: Omar Sandoval
---
fs/btrfs/ctree.h | 4 +
fs/btrfs/inode.c | 6 +-
fs/btrfs/send.c | 230 ++
From: Omar Sandoval
For encoded writes, we need the raw pages for reading compressed data
directly via a bio. So, replace kvmalloc() with vmap() so we have access
to the raw pages. 144k is large enough that it usually gets allocated
with vmalloc(), anyways.
Signed-off-by: Omar Sandoval
---
fs/
From: Boris Burkov
In send stream v2, write commands can now be an arbitrary size. For that
reason, we can no longer allocate a fixed array in sctx for read_cmd.
Instead, read_cmd dynamically allocates sctx->read_buf. To avoid
needless reallocations, we reuse read_buf between read_cmd calls by al
From: Omar Sandoval
The implementation resembles direct I/O: we have to flush any ordered
extents, invalidate the page cache, and do the io tree/delalloc/extent
map/ordered extent dance. From there, we can reuse the compression code
with a minor modification to distinguish the write from writebac
From: Omar Sandoval
This adds the definitions of the new commands for send stream version 2
and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a.
chattr), and encoded writes. It also documents two changes to the send
stream format in v2: the receiver shouldn't assume a maximum comma
From: Boris Burkov
An encoded extent can be up to 128K in length, which exceeds the largest
value expressible by the current send stream format's 16 bit tlv_len
field. Since encoded writes cannot be split into multiple writes by
btrfs send, the send stream format must change to accommodate encode
From: Omar Sandoval
There are 4 main cases:
1. Inline extents: we copy the data straight out of the extent buffer.
2. Hole/preallocated extents: we fill in zeroes.
3. Regular, uncompressed extents: we read the sectors we need directly
from disk.
4. Regular, compressed extents: we read the ent
From: Omar Sandoval
This series uses the interface added in "fs: interface for directly
reading/writing compressed data" to send and receive compressed data
without wastefully decompressing and recompressing it. It does so by
1. Bumping the send stream protocol version to 2.
2. Adding a new comm
From: Omar Sandoval
Currently, an inline extent is always created after i_size is extended
from btrfs_dirty_pages(). However, for encoded writes, we only want to
update i_size after we successfully created the inline extent. Add an
update_i_size parameter to cow_file_range_inline() and
insert_inl
From: Omar Sandoval
Currently, we always reserve the same extent size in the file and extent
size on disk for delalloc because the former is the worst case for the
latter. For RWF_ENCODED writes, we know the exact size of the extent on
disk, which may be less than or greater than (for bookends) t
From: Omar Sandoval
Currently, we only create ordered extents when ram_bytes == num_bytes
and offset == 0. However, RWF_ENCODED writes may create extents which
only refer to a subset of the full unencoded extent, so we need to plumb
these fields through the ordered extent infrastructure and pass
From: Omar Sandoval
btrfs_csum_one_bio() loops over each filesystem block in the bio while
keeping a cursor of its current logical position in the file in order to
look up the ordered extent to add the checksums to. However, this
doesn't make much sense for compressed extents, as a sector on disk
From: Omar Sandoval
Commit 1dae796aabf6 ("btrfs: inode: sink parameter start and len to
check_data_csum()") replaced the start parameter to check_data_csum()
with page_offset(), but page_offset() is not meaningful for direct I/O
pages. Bring back the start parameter.
Fixes: 265d4ac03fdf ("btrfs:
From: Omar Sandoval
The upcoming RWF_ENCODED operation introduces some security concerns:
1. Compressed writes will pass arbitrary data to decompression
algorithms in the kernel.
2. Compressed reads can leak truncated/hole punched data.
Therefore, we need to require privilege for RWF_ENCODED
From: Omar Sandoval
Btrfs supports transparent compression: data written by the user can be
compressed when written to disk and decompressed when read back.
However, we'd like to add an interface to write pre-compressed data
directly to the filesystem, and the matching interface to read
compresse
From: Omar Sandoval
This adds a new page, encoded_io(7), providing an overview of encoded
I/O and updates fcntl(2), open(2), and preadv2(2)/pwritev2(2) to
reference it.
Signed-off-by: Omar Sandoval
---
man2/fcntl.2 | 8 +
man2/open.2 | 13 ++
man2/readv.2 | 69 +
ma
From: Omar Sandoval
This is essentially copy_struct_from_user() but for an iov_iter.
Suggested-by: Aleksa Sarai
Reviewed-by: Josef Bacik
Signed-off-by: Omar Sandoval
---
include/linux/uio.h | 2 ++
lib/iov_iter.c | 82 +
2 files changed, 84 i
From: Omar Sandoval
This series adds an API for reading compressed data on a filesystem
without decompressing it as well as support for writing compressed data
directly to the filesystem. As with the previous submissions, I've
included a man page patch describing the API. I have test cases
(inclu
On Tue, Mar 16, 2021 at 04:54:13PM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> btrfs_orphan_cleanup() has a comment referring to find_dead_roots, but
> function does not exists since commit cb517eabba4f10 ("Btrfs: cleanup the
> similar code of the fs root read"). What we use now t
Hi,
The problem exceeds my knowledge of both Btrfs and bcache/ssd failure
modes. I'm not sure what professional data recovery can really do,
other than throw a bunch of people at stitching things back together
again without any help from the file system. I know that the state of
the repair tools i
On Tue, Mar 16, 2021 at 02:21:45PM -0400, Paul Moore wrote:
> On Tue, Mar 16, 2021 at 10:48 AM Ondrej Mosnacek wrote:
> >
> > When SELinux security options are passed to btrfs via fsconfig(2) rather
> > than via mount(2), the operation aborts with an error. What happens is
> > roughly this sequenc
On Fri, Mar 05, 2021 at 11:26:30AM -0800, Boris Burkov wrote:
> +/*
> + * fsverity op that ends enabling verity.
> + * fsverity calls this when it's done with all of the pages in the file
> + * and all of the merkle items have been inserted. We write the
> + * descriptor and update the inode in th
On Tue, Mar 16, 2021 at 10:48 AM Ondrej Mosnacek wrote:
>
> When SELinux security options are passed to btrfs via fsconfig(2) rather
> than via mount(2), the operation aborts with an error. What happens is
> roughly this sequence:
>
> 1. vfs_parse_fs_param() eats away the LSM options and parses th
On Thu, Mar 11, 2021 at 02:31:04PM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> This patchset fixes a couple bugs, in the two first patches, with the tree
> mod log code. The remaining patches just move all that code into a separate
> file, since it's quite large and ctree.c is hug
On Tue, Mar 16, 2021 at 11:43 AM Filipe Manana wrote:
>
> On Tue, Mar 16, 2021 at 6:49 AM robbieko wrote:
> >
> > Hi All,
> >
> > The patch delayed find orphan roots.
> > Move to after orphan cleanup with tree_root.
> > I think this will cause all orphan items to be deleted
> > when orphan cleanu
From: Filipe Manana
Test that subvolume deletion is resumed on RW mounts, that it is not
performed on RO mounts and that after remounting a filesystem from RO
to RW mode, it is also performed.
This triggers a regression introduced in kernel 5.11 which is fixed
by a patch that has the following s
From: Filipe Manana
btrfs_orphan_cleanup() has a comment referring to find_dead_roots, but
function does not exists since commit cb517eabba4f10 ("Btrfs: cleanup the
similar code of the fs root read"). What we use now to find and load dead
roots is btrfs_find_orphan_roots(). So update the comment
From: Filipe Manana
During the mount procedure we are calling btrfs_orphan_cleanup() against
the root tree, which will find all orphans items in this tree. When an
orphan item corresponds to a deleted subvolume/snapshot (instead of an
inode space cache), it must not delete the orphan item, becaus
When SELinux security options are passed to btrfs via fsconfig(2) rather
than via mount(2), the operation aborts with an error. What happens is
roughly this sequence:
1. vfs_parse_fs_param() eats away the LSM options and parses them into
fc->security.
2. legacy_get_tree() finds nothing in ctx->
When user assign qgroup with qgroup id that is too big to exceeds
range and invade level value, and it works without any error. but
this action would be make undefined error. this code make sure that
qgroup id doesn't exceed range(0 ~ 2^48-1).
Signed-off-by: Sidong Yang
---
v2:
Use btrfs_qgroup
On 2021/3/16 下午8:58, Sidong Yang wrote:
On Tue, Mar 16, 2021 at 01:44:33PM +0800, Qu Wenruo wrote:
On 2021/3/15 下午11:56, Sidong Yang wrote:
When user assign qgroup with qgroup id that is too big to exceeds
range and invade level value, and it works without any error. but
this action would
On Tue, Mar 16, 2021 at 01:44:33PM +0800, Qu Wenruo wrote:
>
>
> On 2021/3/15 下午11:56, Sidong Yang wrote:
> > When user assign qgroup with qgroup id that is too big to exceeds
> > range and invade level value, and it works without any error. but
> > this action would be make undefined error. this
From: Filipe Manana
Test cases where a direct IO write, with O_DSYNC, can not be done and has
to fallback to a buffered write.
This is motivated by the fact we don't have existing tests for these cases
and in fact we had a regression for one case in the 5.10 kernel. This was
the case when doing
From: Filipe Manana
Test that subvolume deletion is resumed on RW mounts, that it is not
performed on RO mounts and that after remounting a filesystem from RO
to RW mode, it is performed.
This currently passes on btrfs and it is not motivated by any recent
regression. This test is being added ju
On 16/03/2021 12:50, Filipe Manana wrote:
> On Mon, Mar 15, 2021 at 7:31 PM David Sterba wrote:
>>
>> On Thu, Mar 11, 2021 at 02:31:06PM +, fdman...@kernel.org wrote:
>>> From: Filipe Manana
>>>
>>> When freeing a tree block we may end up adding its extent back to the
>>> free space cache/tre
On Mon, Mar 15, 2021 at 7:31 PM David Sterba wrote:
>
> On Thu, Mar 11, 2021 at 02:31:06PM +, fdman...@kernel.org wrote:
> > From: Filipe Manana
> >
> > When freeing a tree block we may end up adding its extent back to the
> > free space cache/tree, as long as there are no more references for
On Tue, Mar 16, 2021 at 6:49 AM robbieko wrote:
>
> Hi All,
>
> The patch delayed find orphan roots.
> Move to after orphan cleanup with tree_root.
> I think this will cause all orphan items to be deleted
> when orphan cleanup with tree_root.
> Afterwards, find orphan roots cannot find
> the subvo
On Tue, Mar 16, 2021 at 08:05:31AM +0800, Qu Wenruo wrote:
> > In that case one file with the list of supported values is a better
> > option. The main point is to have full RW support, until then it's
> > interesting only for developers and they know what to expect.
> >
>
> Indeed only full RW su
On Tue, Mar 16, 2021 at 08:10:13AM +0800, Anand Jain wrote:
>
>
> On 16/03/2021 08:05, Qu Wenruo wrote:
> >
> >
> > On 2021/3/16 上午2:44, David Sterba wrote:
> >> On Mon, Mar 15, 2021 at 08:39:31PM +0800, Qu Wenruo wrote:
> >>>
> >>>
> >>> On 2021/3/15 下午7:59, Anand Jain wrote:
> On 10/03/2
On Mon, Mar 15, 2021 at 01:39:13PM +0800, Qu Wenruo wrote:
> During the fstests run for btrfs subpage read-write support, generic/475
> crashes the system with a very high chance.
>
> It turns out the cause is also affecting btrfs subpage read-only mount
> so it's worthy a quick fix.
>
> Also the
On Tue, Mar 16, 2021 at 08:29:35AM +0800, Qu Wenruo wrote:
>
>
> On 2021/3/15 下午11:42, David Sterba wrote:
> > On Mon, Mar 15, 2021 at 01:39:13PM +0800, Qu Wenruo wrote:
> >> During the fstests run for btrfs subpage read-write support, generic/475
> >> crashes the system with a very high chance.
The in_range() macro is defined twice in btrfs' source, once in ctree.h
and once in misc.h.
Remove the definition in ctree.h and include misc.h in the files depending
on it.
Signed-off-by: Johannes Thumshirn
---
fs/btrfs/ctree.h | 2 --
fs/btrfs/extent_io.c | 1 +
fs/btrfs/file-item.c | 1 +
Hi again.
> Looks like the answer is no. The chunk tree really has to be correct
> first before anything else because it's central to doing all the
> logical to physical address translation. And if it's busted and can't
> be repaired then nothing else is likely to work or be repairable. It's
> tha
63 matches
Mail list logo
