On 2017-11-01 10:05, ST wrote:
3. in my current ext4-based setup I have two servers while one syncs
files of certain dir to the other using lsyncd (which launches rsync on
inotify events). As far as I have understood it is more efficient to use
btrfs send/receive (over ssh) than rsync (over ssh) to sync two boxes.
Do you think it would be possible to make lsyncd to use btrfs for
syncing instead of rsync? I.e. can btrfs work with inotify events? Did
somebody try it already?
BTRFS send/receive needs a read-only snapshot to send from. This means
that triggering it on inotify events is liable to cause performance
issues and possibly lose changes
Actually triggering doesn't happen on each and every inotify event.
lsyncd has an option to define a time interval within which all inotify
events are accumulated and only then rsync is launched. It could be 5-10
seconds or more. Which is quasi real time sync. Do you still hold that
it will not work with BTRFS send/receive (i.e. keeping previous snapshot
around and creating a new one)?
Okay, I actually didn't know that. Depending on how lsyncd invokes
rsync though (does it call out rsync with the exact paths or just on the
whole directory?), it may still be less efficient to use BTRFS send/receive.
I assume on the whole directory, but I'm not sure...
4. In a case when compression is used - what quota is based on - (a)
amount of GBs the data actually consumes on the hard drive while in
compressed state or (b) amount of GBs the data naturally is in
uncompressed form. I need to set quotas as in (b). Is it possible? If
not - should I file a feature request?
I can't directly answer this as I don't know myself (I don't use
quotas), but have two comments I would suggest you consider:
1. qgroups (the BTRFS quota implementation) cause scaling and
performance issues. Unless you absolutely need quotas (unless you're a
hosting company, or are dealing with users who don't listen and don't
pay attention to disk usage, you usually do not need quotas), you're
almost certainly better off disabling them for now, especially for a
production system.
Ok. I'll use more standard approaches. Which of following commands will
work with BTRFS:
https://debian-handbook.info/browse/stable/sect.quotas.html
None, qgroups are the only option right now with BTRFS, and it's pretty
likely to stay that way since the internals of the filesystem don't fit
well within the semantics of the regular VFS quota API. However,
provided you're not using huge numbers of reflinks and subvolumes, you
should be fine using qgroups.
I want to have 7 daily (or 7+4) read-only snapshots per user, for ca.
100 users. I don't expect users to invoke cp --reflink or take
snapshots.
Based on what you say below about user access, you should be absolutely
fine then.
There's one other caveat though, only root can use the qgroup ioctls,
which means that only root can check quotas.
However, it's important to know that if your users have shell access,
they can bypass qgroups. Normal users can create subvolumes, and new
subvolumes aren't added to an existing qgroup by default (and unless I'm
mistaken, aren't constrained by the qgroup set on the parent subvolume),
so simple shell access is enough to bypass quotas.
I never did it before, but shouldn't it be possible to just whitelist
commands users are allowed to use in the SSH config (and so block
creation of subvolumes/cp --reflink)? I actually would have restricted
users to sftp if I knew how to let them change their passwords once they
wish to. As far as I know it is not possible with OpenSSH...
Yes, but not with OpenSSH. Assuming you just want SFTP/SCP, and the
ability to change passwords, you can use a program called 'scponly' [1].
It's a replacement shell that only allows the things needed for a very
small set of commands, and it includes support for restricting things to
just SCP/SFTP, and the passwd command.
2. Compression and quotas cause issues regardless of how they interact.
In case (a), the user has no way of knowing if a given file will fit
under their quota until they try to create it. In case (b), actual disk
usage (as reported by du) will not match up with what the quota says the
user is using, which makes it harder for them to figure out what to
delete to free up space. It's debatable which is a less objectionable
situation for users, though most people I know tend to think in a way
that the issue with (a) doesn't matter, but the issue with (b) does.
I think both (a) and (b) should be possible and it should be up to
sysadmin to choose what he prefers. The concerns of the (b) scenario
probably could be dealt with some sort of --real-size to the du command,
while by default it could have behavior (which might be emphasized with
--compressed-size).
Reporting anything but the compressed size by default in du would mean
it doesn't behave as existing software expect it to. It's supposed to
report actual disk usage (in contrast to the sum of file sizes), which
means for example that a 1G sparse file with only 64k of data is
supposed to be reported as being 64k by du.
Yes, it shouldn't be default behavior, but an optional one...
Two more question came to my mind: as I've mentioned above - I have two
boxes one syncs to another. No RAID involved. I want to scrub (or scan -
don't know yet, what is the difference...) the whole filesystem once in
a month to look for bitrot. Questions:
1. is it a stable setup for production? Let's say I'll sync with rsync -
either in cron or in lsyncd?
Reasonably, though depending on how much data and other environmental
constraints, you may want to scrub a bit more frequently.
2. should any data corruption be discovered - is there any way to heal
it using the copy from the other box over SSH?
Provided you know which file is affected, yes, you can fix it by just
copying the file back from the other system.
Ok, but there is no automatic fixing in such a case, right?
Correct.
[1] https://github.com/scponly/scponly/wiki
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
