Dear Thomas,
Am 05.09.19 um 10:50 schrieb Thomas Schneider:
> Hi,
>
> I would agree with Oliver's conclusion:
> the relevant directories are all tmpfs mounts.
this of course easily explains it and btrfs can't do anything about these when
snapshotting.
In case you want to debug the issue in more detail (e.g. how to extract this
information stored in tmpfs from LVM for example), we can of course take this
to the ceph-users list (or contact me directly).
Cheers,
Oliver
>
> root@ld5505:~# mount | grep /var
> /dev/sdbq3 on /var/cache type btrfs
> (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=260,subvol=/@cache)
> /dev/sdbq3 on /var/lib/vz/images type btrfs
> (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=262,subvol=/@images)
> lxcfs on /var/lib/lxcfs type fuse.lxcfs
> (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
> tmpfs on /var/lib/ceph/osd/ceph-122 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-123 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-105 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-92 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-112 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-77 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-76 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-87 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-119 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-79 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-115 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-98 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-100 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-89 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-108 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-120 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-82 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-118 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-104 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-95 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-1 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-96 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-117 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-116 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-99 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-106 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-110 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-97 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-81 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-121 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-88 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-0 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-94 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-113 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-107 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-101 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-78 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-93 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-85 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-103 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-102 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-109 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-114 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-80 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-111 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-83 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-86 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-91 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-84 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-90 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-8 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-9 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-10 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-11 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-12 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-13 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-14 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-15 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-16 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-17 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-18 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-19 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-20 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-21 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-22 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-23 type tmpfs (rw,relatime)
> tmpfs on /var/lib/ceph/osd/ceph-24 type tmpfs (rw,relatime)
>
>
> Am 04.09.2019 um 22:52 schrieb Oliver Freyermuth:
>> Am 04.09.19 um 21:07 schrieb Chris Murphy:
>>> On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <r...@georgianit.com> wrote:
>>>> On 2019-09-04 1:36 p.m., Chris Murphy wrote:
>>>>
>>>>> I don't really know how snapper works.
>>>>>
>>>>> The way 'btrfs subvolume snapshot' works, you must point it to a
>>>>> subvolume. It won't snapshot a regular directory and from what you
>>>>> posted above, there are no subvolumes in /var or /var/lib which means
>>>>> trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe
>>>>> it's failing but snapper doesn't show the error. I'm not really sure.
>>>>>
>>>> In this case, his snapshots are all of the root.
>>>>
>>>> I don't know how Ceph works, but since we already confirmed that there
>>>> are no subvolumes under /var, the only other explanation is that
>>>> /var/lib/ceph/osd/ceph-<n> is a submount
>>>>
>>>> What is the the result of running:
>>>> mount | grep /var
>>>>
>>> Yep.
>>>
>>>
>> Looking at Thomas' mail on the Ceph-users list:
>> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036679.html
>> I deduce he is using Ceph with the Bluestore backend, which indeed means
>> that /var/lib/ceph/osd/ceph.<n>
>> are tmpfs mounts which are completeley ephemeral and are created from LVM
>> metadata of the actual Ceph OSD disks.
>>
>> That would of course also explain why they are not part of any btrfs
>> snapshot of / (and that also means there is no need to backup anything).
>
