Re: Unable to Access a Foreign Volume Group

2019-01-14 Thread Jens Holzhäuser
On Sun, Jan 13, 2019 at 05:39:20PM +0100, Martin wrote:
> Hi Jens,
> 
> my first shot would be setting the actual system id to
> 'zaphod1105820973' like described in the lvmsystemid man page. I'm not
> sure, how this uname or lvmlocal work, so I would try setting
> system_id_source to machineid or file.
> First, look if /etc/machine-id matches. If not, put is may be in a 
> /etc/lvm/system-id.

/etc/machine-id is a seemingly random 32 char hex string, bearing no
resemblance to the system ID of vg00.
/etc/lvm/system-id doesn't exist.

The server is explicitly configured with not having a system ID in
/etc/lvm/lvm.conf:

system_id_source = "none"


> As this alters nothing in the LVM itself, this should not be harmful as a try.

On the one hand, yes. On the other, I am very hesitant to mess with the
server lvm configuration, and potentially having lvm lose access to
vg01, which hosts all of the servers partitions.

I am going to read up more on this topic, and recovery options for the
case of any system ID issues during boot; just to be prepared.
I've not have had ever any trouble with lvm in all the years of using
it ("it just worked"), until now.

The lvmsystemid man page also mentions to use the lvmlocal.conf entry

   local {
   extra_system_ids = [ "my_other_name" ]
   }

instead the command line option; I might give that a try, as it seems
fairly safe.


> Am 13.01.19 um 17:10 schrieb Jens Holzhäuser:
> > Hello,
> > 
> > I have a buster/sid system with two volume groups that have made no
> > issue for years.
> 
> [...]
> 
> > 
> > # lvm systemid
> >   system ID:
> > # vgs -o systemid vg00
> >   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> > system ID.
> >   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> > system ID.
> > # vgs --foreign -o +systemid
> >   VG   #PV #LV #SN Attr   VSize   VFree   System ID
> >   vg00   1   8   0 wz--n- 930.55g  20.00g zaphod1105820973
> >   vg01   1   9   0 wz--n-  <1.50t 534.37g
> > # vgchange --config 'local/extra_system_ids=["zaphod1105820973"]' 
> > --systemid "" vg00
> >   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> > system ID.
> >   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> > system ID.
> > 
> > 
> > Any help is appreciated to get back acess to the VG, vgchange (and
> > vgexport) seem to be ignoring it despite providing the extra system id.
> 
> 
> [...]
> 
> > Thanks,
> > 
> > 
> > Jens
> 
> Martin

Jens



Re: Unable to Access a Foreign Volume Group

2019-01-13 Thread Martin
Hi Jens,

my first shot would be setting the actual system id to 'zaphod1105820973' like 
described in the lvmsystemid man page. I'm not sure, how this uname or lvmlocal 
work, so I would try setting system_id_source to machineid or file.
First, look if /etc/machine-id matches. If not, put is may be in a 
/etc/lvm/system-id.

As this alters nothing in the LVM itself, this should not be harmful as a try.


Am 13.01.19 um 17:10 schrieb Jens Holzhäuser:
> Hello,
> 
> I have a buster/sid system with two volume groups that have made no
> issue for years.

[...]

> 
> # lvm systemid
>   system ID:
> # vgs -o systemid vg00
>   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> system ID.
>   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> system ID.
> # vgs --foreign -o +systemid
>   VG   #PV #LV #SN Attr   VSize   VFree   System ID
>   vg00   1   8   0 wz--n- 930.55g  20.00g zaphod1105820973
>   vg01   1   9   0 wz--n-  <1.50t 534.37g
> # vgchange --config 'local/extra_system_ids=["zaphod1105820973"]' --systemid 
> "" vg00
>   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> system ID.
>   Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
> system ID.
> 
> 
> Any help is appreciated to get back acess to the VG, vgchange (and
> vgexport) seem to be ignoring it despite providing the extra system id.


[...]

> Thanks,
> 
> 
>   Jens

Martin



Unable to Access a Foreign Volume Group

2019-01-13 Thread Jens Holzhäuser
Hello,

I have a buster/sid system with two volume groups that have made no
issue for years.
vg00 has not been actively used for a longer time, I migrated off of it,
but keep it up and around.
vg01 is holding most of the data for the system and works with no
issues. 
Both are on two separate raid1's (see details at the end).

The server has no lvm system id set (never had) and both VGs worked
flawlessly for years. I've never concerned myself with setting a system
id for the standalone server, and am assuming neither VG ever had one
set.

During a recent package update apparently vg00 got a system ID set (see
below) and lvm now is ignoring it.

I am struggling to get access back to this VG, going by the
documentation:

# lvm systemid
  system ID:
# vgs -o systemid vg00
  Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
system ID.
  Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
system ID.
# vgs --foreign -o +systemid
  VG   #PV #LV #SN Attr   VSize   VFree   System ID
  vg00   1   8   0 wz--n- 930.55g  20.00g zaphod1105820973
  vg01   1   9   0 wz--n-  <1.50t 534.37g
# vgchange --config 'local/extra_system_ids=["zaphod1105820973"]' --systemid "" 
vg00
  Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
system ID.
  Cannot access VG vg00 with system ID zaphod1105820973 with unknown local 
system ID.


Any help is appreciated to get back acess to the VG, vgchange (and
vgexport) seem to be ignoring it despite providing the extra system id.




Here are more details on how I think the situation arose, and the system
setup.

During an update via aptitude (2019-01-06  12:08:38 to 2019-01-06  12:15:32)

# grep liblvm2cmd2 /var/log/apt/term.log
Selecting previously unselected package liblvm2cmd2.03:amd64.
Preparing to unpack .../liblvm2cmd2.03_2.03.02-1_amd64.deb ...
Unpacking liblvm2cmd2.03:amd64 (2.03.02-1) ...
Removing liblvm2cmd2.02:amd64 (2.02.176-4.1) ...
Setting up liblvm2cmd2.03:amd64 (2.03.02-1) ...


the following was logged in syslog, the first time the "foreign system
ID" is showing up in my logs:


Jan  6 12:14:39 localhost systemd[1]: lvm2-lvmetad.socket: Socket unit 
configuration has changed whi
le unit has been running, no open socket file descriptor left. The socket unit 
is not functional unt
il restarted.
Jan  6 12:14:39 localhost systemd[1]: Stopping Monitoring of LVM2 mirrors, 
snapshots etc. using dmev
entd or progress polling...
Jan  6 12:14:40 localhost lvm[32256]:   9 logical volume(s) in volume group 
"vg01" unmonitored
Jan  6 12:14:40 localhost lvm[32256]:   WARNING: Found LVs active in VG vg00 
with foreign system ID zaphod1105820973.  Possible data corruption.
Jan  6 12:14:40 localhost systemd[1]: lvm2-monitor.service: Succeeded.
Jan  6 12:14:40 localhost systemd[1]: Stopped Monitoring of LVM2 mirrors, 
snapshots etc. using dmeventd or progress polling.
Jan  6 12:14:40 localhost systemd[1]: dm-event.socket: Succeeded.
Jan  6 12:14:40 localhost systemd[1]: Closed Device-mapper event daemon FIFOs.
Jan  6 12:14:40 localhost systemd[1]: Stopping Device-mapper event daemon FIFOs.
Jan  6 12:14:40 localhost systemd[1]: Listening on Device-mapper event daemon 
FIFOs.
Jan  6 12:14:40 localhost systemd[1]: Starting Monitoring of LVM2 mirrors, 
snapshots etc. using dmeventd or progress polling...
Jan  6 12:14:40 localhost lvm[32257]:   9 logical volume(s) in volume group 
"vg01" monitored
Jan  6 12:14:40 localhost lvm[32257]:   WARNING: Found LVs active in VG vg00 
with foreign system ID zaphod1105820973.  Possible data corruption.
Jan  6 12:14:40 localhost systemd[1]: Started Monitoring of LVM2 mirrors, 
snapshots etc. using dmeventd or progress polling.


Apparently lvm2 was already updated at that point to a newer version
earlier that day (2019-01-04  08:28:56 to 2019-01-04  08:32:37), not
sure how/why this happened:

# grep lvm2 /var/log/apt/term.log
Preparing to unpack .../lvm2_2.03.02-1_amd64.deb ...
Unpacking lvm2 (2.03.02-1) over (2.02.176-4.1) ...
Removing liblvm2app2.2:amd64 (2.02.176-4.1) ...
Setting up lvm2 (2.03.02-1) ...


# apt-cache policy lvm2 liblvm2cmd2.03
lvm2:
  Installed: 2.03.02-1
  Candidate: 2.03.02-1
  Version table:
 *** 2.03.02-1 500
500 http://ftp.us.debian.org/debian testing/main amd64 Packages
   -100 http://ftp.us.debian.org/debian unstable/main amd64 Packages
100 /var/lib/dpkg/status
liblvm2cmd2.03:
  Installed: 2.03.02-1
  Candidate: 2.03.02-1
  Version table:
 *** 2.03.02-1 500
500 http://ftp.us.debian.org/debian testing/main amd64 Packages
   -100 http://ftp.us.debian.org/debian unstable/main amd64 Packages
100 /var/lib/dpkg/status


# uname -a
Linux zaphod 4.20.1-20190110.zaphod #1 SMP Thu Jan 10 11:38:55 EST 2019 x86_64 
GNU/Linux

Relevant output from lsblk only, vg00 is on md1, vg01 is on md2:

# lsblk -i
sdb  8:16   0 931.5G  0 disk
|-sdb1   8:17   0   979M  0 part
`-sdb2   8:18   0 930.6G