Re: [ZFS] files in a weird situtation
On Sat, Dec 17, 2016 at 9:01 PM, David Marec wrote: > > david:~>ls /lib/libjail.so.1 > ls: /lib/libjail.so.1: No such file or directory > david:~>find /lib -name "libjail.so.1" -print > /lib/libjail.so.1 > david:~>find /lib -name "libjail.so.1" -ls > find: /lib/libjail.so.1: No such file or directory > You have a directory entry pointing at a freed inode (or zfs equivalent). -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On 17.12.2016 22:15, Brandon Allbery wrote: You have a directory entry pointing at a freed inode (or zfs equivalent). ZFS may have mapped this inode that points to nowhere. That makes sense. In this case, what should be the best solution to clean this up ? As I said, as far scrubbing the pool didn't show any error, it didn't solve the issue. -- David Marec https:lapinbilly.eu ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On Sat, Dec 17, 2016 at 10:25 PM, David Marec wrote: > In this case, what should be the best solution to clean this up ? > > As I said, as far scrubbing the pool didn't show any error, it didn't > solve the issue. > That I don't know. With a nore Unix-like filesystem I'd run fsck; if scrubbing the zpool didn't fix it, you may need a zfs expert :/ -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On Sat, Dec 17, 2016 at 3:01 PM, David Marec
wrote:
> [I had first posted onto the Forum about this issue]
>
> Two months ago,
>
> - next to a call to |`||delete-old-libs`| or `install world`, I don't
> really know -
>
> my box that is following FreeBSD-11 Stable ran into a weird situation.
>
> A set of files, especially `/lib/libjail.so.1` are in both states
> `existing` and `not existing`:
>
> I means:
>
> david:~>cp ~david/libjail.so.1 /lib
> cp: /lib/libjail.so.1: File exists
>
> But:
>
> david:~>ls /lib/libjail.so.1
> ls: /lib/libjail.so.1: No such file or directory
> david:~>find /lib -name "libjail.so.1" -print
> /lib/libjail.so.1
> david:~>find /lib -name "libjail.so.1" -ls
> find: /lib/libjail.so.1: No such file or directory
>
> With deeper investigation, the file is in fact mapped to an `inode`:
>
> root@dmarec:~ # ls -di /lib
> 13 /lib
> root@dmarec:~ # zdb - zroot/ 13 | grep libjail.so.1
> libjail.so.1 = 10552574 (type: Regular File)
>
> Which fails with `zdb` on:
>
> root@dmarec:~ # zdb - zroot/ 10552574
> Dataset zroot [ZPL], ID 21, cr_txg 1, 114G, 2570002 objects, rootbp
> DVA[0]=<0:b97d6ea00:200> DVA[1]=<0:1c212b0400:200> [L0 DMU objset]
> fletcher4 lz4 LE contiguous unique double size=800L/200P
> birth=3852240L/3852240P fill=2570002
> cksum=17b78fb7e4:7c87a526a07:16251edfaae60:2ce0c5734ccf2f
>
> Object lvl iblk dblk dsize lsize %full type
> zdb: dmu_bonus_hold(10552574) failed, errno 2
>
>
> `stat (2)` returns ENOENT when checking for the file:
> david:~>truss stat -L /lib/libjail.so.1
> ...
> stat("/lib/libjail.so.1",0x7fffe7e8) ERR#2 'No such
> file or
> directory'david:~>truss stat -L /lib/libjail.so.1
>
> A pass with `zfs scrub` didn't help.
>
> Any clue is welcome. What's that `dmu_bonus_hold` stands for ?
>
I am unable to understand what your intent is here. If you wish to delete
it, you can do:
find . -inum 10552574 -exec rm {} \;
--
Adam
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On 18.12.2016 01:38, Adam Vande More wrote:
I am unable to understand what your intent is here. If you wish to
delete it, you can do:
find . -inum 10552574 -exec rm {} \;
It fails on «No such file or directory».
This file missing, not much works.
I have installed a copy into `/usr/lib` to make the system run again.
So, I can't install a new world: `install -C` fails in the same way.
--
David Marec
https://lapinbilly.eu
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
David Marec wrote on 2016/12/18 09:30:
On 18.12.2016 01:38, Adam Vande More wrote:
I am unable to understand what your intent is here. If you wish to
delete it, you can do:
find . -inum 10552574 -exec rm {} \;
It fails on «No such file or directory».
This file missing, not much works.
I have installed a copy into `/usr/lib` to make the system run again.
So, I can't install a new world: `install -C` fails in the same way.
Then you can boot from recovery media like mfsBSD and unpack good known
distribution from kernel.txz and base.txz (or do make installkernel &&
make installworld from NFS mounted /usr/src and /usr/obj).
But if dataset in question is somehow corrupted maybe you need to make
different dataset, extract all files, remove old dataset and rename this
new dataset.
If this is pool error than it will be better to scratch it and start
again from backups.
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On Sat, Dec 17, 2016 at 11:25:41PM +0100, David Marec wrote: > On 17.12.2016 22:15, Brandon Allbery wrote: > > > > You have a directory entry pointing at a freed inode (or zfs equivalent). > > ZFS may have mapped this inode that points to nowhere. That makes sense. > > In this case, what should be the best solution to clean this up ? > > As I said, as far scrubbing the pool didn't show any error, it didn't > solve the issue. Have you tried deleting the /lib directory, or at least renaming it to /lib.bad (or something similar)? You'll need to boot from a rescue disc or boot single user with /rescue/sh as your shell, and then reinstall /lib from known good media (or "make installworld" in /usr/src) That may leave problems in the ZFS metadata still, but it should (in theory) let you continue using the system Gary P.S. I haven't tried any of this myself so be careful ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On Sun, Dec 18, 2016 at 2:30 AM, David Marec wrote: > It fails on «No such file or directory». > I can't even replicate this portion of things. Running it under truss might provide more insight into what is happening. > > This file missing, not much works. > I have installed a copy into `/usr/lib` to make the system run again. > > So, I can't install a new world: `install -C` fails in the same way. I think you have larger issues than a single corrupt file. -- Adam ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
On 2016-12-18 11:16, Adam Vande More wrote: On Sun, Dec 18, 2016 at 2:30 AM, David Marec wrote: It fails on «No such file or directory». I can't even replicate this portion of things. Running it under truss might provide more insight into what is happening. This file missing, not much works. I have installed a copy into `/usr/lib` to make the system run again. So, I can't install a new world: `install -C` fails in the same way. I think you have larger issues than a single corrupt file. I had a similar weirdness back a few weeks ago. I basically rebuilt my boot filesystem with a full make installworld destdir= and killed off the old one once I had everything there. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
find also has -delete which avoids the exec overhead, not much of an impact
here but worth noting if you're removing lots.
On 18 December 2016 at 00:38, Adam Vande More wrote:
> On Sat, Dec 17, 2016 at 3:01 PM, David Marec
> wrote:
>
> > [I had first posted onto the Forum about this issue]
> >
> > Two months ago,
> >
> > - next to a call to |`||delete-old-libs`| or `install world`, I don't
> > really know -
> >
> > my box that is following FreeBSD-11 Stable ran into a weird situation.
> >
> > A set of files, especially `/lib/libjail.so.1` are in both states
> > `existing` and `not existing`:
> >
> > I means:
> >
> > david:~>cp ~david/libjail.so.1 /lib
> > cp: /lib/libjail.so.1: File exists
> >
> > But:
> >
> > david:~>ls /lib/libjail.so.1
> > ls: /lib/libjail.so.1: No such file or directory
> > david:~>find /lib -name "libjail.so.1" -print
> > /lib/libjail.so.1
> > david:~>find /lib -name "libjail.so.1" -ls
> > find: /lib/libjail.so.1: No such file or directory
> >
> > With deeper investigation, the file is in fact mapped to an `inode`:
> >
> > root@dmarec:~ # ls -di /lib
> > 13 /lib
> > root@dmarec:~ # zdb - zroot/ 13 | grep libjail.so.1
> > libjail.so.1 = 10552574 (type: Regular File)
> >
> > Which fails with `zdb` on:
> >
> > root@dmarec:~ # zdb - zroot/ 10552574
> > Dataset zroot [ZPL], ID 21, cr_txg 1, 114G, 2570002 objects,
> rootbp
> > DVA[0]=<0:b97d6ea00:200> DVA[1]=<0:1c212b0400:200> [L0 DMU
> objset]
> > fletcher4 lz4 LE contiguous unique double size=800L/200P
> > birth=3852240L/3852240P fill=2570002
> > cksum=17b78fb7e4:7c87a526a07:16251edfaae60:2ce0c5734ccf2f
> >
> > Object lvl iblk dblk dsize lsize %full type
> > zdb: dmu_bonus_hold(10552574) failed, errno 2
> >
> >
> > `stat (2)` returns ENOENT when checking for the file:
> > david:~>truss stat -L /lib/libjail.so.1
> > ...
> > stat("/lib/libjail.so.1",0x7fffe7e8) ERR#2 'No such
> > file or
> > directory'david:~>truss stat -L /lib/libjail.so.1
> >
> > A pass with `zfs scrub` didn't help.
> >
> > Any clue is welcome. What's that `dmu_bonus_hold` stands for ?
> >
>
> I am unable to understand what your intent is here. If you wish to delete
> it, you can do:
>
> find . -inum 10552574 -exec rm {} \;
>
> --
> Adam
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [ZFS] files in a weird situtation
Hi, On 18.12.2016 02:01, David Marec wrote: > > A pass with `zfs scrub` didn't help. > > Any clue is welcome. What's that `dmu_bonus_hold` stands for ? > Just out of the curiosity - is it on a redundant pool and does the 'zpool status' report any error ? Eugene. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
