Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-12 Thread Nuno Teixeira
(...)

Trying :

`zpool set feature@block_cloning=disabled zroot`:
cannot set property for 'zroot': property 'feature@block_cloning' can only
be set to 'disabled' at creation time

Nuno Teixeira  escreveu no dia quarta, 12/04/2023 à(s)
13:57:

> Hello all,
>
> at current 3fdb40d1befe after `zfs upgrade XXX`:
>
> same problem when running compiler:
>
> - poudriere: crash without dump
> - make buildworld (/usr/src): shutdown -p (I will try to get a photo)
>
> Is there a way to disable block clone?
>
> Cy Schubert  escreveu no dia terça, 11/04/2023
> à(s) 15:47:
>
>> In message <20230411142831.db824...@slippy.cwsent.com>, Cy Schubert
>> writes:
>> > In message <434b83db-f6bb-436f-8aa5-385730d20...@dawidek.net>,
>> > =?utf-8?Q?Pawe=C
>> > 5=82_Jakub_Dawidek?= writes:
>> > >
>> > >
>> > > > On Apr 11, 2023, at 11:31, Cy Schubert 
>> wrote:
>> > > >=20
>> > > > =EF=BB=BFIn message
>> <20230409161436.5412fa6e@thor.intern.walstatt.dynvpn.
>> > d=
>> > > e>,=20
>> > > > FreeBSD Us
>> > > > er writes:
>> > > >> Am Sun, 9 Apr 2023 14:37:03 +0200
>> > > >> Mateusz Guzik  schrieb:
>> > > >>=20
>> > >  On 4/9/23, FreeBSD User  wrote:
>> > > > Today, after upgrading to FreeBSD 14.0-CURRENT #8
>> main-n262052-0d4038
>> > e=
>> > > 301
>> > > >>> 2b:
>> > > > Sun Apr  9
>> > > > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
>> > > >=20
>> > > > zpool upgrade POOLNAME
>> > > >=20
>> > > > some boxes keep crashing when starting compiler runs (the
>> trigger is
>> > > > different on boxes).
>> > > >=20
>> > > > ZFS module is statically compiled into the kernel (if this is of
>> > > > importance)
>> > > >=20
>> > > > Last known good was:
>> > > >=20
>> > > > [...]
>> > > > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
>> > > > main-n262051-75379ea2e461: Sun Apr
>> > > > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
>> > > > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9
>> 07:10:04
>> >  <
>> > > =
>> > > 0.
>> > > >>> 2>
>> > > > thor kernel:
>> > > > FreeBSD clang version 15.0.7 (
>> https://github.com/llvm/llvm-project.gi
>> > t=
>> > >
>> > > > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor
>> kernel:
>> > > > VT(efifb): resolution
>> > > > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl
>> already
>> > > > present!
>> > > > [...]
>> > > >=20
>> > > > The file /var/crash/info.X
>> > > >=20
>> > > > contains:
>> > > >=20
>> > > > [...]
>> > > >=20
>> > > > root@thor:/var/crash # more info.2
>> > > > Dump header from device: /dev/gpt/swap
>> > > >  Architecture: amd64
>> > > >  Architecture Version: 2
>> > > >  Dump Length: 1095192576
>> > > >  Blocksize: 512
>> > > >  Compression: none
>> > > >  Dumptime: 2023-04-09 11:43:41 +
>> > > >  Hostname: thor.local
>> > > >  Magic: FreeBSD Kernel Dump
>> > > >  Version String: FreeBSD 14.0-CURRENT #8
>> main-n262052-0d4038e3012b: S
>> > u=
>> > > n=20
>> > > >>> Apr
>> > > > 9 12:01:02 CEST
>> > > > 2023
>> > > >root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
>> > > >  Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
>> > > >=20
>> > > >  Dump Parity: 2961465682
>> > > >  Bounds: 2
>> > > >  Dump Status: good
>> > > >=20
>> > > > Until reconfigured for more debug stuff I do not have more to
>> present
>> > .=
>> > >
>> > > >=20
>> > > > I rememeber now really scraed that there was a HEADSUP in the
>> list re
>> > g=
>> > > ard
>> > > >>> ing
>> > > > some serious ZFS
>> > > > problems - I didn't find it right now.
>> > > >=20
>> > > > Thanks in advance,
>> > > >=20
>> > > >>>=20
>> > > >>> That's fallout from the new block cloning feature, adding the
>> author
>> > > >>>=20
>> > > >>=20
>> > > >> Thanks.
>> > > >>=20
>> > > >> As of this moment, all systems with the newest kernel and the new
>> ZFS op
>> > t=
>> > > ion=20
>> > > >> enabled, crash -
>> > > >> the reason is mostly in  different ZFS datasets. I guess there is
>> no way
>> >  b
>> > > =
>> > > ack
>> > > >> once this faulty
>> > > >> option is enabled?
>> > > >=20
>> > > > I've run a test on a scratch pool here, first without
>> block_cloning=20
>> > > > enabled, then with. There was no corruption when block_cloning
>> was=20
>> > > > disabled. There was corruption when block_cloning was enabled.
>> > > >=20
>> > > > I don't know of any way to revert back nor is there any way to fix
>> or=20
>> > > > recover the corrupted blocks.
>> > >
>> > > Is the corruption still present after EXDEV fixes?
>> >
>> > Yes and no.
>> >
>> > Yes, there is corruption when block_cloning is enabled.
>> >
>> > There is no corruption when block_cloning is disabled.
>>
>> I should add some detail to this.
>>
>> The corruption experienced when block cloning is disabled was fixed by:
>>
>> - eb1feadc201a
>> - 

Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-12 Thread Nuno Teixeira
Hello all,

at current 3fdb40d1befe after `zfs upgrade XXX`:

same problem when running compiler:

- poudriere: crash without dump
- make buildworld (/usr/src): shutdown -p (I will try to get a photo)

Is there a way to disable block clone?

Cy Schubert  escreveu no dia terça, 11/04/2023
à(s) 15:47:

> In message <20230411142831.db824...@slippy.cwsent.com>, Cy Schubert
> writes:
> > In message <434b83db-f6bb-436f-8aa5-385730d20...@dawidek.net>,
> > =?utf-8?Q?Pawe=C
> > 5=82_Jakub_Dawidek?= writes:
> > >
> > >
> > > > On Apr 11, 2023, at 11:31, Cy Schubert 
> wrote:
> > > >=20
> > > > =EF=BB=BFIn message
> <20230409161436.5412fa6e@thor.intern.walstatt.dynvpn.
> > d=
> > > e>,=20
> > > > FreeBSD Us
> > > > er writes:
> > > >> Am Sun, 9 Apr 2023 14:37:03 +0200
> > > >> Mateusz Guzik  schrieb:
> > > >>=20
> > >  On 4/9/23, FreeBSD User  wrote:
> > > > Today, after upgrading to FreeBSD 14.0-CURRENT #8
> main-n262052-0d4038
> > e=
> > > 301
> > > >>> 2b:
> > > > Sun Apr  9
> > > > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
> > > >=20
> > > > zpool upgrade POOLNAME
> > > >=20
> > > > some boxes keep crashing when starting compiler runs (the
> trigger is
> > > > different on boxes).
> > > >=20
> > > > ZFS module is statically compiled into the kernel (if this is of
> > > > importance)
> > > >=20
> > > > Last known good was:
> > > >=20
> > > > [...]
> > > > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> > > > main-n262051-75379ea2e461: Sun Apr
> > > > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> > > > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9
> 07:10:04
> >  <
> > > =
> > > 0.
> > > >>> 2>
> > > > thor kernel:
> > > > FreeBSD clang version 15.0.7 (
> https://github.com/llvm/llvm-project.gi
> > t=
> > >
> > > > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor
> kernel:
> > > > VT(efifb): resolution
> > > > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl
> already
> > > > present!
> > > > [...]
> > > >=20
> > > > The file /var/crash/info.X
> > > >=20
> > > > contains:
> > > >=20
> > > > [...]
> > > >=20
> > > > root@thor:/var/crash # more info.2
> > > > Dump header from device: /dev/gpt/swap
> > > >  Architecture: amd64
> > > >  Architecture Version: 2
> > > >  Dump Length: 1095192576
> > > >  Blocksize: 512
> > > >  Compression: none
> > > >  Dumptime: 2023-04-09 11:43:41 +
> > > >  Hostname: thor.local
> > > >  Magic: FreeBSD Kernel Dump
> > > >  Version String: FreeBSD 14.0-CURRENT #8
> main-n262052-0d4038e3012b: S
> > u=
> > > n=20
> > > >>> Apr
> > > > 9 12:01:02 CEST
> > > > 2023
> > > >root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
> > > >  Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
> > > >=20
> > > >  Dump Parity: 2961465682
> > > >  Bounds: 2
> > > >  Dump Status: good
> > > >=20
> > > > Until reconfigured for more debug stuff I do not have more to
> present
> > .=
> > >
> > > >=20
> > > > I rememeber now really scraed that there was a HEADSUP in the
> list re
> > g=
> > > ard
> > > >>> ing
> > > > some serious ZFS
> > > > problems - I didn't find it right now.
> > > >=20
> > > > Thanks in advance,
> > > >=20
> > > >>>=20
> > > >>> That's fallout from the new block cloning feature, adding the
> author
> > > >>>=20
> > > >>=20
> > > >> Thanks.
> > > >>=20
> > > >> As of this moment, all systems with the newest kernel and the new
> ZFS op
> > t=
> > > ion=20
> > > >> enabled, crash -
> > > >> the reason is mostly in  different ZFS datasets. I guess there is
> no way
> >  b
> > > =
> > > ack
> > > >> once this faulty
> > > >> option is enabled?
> > > >=20
> > > > I've run a test on a scratch pool here, first without
> block_cloning=20
> > > > enabled, then with. There was no corruption when block_cloning was=20
> > > > disabled. There was corruption when block_cloning was enabled.
> > > >=20
> > > > I don't know of any way to revert back nor is there any way to fix
> or=20
> > > > recover the corrupted blocks.
> > >
> > > Is the corruption still present after EXDEV fixes?
> >
> > Yes and no.
> >
> > Yes, there is corruption when block_cloning is enabled.
> >
> > There is no corruption when block_cloning is disabled.
>
> I should add some detail to this.
>
> The corruption experienced when block cloning is disabled was fixed by:
>
> - eb1feadc201a
> - e2d997d1cbb9
> - d012836fb616 (specifically this commit)
> - 20be1b4fc4b7
>
> When block_cloning is enabled, the pool is corrupted. This has not been
> fixed.
>
>
> --
> Cheers,
> Cy Schubert 
> FreeBSD UNIX: Web:  https://FreeBSD.org
> NTP:   Web:  https://nwtime.org
>
> e^(i*pi)+1=0
>
>
>
>

-- 
Nuno Teixeira
FreeBSD Committer (ports)


Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-11 Thread Cy Schubert
In message <20230411142831.db824...@slippy.cwsent.com>, Cy Schubert writes:
> In message <434b83db-f6bb-436f-8aa5-385730d20...@dawidek.net>, 
> =?utf-8?Q?Pawe=C
> 5=82_Jakub_Dawidek?= writes:
> > 
> >
> > > On Apr 11, 2023, at 11:31, Cy Schubert  wrote:
> > >=20
> > > =EF=BB=BFIn message <20230409161436.5412fa6e@thor.intern.walstatt.dynvpn.
> d=
> > e>,=20
> > > FreeBSD Us
> > > er writes:
> > >> Am Sun, 9 Apr 2023 14:37:03 +0200
> > >> Mateusz Guzik  schrieb:
> > >>=20
> >  On 4/9/23, FreeBSD User  wrote:
> > > Today, after upgrading to FreeBSD 14.0-CURRENT #8 main-n262052-0d4038
> e=
> > 301
> > >>> 2b:
> > > Sun Apr  9
> > > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
> > >=20
> > > zpool upgrade POOLNAME
> > >=20
> > > some boxes keep crashing when starting compiler runs (the trigger is
> > > different on boxes).
> > >=20
> > > ZFS module is statically compiled into the kernel (if this is of
> > > importance)
> > >=20
> > > Last known good was:
> > >=20
> > > [...]
> > > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> > > main-n262051-75379ea2e461: Sun Apr
> > > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> > > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9 07:10:04
>  <
> > =
> > 0.
> > >>> 2>
> > > thor kernel:
> > > FreeBSD clang version 15.0.7 (https://github.com/llvm/llvm-project.gi
> t=
> >
> > > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor kernel:
> > > VT(efifb): resolution
> > > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl already
> > > present!
> > > [...]
> > >=20
> > > The file /var/crash/info.X
> > >=20
> > > contains:
> > >=20
> > > [...]
> > >=20
> > > root@thor:/var/crash # more info.2
> > > Dump header from device: /dev/gpt/swap
> > >  Architecture: amd64
> > >  Architecture Version: 2
> > >  Dump Length: 1095192576
> > >  Blocksize: 512
> > >  Compression: none
> > >  Dumptime: 2023-04-09 11:43:41 +
> > >  Hostname: thor.local
> > >  Magic: FreeBSD Kernel Dump
> > >  Version String: FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b: S
> u=
> > n=20
> > >>> Apr
> > > 9 12:01:02 CEST
> > > 2023
> > >root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
> > >  Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
> > >=20
> > >  Dump Parity: 2961465682
> > >  Bounds: 2
> > >  Dump Status: good
> > >=20
> > > Until reconfigured for more debug stuff I do not have more to present
> .=
> >
> > >=20
> > > I rememeber now really scraed that there was a HEADSUP in the list re
> g=
> > ard
> > >>> ing
> > > some serious ZFS
> > > problems - I didn't find it right now.
> > >=20
> > > Thanks in advance,
> > >=20
> > >>>=20
> > >>> That's fallout from the new block cloning feature, adding the author
> > >>>=20
> > >>=20
> > >> Thanks.
> > >>=20
> > >> As of this moment, all systems with the newest kernel and the new ZFS op
> t=
> > ion=20
> > >> enabled, crash -
> > >> the reason is mostly in  different ZFS datasets. I guess there is no way
>  b
> > =
> > ack
> > >> once this faulty
> > >> option is enabled?
> > >=20
> > > I've run a test on a scratch pool here, first without block_cloning=20
> > > enabled, then with. There was no corruption when block_cloning was=20
> > > disabled. There was corruption when block_cloning was enabled.
> > >=20
> > > I don't know of any way to revert back nor is there any way to fix or=20
> > > recover the corrupted blocks.
> >
> > Is the corruption still present after EXDEV fixes?
>
> Yes and no.
>
> Yes, there is corruption when block_cloning is enabled.
>
> There is no corruption when block_cloning is disabled.

I should add some detail to this.

The corruption experienced when block cloning is disabled was fixed by:

- eb1feadc201a
- e2d997d1cbb9
- d012836fb616 (specifically this commit)
- 20be1b4fc4b7

When block_cloning is enabled, the pool is corrupted. This has not been 
fixed.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  https://FreeBSD.org
NTP:   Web:  https://nwtime.org

e^(i*pi)+1=0





Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-11 Thread Cy Schubert
In message <434b83db-f6bb-436f-8aa5-385730d20...@dawidek.net>, 
=?utf-8?Q?Pawe=C
5=82_Jakub_Dawidek?= writes:
> 
>
> > On Apr 11, 2023, at 11:31, Cy Schubert  wrote:
> >=20
> > =EF=BB=BFIn message <20230409161436.5412fa6e@thor.intern.walstatt.dynvpn.d=
> e>,=20
> > FreeBSD Us
> > er writes:
> >> Am Sun, 9 Apr 2023 14:37:03 +0200
> >> Mateusz Guzik  schrieb:
> >>=20
>  On 4/9/23, FreeBSD User  wrote:
> > Today, after upgrading to FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e=
> 301
> >>> 2b:
> > Sun Apr  9
> > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
> >=20
> > zpool upgrade POOLNAME
> >=20
> > some boxes keep crashing when starting compiler runs (the trigger is
> > different on boxes).
> >=20
> > ZFS module is statically compiled into the kernel (if this is of
> > importance)
> >=20
> > Last known good was:
> >=20
> > [...]
> > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> > main-n262051-75379ea2e461: Sun Apr
> > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9 07:10:04 <
> =
> 0.
> >>> 2>
> > thor kernel:
> > FreeBSD clang version 15.0.7 (https://github.com/llvm/llvm-project.git=
>
> > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor kernel:
> > VT(efifb): resolution
> > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl already
> > present!
> > [...]
> >=20
> > The file /var/crash/info.X
> >=20
> > contains:
> >=20
> > [...]
> >=20
> > root@thor:/var/crash # more info.2
> > Dump header from device: /dev/gpt/swap
> >  Architecture: amd64
> >  Architecture Version: 2
> >  Dump Length: 1095192576
> >  Blocksize: 512
> >  Compression: none
> >  Dumptime: 2023-04-09 11:43:41 +
> >  Hostname: thor.local
> >  Magic: FreeBSD Kernel Dump
> >  Version String: FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b: Su=
> n=20
> >>> Apr
> > 9 12:01:02 CEST
> > 2023
> >root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
> >  Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
> >=20
> >  Dump Parity: 2961465682
> >  Bounds: 2
> >  Dump Status: good
> >=20
> > Until reconfigured for more debug stuff I do not have more to present.=
>
> >=20
> > I rememeber now really scraed that there was a HEADSUP in the list reg=
> ard
> >>> ing
> > some serious ZFS
> > problems - I didn't find it right now.
> >=20
> > Thanks in advance,
> >=20
> >>>=20
> >>> That's fallout from the new block cloning feature, adding the author
> >>>=20
> >>=20
> >> Thanks.
> >>=20
> >> As of this moment, all systems with the newest kernel and the new ZFS opt=
> ion=20
> >> enabled, crash -
> >> the reason is mostly in  different ZFS datasets. I guess there is no way b
> =
> ack
> >> once this faulty
> >> option is enabled?
> >=20
> > I've run a test on a scratch pool here, first without block_cloning=20
> > enabled, then with. There was no corruption when block_cloning was=20
> > disabled. There was corruption when block_cloning was enabled.
> >=20
> > I don't know of any way to revert back nor is there any way to fix or=20
> > recover the corrupted blocks.
>
> Is the corruption still present after EXDEV fixes?

Yes and no.

Yes, there is corruption when block_cloning is enabled.

There is no corruption when block_cloning is disabled.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  https://FreeBSD.org
NTP:   Web:  https://nwtime.org

e^(i*pi)+1=0





Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-10 Thread Cy Schubert
In message <20230409161436.5412f...@thor.intern.walstatt.dynvpn.de>, 
FreeBSD Us
er writes:
> Am Sun, 9 Apr 2023 14:37:03 +0200
> Mateusz Guzik  schrieb:
>
> > On 4/9/23, FreeBSD User  wrote:
> > > Today, after upgrading to FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e301
> 2b:
> > > Sun Apr  9
> > > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
> > >
> > > zpool upgrade POOLNAME
> > >
> > > some boxes keep crashing when starting compiler runs (the trigger is
> > > different on boxes).
> > >
> > > ZFS module is statically compiled into the kernel (if this is of
> > > importance)
> > >
> > > Last known good was:
> > >
> > > [...]
> > > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> > > main-n262051-75379ea2e461: Sun Apr
> > > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> > > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9 07:10:04 <0.
> 2>
> > > thor kernel:
> > > FreeBSD clang version 15.0.7 (https://github.com/llvm/llvm-project.git
> > > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor kernel:
> > > VT(efifb): resolution
> > > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl already
> > > present!
> > > [...]
> > >
> > > The file /var/crash/info.X
> > >
> > > contains:
> > >
> > > [...]
> > >
> > > root@thor:/var/crash # more info.2
> > > Dump header from device: /dev/gpt/swap
> > >   Architecture: amd64
> > >   Architecture Version: 2
> > >   Dump Length: 1095192576
> > >   Blocksize: 512
> > >   Compression: none
> > >   Dumptime: 2023-04-09 11:43:41 +
> > >   Hostname: thor.local
> > >   Magic: FreeBSD Kernel Dump
> > >   Version String: FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b: Sun 
> Apr
> > >  9 12:01:02 CEST
> > > 2023
> > > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
> > >   Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
> > >
> > >   Dump Parity: 2961465682
> > >   Bounds: 2
> > >   Dump Status: good
> > >
> > > Until reconfigured for more debug stuff I do not have more to present.
> > >
> > > I rememeber now really scraed that there was a HEADSUP in the list regard
> ing
> > > some serious ZFS
> > > problems - I didn't find it right now.
> > >
> > > Thanks in advance,
> > >  
> > 
> > That's fallout from the new block cloning feature, adding the author
> > 
>
> Thanks.
>
> As of this moment, all systems with the newest kernel and the new ZFS option 
> enabled, crash -
> the reason is mostly in  different ZFS datasets. I guess there is no way back
>  once this faulty
> option is enabled?

I've run a test on a scratch pool here, first without block_cloning 
enabled, then with. There was no corruption when block_cloning was 
disabled. There was corruption when block_cloning was enabled.

I don't know of any way to revert back nor is there any way to fix or 
recover the corrupted blocks.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  https://FreeBSD.org
NTP:   Web:  https://nwtime.org

e^(i*pi)+1=0





Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-09 Thread FreeBSD User
Am Sun, 9 Apr 2023 14:37:03 +0200
Mateusz Guzik  schrieb:

> On 4/9/23, FreeBSD User  wrote:
> > Today, after upgrading to FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b:
> > Sun Apr  9
> > 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
> >
> > zpool upgrade POOLNAME
> >
> > some boxes keep crashing when starting compiler runs (the trigger is
> > different on boxes).
> >
> > ZFS module is statically compiled into the kernel (if this is of
> > importance)
> >
> > Last known good was:
> >
> > [...]
> > Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> > main-n262051-75379ea2e461: Sun Apr
> > 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9 07:10:04 <0.2>
> > thor kernel:
> > FreeBSD clang version 15.0.7 (https://github.com/llvm/llvm-project.git
> > llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor kernel:
> > VT(efifb): resolution
> > 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl already
> > present!
> > [...]
> >
> > The file /var/crash/info.X
> >
> > contains:
> >
> > [...]
> >
> > root@thor:/var/crash # more info.2
> > Dump header from device: /dev/gpt/swap
> >   Architecture: amd64
> >   Architecture Version: 2
> >   Dump Length: 1095192576
> >   Blocksize: 512
> >   Compression: none
> >   Dumptime: 2023-04-09 11:43:41 +
> >   Hostname: thor.local
> >   Magic: FreeBSD Kernel Dump
> >   Version String: FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b: Sun Apr
> >  9 12:01:02 CEST
> > 2023
> > root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
> >   Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
> >
> >   Dump Parity: 2961465682
> >   Bounds: 2
> >   Dump Status: good
> >
> > Until reconfigured for more debug stuff I do not have more to present.
> >
> > I rememeber now really scraed that there was a HEADSUP in the list regarding
> > some serious ZFS
> > problems - I didn't find it right now.
> >
> > Thanks in advance,
> >  
> 
> That's fallout from the new block cloning feature, adding the author
> 

Thanks.

As of this moment, all systems with the newest kernel and the new ZFS option 
enabled, crash -
the reason is mostly in  different ZFS datasets. I guess there is no way back 
once this faulty
option is enabled?

Regards,

oh

-- 
O. Hartmann



Re: CURRENT: Panic VERIFY(!zil_replaying(zilog, tx)) failed (and crashing)

2023-04-09 Thread Mateusz Guzik
On 4/9/23, FreeBSD User  wrote:
> Today, after upgrading to FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b:
> Sun Apr  9
> 12:01:02 CEST 2023  amd64, AND upgrading ZPOOLs via
>
> zpool upgrade POOLNAME
>
> some boxes keep crashing when starting compiler runs (the trigger is
> different on boxes).
>
> ZFS module is statically compiled into the kernel (if this is of
> importance)
>
> Last known good was:
>
> [...]
> Apr  9 07:10:04 <0.2> thor kernel: FreeBSD 14.0-CURRENT #7
> main-n262051-75379ea2e461: Sun Apr
> 9 00:12:57 CEST 2023 Apr  9 07:10:04 <0.2> thor kernel:
> root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR amd64 Apr  9 07:10:04 <0.2>
> thor kernel:
> FreeBSD clang version 15.0.7 (https://github.com/llvm/llvm-project.git
> llvmorg-15.0.7-0-g8dfdcc7b7bf6) Apr  9 07:10:04 <0.2> thor kernel:
> VT(efifb): resolution
> 2560x1440 Apr  9 07:10:04 <0.2> thor kernel: module zfsctrl already
> present!
> [...]
>
> The file /var/crash/info.X
>
> contains:
>
> [...]
>
> root@thor:/var/crash # more info.2
> Dump header from device: /dev/gpt/swap
>   Architecture: amd64
>   Architecture Version: 2
>   Dump Length: 1095192576
>   Blocksize: 512
>   Compression: none
>   Dumptime: 2023-04-09 11:43:41 +
>   Hostname: thor.local
>   Magic: FreeBSD Kernel Dump
>   Version String: FreeBSD 14.0-CURRENT #8 main-n262052-0d4038e3012b: Sun Apr
>  9 12:01:02 CEST
> 2023
> root@thor:/usr/obj/usr/src/amd64.amd64/sys/THOR
>   Panic String: VERIFY(!zil_replaying(zilog, tx)) failed
>
>   Dump Parity: 2961465682
>   Bounds: 2
>   Dump Status: good
>
> Until reconfigured for more debug stuff I do not have more to present.
>
> I rememeber now really scraed that there was a HEADSUP in the list regarding
> some serious ZFS
> problems - I didn't find it right now.
>
> Thanks in advance,
>

That's fallout from the new block cloning feature, adding the author

-- 
Mateusz Guzik