Bug#1050663: Fwd: Re: [Dar-support] Fwd: Bug#1050663: dar option --on-fly-isolate creates catalogue with broken slice_layout
>From the thread on the dar support mailing list:
https://sourceforge.net/p/dar/mailman/message/37890758/
On Sat, Sep 02 2023, Thomas wrote:
> On Wed, Aug 30, 2023 at 03:24:12PM -0500, John Goerzen wrote:
>> On Wed, Aug 30 2023, Denis Corbin wrote:
>>
>> >> Summary
>> >> ===
>> >> g++ 10 works fine with optimization.
>> >> g++ 11 or newer work only without optimization.
>> >> Hope this helps.
>> >
>> > Thanks for confirming this is a gcc issue. I don't know what should be the
>> > next
>> > step, if someone fills a bug report to gcc Debian maintainer?
>>
>> Well, to be sure, we have a couple of possibilities:
>>
>> 1) This is a gcc bug,
>>
>> or 2) There is something in dar (or, for that matter, bzip2 or some
>> other library) that is relying on some sort of C/C++ undefined behavior
>> (UB) that the optimizer is taking in a different direction. Other
>> possibilities could include race conditions, etc.
>
> To find the root cause why the optimizer does something harmfull in our
> case is way over my head.
>
> But...
> I have found a workaround for dar. The appended patch works for me with
> dar v2.7.11 and g++ v13.2.0, v12.3.0 and v11.4.0. Means, files generated
> with OnFlyIsolation are readable again with default optimizations
> activated.
> It seems the optimizer does not like the used ternary operator.
>
> I tested the patch with g++ v10.4.0 and 9.3.0, too without problems but
> these versions are working with or without this patch anyway.
>
> Hope this helps.
>
> Tom
>
> [2. text/x-diff; slice_layout.cpp-remove_ternary_operator.patch]...
diff -rupN DAR_a/src/libdar/slice_layout.cpp DAR_b/src/libdar/slice_layout.cpp
--- DAR_a/src/libdar/slice_layout.cpp 2023-09-02 09:08:49.657051708 +0200
+++ DAR_b/src/libdar/slice_layout.cpp 2023-09-02 09:11:39.240669572 +0200
@@ -54,7 +54,11 @@ namespace libdar
void slice_layout::write(generic_file & f) const
{
- char tmp = older_sar_than_v8 ? OLDER_THAN_V8 : V8;
+ char tmp = V8;
+ if(older_sar_than_v8)
+ {
+ tmp = OLDER_THAN_V8;
+ }
first_size.dump(f);
other_size.dump(f);
first_slice_header.dump(f);
Bug#1050663: dar option --on‐fly‐isolate creates catalogue with broken slice_layout
tags 1050663 upstream
thanks
Hi Thomas,
Thank you for this clear and well-documented report. This looks like a
bug in upstream dar. Would you please report it to Denis (upstream
author) on the mailing list at
https://sourceforge.net/projects/dar/lists/dar-support ?
I do also monitor that list and will chime in if there is anything
Debian-specific to add. Denis is very responsive and is going to be
making a new release soon, so maybe this could get into the release he
is preparing.
Thanks,
John
On Sun, Aug 27 2023, Thomas wrote:
> Package: dar
> Version: 2.7.10-2+b2
> Severity: normal
> X-Debbugs-Cc: debianbts-20230827181...@racbu.de
>
> Dear Maintainer,
>
> I try to do a differential backup with an earlier on-fly-isolated
> catalogue. It seems dar can't read the isolated catalogues anymore.
>
> , [ Error ]
> | Final memory cleanup...
> | exception type = [BUG] --
> | [source]
> | File ../../../src/libdar/slice_layout.cpp line 48 : it seems to be
> a bug here
> | stack dump :
> |
> /lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar4EbugC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi+0x140)
> | [0x7f4ff3f79d50]
> | stack dump : /lib/x86_64-linux-gnu/libdar64.so.6000(+0xf83bb)
> [0x7f4ff3ef83bb]
> | stack dump :
> |
> /lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar14header_version4readERNS_12generic_fileERNS_16user_interactionEb+0x264)
> | [0x7f4ff3fbc944]
> | stack dump :
> |
> /lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar24macro_tools_open_archiveERKSt10shared_ptrINS_16user_interactionEERKS0_INS_8entrepotEERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_8limitintImEESG_NS_11crypto_algoERKNS_11secu_stringEjRNS_4pileERNS_14header_versionESG_SG_SG_RSI_RNS9_4listINS_8signatorESaISV_EEERNS_12slice_layoutEmmb+0x3aa)
> | [0x7f4ff3fe526a]
> | stack dump :
> |
> /lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar7archive9i_archiveC1ERKSt10shared_ptrINS_16user_interactionEERKNS_4pathERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESH_RKNS_20archive_options_readE+0x44f)
> | [0x7f4ff3fc1d6f]
> | stack dump :
> |
> /lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar7archiveC2ERKSt10shared_ptrINS_16user_interactionEERKNS_4pathERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESG_RKNS_20archive_options_readE+0xd6)
> | [0x7f4ff3f248c6]
> | stack dump : dar(+0x49306) [0x560a2c6b1306]
> | stack dump : dar(+0x5158c) [0x560a2c6b958c]
> | stack dump : dar(+0x247d1) [0x560a2c68c7d1]
> | stack dump : /lib/x86_64-linux-gnu/libc.so.6(+0x276ca)
> [0x7f4ff38456ca]
> | stack dump :
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7f4ff3845785]
> | stack dump : dar(+0x24821) [0x560a2c68c821]
> | [most outside call]
> | ---
> `
>
> Reproduce the error
>
> This is a minimal example to reproduce the error.
>
> ### Create som structure to backup
> $ export BASE=/tmp/dar_debug
> $ mkdir -p ${BASE}/bak ${BASE}/cat ${BASE}/files
> $ touch ${BASE}/files/file1
>
> ### Create the backup and the on-fly-isolation
> $ dar --create ${BASE}/bak/full --fs-root ${BASE}/files/ --aux
> ${BASE}/cat/full_onthefly
>
> ### Do a manual isolation after the backup is done
> $ dar --ref ${BASE}/bak/full --isolate ${BASE}/cat/full_isolate
>
> ### As described in the man page on-fly-isolation is always compressed so
> ### lets do this manually, too
> $ dar --ref ${BASE}/bak/full --compression=bzip2 --isolate
> ${BASE}/cat/full_isolate_compressed
>
> ### checking the results
> # the backup itself and the after backup-catalogues are working:
> $ dar --list ${BASE}/bak/full
> [Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
> Date |filename
> ++---+---+-+---+
> [Saved][ ] [-L-][ ][ ] -rw-r--r-- thomas thomas 0 Sun Aug
> 27 18:45:31 2023file1
>
> $ dar --list ${BASE}/cat/full_isolate
> [Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
> Date |filename
> ++---+---+-+---+
> [InRef][ ] [-L-][-][ ] -rw-r--r-- thomas thomas 0 Sun Aug
> 27 18:45:31 2023file1
>
> $ dar --list ${BASE}/cat/full_isolate_compressed
> [Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
> Date |filename
> ++---+---+-+---+
> [InRef][ ] [-L-][-][ ] -rw-r--r-- thomas thomas 0 Sun Aug
> 27 18:45:31 2023file1
>
>
> # dar can't read the on-fly-catalogue and reports the error above:
> $ dar --list ${BASE}/cat/full_onthefly
>
Bug#1050663: dar option --on‐fly‐isolate creates catalogue with broken slice_layout
Package: dar
Version: 2.7.10-2+b2
Severity: normal
X-Debbugs-Cc: debianbts-20230827181...@racbu.de
Dear Maintainer,
I try to do a differential backup with an earlier on-fly-isolated
catalogue. It seems dar can't read the isolated catalogues anymore.
, [ Error ]
| Final memory cleanup...
| exception type = [BUG] --
| [source]
| File ../../../src/libdar/slice_layout.cpp line 48 : it seems to be a
bug here
| stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar4EbugC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi+0x140)
[0x7f4ff3f79d50]
| stack dump : /lib/x86_64-linux-gnu/libdar64.so.6000(+0xf83bb)
[0x7f4ff3ef83bb]
| stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar14header_version4readERNS_12generic_fileERNS_16user_interactionEb+0x264)
[0x7f4ff3fbc944]
| stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar24macro_tools_open_archiveERKSt10shared_ptrINS_16user_interactionEERKS0_INS_8entrepotEERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_8limitintImEESG_NS_11crypto_algoERKNS_11secu_stringEjRNS_4pileERNS_14header_versionESG_SG_SG_RSI_RNS9_4listINS_8signatorESaISV_EEERNS_12slice_layoutEmmb+0x3aa)
[0x7f4ff3fe526a]
| stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar7archive9i_archiveC1ERKSt10shared_ptrINS_16user_interactionEERKNS_4pathERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESH_RKNS_20archive_options_readE+0x44f)
[0x7f4ff3fc1d6f]
| stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar7archiveC2ERKSt10shared_ptrINS_16user_interactionEERKNS_4pathERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESG_RKNS_20archive_options_readE+0xd6)
[0x7f4ff3f248c6]
| stack dump : dar(+0x49306) [0x560a2c6b1306]
| stack dump : dar(+0x5158c) [0x560a2c6b958c]
| stack dump : dar(+0x247d1) [0x560a2c68c7d1]
| stack dump : /lib/x86_64-linux-gnu/libc.so.6(+0x276ca)
[0x7f4ff38456ca]
| stack dump : /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)
[0x7f4ff3845785]
| stack dump : dar(+0x24821) [0x560a2c68c821]
| [most outside call]
| ---
`
Reproduce the error
This is a minimal example to reproduce the error.
### Create som structure to backup
$ export BASE=/tmp/dar_debug
$ mkdir -p ${BASE}/bak ${BASE}/cat ${BASE}/files
$ touch ${BASE}/files/file1
### Create the backup and the on-fly-isolation
$ dar --create ${BASE}/bak/full --fs-root ${BASE}/files/ --aux
${BASE}/cat/full_onthefly
### Do a manual isolation after the backup is done
$ dar --ref ${BASE}/bak/full --isolate ${BASE}/cat/full_isolate
### As described in the man page on-fly-isolation is always compressed so
### lets do this manually, too
$ dar --ref ${BASE}/bak/full --compression=bzip2 --isolate
${BASE}/cat/full_isolate_compressed
### checking the results
# the backup itself and the after backup-catalogues are working:
$ dar --list ${BASE}/bak/full
[Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
Date |filename
++---+---+-+---+
[Saved][ ] [-L-][ ][ ] -rw-r--r-- thomas thomas 0 Sun Aug
27 18:45:31 2023file1
$ dar --list ${BASE}/cat/full_isolate
[Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
Date |filename
++---+---+-+---+
[InRef][ ] [-L-][-][ ] -rw-r--r-- thomas thomas 0 Sun Aug
27 18:45:31 2023file1
$ dar --list ${BASE}/cat/full_isolate_compressed
[Data ][D][ EA ][FSA][Compr][S]| Permission | User | Group | Size|
Date |filename
++---+---+-+---+
[InRef][ ] [-L-][-][ ] -rw-r--r-- thomas thomas 0 Sun Aug
27 18:45:31 2023file1
# dar can't read the on-fly-catalogue and reports the error above:
$ dar --list ${BASE}/cat/full_onthefly
Final memory cleanup...
exception type = [BUG] --
[source]
File ../../../src/libdar/slice_layout.cpp line 48 : it seems to be a
bug here
stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar4EbugC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi+0x140)
[0x7f4ff3f79d50]
stack dump : /lib/x86_64-linux-gnu/libdar64.so.6000(+0xf83bb)
[0x7f4ff3ef83bb]
stack dump :
/lib/x86_64-linux-gnu/libdar64.so.6000(_ZN6libdar14header_version4readERNS_12generic_fileERNS_16user_interactionEb+0x264)
[0x7f4ff3fbc944]
stack dump :
