Re: [RFE] calm needs to know about ZStandard compressed archives
On 31/01/2021 17:13, Achim Gratz wrote: Jon Turney writes: This is also deployed on sourceware. At least one of xtarfile or zstandard Pythono modules seems to be missing, though. I tried to upload binutils and got this: ERROR: exception NotImplementedError while reading /sourceware/cygwin-staging/home/Achim Gratz/x86/release/binutils/binutils-2.35.2-1-src.tar.zst Yeah, seems like I screwed up installing the python modules. Should be fixed now. Thanks for reporting.
Re: [RFE] calm needs to know about ZStandard compressed archives
On 24/01/2021 09:35, Achim Gratz wrote: Jon Turney writes: I uploaded a calm-20210110-1 package, which adds support for .zst package archives. The package seems to be missing a dependency on python38-xtarfile, though. Thanks for reporting. Should be fixed.
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: > This is also deployed on sourceware. At least one of xtarfile or zstandard Pythono modules seems to be missing, though. I tried to upload binutils and got this: ERROR: exception NotImplementedError while reading /sourceware/cygwin-staging/home/Achim Gratz/x86/release/binutils/binutils-2.35.2-1-src.tar.zst Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: > I uploaded a calm-20210110-1 package, which adds support for .zst > package archives. The package seems to be missing a dependency on python38-xtarfile, though. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: > I uploaded a calm-20210110-1 package, which adds support for .zst > package archives. > > This is also deployed on sourceware. Awesome, thank you! Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [RFE] calm needs to know about ZStandard compressed archives
On 03/01/2021 21:55, Marco Atzeri via Cygwin-apps wrote: On 03.01.2021 16:19, Jon Turney wrote: I can arrange to install it on sourceware, but it would be nice if python-xtarfile was packaged so it can be made a prerequisite of the calm package. I just uploaded test versions of python3X-xtarfile and python3X-zstandard that seems are needed for that compression format.. Thanks. I uploaded a calm-20210110-1 package, which adds support for .zst package archives. This is also deployed on sourceware.
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: >> I think the right thing to do is to stop having actual packages for >> OBSOLETES and just emit the obsoltes: hint instead. > > Yeah, this change to cygport should be made and we can turn off > support for setup versions which don't understand the obsoletes: hint. The obsolete hints are already generated for some time, so at least the newer packages should have them. The older packages would need to retroactively get them (or keep their equally old obsoleteion packages around still). So I think this is what's left to do in cygport (untested, on top of my to-upstream branch): --8<---cut here---start->8--- >From 0491ea2934d83c656ba5dacf7f02c31f02b46a38 Mon Sep 17 00:00:00 2001 Subject: [PATCH] lib/pkg_pkg.cygpart: stop generating packages for obsoletions --- lib/pkg_pkg.cygpart | 46 - 1 file changed, 46 deletions(-) diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart index 2b2f8bc..80003e9 100644 --- a/lib/pkg_pkg.cygpart +++ b/lib/pkg_pkg.cygpart @@ -828,32 +828,6 @@ _EOF warning "${pkg_hint[${n}]%.hint}.hint is missing"; fi - for obspkg in ${!pkg_obsoletes_var} - do - if [ ${obspkg} = ${PN} ] - then - obssubdir= ; - else - obssubdir=${obspkg}; - fi - - mkdir -p ${distdir}/${PN}/${obssubdir}; - ${CYGPORT_TAR_CMD-tar -J} -cf ${distdir}/${PN}/${obssubdir}/${obspkg}-${PVR}${CYGPORT_TAR_EXT-.tar.xz} --files-from /dev/null - - __step "${pkg_name[${n}]} OBSOLETES: ${obspkg}" - - cat > ${distdir}/${PN}/${obssubdir}/${obspkg}-${PVR}.hint <<-_EOF -category: _obsolete -requires: ${pkg_name[${n}]} -sdesc: "Obsoleted by ${pkg_name[${n}]}" -ldesc: "The ${obspkg} package is obsolete. Selecting this package for -installation will cause the ${pkg_name[${n}]} package, which replaces this -one, to be installed instead." -${obssubdir:+external-source: ${PN}} -${pkg_tag} -_EOF - done - n+=1; done @@ -887,26 +861,6 @@ _EOF fi fi - for obspkg in ${!dbg_obsoletes_var} - do - mkdir -p ${distdir}/${PN}/${obspkg}; - ${CYGPORT_TAR_CMD-tar -J} -cf ${distdir}/${PN}/${obspkg}/${obspkg}-${PVR}${CYGPORT_TAR_EXT-.tar.xz} --files-from /dev/null - - __step "${PN}-debuginfo OBSOLETES: ${obspkg}" - - cat > ${distdir}/${PN}/${obspkg}/${obspkg}-${PVR}.hint <<-_EOF -category: _obsolete -requires: ${PN}-debuginfo -sdesc: "Obsoleted by ${PN}-debuginfo" -ldesc: "The ${obspkg} package is obsolete. Selecting this package for -installation will cause the ${PN}-debuginfo package, which replaces this -one, to be installed instead." -external-source: ${PN} -${pkg_tag} -_EOF - done - fi - # source package hint if [ ! -f ${distdir}/${PN}/${PN}-${PVR}-src.hint ] then -- 2.29.2 --8<---cut here---end--->8--- Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [RFE] calm needs to know about ZStandard compressed archives
On 03/01/2021 15:57, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: We will also need to modify cygport to stop automatically add OBSOLETES as most of this packages never existed I think the right thing to do is to stop having actual packages for OBSOLETES and just emit the obsoltes: hint instead. Yeah, this change to cygport should be made and we can turn off support for setup versions which don't understand the obsoletes: hint. an overall cleaning of obsoletes package may be useful in the near future. I need to work on it.
Re: [RFE] calm needs to know about ZStandard compressed archives
On 03.01.2021 16:19, Jon Turney wrote: I can arrange to install it on sourceware, but it would be nice if python-xtarfile was packaged so it can be made a prerequisite of the calm package. I just uploaded test versions of python3X-xtarfile and python3X-zstandard that seems are needed for that compression format.. let me know if find any issue. Regards Marco
Re: [RFE] calm needs to know about ZStandard compressed archives
On 03.01.2021 16:57, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: We will also need to modify cygport to stop automatically add OBSOLETES as most of this packages never existed I think the right thing to do is to stop having actual packages for OBSOLETES and just emit the obsoltes: hint instead. an overall cleaning of obsoletes package may be useful in the near future. I need to work on it. The mailing list is full of folks that seem to update their installations only after several years, so while I'm all for removing the empty packages, the obsolete hints need to stay. I was referring only to the "empty packages" artificially created of course. Regards, Achim.
Re: [RFE] calm needs to know about ZStandard compressed archives
Marco Atzeri via Cygwin-apps writes: > We will also need to modify cygport to stop automatically add > OBSOLETES as most of this packages never existed I think the right thing to do is to stop having actual packages for OBSOLETES and just emit the obsoltes: hint instead. > an overall cleaning of obsoletes package may be useful in the > near future. I need to work on it. The mailing list is full of folks that seem to update their installations only after several years, so while I'm all for removing the empty packages, the obsolete hints need to stay. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [RFE] calm needs to know about ZStandard compressed archives
On 03.01.2021 16:19, Jon Turney wrote: On 31/12/2020 17:08, Jon Turney wrote: On 31/12/2020 16:01, Marco Atzeri via Cygwin-apps wrote: On 31.12.2020 16:52, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: On 31.12.2020 16:47, Achim Gratz wrote: Jon Turney writes: This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression. I can arrange to install it on sourceware, but it would be nice if python-xtarfile was packaged so it can be made a prerequisite of the calm package. I will do it, but I will appreciate if package maintainers can validate the python/pip test release I recently uploaded. As I am rebuiling/adding python packages using latest test and going backward is not so nice. We will also need to modify cygport to stop automatically add OBSOLETES as most of this packages never existed eg python3-pytest >>> python36-pytest requires: python36 python36-attrs python36-iniconfig python36-pluggy python36-py python36-packaging python36-toml >>> python36-pytest OBSOLETES: python3-pytest >>> python37-pytest requires: python37 python37-attrs python37-iniconfig python37-pluggy python37-py python37-packaging python37-toml >>> python38-pytest requires: python38 python38-attrs python38-iniconfig python38-pluggy python38-py python38-packaging python38-toml an overall cleaning of obsoletes package may be useful in the near future. I need to work on it. regards Marco
Re: [RFE] calm needs to know about ZStandard compressed archives
On 31/12/2020 17:08, Jon Turney wrote: On 31/12/2020 16:01, Marco Atzeri via Cygwin-apps wrote: On 31.12.2020 16:52, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: On 31.12.2020 16:47, Achim Gratz wrote: Jon Turney writes: This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression. Maybe this would help until upstream gets their act together… https://pypi.org/project/xtarfile/ do you need it as python package ? The tricky part is if it's available on sourceware, I'd guess. locate says no, for the time being Thanks, but since this is only done in one spot, I think I can just arrange to pipe ztd archives through 'unzstd' before validating. Actually, on second look, we need to look inside the archive in a couple of places (to check if it's empty, and to generate the package file listings), so using xtarfile seems like the right approach. I can arrange to install it on sourceware, but it would be nice if python-xtarfile was packaged so it can be made a prerequisite of the calm package.
Re: [RFE] calm needs to know about ZStandard compressed archives
Brian Inglis writes: > To which part does "No" apply: you specified -J which is --xz; you > could instead specify -a, --auto-compress to use the archive suffix. Here's the actual content of my test installation that I've used for the Perl repository: --8<---cut here---start->8--- CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 -a" CYGPORT_TAR_EXT=".tar.zst" --8<---cut here---end--->8--- Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [RFE] calm needs to know about ZStandard compressed archives
On 2020-12-31 12:18, Achim Gratz wrote: Brian Inglis writes: CYGPORT_TAR_EXT=".tar.zst" ??? Yes, typo. CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 --zstd" ??? No. CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 -J" To which part does "No" apply: you specified -J which is --xz; you could instead specify -a, --auto-compress to use the archive suffix. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]
Re: [RFE] calm needs to know about ZStandard compressed archives
Brian Inglis writes: > CYGPORT_TAR_EXT=".tar.zst" ??? Yes, typo. > CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner > nobody:65534 --zstd" ??? No. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [RFE] calm needs to know about ZStandard compressed archives
On 2020-12-31 11:33, Achim Gratz wrote: Jon Turney writes: Can you publish your cygport changes, when you are happy with them, so I can make some packages to test with? https://repo.or.cz/cygport/rpm-style.git/shortlog/refs/heads/to-upstream Then you need to put this into .cygport.conf to actually produce .tar.zst archives: CYGPORT_TAR_EXT=".tar.xz" CYGPORT_TAR_EXT=".tar.zst" ??? The default compression level can be changed as follows: CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 -J" CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 --zstd" ??? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: > Can you publish your cygport changes, when you are happy with them, so > I can make some packages to test with? https://repo.or.cz/cygport/rpm-style.git/shortlog/refs/heads/to-upstream Then you need to put this into .cygport.conf to actually produce .tar.zst archives: CYGPORT_TAR_EXT=".tar.xz" The default compression level can be changed as follows: CYGPORT_TAR_CMD="env ZSTD_CLEVEL=19 tar --group nobody:65534 --owner nobody:65534 -J" Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables
Re: [RFE] calm needs to know about ZStandard compressed archives
On 31/12/2020 16:01, Marco Atzeri via Cygwin-apps wrote: On 31.12.2020 16:52, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: On 31.12.2020 16:47, Achim Gratz wrote: Jon Turney writes: This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression. Maybe this would help until upstream gets their act together… https://pypi.org/project/xtarfile/ do you need it as python package ? The tricky part is if it's available on sourceware, I'd guess. locate says no, for the time being Thanks, but since this is only done in one spot, I think I can just arrange to pipe ztd archives through 'unzstd' before validating. Achim, Can you publish your cygport changes, when you are happy with them, so I can make some packages to test with?
Re: [RFE] calm needs to know about ZStandard compressed archives
On 31.12.2020 16:52, Achim Gratz wrote: Marco Atzeri via Cygwin-apps writes: On 31.12.2020 16:47, Achim Gratz wrote: Jon Turney writes: This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression. Maybe this would help until upstream gets their act together… https://pypi.org/project/xtarfile/ do you need it as python package ? The tricky part is if it's available on sourceware, I'd guess. locate says no, for the time being
Re: [RFE] calm needs to know about ZStandard compressed archives
Marco Atzeri via Cygwin-apps writes: > On 31.12.2020 16:47, Achim Gratz wrote: >> Jon Turney writes: >>> This isn't totally straightforward as calm uses the python tar module >>> to validate the upload archives, but that doesn't (yet) support zstd >>> compression. >> Maybe this would help until upstream gets their act together… >> https://pypi.org/project/xtarfile/ > > do you need it as python package ? The tricky part is if it's available on sourceware, I'd guess. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [RFE] calm needs to know about ZStandard compressed archives
On 31.12.2020 16:47, Achim Gratz wrote: Jon Turney writes: This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression. Maybe this would help until upstream gets their act together… https://pypi.org/project/xtarfile/ do you need it as python package ? Regards, Achim.
Re: [RFE] calm needs to know about ZStandard compressed archives
Jon Turney writes: > This isn't totally straightforward as calm uses the python tar module > to validate the upload archives, but that doesn't (yet) support zstd > compression. Maybe this would help until upstream gets their act together… https://pypi.org/project/xtarfile/ Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [RFE] calm needs to know about ZStandard compressed archives
On 25/12/2020 19:59, Achim Gratz wrote: While waiting for the Perl bootstrap I've been experimenting with having cygport produce .tar.zst packages. Calm (mksetupini) does not recognize these at the moment, but I've been able to monkey-patch it by adding '|zst' in the obvious places. Can this be done officially, please? I can't help noting that these really shouldn't be literal strings so that you only need to change them in a single place if necessary; so perhaps common_constants.py would be a good place for that definition. Yeah, this list should be done in a less ad-hoc way. This isn't totally straightforward as calm uses the python tar module to validate the upload archives, but that doesn't (yet) support zstd compression.
[RFE] calm needs to know about ZStandard compressed archives
While waiting for the Perl bootstrap I've been experimenting with having cygport produce .tar.zst packages. Calm (mksetupini) does not recognize these at the moment, but I've been able to monkey-patch it by adding '|zst' in the obvious places. Can this be done officially, please? I can't help noting that these really shouldn't be literal strings so that you only need to change them in a single place if necessary; so perhaps common_constants.py would be a good place for that definition. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada