Bug#942756: cme: please provide more description of the format of d/fix.scanned.copyright
On Wednesday, 23 October 2019 04:40:11 CEST Ross Vandegrift wrote: > Makes sense - though am I correct to think that fix.scanned.copyright is for > fixing extractions that include html entities, weird chars from upstream, > etc? Yes. In this case, overriding the bad entry with fill.copyright.blanks is less practical because future upstream changes are masked. > > On the other hand, the "load string syntax" section was intended to > > provide > > instructions usable for non-perl programmers. Was that section > > overwhelming > > as well ? > > It was originally, but after your message, it's much more clear. There's > not nearly as much to learn as it originally looked like. :-) > > In any case, I may need to split this man page... What do you think ? > > No, I don't think that's necessary anymore. Maybe a few more words of > description, and some pointers to the Config::Model::Loader section. > Possible patches are attached that use the explanations you sent me. Although 'cme update' is intended to be a generic sub-command of cme. update can be applied to any application supported by cme, like systemd, ssh... On the other hand, dpkg is the only application where update provides more functionality than 'cme migrate'.. :-/ Anyway, I'll modify the wording to be less specific to dpkg. And may be keep dpkg as a real world example. Many thanks for the discussion and the patches :-) All the best
Bug#942756: cme: please provide more description of the format of d/fix.scanned.copyright
Control: tags -1 patch On Mon, Oct 21, 2019 at 05:17:59PM +0200, Dominique Dumont wrote: > The data contained in debian/copyright is mapped to a tree structure inside > cme. You can have a view of this structure by running 'cme dump dpkg- > copyright' or get a graphical view with 'cme edit dpkg-copyright' > > The instructions specified in fix.scanned.copyright are steps to walk in that > tree and alter the values when specified. Aha, now this is coming together. > > Things that I'd find helpful: > > - what is the general form of these lines? If that's too complex, examples > > covering more use-cases would be helpful. > > In fact, I've found that tweaking debian/fill.copyright.blanks.yml often > provide better results. All in all, I've found very few cases where tweaking > d/fix.scanned.copyright provided better results than tweaking debian/ > fill.copyright.blanks.yml Makes sense - though am I correct to think that fix.scanned.copyright is for fixing extractions that include html entities, weird chars from upstream, etc? > I understand that Config::Model::Loader synopsis is for only for Perl > programmer. > > On the other hand, the "load string syntax" section was intended to provide > instructions usable for non-perl programmers. Was that section overwhelming > as well ? It was originally, but after your message, it's much more clear. There's not nearly as much to learn as it originally looked like. > In any case, I may need to split this man page... What do you think ? No, I don't think that's necessary anymore. Maybe a few more words of description, and some pointers to the Config::Model::Loader section. Possible patches are attached that use the explanations you sent me. Ross diff --git a/lib/App/Cme/Command/update.pm b/lib/App/Cme/Command/update.pm index b4a36ea..997cc3d 100644 --- a/lib/App/Cme/Command/update.pm +++ b/lib/App/Cme/Command/update.pm @@ -103,6 +103,13 @@ Update a configuration file. The update is done scanning external resource. For the update of dpkg-copyright is done by scanning the headers of source files. (Actually, only dpkg-copyright model currently supports updates) +The data contained in debian/copyright is mapped to a tree structure inside cme. You can have +a view of this structure by running 'cme dump dpkg-copyright' or get a graphical view with +'cme edit dpkg-copyright'. + +Sometimes the output will need adjusting. See the section "Tweak results" in +Config::Model::Dpkg::Copyright for ways to control cme's output. + Example: cme update dpkg-copyright diff --git a/lib/Config/Model/Dpkg/Copyright.pm b/lib/Config/Model/Dpkg/Copyright.pm index 961cc360..272734c0 100644 --- a/lib/Config/Model/Dpkg/Copyright.pm +++ b/lib/Config/Model/Dpkg/Copyright.pm @@ -423,8 +423,14 @@ based on comments, the result is sometimes lackluster. Your may specify instruction to alter or set specific copyright entries in C file (or C<< debian/.fix.scanned.copyright >>). -Each line of this file will be handled -by L to modify copyright information. + +cme stores the copyright information in a tree. Entries in +fix.scanned.copyright provide instructions for traversing the cme tree +and modifying entries. You can have a view of debian/copyright file +translated in this syntax by running 'cme dump --format cml +dpkg-copyright'. Each line of this file will be handled by +L to modify copyright information; the full +syntax is documented in the "load string syntax" section. =head2 Example @@ -464,6 +470,12 @@ Here's another more complex example: # and modify the copyright entry with a Perl substitution ! Files:~/^3rdparty/ Copyright=~s/@/(at)/ +Sometimes, you might want to find an entry that spans multiple lines. +You can do this by double quoting the whole value: + + ! Files:"uulib/crc32.h + uulib/uustring.h" Copyright="2019 John Doe" + =head1 Under the hood This section explains how cme merges the information from the existing
Bug#942756: cme: please provide more description of the format of d/fix.scanned.copyright
Hello
On Monday, 21 October 2019 04:35:47 CEST you wrote:
> I started to use cme to maintain d/copyright today. It'll be a big help, so
> thanks!
Thanks for the feedback :-)
> But I ran into a few snags.
Which is to be expected with complex software :-p
> I'm not sure how d/fix.scanned.copyright is supposed to work. Some examples
> in Config::Model::Dpkg::Copyright(3pm) are clear but trivial.
Yes, I failed to come up with more complex but realistic examples. I hope
you'll help me there.
> They allude
> to some complex capabilities without any explanation ("don't forget '!' to
> go back to tree root": but what tree is there? This is a text file with a
> few lines.).
The data contained in debian/copyright is mapped to a tree structure inside
cme. You can have a view of this structure by running 'cme dump dpkg-
copyright' or get a graphical view with 'cme edit dpkg-copyright'
The instructions specified in fix.scanned.copyright are steps to walk in that
tree and alter the values when specified.
> I think I need some more help from the point of view of
> someone trying to automate management of d/copyright.
Sure. Note that this automation cannot be bulletproof because the copyright
extraction process relies on unstructured data so it's somewhat fuzzy.
> Things that I'd find helpful:
> - what is the general form of these lines? If that's too complex, examples
> covering more use-cases would be helpful.
In fact, I've found that tweaking debian/fill.copyright.blanks.yml often
provide better results. All in all, I've found very few cases where tweaking
d/fix.scanned.copyright provided better results than tweaking debian/
fill.copyright.blanks.yml
Note that debian/fill.copyright.blanks.yml can also be edited with 'cme edit
dpkg'
> - how should I handle stanzas with multiple lines in Files, Copyright, etc.?
Right. That's a realistic example I did not think of.. :-}
Please put the multiline stanza between double quotes. For instance:
Files:"uulib/crc32.h
uulib/uustring.h"
You can have a view of debian/copyright file translated in this syntax by
running 'cme dump --format cml dpkg-copyright'
> - there appears to be a few ways to modify d/copyright stanzas. What are
> the possibilities?
Mostly assignment and using regular expression on value. These are detailed on
Config::Model::Loader man page. I'd rather fix this man page than repeat the
instructions.
> I found Config::Model::Loader(3pm) but it was overwhelming. Documenting the
> data structures doesn't help much since I'm not a perl programmer and I'd
> rather avoid learning about the generic serialization described there.
I understand that Config::Model::Loader synopsis is for only for Perl
programmer.
On the other hand, the "load string syntax" section was intended to provide
instructions usable for non-perl programmers. Was that section overwhelming as
well ?
In any case, I may need to split this man page... What do you think ?
All the best
Dod
Bug#942756: cme: please provide more description of the format of d/fix.scanned.copyright
Package: cme
Version: 1.030-1
Severity: wishlist
Hello,
I started to use cme to maintain d/copyright today. It'll be a big help, so
thanks! But I ran into a few snags.
I'm not sure how d/fix.scanned.copyright is supposed to work. Some examples in
Config::Model::Dpkg::Copyright(3pm) are clear but trivial. They allude to some
complex capabilities without any explanation ("don't forget '!' to go back to
tree root": but what tree is there? This is a text file with a few lines.) I
think I need some more help from the point of view of someone trying to
automate management of d/copyright.
Things that I'd find helpful:
- what is the general form of these lines? If that's too complex, examples
covering more use-cases would be helpful.
- how should I handle stanzas with multiple lines in Files, Copyright, etc.?
- there appears to be a few ways to modify d/copyright stanzas. What are the
possibilities?
I found Config::Model::Loader(3pm) but it was overwhelming. Documenting the
data structures doesn't help much since I'm not a perl programmer and I'd
rather avoid learning about the generic serialization described there.
Thanks,
Ross
-- System Information:
Debian Release: bullseye/sid
APT prefers testing
APT policy: (500, 'testing'), (40, 'unstable'), (30, 'experimental'), (1,
'experimental-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.2.0-3-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages cme depends on:
ii libapp-cmd-perl 0.331-1
ii libconfig-model-perl 2.136-1
ii libjson-perl 4.02000-1
ii liblog-log4perl-perl 1.49-1
ii libpath-tiny-perl 0.108-1
ii libpod-pom-perl 2.01-3
ii libyaml-perl 1.29-1
ii perl 5.28.1-6
Versions of packages cme recommends:
ii libconfig-model-approx-perl 1.011-1
ii libconfig-model-dpkg-perl 2.126
ii libconfig-model-lcdproc-perl 2.052-2
ii libconfig-model-openssh-perl 2.8.0.1-1
ii libconfig-model-systemd-perl 0.240.1-1
ii libconfig-model-tkui-perl 1.370-1
Versions of packages cme suggests:
pn libconfig-model-cursesui-perl
pn libconfig-model-itself-perl
-- no debconf information
