Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Dominique Dumont]

On Sunday, 22 October 2017 21:47:12 CEST Andreas Tille wrote:
> Could you please explain what you mean by "main section"?  For me
> 
>   Files: *
> 
> would qualify as "main section" but you seem to have a different
> understanding of this term.

ok. Let's use the same terminology as debian/copyright. I meant the section 
made of one or more  "Stand-alone License paragraph" [1] . This one was 
missing from the file, the CeCILL license was not defined, hence the file was 
considered as invalid by cme.

> >  May be I should just display a
> > "changed" message when summarising the changes applied to a text
> > parameter.
> 
> I think that's a more helpful output.

ok. Will do.

> I don't say that the GUI is bad - I just prefer a command line tool for
> this job.

ok. fair enough. 


> > $ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) !
> > Files:"*" License short_name=CeCILL ! Files:"debian/*" License
> > short_name=CeCILL'
> 
> Ahhh, this helps. :-)
> I just would love a way shorter command line since I somehow will never
> remember this one. :-P

You should be able to use paste-license script in you add first the License 
text as a standalone paragraph (using paste-license), *then* add the Files:* 
paragraphs. 

> Just to let me understand better:  I understood this thread that way
> that creating the license text in a proper form is a goal of this
> specific cme option.  In how far is the problem above a corner case.

It's a corner case because you started from an copyright file that is considred 
invalid by cme because it lacks the standalone license paragraph that defines 
CeCILL license text). 

This challenges the way error tolerance is done in cme, which is not much 
tested yet.

All the best

Dod

[1] 
https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#stand-alone-license-paragraph
-- 
 https://github.com/dod38fr/   -o- http://search.cpan.org/~ddumont/
http://ddumont.wordpress.com/  -o-   irc: dod at irc.debian.org

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Andreas Tille]

Hi Dominique,

On Sun, Oct 22, 2017 at 07:39:10PM +0200, Dominique Dumont wrote:
> > 
> > Sorry, I do not understand.  I the string CeCILL (with capital I) is in the
> > main license section.  Could you please be more verbose how d/copyright
> > needs to look like to make cme add the license text?
> 
> Uh ? From a fresh git clone of beads, cat debian/copyright shows:
> 
> +
> Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
> Upstream-Name: BEADS
> Upstream-Contact: Olivier Langella 
> Source: http://pappso.inra.fr/bioinfo/beads/index.php
> Files-Excluded: */CImg.h
> 
> Files: *
> Copyright: 2008-2010 Michel Zivy , Olivier Langella 
> 
> License: CeCILL
> 
> Files: debian/*
> Copyright: Andreas Tille 
> License: CeCILL
> 
> 
> There's no main CeCILL section.

Could you please explain what you mean by "main section"?  For me

  Files: *

would qualify as "main section" but you seem to have a different
understanding of this term.
 
> And cme complains:
> $ cme check dpkg-copyright 
> cme: using Dpkg::Copyright model
> loading data
> Configuration item 'Files:"*" License short_name' has a wrong value:
> license 'CeCILL' is not declared in main License section. Expected
> 
> Admitedly, the error message is lackluster when main License section is 
> empty. 
> This will be fixed.

:-)
 
> It's just that cme uses a diff to show the delta between old and new value. 
> It's not supposed to be used as a patch. May be I should just display a 
> "changed" message when summarising the changes applied to a text parameter.

I think that's a more helpful output.
 
> > > cme should write debian/copyright provided no error is left.
> > 
> > That's what I'd prefer.
> 
> So we have a common goal :)

:-)
 
> > I admit I do not really like that GUI.
> 
> I'm open to idea on how to improve the GUI while keeping it generic for the 
> other models supported by cme (e.g ssh systemd itself ...)

I don't say that the GUI is bad - I just prefer a command line tool for
this job.
 
> > $ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) !
> > Files:"*" License short_name=CeCILL Files:"debian/*" License
> > short_name=CeCILL' cme: using Dpkg::Copyright model
> > Warning: Files:"*" License short_name skipping value CeCILL because of the
> > following errors: license 'CeCILL' is not declared in main License section.
> [snip]
> > Hmmm, sorry, no change to d/copyright.
> 
> Sorry, my bad. I dropped a '!' during cut'n'paste. 
> 
> Here's the right command tested on beads repo:
> 
> $ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) ! 
> Files:"*" License short_name=CeCILL ! Files:"debian/*" License 
> short_name=CeCILL'

Ahhh, this helps. :-)
I just would love a way shorter command line since I somehow will never
remember this one. :-P

> $ git diff --stat
>  debian/copyright | 508 ++
> +-
>  1 file changed, 507 insertions(+), 1 deletion(-)
 
> > No, sorry, I seem I totally fail to understand.  Why exactly can't cme
> > simply copy the text that is specified as CeCILL into my copyright?
> 
> It did so, but refused to write the output due to errors.
> 
> > Sorry for my probably very naive questions but obviously I do not
> > understand the philosphy behind.
> 
> No problem. cme tries to address a very complicated problem and can be 
> confusing when dealing with corner cases. 

Just to let me understand better:  I understood this thread that way
that creating the license text in a proper form is a goal of this
specific cme option.  In how far is the problem above a corner case.
 
> Thanks for your patience and questions. This is invaluable to improve the 
> usability of cme. :-)

You are welcome - the thanks goes to you who does all the coding work

Andreas.
 
-- 
http://fam-tille.de

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Dominique Dumont]

On Sunday, 22 October 2017 12:10:29 CEST Andreas Tille wrote:
> > without the matching section in Licenses (the one you trying to add). cme
> > emits a warning when reading a copyright file with this error. This value
> > is ignored because of this error.
> 
> Sorry, I do not understand.  I the string CeCILL (with capital I) is in the
> main license section.  Could you please be more verbose how d/copyright
> needs to look like to make cme add the license text?

Uh ? From a fresh git clone of beads, cat debian/copyright shows:

+
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: BEADS
Upstream-Contact: Olivier Langella 
Source: http://pappso.inra.fr/bioinfo/beads/index.php
Files-Excluded: */CImg.h

Files: *
Copyright: 2008-2010 Michel Zivy , Olivier Langella 

License: CeCILL

Files: debian/*
Copyright: Andreas Tille 
License: CeCILL


There's no main CeCILL section.

And cme complains:
$ cme check dpkg-copyright 
cme: using Dpkg::Copyright model
loading data
Configuration item 'Files:"*" License short_name' has a wrong value:
license 'CeCILL' is not declared in main License section. Expected

Admitedly, the error message is lackluster when main License section is empty. 
This will be fixed.

> > > Warning: Files:"debian/*" License short_name skipping value CeCILL
> > > because
> > > of the following errors: license 'CeCILL' is not declared in main
> > > License
> > > section. Expected
> > 
> > Likewise.
> 
> Likewise I fail to understand. ;-)

Duh ;-)

> Yes please.  I'd be really happy if you could push a d/copyright that is
> correctly formed to let cme work.

I hope that won't be necessary. See below.

> I redirected simply for this mail here ...

Fair enough.

> > err. it never occurred to me that someone could feed cme output to patch
> > 
> Yes, I had this clue only since the first column '+' is a feature
> frequently used in patch.  Just a wild guess of mine.

It's just that cme uses a diff to show the delta between old and new value. 
It's not supposed to be used as a patch. May be I should just display a 
"changed" message when summarising the changes applied to a text parameter.

> > cme should write debian/copyright provided no error is left.
> 
> That's what I'd prefer.

So we have a common goal :)

> I admit I do not really like that GUI.

I'm open to idea on how to improve the GUI while keeping it generic for the 
other models supported by cme (e.g ssh systemd itself ...)

> $ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) !
> Files:"*" License short_name=CeCILL Files:"debian/*" License
> short_name=CeCILL' cme: using Dpkg::Copyright model
> Warning: Files:"*" License short_name skipping value CeCILL because of the
> following errors: license 'CeCILL' is not declared in main License section.
[snip]
> Hmmm, sorry, no change to d/copyright.

Sorry, my bad. I dropped a '!' during cut'n'paste. 

Here's the right command tested on beads repo:

$ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) ! 
Files:"*" License short_name=CeCILL ! Files:"debian/*" License 
short_name=CeCILL'
[ snip ]
$ git diff --stat
 debian/copyright | 508 ++
+-
 1 file changed, 507 insertions(+), 1 deletion(-)


> No, sorry, I seem I totally fail to understand.  Why exactly can't cme
> simply copy the text that is specified as CeCILL into my copyright?

It did so, but refused to write the output due to errors.

> Sorry for my probably very naive questions but obviously I do not
> understand the philosphy behind.

No problem. cme tries to address a very complicated problem and can be 
confusing when dealing with corner cases. 

Thanks for your patience and questions. This is invaluable to improve the 
usability of cme. :-)

All the best

Dod
-- 
 https://github.com/dod38fr/   -o- http://search.cpan.org/~ddumont/
http://ddumont.wordpress.com/  -o-   irc: dod at irc.debian.org

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Andreas Tille]

On Sun, Oct 22, 2017 at 11:14:04AM +0200, Dominique Dumont wrote:
> On Sunday, 22 October 2017 08:55:44 CEST Andreas Tille wrote:
> > $ cme run paste-license  --arg license=CeCILL --arg file=COPYING >
> > copyright.patch Log4perl: Seems like no initialization happened. Forgot to
> > call init()? 
> 
> That's a bug in cme that will be fixed soon.

OK.
 
> > Warning: Files:"*" License short_name skipping value CeCILL
> > because of the following errors: license 'CeCILL' is not declared in main
> > License section. Expected
> 
> Looks like you copyright file already has
> Files: *
> License: CeCiLL
> 
> without the matching section in Licenses (the one you trying to add). cme 
> emits a warning when reading a copyright file with this error. This value is 
> ignored because of this error.

Sorry, I do not understand.  I the string CeCILL (with capital I) is in the
main license section.  Could you please be more verbose how d/copyright needs
to look like to make cme add the license text?

> > Warning: Files:"debian/*" License short_name skipping value CeCILL because
> > of the following errors: license 'CeCILL' is not declared in main License
> > section. Expected
> 
> Likewise.

Likewise I fail to understand. ;-)
 
> > License CeCILL is not used in Files: section
> 
> The new added license is seen as unused because the CeCILL values were 
> ignored 
> above. (*)
> 
> > Configuration item 'Files:"*" License short_name' has a wrong value:
> > Undefined mandatory value.
> 
> This is an error shown while writing the file. cme does not accept to write 
> back a copyright file containing errors. The values are missing because, err, 
> they were ignored above because, err, the main license was missing. 
> 
> I guess that error handling in cme can be improved ...

Yes please.  I'd be really happy if you could push a d/copyright that is
correctly formed to let cme work.
 
> > I admit I do not really understand all the output to stderr. 
> 
> I hope I gave some clue above ...

Not really - but it might be probably me. ;-)
 
> > The output
> > to stdout 
> 
> The fact that stdout is redirect makes the errors above harder to understand.

I redirected simply for this mail here ...
 
> > (in my example redirected to copyright.patch) is
> 
> err. it never occurred to me that someone could feed cme output to patch 

Yes, I had this clue only since the first column '+' is a feature
frequently used in patch.  Just a wild guess of mine.
 
> > So this does not really help since its neither a valid patch for
> > d/copyright nor can I add this content there without editing.  It just
> > added a '+' to the original license text which is not really helpful.
> > May be I'm continuously missing the point but shouldn't it add rather
> > a ' ' instead of a '+' and replace empty lines by ' .'?
> 
> cme should write debian/copyright provided no error is left.

That's what I'd prefer.
 
> Following Perl's TIMTOWTDI tradition, I suggest to fix this problem by either:
> 
> - use -force option with cme and add back the License entries after cme has 
> saved the file
> - use the GUI (cme edit dpkg-copyright) and cut'n'paste CeCILL license text 
> in 
> the License section (see [1] for details)

I admit I do not really like that GUI.

> - tweak your file so that cme check returns no warning before running paste-
> license
> - fix everything at once with: cme modify dpkg-copyright -force 
> 'License:CeCILL 
> text=.file(COPYING) ! Files:"*" License short_name=CeCILL Files:"debian/*" 
> License short_name=CeCILL'

$ cme modify dpkg-copyright -force 'License:CeCILL text=.file(COPYING) ! 
Files:"*" License short_name=CeCILL Files:"debian/*" License short_name=CeCILL'
cme: using Dpkg::Copyright model
Warning: Files:"*" License short_name skipping value CeCILL because of the 
following errors:
license 'CeCILL' is not declared in main License section. Expected 

Warning: Files:"debian/*" License short_name skipping value CeCILL because of 
the following errors:
license 'CeCILL' is not declared in main License section. Expected 

License CeCILL is not used in Files: section
object 'Files:"*" License' error: unknown element 'Files'. Either your file has 
an error or Dpkg::Copyright::FileLicense model is lagging behind. In the latter 
case, please submit a bug report using 'reportbug libconfig-model-dpkg-perl'. 
See cme man page for details.
Expected elements: 'full_license','short_name'


Hmmm, sorry, no change to d/copyright.
 
> Hope this helps

No, sorry, I seem I totally fail to understand.  Why exactly can't cme
simply copy the text that is specified as CeCILL into my copyright?

Sorry for my probably very naive questions but obviously I do not
understand the philosphy behind.

Kind regards

 Andreas.

> [1] 
> https://github.com/dod38fr/config-model/wiki/Managing-Debian-packages-with-cme#maintaining-debian-copyright-file
> 
> (*) I always wondered if an erroneous value found in a file should be ignored 
> or loaded. 

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Dominique Dumont]

On Sunday, 22 October 2017 08:55:44 CEST Andreas Tille wrote:
> $ cme run paste-license  --arg license=CeCILL --arg file=COPYING >
> copyright.patch Log4perl: Seems like no initialization happened. Forgot to
> call init()? 

That's a bug in cme that will be fixed soon.

> Warning: Files:"*" License short_name skipping value CeCILL
> because of the following errors: license 'CeCILL' is not declared in main
> License section. Expected

Looks like you copyright file already has
Files: *
License: CeCiLL

without the matching section in Licenses (the one you trying to add). cme 
emits a warning when reading a copyright file with this error. This value is 
ignored because of this error.

> Warning: Files:"debian/*" License short_name skipping value CeCILL because
> of the following errors: license 'CeCILL' is not declared in main License
> section. Expected

Likewise.

> License CeCILL is not used in Files: section

The new added license is seen as unused because the CeCILL values were ignored 
above. (*)

> Configuration item 'Files:"*" License short_name' has a wrong value:
> Undefined mandatory value.

This is an error shown while writing the file. cme does not accept to write 
back a copyright file containing errors. The values are missing because, err, 
they were ignored above because, err, the main license was missing. 

I guess that error handling in cme can be improved ...

> I admit I do not really understand all the output to stderr. 

I hope I gave some clue above ...

> The output
> to stdout 

The fact that stdout is redirect makes the errors above harder to understand.

> (in my example redirected to copyright.patch) is

err. it never occurred to me that someone could feed cme output to patch 

> So this does not really help since its neither a valid patch for
> d/copyright nor can I add this content there without editing.  It just
> added a '+' to the original license text which is not really helpful.
> May be I'm continuously missing the point but shouldn't it add rather
> a ' ' instead of a '+' and replace empty lines by ' .'?

cme should write debian/copyright provided no error is left.

Following Perl's TIMTOWTDI tradition, I suggest to fix this problem by either:

- use -force option with cme and add back the License entries after cme has 
saved the file
- use the GUI (cme edit dpkg-copyright) and cut'n'paste CeCILL license text in 
the License section (see [1] for details)
- tweak your file so that cme check returns no warning before running paste-
license
- fix everything at once with: cme modify dpkg-copyright -force 'License:CeCILL 
text=.file(COPYING) ! Files:"*" License short_name=CeCILL Files:"debian/*" 
License short_name=CeCILL'

Hope this helps

Dod

[1] 
https://github.com/dod38fr/config-model/wiki/Managing-Debian-packages-with-cme#maintaining-debian-copyright-file

(*) I always wondered if an erroneous value found in a file should be ignored 
or loaded. I've chosen the first option, which leads to the cascading errors 
you've found. I guess that I should implement the second option at least when 
-force option is used.

-- 
 https://github.com/dod38fr/   -o- http://search.cpan.org/~ddumont/
http://ddumont.wordpress.com/  -o-   irc: dod at irc.debian.org

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Andreas Tille]

On Sat, Oct 21, 2017 at 06:40:37PM +0200, gregor herrmann wrote:
> On Sat, 21 Oct 2017 16:20:58 +0200, Andreas Tille wrote:
> 
> > $ gbp clone https://anonscm.debian.org/git/debian-med/beads.git
> > $ cd beads
> > beads(master) $ cme run paste-license  --arg license=CeCILL --arg COPYING 
> 
> % cme run paste-license  --arg license=CeCILL --arg file=COPYING
> 
> seems to work; note the "file=" part as in dod's original mail:

Well, OK, when I do not miss the "file=" part I can do
 
$ cme run paste-license  --arg license=CeCILL --arg file=COPYING > 
copyright.patch
Log4perl: Seems like no initialization happened. Forgot to call init()?
Warning: Files:"*" License short_name skipping value CeCILL because of the 
following errors:
license 'CeCILL' is not declared in main License section. Expected 

Warning: Files:"debian/*" License short_name skipping value CeCILL because of 
the following errors:
license 'CeCILL' is not declared in main License section. Expected 

License CeCILL is not used in Files: section
Configuration item 'Files:"*" License short_name' has a wrong value:
Undefined mandatory value.


I admit I do not really understand all the output to stderr.  The output
to stdout (in my example redirected to copyright.patch) is


-- 
hanges applied to dpkg-copyright configuration:
- License:CeCILL text:
@@ -1 +1,505 @@
-
+CeCILL FREE SOFTWARE LICENSE AGREEMENT
+
+
+Notice
+
+This Agreement is a Free Software license agreement that is the result
+of discussions between its authors in order to ensure compliance with
+the two main principles guiding its drafting:
+
+* firstly, compliance with the principles governing the distribution
...


So this does not really help since its neither a valid patch for
d/copyright nor can I add this content there without editing.  It just
added a '+' to the original license text which is not really helpful.
May be I'm continuously missing the point but shouldn't it add rather
a ' ' instead of a '+' and replace empty lines by ' .'?

Kind regards

Andreas.

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[gregor herrmann]

On Sat, 21 Oct 2017 16:20:58 +0200, Andreas Tille wrote:

> $ gbp clone https://anonscm.debian.org/git/debian-med/beads.git
> $ cd beads
> beads(master) $ cme run paste-license  --arg license=CeCILL --arg COPYING 

% cme run paste-license  --arg license=CeCILL --arg file=COPYING

seems to work; note the "file=" part as in dod's original mail:

> On Tue, Oct 17, 2017 at 01:20:15PM +0200, Dominique Dumont wrote:
> > $ cme run paste-license  --arg license=MyTest --arg file=my-lic.txt



Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   NP: Flying Pickets: Only You


signature.asc
Description: Digital Signature

Re: New: "cme run paste-license script" (was: Re: pasting license text into debian/copyright)

[Andreas Tille]

Hi Dominique,

I wanted to try this cool feature but failed may be I did something wrong:

$ gbp clone https://anonscm.debian.org/git/debian-med/beads.git
$ cd beads
beads(master) $ cme run paste-license  --arg license=CeCILL --arg COPYING 
Log4perl: Seems like no initialization happened. Forgot to call init()?
Odd number of elements in hash assignment at 
/usr/share/perl5/App/Cme/Command/run.pm line 110.
cme: using Dpkg::Copyright model
Warning: Files:"*" License short_name skipping value CeCILL because of the 
following errors:
license 'CeCILL' is not declared in main License section. Expected 

Warning: Files:"debian/*" License short_name skipping value CeCILL because of 
the following errors:
license 'CeCILL' is not declared in main License section. Expected 

License CeCILL is not used in Files: section

^C


I needed to ^C since cme did not finished otherwise.

Am I doing something wrong or is there some issue with this new
cool feature?

Kind regards

   Andreas.

On Tue, Oct 17, 2017 at 01:20:15PM +0200, Dominique Dumont wrote:
> Hi
> 
> People have complained that adding license text in debian/copyright file is 
> tedious.
> 
> To avoid this problem, libconfig-model-dpkg-perl 2.102 now ships a new cme 
> script to copy a license text in debian/copyright. This script is run with 
> "cme run" command [1]
> 
> For instance:
> 
> $ echo -e "blah\n\nblah\n\nblah"  > my-lic.txt
> $ cme run paste-license  --arg license=MyTest --arg file=my-lic.txt
> cme: using Dpkg::Copyright model
> License MyTest is not used in Files: section
> 
> Changes applied to dpkg-copyright configuration:
> - License:MyTest text: 
> @@ -1 +1,5 @@
> -
> +blah
> +
> +blah
> +
> +blah
> 
> $ git diff
> diff --git a/debian/copyright b/debian/copyright
> index 60bf1722..6e85dadb 100644
> --- a/debian/copyright
> +++ b/debian/copyright
> @@ -22,3 +22,10 @@ License: LGPL-2.1+
>  License, or (at your option) any later version.
>  On Debian GNU/Linux systems, the complete text of version 2.1 of the GNU
>  Lesser General Public License can be found in `/usr/share/common-
> licenses/LGPL-2.1'
> +
> +License: MyTest
> + blah
> + .
> + blah
> + .
> + blah
> 
> The doc specific to this script is shown with -doc option:
> 
> $ cme run paste-license --doc
> paste license text in License paragraph
> paste file: cme run paste-license --arg license=Expat --arg file=MIT.txt
> paste STDIN: cat MIT.txt | cme run paste-license --arg license=Expat
> 
> Please ignore a warning message about missing initialisation of Log4Perl. 
> This 
> is harmless and will be fixed soon.
> 
> I hope this command will be useful to help you in your packaging activities.
> 
> All the best
> 
> [1] https://manpages.debian.org/testing/cme/App::Cme::Command::run.3pm.en.html
> 
> 
> -- 
>  https://github.com/dod38fr/   -o- http://search.cpan.org/~ddumont/
> http://ddumont.wordpress.com/  -o-   irc: dod at irc.debian.org
> 
> 

-- 
http://fam-tille.de