Re: Unaddressed use cases for machine-readable debian/copyright files

[Dominique Dumont]

On Saturday, 25 March 2017 16:25:38 CEST Guillem Jover wrote:
> Personally I have no issue with coalescing
> copyright notices, as long as they are all for the same license, etc.
> I even coalesce copyright years for the same owner.

Coalescing copyright notices and years is also done when running "cme update 
dpkg-copyright" 

See [1] for more details.

HTH

[1] 
https://github.com/dod38fr/config-model/wiki/Managing-Debian-packages-with-cme#update-a-package

-- 
 https://github.com/dod38fr/   -o- http://search.cpan.org/~ddumont/
http://ddumont.wordpress.com/  -o-   irc: dod at irc.debian.org

Re: Unaddressed use cases for machine-readable debian/copyright files

[G. Branden Robinson]

On Sat, Mar 25, 2017 at 04:25:38PM +0100, Guillem Jover wrote:
> Hi!

Thanks for your feedback!  Been a while since we've chatted.  :)

> On Fri, 2017-03-24 at 14:02:49 -0400, G. Branden Robinson wrote:
> > In returning my attention to current Debian packaging practices and
> > conventions I took my first serious look at good old DEP5, and brought
> > the debian/copyright file for my first-ever package, xtrs[1], into
> > conformance with the new[2] standard[3].
> > 
> > However, in doing so, I encountered some use cases that are not covered
> > by this standard.  Happily, all of them triggered lintian warnings as
> > well, which gives me a framework within which to present my problems.
> > 
> > The attached debian/copyright file may also illuminate these matters.
> 
> I think all your concerns are already mostly covered by the spec,
> except notably for the license namespacing.

Thanks!  I did completely overlook the section "Stand-alone License
Paragraph", which you brought to my attention through your patches.

However, I still think the copyright file specification could be clearer
with respect to what is deliberately left unspecified, rather than
leaving it implicit.

> Something that is also a common source of confusion, is that because
> it specifies a Files field, it seems it compels people to do very
> fine-grained splitting. Personally I have no issue with coalescing
> copyright notices, as long as they are all for the same license, etc.
> I even coalesce copyright years for the same owner. See for example
> the libbsd copyright file, being more fine-grained would be madness
> IMO. And it is my understanding that ftpmasters find this to be fine.

I agree that this is a maintainer discretion issue; perhaps ultimately
an ftpmaster discretion issue, at least at the lower bound of precision.

That such discretion is available to the maintainer should be made more
clear by the spec.  If we said this clearly, you might find fewer people
picking nits below your threshold of attention-worthiness.

xtrs is small enough that I could go atomic on it without much effort;
it's only 24k lines in 37 *.[ch] files.  I once did a license
enumeration deep dive on glibc that surprised me.  Packages that are
large and old (like glibc) or in languages that encourage rampant
cut-and-paste of source snippets (JavaScript) make exhaustive analysis
more challenging, and I would not mandate that my peers undertake such
without good reason, such as a hostile copyright holder.

> So instead of going over all your fine points, I've taken the liberty
> instead to fix the copyright file in what I'd consider to be correctly
> formatted; in two ways, first following the original spirit, of very
> detailed listing, and then another condensed one which is what I'd
> have done. They are incremental patches over your original one.

I was a little confused by this because your first diff is against the
xtrs debian/copyright file currently in the archive.  I.e., it's against
xtrs 4.9c, not the one for 4.9d which I attached in my mail.

I don't see much to distinguish our approaches:
1. You use forward references to license short names; my preference
   would be to use backward references.
2. You put nothing on the first line of a multi-line Files field; I do.
   But I'll change that.
3. You don't use bracket notation in your globs.  Re-reading the
   copyright format doc, I see I'm out of spec on this.  I'll fix that.
4. You (in your condensed form) always put the contents of the Copyright
   field on the subsequent line, even if it would fit on the first.
5. You place importance on the varying renditions of the copyright sign
   ( (c), ©, etc.); I don't.  My lay understanding of U.S. copyright law
   and the various international treaties on the subject is that
   copyright notices do not require any particular rendition of this
   symbol to be effective.  Either of the foregoing, or "(C)" or the
   word "copyright" in any case rendition is adequate, even leaving
   aside the implicit copyright that attaches to any copyright-eligible
   work that goes into "fixed form", as which a text file surely
   qualifies.  But I'm not quite ready to wade back into debian-legal
   yet...

I don't find any of the foregoing to be hills worth dying on; certainly
not #3, where I am surrendering the field. ;-)

I'm attaching my latest revision of debian/copyright based on your
guidance.  It still reflects a degree of wild card phobia, but that's
mainly because I had already done the work to categorize things.

My file (and the whole package, for that matter) is lintian-clean
now--thank you!

In my view, two questions remain:
A. Does the debian/copyright file format spec require an update to
   improve clarity and guidance?  My opinion is that it does.  What do
   other folks on the list think?
B. What shall we do about the namespace problem?  I'm inclined to start
   a new thread on that.

Regards,
Branden

[1] https://www.debian.org/doc/packaging-manua

Re: Unaddressed use cases for machine-readable debian/copyright files

[Russ Allbery]

Guillem Jover  writes:

> Something that is also a common source of confusion, is that because
> it specifies a Files field, it seems it compels people to do very
> fine-grained splitting. Personally I have no issue with coalescing
> copyright notices, as long as they are all for the same license, etc.
> I even coalesce copyright years for the same owner.

+1.  I would strongly encourage people to do this wherever possible.
There doesn't seem to be much purpose served by being more granular,
unless it's a side effect of automatically generating the file or
something.

-- 
Russ Allbery (r...@debian.org)   

Re: Unaddressed use cases for machine-readable debian/copyright files

[Guillem Jover]

Hi!

On Fri, 2017-03-24 at 14:02:49 -0400, G. Branden Robinson wrote:
> In returning my attention to current Debian packaging practices and
> conventions I took my first serious look at good old DEP5, and brought
> the debian/copyright file for my first-ever package, xtrs[1], into
> conformance with the new[2] standard[3].
> 
> However, in doing so, I encountered some use cases that are not covered
> by this standard.  Happily, all of them triggered lintian warnings as
> well, which gives me a framework within which to present my problems.
> 
> The attached debian/copyright file may also illuminate these matters.

I think all your concerns are already mostly covered by the spec,
except notably for the license namespacing.

Something that is also a common source of confusion, is that because
it specifies a Files field, it seems it compels people to do very
fine-grained splitting. Personally I have no issue with coalescing
copyright notices, as long as they are all for the same license, etc.
I even coalesce copyright years for the same owner. See for example
the libbsd copyright file, being more fine-grained would be madness
IMO. And it is my understanding that ftpmasters find this to be fine.

So instead of going over all your fine points, I've taken the liberty
instead to fix the copyright file in what I'd consider to be correctly
formatted; in two ways, first following the original spirit, of very
detailed listing, and then another condensed one which is what I'd
have done. They are incremental patches over your original one.

Thanks,
Guillem
diff --git a/debian/copyright b/debian/copyright
index 92365ec..5101554 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -1,46 +1,164 @@
-Package: xtrs
-Obtained from: http://www.tim-mann.org/trs80/xtrs-4.9c.tar.gz
-   http://www.tim-mann.org/trs80faq.html
-Upstream authors: David Gingold, Alec Wolman, Timothy Mann
-Debian package author: Branden Robinson
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: xtrs
+Source: http://www.tim-mann.org/xtrs.html
+Disclaimer: Requires non-DFSG-free ROM images and/or operating systems
+ to be useful for most purposes.
+ .
+ There is a freely-licensed boot ROM for Model 4P emulation provided with xtrs;
+ however, this boot image can only be used to boot an operating system designed
+ for the Model 4 (it is not sophisticated enough to load the BASIC interpreter
+ ROM for Model III compatiblity mode, provided on Model 4P TRSDOS disks as a
+ file called MODELA/III).  Since most users will likely be using this emulator
+ to run proprietary legacy applications for the TRS-80 computers, I do not
+ regard this exception as sufficient to recategorize xtrs for inclusion in main.
+ .
+ It is worth keeping an eye on projects like Contiki and FUZIX; if one of them
+ becomes useful under xtrs, that would be an argument for moving xtrs to main.
+ + http://www.contiki-os.org/
+ + https://github.com/EtchedPixels/FUZIX
 
-Debian copyright notice and license terms:
+Files: *
+Copyright: 1998 Timothy Mann
+License: Timothy-Mann-xtrs-permissive-non-copyleft
 
-Unless otherwise noted, all independently copyrightable modifications and
-additions to xtrs found in its Debian packages bear the following copyright
-and license terms:
+Files:
+ ChangeLog
+ Makefile
+ cassette
+ cassette.man
+ cassette.sh
+ cassette.txt
+ cpmutil.html
+ crc.c
+ do6.jcl
+ dskspec.html
+ expall.bas
+ fakerom.lst
+ fakerom.z80
+ hex2cmd.man
+ hex2cmd.txt
+ m1format.fix
+ mkdisk.man
+ mkdisk.txt
+ reed.h
+ trs_chars.c
+ utility.jcl
+ xtrs.man
+ xtrs.txt
+Copyright: no explicit notices
+License: Timothy-Mann-xtrs-permissive-non-copyleft
+Comment:
+ I (Branden Robinson) presume them to be under Tim Mann's license (see below).
+ Some of them are plainly derived from others (such as cassette.txt from
+ cassette.man), and some may have been written by other people (e.g.,
+ cassette.sh was written by me, albeit obviously a it's a sort of
+ transliteration of cassette [csh]).
 
-Copyright 1998-2006, 2008 Branden Robinson
+Files:
+ cd.ccc
+ mount.ccc
+ pwd.ccc
+ truedam.ccc
+ umount.ccc
+ unix.ccc
+ xtrs8.lst
+ xtrs8.z80
+ xtrshard.lst
+ xtrshard.z80
+ xtrsmous.lst
+ xtrsmous.z80
+Copyright: 1998 Timothy Mann
+License: Timothy-Mann-xtrs-permissive-non-copyleft
 
-This software may be copied, modified, and used for any purpose
-without fee, provided that (1) the above copyright notice is
-retained, and (2) modified versions are clearly marked as having
-been modified, with the modifier's name and the date included.
+Files:
+ cmd.c
+ cmd.h
+ hex2cmd.c
+ trs_disk.c
+ trs_disk.h
+ trs_imp_exp.c
+ trs_imp_exp.h
+ trs_interrupt.c
+Copyright: 1996 Timothy Mann
+License: Timothy-Mann-xtrs-permissive-non-copyleft
 
-Original copyright notice and license terms:
+Files:
+ cmddump.c
+ load_cmd.c
+ load_cmd.h
+ mkdisk.c
+Copyright: 1996-98 Timothy Mann
+License: Timothy-Mann-xtrs-permissive-non-copyleft
 
-*

Unaddressed use cases for machine-readable debian/copyright files

[G. Branden Robinson]

Hi folks,

Sorry to bother you again so soon.

In returning my attention to current Debian packaging practices and
conventions I took my first serious look at good old DEP5, and brought
the debian/copyright file for my first-ever package, xtrs[1], into
conformance with the new[2] standard[3].

However, in doing so, I encountered some use cases that are not covered
by this standard.  Happily, all of them triggered lintian warnings as
well, which gives me a framework within which to present my problems.

The attached debian/copyright file may also illuminate these matters.

W: xtrs source: empty-short-license-in-dep5-copyright (paragraph at line 48)
N:
N:The short license field in the machine readable copyright file is empty.
N:
N:Refer to
N:https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N:details.
N:
N:Severity: normal, Certainty: possible
N:
N:Check: source-copyright, Type: source
N:

1. I left this field blank because there are several files which bear no
copyright notice at all.  There is also no "top-level" copyright for
this package, and some of the files are probably not copyrightable
anyway, or are not independent works from other files in the
distribution.  (See my Comment in the file.)  I would like a successor
to DEP, call it DEP5++ for the nonce, to give me a mechanism for
declaring that a set of files fall into one of these categories which
are not really subject to machine analysis.  This is not a rare or
unusual case.  Based on my work experience in this area, I am strongly
confident that this issue affects many, many packages in main.  Many of
them are old, or come from the BSD tradition, or are from authors who
for whatever reason simply don't care to slap a copyright notice on
everything in sight.

My ask: As a package maintainer, I'd like a way to designate files in my
package as having been reviewed by me and, in my opinion, are at least
as freely licensed as some license used elsewhere in the package, and
which I impute to these files for analytical convenience.

W: xtrs source: missing-license-paragraph-in-dep5-copyright timothy mann xtrs 
permissive non-copyleft license (paragraph at line 142)
N:
N:The files paragraph in the machine readable copyright file references a
N:license, for which no standalone license paragraph exists.
N:
N:Refer to
N:https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N:details.
N:
N:Severity: normal, Certainty: possible
N:
N:Check: source-copyright, Type: source
N:

2. DEP5 is asking me to violate DRY.  It seems like a terrible idea to
do this.  I've already pasted the license boilerplate into this file
once, why can't I take advantage of that by citing to it by some form of
reference?  Why don't I just merge the filename patterns in with the
first occurrence of the license boilerplate?  Because the copyright
holder is different.  This package has 4 different copyright holders[5]
and 2 different licenses in it.  3 of those 4 copyright holders use the
"Tim Mann xtrs license", and the last, the original authors of xtrs,
Clarendon Hill Software, use another, which is just as permissive but
not as terse or clean.

My ask: As a package maintainer, I'd like to be able to designate a
license expression as an anchor to which subsequent Files->License
fields can refer, reducing the length, redundancy, and human parse time
of the document.

As a fan of YAML, I'd be thrilled if I could do something like this:

Files: foo.c bar.c
Copyright: 2013 Moe Szyslak
License: *FOOBAR-1
 This program is free and libre software; yadda yadda yadda...
 .
 This program is distributed in the hope that yadda yadda yadda...
 .
 You should have received a copy of the Foo Bar License yadda...

Files: baz.c
Copyright: 2017 Branden Robinson
License: &FOOBAR-1

W: xtrs source: space-in-std-shortname-in-dep5-copyright timothy mann xtrs 
permissive non-copyleft license (paragraph at line 142)
N:
N:The license header contains a short name with a space, which does not
N:conform to the specification.
N:
N:Refer to
N:https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N:details.
N:
N:Severity: minor, Certainty: certain
N:
N:Check: source-copyright, Type: source
N:
W: xtrs source: space-in-std-shortname-in-dep5-copyright clarendon hill 
software xtrs permissive non-copyleft license (paragraph at line 89)

3. This one's relatively simple.  What sort of identifier am I suppose
to use if a license in use by my package appears neither in the Debian
spec[3] nor in the SPDX license list[6]?  I guess I can make up my own,
but shouldn't we offer the package maintainer a bit more guidance in
this respect?  Shouldn't we have some naming conventions[7]?  I spoke to
Paul Wise on #debian-devel about this and he noted that (I'm
paraphrasing) Debian institutionally does not feel bound to SPDX
namespace or analytical precedents.  That's fine, but tools and users
need to be a