Re: [Development] Documenting 3rd party code Qt

2016-07-21 Thread Kai Koehne 


> -Original Message-
> From: Development [mailto:development-bounces+kai.koehne=qt.io@qt-
> project.org] On Behalf Of Thiago Macieira
> Sent: Wednesday, July 20, 2016 6:57 PM
> To: development@qt-project.org
> Subject: Re: [Development] Documenting 3rd party code Qt
> 
> On quarta-feira, 20 de julho de 2016 11:13:39 PDT Kai Koehne wrote:
> > I had a look at SPDX, README.Chromium, debian/copyright (btw thanks
> > for the pointer!). In the end I went for a custom format, because they
> > all seem to not quite fit for our use case. Anyhow, it's easy to
> > extend licensescanner to generate other formats, too.
> 
> What's missing from SPDX and have you tried to talk to them about adding
> the missing information?

I'm not sure whether anything particular is missing in the standard - my best
guess is we could create a syntactically valid SPDX file containing the same
information that we currently have in the proposed qt_attribution.json file 
format.

However, SPDX is a standard for "software packages" - it would probably make
more sense to add an SPDX file for QtCore or even qtbase. Using a SPDX file for
a single file in 3rdparty feels a bit like a misuse. 

Also, the tools I found to process spdx files are written in Java, which I 
certainly 
do not want to add as a build dependency. This would mean we'd write a custom 
parser for the subset of SPDX we'd support.

In the end I concluded it's just easier to have a tailored format we have full
control over, but can be used as a source for generating 
SPDX/debian_copyright/...
files as they're needed. I'd be happy to have a discussion about this with
someone interested in this.

My proposal is to use the names and matching guidelines from
https://spdx.org/licenses/ though. 


Regards

Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Documenting 3rd party code Qt

2016-07-20 Thread Thiago Macieira 
On quarta-feira, 20 de julho de 2016 11:13:39 PDT Kai Koehne wrote:
> I had a look at SPDX, README.Chromium, debian/copyright (btw thanks for the
> pointer!). In the end I went for a custom format, because they all seem to
> not quite fit for our use case. Anyhow, it's easy to extend licensescanner
> to generate other formats, too.

What's missing from SPDX and have you tried to talk to them about adding the 
missing information?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Documenting 3rd party code Qt

2016-07-20 Thread Kai Koehne 


> -Original Message-
> From: Development [mailto:development-bounces+kai.koehne=qt.io@qt-
> [...]
> >  That might be helpful, although the current format lacks, for
> > example, per- entity copyright (yes, each contributor name).
> [...]
> I'm genuinely curious though why debian/copyright needs this on a file level.
> Do you know the reasons?

Actually re-reading " Example files paragraphs" from 
https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ :

> Files: *
> Copyright: 1975-2010 Ulla Upstream
> License: GPL-2+
>
> Files: debian/*
> Copyright: 2010 Daniela Debianizer
> License: GPL-2+
>
> Files: debian/patches/fancy-feature
> Copyright: 2010 Daniela Debianizer
> License: GPL-3+
> 
> Files: */*.1
> Copyright: 2010 Manuela Manpager
> License: GPL-2+
>
> [...]
> Since the license of the manual pages is the same as the other files in the 
> package, the last paragraph above could instead be combined with the first 
> paragraph, listing both copyright 
> statements in one Copyright field. Whether to combine paragraphs with the 
> same license is left to the discretion of the author of the debian/copyright 
> file.

If I got this right it says you can combine copyright lines from multiple files 
if the license is the same. So far we don't actually support differently 
licensed files in one json object (that's why e.g. 
https://codereview.qt-project.org/#/c/160970/11/src/3rdparty/angle/qt_attribution.json
 is split up in multiple objects). Hence I'm questioning whether we really need 
to support separating Copyright's per file.

Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Documenting 3rd party code Qt

2016-07-20 Thread Kai Koehne 


> -Original Message-
> From: Development [mailto:development-bounces+kai.koehne=qt.io@qt-
> project.org] On Behalf Of Lisandro Damián Nicanor Pérez Meyer
> Sent: Wednesday, July 20, 2016 4:04 PM
> To: development@qt-project.org
> Subject: Re: [Development] Documenting 3rd party code Qt
> 
> On miércoles, 20 de julio de 2016 11:13:39 A. M. ART Kai Koehne wrote:
> > Hi,
> [snip]
> > # File Format
> >
> > I had a look at SPDX, README.Chromium, debian/copyright (btw thanks
> > for the pointer!).
> 
> My pleasure :)
> 
> > In the end I went for a custom format, because they all seem to not
> > quite fit for our use case. Anyhow, it's easy to extend licensescanner
> > to generate other formats, too.
> 
>  That might be helpful, although the current format lacks, for example, per-
> entity copyright (yes, each contributor name).
>
> I *know* this is a big burden because I have to do it :) But if this can be
> added, even if optional, we the ones who need this info can fill these details
> on each of our reviews.

It can easily get laborious, but I don't see why we shouldn't allow something 
like

"CopyrightFiles" : [
   { "*": "Copyright (c) 1920 Joe Doe", },
   { "special_file": "Copyright (c) 2040 Jane Roe" }
]

in the format, sure. 

I'm genuinely curious though why debian/copyright needs this on a file level.  
Do you know the reasons?

Regards

Kai

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Documenting 3rd party code Qt

2016-07-20 Thread Lisandro Damián Nicanor Pérez Meyer 
On miércoles, 20 de julio de 2016 11:13:39 A. M. ART Kai Koehne wrote:
> Hi,
[snip]
> # File Format
> 
> I had a look at SPDX, README.Chromium, debian/copyright (btw thanks for the
> pointer!).

My pleasure :)

> In the end I went for a custom format, because they all seem to
> not quite fit for our use case. Anyhow, it's easy to extend licensescanner
> to generate other formats, too.

 That might be helpful, although the current format lacks, for example, per-
entity copyright (yes, each contributor name).

I *know* this is a big burden because I have to do it :) But if this can be 
added, even if optional, we the ones who need this info can fill these details 
on each of our reviews.

Would that be a good compromise for you?

-- 
Si vives cada día de tu vida como si fuera el último,
algún día realmente tendrás razón.
  Steve Jobs

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/


signature.asc
Description: This is a digitally signed message part.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

[Development] Documenting 3rd party code Qt

2016-07-20 Thread Kai Koehne 
Hi,

A while ago I proposed to generate 3rd party code attributions out of files 
that are right beside the code, instead of hand-editing 3rdparty.html in qtdoc. 
I've now a set of patches that are IMO worth to take a look at (also because FF 
for 5.8 is coming ;)

# Information Flow

Every 3rd party code in our repositories should get a 'qt_attribution.json' 
file containing metadata about the code. When the documentation of a module is 
built, a new tool 'licensescanner' is automatically run by qmake that scans the 
sources for qt_attribution.json files, and generates a .qdoc file with pages 
for every attribution it finds. An overview of the attributions is shown on 
each modules entry page. Finally, the current 3rdparty overview page should be 
automatically generated from all modules' attribution pages.

# File Format

I had a look at SPDX, README.Chromium, debian/copyright (btw thanks for the 
pointer!). In the end I went for a custom format, because they all seem to not 
quite fit for our use case. Anyhow, it's easy to extend licensescanner to 
generate other formats, too.

The current format is documented in  https://wiki.qt.io/Qt_attribution.json .

Real-life examples for qtbase can be found in 
https://codereview.qt-project.org/#/c/160970/ .

# QDoc Extension

To be able to document and structure the 3rd party content I introduced a new 
'attribution' attribute to qdoc's \page command, and added 
\generatelist{attributions}, \generatelist{annotatedattributions} commands.

# Patches

qttools patches (licensescanner, qdoc modifications): 
https://codereview.qt-project.org/#/q/status:open+project:qt/qttools+branch:dev+topic:3rdparty,n,z
qtbase patches (qt_attributions.json files, build integration, overview pages): 
https://codereview.qt-project.org/#/q/status:open+project:qt/qtbase+branch:dev+topic:3rdparty,n,z
qtdoc patch (for general overview page): 
https://codereview.qt-project.org/#/q/status:open+project:qt/qtdoc+branch:dev+topic:3rdparty,n,z

# FAQ

Q: Why do we need such an elaborate setup? Wouldn't it be easier to just write 
.qdoc files?

Keeping the 'documentation' close to the actual sources hopefully makes sure 
they're up to date and complete. Admittedly we could also just have .qdoc files 
right beside each code snipped, and include this one. In the future I'd like to 
extend the tool though so that attribution information (say HTML) for an app 
bundling Qt can be generated. The qt_attribution.json file could also be used 
as input to 3rd party code scanners popular in the industry, or used as a 
source for generating SPDX files ... Keeping the metadata outside of .qdoc 
makes this a lot easier.

Q: What about 3rd party code in tests/examples ...?

I think they should get a qt_attribution.json file too. We have to mark though 
what actually ends up in the Qt libs, and what only ends up in tests / build 
system / host tools ... I'm still contemplating how to best document this in 
the qt_attribution.json file.

Regards

Kai

-- 
Kai Köhne, Senior Manager R&D | The Qt Company

The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der 
Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development