Re: New deprecation warning in CMake 3.27

[Kyle Edwards]

On 6/17/23 12:29, Timo Röhling wrote:


Being backwards compatible is CMake's greatest blessing and greatest
curse at the same time, because people still run into crappy,
20 year old tutorials and needlessly complicated (but working) code
snippets from CMake 2.x on the Internet, making them believe that
CMake is crappy and needlessly complicated. It is better than
its reputation. It does lack good, recent tutorials though.

CMake upstream here. We do have a tutorial that we've put together and 
improved over the last few years that teaches modern CMake and avoids 
using the old directory-level commands. You can find it here: 
https://cmake.org/cmake/help/latest/guide/tutorial

Can you recommend a relatively safe & old version to use instead of
< 3.5 which doesn't need bumping next month but is also available in
most semi-current releases of all major distributions (as that is what
most upstreams will care about if they don't have special needs)?

The *correct* minimum version is actually not that easy to
determine, because CMake, for some reason, will let you say
"cmake_minimum_required(VERSION 3.0)" and still use newer commands
such as "add_link_options" (introduced in CMake 3.13) without
warning.


If you have a cmake_minimum_required() that's older than some of the 
commands you're using, the expected pattern to avoid using them is:


if(NOT CMAKE_VERSION VERSION_LESS 3.25)
  # Version-gated code here
endif()

Speaking of 3.13, Buster aka oldoldstable ships with CMake 3.13
(released in 2018) and Repology tells me that any relevant
distribution with an older CMake is either completely unsupported or
you need to buy LTS for it. That sounds like a reasonable support
baseline to me.

More elegant is a version range: if your script does not give
any policy warnings with a recent CMake (say CMake 3.26), you can
write something like "cmake_minimum_required(VERSION 3.0...3.26)",
which will continue to work with old CMake releases but not trigger
any support warnings for the forseeable future, because only the
upper bound is considered for that.


Please keep in mind that if you set an upper bound, any policies up to 
that upper bound will automatically be set to NEW, so be sure that your 
code is ready for that. You could potentially get some unexpected 
behavior if you have code that uses the OLD behavior of a policy in an 
older version of CMake and the NEW behavior in a newer version.


Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/20/21 2:37 PM, Simon Richter wrote:


This is a use case where HTTPS does hurt, and where I can't think of 
any good mitigation strategies that wouldn't be worse from a security 
PoV than the status quo.


Such situations are the exception rather than the norm. If https is 
detrimental to their setup, they can choose to opt out of it.


For everyone else, I think https should be the default.

Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/20/21 10:56 AM, Russ Allbery wrote:

Do you think using HTTPS makes security worse?

No idea whether I qualify as a "security expert" but as someone who has
spent a fair amount of time working in security, my concern is making
advice simple enough for people to follow.  Complicated, conditional, or
inconsistent advice means you lose people who decide this is all too hard
to understand and just do nothing.

"Use HTTPS everywhere that supports it" is simple and actionable advice
for the average person that will make them more secure.  There are
applications and sites where HTTPS doesn't really help, but other than
some unusual performance edge cases that are pretty rare in practice, it
doesn't hurt.  It's not magic fairy dust, but it does raise the bar
against a set of attacks, provides some additional privacy against casual
non-targeted snooping, and is a better default than not using TLS.

Personally, I think we should switch our default to HTTPS not because we
have a specific security flaw in mind against which HTTPS provides some
protection but because it's consistent with the general message that a lot
of us (including, for example, the EFF and the IETF) are trying to send to
average users who don't have the expertise to analyze any of this: use TLS
by default wherever you can.  It's not a panacea, but ubiquitous, default
use of TLS helps both your security and your privacy compared to either
the previous default of no TLS or spending a bunch of mental energy
picking and choosing.


I agree with all of this. While HTTPS might not be sufficient, it is 
still necessary. If the last decade of leaks, breaches, and scandals 
have taught us anything, it's that strong encryption should be the 
default, not an afterthought. We should be moving towards more 
encryption, not less.


Kyle

Re: Q: Use https for {deb,security}.debian.org by default

[Kyle Edwards]

On 8/19/21 3:46 PM, Simon Richter wrote:
For the most part, users would configure https if they are behind a 
corporate firewall that disallows http, or modifies data in-flight so 
signature verification fails, everyone else is better off using plain 
http.


Or they might configure https on the sheer principle of not wanting to 
have their traffic hoovered up by their ISP or anyone else who might be 
listening.


Kyle

Re: Debian package manager privilege escalation attack

[Kyle Edwards]

On 8/12/21 2:32 AM, Vincent Bernat wrote:

  ❦ 12 August 2021 10:39 +05, Andrey Rahmatullin:


I just ran across this article
https://blog.ikuamike.io/posts/2021/package_managers_privesc/ I tested
the attacks on Debian 11 and they work successfully giving me a root
shell prompt.

I don't think calling this "privilege escalation" or "attack" is correct.
The premise of the post is "the user should not be a root/admin user but
has been assigned sudo permissions to run the package manager" and one
doesn't really need a long article to prove that it's not secure.

I think the article is interesting nonetheless. Some people may think
that granting sudo on apt is OK. In the past, I think "apt install
./something.deb" was not possible.


Random thought: could it be possible to restrict non-sudo users to 
installing packages from repos that are signed by a GPG key that is 
already trusted by the system (the Debian archive key)? That way this 
attack could not be carried out. Then add a Unix group that allows apt 
installation from trusted repos, make apt setuid so it can do the 
privileged operations, and have it check that the user is root or part 
of the non-privileged group.


Just my $0.02.

Kyle

Re: Code

[Kyle Edwards]

On 8/11/21 7:34 AM, Horler, Johannes wrote:


Dear Debian Team,


hopefully I am writing this to the right email address. (In case I am 
not, I would be happy about being refered.) Recently I got interested 
in operating systems. Now I want to try to experiment with modifying one.


Is the complete source code of any Debian Version available somewhere 
in full (meaning everything ultimately written - in translated form - 
to the bootable device)? Maybe I am a bit slow to understand but I 
have failed to find something like it. Therefore I have decided to 
just ask you.


Yes, all of the source code for Debian is available, though it's divided 
into thousands of different packages by different authors. You can find 
links to the source code from packages.debian.org - just search a 
package name and you should be able to find the source from there.



Which component(s) are you looking for specifically? The bootloader? The 
kernel? systemd? glibc? The core userspace utilities?



Kyle

Re: Bug#990521: I wonder whether bug #990521 "apt-secure points to apt-key which is deprecated" should get a higher severity

[Kyle Edwards]

On 7/1/21 2:45 PM, Jeremy Stanley wrote:

Check out the sources.list manpage:

 "The files list one source per line (one-line style) or contain
 multiline stanzas defining one or more sources per stanza
 (deb822 style), ..."

And then there's an entire DEB822-STYLE FORMAT section which
explains in greater detail.


Thanks, I see this now. It sounds like sources.list is slowly becoming 
deprecated - I will consider migrating our repository instructions (and 
the associated convenience script) to use the deb822 *.sources format 
instead next time I'm mucking around with our repository.



Ubuntu ships tweaked snapshots of Sid for most stuff, and that dates
since well after Stretch froze for release. Of course, try it to be
sure, but I just checked an Ubuntu 16.04 LTS machine and the
sources.list manpage there indicates support for deb822. As for
Signed-By in Release files, Ubuntu 18.04 does seem to have support
according to the apt changelog, any version newer than 1.3 ought to
include support and it has 1.6 so should be plenty new enough for
that.


Thanks for the info.

Kyle

Re: Bug#990521: I wonder whether bug #990521 "apt-secure points to apt-key which is deprecated" should get a higher severity

[Kyle Edwards]

On 7/1/21 2:19 PM, Jeremy Stanley wrote:

Also, as other's have stated, deb822 might be a cleaner way to
express this.


I'm a little confused - I thought deb822 was just a generic format used 
in various places throughout Debian, including in the Release files. 
Where specifically would the signed-by information be stored? In the 
Release file as you said below, or somewhere on the user's machine?



On top of that, you can embed Signed-By fields with
your key fingerprint in your repository's Release files, in order to
highlight if someone gets an updated index which is signed by a
different key than you previously indicated it should be. I think
anything as recent as Stretch should support all of this.


Thanks. Our primary target is Ubuntu - does Ubuntu 18.04 support this?

Kyle

Re: Bug#990521: I wonder whether bug #990521 "apt-secure points to apt-key which is deprecated" should get a higher severity

[Kyle Edwards]

On 7/1/21 10:40 AM, Jeremy Stanley wrote:

Yes, that's a community-maintained wiki article with a few editors
(at least most of whom are also DDs in this case), started in
2017-03-22 to describe a specific model which discourages it, but
nowhere does that claim use of /etc/apt/trusted.gpg.d is officially
deprecated and when, much less link to official documentation
stating so. The article also does essentially nothing to explain the
risks that model wants to counter. Reading between the lines, it may
protect you from accidentally using a package repository (whose
maintainers you implicitly trust) in unintended ways. If the people
in control of those keys wanted to take control of your machine,
they still could, so it's not protecting you from any intentionally
malicious threats.

It might be good to get input on this from anarcat and dkg, as the
primary authors of that document, on the underlying intention, and
maybe add some further explanation to it indicating the real-world
threats this recommendation mitigates. Security policy should be
informed by risk analysis, and complicating things with additional
security controls which bring no appreciable improvement to the
actual security of the system but just "because you can" is
ultimately detrimental.


Hmmm. Perhaps the messaging around this could be cleared up then, to let 
people know that /etc/apt/trusted.gpg.d is not actually deprecated. A 
while back we received a bug report about our company's APT repository:


https://gitlab.kitware.com/cmake/cmake/-/issues/22245

The filer pointed us to this article claiming we should not use 
/etc/apt/trusted.gpg.d:


https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html

In response, we updated our keyring package to remove the 
/etc/apt/trusted.gpg.d files that had been added, and automatically 
replace them with [signed-by=] attributes in the sources.list (with 
permission from the user.) It sounds like this move was not necessary. 
Nevertheless, is it considered "wrong" to do it this way? Should I have 
left it alone?


Kyle

Re: Bug#990521: I wonder whether bug #990521 "apt-secure points to apt-key which is deprecated" should get a higher severity

[Kyle Edwards]

On 7/1/21 9:27 AM, Jeremy Stanley wrote:

It's not clear (to me at least) that placing keys into
/etc/apt/trusted.gpg.d is deprecated


According to https://wiki.debian.org/DebianRepository/UseThirdParty it is:

> The key MUST NOT be placed in /etc/apt/trusted.gpg.d or loaded by 
apt-key add.



There's nothing especially wrong about using signed-by, but
it's not the security fix some people seem to believe. In short,
*any* package you install can run arbitrary commands as the root
user on your system during installation.


Obviously, and the page linked above even says as much:

> However, the installation of any single malicious package from a 
malicious repository can currently undo these protections, for example 
by running a MaintainerScripts 
 command to override the 
configured preferences or by authorizing new OpenPGP keys. For the 
purposes of this page, /attacks by a package that belongs to a given 
repository/ are out of scope. To restrict /what an installed package can 
do/, see the larger UntrustedDebs 
 problem, and particularly 
Teams/Dpkg/Spec/DeclarativePackaging 
 for a 
potential solution.


In fact, the automatic [signed-by=] migration that I implemented uses 
exactly this avenue, albeit in an explicitly non-malicious way that 
prompts the user first.


Kyle

Re: Bug#990521: I wonder whether bug #990521 "apt-secure points to apt-key which is deprecated" should get a higher severity

[Kyle Edwards]

On 7/1/21 8:27 AM, Julian Andres Klode wrote:

I don't want to advertise signed-by=. We should aim to get deb822 format
supported in python-apt next cycle, and then advertise a consistent use
of deb822 .sources files.

Including, but not limited to, having d-i create
sources.list.d/.sources instead of sources.list.

It just looks bad in the legacy file format.

I'm still concerned having signed-by leads people to adding sources
they trust less, only to then be rootkitted by evil maintainer scripts
of packages in that repo.


If [signed-by=] isn't the way to go, then what is? I recently updated 
the keyring package in our company's APT repository to automatically 
migrate people to [signed-by=] since apt-key (and with it 
/etc/apt/trusted.gpg.d) is deprecated. This page suggested using 
[signed-by=] instead:


https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html

In addition, this Debian Wiki page (linked from the article above) 
suggests using [signed-by=] and not /etc/apt/trusted.gpg.d:


https://wiki.debian.org/DebianRepository/UseThirdParty

Kyle

Re: Hosting the original youtube-dl sources on salsa?

[Kyle Edwards]

On 10/30/20 8:29 AM, Domenico Andreoli wrote:

First things first, we need to have many copies of that and make it
available it all over the net.

Do you have a complete git repo that can be cloned?


This one is still online at the time of writing:

https://github.com/youtube-dl2/youtube-dl

Kyle

Re: Bug#963615: ITP: jakarta-annotation-api -- Annotations for common semantic concepts in the Java SE and Jakarta EE platforms

[Kyle Edwards]

On Thu, 2020-06-25 at 22:19 +0200, Emmanuel Bourg wrote:
> I'm not sure to understand what bothers you. In this case the final
> version will be nearly identical. The JakartaEE APIs consist
> basically
> in the JavaEE versions that have been stable and in use for years
> with
> the base package renamed from 'javax' to 'jakarta'. I don't expect
> much
> surprise in the final version.

Speaking as one of the CMake developers, we sometimes make breaking
changes during our RC cycles, and don't freeze the interface until the
official .0 release. There's a reason our RCs don't typically go into
production distros, and there is the possibility that JakartaEE could
do the same thing.

However, the "experimental" repo does take RC packages. Perhaps
experimental would be a better place for this?

Kyle

Re: Free open-source ONLYOFFICE Desktop Editors: Compile from source code for Debian repository

[Kyle Edwards]

On Tue, 2020-05-19 at 14:25 +, Kseniya Fedoruk wrote:
> Hi team,
> 
> I'm Kseniya from ONLYOFFICE. We are developing open-source office
> applications, including the absolutely free ONLYOFFICE Desktop
> Editors that provide alternatives to MS Office Word, Excel, and
> PowerPoint distributed under GNU AGPL v3.0.
> 
> Currently, users can install the suite on various Linux distros from
> deb, rpm, Snap, Flatpak, and AppImage. We often receive requests for
> the native default support. Therefore, we’ve decided to contact you
> to ask about a possible integration of ONLYOFFICE Desktop Editors
> into Debian.
> 
> ...
> 
> So, will you be interested in adding ONLYOFFICE Desktop Editors to
> the Debian system so that lots of users get one more office suite
> option highly compatible with OOXML formats and giving access to
> cloud collaboration?

Hi Kseniya,

Welcome to the debian-devel mailing list. If you would like to see
ONLYOFFICE in Debian, feel free to file an RFP (request for package)
bug against Debian's bug tracking system:

https://wiki.debian.org/RFP

Or, if you would like to package it yourself, you can file an ITP
(intent to package):

https://wiki.debian.org/ITP

Note that you will need a sponsor (a Debian Developer who reviews the
package for compliance with Debian policy) if you choose to package it
yourself.

Kyle

Re: [Fwd: Bug#959057: RFS: dh-cmake/0.4 [ITP] -- Debhelper programs for CMake projects]

[Kyle Edwards]

Hi Alastair,

On Tue, 2020-05-12 at 10:47 +0100, Alastair McKinstry wrote:
> Dear Kyle,
> 
> I'm willing to sponsor this.

Thank you very much for the sponsorship offer - I gladly accept! :)

So what's next? I am new to Debian development, and this is my first
time submitting a package.

> It overlaps /complements a package I maintain "ecbuild", which is a
> set
> of cmake macros and wrappers used by ECMWF (www.ecmwf.int) for their
> software stack.
> At the moment ecbuild provides some standard flags, etc when
> --buildsystem=ecbuild is used  but the plan is to expand this for
> other
> cmake fragments and dependencies similar to work here Packages in the
> ECMWF stack typically include cmake fragments in
> /usr/lib/$arch/cmake/$package/* which define for example which
> libraries
> and headers are needed; eg. when something at the botttom of the
> stack
> is built with libjemalloc, packages throughout the stack now need to
> link against it and include libjemalloc-dev in the dependencies for
> their libX-dev packages. ;

Very nice. It's always interesting to see the CMake frameworks that
people have developed. Would you have any interest in sharing it on the
CMake Discourse server? We may be able to offer some feedback. https://
discourse.cmake.org/c/community

Kyle

[Fwd: Bug#959057: RFS: dh-cmake/0.4 [ITP] -- Debhelper programs for CMake projects]

[Kyle Edwards]

 Forwarded Message 
From: Kyle Edwards 
Reply-to: Kyle Edwards ,
959...@bugs.debian.org
To: Debian Bug Tracking System 
Subject: Bug#959057: RFS: dh-cmake/0.4 [ITP] -- Debhelper programs for
CMake projects
Date: Tue, 28 Apr 2020 20:54:33 +

> Package: sponsorship-requests
> Severity: wishlist
> 
> Dear mentors,
> 
> I am looking for a sponsor for my package "dh-cmake":
> 
>  * Package name: dh-cmake
>    Version     : 0.4
>    Upstream Author : Kyle Edwards 
>  * URL : https://gitlab.kitware.com/debian/dh-cmake
>  * License : BSD-3
>  * Vcs : https://gitlab.kitware.com/debian/dh-cmake.git
>    Section : devel
> 
> It builds those binary packages:
> 
>   dh-cmake - Debhelper programs for CMake projects
> 
> To access further information about this package, please visit the
> following URL:
> 
>   https://gitlab.kitware.com/debian/dh-cmake
> 
> Regards,
> 
> --
>   Kyle Edwards

Re: Third-party forks of packaged projects

[Kyle Edwards]

On Wed, 2020-04-29 at 10:21 +0100, Alastair McKinstry wrote:
> I think vendoring libraries that are only used by this package is
> fine.
> 
> I would however put them in a "public" place and namespace (eg
> /usr/lib/$ARCH/libatl.* ) rather than a subdir or namespace
> ('libatl-adios')  so that there is a clear collision if someone else
> attempts to package it; in which case it can be 'unvendored' and
> shared.

Thanks Alastair - these libraries are already in the public namespace
and named libadios2_atl.so, etc. so as not to conflict with an
unvendored version. If we decide to unvendor them later on, it won't be
difficult at all.

Kyle

Re: Third-party forks of packaged projects

[Kyle Edwards]

On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote:
> I'd say that it depends, and that it should be addressed on the
> case-by-case basis.

I do have another scenario I'd like to address. ADIOS uses a stack of
closely related but separate projects, all developed by Greg
Eisenhauer, which, as far as I know, are not used by any other major
software project. ADIOS is the main driver behind them. The projects
are the following:

https://github.com/GTkorvo/atl
https://github.com/GTkorvo/dill
https://github.com/GTkorvo/EVPath
https://github.com/GTkorvo/ffs

ADIOS vendors these projects as well. Are these reasonable to be
packaged with ADIOS, or should they be packaged separately?

Kyle

Re: Third-party forks of packaged projects

[Kyle Edwards]

On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote:
> If the library you need to package isn't needed by any other package,
> simply apply the needed patch and upload. Even better if it's only a
> build-dependency, in which case it wont ever be a problem.

This brings up an interesting question: are things that are build-
dependencies only covered by the no-vendoring rule? For example,
RapidJSON appears to be header-only (rapidjson-dev). If ADIOS ships
with its own copy of RapidJSON, does it still have to use the one
provided by Debian even though it will have little to no effect on the
final package?

> The other possibility (which is probably the best way forward) is to
> convince your upstream that he did a fork, and that it should be
> renamed
> to avoid conflicts.

In my case, I *am* upstream. So, for example, we forked enet and made
some changes to how it works. In that case, I could make a new package
called adios2-enet or ornl-enet, and depend on it for ADIOS. Is that
correct?

Kyle

Re: Third-party forks of packaged projects

[Kyle Edwards]

On Sat, 2020-04-25 at 00:02 +0200, Mattia Rizzolo wrote:
> If the upstream maintainer of that library is not available anymore
> and
> the project is clearly dormient, perhaps you can take over
> officially?
> Or if that patch is "acceptable" just leave it on the bug tracker,
> and
> within debian that patch could be applied, so that at least downstram
> we
> can strip off that vendored library.  That's cleraly possible only if
> that patch doesn't break stuff, etc.

Mattia,

Thanks for the insight. Two years ago I encountered this issue with
VTK. We use a modified version of libharu, and submitted our patches
upstream, but upstream has not been active in 5 years. I attempted to
submit our patches to Debian, but was told it would be better if we
forked it outright and made our own release. In the end, I decided it
wasn't worth the effort.

I am now encountering a similar issue with ADIOS2 (https://github.com/o
rnladios/ADIOS2). We use a modified version of enet (https://github.com
/GTkorvo/enet), and from what I understand, we can't use the upstream
version, so this is a situation where either enet would have to be
vendored with ADIOS or we would have to package our fork.

> vendored dependencies are not really a strict no-go.  cases like what
> you describe are one reason to keep them vendored within a package,
> and
> the security team tries keep a list of such vendored libraries, but
> since few maintainers reports back changes in this area, that list is
> not really that good (reason not to vendor libraries!!).

I did not realize that exceptions were occasionally made for vendored libraries 
- I thought the policy was "if the dependency can't be externalized, then it 
can't be used"... though I do recall a recent discussion about Docker/k8s 
having this issue.


Kyle

Third-party forks of packaged projects

[Kyle Edwards]

Hello,

I have a question about how Debian handles modifications to third-party 
dependencies. Sometimes a project relies on another project, but has
made modifications to that project that never went into upstream,
either because upstream has abandoned the project or because the
changes are not appropriate for upstream. In that case, the depending
project "vendors" the third-party dependency with the modifications
that it needs.

Obviously, "vendored" dependencies are a no-go in Debian, but how do
situations like this get resolved? I'm imagining the modified project
could be packaged on its own the way any fork would - in that manner,
it would not be much different from packaging MariaDB even though a
package for MySQL already exists. Is my intuition correct here?

Kyle

Re: What should I learn for applications befor, C or C++?

[Kyle Edwards]

On Wed, 2020-04-15 at 16:05 +0300, Constantine Ryzhikov wrote:
> Hello.
> Today, my main programming language is Python. I have worked with
> other 
> programming languages.
> I'd like to become a stronger developer, and be able to write 
> performance code.
> I'm interested in developing applications for Windows and Linux.
> I was interested in C#. But this is not fully supported by Linux, 
> unfortunately.
> I hate system programming.
> What should I learn first? C or C++?
> Thanks in advance!

I would suggest learning C++ first. Although C and C++ both require
memory management, C++ does a lot of the legwork for you, and also
abstracts away a number of things that would have to be spelled out
explicitly in C (vtables, destructors, etc.)

By the way, C# has at least partial support on Linux, through the Mono
project.

Happy hacking!

Kyle

Re: What to do when DD considers policy to be optional? [kubernetes]

[Kyle Edwards]

On Thu, 2020-03-26 at 19:57 +0100, Andrej Shadura wrote:
> An example: commercial users. They need to know *exactly* what they
> are running and under which licenses. They often want to be holier
> not
> only than the Pope, but holier than the whole population of Poland,
> Italy and Spanish-speaking countries altogether (I hope I don’t
> offend
> anyone with this comparison, it’s meant as a joke). They are often
> bound by regulations with heavy fines for violating them, and not
> only
> fines, but a threat of your product being banned, and that often
> means
> they want only specific licenses in their products.
> 
> And then there are Debian derivatives that cater to such commercial
> users. And guess what? The users tell the makers of the derivatives
> debian/copyright data that comes from Debian is not sufficiently
> strict or precise, and for that they need to set up their own
> processes to double-check and review the Debian copyright data.

Is it Debian's responsibility to cater to commercial users though?
Debian is not receiving commercial funding, or at least not directly.
It seems to me like the responsibility for this work falls on the
commercial users, not Debian.

Kyle

Re: OpenLiteSpeed Build Script Violating Debian Upstream Guide

[Kyle Edwards]

On Fri, 2020-03-06 at 20:38 +0700, Bagas Sanjaya wrote:
> 1. The script builds OLS using specific version of CMake (3.14.5) and
> Go 
> (1.6), though it did mention installing cmake as build dependency
> from 
> apt. The script downloads both build tools and install them to 
> `/usr/local` (which Debian packages never do).

The CMake dependency should be easy enough to fix, as CMake 3.16 is
available in Debian Sid.

Kyle

Re: Rethinking tensorflow build (taking shortcuts)

[Kyle Edwards]

On Fri, 2019-10-04 at 12:27 -0600, Daniele Nicolodi wrote:
> For those not following closely, can you please point to the reasons
> why
> bazel (cannot?) is (yet?) packaged for Debian?  I recently pumped
> into
> another software that proposes to use bazel as its build system, thus
> I
> would like to be aware of any problems with it.
> 
> Thanks!
> 
> Cheers,
> Dan

The issues have been discussed on the ITP for Bazel:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782654

Kyle

Re: Changing quote signs in GPL allowed? [Was: Bug#929752: installation-guide: left quotes in gpl.xml are not correctly rendered in pdf ]

[Kyle Edwards]

On Tue, 2019-08-06 at 21:05 +0200, Jonas Smedegaard wrote:
> Quoting Holger Wansing (2019-08-06 20:30:56)
> > 
> > [Adding debian-devel for a bigger audience]
> > 
> > 
> > Hi,
> > 
> > Holger Wansing  wrote:
> > > 
> > > Control: retitle -1 Replace grave accents/single quotes by
> > >  in gpl.xml
> > > 
> > > Samuel Thibault  wrote:
> > > > 
> > > > Hello,
> > > > 
> > > > Changwoo Ryu, le jeu. 30 mai 2019 22:22:05 +0900, a ecrit:
> > > > > 
> > > > > In en/appendix/gpl.xml, ordinary double quotes (") and grave 
> > > > > accent character (`) are used for left single/double quotes.
> > > > > But 
> > > > > they are not correctly rendered in pdf.
> > > > > 
> > > > > See the English PDF at 
> > > > > https://www.debian.org/releases/stable/amd64/install.pdf.en. 
> > > > > Opening double quotes are rendered as right double quotes.
> > > > > And 
> > > > > grave accents are rendered just as grave accents, not a left 
> > > > > single quotes.
> > > > AIUI we should just replace "" and `' in the xml file with 
> > > > .
> > I was about to commit these changes, however it came to my mind if 
> > such changes to the GPL are allowed?
> > 
> > At least the English variant of the GPL is 'official' and is not to
> > be 
> > changed, so what about changing the quoting signs into  
> >  entities?
> Some very narrow interpretation of "not to be changed" disallow
> making 
> the text into a PDF at all, as it is not bit-by-bit identical.
> 
> GNU GPL-1, GPL-2, and GPL-3 however all permit "verbatim copies"
> which I 
> can only interpret as permitting changes to punctuation and
> whitespace.

IANAL. In my opinion, replacing punctuation with equivalent markup to
render correctly while still keeping the original wording is within the
spirit of GPL.

I think the best thing to do would be to contact the FSF and ask for
clarification. I'm sure they'd be happy to answer this.

Kyle

Re: Question about embedded Lua

[Kyle Edwards]

On Thu, 2019-07-25 at 09:45 -0700, Russ Allbery wrote:
> Tobias Frost  writes:
> 
> > 
> > I think there is another option, as embedding lua is a bad idea for
> > the
> > reason you have already quoted: There are currently two (three with
> > experimental) lua versions available in Debian, so you should run
> > one of
> > those. This is especially true if the to-embedded-library is
> > someting
> > like the caliber of lua.
> I think you may be interpreting this with wrong definition of
> "embed."
> "Embedded Lua" normally means that the compiled program contains or
> links
> with a Lua interpreter and some configuration or plugins can be
> written in
> Lua.  I would just read "embed Lua" as "link the program with a Lua
> interpreter."
> 
> It doesn't mean that the Lua source code ships with the source of the
> program, or that the program has its own forked copy of Lua.  That's
> normally called "vendoring" or something more informal.
> 
> The problem is what version of Lua to link with, and how to handle
> upgrades (which are inevitable since option 2 from the original
> message
> isn't viable in the long run; eventually, we're going to remove the
> old
> version of Lua from the archive).

Exactly. If you choose 5.3 now, and backwards compatibility is a core
goal, you're stuck supporting 5.3 for a VERY long time, since 5.4 and
beyond are not backwards compatible. But eventually, perhaps 10 or 20
years from now, Debian is going to drop 5.3, meaning you need an
upgrade path. That upgrade path means simultaneously linking against
two versions of Lua, which isn't feasible unless you mangle the symbol
names.

Kyle

Question about embedded Lua

[Kyle Edwards]

Hi all,

As per Debian policy, programs and libraries are generally not allowed
to embed their own copies of libraries that are present in another
package, in order to avoid duplication of code and to enable security
updates by updating shared libraries. In general, this is a good rule,
but I'm wondering if an exception can be made in certain circumstances.
Specifically, I'm thinking about Lua.

New versions of Lua aren't backwards compatible with old ones. In other
words, Lua 5.3 is as different from Lua 5.2 as Python 3 is from Python
2. If someone wants to include Lua in their program, they have several
options:

1. Always embed the latest version of Lua, breaking backwards
compatibility as it gets upgraded
2. Pick a single version of Lua and stick with it forever, missing out
on the benefits of newer versions but retaining backwards compatibility
3. Embed *multiple* versions of Lua, allowing the end user to select
the version they want, which allows them to reap the benefits of the
upgrade while still retaining backwards compatibility

Let's say someone goes with the 3rd option. I don't see how that could
be made to work without embedding a copy of Lua inside the program
itself, mangling the symbol names to avoid a conflict. In other words,
such a program could not link against the system versions of Lua, since
trying to link against liblua5.2 and liblua5.3 at the same time would
cause a symbol conflict. (I know there are versioned symbols, but does
this allow a program to link against multiple versions of the same
symbol? Even if it does, this seems very error prone.)

If such a program chose to embed multiple, modified copies of Lua for
symbol mangling, and did not provide the option to build against an
external Lua, would such a program still be eligible for inclusion in
Debian even though it violates the "don't include your own copies of
libraries" rule? Could an exception be made for this circumstance?

Kyle

Re: Debian distribution

[Kyle Edwards]

On Tue, 2019-07-16 at 10:22 +, Javeed Ahmed wrote:
> sir/madam
> can i make my own  os using debian as a base and distribute it?

Absolutely! Debian is free software, and you are free to use, modify,
and distribute it for any purpose. Please make sure to follow the rules
of each package's license (GPL, BSD, etc.), but in general, yes, all of
these licenses allow modification and redistribution. See:

https://www.debian.org/social_contract#guidelines

for a description of the Debian philosophy of what consitutes "free
software".

Kyle

Re: Debian and our frenemies of containers and userland repos

[Kyle Edwards]

On Thu, 2019-07-11 at 23:25 +0800, Yao Wei wrote:
> Hi,
> 
> Following to Mo Zhou's thread of Conda and Debian, it reminds me
> that,
> could Debian reduced into a "proof of concept" as an operating system
> with collection of apps and things composed of completely free
> software,
> as more and more software repositories are moving away from the free
> software repositories like Debian, and userland repositories and app
> containers becomes more prominent and easier to access.  I feel the
> fear
> when I was in a flatpak session in DebConf17.
> 
> It can be a "solid base" of container images and barebone systems,
> but
> the days are numbered as operating systems as free and focused on its
> mission (like Google COOS, Yocto, Alpine etc.) is evolving steady.
> 
> Could it be a disaster for us?  And more importantly, do users care?
> 
> Best regards,
> Yao Wei

I think that there will always be some demand for APT-style packages as
opposed to containers. I know that I personally am a lot more likely to
use software if I can get it from apt-get than if I have to download a
Docker image.

I have also seen this first-hand from other people. Since we launched
our Ubuntu CMake APT repository earlier this year, it has become
massively popular. It is regularly getting hundreds of downloads per
week, both from people inside our company and from external users. I
suspect it is only going to get bigger over time, especially with our
next release cycle. I get emails about it almost every day.

The demand is there. APT-style repositories aren't going away any time
soon.

Kyle

Re: Question about Debian build infrastructure

[Kyle Edwards]

On Mon, 2019-06-10 at 16:56 +, Jeremy Stanley wrote:
> 2. Don't place all your trust key revocation, instead plan a
> rotation schedule so that even if a key falls into the wrong hands
> it's more likely users will smell something fishy when they see it
> used to sign new artifacts after expiration, even if they're not
> regularly checking for key revocations.

We generate new keys once a year, publishing them in a keyring package
6 months before they start being used. We are currently signing with
the 2019 key, and will generate the 2020 key in a few weeks.

> 3. Use signing-only subkeys in your automation, not your master
> keys; this way the master keys can kept symmetrically encrypted
> somewhere you only access to create or revoke subkeys (maybe even on
> the other side of an air gap or locked up in an HSM).

We are doing this already :)

> 4. Package the public keys and make sure your pregeneration/rotation
> schedule takes into account a slowest reasonable update frequency,
> so that your users are likely to have your next key in place and
> trusted before you transition to signing anything with it.

We are publishing a keyring package which takes care of this.

> 5. It probably goes without saying, but always generate a revocation
> certificate for every master key when you create that key, and keep
> it somewhere safe as a precaution in case you lose control of the
> master.

We are doing this already :)

> 6. To allow for easier manual verification of key transitions,
> always sign new keys with their predecessors when creating them.

We haven't signed the new key at the GPG key-signing level, but we've
effectively signed it at the APT level by publishing it in the
repository 6 months before it gets used.

> 7. You *may* want to separate signing and artifact building onto
> different systems so you have tighter control over key distribution
> and can shield key material from processes involving the running of
> less-trusted/arbitrary code, though this still comes with its own
> "chain of custody" problem as you need some way for the signing
> system to trust not only the build system but now also the channel
> through which the artifacts are transferred.

The only place where the APT key gets used is to publish the APT
repository itself. Internally, the build process signs with a
disposable, unpublished GPG key that can easily be revoked if it's
compromised. (The Aptly server is also only visible inside our network,
so outsiders can't upload to it anyway.)

> 8. Publishing fingerprints and validity dates for your keys
> alongside your release documentation/announcements is sometimes
> beneficial for bootstrapping initial trust (we also publish full
> copies of our public keys on their own page, in addition to pushing
> them into the public keyserver network).

I've published the key and installation instructions on our APT landing
page.

> There's probably more I'm forgetting, but that's at least a good
> start at mitigating unattended use of unencrypted keys while
> maintaining a robust and resilient signing infrastructure.

It sounds like I'm doing most of this already. Thank you for all the
advice!

Kyle

Re: Question about Debian build infrastructure

[Kyle Edwards]

On Sat, 2019-06-08 at 02:25 +0200, Matthias Klumpp wrote:
> Am Do., 6. Juni 2019 um 20:33 Uhr schrieb Kyle Edwards
> :
> > 
> > 
> > Hello all,
> > 
> > I have been preparing Ubuntu releases for CMake on our own APT
> > repository for several months now. We did this by preparing our own
> > repository infrastructure - we have a machine that builds packages,
> > and
> > a machine that hosts an Aptly instance and pushes the repository to
> > our
> > web server. However, all of these things had a lot of manual steps
> > to
> > set up and use, and I'm wondering how Debian handles this problem.
> > Here
> > are some of my questions:
> Hello! :-)
> I am not involved with either the ftpmasters team or the wanna-build
> admins, but I did set up infrastructure like this a few times
> already.
> For the last time for the PureOS derivative and for the first time
> for
> the Tanglu derivative where the goal was to explicitly "use what
> Debian uses", so I gained quite a bit of insight into the process.
> Some information I have may be dated by now though, so please correct
> me in these cases.
> 
> > 
> > 1. What software do the official Debian repositories use? Do they
> > use
> > Aptly or reprepro or something else?
> The main Debian repositories use dak, the Debian Archive Kit.
> You can find more information on it at [1] and get its code on Salsa
> at [2].
> Dak was written specifically for Debian's needs and was in the past
> quite Debian-specific with lots of hardcoding of Debianisms (like
> suite names) and expectations on the host environment. This has
> changed quite a bit in the past few years, and while there are still
> a
> bunch of Debian-hardcoded parts, dak is generally useful for
> non-Debian repositories as well.
> Its setup is still orders of magnitudes more complex than using
> reprepro or Aptly (you will need to write some dedicated scripts for
> your distribution), but for huge package repositories it is from my
> experience one of the most performant and painless options. You also
> gain all the features the Debian archive has instantly.
> If your repository is small though, using dak may be overkill - it
> really shines if you have thousands of packages, with just a few
> reprepro could get the job done easier with less manual work.
> 
> > 
> > Is there a downloadable OS image which comes with this pre-set up?
> No, unfortunately not.
> 
> > 
> > Does it run on a cron job or does it
> > have some sort of continuous monitoring? (We have ours run on a
> > cron
> > job every 10 minutes.)
> Dak actions are triggered by multiple cronjobs which run different
> actions. There is one to process incoming uploads which runs roughly
> every 15min, hourly, daily, weekly and monthly cleanup and
> statistics-generating actions as well as the "dinstall" task which
> will run about 4 times a day and publish new packages in the archive
> so users can access them. The dinstall task is actually comprised of
> many individual actions, which deal with different aspects of the
> archive (e.g. translations and AppStream metadata), so summarizing it
> is not that easy.
> In order to not have the autobuilder network wait on publication, dak
> can maintain special queues for the builders to fetch packages from
> prior to them being published officially.
> 
> > 
> > 2. According to https://wiki.debian.org/BuilddSetup, there seems to
> > be
> > a distinction between the build broker (wanna-build) and the build
> > workers (buildd). Do either of these roles have their own OS images
> > one
> > can download?
> AFAIK there are no OS images, but I would bet that the buildd
> machines
> have an Ansible recipe or something similar somewhere, as those are
> continuously updated and refreshed. I would strongly advise against
> using wanna-build for anything - when I tried to use it in the past
> that attempt turned out to be virtually impossible because there was
> no documentation on it and it heavily relied on grown structures
> within Debian itself. If you dig into it, you will also find some
> interesting historical trivia, e.g. apparently in the past an
> autobuilder was building a package and then sending the build log to
> a
> developer, which then looked over it, signed it and submitted it back
> to get the build actually accepted in the archive.
> So, IMHO wanna-build is really not something that should be used in
> new projects...
> 
> > 
> > 3. I understand that source packages are signed by developers
> > before
> > being sent to the build farm, but what about the binary packages
> > bui

Question about Debian build infrastructure

[Kyle Edwards]

Hello all,

I have been preparing Ubuntu releases for CMake on our own APT
repository for several months now. We did this by preparing our own
repository infrastructure - we have a machine that builds packages, and
a machine that hosts an Aptly instance and pushes the repository to our
web server. However, all of these things had a lot of manual steps to
set up and use, and I'm wondering how Debian handles this problem. Here
are some of my questions:

1. What software do the official Debian repositories use? Do they use
Aptly or reprepro or something else? Is there a downloadable OS image
which comes with this pre-set up? Does it run on a cron job or does it
have some sort of continuous monitoring? (We have ours run on a cron
job every 10 minutes.)
2. According to https://wiki.debian.org/BuilddSetup, there seems to be
a distinction between the build broker (wanna-build) and the build
workers (buildd). Do either of these roles have their own OS images one
can download?
3. I understand that source packages are signed by developers before
being sent to the build farm, but what about the binary packages built
by the build farm and uploaded to the repository? Do the build farm
servers have their own GPG keys? Does the repository server recognize
these keys?

Thanks in advance, all this info would be helpful for me as I expand
our Ubuntu build infrastructure.

Kyle

Re: RE:[Idea] Debian User Repository? (Not simply mimicing AUR)

[Kyle Edwards]

On Mon, 2019-04-08 at 16:05 +, PICCA Frederic-Emmanuel wrote:
> now we have the salsa pipeline.
> 
> does it fit your needs ?

Does this allow non-DD's to host packages? Nobody at Kitware is a DD,
we just host an unofficial third-party repository, similar to PPA.

Kyle

Re: [Idea] Debian User Repository? (Not simply mimicing AUR)

[Kyle Edwards]

On Mon, 2019-04-08 at 00:02 -0400, Peter Silva wrote:
> > If one needs to keep a close eye on changes to make sure they can
> > still
> > be installed even on a years-old OS, the resulting packages can be
> > placed in a custom repository set up with the instructions at
> > https://wiki.debian.org/DebianRepository/Setup>;. What am I
> > missing?
> > 
> yes, it can be done, but it is a lot more work for individual
> packagers.
> 
> launchpad.net combines:
>    - very few clicks to build custom repositories.
>    - a build environment for each OS, so that it runs "debuild" in
> the currently patched version of the OS for which the package it
> built.
> 
> It saves people from having to build their own custom repository, and
> from having to maintain a build environment for all supported OS
> versions and architectures.  on Ubuntu,  packages are built for
> 14.04, 16,04, 18.04, 18.10, 19.04, and I get all those just from
> clicking one box for each one. I think it also propagates re-building 
> of packages when a build-dependency changes, without my knowledge or
> interaction.  It leverages the ubuntu build-farm for third-party
> packages.
> 
> With debian, it's kind of all or nothing.  Etiher you're in Debian,
> and it gets built on every platform using the build farm, or it's
> not, so you get no help at all. Launchpad gives a nice middle road
> that suits us right now, and if something similar were available for
> debian, it would provide a stepping stone to being in Debian proper.
CMake and VTK upstream here. This was my exact observation a while back
when I tried to package VTK for Debian and Ubuntu. Packaging for Ubuntu
was very easy: just upload your packaging script to launchpad.net and
you're done. For Debian, I had to set up a build machine and an Aptly
repository on a virtual machine that pulls in built packages from
"incoming", inserts them into the Aptly repo, and rsyncs them to a web
server, basically mimicking what the Debian repo infrastructure does.
This was non-trivial to set up, and I found it frustrating that
Launchpad-like infrastructure did not exist for Debian. (Building lots
of packages is easy once you have the infrastructure, but the initial
investment is rather high.) Just my $0.02.
Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

On Wed, 2018-07-11 at 08:57 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> But no, my real fear here was a tool more in the cpack kind. Years
> ago we had packagers trying to get their stuff in by using cpack.
> While it might be of some help for non official packages it was not
> really fit for official ones.

Yes, the idea here is to make something that is fit for official Debian
packaging.

> Kyle has made it really clear that they intend to support the various
> realities they will probably face if people start using it everywhere
> to create good quality packaging, so yes, I also applaud the
> effort... now ;-)

Thank you Lisandro! I'm glad I was able to address your concerns, and I
think this has been a really good discussion.

Ian, thanks for the feedback, and if you have minor technical questions
please feel free to ask them, either on- or off-list :)

I think it makes sense for dh-cmake to live primarily in VTK for now,
and then, as we work out the issues, hopefully we will see other
packages start to adopt it as well.

I will keep everyone updated as both dh-cmake and the VTK packaging are
developed. We are hoping for an official Debian release of both
packages towards the end of the year.

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

On Tue, 2018-07-10 at 12:52 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> Well, there are cases when upstream is doing things the right way
> with respect to Debian but... what about derivatives (distributions
> which base themselves in Debian)? Sometimes they need something
> different, and even if the package fits perfectly well in Debian it
> might not be true for them. So they need to either patch CMake files
> or re do the whole packaging using traditional tools.

I understand what you're saying. As a concrete example, we all know
that Debian requires *.so library symlinks to live in the -dev package.
But let's say there's a hypothetical Debian derivative that requires
them to live in the library binary package instead.

If upstream has both the headers and the symlink in the "Development"
component, then this would certainly be a problem. Perhaps the solution
is for upstream to make this more flexible, through one of several
options:

1. Further divide the "Development" component into "Headers" and
"Symlinks", allowing downstream to put the symlink in whichever package
is required to meet the distribution's policy. Remember: dh-cmake
allows you to specify *more than one* component per package. In this
case, erring on the side of smaller components would give downstream
the maximum flexibility for grouping them as needed.

2. Make the component names configurable and intelligent through a new,
standardized CMake module, similar to GNUInstallDirs. For example,
there could be new cache variables such as
CMAKE_INSTALL_LIBRARY_COMPONENT, CMAKE_INSTALL_NAMELINK_COMPONENT,
CMAKE_INSTALL_HEADER_COMPONENT, etc. They could be set to defaults that
make sense for the detected distribution, but also configurable to
allow downstream to override them as needed.

I would also add that while this component system might not be as
valuable for smaller, single-library packages, it makes a lot of sense
for VTK. VTK has a module system, with its own internal dependency
graph, that builds and installs dozens of modules, each with their own
headers and CMake files that have to be installed with their respective
shared libraries.

Without the component-based approach taken by dh-cmake, trying to break
up this installation into multiple packages is very difficult. The VTK7
package divides it up into Qt and non-Qt modules (along with docs,
executables, bindings, etc.) To separate Qt and non-Qt modules, it has
to install everything into the non-Qt package, and then manually remove
the Qt modules from it.

But if VTK's module system was updated to install each module into its
own set of components (vtkCommonCore-Libaries, vtkCommonCore-Headers,
etc.), then the debian/*.cpack-components files used by dh-cmake could
simply list these module components, making it very easy to group the
modules together as needed. (Perhaps we could even add a globbing
mechanism so you can just say "vtkCommon*-Libraries" as a shorthand.)
Then, we could further break the VTK package into smaller packages so
you can install some modules without having to install everything.

Now let's say a problem is encountered with the package later on, and
one of the modules needs to be moved into a different package. No
problem - just move the problem module into the correct *.cpack-
components file, and you're done.

If there's another approach we could take to solve this particular
packaging issue, I would love to hear about it, but this is the best
we've come up with so far.

> To sum it up: I *do* think there is a *huge* potential for this
> helper, just not for Debian proper. Of course I would *love* to see
> it packaged in Debian because it will help projects trying to create
> their own Debian packages, but I would expect a very clear
> explanation that this is not a suitable way to maintain packages in
> Debian proper.

In fact, CPack already provides its own method for maintaining 3rd
party Debian packages - it has its own .deb generator. But dh-cmake is
our attempt to make something that fits better into the Debian
workflow, so that it *can* be used to maintain packages in Debian
proper.

We want CMake packaging to be as easy as Python packaging, where you
just activate dh-python. The end goal of dh-cmake is to make CMake
packaging as easy as writing a few configuration files, and then
declaring "dh $@ --buildsystem=cmake --with cmake --with ctest --with
cpack".

In our case, our goal is to maintain an official VTK package in both
Debian proper and Ubuntu proper. VTK is a huge project which has proven
to be difficult to package, and dh-cmake is being created to solve this
problem. We've also made changes to both VTK and CMake itself to
support the VTK packaging effort, and we can and will make more.

> Except we can find smart work arounds for this cases, of course.

I think making the component names configurable and/or standardized, as
I described above, would help tremendously with this.

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

On Fri, 2018-07-06 at 21:00 +0100, Colin Watson wrote:
> If the libraries in question are DFSG-free themselves, there's no
> DFSG issue and you don't need to remove them from the tarball (and
> we'd generally encourage not modifying the upstream tarball
> unnecessarily for upload to Debian).  The policy about bundling is
> separate from the DFSG. Of course it'd be incumbent on whoever's
> doing the Debian upload to actually check the licensing status.

Thank you Colin, this is good to know. In that case, I will investigate
VTK's DFSG issues when I get a chance. If there's something in there
with a licensing issue, then we as upstream would like to fix it.

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

On Thu, 2018-07-05 at 14:04 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> From what you write above I tend to think that simply by not using
> dh-cmake whatever upstream has defined as packaging it will be simply
> ignored (ie, it will become a "standard" CMake project).

Yes, this is true. dh-cmake only adds optional new functionality and
doesn't take anything away from existing packages.

> Now it turns out that you get a bug report where you need to split
> the packaging. It's not an upstream issue per-se, but rather a
> packaging one. Following normal Debian workflow that would mean
> simply creating a new Debian revision.

In our opinion, this actually *is* an upstream issue. To explain why, I
think it would help if I give a brief explanation of the CMake
philosophy.

CMake is intended to be a high-level, cross-platform buildsystem that
lets the end user use whatever tools they like most. Barring hard
technical limitations (such as functionality that is highly specific to
one operating system), if a CMake project is written properly, the
developer and end user should be able to:

* build and run the software on their operating system of choice,
whether it be Windows, Mac, or Linux (among others),
* generate build files for their favorite build tools (Makefiles, Ninja
files, or project files for various IDEs), and
* through the use of CPack, generate installers in whichever format
they desire, including .deb, .rpm, WiX for Windows, and .dmg for Mac.

Expanding on the third point, ideally, upstream would properly split
the project output into components that have all the files in the right
place. Then, the end user can generate well-formed packages in any
format they want with CPack, or, in this case, with dh-cmake in
cooperation with CPack.

If any of these assumptions don't hold true, then the project isn't
utilizing CMake to its full potential, and it can create a poor
experience for the end user if they can't use the tools they feel most
comfortable with. In this case, if a Debian developer can't generate a
package from it using dh-cmake, then we consider it an upstream bug.

> What would happen with dh-cmake? Would be a new upstream release
> required? Would the package maintainer need to patch the packaging
> editing a CMake file?

In a word, yes. See my comment above about this being an upstream
issue.

Of course, many upstream projects won't do things correctly, and the
Debian maintainer will have to do a lot of work no matter what tools
they use. We realize this is unavoidable. In that case, the solution is
to package it as a "traditional" Debian CMake package, without dh-
cmake, the way it's always been done.

This project is meant for the subset of cases where upstream DOES
package it correctly, leaving almost no work for the Debian maintainer
- just add configuration files for dh-cmake and it's ready to go right
away, complete with upstream's dependency graph and project
description.

> [+dfsg] Maybe you can work out to fix this so a repackaging is not
> needed? That would be *really* welcomed.

I have a question about this (and I apologize if this is slightly off
topic): VTK includes "convenience copies" of third-party libraries it
uses to avoid "dependency hell", and also because we've made
modifications to them that haven't yet been upstreamed. I know that on
Debian it's not allowed to *use* these libraries (and we have flags to
use the system versions instead), but do they have to be removed from
the tarball entirely to comply with DFSG?

If the DFSG modifications are because of a non-free file in VTK, we
would certainly like to know about it, and possibly try to fix it. We
don't intend to distribute anything with VTK that's under a non-free,
restrictive license.

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

So, to clarify: we've changed VTK to use GNUInstallDirs, which *itself*
sets the proper directories for Debian, as I will explain below.

On Thu, 2018-07-05 at 15:38 +0100, Simon McVittie wrote:
> debhelper's Debian::Debhelper::BuildSystem::cmake already passes
> -DCMAKE_INSTALL_LIBDIR=lib/$DEB_HOST_MULTIARCH (among other options)
> to packages built using cmake, although for some reason it only does
> this when cross-compiling (this seems sufficiently odd that I've
> reported it as a bug).

The CMAKE_INSTALL_LIBDIR variable comes from the GNUInstallDirs module,
which already sets it to lib/$DEB_HOST_MULTIARCH if it detects a Debian
installation and if CMAKE_INSTALL_PREFIX is set to /usr. Though, as you
have pointed out, CMAKE_INSTALL_LIBDIR can be overridden if so desired.

> The most helpful thing that CMake could do here would be to have a
> predictable set of conventional installation paths, similar to the
> --libdir, --bindir etc. in Autotools and Meson, so that debhelper can
> define the same paths for all CMake-built packages and have the right
> things happen 99% of the time, as it already does for Autotools and
> Meson. If I understand correctly, CMake doesn't *necessarily* provide
> anything like this, but individual CMake-using projects can opt-in to
> it
> by using  tml>?
> (debhelper does set many of those variables, in the hope that the
> project
> being built uses GNUInstallDirs.)

Yes, we've changed VTK to use GNUInstallDirs for this exact reason.
GNUInstallDirs should be considered the standard set of installation
directories for CMake projects.

> In general I think we prefer upstream build systems to do something
> predictable rather than something clever, because we're usually going
> to be passing options to them anyway

I think we're on the same page here. In fact, another predictable thing
we'd like to see upstream buildsystems do is install their libraries in
the "Libraries" or "Runtime" component (via the install() command's
COMPONENT argument) and install the headers and symlinks in the
"Development" component. Then, Debian maintainers can use dh-cmake with
these standard options and have everything go in the correct packages.
Or, if it's a large project with lots of output packages, it could have
more fine-grained components like "example1-Development", "example2-
Development", etc., and then the Debian packaging can take this into
account. (This is exactly what we plan to do with VTK.)

The whole point of this project is to encourage upstream CMake projects
to follow a set of standards that are Debian-friendly (and distro-
friendly in general), so that they "do something predictable."

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

On Wed, 2018-07-04 at 11:30 +0200, Andreas Tille wrote:
> I think you can solve the lintian warning
>   W: dh-cmake source: ancient-python-version-field x-python3-version
> 3.2
> by simply removing
>   X-Python3-Version: >= 3.2
> from d/control.

Thanks for the tip, we will fix this in the next release.

> Manpages for the tools in /usr/bin would be nice.

Will do.

> I'll try with the next cmake package I'll touch.  If you can give
> some examples this would be helpful.

The README.md file has some examples in it, but I'll make a separate
"examples" folder which has examples that can be built out-of-the-box.

Kyle

Re: A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

Hi Lisandro,

Thank you for expressing your concerns. You bring up some very valid
points, and I will try to address all of them here.

On Wed, 2018-07-04 at 14:40 -0300, Lisandro Damián Nicanor Pérez Meyer
wrote:
> I even use it for 99% of my personal/job projects!

Glad to hear it!

> If upstream happens to be the Debian maintainer then *maybe* this
> might be desirable. But if upstream is *not* the Debian maintainer
> then the later must be able to easily override whatever upstream has
> planned as "packaging".

You are correct that this works best if the upstream project has done
its packaging correctly, and/or if the upstream maintainer is also the
Debian maintainer. We would certainly like to see projects use CMake in
a way that makes the life of downstream maintainers easier, and the
install() command's COMPONENT parameter is designed specifically for
this purpose.

If upstream hasn't done its packaging correctly, then we would
certainly advise you to use your own judgment and possibly not use it,
or even try to get your changes upstreamed. (Though, there may be cases
where upstream gets it 99% correct, in which case the solution is
"install the 'Development' component in libexample-dev, then remove
this one file that shouldn't be there.")

In our case, we plan on using this to package VTK. Our plan is to
change VTK's upstream CMake scripts to make it more distro-friendly,
then provide packaging scripts that take advantage of these changes.
(We've already made some of these changes in the latest VTK master - it
now automatically installs its libraries in /usr/lib/ if built as
a Debian package.)

> Debian buildds do not allow network connections. Except maybe if some
> day we deploy something specifically for this.

Not to worry, we've already thought about this. Even if the packaging
turns on the CTest functionality, dh-cmake won't actually try to submit
anything to a server unless the builder/developer gives it explicit
permission to do so, via the DEB_CTEST_OPTIONS environment variable
(this was inspired by DEB_BUILD_OPTIONS). This allows for packages that
simulaneously support two workflows: one for in-house development,
where the build machine gives permission to submit results to CDash,
and one for production (Debian's buildd instances) where it doesn't
submit anything, and just acts as a thin wrapper around the dh_auto_*
commands.

The CTest functionality isn't meant to be turned on in production
buildd instances. It's meant to be used as part of the continuous
integration/nightly build process of an upstream software project,
where a Debian package is one of the supported builds. In our case, we
plan on making Debian one of the supported nightly builds of VTK, and
having this information submitted to CDash is very valuable to us, as
it allows us to spot problems with the Debian build as they occur. VTK
is a very large project that is constantly growing, evolving, and
changing, and supporting it on Debian now and for the rest of the
foreseeable future requires the use of this workflow.

> All that being said discussing details in this list might be
> appropiate. We might find a use for it which suites both sides :-)

Yes, my hope is that we've designed this in a way that works for both
upstream and downstream. I think we've done our best to address all of
your concerns while also supporting upstream's needs.

Thanks again for the feedback!

Kyle

A message from CMake upstream: announcing dh-cmake

[Kyle Edwards]

Hello everyone!

My name is Kyle. I work at Kitware, Inc., the upstream maintainer of
the CMake buildsystem (https://www.cmake.org/) and VTK, the
Visualization Toolkit (https://www.vtk.org/). As some of you on the
Debian Science list may have heard, we are making an effort to
officially support our flagship product, VTK, on Debian and its
derivatives. To that end, we have created a new set of Debhelper
utilities to meet the unique challenges of packaging a large CMake-
based project like VTK. We have named this project "dh-cmake". It
allows Debhelper to take advantage of some of the more advanced
capabilities of CMake. For example:

* CMake's install() command takes an optional COMPONENT parameter,
  which allows you to break the installation up into multiple
  "components", such as "Libraries" and "Development". dh-cmake allows
  you to assign these components to separate binary packages, to avoid
  having to enumerate every file or file glob in the *.install files.
* Projects that are CTest-aware can optionally have the output of
  dh_auto_configure, dh_auto_build, and dh_auto_test captured by CTest
  and submitted to a CDash server as part of a continuous integration
  process. This is very useful for making sure a large software project
  builds properly on Debian.
* CPack includes a mechanism to declare dependencies between
  installation components, for example, stating that the "Development"
  component depends on the "Libraries" component. dh-cmake can
  propagate this information into the output packages, so that
  libexample-dev will automatically depend on libexample.

You can download the source code at
https://gitlab.kitware.com/debian/dh-cmake, and read more details about
the rationale and how it works. You can also install the binaries from
our own APT repository. Follow the instructions at
https://apt.kitware.com/ to set up the repository, and then install the
"dh-cmake" package.

Our end goal is to get both dh-cmake and VTK into Debian proper, but it
is still in an experimental state, and there is still a lot of work to
be done yet. We would like to get some feedback on dh-cmake, and we
will eventually file a formal ITP and RFS for it as it becomes more
mature. We would also like to see other CMake-based packages follow our
lead and use these utilities. If you have a package that uses CMake, we
encourage you to give dh-cmake a try.

Thank you in advance for the feedback. We are very excited to venture
into Debian development.

Kyle