Re: Third-party forks of packaged projects
On Wed, 2020-04-29 at 10:21 +0100, Alastair McKinstry wrote:
> I think vendoring libraries that are only used by this package is
> fine.
>
> I would however put them in a "public" place and namespace (eg
> /usr/lib/$ARCH/libatl.* ) rather than a subdir or namespace
> ('libatl-adios') so that there is a clear collision if someone else
> attempts to package it; in which case it can be 'unvendored' and
> shared.
Thanks Alastair - these libraries are already in the public namespace
and named libadios2_atl.so, etc. so as not to conflict with an
unvendored version. If we decide to unvendor them later on, it won't be
difficult at all.
Kyle
Re: Third-party forks of packaged projects
On 28/04/2020 21:44, Kyle Edwards wrote:
> On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote:
>> I'd say that it depends, and that it should be addressed on the
>> case-by-case basis.
> I do have another scenario I'd like to address. ADIOS uses a stack of
> closely related but separate projects, all developed by Greg
> Eisenhauer, which, as far as I know, are not used by any other major
> software project. ADIOS is the main driver behind them. The projects
> are the following:
>
> https://github.com/GTkorvo/atl
> https://github.com/GTkorvo/dill
> https://github.com/GTkorvo/EVPath
> https://github.com/GTkorvo/ffs
>
> ADIOS vendors these projects as well. Are these reasonable to be
> packaged with ADIOS, or should they be packaged separately?
I think vendoring libraries that are only used by this package is fine.
I would however put them in a "public" place and namespace (eg
/usr/lib/$ARCH/libatl.* ) rather than a subdir or namespace
('libatl-adios') so that there is a clear collision if someone else
attempts to package it; in which case it can be 'unvendored' and shared.
> Kyle
Regards
Alastair
--
Alastair McKinstry, , ,
https://diaspora.sceal.ie/u/amckinstry
Misentropy: doubting that the Universe is becoming more disordered.
Re: Third-party forks of packaged projects
On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote: > I'd say that it depends, and that it should be addressed on the > case-by-case basis. I do have another scenario I'd like to address. ADIOS uses a stack of closely related but separate projects, all developed by Greg Eisenhauer, which, as far as I know, are not used by any other major software project. ADIOS is the main driver behind them. The projects are the following: https://github.com/GTkorvo/atl https://github.com/GTkorvo/dill https://github.com/GTkorvo/EVPath https://github.com/GTkorvo/ffs ADIOS vendors these projects as well. Are these reasonable to be packaged with ADIOS, or should they be packaged separately? Kyle
Re: Third-party forks of packaged projects
On 4/27/20 10:34 PM, Kyle Edwards wrote: > On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote: >> If the library you need to package isn't needed by any other package, >> simply apply the needed patch and upload. Even better if it's only a >> build-dependency, in which case it wont ever be a problem. > > This brings up an interesting question: are things that are build- > dependencies only covered by the no-vendoring rule? Of course yes. Otherwise, we would leave all Go stuff vendored and never care, for example. > In my case, I *am* upstream. So, for example, we forked enet and made > some changes to how it works. In that case, I could make a new package > called adios2-enet or ornl-enet, and depend on it for ADIOS. Is that > correct? That's IMO the recommended way, yes. Thomas Goirand (zigo)
Re: Third-party forks of packaged projects
On Sat, 2020-04-25 at 20:31 +0200, Thomas Goirand wrote: > If the library you need to package isn't needed by any other package, > simply apply the needed patch and upload. Even better if it's only a > build-dependency, in which case it wont ever be a problem. This brings up an interesting question: are things that are build- dependencies only covered by the no-vendoring rule? For example, RapidJSON appears to be header-only (rapidjson-dev). If ADIOS ships with its own copy of RapidJSON, does it still have to use the one provided by Debian even though it will have little to no effect on the final package? > The other possibility (which is probably the best way forward) is to > convince your upstream that he did a fork, and that it should be > renamed > to avoid conflicts. In my case, I *am* upstream. So, for example, we forked enet and made some changes to how it works. In that case, I could make a new package called adios2-enet or ornl-enet, and depend on it for ADIOS. Is that correct? Kyle
Re: Third-party forks of packaged projects
On 4/24/20 11:28 PM, Kyle Edwards wrote: > Hello, > > I have a question about how Debian handles modifications to third-party > dependencies. Sometimes a project relies on another project, but has > made modifications to that project that never went into upstream, > either because upstream has abandoned the project or because the > changes are not appropriate for upstream. In that case, the depending > project "vendors" the third-party dependency with the modifications > that it needs. > > Obviously, "vendored" dependencies are a no-go in Debian, but how do > situations like this get resolved? I'm imagining the modified project > could be packaged on its own the way any fork would - in that manner, > it would not be much different from packaging MariaDB even though a > package for MySQL already exists. Is my intuition correct here? > > Kyle I'd say that it depends, and that it should be addressed on the case-by-case basis. If the library you need to package isn't needed by any other package, simply apply the needed patch and upload. Even better if it's only a build-dependency, in which case it wont ever be a problem. I did this for python3-antlr3 which was needed at build time only by Congress, and which I don't think any other package will ever need in Debian. If some day, a package will need python3-antlr3, then probably we can make 2 different binaries conflicting each other. Python3-antlr3 is only a build dependency, so it's really not a problem if we need to add such conflicts: statement. But we could generalize this even for libraries needed at runtime, analyzing if there's a big chance or not that 2 programs will need different flavor of the same library. The other possibility (which is probably the best way forward) is to convince your upstream that he did a fork, and that it should be renamed to avoid conflicts. Cheers, Thomas Goirand (zigo)
Re: Third-party forks of packaged projects
On Fri, Apr 24, 2020 at 10:16 PM Kyle Edwards wrote: > I did not realize that exceptions were occasionally made for vendored > libraries There is no exception for vendored libraries, but occasionally people just upload without doing the work needed to find them or deliberately ignoring the rules etc. Consequently there are a ton of embedded code copies in Debian, probably many more than even the current giant list of registered embedded code copies indicates. https://wiki.debian.org/EmbeddedCopies -- bye, pabs https://wiki.debian.org/PaulWise
Re: Third-party forks of packaged projects
On Fri, Apr 24, 2020 at 9:29 PM Kyle Edwards wrote: > I have a question about how Debian handles modifications to third-party > dependencies. Sometimes a project relies on another project, but has > made modifications to that project that never went into upstream, > either because upstream has abandoned the project or because the > changes are not appropriate for upstream. In that case, the depending > project "vendors" the third-party dependency with the modifications > that it needs. > > Obviously, "vendored" dependencies are a no-go in Debian, but how do > situations like this get resolved? I'm imagining the modified project > could be packaged on its own the way any fork would - in that manner, > it would not be much different from packaging MariaDB even though a > package for MySQL already exists. Is my intuition correct here? It is easiest to discuss this if you provide specifics rather than hypothetical scenarios. The best option would be that the upstream project comes alive again; either by the previously inactive maintainers, or by the people depending on it taking it over, or forking it if taking it over isn't possible. Once that happens the downstream modifications from projects that depend on it or from the distros can be merged. -- bye, pabs https://wiki.debian.org/PaulWise
Re: Third-party forks of packaged projects
On Sat, 2020-04-25 at 00:02 +0200, Mattia Rizzolo wrote: > If the upstream maintainer of that library is not available anymore > and > the project is clearly dormient, perhaps you can take over > officially? > Or if that patch is "acceptable" just leave it on the bug tracker, > and > within debian that patch could be applied, so that at least downstram > we > can strip off that vendored library. That's cleraly possible only if > that patch doesn't break stuff, etc. Mattia, Thanks for the insight. Two years ago I encountered this issue with VTK. We use a modified version of libharu, and submitted our patches upstream, but upstream has not been active in 5 years. I attempted to submit our patches to Debian, but was told it would be better if we forked it outright and made our own release. In the end, I decided it wasn't worth the effort. I am now encountering a similar issue with ADIOS2 (https://github.com/o rnladios/ADIOS2). We use a modified version of enet (https://github.com /GTkorvo/enet), and from what I understand, we can't use the upstream version, so this is a situation where either enet would have to be vendored with ADIOS or we would have to package our fork. > vendored dependencies are not really a strict no-go. cases like what > you describe are one reason to keep them vendored within a package, > and > the security team tries keep a list of such vendored libraries, but > since few maintainers reports back changes in this area, that list is > not really that good (reason not to vendor libraries!!). I did not realize that exceptions were occasionally made for vendored libraries - I thought the policy was "if the dependency can't be externalized, then it can't be used"... though I do recall a recent discussion about Docker/k8s having this issue. Kyle
Re: Third-party forks of packaged projects
On Fri, Apr 24, 2020 at 05:28:51PM -0400, Kyle Edwards wrote: > In that case, the depending > project "vendors" the third-party dependency with the modifications > that it needs. Which is always horrible for us. If you have any power, please don't do it. Rather find a way to monkey-patch whatever dependency function you are using or whatever is possible in your case. If the upstream maintainer of that library is not available anymore and the project is clearly dormient, perhaps you can take over officially? Or if that patch is "acceptable" just leave it on the bug tracker, and within debian that patch could be applied, so that at least downstram we can strip off that vendored library. That's cleraly possible only if that patch doesn't break stuff, etc. > Obviously, "vendored" dependencies are a no-go in Debian, but how do > situations like this get resolved? vendored dependencies are not really a strict no-go. cases like what you describe are one reason to keep them vendored within a package, and the security team tries keep a list of such vendored libraries, but since few maintainers reports back changes in this area, that list is not really that good (reason not to vendor libraries!!). > I'm imagining the modified project > could be packaged on its own the way any fork would - in that manner, > it would not be much different from packaging MariaDB even though a > package for MySQL already exists. Is my intuition correct here? pacakaging that as a fork it's clearly possible, but it's much more work, and usually doesn't make sense since that vendorerd patches libraries likely won't be used by anything else, and would be updated only together with the "main" software, so separating them is not really useful. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. More about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Third-party forks of packaged projects
Hello, I have a question about how Debian handles modifications to third-party dependencies. Sometimes a project relies on another project, but has made modifications to that project that never went into upstream, either because upstream has abandoned the project or because the changes are not appropriate for upstream. In that case, the depending project "vendors" the third-party dependency with the modifications that it needs. Obviously, "vendored" dependencies are a no-go in Debian, but how do situations like this get resolved? I'm imagining the modified project could be packaged on its own the way any fork would - in that manner, it would not be much different from packaging MariaDB even though a package for MySQL already exists. Is my intuition correct here? Kyle
