Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi Guillem,

On Wed, Aug 7, 2019 at 8:03 AM Guillem Jover  wrote:
>
> In the lintian case, I'm not sure there's really a need? ISTM that most
> of the independence and verifiability relies more on the testing data,
> and how any such parser is being driven than the parser itself?

You are right. Your comment actually illustrates Lintian's
transformation, over twenty years, from a format checker (the parser)
to a maintainer checker (whose input is the testing data).

I have had to rethink Lintian's relationship with Dpkg on several
occasions. For example, several tags became untestable because of the
advancement of your tools. [1] [2] [3] [4] [5] [6] Lintian is no
longer a testsuite for packaging tools like Dpkg (I am not sure there
are others) but rather an expert system rendering packaging advice for
the benefit of maintainers.

My desire to stay independent was rooted in the outdated concept, and
wrong-headed. I will convert Lintian to use Dpkg::Changelog::Debian.
Thanks to intrigeri for sharing his development branch!

> I'm not sure what you find lacking in the current Dpkg::Changelog
> implementation, even though it is too a direct successor to the
> Parse::DebianChangelog modules which got imported into libdpkg-perl
> many years ago [C] as the foundation for the new code.

Some of my comments may have been based on older versions. I
discovered later that someone (presumably you) made substantial
improvements. My apologies.

> But I'm happy to improve the current interface, implementation or
> documentation to make its use easier/better.

Thank you for your offer. I will make suggestions as long as they
appear useful to you and others.

> I'm interested in this, and would like to hear more concrete issues
> with the current interface.

I can probably implement the planned data structures (based on Moo)
with your module as a parser backend. The point is therefore moot. I
look forward to working with Dpkg::Changelog::Debian!

Kind regards,
Felix

[1] 
https://salsa.debian.org/lintian/lintian/commit/ff6a5fef762146027dfa295f73503476fe76535f
[2] 
https://salsa.debian.org/lintian/lintian/commit/9ae1abd7d3dac631707a4c46badba3086608e5b9
[3] 
https://salsa.debian.org/lintian/lintian/commit/4f019cb2c1d754cdcb1c601f34b85d5015b2d90d
[4] 
https://salsa.debian.org/lintian/lintian/commit/769117ae8083cbedc61cfd08660aa7e823021beb
[5] 
https://salsa.debian.org/lintian/lintian/commit/00bd64fe3876da0b26d4b85308c3bf350dd97acb
[6] 
https://salsa.debian.org/lintian/lintian/commit/e01a8f86890a2390e8eded2ae7bda02b7f855636

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Guillem Jover]

Hi!

On Wed, 2019-07-31 at 09:56:31 -0700, Felix Lechner wrote:
> On Wed, Jul 31, 2019 at 8:29 AM intrigeri  wrote:
> >  - Your MR introduces yet another debian/changelog parser.
> >
> > Given the trouble we had with keeping Parse::DebianChangelog somewhat
> > working and up-to-date so far, I would argue that relying on existing,
> > well-maintained Dpkg::* code, is a safer approach on the long term.
> 
> Having converted the Lintian test suite from a custom approach to
> TAP::Parser, I share your sentiment about using established, and
> tested, third-party packages. I am more tolerant, however, when
> duplicating efforts related to Debian---such as analyzing the
> changelog. Lintian is supposed to be an independent critic, and there
> is a value to, well, being independent.

I also agree and support the existence of independent implementations
whenever they make sense, so that the specified format gets exercised
by various implementations and can uncover ambiguities or issues in
the spec itself. But these also incur a cost, when we need to extend
the format itself for example.

In the lintian case, I'm not sure there's really a need? ISTM that most
of the independence and verifiability relies more on the testing data,
and how any such parser is being driven than the parser itself?

> The changelog parser is also a relatively small piece of code.

Right, although all these small pieces add up. :)

> A more significant issue is that Dpkg::Changelog::Debian may not offer
> the kind of data structures I hope to offer to writers of Lintian
> checks. At least, I felt that way when implementing centralized
> version parsing as part of the $info data structure for source
> packages (MR in preparation) pursuant to this message:

I'm not sure what you find lacking in the current Dpkg::Changelog
implementation, even though it is too a direct successor to the
Parse::DebianChangelog modules which got imported into libdpkg-perl
many years ago [C] as the foundation for the new code.

  [C] 


But I'm happy to improve the current interface, implementation or
documentation to make its use easier/better.

> > But it requires more initial work because the API differs slightly,
> > and more importantly, the Dpkg::Changelog::Debian parser is stricter,
> > so we lose some granularity wrt. error reporting (some syntax errors
> > Lintian would previously catch itself later on are now reported as
> > syntax errors, as soon as we parse the changelog file).
> 
> I am not worried about the amount of work, but about its outcome: As
> you acknowledge, Dpkg::Changelog::Debian's lack of granularity may
> make it harder to implement distinct tags. That results less detailed
> advice on how to improve packages, and is less customer-friendly.

I'm interested in this, and would like to hear more concrete issues
with the current interface.

Thanks,
Guillem

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi intrigeri,

On Wed, Jul 31, 2019 at 8:29 AM intrigeri  wrote:
>
>I've pushed my current WIP to
>https://salsa.debian.org/intrigeri/lintian/tree/bug933134.

Your branch is now in my repo at
https://salsa.debian.org/lechner-guest/lintian/tree/bug933134.

>I'm not sure if/when I'll be able to resume work on it,
>so please don't block on me.

Please delete your copy when you do not need it anymore.

Kind regards,
Felix

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi,

On Wed, Jul 31, 2019 at 8:29 AM intrigeri  wrote:
>
>I've pushed my current WIP to
>https://salsa.debian.org/intrigeri/lintian/tree/bug933134.
>I'm not sure if/when I'll be able to resume work on it,
>so please don't block on me.

Thank you. I will start with your code when we decide to go with
Dpkg::Changelog::Debian.

>  - Your MR introduces yet another debian/changelog parser.
>
> Given the trouble we had with keeping Parse::DebianChangelog somewhat
> working and up-to-date so far, I would argue that relying on existing,
> well-maintained Dpkg::* code, is a safer approach on the long term.

Having converted the Lintian test suite from a custom approach to
TAP::Parser, I share your sentiment about using established, and
tested, third-party packages. I am more tolerant, however, when
duplicating efforts related to Debian---such as analyzing the
changelog. Lintian is supposed to be an independent critic, and there
is a value to, well, being independent.

The changelog parser is also a relatively small piece of code.

A more significant issue is that Dpkg::Changelog::Debian may not offer
the kind of data structures I hope to offer to writers of Lintian
checks. At least, I felt that way when implementing centralized
version parsing as part of the $info data structure for source
packages (MR in preparation) pursuant to this message:

https://lists.debian.org/debian-lint-maint/2019/07/msg00141.html

and based on this code:

https://salsa.debian.org/lintian/lintian/blob/master/checks/source-changelog.pm#L28-84

If it helps us with a decision, I could submit that MR first. Or,
Chris could accept my merge request !234 for now. We could look at
alternatives later. That would immediately resolve #933134, which
prevents the removal of libparse-debianchangelog-perl. (From your bug
filing, it sounded like a priority for the Perl team.) Lintian would
be off no worse than before: We have an old parser that does not use
Dpkg::Changelog::Debian. :)

This last course of action is my favorite. I have so many changes in
the pipeline, it would be far easier for me to adapt Lintian to
Dpkg::Changelog::Debian later than to rebase my branches. I promise to
undertake the integration work, if necessary. Would that be agreeable
to you?

> But it requires more initial work because the API differs slightly,
> and more importantly, the Dpkg::Changelog::Debian parser is stricter,
> so we lose some granularity wrt. error reporting (some syntax errors
> Lintian would previously catch itself later on are now reported as
> syntax errors, as soon as we parse the changelog file).

I am not worried about the amount of work, but about its outcome: As
you acknowledge, Dpkg::Changelog::Debian's lack of granularity may
make it harder to implement distinct tags. That results less detailed
advice on how to improve packages, and is less customer-friendly.

Kind regards,
Felix

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[intrigeri]

Hi,

Felix Lechner:
> https://salsa.debian.org/lintian/lintian/merge_requests/234

OK, thanks for sharing. The main differences between our attempts is:

 - My own WIP ports the code from Parse::DebianChangelog to
   Dpkg::Changelog::Debian (from libdpkg-perl), which should be kept
   up-to-date by the dpkg maintainers wrt. debian/changelog
   format changes.

   I've pushed my current WIP to
   https://salsa.debian.org/intrigeri/lintian/tree/bug933134.
   I'm not sure if/when I'll be able to resume work on it,
   so please don't block on me.

 - Your MR introduces yet another debian/changelog parser.

Given the trouble we had with keeping Parse::DebianChangelog somewhat
working and up-to-date so far, I would argue that relying on existing,
well-maintained Dpkg::* code, is a safer approach on the long term.
But it requires more initial work because the API differs slightly,
and more importantly, the Dpkg::Changelog::Debian parser is stricter,
so we lose some granularity wrt. error reporting (some syntax errors
Lintian would previously catch itself later on are now reported as
syntax errors, as soon as we parse the changelog file).

Cheers,
-- 
intrigeri

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi intrigeri,

On Tue, Jul 30, 2019 at 6:16 AM intrigeri  wrote:
>
> I have WIP locally that also migrates Lintian away from
> Parse::DebianChangelog. I did not finish it yet but if it helps,
> I could push my current branch somewhere.

I knew Lintian would be high on your list and got to work immediately:

https://salsa.debian.org/lintian/lintian/merge_requests/234

I plan more improvements and will share them when appropriate. We
could also cooperate on making a new library available to others.

Kind regards,
Felix Lechner

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[intrigeri]

Chris Lamb:
> Felix Lechner wrote:

>> After you filed this bug I migrated Lintian away from that library.
>> With some luck, it will be in master soon---and well before the
>> bullseye freeze.

> I still see a "Depends" on libparse-debianchangelog-perl. By "migrated
> Lintian" away does this infer that this is in a local branch or
> perhaps did this change get lost somewhere?

Interesting! I have WIP locally that also migrates Lintian away from
Parse::DebianChangelog. I did not finish it yet but if it helps,
I could push my current branch somewhere.

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi Chris,
.
On Tue, Jul 30, 2019 at 5:46 AM Chris Lamb  wrote:
>
> is in a local branch or
> perhaps did this change get lost somewhere?

You don't have it yet. It works, but I am contemplating better
parsing, possibly using Parser::MGC, and changes to the API. Coming in
the next few days.

Kind regards,
Felix

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Chris Lamb]

Felix Lechner wrote:

> After you filed this bug I migrated Lintian away from that library.
> With some luck, it will be in master soon---and well before the
> bullseye freeze.

I still see a "Depends" on libparse-debianchangelog-perl. By "migrated
Lintian" away does this infer that this is in a local branch or
perhaps did this change get lost somewhere?


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org 🍥 chris-lamb.co.uk
   `-

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[Felix Lechner]

Hi intrigeri,

On Fri, Jul 26, 2019 at 1:45 PM  wrote:
>
> lintian depends, or build-depends, on libparse-debianchangelog-perl,
> which has had no upstream maintainer since 2011.

After you filed this bug I migrated Lintian away from that library.
With some luck, it will be in master soon---and well before the
bullseye freeze.

> see https://bugs.debian.org/933128, where we can discuss our options.

Our solution does not presently use Dpkg::Changelog::Debian. If it
continues to work well, I will post a reference to it on your main
bug.

Kind regards
Felix Lechner

Bug#933134: lintian: depends on libparse-debianchangelog-perl that has no upstream maintainer

[intrigeri]

Package: lintian
Severity: important
User: debian-p...@lists.debian.org
Usertags: libparse-debian-changelog-perl-removal

Hi,

lintian depends, or build-depends, on libparse-debianchangelog-perl,
which has had no upstream maintainer since 2011.

The Debian Perl group has decided today that we don't want
libparse-debianchangelog-perl to be included in the Bullseye release,
unless someone steps up and volunteers to be its upstream maintainer:
see https://bugs.debian.org/933128, where we can discuss our options.

Cheers, 
-- 
intrigeri, for the Debian Perl group