Bug#1042947: UDD: create a duck importer
Hi Lucas,
On Wed, 2023-10-25 at 20:57 +0200, Lucas Nussbaum wrote:
> On 08/08/23 at 06:42 +0200, Lucas Nussbaum wrote:
> > Hi Baptiste,
> >
> > On 07/08/23 at 22:07 +0200, Baptiste Beauplat wrote:
> > > Hi Lucas,
> > >
> > > On 2023-08-03 10:30, Lucas Nussbaum wrote:
> > > > duck-as-a-service (duck.debian.net) has been broken for a long
> > > > time,
> > > > and
> > > > the corresponding UDD importer is broken as well (see #949009,
> > > > #963887).
> > > > In the meantime, duck continued evolving (was rewritten?) and
> > > > is now
> > > > checking a lot more places for URLs.
> > > >
> > > > It would probably be useful to re-create a way to provide duck
> > > > results
> > > > as a service, based on UDD, similarly to what is done for
> > > > upstream or
> > > > lintian data.
> > > >
> > > > Ideally, this would be done in cooperation with the duck
> > > > maintainer
> > > > to
> > > > do the following changes:
> > > > - in duck, separate the logic to get URLs from sources, from
> > > > the
> > > > logic
> > > > to check those URLs (for example, allow dumping a list of
> > > > URLs, and
> > > > also using a list of URLs as source)
> > > > - in duck, provide machine-readable outputs (JSON?)
> > >
> > > Currently duck has two features which can help us:
> > >
> > > - The `-n` switch, which gets all URLs and prints them to stdout
> > > - The `-l filename` switch, which takes a file with one URL per
> > > line
> > > and checks them
> > >
> > > Theoretically, what's missing in only a `--json` switch, which
> > > would
> > > change the output from console/text to JSON.
> > >
> > > But, as I see it, the `-l` argument is limited in two aspects:
> > >
> > > - It provides only the URL, loosing the checker type which is
> > > used to
> > > select what kind of validation will be performed.
> > >
> > > For instance, a https://salsa.debian.org/rfrancoise/tmux.git of
> > > type
> > > VCS-Git would be tested as a standard URL in the `-l` context,
> > > instead
> > > of a git repository.
> > >
> > > - It requires a file
> > >
> > > I'm thinking of implementing a new JSON specific input format
> > > (`--input-json`?), including the two information, which would
> > > read from
> > > stdout instead of a file.
> > >
> > > The format would be as simple as:
> > >
> > > ```json
> > > [
> > > {"type": "VCS-Git",
> > > "url": "https://salsa.debian.org/rfrancoise/tmux.git;,
> > > "filename": "debian/control", # optional key
> > > "line_number": 10}, # optional key
> > > ...
> > > ]
> > > ```
> > >
> > > Following this logic, the output format for checking URLs would
> > > be the
> > > same, as to have `duck --json -n | duck --input-json` working.
> > >
> > > The JSON result would hold an additional dictionary for each URL
> > > entries
> > > named "result", described as follows:
> > >
> > > ```json
> > > [
> > > {"type": "VCS-Git",
> > > "url": "https://salsa.debian.org/rfrancoise/tmux.git;,
> > > "filename": "debian/control", # optional key
> > > "line_number": 10, # optional key
> > > "result": {
> > > "state": 0, # 0 for OK, 1 for Error, 2 for Information
> > > "detail": "Informative message",
> > > "certainty": "possible" # optional key
> > > }},
> > > ...
> > > ]
> > > ```
> > >
> > > Let me know what you think of it.
> >
> > That would be perfect!
> >
> > In the context of UDD, I will probably implement that as two
> > tables:
> > - one to store the mapping between source packages and urls
> > (source, version, url, type, filename, line_number)
> > which would be updated when a new source version gets uploaded
> > - one to store the status of urls
> > (url, type, result, timestamp of last check)
> > which would be updated with a retry policy to be defined
> >
> > I would not use (filename, line_number) in the input of the URL
> > testing part.
> > The reason for that design is that it will easily allow to gather
> > the
> > status for several versions of the package (testing + unstable +
> > experimental for example), while not duplicating the checks for
> > URLs.
>
> Just checking: did you make progress on this?
Sort of.
I could not see a clean way to add this feature without a total rewrite
of duck. So that's what I've started, and I'm making steady progress on
that front.
However, I have not started working on the json interface
implementation just yet.
I'll keep you posted once I have a working version of that.
Best,
--
Baptiste Beauplat
signature.asc
Description: This is a digitally signed message part
Bug#1042947: UDD: create a duck importer
Hi Lucas,
On 2023-08-03 10:30, Lucas Nussbaum wrote:
> duck-as-a-service (duck.debian.net) has been broken for a long time,
> and
> the corresponding UDD importer is broken as well (see #949009,
> #963887).
> In the meantime, duck continued evolving (was rewritten?) and is now
> checking a lot more places for URLs.
>
> It would probably be useful to re-create a way to provide duck
> results
> as a service, based on UDD, similarly to what is done for upstream or
> lintian data.
>
> Ideally, this would be done in cooperation with the duck maintainer
> to
> do the following changes:
> - in duck, separate the logic to get URLs from sources, from the
> logic
> to check those URLs (for example, allow dumping a list of URLs, and
> also using a list of URLs as source)
> - in duck, provide machine-readable outputs (JSON?)
Currently duck has two features which can help us:
- The `-n` switch, which gets all URLs and prints them to stdout
- The `-l filename` switch, which takes a file with one URL per line
and checks them
Theoretically, what's missing in only a `--json` switch, which would
change the output from console/text to JSON.
But, as I see it, the `-l` argument is limited in two aspects:
- It provides only the URL, loosing the checker type which is used to
select what kind of validation will be performed.
For instance, a https://salsa.debian.org/rfrancoise/tmux.git of type
VCS-Git would be tested as a standard URL in the `-l` context, instead
of a git repository.
- It requires a file
I'm thinking of implementing a new JSON specific input format
(`--input-json`?), including the two information, which would read from
stdout instead of a file.
The format would be as simple as:
```json
[
{"type": "VCS-Git",
"url": "https://salsa.debian.org/rfrancoise/tmux.git;,
"filename": "debian/control", # optional key
"line_number": 10}, # optional key
...
]
```
Following this logic, the output format for checking URLs would be the
same, as to have `duck --json -n | duck --input-json` working.
The JSON result would hold an additional dictionary for each URL
entries
named "result", described as follows:
```json
[
{"type": "VCS-Git",
"url": "https://salsa.debian.org/rfrancoise/tmux.git;,
"filename": "debian/control", # optional key
"line_number": 10, # optional key
"result": {
"state": 0, # 0 for OK, 1 for Error, 2 for Information
"detail": "Informative message",
"certainty": "possible" # optional key
}},
...
]
```
Let me know what you think of it.
> Then UDD could process source packages to extract URLs, check those
> URLs
> on a regular basis (similarly to what is done for lintian), and
> publish/export the results in all relevant places.
Best,
--
Baptiste Beauplat
signature.asc
Description: This is a digitally signed message part
Bug#963887: UDD: 'duck' importer broken since 2020-05-25
Hi Lucas, On Thu, 2023-08-03 at 10:38 +0200, Lucas Nussbaum wrote: > I submitted #1042947 to discuss re-creating a UDD duck importer, > using the same model as the lintian importer. > > @Baptiste: could you take a look? There would be a few changes on the > duck side that would make it much easier. Sure, I'll have a look before next week. Best, -- Baptiste Beauplat signature.asc Description: This is a digitally signed message part
Re: Package with "wrong" homepage
On 2021/03/08 08:25 PM, Jelmer Vernooij wrote: > On Mon, Mar 08, 2021 at 09:17:15PM +0100, Davide Prina wrote: > > I will try to propose a new check to improve Debian Quality :-) > > > > I'm using repology (https://repology.org) to report packages with home page > > not work anymore and where I found a possible new home page. > > But a lot of what I'm doing can be automatized. > > > > When upstream abandon a home page, the Debian link can be used: > > * by attackers to build a fake home page > > * by person to register the old home page to have a lot of referrals from a > > lot of GNU/Linux distro, for something totally different > > * ... > > > > but also, if upstream change to a new one can cause: > > * outdated software in repository > > * software based on outdated libraries > > * software that seem not be maintained upstream > > * removed software from Debian repository for the previous motivations > > when, probably, in the new upstream there is the solution of all these > > problems. > > > > For example, for Debian testing, you can see what packages have home page > > problems: > > https://repology.org/repository/debian_testing/problems > > > > most have point to the htpp URI that is redirect to the htpps one, but a lot > > do not respond anymore or have other problems (for example point to a not > > more maintained repository, for example goolge code; there are also some > > case when not all the certificate chain is validated or similar issues). > > > > If you open for a package the repology detail you can see which distro are > > using with version (note: I see that sometime different distro use the same > > package name for different upstream software): > > https://repology.org/project/jansi-native/versions > > > > and if you go to the information tab > > https://repology.org/project/jansi-native/information > > > > you can see in the "Homepage links" section what home page link all the > > distro are using; where a number is in green so that URI is working and must > > be checked to know if it is the new home page of that Debian package. > > > > So, for example, in PTS (or in a bug report) can be reported to the DD: > > 1) that the package home page has some problem > > 2) a possible solution (in the repology page above) > > > > For all packages that there isn't a possible solution can be created a list > > (in the wiki, for example) and ask user help to find if there is a new home > > page. > > > > I wish that can be a good suggestion for the qa team. > > > > I think that repology can also be used for other checks. > > The Debian Janitor (through lintian-brush/upstream-ontologist) looks > at repology (as well as other data sources) to determine the Homepage field > for Debian packages where it is missing. > > Today, janitor/lintian-brush only sets the homepage field when it > is not set - it doesn't remove the homepage field when it is missing. > I was hoping to rely on duck (https://duck.debian.net/) to detect when > the Homepage field has gone bad, but it looks like duck is no longer > maintainer :( While the website is indeed down, one can still use duck as a standalone program to highlight dead urls in a source package. Granted that this is not helping an archive wide QA check. There was some talk about reviving duck.debian.net as part of distro-tracker [1][2], unfortunatly, I haven't managed to free enough time to start working on it (being busy on other projects ATM). [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=963887 [2]: https://salsa.debian.org/qa/distro-tracker/-/issues/51 -- Baptiste Beauplat - lyknode signature.asc Description: PGP signature
Re: Intent to orphan packages with unreachable maintainer address
Small update on those packages:
On 2021/01/17 09:53 PM, Mattia Rizzolo wrote:
> Hello,
>
> On Tue, Jan 12, 2021 at 09:43:20PM +0100, Baptiste Beauplat wrote:
> > While discussing on #debian-qa how to handle those bugs on packages
> > without reachable maintainer, Mattia suggested to simply orphan those,
> > without the need of going through the MIA process. Just to be sure of
> > what I'm doing, I'm writing this mail as an heads-up.
>
> Let me elaborate a bit while commenting on the list.
> In particular, for DDs and DMs, IMHO it's fine to "just orphan" in some
> cases, and the MIA team will eventually pick up the rest of the work.
>
> > Al Nikolov
>
> Already went through a MIA process in 2016, but guess he needs another.
> Added to the list, feel free to orphan his packages in the meantime.
orphaned,
> > Alberto Furia
>
> Feel free to orphan both of his packages.
orphaned,
> > Andy Shevchenko
orphaned,
> > Angel Ramos
>
> Already in a low-prio MIA list, go orphan the packages.
orphaned,
> > Antonio Ospite
>
> This one is active in other packages with the email a...@ao2.it (assuming
> it's the same person by the same name). Should probably forward the
> kboot-utils bug to him.
forwarded, with a reply. It will most likely end-up with an RM.
> > C. Chad Wallace
> >liblingua-en-namecase-perl
>
> I asked the perl team if they want the package, otherwise I'd propose
> removal (not maintaed for 12 years, low popcon).
> gregoa said he is going to take the package.
adopted,
> > Carlos Alberto Silombria Ibarra
> >maildirsync
>
> one past upload was done by a person of the same name using
> silver...@gmail.com - perhaps one could try to that address.
> If we can't be bothered, orphan it away.
orphaned,
> > Chris Grzegorczyk
> >geronimo-ejb-3.0-spec
> >geronimo-j2ee-connector-1.5-spec
> >geronimo-jms-1.1-spec
>
> This one has a tad too many packages on his name (9) which makes me
> slighly unconfortable in just removing him, otoh that was 10+ years ago,
> so meh (to do this cleanly we should also drop him from Uploaders of
> some java packages)
forwared to the Java team,
> > Daniel Jared Dominguez
> >wsl
>
> When I noticed that other packages used "jared.doming...@dell.com" I
> though that was a typo, but a quick telnet to dell.com's MX revelead
> that also is "user unknown", so whatever.
>
> I can't quickly spot other mail addresses for him, so we should orphan
> them all (including uploaders).
wsl orphaned, request to remove as uploaders sent to the EFI team (for
all other packages),
> > Dmitry Borisyuk
> >fbxkb
>
> one single upload 6 years ago → orphan.
orphaned,
> > Graziano Obertelli
> >geronimo-ejb-3.0-spec (U)
> >geronimo-j2ee-connector-1.5-spec (U)
> >geronimo-jms-1.1-spec (U)
>
> I now notice that's a pretty similar set of packages of
> g...@eucalyptus.com - which let me notice that the domain is gone.
> I'd ask the java team if they are interested in those not yet in the
> team, else orphan.
forwarded to the Java Team,
> > gustavo panizzo
> >numad (U)
>
> That's a DD: g...@debian.org - just ask forward the bug to him.
Yeah, that's my bad, I didn't filter out Uploaders from dd-list output
of the packages with an unreachable maintainers.
adopted by gfa,
> > Hamish Moffatt
> >extra-xdg-menus (U)
>
> This DD is already retired, and indeed MIA already handled him, that's
> apparently the last package left with his name: https://bugs.debian.org/831474
orphaned,
> > Jeremías Casteglione
> >numad
>
> should probably ask gfa to take it over, since he is in Uploaders.
adopted by gfa,
> > Juliane Holzt
> >gwhois
>
> That's a DD. They have been inactive enough, I've added them to MIA's
> list. Feel free to orphan the package though.
orhpaned,
> > Julián Moreno Patiño
> >john (U)
>
> Last upload of this DD was Feb 2019, which kind of would make the cut
> for MIA. I'm not adding him just yet, but perhaps somebody could try to
> send a mail to their other address arkju...@gmail.com.
Same as gfa, Julián should have not ended-up on this list.
> > Kamal Mostafa
> >numad (U)
>
> Already in a low-prio MIA list, though their last activity is actually
> nearer than the last person.
> They are quite active in launchpad, so likely just mail them on any
> other of their addresses (kamal@{canonical.com,debian.org}).
adopted by gfa,
> > Kyo Lee
> >geronimo-ejb-3.0-spec (U)
> >geronimo-j2ee-connector-1.5-spec (U)
> >
Bug#980949: O: wbox -- HTTP testing tool and configuration-less HTTP server
Package: wnpp The current maintainer of wbox, Alberto Furia , is apparently not active anymore. Therefore, I orphan this package now. Maintaining a package requires time and skills. Please only adopt this package if you will have enough time and attention to work on it. If you want to be the new maintainer, please see https://www.debian.org/devel/wnpp/#howto-o for detailed instructions how to adopt a package properly. Some information about this package: Package: wbox Binary: wbox Version: 5-1 Maintainer: Alberto Furia Build-Depends: debhelper (>= 7), dpatch Architecture: any Standards-Version: 3.8.3 Format: 1.0 Files: 39212ed0d6f433f9a7557250f9c34b70 932 wbox_5-1.dsc a95ca2c69982db10704b5ed482c9c722 16465 wbox_5.orig.tar.gz 04852a510d31c699bf5fc9b47beed653 6080 wbox_5-1.diff.gz Checksums-Sha256: d9ce54c32b3cdbaa6e7c15f6f86416e1c28ce7a5d8be915eb65fb8fd61ce246e 932 wbox_5-1.dsc 1589d85e83c8ee78383a491d89e768ab9aab9f433c5f5e035cfb5eed17efaa19 16465 wbox_5.orig.tar.gz 69cbea56623ddd744923c6f1fa1b304749747d4207b1656b3de87ab57d592efc 6080 wbox_5-1.diff.gz Homepage: http://www.hping.org/wbox/ Directory: pool/main/w/wbox Priority: source Section: web Package: wbox Source: wbox (5-1) Version: 5-1+b2 Installed-Size: 54 Maintainer: Alberto Furia Architecture: amd64 Depends: libc6 (>= 2.14) Description: HTTP testing tool and configuration-less HTTP server Description-md5: d4e227ddc208e8ac7a1306a287a6eeb8 Homepage: http://www.hping.org/wbox/ Tag: implemented-in::c, role::program Section: web Priority: optional Filename: pool/main/w/wbox/wbox_5-1+b2_amd64.deb Size: 21452 MD5sum: 0837ee534c0279262dc8ce5f2e93fdf1 SHA256: 15ac083b4573582848dc168651871129a9e8709131181d9cae5afb1c3118f783 -- Baptiste Beauplat - lyknode signature.asc Description: PGP signature
Bug#980948: O: archmbox -- a simple email archiver written in perl
Package: wnpp The current maintainer of archmbox, Alberto Furia , is apparently not active anymore. Therefore, I orphan this package now. Maintaining a package requires time and skills. Please only adopt this package if you will have enough time and attention to work on it. If you want to be the new maintainer, please see https://www.debian.org/devel/wnpp/#howto-o for detailed instructions how to adopt a package properly. Some information about this package: Package: archmbox Binary: archmbox Version: 4.10.0-2.1 Maintainer: Alberto Furia Build-Depends: debhelper (>= 5.0.0) Build-Depends-Indep: perl, psmisc, bzip2, uw-mailutils Architecture: all Standards-Version: 3.8.3 Format: 1.0 Files: bdd442520e9317007ccb3d3f733bbb6f 1756 archmbox_4.10.0-2.1.dsc 44f9f155d45d2eae4b9de33314adf841 82370 archmbox_4.10.0.orig.tar.gz e27302880e3d9669201ae89d94cd5986 25788 archmbox_4.10.0-2.1.diff.gz Checksums-Sha256: 4def4a5146aca01bb6ee3df18f29e4141cf1b9a6c6aee8c5044c85001a5cf17f 1756 archmbox_4.10.0-2.1.dsc 0c0ec8bf6340dbb625306e5c129abe50c070abb3d84e00ab2559cdfbc366f329 82370 archmbox_4.10.0.orig.tar.gz 2346ece15b2b4c6bb0b4795eb8b72bedccaa3e3e3c80fb377a0f02a2d84bde51 25788 archmbox_4.10.0-2.1.diff.gz Homepage: http://adc-archmbox.sourceforge.net/ Package-List: archmbox deb mail optional arch=all Directory: pool/main/a/archmbox Priority: source Section: mail Package: archmbox Version: 4.10.0-2.1 Installed-Size: 83 Maintainer: Alberto Furia Architecture: all Depends: perl, psmisc, bzip2, uw-mailutils Description: a simple email archiver written in perl Description-md5: aea0792059a321f0195e8aaee4d7d4c2 Homepage: http://adc-archmbox.sourceforge.net/ Tag: admin::backup, implemented-in::perl, interface::commandline, role::program, scope::utility, use::compressing, use::storing, works-with::mail Section: mail Priority: optional Filename: pool/main/a/archmbox/archmbox_4.10.0-2.1_all.deb Size: 35228 MD5sum: e3742ac03ae28d24a575ff794366e352 SHA256: ea6d8b1b051809a63b21bd98734382a51a8c5196d620fb4e0906990ce43ca09d -- Baptiste Beauplat - lyknode signature.asc Description: PGP signature
Re: Intent to orphan packages with unreachable maintainer address
On 1/13/21 1:17 AM, Steve Langasek wrote: > On Tue, Jan 12, 2021 at 09:43:20PM +0100, Baptiste Beauplat wrote: >> Hi, >> >> Following Holger's no-change source-only uploads to have all binaries >> built on buildd [1], a couple of RC bugs have been opened because >> Maintainers email's are unreachable. >> >> While discussing on #debian-qa how to handle those bugs on packages >> without reachable maintainer, Mattia suggested to simply orphan those, >> without the need of going through the MIA process. Just to be sure of >> what I'm doing, I'm writing this mail as an heads-up. >> >> After carefully reviewing all bugs, I've extracted all packages that, I >> believe, qualify for being Orphaned, based on: >> >> - Maintainer email unreachable >> - No substitute address found > > Kamal Mostafa is a DD: > > https://nm.debian.org/person/kamal/ > > So that seems like a substitute address it should've been possible to find? Indeed he is, but he is "only" listed as Uploaders and his email address isn't part of the ones marked as unreachable. That being said, thank you for pointing this out because I made a pretty big oversight. Indeed, as pointed out by Mattia, packages belonging to DD or DM have to be excluded from this list. Those do have to go through the MIA team. So I'm adding the following condition: - Maintainer/Uploaders is no DD or DM And regenerating the list (cut by about half the size). >> - Last maintainer upload is over 900 days >> - Not part of a Team >> - No active Uploaders >> >> I've compile all that data into the attached file packages-data.csv. >> >> The final list of source package that would be orphaned is available in >> the attached file dd-list.txt >> >> The MIA Team might want to have look at maintainers that have other >> packages (Other packages column in the CSV). >> >> Let me know if I shouldn't process with orphaning those packages, >> otherwise, I'll do so in a week. >> >> [1]: https://lists.debian.org/debian-devel/2020/12/msg00419.html -- Baptiste Beauplat - lyknode Alberto Furia archmbox Andy Shevchenko renrot Antonio Ospite kboot-utils C. Chad Wallace liblingua-en-namecase-perl Carlos Alberto Silombria Ibarra maildirsync Chris Grzegorczyk geronimo-ejb-3.0-spec geronimo-j2ee-connector-1.5-spec geronimo-jms-1.1-spec Daniel Jared Dominguez wsl Dmitry Borisyuk fbxkb Graziano Obertelli geronimo-ejb-3.0-spec (U) geronimo-j2ee-connector-1.5-spec (U) geronimo-jms-1.1-spec (U) Kyo Lee geronimo-ejb-3.0-spec (U) geronimo-j2ee-connector-1.5-spec (U) geronimo-jms-1.1-spec (U) Leo Iannacone apt-venv Valerie R Young blag-fortune Vedran Furač aspell-hr Source;Maintainer;Uploaders;RC Bug;Reason;Popcorn;Total bugs;RC bugs;Time since last maintainer upload;Time since last maintainer upload (days);Other packages;Comment apt-venv;Leo Iannacone ;;979347;Unknown user;31;3;1;2014-08-23;2334;60;MIA since 2016 archmbox;Alberto Furia ;;979435;Unknown user;21;3;1;2009-11-23;4068;1; aspell-hr;Vedran Furač ;;979051;Unknown user;435;3;1;2006-10-10;5208;0; blag-fortune;Valerie R Young ;;979530;Mailbox full;176;3;1;2017-08-07;1254;0; fbxkb;Dmitry Borisyuk ;;978624;Unknown user;195;5;1;2014-11-05;2260;0; geronimo-ejb-3.0-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979222;NXFAILED;144;2;1;2010-01-03;4027;10;Also affects Kyo Lee , Both MIA since 2010 geronimo-j2ee-connector-1.5-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979058;NXFAILED;597;2;1;2010-01-29;4001;10; geronimo-jms-1.1-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979056;NXFAILED;1289;4;1;2010-01-03;4027;10; kboot-utils;Antonio Ospite ;;979641;Unknown user;2;1;1;2013-12-14;2586;0; liblingua-en-namecase-perl;C. Chad Wallace ;;979502;Unknown user;17;2;1;2008-03-11;4690;0; maildirsync;Carlos Alberto Silombria Ibarra ;;979345;Unknown user;39;3;1;2015-03-03;2142;0; renrot;Andy Shevchenko ;;979218;Relay error;116;4;1;2009-01-04;4391;0; wsl;Daniel Jared Dominguez ;;979513;Unknown user;16;1;1;2012-11-17;2978;0; OpenPGP_signature Description: OpenPGP digital signature
Intent to orphan packages with unreachable maintainer address
Hi, Following Holger's no-change source-only uploads to have all binaries built on buildd [1], a couple of RC bugs have been opened because Maintainers email's are unreachable. While discussing on #debian-qa how to handle those bugs on packages without reachable maintainer, Mattia suggested to simply orphan those, without the need of going through the MIA process. Just to be sure of what I'm doing, I'm writing this mail as an heads-up. After carefully reviewing all bugs, I've extracted all packages that, I believe, qualify for being Orphaned, based on: - Maintainer email unreachable - No substitute address found - Last maintainer upload is over 900 days - Not part of a Team - No active Uploaders I've compile all that data into the attached file packages-data.csv. The final list of source package that would be orphaned is available in the attached file dd-list.txt The MIA Team might want to have look at maintainers that have other packages (Other packages column in the CSV). Let me know if I shouldn't process with orphaning those packages, otherwise, I'll do so in a week. [1]: https://lists.debian.org/debian-devel/2020/12/msg00419.html -- Baptiste Beauplat - lyknode Al Nikolov goban libdbix-class-htmlwidget-perl Alberto Furia archmbox Andy Shevchenko renrot Angel Ramos cvsdelta Antonio Ospite kboot-utils C. Chad Wallace liblingua-en-namecase-perl Carlos Alberto Silombria Ibarra maildirsync Chris Grzegorczyk geronimo-ejb-3.0-spec geronimo-j2ee-connector-1.5-spec geronimo-jms-1.1-spec Daniel Jared Dominguez wsl Dmitry Borisyuk fbxkb Graziano Obertelli geronimo-ejb-3.0-spec (U) geronimo-j2ee-connector-1.5-spec (U) geronimo-jms-1.1-spec (U) gustavo panizzo numad (U) Hamish Moffatt extra-xdg-menus (U) Jeremías Casteglione numad Juliane Holzt gwhois Julián Moreno Patiño john (U) Kamal Mostafa numad (U) Kyo Lee geronimo-ejb-3.0-spec (U) geronimo-j2ee-connector-1.5-spec (U) geronimo-jms-1.1-spec (U) Leo Iannacone apt-venv Peter Clifton extra-xdg-menus extra-xdg-menus (U) Petr Baranov libloki Robert James Clay crashmail Ruben Molina john Valerie R Young blag-fortune Vedran Furač aspell-hr Source;Maintainer;Uploaders;RC Bug;Reason;Popcorn;Total bugs;RC bugs;Time since last maintainer upload;Time since last maintainer upload (days);Other packages;Comment apt-venv;Leo Iannacone ;;979347;Unknown user;31;3;1;2014-08-23;2334;60;MIA since 2016 archmbox;Alberto Furia ;;979435;Unknown user;21;3;1;2009-11-23;4068;1; aspell-hr;Vedran Furač ;;979051;Unknown user;435;3;1;2006-10-10;5208;0; blag-fortune;Valerie R Young ;;979530;Mailbox full;176;3;1;2017-08-07;1254;0; crashmail;Robert James Clay ;;976832;Relay error;7;3;1;2018-07-19;908;31;Last upload on feb 2019 cvsdelta;Angel Ramos ;;979428;No MX;28;2;1;2009-05-08;4267;3;MIA since 2009 extra-xdg-menus;Peter Clifton ;Peter Clifton , Hamish Moffatt ;979055;Unknown user;4965;8;1;2008-02-09;4721;1;Inactive DD as uploader (last upload 2010) fbxkb;Dmitry Borisyuk ;;978624;Unknown user;195;5;1;2014-11-05;2260;0; geronimo-ejb-3.0-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979222;NXFAILED;144;2;1;2010-01-03;4027;10;Also affects Kyo Lee , Both MIA since 2010 geronimo-j2ee-connector-1.5-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979058;NXFAILED;597;2;1;2010-01-29;4001;10; geronimo-jms-1.1-spec;Chris Grzegorczyk ;Graziano Obertelli , Kyo Lee ;979056;NXFAILED;1289;4;1;2010-01-03;4027;10; goban;Al Nikolov ;;979875;Timeout;29;1;1;2017-06-08;1314;7;Last upload jan 2019 gwhois;Juliane Holzt ;;979221;Unknown user;162;13;1;2015-04-17;2097;1; john;Ruben Molina ;Julián Moreno Patiño ;979054;Unknown user;2527;24;1;2013-09-05;2686;5; kboot-utils;Antonio Ospite ;;979641;Unknown user;2;1;1;2013-12-14;2586;0; libdbix-class-htmlwidget-perl;Al Nikolov ;;979529;Timeout;9;1;1;2017-06-08;1314;7; liblingua-en-namecase-perl;C. Chad Wallace ;;979502;Unknown user;17;2;1;2008-03-11;4690;0; libloki;Petr Baranov ;;979357;Unknown user;32;4;1;2011-07-03;3481;0; maildirsync;Carlos Alberto Silombria Ibarra ;;979345;Unknown user;39;3;1;2015-03-03;2142;0; numad;Jeremías Casteglione ;gustavo panizzo , Kamal Mostafa ;976328;Unknown user;154;2;1;2016-12-26;1478;0; renrot;Andy Shevchenko ;;979218;Relay error;116;4;1;2009-01-04;4391;0; wsl;Daniel Jared Dominguez ;;979513;Unknown user;16;1;1;2012-11-17;2978;0; OpenPGP_signature Description: OpenPGP digital signature
Bug#939527: vcdimager contains one file with different license
Control: owner -1 ! On Thu, 05 Sep 2019 21:42:27 + bendik...@vfemail.net wrote: > https://metadata.ftp-master.debian.org/changelogs//main/v/vcdimager/vcdimager_2.0.1+dfsg-3_copyright > > The file changed on 17-03-2011 from gpl 2 or later to gpl 2 > https://cvs.savannah.gnu.org/viewvc/vcdimager/vcdimager/lib/sector.c?view=markup > > Already send a message to (not visible at the moment) > https://lists.gnu.org/mailman/listinfo/bug-vcdimager/ > also > https://directory.fsf.org/wiki/User_talk:Rockyb > more info > https://directory.fsf.org/wiki/vcdimager I asked upstream and apparently the change was requested by one of its author. They don't seems to want to change it back. See https://github.com/rocky/vcdimager/issues/5 For Debian, we just need to fix the copyright listing. I'll get on it. -- Baptiste Beauplat - lyknode OpenPGP_signature Description: OpenPGP digital signature
Bug#966297: UDD: mentors.debian.net schema change
Hi Lucas, The API is in production on mentors. I've worked a bit on udd and came up with a proposal: https://salsa.debian.org/qa/udd/-/merge_requests/29 Best, -- Baptiste BEAUPLAT - lyknode signature.asc Description: OpenPGP digital signature
Bug#966297: UDD: mentors.debian.net schema change
On 8/15/20 10:54 AM, Lucas Nussbaum wrote: > On 01/08/20 at 14:07 +0200, Baptiste BEAUPLAT wrote: >> On 8/1/20 9:01 AM, Lucas Nussbaum wrote: >>> Hi Baptiste, >>> >>> I think that it would be better if mentors.d.n would provide a JSON >>> export of its data that it useful for UDD. >> >> Would a JSON rest api work? > > Sure, provided it doesn't require looping over packages or uploads. Yeah. In this proposal [1], I've added a standalone /api/uploads/ route, listing all uploads with some additional hints (package name instead of just the id), in order for scripts not to have to fetch resources everywhere. [1]: https://salsa.debian.org/mentors.debian.net-team/debexpo/-/merge_requests/146 -- Baptiste BEAUPLAT - lyknode signature.asc Description: OpenPGP digital signature
Bug#966297: UDD: mentors.debian.net schema change
On 8/1/20 9:01 AM, Lucas Nussbaum wrote: > Hi Baptiste, > > I think that it would be better if mentors.d.n would provide a JSON > export of its data that it useful for UDD. Would a JSON rest api work? UDD could fetch something like https://mentors.debian.net/api/uploads/ and retrieve all information needed. > Regarding data that should be replicated in UDD, I would prefer > something simpler that is enough, typically for DMD, to notify that > there's a new version waiting, but that still requires going to > mentors.d.n for details. I wonder if we should have more than one table, > that just describes the last upload for a given source or (source,user) > (it's not clear to me if several users can simultaneously upload > competing versions of the same source package). Yes, several users can upload the same package using different version and distributions. From what I can see in the DMD page, it fetches from UDD's tables: - package name - version - distribution - uploaded date From that, distribution is actually unused. So fetching the list of all uploads and storing that into a single table sould be sufficient. -- Baptiste BEAUPLAT - lyknode signature.asc Description: OpenPGP digital signature
Bug#963887: UDD: 'duck' importer broken since 2020-05-25
On 6/30/20 9:29 AM, Mattia Rizzolo wrote: > Just a note before you head toward implementing that: the Homepage field > is similar to Section, in the way that it can also be specified in the > binary paragraphs, not just the source paragraphs. > You can see that as the Homepage field is present in the DEBIAN binary > control field of the .debs, and clearly that value might be different > than the one in Homepage of the .dsc. > > So please, look harder for Homepage, not just in the first paragraph of > d/control ;) A good list of places to look can be found in: https://salsa.debian.org/debian/duck/-/tree/master/lib/checks -- Baptiste BEAUPLAT - lyknode signature.asc Description: OpenPGP digital signature
Bug#963887: UDD: 'duck' importer broken since 2020-05-25
On 6/29/20 11:34 PM, Raphael Hertzog wrote: > On Mon, 29 Jun 2020, Baptiste BEAUPLAT wrote: >>> Indeed, creating a dedicated service for this does not seem a good idea. >> >> I would love to have this feature integrated directly with >> distro-tracker. However, I'm wondering about the load that would case >> for the service. > > Network request do not generate much "load", such processes spend the bulk > of their time waiting on the network. True that. >> The duck worker has to process around 46 urls (only counting >> Homepage) in less than 24h. > > How do you get to that figure? We don't have that many source package > and even if you consider multiple URL for each source package due to > changes over time (in multiple releases), that makes way too many URLs > per source package. Err, sorry about that. That figure is the result of: $ curl -s http://deb.debian.org/debian/dists/unstable/main/source/Sources.gz | zgrep -v Homepage: | sort -u | wc -l 458804 Which is obviously wrong. Here is the real number: $ curl -s http://deb.debian.org/debian/dists/unstable/main/source/Sources.gz | zgrep Homepage: | sort -u | wc -l 26250 >> I'm not sure that can done properly using >> the distro-tracker tasks (parallel workers are needed to work around >> timeout). Obviously that can be optimized (different check delay for >> different results) but that's still bulk network related tasks. > > Nothing forbids parallel workers and in any case, I welcome any > improvement to the task mechanism to make that kind of parallelism easier > to handle. > > There are other tasks that could benefit from this (and in general I want > to merge more of such features in distro-tracker to make them available to > derivatives too). Then, let's add this to distro-tracker :) I've created an issue on the project on salsa so we can discuss technical details : https://salsa.debian.org/qa/distro-tracker/-/issues/51 As I've said before, I would like to finish up on a couple of other projects (namely mentors.d.n and snapshot.d.o) and I will be available right after that. Best, -- Baptiste BEAUPLAT - lyknode signature.asc Description: OpenPGP digital signature
Bug#963887: UDD: 'duck' importer broken since 2020-05-25
Hi Bastian, Raphael,
On 6/29/20 3:55 PM, Raphael Hertzog wrote:
> On Sun, 28 Jun 2020, Bastian Blank wrote:
>>> Baptiste (CCed) volunteered to write it over again, but for now there is
>>> no clear timeline as for when the new project will be started.
>>
>> Maybe you could add that to vcswatch?
The main differences between vcswatch and duck.d.n are:
- history: duck used to keep 6 runs for each package, reporting only
after 3 failures. vcswatch only keeps the last run.
- d/control: duck processed the Homepage as well as the
Vcs-{Git,SVN,Hg,Darcs} fields. vcswatch has a wider support for all Vcs-*.
- d/upstream/metadata: duck processed any urls found here.
- worker: parallel worker for duck, single instance for vcswatch.
(sorry if I got anything wrong here. Please correct me!)
I'm not convinced that adding those features would result in an
improvement for vcswatch (Cc'ing Christoph to have his input on that).
Creating a new sub-project, like vcswatch, to qa.debian.org would be
more sensible IMHO. The new duck could only take care of the http urls
and leave Vcs stuff to vcswatch.
> or distro-tracker?
>
> Indeed, creating a dedicated service for this does not seem a good idea.
I would love to have this feature integrated directly with
distro-tracker. However, I'm wondering about the load that would case
for the service.
The duck worker has to process around 46 urls (only counting
Homepage) in less than 24h. I'm not sure that can done properly using
the distro-tracker tasks (parallel workers are needed to work around
timeout). Obviously that can be optimized (different check delay for
different results) but that's still bulk network related tasks.
Another thing is that duck.d.n was delegating the actual checks to the
`duck` perl library. To work with distro-tracker I would need to drop
that an implement something silimar in python. Not a huge task per se,
but something to keep in mind.
I'm not sure what is best here and I'm looking forward to your
suggestions and remarks.
--
Baptiste BEAUPLAT - lyknode
signature.asc
Description: OpenPGP digital signature
