Re: Q to all candidates: NEW queue
Lucas Nussbaum writes ("Re: Q to all candidates: NEW queue"):
> So in my original mail, I proposed that new packages would get
> immediately accepted into unstable, but would still require a review
> before migrating to testing. I believe that it's an interesting compromise,
> because:
> - while in unstable, they would get tested by our regular QA tools, that
> are likely to find some of the issues ftpmasters would have found
> - it makes it possible for the maintainer to get early feedback from
> users, and to continue working on packaging reverse dependencies.
> - it's unstable, so even if it's severely broken, it's probably not a
> big deal. We have lots of packages in unstable that have been severely
> broken for years anyway.
> - it protects 'testing' (and our stable releases) from unreviewed
> packages.
Because we build everything on unstable, it does not protect testing
from .debs which were built using un-reviewed packages, does it ?
In general maybe we shouldn't be migrating X.deb if its build-info
says it was built using (a version of) Y which isn't in testing.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Q to all candidates: NEW queue
On 26/03/20 1:03 pm, Lucas Nussbaum wrote: > Hi, > > For as long as I can remember, there has been complaints about the > delays caused by NEW processing (and > https://ftp-master.debian.org/stat.html shows that we constantly have > 250-300 packages waiting to be processed). > > What is your diagnostic of this issue? > What solutions do you envision about this issue? Is that just something > that we have to live with? Delays in NEW processing is something I am concerned personally. I assume manpower is the main issue in these delays. One thing is sure, more people we have looking into NEW queue, faster the process would be. The call for FTP-trainees which happened recently is a good start (I have applied to be an FTP-trainee). I would suggest that there should be regular induction of new people to FTP-team and growing the team so that there is no shortage of active people working on NEW queue at any time. I am also aware of some alternative solutions/technical workflows being proposed for NEW processing in -devel. I do not have any clear cut solution as of now, but as DPL I would definitely encourage discussions on this topic and come up a solution acceptable for both FTP-team and package maintainers. > > Specifically, Jonathan writes that he would like to "Reduce bottlenecks > that affect our contributors.". That sounds like a good example. > > > Personnally, I wonder if we are being overly cautious about NEW. I > wonder if we could move to checking a posteriori (accept the package in > unstable; maybe don't let it migrate to testing until it is reviewed). I personally had a couple of packages which had some overlooked copyright issues that FTP-team pointed out. I do believe the work done by the team is important for the project, circumventing that may not be a good option. As I mentioned earlier, we should come up with solving manpower issues and ways to simplify the work of FTP-team instead. > Lucas > signature.asc Description: OpenPGP digital signature
Re: Q to all candidates: NEW queue
On Sat, Mar 28, 2020 at 09:50:41AM +, Simon McVittie wrote: > On Fri, 27 Mar 2020 at 23:06:55 +, Holger Levsen wrote: > > another option would be 'unstable-proposed' (or whatever) where packages get > > uploaded to, and which only gets moved to 'unstable' if they don't fail > > piuparts, > > autopkgtests (plain build tests) and so forth... > Do you mean this to be for all package, like Ubuntu does (they run the > unstable->testing migration with a 0-day delay, all packages get uploaded > into the equivalent of unstable, and users of their development branch > are actually getting the equivalent of testing), or do you mean just for > source-NEW and/or binary-NEW packages? for all, including *-NEW. why should ftpmaster/anyone look at a package which fails automated test? (well, anyone except the maintainers of said package.) -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
On Fri, 27 Mar 2020 at 23:06:55 +, Holger Levsen wrote: > another option would be 'unstable-proposed' (or whatever) where packages get > uploaded to, and which only gets moved to 'unstable' if they don't fail > piuparts, > autopkgtests (plain build tests) and so forth... Do you mean this to be for all package, like Ubuntu does (they run the unstable->testing migration with a 0-day delay, all packages get uploaded into the equivalent of unstable, and users of their development branch are actually getting the equivalent of testing), or do you mean just for source-NEW and/or binary-NEW packages? smcv
Re: Q to all candidates: NEW queue
hi, another option would be 'unstable-proposed' (or whatever) where packages get uploaded to, and which only gets moved to 'unstable' if they don't fail piuparts, autopkgtests (plain build tests) and so forth... humans shouldn't look at stuff robots don't wanna see. -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
On 27/03/20 at 08:18 -0700, Sean Whitton wrote: > Hello, > > On Fri 27 Mar 2020 at 09:39AM +01, Ulrike Uhlig wrote: > > > My point was to ask if there are any points in this list that could be > > harmful in the scenario proposed by Lucas. > > We try to stop packages of sufficiently low quality entering the archive > at all, so that other contributors don't have to deal with understanding > what's going on with those low quality packages when trying to fix other > stuff. > > I would say that Lucas' proposal would not be able to achieve that. That's true, however, I think that the question is whether we value this filter so much that we agree that it's OK to cause a delay of weeks or months for good-quality packages to enter unstable. My personal opinion is that I would rather have no delay or a much smaller delay before the package gets accepted in 'unstable', and then deal with quality issues the way we deal with them for other packages, by detecting them using QA checks, and by filing (RC) bugs appropriately. It's true that low-quality packages in unstable are annoying when doing QA work, but on the other hand, it's quite easy to ignore them by looking only at packages that are also in testing (that's what I do for archive rebuilds, where I ignore packages in unstable when I don't have much time; and I think I stole that idea from someone else, so others have been doing it too). We already have 365 packages in unstable whose last appearance in testing was before the buster release, and among those, 81 which were last in testing before the stretch release. So this problem of low-quality packages in unstable-only needs to be dealt with anyway. Lucas signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
Hello Martin, On Fri 27 Mar 2020 at 12:23PM +01, Martin Pitt wrote: > At least during my many years of Ubuntu archive administration I've actually > seen quite a lot of packages which contained non-distributable files, had > hilariously broken maintainer scripts (which could then also damage *other* > software on your system), and the like. For these an initial NEW review was > quite important. Indeed. > @ftpmasters, would it help to try some automation on the 80% case, and e. g. > auto-process packages if they are lintian-clean, suspicious-source is empty, > and checking for some reasonable overlap of licensecheck and grep -i '(c)' > with > names appearing in debian/copyright? So that ftpmasters can concentrate on the > 20% complicated packages? > > Or are the 80% already not a problem/time sink? They're not a problem. They only get stuck for too long sometimes because the code which chooses which order to display the packages to us sometimes puts them much lower down in the queue than you'd expect them to me, for reasons I don't yet understand. -- Sean Whitton signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
Hello, On Fri 27 Mar 2020 at 09:39AM +01, Ulrike Uhlig wrote: > My point was to ask if there are any points in this list that could be > harmful in the scenario proposed by Lucas. We try to stop packages of sufficiently low quality entering the archive at all, so that other contributors don't have to deal with understanding what's going on with those low quality packages when trying to fix other stuff. I would say that Lucas' proposal would not be able to achieve that. -- Sean Whitton signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
On Friday, March 27, 2020 9:37:28 AM EDT Lucas Nussbaum wrote: > On 27/03/20 at 09:23 -0400, Scott Kitterman wrote: > > On Friday, March 27, 2020 8:40:11 AM EDT Lucas Nussbaum wrote: > > > On 27/03/20 at 12:23 +0100, Martin Pitt wrote: > > > > At least during my many years of Ubuntu archive administration I've > > > > actually seen quite a lot of packages which contained > > > > non-distributable > > > > files, had hilariously broken maintainer scripts (which could then > > > > also > > > > damage *other* software on your system), and the like. For these an > > > > initial NEW review was quite important. > > > > > > > > That proposal is assuming that the "package gets reviewed, a bug is > > > > filed" > > > > step actually happens timely, but that is precisely the problem -- > > > > with > > > > such a workflow we would essentially stop having NEW review and just > > > > hope > > > > that someone catches bad packages before they get released. So IMHO > > > > this > > > > is not a solution, and only causes buggy packages to creep into > > > > unstable. > > > > > > So in my original mail, I proposed that new packages would get > > > immediately accepted into unstable, but would still require a review > > > before migrating to testing. I believe that it's an interesting > > > compromise, > > > because: > > > - while in unstable, they would get tested by our regular QA tools, that > > > > > > are likely to find some of the issues ftpmasters would have found > > > > > > - it makes it possible for the maintainer to get early feedback from > > > > > > users, and to continue working on packaging reverse dependencies. > > > > > > - it's unstable, so even if it's severely broken, it's probably not a > > > > > > big deal. We have lots of packages in unstable that have been severely > > > broken for years anyway. > > > > > > - it protects 'testing' (and our stable releases) from unreviewed > > > > > > packages. > > > > > > Of course this only works if Debian doesn't get sued for copyright > > > infringement too often. I wonder if that would be a problem (it's > > > probably less likely to be a problem for packages in 'main' than for > > > packages in 'non-free'). > > > > > > Lucas > > > > What's "too often"? > > I don't know. Has it happened in the past? How frequently does ftpmaster > run into things that would/could trigger a lawsuit? I'm not aware of it ever happening and I think that's the acceptable frequency. Such lawsuits are insanely expensive to defend. I don't know how often it happens, it's not like we track it that way. We did catch one really high risk package this month and it wasn't the code that was risky, it was the data. So it happens. Scott K signature.asc Description: This is a digitally signed message part.
Re: Q to all candidates: NEW queue
On 27/03/20 at 09:23 -0400, Scott Kitterman wrote: > On Friday, March 27, 2020 8:40:11 AM EDT Lucas Nussbaum wrote: > > On 27/03/20 at 12:23 +0100, Martin Pitt wrote: > > > At least during my many years of Ubuntu archive administration I've > > > actually seen quite a lot of packages which contained non-distributable > > > files, had hilariously broken maintainer scripts (which could then also > > > damage *other* software on your system), and the like. For these an > > > initial NEW review was quite important. > > > > > > That proposal is assuming that the "package gets reviewed, a bug is filed" > > > step actually happens timely, but that is precisely the problem -- with > > > such a workflow we would essentially stop having NEW review and just hope > > > that someone catches bad packages before they get released. So IMHO this > > > is not a solution, and only causes buggy packages to creep into unstable. > > > > So in my original mail, I proposed that new packages would get > > immediately accepted into unstable, but would still require a review > > before migrating to testing. I believe that it's an interesting compromise, > > because: > > - while in unstable, they would get tested by our regular QA tools, that > > are likely to find some of the issues ftpmasters would have found > > - it makes it possible for the maintainer to get early feedback from > > users, and to continue working on packaging reverse dependencies. > > - it's unstable, so even if it's severely broken, it's probably not a > > big deal. We have lots of packages in unstable that have been severely > > broken for years anyway. > > - it protects 'testing' (and our stable releases) from unreviewed > > packages. > > > > Of course this only works if Debian doesn't get sued for copyright > > infringement too often. I wonder if that would be a problem (it's > > probably less likely to be a problem for packages in 'main' than for > > packages in 'non-free'). > > > > Lucas > > What's "too often"? I don't know. Has it happened in the past? How frequently does ftpmaster run into things that would/could trigger a lawsuit? Lucas
Re: Q to all candidates: NEW queue
On Friday, March 27, 2020 8:40:11 AM EDT Lucas Nussbaum wrote: > On 27/03/20 at 12:23 +0100, Martin Pitt wrote: > > At least during my many years of Ubuntu archive administration I've > > actually seen quite a lot of packages which contained non-distributable > > files, had hilariously broken maintainer scripts (which could then also > > damage *other* software on your system), and the like. For these an > > initial NEW review was quite important. > > > > That proposal is assuming that the "package gets reviewed, a bug is filed" > > step actually happens timely, but that is precisely the problem -- with > > such a workflow we would essentially stop having NEW review and just hope > > that someone catches bad packages before they get released. So IMHO this > > is not a solution, and only causes buggy packages to creep into unstable. > > So in my original mail, I proposed that new packages would get > immediately accepted into unstable, but would still require a review > before migrating to testing. I believe that it's an interesting compromise, > because: > - while in unstable, they would get tested by our regular QA tools, that > are likely to find some of the issues ftpmasters would have found > - it makes it possible for the maintainer to get early feedback from > users, and to continue working on packaging reverse dependencies. > - it's unstable, so even if it's severely broken, it's probably not a > big deal. We have lots of packages in unstable that have been severely > broken for years anyway. > - it protects 'testing' (and our stable releases) from unreviewed > packages. > > Of course this only works if Debian doesn't get sued for copyright > infringement too often. I wonder if that would be a problem (it's > probably less likely to be a problem for packages in 'main' than for > packages in 'non-free'). > > Lucas What's "too often"? Scott K signature.asc Description: This is a digitally signed message part.
Re: Q to all candidates: NEW queue
Jean-Philippe MENGUAL Le 27/03/2020 à 13:40, Lucas Nussbaum a écrit : On 27/03/20 at 12:23 +0100, Martin Pitt wrote: At least during my many years of Ubuntu archive administration I've actually seen quite a lot of packages which contained non-distributable files, had hilariously broken maintainer scripts (which could then also damage *other* software on your system), and the like. For these an initial NEW review was quite important. That proposal is assuming that the "package gets reviewed, a bug is filed" step actually happens timely, but that is precisely the problem -- with such a workflow we would essentially stop having NEW review and just hope that someone catches bad packages before they get released. So IMHO this is not a solution, and only causes buggy packages to creep into unstable. So in my original mail, I proposed that new packages would get immediately accepted into unstable, but would still require a review before migrating to testing. I believe that it's an interesting compromise, because: - while in unstable, they would get tested by our regular QA tools, that are likely to find some of the issues ftpmasters would have found - it makes it possible for the maintainer to get early feedback from users, and to continue working on packaging reverse dependencies. - it's unstable, so even if it's severely broken, it's probably not a big deal. We have lots of packages in unstable that have been severely broken for years anyway. - it protects 'testing' (and our stable releases) from unreviewed packages. Of course this only works if Debian doesn't get sued for copyright infringement too often. I wonder if that would be a problem (it's probably less likely to be a problem for packages in 'main' than for packages in 'non-free'). How do you manage the license issue with a direct upload? For this reason, I would tend to suggest expermiental repo instead. ftpmasters would focus on license? IF they accept, good idea. Regards Lucas
Re: Q to all candidates: NEW queue
On 27/03/20 at 12:23 +0100, Martin Pitt wrote: > At least during my many years of Ubuntu archive administration I've actually > seen quite a lot of packages which contained non-distributable files, had > hilariously broken maintainer scripts (which could then also damage *other* > software on your system), and the like. For these an initial NEW review was > quite important. > > That proposal is assuming that the "package gets reviewed, a bug is filed" > step > actually happens timely, but that is precisely the problem -- with such a > workflow we would essentially stop having NEW review and just hope that > someone > catches bad packages before they get released. So IMHO this is not a solution, > and only causes buggy packages to creep into unstable. So in my original mail, I proposed that new packages would get immediately accepted into unstable, but would still require a review before migrating to testing. I believe that it's an interesting compromise, because: - while in unstable, they would get tested by our regular QA tools, that are likely to find some of the issues ftpmasters would have found - it makes it possible for the maintainer to get early feedback from users, and to continue working on packaging reverse dependencies. - it's unstable, so even if it's severely broken, it's probably not a big deal. We have lots of packages in unstable that have been severely broken for years anyway. - it protects 'testing' (and our stable releases) from unreviewed packages. Of course this only works if Debian doesn't get sued for copyright infringement too often. I wonder if that would be a problem (it's probably less likely to be a problem for packages in 'main' than for packages in 'non-free'). Lucas
Re: Q to all candidates: NEW queue
Lucas Nussbaum [2020-03-26 14:58 +0100]: > - package is uploaded > - package gets accepted in unstable > - package gets reviewed, a bug is filed > - bug gets fixed > > Except that with (B), we avoid the wait in NEW. > > One important question is: how often does the FTP team run into a > package that is so problematic that accepting it in Debian with an RC > bug is not an option? At least during my many years of Ubuntu archive administration I've actually seen quite a lot of packages which contained non-distributable files, had hilariously broken maintainer scripts (which could then also damage *other* software on your system), and the like. For these an initial NEW review was quite important. That proposal is assuming that the "package gets reviewed, a bug is filed" step actually happens timely, but that is precisely the problem -- with such a workflow we would essentially stop having NEW review and just hope that someone catches bad packages before they get released. So IMHO this is not a solution, and only causes buggy packages to creep into unstable. However, as always in life this appears to be an 80/20 problem. A lot of new packages are small, simple, and harmless, and can be NEW-reviewed in minutes. E. g. a new python or Perl module, where the whole source code has a single license, very few authors, and no "funny" files. But these 80% tend to get stuck behind the large and complicated new packages. @ftpmasters, would it help to try some automation on the 80% case, and e. g. auto-process packages if they are lintian-clean, suspicious-source is empty, and checking for some reasonable overlap of licensecheck and grep -i '(c)' with names appearing in debian/copyright? So that ftpmasters can concentrate on the 20% complicated packages? Or are the 80% already not a problem/time sink? Thanks, Martin
Re: Q to all candidates: NEW queue
Hi Lucas On 2020/03/26 09:33, Lucas Nussbaum wrote: > For as long as I can remember, there has been complaints about the > delays caused by NEW processing (and > https://ftp-master.debian.org/stat.html shows that we constantly have > 250-300 packages waiting to be processed). There was even a brief period in 2017 where it was really low for a while, but I think this was mainly due to hero work from one individual. Watching some old DebConf videos recently, and it was interesting seeing a BoF where Debian was approaching 15000 packages and the idea was to figure out how to scale up and be able to support that. Since then that number has casually exploded and I think it's remarkable that everything has passed through NEW at some point. > What is your diagnostic of this issue? I think that's the most difficult question I've seen during my two DPL campaigns... thanks! I don't have all the details and have just recently been re-accepted as ftp-trainee, but I believe it's a case of it being easier to continue doing a lot of grunt work rather than to do a collective step back and redesigning the machinery. > What solutions do you envision about this issue? Is that just something > that we have to live with? I'll be honest in that I haven't envisioned anything for this yet, I'd like to take some time to get into it and understand all the problems properly first. I do not think it's something we have to live with, no. I think with a combination of process improvements as well as tooling improvements, this could be made a lot better. For example, copyright review seems to be a big chunk of the work. There's probably no reason why a larger group of DD's can't help with this too (currently the ftp-helpers/trainees helps with this, but it depends on them being part of the team and using the team's tooling). Perhaps mentors.debian.net could be augmented for better copyright reviewing, or a similar tool could be set up for DD's who want their copyright reviewed or maybe even just get a second pair of eyes over the package before it gets submitted to NEW, then that might already help. >From a mostly outsider view, it seems that packages that are problematic take up a significantly more amount of ftpmaster time than packages which have no problems. If we can add some kind of filters, whether it's based on code or on social solutions, then it may be possible to reduce ftpmaster load without even making any immediate changes to the ftp team. That said, I like the efforts currently underway with the new ftptrainees, there is now a dedicated person taking care of them (well, us :)) and I think those efforts will pay off. I also think that it's worth while for the ftpmasters to get together somewhere nice at least once a year. Nice as in, not DebCamp but somewhere where they can have relaxed conversation and be creative without being disturbed. > Specifically, Jonathan writes that he would like to "Reduce bottlenecks > that affect our contributors.". That sounds like a good example. Yep, perfect example :) > Personnally, I wonder if we are being overly cautious about NEW. I > wonder if we could move to checking a posteriori (accept the package in > unstable; maybe don't let it migrate to testing until it is reviewed). Not fond of that idea, because that means the packages already get mirrored (so basically distributed) already. It would be nice to be able to access NEW packages via an apt repository, or even links from the NEW page[1] to the package's git repository, so I think there might be tooling that can help but I don't want to get into specific ideas or solutions without having delved deeper into this. I do have confidence that the current measures with the new ftptrainees will start paying off over the next few months. Merely the fact that there's less stress on the team might help it to re-assess a few old processes and tooling and allow it to evolve again. Overall, I think it's good for a DPL to check in on the team and offer assistance, I don't think a DPL should be too pushy on these issues, the strategy should be removing pressure instead of adding more imho. If the team has more time to address their problems internally then the day to day problems will eventually get better too. -Jonathan [1] https://ftp-master.debian.org/new.html -- ⢀⣴⠾⠻⢶⣦⠀ Jonathan Carter (highvoltage) ⣾⠁⢠⠒⠀⣿⡁ https://wiki.debian.org/highvoltage ⢿⡄⠘⠷⠚⠋ https://debian.org | https://jonathancarter.org ⠈⠳⣄ Be Bold. Be brave. Debian has got your back.
Re: Q to all candidates: NEW queue
Le 27/03/2020 à 10:03, Jonas Smedegaard a écrit : > Quoting Xavier (2020-03-27 09:38:54) >> Le 27/03/2020 à 09:22, Ulrike Uhlig a écrit : >>> Hi! >>> >>> On 26.03.20 15:05, Lucas Nussbaum wrote: On 26/03/20 at 14:42 +0100, Jonas Smedegaard wrote: > > For the avoidance of doubt, I wrote none of below quoted text. > A/ - package is uploaded - package waits in NEW - package gets reviewed, gets accepted in unstable with a bug filed - bug gets fixed B/ - package is uploaded - package gets accepted in unstable - package gets reviewed, a bug is filed - bug gets fixed Except that with (B), we avoid the wait in NEW. >>> >>> In scenario B, the wait is shorter, but there is no guarantee that the >>> bug gets fixed in an appropriate time frame. >>> One important question is: how often does the FTP team run into a package that is so problematic that accepting it in Debian with an RC bug is not an option? >>> >>> Another question is: what other things are reviewed by the FTP team? >>> Like, is there some sort of basic security review, are there packages >>> that are not appropriate to be uploaded to the archive for some reason >>> or another? And then this would not just result in an RC bug, I guess? >>> >>> Ulrike >> >> Hi >> >> And what about creating "pre-ftp-master" in teams ? They could fix team >> policy and do a technical review. This will avoid common errors and may >> decrease ftp-master work. >> >> This proposition is based on JS-Team experience: some people pushed JS >> packages without knowing JS tools and practices. These packages often >> contains pre-build objects and a few other common errors. Sadly some DD >> push such packages with JS-Team as maintainer without being member of >> this team, neither asking for help/review from JS team! >> >> Then scenario C: >> - package is uploaded >> - ftp-master redirects package to pre-ftp-master(s) [using BTS?] who >>are concerned >> - package gets pre-ftp-master(s) review [BTS(s) closed] >> - package gets ftp-master's review >> - package is released > > Sorry, maybe just me, but is above a question to DPL candidates? > > I mean, teams certainly do not need DPL to permit them to packaging > prepare packages before bothering ftp-master with them, or...? > > - Jonas This change a little actual process. Example if a package contains plural languages, ftp-master ask for review to the different pre-ftp-masters of concerned teams. Example: a Perl package that contains JS is pre-reviewed by 2 pre-ftp-masters. This may decrease ftp-master work since new packages may have better quality before ftp-master review.
Re: Q to all candidates: NEW queue
Quoting Xavier (2020-03-27 09:38:54) > Le 27/03/2020 à 09:22, Ulrike Uhlig a écrit : > > Hi! > > > > On 26.03.20 15:05, Lucas Nussbaum wrote: > >> On 26/03/20 at 14:42 +0100, Jonas Smedegaard wrote: For the avoidance of doubt, I wrote none of below quoted text. > >> A/ > >> - package is uploaded > >> - package waits in NEW > >> - package gets reviewed, gets accepted in unstable with a bug filed > >> - bug gets fixed > >> > >> B/ > >> - package is uploaded > >> - package gets accepted in unstable > >> - package gets reviewed, a bug is filed > >> - bug gets fixed > >> > >> Except that with (B), we avoid the wait in NEW. > > > > In scenario B, the wait is shorter, but there is no guarantee that the > > bug gets fixed in an appropriate time frame. > > > >> One important question is: how often does the FTP team run into a > >> package that is so problematic that accepting it in Debian with an RC > >> bug is not an option? > > > > Another question is: what other things are reviewed by the FTP team? > > Like, is there some sort of basic security review, are there packages > > that are not appropriate to be uploaded to the archive for some reason > > or another? And then this would not just result in an RC bug, I guess? > > > > Ulrike > > Hi > > And what about creating "pre-ftp-master" in teams ? They could fix team > policy and do a technical review. This will avoid common errors and may > decrease ftp-master work. > > This proposition is based on JS-Team experience: some people pushed JS > packages without knowing JS tools and practices. These packages often > contains pre-build objects and a few other common errors. Sadly some DD > push such packages with JS-Team as maintainer without being member of > this team, neither asking for help/review from JS team! > > Then scenario C: > - package is uploaded > - ftp-master redirects package to pre-ftp-master(s) [using BTS?] who >are concerned > - package gets pre-ftp-master(s) review [BTS(s) closed] > - package gets ftp-master's review > - package is released Sorry, maybe just me, but is above a question to DPL candidates? I mean, teams certainly do not need DPL to permit them to packaging prepare packages before bothering ftp-master with them, or...? - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Re: Q to all candidates: NEW queue
Le 27/03/2020 à 09:22, Ulrike Uhlig a écrit : > Hi! > > On 26.03.20 15:05, Lucas Nussbaum wrote: >> On 26/03/20 at 14:42 +0100, Jonas Smedegaard wrote: > >> A/ >> - package is uploaded >> - package waits in NEW >> - package gets reviewed, gets accepted in unstable with a bug filed >> - bug gets fixed >> >> B/ >> - package is uploaded >> - package gets accepted in unstable >> - package gets reviewed, a bug is filed >> - bug gets fixed >> >> Except that with (B), we avoid the wait in NEW. > > In scenario B, the wait is shorter, but there is no guarantee that the > bug gets fixed in an appropriate time frame. > >> One important question is: how often does the FTP team run into a >> package that is so problematic that accepting it in Debian with an RC >> bug is not an option? > > Another question is: what other things are reviewed by the FTP team? > Like, is there some sort of basic security review, are there packages > that are not appropriate to be uploaded to the archive for some reason > or another? And then this would not just result in an RC bug, I guess? > > Ulrike Hi And what about creating "pre-ftp-master" in teams ? They could fix team policy and do a technical review. This will avoid common errors and may decrease ftp-master work. This proposition is based on JS-Team experience: some people pushed JS packages without knowing JS tools and practices. These packages often contains pre-build objects and a few other common errors. Sadly some DD push such packages with JS-Team as maintainer without being member of this team, neither asking for help/review from JS team! Then scenario C: - package is uploaded - ftp-master redirects package to pre-ftp-master(s) [using BTS?] who are concerned - package gets pre-ftp-master(s) review [BTS(s) closed] - package gets ftp-master's review - package is released Cheers, Xavier
Re: Q to all candidates: NEW queue
Hi, On 27.03.20 09:32, Jonas Smedegaard wrote: > Quoting Ulrike Uhlig (2020-03-27 09:22:46) >> Another question is: what other things are reviewed by the FTP team? >> Like, is there some sort of basic security review, are there packages >> that are not appropriate to be uploaded to the archive for some reason >> or another? And then this would not just result in an RC bug, I guess? > > See https://ftp-master.debian.org/REJECT-FAQ.html Thank you. My point was to ask if there are any points in this list that could be harmful in the scenario proposed by Lucas. ulrike
Re: Q to all candidates: NEW queue
Quoting Ulrike Uhlig (2020-03-27 09:22:46) > Another question is: what other things are reviewed by the FTP team? > Like, is there some sort of basic security review, are there packages > that are not appropriate to be uploaded to the archive for some reason > or another? And then this would not just result in an RC bug, I guess? See https://ftp-master.debian.org/REJECT-FAQ.html - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Re: Q to all candidates: NEW queue
Hi! On 26.03.20 15:05, Lucas Nussbaum wrote: > On 26/03/20 at 14:42 +0100, Jonas Smedegaard wrote: > A/ > - package is uploaded > - package waits in NEW > - package gets reviewed, gets accepted in unstable with a bug filed > - bug gets fixed > > B/ > - package is uploaded > - package gets accepted in unstable > - package gets reviewed, a bug is filed > - bug gets fixed > > Except that with (B), we avoid the wait in NEW. In scenario B, the wait is shorter, but there is no guarantee that the bug gets fixed in an appropriate time frame. > One important question is: how often does the FTP team run into a > package that is so problematic that accepting it in Debian with an RC > bug is not an option? Another question is: what other things are reviewed by the FTP team? Like, is there some sort of basic security review, are there packages that are not appropriate to be uploaded to the archive for some reason or another? And then this would not just result in an RC bug, I guess? Ulrike
Re: Q to all candidates: NEW queue
On Thu, Mar 26, 2020 at 3:34 AM Lucas Nussbaum wrote: > > Hi, > > For as long as I can remember, there has been complaints about the > delays caused by NEW processing (and > https://ftp-master.debian.org/stat.html shows that we constantly have > 250-300 packages waiting to be processed). > > What is your diagnostic of this issue? > What solutions do you envision about this issue? Is that just something > that we have to live with? > > Specifically, Jonathan writes that he would like to "Reduce bottlenecks > that affect our contributors.". That sounds like a good example. > > > Personnally, I wonder if we are being overly cautious about NEW. I > wonder if we could move to checking a posteriori (accept the package in > unstable; maybe don't let it migrate to testing until it is reviewed). > > Lucas I believe package count by itself lacks context. I would encourage FTPmasters to add another metric to their graphs that include the average age of packages in NEW, rather than just a count. In addition to recruiting additional team members, which they are already working on, I think FTPmasters needs to figure a way to get their heads out of the weeds of the overwhelming work, and find a way to accept help, without lowering quality. Cheers, Brian signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
On Thu, Mar 26, 2020 at 10:03:16AM -0700, Sean Whitton wrote: > Hello Roberto, > > On Thu 26 Mar 2020 at 09:51AM -04, Roberto C. Sánchez wrote: > > > Good point. The most recent experience I had with this was a package I > > uploaded in February 2019. The FTP team rejection came on 4th January > > 2020, my requests for additional clarification were made within 24 hours > > of the rejection, but have not yet been replied to by the FTP team. > > Typically we try to follow up on requests for clarification but > sometimes e-mails get lost. So kindly ask again. > Thanks for the suggestion. I have re-sent my request for clarification. > > I have filed an ITP for every new package I have ever uploaded. I have > > never seen the FTP team make comments in the ITP bug whenever there has > > been an issue that had to be resolved, and that remains the case for my > > most recent experience. > > The FTP Team has not adopted the workflow of posting comments to ITPs. > Ah, I misunderstood Jonas' email. He did say "for future cases" and I missed that part. Regards, -Roberto -- Roberto C. Sánchez
Re: Q to all candidates: NEW queue
Hello Roberto, On Thu 26 Mar 2020 at 09:51AM -04, Roberto C. Sánchez wrote: > Good point. The most recent experience I had with this was a package I > uploaded in February 2019. The FTP team rejection came on 4th January > 2020, my requests for additional clarification were made within 24 hours > of the rejection, but have not yet been replied to by the FTP team. Typically we try to follow up on requests for clarification but sometimes e-mails get lost. So kindly ask again. > I have filed an ITP for every new package I have ever uploaded. I have > never seen the FTP team make comments in the ITP bug whenever there has > been an issue that had to be resolved, and that remains the case for my > most recent experience. The FTP Team has not adopted the workflow of posting comments to ITPs. -- Sean Whitton signature.asc Description: PGP signature
Re: Q to all candidates: NEW queue
On 26/03/20 at 14:42 +0100, Jonas Smedegaard wrote: > Quoting Roberto C. Sánchez (2020-03-26 14:28:47) > > That said, I have never had a package rejected for reasons that would > > outright keep it from entering Debian. Each package I have had > > rejected could have as easily been accepted into unstable and then > > fixed after the fact to address whatever the problem was. For > > instance, omitting a particular file with a distinct (but compatible) > > license from listing in the copyright file or some similar minor > > license-related matter. None of those things seem like reasons to > > hold up a package entering Debian for months to nearly a year. > > Though, I may be mistaken in that and biased based on my own > > experience. > > Or you might be describing a procedure that has since been improved. > > When sharing example cases, it helps if you also mention how long ago it > happened (I sometimes get surprised how much time has passed when > checking such facts). > > For future cases, I suggest to always file an ITP bugreport before > pushing a new package, to help encourage ftp team to use that bugreport > to store responses/remarks from them - and when that happens we can in > future reference the bugreport when sharing example cases :-) I think that Roberto's point is that those two workflows are valid: A/ - package is uploaded - package waits in NEW - package gets reviewed, gets accepted in unstable with a bug filed - bug gets fixed B/ - package is uploaded - package gets accepted in unstable - package gets reviewed, a bug is filed - bug gets fixed Except that with (B), we avoid the wait in NEW. One important question is: how often does the FTP team run into a package that is so problematic that accepting it in Debian with an RC bug is not an option? Lucas
Re: Q to all candidates: NEW queue
On Thu, Mar 26, 2020 at 02:42:34PM +0100, Jonas Smedegaard wrote: > Quoting Roberto C. Sánchez (2020-03-26 14:28:47) > > That said, I have never had a package rejected for reasons that would > > outright keep it from entering Debian. Each package I have had > > rejected could have as easily been accepted into unstable and then > > fixed after the fact to address whatever the problem was. For > > instance, omitting a particular file with a distinct (but compatible) > > license from listing in the copyright file or some similar minor > > license-related matter. None of those things seem like reasons to > > hold up a package entering Debian for months to nearly a year. > > Though, I may be mistaken in that and biased based on my own > > experience. > > Or you might be describing a procedure that has since been improved. > > When sharing example cases, it helps if you also mention how long ago it > happened (I sometimes get surprised how much time has passed when > checking such facts). > Good point. The most recent experience I had with this was a package I uploaded in February 2019. The FTP team rejection came on 4th January 2020, my requests for additional clarification were made within 24 hours of the rejection, but have not yet been replied to by the FTP team. One additional point which I initially omitted and which may (or may not) have bearing on the situation is that the package in question targeted experimental rather than unstable. > For future cases, I suggest to always file an ITP bugreport before > pushing a new package, to help encourage ftp team to use that bugreport > to store responses/remarks from them - and when that happens we can in > future reference the bugreport when sharing example cases :-) > I have filed an ITP for every new package I have ever uploaded. I have never seen the FTP team make comments in the ITP bug whenever there has been an issue that had to be resolved, and that remains the case for my most recent experience. Regards, -Roberto -- Roberto C. Sánchez
Re: Q to all candidates: NEW queue
Quoting Roberto C. Sánchez (2020-03-26 14:28:47) > That said, I have never had a package rejected for reasons that would > outright keep it from entering Debian. Each package I have had > rejected could have as easily been accepted into unstable and then > fixed after the fact to address whatever the problem was. For > instance, omitting a particular file with a distinct (but compatible) > license from listing in the copyright file or some similar minor > license-related matter. None of those things seem like reasons to > hold up a package entering Debian for months to nearly a year. > Though, I may be mistaken in that and biased based on my own > experience. Or you might be describing a procedure that has since been improved. When sharing example cases, it helps if you also mention how long ago it happened (I sometimes get surprised how much time has passed when checking such facts). For future cases, I suggest to always file an ITP bugreport before pushing a new package, to help encourage ftp team to use that bugreport to store responses/remarks from them - and when that happens we can in future reference the bugreport when sharing example cases :-) - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Re: Q to all candidates: NEW queue
On Thu, Mar 26, 2020 at 09:49:41AM -0300, Gabriel F. T. Gomes wrote: > Hi, Lucas, > > On Thu, 26 Mar 2020, Lucas Nussbaum wrote: > > > > What solutions do you envision about this issue? Is that just something > > that we have to live with? > > The FTP Team has very recently made a call for volunteers [1], so > perhaps they already have some plans?... > > [1] https://lists.debian.org/debian-devel-announce/2020/03/msg3.html > > > Personnally, I wonder if we are being overly cautious about NEW. I > > wonder if we could move to checking a posteriori (accept the package in > > unstable; maybe don't let it migrate to testing until it is reviewed). > > Anyhow, I wrote to weigh in on this particular comment, because, as a > Debian Maintainer, I had only submitted two packages to the NEW queue. > On both occasions, the FTP Team found licensing issues in the packages, > even though I thought I had done a *really* great job of reading through > the copyright notices, (specially for the second package). So, I'm > glad the package had to be reviewed, even though I would had been > happier if the process had taken less time, :). > I too have found the FTP team review valuable. Like in your case, they identified an issue that I had overlooked. The part I didn't like was waiting 11 months to get that message from the FTP team and (since the FTP team rejection message included some unclear comments) not receiving a reply to my requests for clarification. That said, I have never had a package rejected for reasons that would outright keep it from entering Debian. Each package I have had rejected could have as easily been accepted into unstable and then fixed after the fact to address whatever the problem was. For instance, omitting a particular file with a distinct (but compatible) license from listing in the copyright file or some similar minor license-related matter. None of those things seem like reasons to hold up a package entering Debian for months to nearly a year. Though, I may be mistaken in that and biased based on my own experience. Regards, -Roberto -- Roberto C. Sánchez
Re: Q to all candidates: NEW queue
Hi, Lucas, On Thu, 26 Mar 2020, Lucas Nussbaum wrote: > > What solutions do you envision about this issue? Is that just something > that we have to live with? The FTP Team has very recently made a call for volunteers [1], so perhaps they already have some plans?... [1] https://lists.debian.org/debian-devel-announce/2020/03/msg3.html > Personnally, I wonder if we are being overly cautious about NEW. I > wonder if we could move to checking a posteriori (accept the package in > unstable; maybe don't let it migrate to testing until it is reviewed). Anyhow, I wrote to weigh in on this particular comment, because, as a Debian Maintainer, I had only submitted two packages to the NEW queue. On both occasions, the FTP Team found licensing issues in the packages, even though I thought I had done a *really* great job of reading through the copyright notices, (specially for the second package). So, I'm glad the package had to be reviewed, even though I would had been happier if the process had taken less time, :). Cheers, Gabriel
Q to all candidates: NEW queue
Hi, For as long as I can remember, there has been complaints about the delays caused by NEW processing (and https://ftp-master.debian.org/stat.html shows that we constantly have 250-300 packages waiting to be processed). What is your diagnostic of this issue? What solutions do you envision about this issue? Is that just something that we have to live with? Specifically, Jonathan writes that he would like to "Reduce bottlenecks that affect our contributors.". That sounds like a good example. Personnally, I wonder if we are being overly cautious about NEW. I wonder if we could move to checking a posteriori (accept the package in unstable; maybe don't let it migrate to testing until it is reviewed). Lucas
