Bug#981515: kcoreaddons: please replace fam with gamin
> Hi, > > In data venerdì 20 agosto 2021 13:32:08 CEST, Chris Hofstaedtler ha scritto: > > given we are now in the bookworm cycle - do you think kcoreaddons > > could remove fam (or replace it with gamin if really needed)? > > As I wrote a couple of times in this bug already: #510368 must be fixed > first. It seems like I forgot to set this dependency, so I'm adding it > now. > > Also, considering that there is a huge queue of RM bugs on the > FTP-masters side that have been attended so far a couple of times in > this year (sigh), rushing this won't make any change, since the RM will > be ignored for some time anyway. > Also #2: considering that we are not even a week since the lift of the > freeze and everybody and their dogs have already uploaded things of > every sort of unstable, please give (me) some time for what is really > not a priority. FTP team is caught up on removals. kcoreaddons is the last package keeping fam in the archive, so I would appreciate it if this could be resolved so we can just get rid of it. Scott K signature.asc Description: This is a digitally signed message part.
Processed: Re: Bug#981515: kcoreaddons: please replace fam with gamin
Processing commands for cont...@bugs.debian.org: > block 981515 by 510368 Bug #981515 [src:kcoreaddons] kcoreaddons: please replace fam with gamin 981515 was not blocked by any bugs. 981515 was blocking: 966273 Added blocking bug(s) of 981515: 510368 > thanks Stopping processing here. Please contact me if you need assistance. -- 981515: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981515 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#981515: kcoreaddons: please replace fam with gamin
block 981515 by 510368 thanks Hi, In data venerdì 20 agosto 2021 13:32:08 CEST, Chris Hofstaedtler ha scritto: > given we are now in the bookworm cycle - do you think kcoreaddons > could remove fam (or replace it with gamin if really needed)? As I wrote a couple of times in this bug already: #510368 must be fixed first. It seems like I forgot to set this dependency, so I'm adding it now. Also, considering that there is a huge queue of RM bugs on the FTP-masters side that have been attended so far a couple of times in this year (sigh), rushing this won't make any change, since the RM will be ignored for some time anyway. Also #2: considering that we are not even a week since the lift of the freeze and everybody and their dogs have already uploaded things of every sort of unstable, please give (me) some time for what is really not a priority. Thanks, -- Pino Toscano signature.asc Description: This is a digitally signed message part.
Bug#981515: kcoreaddons: please replace fam with gamin
Hi Pino, given we are now in the bookworm cycle - do you think kcoreaddons could remove fam (or replace it with gamin if really needed)? Do you want a wider audience informed of the change, or would the change in kcoreaddons be self-contained enough? Thanks, Chris
Bug#981515: kcoreaddons: please replace fam with gamin
In data venerdì 5 marzo 2021 18:16:18 CET, Glenn Strauss ha scritto: > On Fri, Mar 05, 2021 at 05:12:17PM +0100, Pino Toscano wrote: > > > Personally, I'd argue that switching the FAM implementation across the > > distribution _is_ a "transition", and as such it ought to have been > > requested (if not even started) two months ago. > > In July 2020, #966273 was filed: RFA: fam -- File Alteration Monitor > > I posted in October 2020 on that bug where FAM was abandoned. > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966273#15 > Debian developers did not suggest "next steps" for over 3 months, > until the freeze occurred. > > The bug was not touched by a Debian Developer until 31 Jan 2021. This message was addressed only to bugs, one of them was assigned to "wnpp" and the other to libgamon0: this means that only the gamin maintainer (1 specific person, as there is no group maintenance) and who watches the bugs of wnpp and src:gamin actually read it. Becuase of this, its audience was limited, and neither the general list for Debian development (debian-devel) nor the release team (release-team) were notified/aware of it by default. I can understand your frustation, but that is not a reason to rush things just because of this. All the deadline for Debian releases have been more or less streamlined/standardized for at least the past 2 stable releases already, so every step is well known in advance. Because of this, for example, we were not able to provide Plasma 5.12 in Bullseye. > The solution is to remove FAM. And nobody, really, *nobody* ever said the opposite, so please stop repeating that it is not wanted. > Back on topic: > > If you take a moment to look in the kcoreaddons code, you can see that > kcoreaddons has multiple mechanisms for filesystem notification. > If FAM interfaces fail for any reason, kcoreaddons switches to an > alternative mechanism. > > https://invent.kde.org/frameworks/kcoreaddons/-/blob/master/src/lib/io/kdirwatch.cpp#L1611 > > Therefore, your FUD is unsubstantiated. Changing kcoreaddons to use > gamin, or to not use FAM or gamin, are both reasonable options. > As I posted in > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981515#49 > gamin can be configured to poll NFS locations and so is a reasonable > substitution for FAM, not limited to inotify() as Sune suggested in > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981515#39 Sure, I well know that KDirWatch in kcoreaddons builds fine with either libfam or gamin as "FAM", I remember people doing this many years ago in other distributions. However, what I said is something completely different, let me summarize it in short points for easier understanding: - I am fine switching from libfam to gamin in the future - I am fine dropping libfam from Debian in the future - I want first libgamin to stop providing libfam - switching from libfam from gamin *is* switching FAM implementation, and thus which code is used for "FAM", and possibily different bugs; hence, this is *too late* to properly test is in Bullseye There is no FUD in what I said. > It is true that this relatively safe change is being requested during > the soft freeze and so should be scrutinized. However, that does not > make the requested change any more or less dangerous. It does mean > less time to test by people who, in your own words, might not be using > this feature: > > and these FAM/gamin bits do not get much of use these days So if, "less time to test by people who [...] might not be using this feature", this switch is even more dangerous. Thanks from proving my point. > I posit that the code in upstream kcoreaddons is already better tested > than would be Debian "testing" (existence in tree?) of an additional > month or two or three of the Debian kcoreaddons package configured to > use gamin (or to use neither FAM nor gamin). This "additional month or two or three" we are talking about is called "Debian freeze". Dependency changes like this are very much *not* the kind of changes we want to introduce during a freeze. -- Pino Toscano signature.asc Description: This is a digitally signed message part.
Bug#981515: kcoreaddons: please replace fam with gamin
On Fri, Mar 05, 2021 at 05:12:17PM +0100, Pino Toscano wrote: > Personally, I'd argue that switching the FAM implementation across the > distribution _is_ a "transition", and as such it ought to have been > requested (if not even started) two months ago. In July 2020, #966273 was filed: RFA: fam -- File Alteration Monitor I posted in October 2020 on that bug where FAM was abandoned. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966273#15 Debian developers did not suggest "next steps" for over 3 months, until the freeze occurred. The bug was not touched by a Debian Developer until 31 Jan 2021. All this reflects poorly on Debian Developers, who we all understand are volunteers. I am not questioning the skills of Debian Developers. I am criticizing the execution. The failure to triage bugs in a timely fashion, the failure to encourage outside contributions, and the failure to recruit and nurture additional help all reflect poorly on Debian as a project, as does the 12+ year old bug which is being deferred, yet again. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=510368 The solution is to remove FAM. It just FUD that is delaying action. (It is FUD hiding behind policy, since FAM package removal was a blocking bug for Bullseye.) Back on topic: If you take a moment to look in the kcoreaddons code, you can see that kcoreaddons has multiple mechanisms for filesystem notification. If FAM interfaces fail for any reason, kcoreaddons switches to an alternative mechanism. https://invent.kde.org/frameworks/kcoreaddons/-/blob/master/src/lib/io/kdirwatch.cpp#L1611 Therefore, your FUD is unsubstantiated. Changing kcoreaddons to use gamin, or to not use FAM or gamin, are both reasonable options. As I posted in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981515#49 gamin can be configured to poll NFS locations and so is a reasonable substitution for FAM, not limited to inotify() as Sune suggested in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981515#39 It is true that this relatively safe change is being requested during the soft freeze and so should be scrutinized. However, that does not make the requested change any more or less dangerous. It does mean less time to test by people who, in your own words, might not be using this feature: > and these FAM/gamin bits do not get much of use these days I posit that the code in upstream kcoreaddons is already better tested than would be Debian "testing" (existence in tree?) of an additional month or two or three of the Debian kcoreaddons package configured to use gamin (or to use neither FAM nor gamin). With that, I've said my piece. I shall not argue further.
Bug#981515: kcoreaddons: please replace fam with gamin
Dear Glenn, In data venerdì 5 marzo 2021 16:41:40 CET, Glenn Strauss ha scritto: > In #981513, courier changed to use libgamin-dev, so > kcoreaddons is now the *only* remaining package using FAM. > > As such, there is considerably more risk to doing nothing > than there is to migrating to gamin. No, there is more risk in switching to a different library at this phase of the Debian freeze. > ==> Please change kcoreaddons to use libgamin-dev for Bullseye. > https://salsa.debian.org/qt-kde-team/kde/kcoreaddons/-/merge_requests/3 While I understand your motivation behind this change, I'll repeat once again what I said previously in this bug: this is *not* going to happen for Bullseye, full stop. The reason is that we are talking about switching to a different library for a functionality that is rarely used these days (but potentially can be), hence a switch at this phase is very risky, and gives basically no time to test or even get feedback about it. The freeze for transitions started almost two months ago, on January 13th: https://lists.debian.org/debian-devel-announce/2021/01/msg2.html Personally, I'd argue that switching the FAM implementation across the distribution _is_ a "transition", and as such it ought to have been requested (if not even started) two months ago. On February 13th, a "mild freeze" started: https://lists.debian.org/debian-devel-announce/2021/02/msg2.html while changes at the beginning of it still migrated to testing, IMHO the switch of a dependency raises the bar of the risk; while I can check it for things I upload and work for, this feature represents something corner-case, which I don't have neither the setup nor the time to properly test. This request was opened at the end of January (so in transition freeze already, and IMHO enough to make it out of scope for Bullseye), and my question about the timing for this got "not for Bullseye" as answer. All the more traffic for you, Glenn, started two days ago, already in a time frame where uploads to unstable will not migrate to testing anymore [1], and thus it will need exception from release-team, meaning it has to be something importat/serious enough (and this is not, as the status of it would be the same as in previous Debian releases). [1] automatic migration ends on March 13th, and the default migration time is 10 days, which means the last day for such uploads was March 2nd Moreover, I already stated that I really want #510368 fixed _before_ switching the dependency, and that bug has not been fixed yet (and unlikely to be for Bullseye). So, thanks again for the time and interest in this, but this will be handled only after both a) Bullseye is released b) #510368 is fixed. Thanks, -- Pino Toscano signature.asc Description: This is a digitally signed message part.
Bug#981515: kcoreaddons: please replace fam with gamin
In #981513, courier changed to use libgamin-dev, so kcoreaddons is now the *only* remaining package using FAM. As such, there is considerably more risk to doing nothing than there is to migrating to gamin. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=510368 is over 12 years old. It's time to get it fixed by removing FAM instead of going through another cycle where end-users continue to run into unnecessary conflicts. There are a large number of issues that go away when FAM gets dropped. I listed some of them in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981513 If you search bugs.debian.org, or Ubuntu forums, or the internet, the general solution to conflicts in between FAM and gamin is to remove FAM, e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=599682 (from 2011!) (solution: replace FAM with gamin) On the other hand, a search of bugs.debian.org did not turn up any bugs with kcoreaddons and gamin, besides this one (#981515). Why are you suggesting "there might be issues with kcoreaddons using gamin", when no such issues have been reported thus far? ==> Please change kcoreaddons to use libgamin-dev for Bullseye. https://salsa.debian.org/qt-kde-team/kde/kcoreaddons/-/merge_requests/3 Thank you. Glenn
Bug#981515: kcoreaddons: please replace fam with gamin
On Wed, Mar 03, 2021 at 02:54:58PM -0500, Nicholas D Steeves wrote: > Hi, > > Glenn Strauss writes: > > > gamin provides libfam0. > > > > kcoreaddons should load just fine with libfam0 from gamin. > > > > I did the research in #510368 and #966273, reviewing the actual code > > and confidentally concluded that FAM can be removed from Bullseye. > > > > The safest choice is to have a single library (gamin) used in the > > distro, rather than having both FAM and gamin. > > > > I don't think the removal of FAM is as clear-cut as it has been > presented to be. > > AFAIK the following is still current: Gamin provides "No NFS support > based on specific RPC and server, instead gamin monitors only the state > as reported locally by the kernel, not that locally done changes on NFS > or AFS filesystems are reported on Linux which is the main criteria when > having user home directories on such filesystems." > > https://people.gnome.org/~veillard/gamin/differences.html > > thus FAM covers a use case that gamin does not, and this case is: users > who want to receive inotify style events for files that have been > remotely created or modified on NFS mounts. > > I can't speak to how widespread the need for this feature is, but if it > is non-zero then it seems to me that FAM should not be removed this late > in the Bullseye cycle. > > Also, IIRC gamin depends on Linux-specific features such as inotify > where FAM provides fallbacks (eg: IIRC FAM is required on kfreebsd and > hurd); this might not be significant, but I felt it was worth mentioning > :-) > > FreeBSD doesn't have inotify, so there is a need for FAM, and maybe > someone from their community has become the defacto upstream (via their > "ports" packaging system)? Or maybe someone from their community would > be willing to officially become FAM's new upstream? Nicholas: gamin can be configured to use different mechanisms for different filesystems, so gamin can be configured to poll an NFS filesystem instead of using inotify(). Also, gamin supports kqueue() on *BSD. https://people.gnome.org/~veillard/gamin/config.html Cheers, Glenn
Bug#981515: kcoreaddons: please replace fam with gamin
On Thu, Mar 04, 2021 at 08:23:44AM +0100, Sune Stolborg Vuorela wrote: > > On 3/3/21 8:54 PM, Nicholas D Steeves wrote: > > > > thus FAM covers a use case that gamin does not, and this case is: users > > who want to receive inotify style events for files that have been > > remotely created or modified on NFS mounts. > > > > Also, IIRC gamin depends on Linux-specific features such as inotify > > where FAM provides fallbacks (eg: IIRC FAM is required on kfreebsd and > > hurd); this might not be significant, but I felt it was worth mentioning > > :-) > > > Please note that these features are what KCoreAddons (or more specifically > KDirWatch) uses FAM for. > > KDirWatch hooks up to inotify directly and prefers that if possible. FAM is > only used for fallback codepath for e.g. nfs-mounts, before KDirWatch > resorts to the last fallback; polling. > > Replacing with libgamin is basically making the inotify-free codepath depend > on inotify, so either keep using FAM, or skip both of them. > > > /Sune > > - who has been debugging KDirWatch and other parts of KCoreAddons over the > years. Sune, thanks for chiming in as someone using and debugging KCoreAddons! It bears repeating that KCoreAddons using FAM is ineffective over NFS unless the NFS server is also running and insecurely exposing FAM, which is hopefully not widely used on the open internet today. Sune, do you know if you were using KCoreAddons on networks that had FAM running on the NFS server? If so, would you recommend those configurations today (versus 20 years ago)? (Yes, it is possible to better secure with firewall rules and tunnels, but takes more effort and is still less appropriate for a multi-user system on which the users have shell access and are using KCoreAddons.) > Replacing with libgamin is basically making the inotify-free codepath depend > on inotify, so either keep using FAM, or skip both of them. I suggest the latter. I think removal of FAM and gamin from KCoreAddons is preferred. As a lighttpd developer, I replaced both FAM and gamin with inotify() on Linux and kqueue() on *BSD. There is a reasonable fallback in lighttpd when neither are available. Cheers, Glenn
Bug#981515: kcoreaddons: please replace fam with gamin
On 3/3/21 8:54 PM, Nicholas D Steeves wrote: thus FAM covers a use case that gamin does not, and this case is: users who want to receive inotify style events for files that have been remotely created or modified on NFS mounts. Also, IIRC gamin depends on Linux-specific features such as inotify where FAM provides fallbacks (eg: IIRC FAM is required on kfreebsd and hurd); this might not be significant, but I felt it was worth mentioning :-) Please note that these features are what KCoreAddons (or more specifically KDirWatch) uses FAM for. KDirWatch hooks up to inotify directly and prefers that if possible. FAM is only used for fallback codepath for e.g. nfs-mounts, before KDirWatch resorts to the last fallback; polling. Replacing with libgamin is basically making the inotify-free codepath depend on inotify, so either keep using FAM, or skip both of them. /Sune - who has been debugging KDirWatch and other parts of KCoreAddons over the years.
Bug#981515: kcoreaddons: please replace fam with gamin
On Wed, Mar 03, 2021 at 02:54:58PM -0500, Nicholas D Steeves wrote: > I don't think the removal of FAM is as clear-cut as it has been > presented to be. > > AFAIK the following is still current: Gamin provides "No NFS support > based on specific RPC and server, instead gamin monitors only the state > as reported locally by the kernel, not that locally done changes on NFS > or AFS filesystems are reported on Linux which is the main criteria when > having user home directories on such filesystems." > > https://people.gnome.org/~veillard/gamin/differences.html > > thus FAM covers a use case that gamin does not, and this case is: users > who want to receive inotify style events for files that have been > remotely created or modified on NFS mounts. > > I can't speak to how widespread the need for this feature is, but if it > is non-zero then it seems to me that FAM should not be removed this late > in the Bullseye cycle. FAM attempts to get notifications for NFS by requiring that a remote FAM server is running on the NFS server. No encryption is used. I hope this is not widely used. FAM code was last modified circa 2003 in the tar ball I downloaded from debian and predates inotify(). However, using FAM or gamin in kcoreaddons is not necessary. Quoting from kcoreaddons src/lib/io/kdirwatch.h: https://invent.kde.org/frameworks/kcoreaddons/-/blob/master/src/lib/io/kdirwatch.h#L44 * The implementation uses the INOTIFY functionality on LINUX. * Otherwise the FAM service is used, when available. * As a last resort, a regular polling for change of modification times * is done; the polling interval is a global config option: * DirWatch/PollInterval and DirWatch/NFSPollInterval for NFS mounted * directories. Please consider removing kcoreaddons dependency on FAM and on gamin in the Debian package. kcoreaddons has its own fallbacks. Also, kcoreaddons is not using FAM or gamin on Linux. > FreeBSD doesn't have inotify, so there is a need for FAM, and maybe > someone from their community has become the defacto upstream (via their > "ports" packaging system)? Or maybe someone from their community would > be willing to officially become FAM's new upstream? FYI: *BSD has kqueue() with EVFILT_VNODE. Please consider removing FAM and gamin from the Debian package for kcoreaddons by removing libfam-dev and libgamin-dev from debian/control, or at least changing it to libgamin-dev. https://salsa.debian.org/qt-kde-team/kde/kcoreaddons/-/merge_requests/3 Cheers, Glenn
Bug#981515: kcoreaddons: please replace fam with gamin
Hi, Glenn Strauss writes: > gamin provides libfam0. > > kcoreaddons should load just fine with libfam0 from gamin. > > I did the research in #510368 and #966273, reviewing the actual code > and confidentally concluded that FAM can be removed from Bullseye. > > The safest choice is to have a single library (gamin) used in the > distro, rather than having both FAM and gamin. > I don't think the removal of FAM is as clear-cut as it has been presented to be. AFAIK the following is still current: Gamin provides "No NFS support based on specific RPC and server, instead gamin monitors only the state as reported locally by the kernel, not that locally done changes on NFS or AFS filesystems are reported on Linux which is the main criteria when having user home directories on such filesystems." https://people.gnome.org/~veillard/gamin/differences.html thus FAM covers a use case that gamin does not, and this case is: users who want to receive inotify style events for files that have been remotely created or modified on NFS mounts. I can't speak to how widespread the need for this feature is, but if it is non-zero then it seems to me that FAM should not be removed this late in the Bullseye cycle. Also, IIRC gamin depends on Linux-specific features such as inotify where FAM provides fallbacks (eg: IIRC FAM is required on kfreebsd and hurd); this might not be significant, but I felt it was worth mentioning :-) FreeBSD doesn't have inotify, so there is a need for FAM, and maybe someone from their community has become the defacto upstream (via their "ports" packaging system)? Or maybe someone from their community would be willing to officially become FAM's new upstream? Regards, Nicholas signature.asc Description: PGP signature
Bug#981515: kcoreaddons: please replace fam with gamin
gamin provides libfam0. kcoreaddons should load just fine with libfam0 from gamin. I did the research in #510368 and #966273, reviewing the actual code and confidentally concluded that FAM can be removed from Bullseye. The safest choice is to have a single library (gamin) used in the distro, rather than having both FAM and gamin. Please pick up the trivial change in https://salsa.debian.org/qt-kde-team/kde/kcoreaddons/-/merge_requests/3 so that kcoreaddons is not blocking progress. Thank you. Glenn
Bug#981515: kcoreaddons: please replace fam with gamin
* Pino Toscano [210201 00:28]: > In data lunedì 1 febbraio 2021 00:00:47 CET, Chris Hofstaedtler ha scritto: > I wouldn't say that gamin is more maintained than FAM: > - on the upstream side (gnome.org), it was moved the "Archive" section > of the gnome gitlab > - during the migration to gitlab, all the bugzilla tickets were closed > instead of migrated too, because the project is archived > - on the Debian side, gamin saw 4 uploads (one of them is an NMU) in > the last 10 years This is certainly true, but its better to ship one unmaintained thing than two. > - #510368 prevents to switch from FAM to gamin, as it will build against > libgamin but still pulling libfam (or keep an old libfam as valid > package satisfying the dependency) It will keep an old libfam as valid, yes. Maybe we can get that fixed though. > > Severity of this bug will probably raise over time. > > What is "over time" implying, please? I really hope this is not for > bullseye: we are so close to its freeze, and these FAM/gamin bits do > not get much of use these days, so the risk of release regressions seems > medium/high to me. Not for bullseye, at least not if you don't feel like it. But now I have to ask: why not drop the fam/gamin dependency, if it "does not get much of use these days"? For bookworm, that is. Chris
Bug#981515: kcoreaddons: please replace fam with gamin
In data lunedì 1 febbraio 2021 00:00:47 CET, Chris Hofstaedtler ha scritto: > Source: kcoreaddons > Version: 5.54.0-1 > > Dear Maintainer, > > your package currently Depends on libfam0 (fam), which has > been requested to be removed in #966273. Please switch to gamin > instead. You can find more info, and an analysis of what probably > needs to be done for your package in this message from Glenn Strauss: > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966273#15 I wouldn't say that gamin is more maintained than FAM: - on the upstream side (gnome.org), it was moved the "Archive" section of the gnome gitlab - during the migration to gitlab, all the bugzilla tickets were closed instead of migrated too, because the project is archived - on the Debian side, gamin saw 4 uploads (one of them is an NMU) in the last 10 years - #510368 prevents to switch from FAM to gamin, as it will build against libgamin but still pulling libfam (or keep an old libfam as valid package satisfying the dependency) > Severity of this bug will probably raise over time. What is "over time" implying, please? I really hope this is not for bullseye: we are so close to its freeze, and these FAM/gamin bits do not get much of use these days, so the risk of release regressions seems medium/high to me. -- Pino Toscano signature.asc Description: This is a digitally signed message part.
Bug#981515: kcoreaddons: please replace fam with gamin
Source: kcoreaddons Version: 5.54.0-1 Dear Maintainer, your package currently Depends on libfam0 (fam), which has been requested to be removed in #966273. Please switch to gamin instead. You can find more info, and an analysis of what probably needs to be done for your package in this message from Glenn Strauss: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966273#15 Severity of this bug will probably raise over time. Thanks, Chris
