Re: Announcing DNF 3 development
Matěj Cepl wrote: > When switching the programming langauge than I would think there > are some better C-successors than C++, namely Rust? Mad rush of > giving up on 46 years old language and switching to one which is > just 33 years old seems a bit bizarre to me. But the "33 years old" language WORKS. :-) As for why switching from C to C++, that's because C++ has some features that C does not have and that are useful for the DNF developers, in particular, native support for object orientation (without the GObject hack). Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018 at 12:10 PM, Martin Kolmanwrote: > On Mon, 2018-03-26 at 11:59 -0400, Neal Gompa wrote: >> On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruch wrote: >> > >> > >> > Dne 26.3.2018 v 10:16 Tom Hughes napsal(a): >> > > On 26/03/18 09:06, Marcin Juszkiewicz wrote: >> > > > W dniu 22.03.2018 o 13:40, Daniel Mach pisze: >> > > > > We are pleased to announce that development of DNF 3 has started. >> > > > > This >> > > > > version is focused on performance improvements, new API and >> > > > > consolidating >> > > > > the whole software management stack. >> > > > > >> > > > > Please read more details on our blog: >> > > > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >> > > > > >> > > > >> > > > Will it FINALLY use one copy of metadata for all system users? >> > > >> > > Do you have a proposal as to how that might be possible in a >> > > secure way? >> > >> > If DNF was just frontend for some service/daemon, that would be one >> > possibility. It would also help with other issues like updates of X >> > server crashing whole user session and therefore the update. >> > >> >> It would be nice if dnfdaemon was actually merged into dnf itself, but >> I'm not sure whether they'd consider it to be desirable or not. > I have a hunch there will be some gotchas, otherwise everybody would be doing > it. > > Maybe issues with chrooting or something like that ? We'd probably want a no-daemon switch for changing behavior, and trigger that change for circumstances where the daemon cannot run. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018, at 11:02 AM, Vít Ondruch wrote: > If DNF was just frontend for some service/daemon, that would be one > possibility. It would also help with other issues like updates of X > server crashing whole user session and therefore the update. FWIW rpm-ostree is always a daemon today; we're really focused on the "host system" management case so we can just assume that. Obviously there's PackageKit as well. That said I spent a while thinking about this a while ago, and the thing is a vast array of tooling assumes that package managers are CLI tools. Not least e.g. `Dockerfile` style container builds. And `mock`. So you'd really end up with something that splits up the "managing a host system" from "buildroot tool", which is how I think of rpm-ostreed versus dnf today. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, 2018-03-26 at 11:59 -0400, Neal Gompa wrote: > On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruchwrote: > > > > > > Dne 26.3.2018 v 10:16 Tom Hughes napsal(a): > > > On 26/03/18 09:06, Marcin Juszkiewicz wrote: > > > > W dniu 22.03.2018 o 13:40, Daniel Mach pisze: > > > > > We are pleased to announce that development of DNF 3 has started. This > > > > > version is focused on performance improvements, new API and > > > > > consolidating > > > > > the whole software management stack. > > > > > > > > > > Please read more details on our blog: > > > > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > > > > > > > > > > > > > Will it FINALLY use one copy of metadata for all system users? > > > > > > Do you have a proposal as to how that might be possible in a > > > secure way? > > > > If DNF was just frontend for some service/daemon, that would be one > > possibility. It would also help with other issues like updates of X > > server crashing whole user session and therefore the update. > > > > It would be nice if dnfdaemon was actually merged into dnf itself, but > I'm not sure whether they'd consider it to be desirable or not. I have a hunch there will be some gotchas, otherwise everybody would be doing it. Maybe issues with chrooting or something like that ? > > > -- > 真実はいつも一つ!/ Always, there's only one truth! > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018 at 11:02 AM, Vít Ondruchwrote: > > > Dne 26.3.2018 v 10:16 Tom Hughes napsal(a): >> On 26/03/18 09:06, Marcin Juszkiewicz wrote: >>> W dniu 22.03.2018 o 13:40, Daniel Mach pisze: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >>> >>> Will it FINALLY use one copy of metadata for all system users? >> >> Do you have a proposal as to how that might be possible in a >> secure way? > > If DNF was just frontend for some service/daemon, that would be one > possibility. It would also help with other issues like updates of X > server crashing whole user session and therefore the update. > It would be nice if dnfdaemon was actually merged into dnf itself, but I'm not sure whether they'd consider it to be desirable or not. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
Le lundi 26 mars 2018 à 17:02 +0200, Vít Ondruch a écrit : > > If DNF was just frontend for some service/daemon, that would be one > possibility. It would also help with other issues like updates of X > server crashing whole user session and therefore the update. Yes that would be pretty awesome -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
Dne 26.3.2018 v 10:16 Tom Hughes napsal(a): > On 26/03/18 09:06, Marcin Juszkiewicz wrote: >> W dniu 22.03.2018 o 13:40, Daniel Mach pisze: >>> We are pleased to announce that development of DNF 3 has started. This >>> version is focused on performance improvements, new API and >>> consolidating >>> the whole software management stack. >>> >>> Please read more details on our blog: >>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >>> >> >> Will it FINALLY use one copy of metadata for all system users? > > Do you have a proposal as to how that might be possible in a > secure way? If DNF was just frontend for some service/daemon, that would be one possibility. It would also help with other issues like updates of X server crashing whole user session and therefore the update. V. > >> I do not see how fetching megabytes of metadata is better than using >> copy present in /var/cache/dnf/ directory. > > Well it could just use the cached copy sure, but if it was out of > date then it wouldn't be able to update it? > >> It is faster to enter long password to use sudo than to wait until dnf >> fetch useless copy of metadata. > > Agreed, which is why I always run dnf under sudo even for query > operations. > >> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. > > Do we know how? Do they just not allow non-root users to get up > to date data, or do they do something cleverer? > > Tom > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018 at 9:58 AM, Martin Sehnoutkawrote: > > > On 03/26/2018 02:30 PM, Neal Gompa wrote: >> On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutka >> wrote: >>> >>> >>> On 03/26/2018 01:38 PM, Neal Gompa wrote: On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl wrote: > On 2018-03-26, 10:52 GMT, Florian Weimer wrote: >> On 03/22/2018 01:40 PM, Daniel Mach wrote: >>> Please read more details on our blog: >>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >> >> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should >> use Developer Toolset to compile on Red Hat Enterprise Linux >> 7 if you need C++11 support. The system compiler, GCC 4.8, >> has limited support only. > > When switching the programming langauge than I would think there > are some better C-successors than C++, namely Rust? Mad rush of > giving up on 46 years old language and switching to one which is > just 33 years old seems a bit bizarre to me. > >>> >>> Take a look into the code, it is mostly C with few features from C++. >>> >>> btw what is the motivation to use GOBjects? Is the libdnf api supposed >>> to be consumed by dnf frontend via gi repository? >>> >> >> It was a thought a while ago with libhif, and as part of the final >> rationalization for libdnf, it's being dropped. Because libdnf is >> going to be in C++, it's going to use SWIG for bindings generation. >> > > Thanks for clarification. > I'm okay with not dealing with LLVM for my system package manager, thank you very much. I'd be more open to Rust if Rust also could be built with GCC, and thus supported across literally everything, but no one is investing in that effort. >>> >>> Well, investment like this will need some justification, not saying that >>> dnf should be the one, but you will definitely need a big, important >>> project. >>> >> >> Considering all the other "big important things" people don't invest >> in anyway, I don't think that'd help any. >> And frankly, Rust is harder to program in than C++, and creating bindings is no walk in the park. >>> >>> Purely personal opinion. You are probably referring to the learning >>> curve, which is known to be steep, but after this period it is well >>> worth the effort. >>> >> >> Not my personal opinion. That's the opinion of several developers I >> know who are working on Rust based projects. Not everyone gets the >> benefit of GNOME forcing all the things so that stuff _must_ work. >> > > I don't really get the last sentence. What is GNOME forcing a what must > work? > Basically, when you work outside of the GNOME ecosystem, things get much harder because you can't guarantee everything interfaces through GObject and other stuff like it. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/26/2018 02:30 PM, Neal Gompa wrote: > On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutkawrote: >> >> >> On 03/26/2018 01:38 PM, Neal Gompa wrote: >>> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl wrote: On 2018-03-26, 10:52 GMT, Florian Weimer wrote: > On 03/22/2018 01:40 PM, Daniel Mach wrote: >> Please read more details on our blog: >> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > > “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should > use Developer Toolset to compile on Red Hat Enterprise Linux > 7 if you need C++11 support. The system compiler, GCC 4.8, > has limited support only. When switching the programming langauge than I would think there are some better C-successors than C++, namely Rust? Mad rush of giving up on 46 years old language and switching to one which is just 33 years old seems a bit bizarre to me. >> >> Take a look into the code, it is mostly C with few features from C++. >> >> btw what is the motivation to use GOBjects? Is the libdnf api supposed >> to be consumed by dnf frontend via gi repository? >> > > It was a thought a while ago with libhif, and as part of the final > rationalization for libdnf, it's being dropped. Because libdnf is > going to be in C++, it's going to use SWIG for bindings generation. > Thanks for clarification. >>> >>> I'm okay with not dealing with LLVM for my system package manager, >>> thank you very much. I'd be more open to Rust if Rust also could be >>> built with GCC, and thus supported across literally everything, but no >>> one is investing in that effort. >>> >> >> Well, investment like this will need some justification, not saying that >> dnf should be the one, but you will definitely need a big, important >> project. >> > > Considering all the other "big important things" people don't invest > in anyway, I don't think that'd help any. > >>> And frankly, Rust is harder to program in than C++, and creating >>> bindings is no walk in the park. >>> >> >> Purely personal opinion. You are probably referring to the learning >> curve, which is known to be steep, but after this period it is well >> worth the effort. >> > > Not my personal opinion. That's the opinion of several developers I > know who are working on Rust based projects. Not everyone gets the > benefit of GNOME forcing all the things so that stuff _must_ work. > I don't really get the last sentence. What is GNOME forcing a what must work? >> Regarding the bindings, if libdnf is meant to be used via gir (see my >> question above), then there is already an effort to make this much >> easier (I'm referring to gnome-class). >> > > As I noted earlier in this email, gir is a leftover and is being removed. > -- Martin Sehnoutka | Associate Software Engineer PGP: 5FD64AF5 UTC+1 (CET) RED HAT | TRIED. TESTED. TRUSTED. signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018 at 8:26 AM, Martin Sehnoutkawrote: > > > On 03/26/2018 01:38 PM, Neal Gompa wrote: >> On Mon, Mar 26, 2018 at 7:22 AM, Matěj Cepl wrote: >>> On 2018-03-26, 10:52 GMT, Florian Weimer wrote: On 03/22/2018 01:40 PM, Daniel Mach wrote: > Please read more details on our blog: > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should use Developer Toolset to compile on Red Hat Enterprise Linux 7 if you need C++11 support. The system compiler, GCC 4.8, has limited support only. >>> >>> When switching the programming langauge than I would think there >>> are some better C-successors than C++, namely Rust? Mad rush of >>> giving up on 46 years old language and switching to one which is >>> just 33 years old seems a bit bizarre to me. >>> > > Take a look into the code, it is mostly C with few features from C++. > > btw what is the motivation to use GOBjects? Is the libdnf api supposed > to be consumed by dnf frontend via gi repository? > It was a thought a while ago with libhif, and as part of the final rationalization for libdnf, it's being dropped. Because libdnf is going to be in C++, it's going to use SWIG for bindings generation. >> >> I'm okay with not dealing with LLVM for my system package manager, >> thank you very much. I'd be more open to Rust if Rust also could be >> built with GCC, and thus supported across literally everything, but no >> one is investing in that effort. >> > > Well, investment like this will need some justification, not saying that > dnf should be the one, but you will definitely need a big, important > project. > Considering all the other "big important things" people don't invest in anyway, I don't think that'd help any. >> And frankly, Rust is harder to program in than C++, and creating >> bindings is no walk in the park. >> > > Purely personal opinion. You are probably referring to the learning > curve, which is known to be steep, but after this period it is well > worth the effort. > Not my personal opinion. That's the opinion of several developers I know who are working on Rust based projects. Not everyone gets the benefit of GNOME forcing all the things so that stuff _must_ work. > Regarding the bindings, if libdnf is meant to be used via gir (see my > question above), then there is already an effort to make this much > easier (I'm referring to gnome-class). > As I noted earlier in this email, gir is a leftover and is being removed. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, 2018-03-26 at 13:22 +0200, Matěj Cepl wrote: > On 2018-03-26, 10:52 GMT, Florian Weimer wrote: > > On 03/22/2018 01:40 PM, Daniel Mach wrote: > > > Please read more details on our blog: > > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > > > > “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should > > use Developer Toolset to compile on Red Hat Enterprise Linux > > 7 if you need C++11 support. The system compiler, GCC 4.8, > > has limited support only. > > When switching the programming langauge than I would think there > are some better C-successors than C++, namely Rust? Mad rush of > giving up on 46 years old language and switching to one which is > just 33 years old seems a bit bizarre to me. I think it is not bad to be a bit conservative when core system components are concerned. > > Best, > > Matěj > -- > https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz > GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 > > I am a Roman Catholic, so that I do not expect `history' to be > anything but a `long defeat' -- though it contains (and in > a legend may contain more clearly and movingly) some samples or > glimpses of final victory. > -- J.R.R. Tolkien > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/26/2018 01:38 PM, Neal Gompa wrote: > On Mon, Mar 26, 2018 at 7:22 AM, Matěj Ceplwrote: >> On 2018-03-26, 10:52 GMT, Florian Weimer wrote: >>> On 03/22/2018 01:40 PM, Daniel Mach wrote: Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >>> >>> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should >>> use Developer Toolset to compile on Red Hat Enterprise Linux >>> 7 if you need C++11 support. The system compiler, GCC 4.8, >>> has limited support only. >> >> When switching the programming langauge than I would think there >> are some better C-successors than C++, namely Rust? Mad rush of >> giving up on 46 years old language and switching to one which is >> just 33 years old seems a bit bizarre to me. >> Take a look into the code, it is mostly C with few features from C++. btw what is the motivation to use GOBjects? Is the libdnf api supposed to be consumed by dnf frontend via gi repository? > > I'm okay with not dealing with LLVM for my system package manager, > thank you very much. I'd be more open to Rust if Rust also could be > built with GCC, and thus supported across literally everything, but no > one is investing in that effort. > Well, investment like this will need some justification, not saying that dnf should be the one, but you will definitely need a big, important project. > And frankly, Rust is harder to program in than C++, and creating > bindings is no walk in the park. > Purely personal opinion. You are probably referring to the learning curve, which is known to be steep, but after this period it is well worth the effort. Regarding the bindings, if libdnf is meant to be used via gir (see my question above), then there is already an effort to make this much easier (I'm referring to gnome-class). -- Martin Sehnoutka | Associate Software Engineer PGP: 5FD64AF5 UTC+1 (CET) RED HAT | TRIED. TESTED. TRUSTED. signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, Mar 26, 2018 at 7:22 AM, Matěj Ceplwrote: > On 2018-03-26, 10:52 GMT, Florian Weimer wrote: >> On 03/22/2018 01:40 PM, Daniel Mach wrote: >>> Please read more details on our blog: >>> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ >> >> “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should >> use Developer Toolset to compile on Red Hat Enterprise Linux >> 7 if you need C++11 support. The system compiler, GCC 4.8, >> has limited support only. > > When switching the programming langauge than I would think there > are some better C-successors than C++, namely Rust? Mad rush of > giving up on 46 years old language and switching to one which is > just 33 years old seems a bit bizarre to me. > I'm okay with not dealing with LLVM for my system package manager, thank you very much. I'd be more open to Rust if Rust also could be built with GCC, and thus supported across literally everything, but no one is investing in that effort. And frankly, Rust is harder to program in than C++, and creating bindings is no walk in the park. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 2018-03-26, 10:52 GMT, Florian Weimer wrote: > On 03/22/2018 01:40 PM, Daniel Mach wrote: >> Please read more details on our blog: >> https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > > “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should > use Developer Toolset to compile on Red Hat Enterprise Linux > 7 if you need C++11 support. The system compiler, GCC 4.8, > has limited support only. When switching the programming langauge than I would think there are some better C-successors than C++, namely Rust? Mad rush of giving up on 46 years old language and switching to one which is just 33 years old seems a bit bizarre to me. Best, Matěj -- https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 I am a Roman Catholic, so that I do not expect `history' to be anything but a `long defeat' -- though it contains (and in a legend may contain more clearly and movingly) some samples or glimpses of final victory. -- J.R.R. Tolkien ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/22/2018 01:40 PM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ “C++ 11 is supported by GCC in RHEL 7 / CentOS 7” — You should use Developer Toolset to compile on Red Hat Enterprise Linux 7 if you need C++11 support. The system compiler, GCC 4.8, has limited support only. Thanks, Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, 2018-03-26 at 11:44 +0300, Jonathan Dieter wrote: > I've been working on zchunk which will allow downloading only the delta > of the metadata. For reference, the proposal starts at http://lists.rpm.org/pipermail/rp m-ecosystem/2018-February/000534.html. Jonathan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
> On 26. Mar 2018, at 10:39, Oron Peledwrote: > > On Monday, 26 March 2018 11:16:14 IDT Tom Hughes wrote: >> On 26/03/18 09:06, Marcin Juszkiewicz wrote: >>> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. >> >> Do we know how? Do they just not allow non-root users to get up >> to date data, or do they do something cleverer? > > With APT, cache update and using the cache are orthogonal operations: > * Running "apt-get update" is privileged and update the cache. > * There's obviously an optional service to run "apt-get update" periodically > if root prefer this mechanism. > * Doing just query (e.g: apt list ...) always use latest cache data. > * So query by root or any user are the same -- no privileged and work on the > cache (which may/may-not be updated). Question: Would splitting-up cache update and package upgrade operations worth considering for dnf 3.0? (As a long time CentOS, Fedora user I am always envious at how snappy (no pun intended) apt-get update && apt-get upgrade feel when I get my hands on one of these :-) BK ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Mon, 2018-03-26 at 09:30 +0100, Richard Hughes wrote: > With version 3.0 dnf is switching to the codebase we originally wrote > for PackageKit (libzif->libhif->libdnf), and so it inherits the > "download complete cache and switch atomically" logic. This means we > get a shared cache for free. I've been working on zchunk which will allow downloading only the delta of the metadata. AIUI, currently dnf uses librepo to download the metadata, so that's where I was planning to do the zchunk integration. Is there somewhere else I should be looking at for dnf 3.0, or is it still using librepo to do the actual downloads? Jonathan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Monday, 26 March 2018 11:16:14 IDT Tom Hughes wrote: > On 26/03/18 09:06, Marcin Juszkiewicz wrote: > > Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. > > Do we know how? Do they just not allow non-root users to get up > to date data, or do they do something cleverer? With APT, cache update and using the cache are orthogonal operations: * Running "apt-get update" is privileged and update the cache. * There's obviously an optional service to run "apt-get update" periodically if root prefer this mechanism. * Doing just query (e.g: apt list ...) always use latest cache data. * So query by root or any user are the same -- no privileged and work on the cache (which may/may-not be updated). KISS, -- Oron Peled Voice: +972-4-8228492 May the Source be with you! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 26/03/18 09:30, Richard Hughes wrote: On 26 March 2018 at 09:16, Tom Hugheswrote: Will it FINALLY use one copy of metadata for all system users? Do you have a proposal as to how that might be possible in a secure way? I think he means removing the duplication of the cache between PackageKit and dnf. The former only uses a separate cache as it updates the cache all at one, and switches to new versions atomically so applications like gnome-software can always run transactions with known latencies. dnf (the command line tool) only downloaded the metadata files it needed for the current request, and so it could be you had to start downloading something like the filelists when you actually depsolved a transaction (which is somewhat incompatible with offline updates for instance). Well that's an issue as well, but he specifically referred to the per-user caching behaviour of dnf and the need to run it under sudo to use the system cache instead of building a new one. In other words if I do "dnf search foo" as myself it will fetch a new cache from scratch into /tmp instead of using the system one. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
W dniu 26.03.2018 o 10:16, Tom Hughes pisze: > On 26/03/18 09:06, Marcin Juszkiewicz wrote: >> Will it FINALLY use one copy of metadata for all system users? > > Do you have a proposal as to how that might be possible in a > secure way? Use. Not download. And inform (like always) how old that data is. >> I do not see how fetching megabytes of metadata is better than using >> copy present in /var/cache/dnf/ directory. > > Well it could just use the cached copy sure, but if it was out of > date then it wouldn't be able to update it? Then it is out-of-date. But still is. And not everyone runs rawhide to make it a difference when metadata is few days old. >> It is faster to enter long password to use sudo than to wait until dnf >> fetch useless copy of metadata. > > Agreed, which is why I always run dnf under sudo even for query > operations. I hate having to do that. Also dislike fact that if I do not do that then have to wait and wait and wait until it finally fetch metadata to show me result. DNF started as user can not install packages so why it can not just use systemwide metadata? >> Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. > > Do we know how? Do they just not allow non-root users to get up > to date data, or do they do something cleverer? APT has separate command for updating metadata. YUM/DNF tries to do that every time they think that metadata is too old. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 26 March 2018 at 09:16, Tom Hugheswrote: >> Will it FINALLY use one copy of metadata for all system users? > Do you have a proposal as to how that might be possible in a > secure way? I think he means removing the duplication of the cache between PackageKit and dnf. The former only uses a separate cache as it updates the cache all at one, and switches to new versions atomically so applications like gnome-software can always run transactions with known latencies. dnf (the command line tool) only downloaded the metadata files it needed for the current request, and so it could be you had to start downloading something like the filelists when you actually depsolved a transaction (which is somewhat incompatible with offline updates for instance). With version 3.0 dnf is switching to the codebase we originally wrote for PackageKit (libzif->libhif->libdnf), and so it inherits the "download complete cache and switch atomically" logic. This means we get a shared cache for free. Richard. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 26/03/18 09:06, Marcin Juszkiewicz wrote: W dniu 22.03.2018 o 13:40, Daniel Mach pisze: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ Will it FINALLY use one copy of metadata for all system users? Do you have a proposal as to how that might be possible in a secure way? I do not see how fetching megabytes of metadata is better than using copy present in /var/cache/dnf/ directory. Well it could just use the cached copy sure, but if it was out of date then it wouldn't be able to update it? It is faster to enter long password to use sudo than to wait until dnf fetch useless copy of metadata. Agreed, which is why I always run dnf under sudo even for query operations. Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. Do we know how? Do they just not allow non-root users to get up to date data, or do they do something cleverer? Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
W dniu 22.03.2018 o 13:40, Daniel Mach pisze: > We are pleased to announce that development of DNF 3 has started. This > version is focused on performance improvements, new API and consolidating > the whole software management stack. > > Please read more details on our blog: > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ Will it FINALLY use one copy of metadata for all system users? I do not see how fetching megabytes of metadata is better than using copy present in /var/cache/dnf/ directory. It is faster to enter long password to use sudo than to wait until dnf fetch useless copy of metadata. 10:01 (1s) hrw@puchatek:~$ LANGUAGE=C LANG=C COLUMNS=60 dnf list nano Spotify (negativo17) 31 kB/s | 11 kB 00:00 Fedora 27 - x86_64 - Update 8.4 MB/s | 21 MB 00:02 Fedora 27 - x86_64 15 MB/s | 58 MB 00:03 Google Chrome (stable) 57 kB/s | 3.7 kB 00:00 google-earth 72 kB/s | 4.7 kB 00:00 RPM Fusion for Fedora 27 - 984 kB/s | 351 kB 00:00 RPM Fusion for Fedora 27 - 1.3 MB/s | 717 kB 00:00 RPM Fusion for Fedora 27 - 50 kB/s | 87 kB 00:01 RPM Fusion for Fedora 27 - 128 kB/s | 163 kB 00:01 Skype Repository 40 kB/s | 1.8 kB 00:00 Last metadata expiration check: 0:00:00 ago on Mon Mar 26 10:03:01 2018. Available Packages nano.x86_64 2.8.7-1.fc27fedora 10:02 (40s) hrw@puchatek:~$ 40 seconds just because of good download speed. Debian has APT. APT uses one copy of metadata. Be wise. Like Debian. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Fri, 2018-03-23 at 10:55 -0500, Ian Pilcher wrote: > On 03/22/2018 07:40 AM, Daniel Mach wrote: > > We are pleased to announce that development of DNF 3 has started. This > > version is focused on performance improvements, new API and > > consolidating the whole software management stack. > > > > Please read more details on our blog: > > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > > Can someone explain how DNF 2 can be considered "finished", when it > still can't provide the dependency information that yum did? Neither the mail nor the announcement makes any claim that DNF 2 is considered "finished". -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/22/2018 07:40 AM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ Can someone explain how DNF 2 can be considered "finished", when it still can't provide the dependency information that yum did? https://bugzilla.redhat.com/show_bug.cgi?id=1549851 -- Ian Pilcher arequip...@gmail.com "I grew up before Mark Zuckerberg invented friendship" ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Thu, Mar 22, 2018, 23:13 Nico Kadel-Garciawrote: > On Thu, Mar 22, 2018 at 5:49 PM, John Reiser wrote: > > On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote: > >> > >> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser > >> wrote: > >>> > >>> On 03/22/2018 05:40 AM, Daniel Mach wrote: > > > We are pleased to announce that development of DNF 3 has started. This > version is focused on performance improvements, new API and > consolidating > the whole software management stack. > >>> > >>> > >>> > >>> How does RPM fit into DNF's view of "the whole software management > >>> stack"? > >>> RPM is a slug (moves very slowly): no parallelism (at any point all > >>> packages > >>> with no remaining predecessors could be updated/installed in parallel), > >>> not even manually pipelined (decompress to memory, manipulate > filesystem, > >>> update database.) > >> > >> > >> Parallelizing software updates or installations would be *begging* for > >> pain. It would be difficult for me to recommend strongly enough > >> against this. > > > > > > Please be specific about the pain points that you fear. > > RPM, itself, is single threaded. > > %pre and %post operations would have to be re-evaluated for > parallelization. system account creation, in particular, would have to > be made thread safe. > > RPM installation can fail partly through deployment due to SELinux, > disk space, or network based mount point failure: keeping it single > threaded makes it much safer to unravel failed or partial RPM > installation. > > Unweaving partial dependency deployment could be quite destructive > with a parallelized approach. > > Daemons that need to be restarted and may have incompatible component > updates, such httpd with its modules, are particularly vulnerable to > fascinating failures from the daemon restarting with only some updated > components. Avoiding that would seem to require even more dependency > management for RPM installation, rather than each update itself > triggering an update. > > > The three-stage "manual" pipeline achieves 2x to 3x faster throughput > > with error states that are isomorphic to present RPM. (Consider the > > Turning machine model: if you don't write to the filesystem, then > > there is no change of external state.) > > Turing machines don't have to deal with all the possible > > > The "parallelize everything that has no remaining predecessors" strategy > > requires parallel transactions in the database (they cannot interfere > > because that would be a predecessor constraint) and checking for > > resource exhaustion (file space, inodes, etc.) as a global > > predecessor constraint. What else? > > Parallelizing the installations means losing the milestones at which > one update has succeeded, and the second update has not. Unweaving > that to find out which update triggered the failure sounds like pain, > and makes testing the update process more difficult. It becomes > difficult to manage or guess what the state of the system was at the > time of the RPM update, since another RPM update may be in progress at > the time. > > There is an infamous quote by Donald Knuth that "premature > optimization is the root of all evil". There are systems that benefit > the time benefits of parallelization, but for ordinary RPM > installations and system updates, I think that the slow update time is > because of other factors, such as disk IO and download time of > repodata, RPM database updates, and download times for the packages. > Well ... if you want to take this argument to the extreme, an update process where the update is downloaded and prepared in the background, and then applied atomically, would be the most "efficient" (because it happens instantaneous, with a reboot) and resilient against errors (at no point in time there is a broken, half-updated system) - just like rpm-ostree does things ... *ducks* ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Thu, Mar 22, 2018 at 5:49 PM, John Reiserwrote: > On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote: >> >> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser >> wrote: >>> >>> On 03/22/2018 05:40 AM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. >>> >>> >>> >>> How does RPM fit into DNF's view of "the whole software management >>> stack"? >>> RPM is a slug (moves very slowly): no parallelism (at any point all >>> packages >>> with no remaining predecessors could be updated/installed in parallel), >>> not even manually pipelined (decompress to memory, manipulate filesystem, >>> update database.) >> >> >> Parallelizing software updates or installations would be *begging* for >> pain. It would be difficult for me to recommend strongly enough >> against this. > > > Please be specific about the pain points that you fear. RPM, itself, is single threaded. %pre and %post operations would have to be re-evaluated for parallelization. system account creation, in particular, would have to be made thread safe. RPM installation can fail partly through deployment due to SELinux, disk space, or network based mount point failure: keeping it single threaded makes it much safer to unravel failed or partial RPM installation. Unweaving partial dependency deployment could be quite destructive with a parallelized approach. Daemons that need to be restarted and may have incompatible component updates, such httpd with its modules, are particularly vulnerable to fascinating failures from the daemon restarting with only some updated components. Avoiding that would seem to require even more dependency management for RPM installation, rather than each update itself triggering an update. > The three-stage "manual" pipeline achieves 2x to 3x faster throughput > with error states that are isomorphic to present RPM. (Consider the > Turning machine model: if you don't write to the filesystem, then > there is no change of external state.) Turing machines don't have to deal with all the possible > The "parallelize everything that has no remaining predecessors" strategy > requires parallel transactions in the database (they cannot interfere > because that would be a predecessor constraint) and checking for > resource exhaustion (file space, inodes, etc.) as a global > predecessor constraint. What else? Parallelizing the installations means losing the milestones at which one update has succeeded, and the second update has not. Unweaving that to find out which update triggered the failure sounds like pain, and makes testing the update process more difficult. It becomes difficult to manage or guess what the state of the system was at the time of the RPM update, since another RPM update may be in progress at the time. There is an infamous quote by Donald Knuth that "premature optimization is the root of all evil". There are systems that benefit the time benefits of parallelization, but for ordinary RPM installations and system updates, I think that the slow update time is because of other factors, such as disk IO and download time of repodata, RPM database updates, and download times for the packages. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 22 March 2018 at 17:49, John Reiserwrote: > On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote: >> >> On Thu, Mar 22, 2018 at 10:52 AM, John Reiser >> wrote: >>> >>> On 03/22/2018 05:40 AM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. >>> >>> >>> >>> How does RPM fit into DNF's view of "the whole software management >>> stack"? >>> RPM is a slug (moves very slowly): no parallelism (at any point all >>> packages >>> with no remaining predecessors could be updated/installed in parallel), >>> not even manually pipelined (decompress to memory, manipulate filesystem, >>> update database.) >> >> >> Parallelizing software updates or installations would be *begging* for >> pain. It would be difficult for me to recommend strongly enough >> against this. > > > Please be specific about the pain points that you fear. > > The three-stage "manual" pipeline achieves 2x to 3x faster throughput > with error states that are isomorphic to present RPM. (Consider the > Turning machine model: if you don't write to the filesystem, then > there is no change of external state.) > > The "parallelize everything that has no remaining predecessors" strategy > requires parallel transactions in the database (they cannot interfere > because that would be a predecessor constraint) and checking for > resource exhaustion (file space, inodes, etc.) as a global > predecessor constraint. What else? > Having some way for triggers/postun/pre etc scripts to know they actually interfere with a predecessor constraint. This would mean itemizing everything in every script which could be run during these steps and working out blocks/conflicts/etc. [AKA if bash is not there when a parallel scriplet runs... boom.] [Not saying it is impossible.. but it might mean a multipass update where certain things are updated, then the next set and then the third ... ] > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/22/2018 01:51 PM, Nico Kadel-Garcia wrote: On Thu, Mar 22, 2018 at 10:52 AM, John Reiserwrote: On 03/22/2018 05:40 AM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. How does RPM fit into DNF's view of "the whole software management stack"? RPM is a slug (moves very slowly): no parallelism (at any point all packages with no remaining predecessors could be updated/installed in parallel), not even manually pipelined (decompress to memory, manipulate filesystem, update database.) Parallelizing software updates or installations would be *begging* for pain. It would be difficult for me to recommend strongly enough against this. Please be specific about the pain points that you fear. The three-stage "manual" pipeline achieves 2x to 3x faster throughput with error states that are isomorphic to present RPM. (Consider the Turning machine model: if you don't write to the filesystem, then there is no change of external state.) The "parallelize everything that has no remaining predecessors" strategy requires parallel transactions in the database (they cannot interfere because that would be a predecessor constraint) and checking for resource exhaustion (file space, inodes, etc.) as a global predecessor constraint. What else? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On Thu, Mar 22, 2018 at 10:52 AM, John Reiserwrote: > On 03/22/2018 05:40 AM, Daniel Mach wrote: >> >> We are pleased to announce that development of DNF 3 has started. This >> version is focused on performance improvements, new API and consolidating >> the whole software management stack. > > > How does RPM fit into DNF's view of "the whole software management stack"? > RPM is a slug (moves very slowly): no parallelism (at any point all packages > with no remaining predecessors could be updated/installed in parallel), > not even manually pipelined (decompress to memory, manipulate filesystem, > update database.) Parallelizing software updates or installations would be *begging* for pain. It would be difficult for me to recommend strongly enough against this. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Announcing DNF 3 development
On 03/22/2018 05:40 AM, Daniel Mach wrote: We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. How does RPM fit into DNF's view of "the whole software management stack"? RPM is a slug (moves very slowly): no parallelism (at any point all packages with no remaining predecessors could be updated/installed in parallel), not even manually pipelined (decompress to memory, manipulate filesystem, update database.) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [Rpm-ecosystem] Announcing DNF 3 development
On Thu, Mar 22, 2018 at 8:40 AM, Daniel Machwrote: > We are pleased to announce that development of DNF 3 has started. This > version is focused on performance improvements, new API and consolidating > the whole software management stack. > > Please read more details on our blog: > https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ > Congratulations! I look forward to integrating the first releases of the DNF 3.x stack in Mageia, OpenMandriva, and openSUSE. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Announcing DNF 3 development
We are pleased to announce that development of DNF 3 has started. This version is focused on performance improvements, new API and consolidating the whole software management stack. Please read more details on our blog: https://rpm-software-management.github.io/announcement/2018/03/22/dnf-3-announcement/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org