Re: [Discussion] Future of the CLI APP packaging
Hi, Jiri Konecny writes: > Dne 07. 01. 22 v 21:54 Major Hayden napsal(a): >> On 1/7/22 14:46, Jiri Konecny wrote: >>> I would like to do here a bit of brainstorming and ask if there is an >>> existing solution to this problem. To explain my problem, >>> recently I found more and more apps likes terminals, prompt and >>> command line tooling, which are great to use but not >>> packaged or are outdated in our repositories. The reason for this is >>> simple. These apps are written in languages like Rust or >>> Go which works the way that there are plenty of smaller libraries and >>> these are really a hell to package or maintain in >>> Fedora. On the other hand it's pretty simple to build it from the >>> source (most of the time) but hard to split it into packages. >> >> Oh gosh, yes, this has been a problem for me for a while. I maintain >> azure-cli, which has a few subpackages. It depends on about 95 other >> Azure Python SDK components which depend on various Python libraries. >> Each of these must be carefully updated when the main azure-cli >> package is updated. 🥵 >> >>> I would say, that for GUI apps we already done that. We have Flatpak >>> which is doing great job to get these apps there >>> and motivate people to do that because it's then consumable on plenty >>> of places. However, this is not really usable if you >>> need app which is core part of the system (like prompt) or emacs/vim >>> which needs a lot of dependencies from the system >>> based on user configuration. >> >> I do like Flatpaks for GUI applications. >> >> For CLIs, I've seen people use container images since it's a single >> item that is easily updated. However, it's not always easy to >> determine how a container was built and the home of its sources. 😬 >> >> The cloud vendors seem to be moving towards bundling everything up >> into a zip file that you download and run. AWS now has their own >> crypto/hashing components and bundled Python in a zip. Google has a >> large zip with plenty of third party vendored content. > Bundling to ZIP is again the same issue. You have to download it from > somewhere and from my PoV that is a bigger problem than not being able > to validate the content. I am afraid that we will not get around bundling dependencies in the long haul if we want packaging to still be a doable effort. There however kind off a middle ground between "package everything" and "bundle everything", that is currently pursued by the openSUSE project for node.js: allow the build system to "inject" system packages into a npm install via a local registry. That would allow packagers to still bundle most of the non-critical™ packages, while the security relevant ones could be maintained as rpms and get more attention. Iirc the plan was to provide this functionality via https://github.com/openSUSE/npm-localhost-proxy, but I am not sure if it's already implemented or whether that's the correct project. Cheers, Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: [Discussion] Future of the CLI APP packaging
Dne 08. 01. 22 v 15:28 Fabio Valentini napsal(a): On Sat, Jan 8, 2022 at 3:08 PM Jiri Konecny wrote: (snip) I really can't blame the people. Not everyone has the week of time to package all the dependencies one by one. But COPR is a nice shortcut to that so I can have what the upstream is producing without spending so much time and just re-using binaries produced upstream. The same works for pinging people to update their packages. On the other hand ... if you don't have the time to properly maintain a package, maybe you shouldn't push it to the official Fedora repositories at all? This only results in the worst outcome: Frustration for the packager (don't have time to properly update) *and* users (package in Fedora is out of date). Pushing things in a not-entirely-packaging-guidelines-compliant-but-maintainable-with-limited-time form to COPR would be a way better solution here from the start. I understand your point but this kind of packaging is starting to be so demanding that I fear that only a few people will be able to have packages in the Fedora in the future. And those will be pretty overloaded. Best Regards, Jirka ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: [Discussion] Future of the CLI APP packaging
On Sat, Jan 8, 2022 at 3:08 PM Jiri Konecny wrote: > (snip) > > I really can't blame the people. Not everyone has the week of time to > package all the dependencies > one by one. But COPR is a nice shortcut to that so I can have what the > upstream is producing > without spending so much time and just re-using binaries produced > upstream. The same works for > pinging people to update their packages. On the other hand ... if you don't have the time to properly maintain a package, maybe you shouldn't push it to the official Fedora repositories at all? This only results in the worst outcome: Frustration for the packager (don't have time to properly update) *and* users (package in Fedora is out of date). Pushing things in a not-entirely-packaging-guidelines-compliant-but-maintainable-with-limited-time form to COPR would be a way better solution here from the start. Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: [Discussion] Future of the CLI APP packaging
Dne 08. 01. 22 v 1:08 Fabio Valentini napsal(a): On Fri, Jan 7, 2022 at 9:48 PM Jiri Konecny wrote: Hi everyone, I would like to do here a bit of brainstorming and ask if there is an existing solution to this problem. To explain my problem, recently I found more and more apps likes terminals, prompt and command line tooling, which are great to use but not packaged or are outdated in our repositories. The reason for this is simple. These apps are written in languages like Rust or Go which works the way that there are plenty of smaller libraries and these are really a hell to package or maintain in Fedora. On the other hand it's pretty simple to build it from the source (most of the time) but hard to split it into packages. Hello! (I'd argue that most popular Rust libraries really are not small at all. Some are, but some are large. Most are medium-sized, as Rust does not have the same incentives as, for example, NodeJS, to publish single-function-modules.) I did not programmed much in Rust so I can't tell. :) Great example of this is Startship command line prompt, we have outdated version but when you look to dependencies[1] you have to completely understand. There is a slight problem with picking starship as an example here. Because it is almost certainly the Rust application with the largest number of (direct) dependencies. The starship project also frequently adds new dependencies with new versions, and aggressively uses new major versions of dependencies, both of which make updating it very painful (which is also why the package maintainer seemingly gave up on the Fedora package and now only updates it in COPR). Another example is chezmoi: https://github.com/twpayne/chezmoi/blob/master/go.mod#L5 Not packaged in Fedora at all. However, a great tool for solving your dotfiles. Upstream recommended way how to install this -> curl install script to bash... :( However, they also have RPM files (without repository). I don't want to start here a discussion if that is insecure or a good idea. That is something which has to be solved by people creating the ecosystem around these languages. What I would like to discuss here is how Fedora could improve the situation. Where you see a technological problem, I see a deeper problem with the way we do package maintenance. Many packagers came to the Rust ecosystem because they wanted to add "that one cool new application" to Fedora. But they usually don't have the time to care about their dependencies too, other than maybe doing the initial packaging and then throwing them over the rust-sig wall. On the other hand, there's people like me, who try to keep the lower-level libraries up-to-date, monitor security advisories, etc., but obviously I can't care about updating shiny application X or the-new-hotness Y as well - because I just don't have the time. And I don't think making sure somebody else can update their package is my responsibility. That disconnect between "application maintainers" and "library maintainers" is often frustrating. Application maintainers grumble about having to file new package review tickets, and that their dependencies are out of date. Library maintainers often update libraries to new major versions only "as needed", because those major changes often involve creating compat packages or pushing coordinated updates of all dependents of a library. But often the "I can't update my shiny application X to the new version! The libraries are out of date, you lazy library maintainers!" (I'm exaggarating, but you get the point) is the result of missing "major" updates that just weren't needed (or possible!) before shiny application X needed the new version. Additionally, some application maintainers are now no longer actively maintaining their Fedora packages, but instead push updates only to COPR, where they don't need to follow Packaging Guidelines. starship is an example here [1]. While the COPR is up-to-date (with using bundled dependencies), the Fedora package could be described as unmaintained. I really can't blame the people. Not everyone has the week of time to package all the dependencies one by one. But COPR is a nice shortcut to that so I can have what the upstream is producing without spending so much time and just re-using binaries produced upstream. The same works for pinging people to update their packages. [1]: https://copr.fedorainfracloud.org/coprs/atim/starship/ I would say, that for GUI apps we already done that. We have Flatpak which is doing great job to get these apps there and motivate people to do that because it's then consumable on plenty of places. However, this is not really usable if you need app which is core part of the system (like prompt) or emacs/vim which needs a lot of dependencies from the system based on user configuration. My question is, what we can do to avoid situations as in Windows? I mean that first thing what Win users are doing is to look on internet and start to download
Re: [Discussion] Future of the CLI APP packaging
On Fri, Jan 7, 2022 at 9:48 PM Jiri Konecny wrote: > > Hi everyone, > > I would like to do here a bit of brainstorming and ask if there is an > existing solution to this problem. To explain my problem, > recently I found more and more apps likes terminals, prompt and command > line tooling, which are great to use but not > packaged or are outdated in our repositories. The reason for this is > simple. These apps are written in languages like Rust or > Go which works the way that there are plenty of smaller libraries and > these are really a hell to package or maintain in > Fedora. On the other hand it's pretty simple to build it from the source > (most of the time) but hard to split it into packages. Hello! (I'd argue that most popular Rust libraries really are not small at all. Some are, but some are large. Most are medium-sized, as Rust does not have the same incentives as, for example, NodeJS, to publish single-function-modules.) > Great example of this is Startship command line prompt, we have outdated > version but when you look to dependencies[1] > you have to completely understand. There is a slight problem with picking starship as an example here. Because it is almost certainly the Rust application with the largest number of (direct) dependencies. The starship project also frequently adds new dependencies with new versions, and aggressively uses new major versions of dependencies, both of which make updating it very painful (which is also why the package maintainer seemingly gave up on the Fedora package and now only updates it in COPR). > I don't want to start here a discussion if that is insecure or a good > idea. That is something which has to be solved by > people creating the ecosystem around these languages. What I would like > to discuss here is how Fedora could improve > the situation. Where you see a technological problem, I see a deeper problem with the way we do package maintenance. Many packagers came to the Rust ecosystem because they wanted to add "that one cool new application" to Fedora. But they usually don't have the time to care about their dependencies too, other than maybe doing the initial packaging and then throwing them over the rust-sig wall. On the other hand, there's people like me, who try to keep the lower-level libraries up-to-date, monitor security advisories, etc., but obviously I can't care about updating shiny application X or the-new-hotness Y as well - because I just don't have the time. And I don't think making sure somebody else can update their package is my responsibility. That disconnect between "application maintainers" and "library maintainers" is often frustrating. Application maintainers grumble about having to file new package review tickets, and that their dependencies are out of date. Library maintainers often update libraries to new major versions only "as needed", because those major changes often involve creating compat packages or pushing coordinated updates of all dependents of a library. But often the "I can't update my shiny application X to the new version! The libraries are out of date, you lazy library maintainers!" (I'm exaggarating, but you get the point) is the result of missing "major" updates that just weren't needed (or possible!) before shiny application X needed the new version. Additionally, some application maintainers are now no longer actively maintaining their Fedora packages, but instead push updates only to COPR, where they don't need to follow Packaging Guidelines. starship is an example here [1]. While the COPR is up-to-date (with using bundled dependencies), the Fedora package could be described as unmaintained. [1]: https://copr.fedorainfracloud.org/coprs/atim/starship/ > I would say, that for GUI apps we already done that. We have Flatpak > which is doing great job to get these apps there > and motivate people to do that because it's then consumable on plenty of > places. However, this is not really usable if you > need app which is core part of the system (like prompt) or emacs/vim > which needs a lot of dependencies from the system > based on user configuration. > > My question is, what we can do to avoid situations as in Windows? I mean > that first thing what Win users are doing is to > look on internet and start to download random binaries. Do we have > something like Flatpaks for core parts of the system > now? Is there something we could adopt? Am I completely wrong with my > assumptions :D? > > Also want to mention that I don't have much experience with this. It's > more that I'm worried and maybe this discussion > could lead to something in the future. The problem I see with "containerizing" CLI applications is that those often require system access, and cannot be sandboxed or containerized at all. Another problem is that while *some* Rust applications can very easily be installed with "cargo install foo", some cannot. Fedora packages for library dependencies provide additional support for buildin
Re: [Discussion] Future of the CLI APP packaging
Dne 07. 01. 22 v 21:54 Major Hayden napsal(a): On 1/7/22 14:46, Jiri Konecny wrote: I would like to do here a bit of brainstorming and ask if there is an existing solution to this problem. To explain my problem, recently I found more and more apps likes terminals, prompt and command line tooling, which are great to use but not packaged or are outdated in our repositories. The reason for this is simple. These apps are written in languages like Rust or Go which works the way that there are plenty of smaller libraries and these are really a hell to package or maintain in Fedora. On the other hand it's pretty simple to build it from the source (most of the time) but hard to split it into packages. Oh gosh, yes, this has been a problem for me for a while. I maintain azure-cli, which has a few subpackages. It depends on about 95 other Azure Python SDK components which depend on various Python libraries. Each of these must be carefully updated when the main azure-cli package is updated. 🥵 I would say, that for GUI apps we already done that. We have Flatpak which is doing great job to get these apps there and motivate people to do that because it's then consumable on plenty of places. However, this is not really usable if you need app which is core part of the system (like prompt) or emacs/vim which needs a lot of dependencies from the system based on user configuration. I do like Flatpaks for GUI applications. For CLIs, I've seen people use container images since it's a single item that is easily updated. However, it's not always easy to determine how a container was built and the home of its sources. 😬 The cloud vendors seem to be moving towards bundling everything up into a zip file that you download and run. AWS now has their own crypto/hashing components and bundled Python in a zip. Google has a large zip with plenty of third party vendored content. Bundling to ZIP is again the same issue. You have to download it from somewhere and from my PoV that is a bigger problem than not being able to validate the content. In general I'm thinking if we shouldn't "shift" our current packager is responsible for safe content to author is responsible for safe content. That is basically happening for Flatpak. It's much easier to package because you can "just take the binary" from author and don't care. Partial solution to that is COPR. Which I'm taking as AUR (Archlinux). That is great but maybe there is something not specific for RPM but working out of the RPM world? That would be much better marketing for the authors to do the packaging. I'm very interested to hear additional use cases and ideas. ___ devel mailing list --devel@lists.fedoraproject.org To unsubscribe send an email todevel-le...@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: [Discussion] Future of the CLI APP packaging
Sorry adding missing link: [0]: https://github.com/starship/starship/blob/master/Cargo.toml#L32 Dne 07. 01. 22 v 21:46 Jiri Konecny napsal(a): Hi everyone, I would like to do here a bit of brainstorming and ask if there is an existing solution to this problem. To explain my problem, recently I found more and more apps likes terminals, prompt and command line tooling, which are great to use but not packaged or are outdated in our repositories. The reason for this is simple. These apps are written in languages like Rust or Go which works the way that there are plenty of smaller libraries and these are really a hell to package or maintain in Fedora. On the other hand it's pretty simple to build it from the source (most of the time) but hard to split it into packages. Great example of this is Startship command line prompt, we have outdated version but when you look to dependencies[1] you have to completely understand. I don't want to start here a discussion if that is insecure or a good idea. That is something which has to be solved by people creating the ecosystem around these languages. What I would like to discuss here is how Fedora could improve the situation. I would say, that for GUI apps we already done that. We have Flatpak which is doing great job to get these apps there and motivate people to do that because it's then consumable on plenty of places. However, this is not really usable if you need app which is core part of the system (like prompt) or emacs/vim which needs a lot of dependencies from the system based on user configuration. My question is, what we can do to avoid situations as in Windows? I mean that first thing what Win users are doing is to look on internet and start to download random binaries. Do we have something like Flatpaks for core parts of the system now? Is there something we could adopt? Am I completely wrong with my assumptions :D? Also want to mention that I don't have much experience with this. It's more that I'm worried and maybe this discussion could lead to something in the future. Best Regards, Jirka Konecny ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: [Discussion] Future of the CLI APP packaging
On 1/7/22 14:46, Jiri Konecny wrote: I would like to do here a bit of brainstorming and ask if there is an existing solution to this problem. To explain my problem, recently I found more and more apps likes terminals, prompt and command line tooling, which are great to use but not packaged or are outdated in our repositories. The reason for this is simple. These apps are written in languages like Rust or Go which works the way that there are plenty of smaller libraries and these are really a hell to package or maintain in Fedora. On the other hand it's pretty simple to build it from the source (most of the time) but hard to split it into packages. Oh gosh, yes, this has been a problem for me for a while. I maintain azure-cli, which has a few subpackages. It depends on about 95 other Azure Python SDK components which depend on various Python libraries. Each of these must be carefully updated when the main azure-cli package is updated. 🥵 I would say, that for GUI apps we already done that. We have Flatpak which is doing great job to get these apps there and motivate people to do that because it's then consumable on plenty of places. However, this is not really usable if you need app which is core part of the system (like prompt) or emacs/vim which needs a lot of dependencies from the system based on user configuration. I do like Flatpaks for GUI applications. For CLIs, I've seen people use container images since it's a single item that is easily updated. However, it's not always easy to determine how a container was built and the home of its sources. 😬 The cloud vendors seem to be moving towards bundling everything up into a zip file that you download and run. AWS now has their own crypto/hashing components and bundled Python in a zip. Google has a large zip with plenty of third party vendored content. I'm very interested to hear additional use cases and ideas. -- Major Hayden OpenPGP_0x737051E0C1011FB1.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
[Discussion] Future of the CLI APP packaging
Hi everyone, I would like to do here a bit of brainstorming and ask if there is an existing solution to this problem. To explain my problem, recently I found more and more apps likes terminals, prompt and command line tooling, which are great to use but not packaged or are outdated in our repositories. The reason for this is simple. These apps are written in languages like Rust or Go which works the way that there are plenty of smaller libraries and these are really a hell to package or maintain in Fedora. On the other hand it's pretty simple to build it from the source (most of the time) but hard to split it into packages. Great example of this is Startship command line prompt, we have outdated version but when you look to dependencies[1] you have to completely understand. I don't want to start here a discussion if that is insecure or a good idea. That is something which has to be solved by people creating the ecosystem around these languages. What I would like to discuss here is how Fedora could improve the situation. I would say, that for GUI apps we already done that. We have Flatpak which is doing great job to get these apps there and motivate people to do that because it's then consumable on plenty of places. However, this is not really usable if you need app which is core part of the system (like prompt) or emacs/vim which needs a lot of dependencies from the system based on user configuration. My question is, what we can do to avoid situations as in Windows? I mean that first thing what Win users are doing is to look on internet and start to download random binaries. Do we have something like Flatpaks for core parts of the system now? Is there something we could adopt? Am I completely wrong with my assumptions :D? Also want to mention that I don't have much experience with this. It's more that I'm worried and maybe this discussion could lead to something in the future. Best Regards, Jirka Konecny ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
