I find this discussion interesting, it caused me to search around for links, blogposts, papers and examples about package management. The things I found so far are accessible here (a online collaborative text editor), but feel free to move it to some other tool, wiki etc.: https://cryptpad.fr/pad/#/2/pad/edit/YBPAp3HwnjdxbaDIVWaEcfrV/
Maybe we can collect links to papers that could be relevant. Probably should be converted to a github repo / wiki at some point? For packaging standalone applications this issue might be partially related to this discussion: "Easy to use and scriptable cross platform distribution support" https://github.com/racket/rhombus-brainstorming/issues/113 I am interested in helping with any potential efforts towards new or extended tools related to this. Simon Am Sa., 2. Mai 2020 um 00:03 Uhr schrieb Robby Findler < [email protected]>: > Thanks! Although I cannot really see the full implications, the basic idea > sounds really great. > > DrRacket does share some things, but tries to minimize the sharing; if it > were possible to share less, that'd be great, but maybe that'll end up > falling under the same category as reasons you'd install something > globally. But certainly it would be important for the system to support the > ability to change which "project" you're in without having to start a new > command-line racket binary somehow. > > Robby > > > On Fri, May 1, 2020 at 11:04 AM Sam Tobin-Hochstadt <[email protected]> > wrote: > >> I'll try to write out in more detail what the kinds of package >> managers Sage is referring to do, to make this clearer for everyone. >> >> The high-level idea is that you have a bunch of directories on your >> system(Alex's analogy to git is a good one), and in each of them, your >> run appropriate commands and you see things set up as if the other >> directories you've worked with didn't exist. This includes both >> executing code (like running `racket foo.rkt`) and managing packages >> (like `raco pkg install`). >> >> There are basically two approaches in wide use here, depending on >> whether the underlying system was developed after this kind of package >> management approach became popular (npm, cargo, stack) or before >> (virtualenv for python and others). >> >> Approach 1 involves mostly working using the package manager tool >> (such as npm), but also setting up the semantics of the module system >> to look in the local directory. >> >> Approach 2 involves setting environment variables and command line >> arguments, and wrapping the underlying command (such as the python >> binary) to trick it into looking for files in the right place. >> >> In both approaches, installing libraries "globally" is not >> recommended, and likely reserved for command line tools that aren't >> depended upon as libraries, or for core system libraries that can't >> change independently of the compiler. >> >> One additional detail -- often performing package operations will >> automatically record the exact versions of everything that was used in >> that directory (with data similar to a pkg catalog), so that a the >> same code can be installed with the same dependent packages in a >> different situation. >> >> I think supporting this mode of working would be very useful for >> Racket, but there are definitely some challenges. The biggest one >> relates to what Matthew mentioned, with globally-installed packages >> shadowed by local-scope packages. This would be particularly hard for >> Racket, where lots of tools we want to install once globally (like >> DrRacket) would pull in packages that would also be needed locally. >> Additionally, how DrRacket would work in this mode would be tricky, >> since DrRacket's architecture relies on sharing of lower-level modules >> between the program and DrRacket itself. >> >> With that said, I think the approach Matthew outlined would take a big >> step, and would let us find out if we needed more steps, and what they >> might be. >> >> Sam >> >> On Thu, Apr 30, 2020 at 11:59 PM Robby Findler >> <[email protected]> wrote: >> > >> > I'm not familiar enough with NPM or Pipenv to fully appreciate what >> you're saying, but I'm glad it sounds like I have the right basic idea. >> raco link is, roughly speaking, a layer below raco pkg that is used to >> implement raco pkg. >> > >> > Well, at any rate, I'll be curious to see how things progress! >> > >> > Robby >> > >> > >> > On Thu, Apr 30, 2020 at 8:18 PM Sage Gerard <[email protected]> >> wrote: >> >> >> >> I'm looking for an experience similar to how NPM, Pipenv, and other >> package managers work in their respective ecosystems. You described the >> desired conditions correctly. >> >> >> >> Racket's separation of packages and collections complicates that a >> bit. Sam mentioned on the Slack that this problem could be approached in >> terms of 'raco link', but it would lead to a large scope of work. I wonder >> how much that overlaps here. >> >> >> >> >> >> >> >> >> >> -------- Original Message -------- >> >> On Apr 30, 2020, 9:01 PM, Robby Findler < [email protected]> >> wrote: >> >> >> >> >> >> I'm trying to understand what you guys are talking about here and am >> not sure I'm getting it. Is the idea that a "project" (new term definition) >> would be a place one can install a package and its dependencies in a way >> that avoids any conflicts with other pkgs that are already installed (even >> if they are different versions of those same packages)? So I would have >> some command to configure my installation so that I would tell it what >> project I want to work with and then uses of `racket` after that would find >> requires inside the pkgs that belong to that project? >> >> >> >> (Sorry if this is a dumb question; I'm very curious about this.) >> >> >> >> Robby >> >> >> >> >> >> On Thu, Apr 30, 2020 at 4:33 PM Matthew Flatt <[email protected]> >> wrote: >> >>> >> >>> That's right: there's no one-step operation right now, and a new >> >>> one-step tool that sets things up the way you need would be a great >> >>> contribution. >> >>> >> >>> To avoid anything currently in user scope, maybe your new tool would >> >>> configure the project space with a fresh 'installation-name. After >> >>> setting 'pkgs-dir to be the project-specific package directory while >> >>> setting 'pkg-search-dir to chain to the original installation's >> package >> >>> directory, it could install some initial packages in the new >> >>> "installation" scope --- without affecting "user" scope or the >> original >> >>> "installation" scope. By setting 'lib-dir and 'shared-dir and similar >> >>> directories to a project-specific location, any files installed by >> >>> packages in the new "installation" scope would not pollute the old >> >>> installation. And so on. >> >>> >> >>> I'm not sure of the details, and it's possible that something new is >> >>> would still be needed in the collection or package or `setup/dirs` >> >>> layers to make the one-step tool work right. It's still possible that >> >>> the existing support doesn't do what you want at all. >> >>> >> >>> I'm unclear on whether you want to install packages in project scope >> >>> that would collide with things in the original "installation" scope, >> >>> but that's tricky at best. Conceptually, it works to install a package >> >>> in a narrower scope and shadow one in a wider scope, but only if >> >>> everything in the wider scope that depends on the shadowed package is >> >>> also shadowed in the narrower scope. If you want to be able to do >> that, >> >>> then probably this isn't the right direction. >> >>> >> >>> At Thu, 30 Apr 2020 20:54:33 +0000, Sage Gerard wrote: >> >>> > Some of the pieces are falling into place, but I'm still chewing on >> something. >> >>> > >> >>> > Let me rephrase my goal with added details: As a library author, I >> need my >> >>> > users to have ONE command that safely installs a package with >> collection paths >> >>> > that would collide in a user or (system) installation scope. >> >>> > >> >>> > The docs on package scopes and your email make it sound like I have >> to ask my >> >>> > users to create and configure an installation, and make sure no >> colliding >> >>> > packages exist at user scope. All before installing the package. >> Can your >> >>> > recipe be done in one step? If not, I think that's where I'd like to >> >>> > contribute. >> >>> > >> >>> > ~slg >> >>> > >> >>> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >> >>> > On Thursday, April 30, 2020 3:24 PM, Matthew Flatt < >> [email protected]> wrote: >> >>> > >> >>> > > Consider viewing "project" scope as "installation" scope, where an >> >>> > > existing installation becomes a wider scope after "installation". >> Then, >> >>> > > the exiting configuration and scope machinery will match what you >> need. >> >>> > > >> >>> > > If you take that view, then a project gets its own "etc" where >> you put >> >>> > > a "config.rktd" to configure the project-as-installation. That >> >>> > > "config.rktd" can point to the original installation as the next >> layer >> >>> > > for various searches: "collects", "lib", and so on. >> >>> > > >> >>> > > You can even set `config-tethered-console-bin-dir` and >> >>> > > `config-tethered-gui-bin-dir` to get project-local `racket`, >> `raco`, >> >>> > > etc., executables that have the right configuration path built in, >> >>> > > instead of having to use something like environment variables to >> select >> >>> > > a project's configuration. >> >>> > > >> >>> > > At least, that's one of the intentions of the existing layers. I >> didn't >> >>> > > push it further (e.g., to a `raco` command that would set up a >> project >> >>> > > in this sense) because there was not demand at the time. >> >>> > > >> >>> > > At Thu, 30 Apr 2020 19:10:11 +0000, Sage Gerard wrote: >> >>> > > >> >>> > > > Alright, so after thinking about how to improve Racket package >> management I >> >>> > > > decided that I want to add project scoping logic to raco pkg >> install. I'm >> >>> > not >> >>> > > > sure --scope-dir is enough because the docs say that the >> installation is >> >>> > > > apparently checked before the scoped directories, and there are >> no lock >> >>> > files. >> >>> > > > Correct me if I'm wrong. >> >>> > > > I've previously made commitments re: a CI dashboard and a syntax >> >>> > highlighter. >> >>> > > > Those remain, but don't seem to need me right this minute. >> Meanwhile, this >> >>> > is >> >>> > > > one of those features that hurts to not have, especially with >> current >> >>> > > > limitations in the default package catalog. >> >>> > > > https://akkuscm.org says it supports plt-r6rs, so I >> >>> > > > see two paths: Either implement the diff between that and >> Racket today in >> >>> > > > Akku, or add just enough of Akku to a raco pkg install --scope >> project >> >>> > > > command. I expect both efforts would be painful, but the latter >> seems >> >>> > better. >> >>> > > > Are there any landmines are in the raco pkg install code that I >> need to >> >>> > know >> >>> > > > about? I'd rather not make a bunch of changes with subtle >> consequences that >> >>> > > > I'm not seeing. I'd also like to know who would be the best >> person to >> >>> > review >> >>> > > > my proposed changes in this area. >> >>> > > > ~slg >> >>> > > > -- >> >>> > > > You received this message because you are subscribed to the >> Google Groups >> >>> > > > "Racket Developers" group. >> >>> > > > To unsubscribe from this group and stop receiving emails from >> it, send an >> >>> > > > email to [email protected]. >> >>> > > > To view this discussion on the web visit >> >>> > > > >> >>> > >> https://groups.google.com/d/msgid/racket-dev/04pE-yKBt1FzXg9NpgHlRoMEPSp3rIKJMJ >> >>> > > > >> >>> > >> fzPRd7HBHxyF1gEwxK70y4uFONwvTXUADMdH5_Yz-Xy7fHA7XrbWe6NYYHXvE65gvPZ4H_uNU%3D%40 >> >>> > > > sagegerard.com. >> >>> > > >> >>> > > -- >> >>> > > >> >>> > > You received this message because you are subscribed to the >> Google Groups >> >>> > "Racket Developers" group. >> >>> > > To unsubscribe from this group and stop receiving emails from it, >> send an >> >>> > email to [email protected]. >> >>> > > To view this discussion on the web visit >> >>> > >> https://groups.google.com/d/msgid/racket-dev/5eab25df.1c69fb81.8f98.25f6SMTPIN_ >> >>> > ADDED_MISSING%40gmr-mx.google.com. >> >>> > >> >>> > >> >>> > -- >> >>> > You received this message because you are subscribed to the Google >> Groups >> >>> > "Racket Developers" group. >> >>> > To unsubscribe from this group and stop receiving emails from it, >> send an >> >>> > email to [email protected]. >> >>> > To view this discussion on the web visit >> >>> > >> https://groups.google.com/d/msgid/racket-dev/il5M1m5pESWE_CWHdsPHg9WvQ5BhXin9sT >> >>> > >> UJ9KMwaoKzuZldA1874eMaM5py5OFrsd23Az6Kr9mH69YblDfqrGS77flNW3BnwVzQGvygG84%3D%40 >> >>> > sagegerard.com. >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> >>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/5eab440d.1c69fb81.87557.2d50SMTPIN_ADDED_MISSING%40gmr-mx.google.com >> . >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAL3TdOOgXhcPV-ppNy%3D%2Be%2B_LU6B-1jd-L1jmUiZT%2B6O9WYCa4g%40mail.gmail.com >> . >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Racket Developers" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAL3TdOMwspLvuha7FoOm_RApfY-YGq%2BsB2625%3DLxBBJ0vhiYng%40mail.gmail.com >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-dev/CAK%3DHD%2BYR_x4nuGV78-TB0dDzbAF%3D6hZDWQ%3DkbY9Pn4QqBJawcw%40mail.gmail.com >> . >> > -- > You received this message because you are subscribed to the Google Groups > "Racket Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/racket-dev/CAL3TdOP0T693W2Avw3R-rgmAUPbdovJcVn2zJRqx0Cb-gwD66w%40mail.gmail.com > <https://groups.google.com/d/msgid/racket-dev/CAL3TdOP0T693W2Avw3R-rgmAUPbdovJcVn2zJRqx0Cb-gwD66w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Racket Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/CAHs8QBqfxDHvDbEA73cO7ak_sny4J0G03UMBJUXJeLrN0iVJsg%40mail.gmail.com.
