Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote: On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. I've now fixed this (with an assist from Matthew). If you look at that commit again, it should reference an original commit that you can find in `plt/racket`. Furthermore, I've renamed the repository with the history. It's now here: github.com/samth/plt-history It has the original commit for everything that ends up in a split. It also has some instructions on how to see the history including all the original commits. Thanks for noticing the error. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: # Changes for git users If you build Racket from source from Git, that build now contains fewer packages. There is not yet an single-step way to get all of the split pkgs as git repositories; we plan to write a script for this soon. I've now written a first draft of this script. You can get it at https://github.com/samth/clone-all If you run: $ racket clone-all.rkt dir it will create clones of all the packages split out of the repository in dir. We'll integrate this into the main repository soon. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
Sam gave me a heads-up on IRC. Also, DrDr now builds several packages that it didn't before: the s3-sync, sha, http, and aws packages. Currently, these packages have some test failures, but we hope to fix that soon. I'm responsible for aws, http, and sha. I'm working on this. I'm close to merging some commits[^1] to aws/master. Will look at http and sha after that. Also this is an opportunity to set up aws to be built by Travis CI[^2]. If tests sneak back in that would fail on DrDr, they should fail for the same reason on Travis as an early warning for me. (The basic issue is that I have tests that actually interact with AWS, but these need credentials and cost $ so they need to be skipped in scenarios like DrDr and Travis CI. Some other tests can still be run, and will be.) [^1]: https://github.com/greghendershott/aws/commits/drdr [^2]: https://travis-ci.org/greghendershott/aws/builds/42570181 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
At Sat, 29 Nov 2014 22:00:44 -0500, Eli Barzilay wrote: On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote: On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: [...] This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? You just need to edit and change the relevant repository -- nothing else is required. You don't have to use --clone or any other mechanism. [...] This doesn't answer my question -- so I think that I wasn't clear enough. Say that I have some change to a specific file. I want to know how to do it in the following two ways: [...] 2. The thing that is more relevant for others (and that's the important one): I have a normal installation, I find a bug -- what do I do now? [...] A strategy is explained in the Developing Packages with Git section of the package-manager documentation. To elaborate for the running example: If you have remote-shell installed via pkgs.racket-lang.org and you want to modify it, then raco pkg update --clone .../remote-shell creates a clone at `.../remote-shell'. You can then use just `remote-shell` to make a subdirectory of the current directory. If you do that with the version of `raco pkg` as of this morning, it will suggest that you use `--multi-clone ask`. (As of just now, `--multi-clone ask` is the default.) When you're in `--multi-clone ask` mode, `raco pkg` will suggest that remote-shell, remote-shell-lib, and remote-shell-doc, are all cloned together. The argument after `--clone` is a directory path, but if no other arguments are provided to `raco pkg update`, a package name is inferred from the directory path (and a Git repo is inferred from that package name). In the example above, if you want to clone remote-shell to the current directory instead of a subdirectory of the current directory, supply the package name explicitly after . as the directory path: raco pkg update --clone . remote-shell That is, a clone of the package remote-shell is made in the current directory as linked as the package installation. If you installed a Git-based package via pkgs.racket-lang.org using a version of `raco pkg` that's older than a week or so, then `raco pkg` didn't record that the package came from a Git-repo URL, and so the simple `--clone` form above doesn't work. More significantly, if you install a snapshot of Racket, then the simple `--clone` form won't work with a package that is provided in built form by the snapshot's catalog. I'm working on a refinement to better address those situations. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
I believe I've fixed everything shown on http://drdr.racket-lang.org/29582/racket/share/pkgs/ for which I'm responsible: aws: Commits merged to master and pushed. http: Fixed a buggy test mishandling 301/302 redirects. sha: Nothing needed to be fixed. I'll keep an eye on DrDr; Sam said he'd add me to the email alerts. On Sun, Nov 30, 2014 at 4:13 PM, Greg Hendershott greghendersh...@gmail.com wrote: Sam gave me a heads-up on IRC. Also, DrDr now builds several packages that it didn't before: the s3-sync, sha, http, and aws packages. Currently, these packages have some test failures, but we hope to fix that soon. I'm responsible for aws, http, and sha. I'm working on this. I'm close to merging some commits[^1] to aws/master. Will look at http and sha after that. Also this is an opportunity to set up aws to be built by Travis CI[^2]. If tests sneak back in that would fail on DrDr, they should fail for the same reason on Travis as an early warning for me. (The basic issue is that I have tests that actually interact with AWS, but these need credentials and cost $ so they need to be skipped in scenarios like DrDr and Travis CI. Some other tests can still be run, and will be.) [^1]: https://github.com/greghendershott/aws/commits/drdr [^2]: https://travis-ci.org/greghendershott/aws/builds/42570181 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. If you build Racket from source from Git, that build now contains fewer packages. There is not yet an single-step way to get all of the split pkgs as git repositories; we plan to write a script for this soon. Any reason they are not git modules? (They've improved a lot, and they're even supported in github as links as long as they point to other GH repos. See for example: https://github.com/elibarzilay/test) To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: raco pkg install --clone pkg-build git://github.com/racket/pkg-build or for packages that are grouped together in a single repository raco pkg install --clone remote-shell git://github.com/racket/remote-shell?path=remote-shell-lib git://github.com/racket/remote-shell?path=remote-shell git://github.com/racket/remote-shell?path=remote-shell-doc Note that the clones created by `raco pkg install` cannot be pushed to with the default origin remote. This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? (Hopefully there's obvious agreement on the need for this to be compact.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote: On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. For packages with old history, the shas are in reference to https://github.com/samth/old-plt-new/ (which should maybe have a better name). If you build Racket from source from Git, that build now contains fewer packages. There is not yet an single-step way to get all of the split pkgs as git repositories; we plan to write a script for this soon. Any reason they are not git modules? (They've improved a lot, and they're even supported in github as links as long as they point to other GH repos. See for example: https://github.com/elibarzilay/test) The goal is to have packages that are in the main distribution not have a particular special status, beyond being in the list of things that gets put in the distribution. Also, a situation where you have to update two things when you do a commit is not ideal. To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: raco pkg install --clone pkg-build git://github.com/racket/pkg-build or for packages that are grouped together in a single repository raco pkg install --clone remote-shell git://github.com/racket/remote-shell?path=remote-shell-lib git://github.com/racket/remote-shell?path=remote-shell git://github.com/racket/remote-shell?path=remote-shell-doc Note that the clones created by `raco pkg install` cannot be pushed to with the default origin remote. This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? You just need to edit and change the relevant repository -- nothing else is required. You don't have to use --clone or any other mechanism. Note that the command line above can be shortened to: raco pkg install --clone remote-shell remote-shell-lib remote-shell-doc remote-shell (that `remote-shell` appears twice there is intentional). Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote: On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. For packages with old history, the shas are in reference to https://github.com/samth/old-plt-new/ (which should maybe have a better name). Yeah, very bad name but an even worse location... Since this is a major reorganization it would also be a reasonable to do a forced push of that thing to the main repository so the references make sense. (This is assuming that you cannot refer to the proper commits in the current repo for some reason.) If all else fails, you should at least include in the commits some pointer to a repo that has it. (Sounds picky, but having no reliable history is one thing that can make a company avoid using it.) Any reason they are not git modules? (They've improved a lot, and they're even supported in github as links as long as they point to other GH repos. See for example: https://github.com/elibarzilay/test) The goal is to have packages that are in the main distribution not have a particular special status, beyond being in the list of things that gets put in the distribution. I'm not talking about the distribution here -- just some meta repository that has everything in it in a convenient way for the people who have code in there. (And assuming that this meta repository will build with all of these packages that still not be related to a distribution.) (A different issue is organizing distributions: at some point in the past I pointed at the fact that git modules could be used for this, essentially being a collection of pointers to code that is included. But that bus has long gone, so I'm definitely not talking about that.) Also, a situation where you have to update two things when you do a commit is not ideal. I absolutely agree with that. Probably even more than you do, and certainly for a much longer time (that's the essense of my bundle-related rants years ago about spaghetti dependencies). I suppose that you're worrying about modules making it easy to do such cross-repo commits -- but it's no easier than just having a bunch of repos and doing a commit over them simultaneously. So it's as doable, and in both cases will be awkward enough to make it clear to the committer that something is wrong. To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: [...] This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? You just need to edit and change the relevant repository -- nothing else is required. You don't have to use --clone or any other mechanism. [...] This doesn't answer my question -- so I think that I wasn't clear enough. Say that I have some change to a specific file. I want to know how to do it in the following two ways: 1. Get the equivalent of the whole distribution in the form of a bunch of repositories, including the repository that I want to change. I make the change in the file, re-build, test things out, and eventually push. This is the way that is more relevant for internal people, and I suppose that there would be some script that will do all of that. (The equivalent of a toplevel `make'). 2. The thing that is more relevant for others (and that's the important one): I have a normal installation, I find a bug -- what do I do now? I can't just report the bug, the file, and my patch since that's using a path in the distribution that is mostly unrelated to the source repo for the file. Ideally, there would be some way to turn some containing directory into a repository so I can commit my change, and push it to a fork to do a PR. But that's a little extreme -- so what I'm talking about is more like a way to say: get this repository and use it instead of the stuff in my distribution, so I can work and push from that. I'm guessing that this is the intention behind that command, but it's obscure since it's not obvious what to do. (And obviousness is important to encourage contributions.) As a concrete example: I find a bug in a file like share/pkgs/games/gcalc/gcalc.rkt and want to fork, edit, and push a fix. What do I do? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: