Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
> I understand the concerns of David Craven regarding > ‘cargo-build-system’, though OTOH I’m tempted to think > ‘cargo-build-system’ is probably not so bad if it managed to build all > this. ;-) But anyway, I’ll David and you figure out what the best > approach is! I haven't inspected all the patches, but I don't think it's possible that any package was actually built (unless danny's patch was applied). All the build-system does with these packages is recursively download the rust-psych dependency tree without building anything. This would not show any problems with some crates requiring a nightly rustc, or circular dependencies, or version mismatches (like requiring an older semver major version or multiple versions of the same package). The downside of doing it like this is that we can't run tests on the dependencies. But I think that building binaries like cargo that have locked dependencies side steps all the problems mentioned above, and would be a big milestone. It would allow us to do cool things like patch crates with security fixes; and c wrapper crates like rust-pkg-config, rust-cmake, rust-openssl, etc. could declare/propagate/retain their c dependencies directly. Danny has been exploring a different (or complementary) approach that would allow running the tests on the crates and solves the above issues manually. I currently think that doing this manually would make packaging and much more important updating packages a large amount of manual work, but I am open to reconsider if someone shows this to be a surmountable task. Another issue with Danny's approach is that as it is now, I don't see an immediate advantage of running tests on dependencies of a package before we have managed to build the package. The reason being that trying to use the latest versions of all packages recursively might yield big issues when trying to build the final thing even if all tests passed on the individual crates. This results in false confidence in the taken path and I don't think it can be considered a reliable intermediate step to packaging a rust package, as it would be in other build-systems. At least not until someone has provided evidence by showing at least one counter example.
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
Ludovic Courtès writes: > Hi! > > Thanks for all the work! > > I second Ricardo’s comment about systematically running ‘guix lint’ > before submitting, especially for a patch series that is this large. > > If you think the review might take several round trips, it might be more > convenient to publish it as a branch somewhere, to avoid resending 86 > messages. It is published, but I can move it to my pagure.io account as long as our (secushare) git+issue instance hasn't moved to the new server. Right now it's .onion and invite-only, which as we already discussed and voted on is suboptimal, resulting in the move to a new server which shall be more open. If that's really desired. I really dislike using email, so if you would really make use of just using the remote branch or checking it out in some way I'm all for it. Should I just sent a short notive with the new branch and its location once it is done? > I understand the concerns of David Craven regarding > ‘cargo-build-system’, though OTOH I’m tempted to think > ‘cargo-build-system’ is probably not so bad if it managed to build all > this. ;-) But anyway, I’ll David and you figure out what the best > approach is! I'm confused though. Before the latest patch it managed to build it all, but it starts to break at the circular dependencies where I did not check my solution and if it still builds. So up until the point where the patches just change an input to become available, it's alright, or seems so. The circular dependencies were confusing, I ended up moving from gitlab+markdown to my board and to paper to figure out the correct way to bootstrap over several levels but it ended up being circular. Then I saw that some where just devel-inputs or what they are called in rust again, so I thought it'd work out to first leave them out and add them once the right inputs are available. > Ludo’. > -- ♥Ⓐ ng0 PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
Hi! Thanks for all the work! I second Ricardo’s comment about systematically running ‘guix lint’ before submitting, especially for a patch series that is this large. If you think the review might take several round trips, it might be more convenient to publish it as a branch somewhere, to avoid resending 86 messages. I understand the concerns of David Craven regarding ‘cargo-build-system’, though OTOH I’m tempted to think ‘cargo-build-system’ is probably not so bad if it managed to build all this. ;-) But anyway, I’ll David and you figure out what the best approach is! Ludo’.
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
Danny Milosavljevic writes: > Hi ng0, > >> > For those I checked (like rust-openssl-sys) left me in confusion. Does our >> > cargo build-system just build nothing? > > Most Rust libraries are installed as source code (similar to C++ templates > which are also installed as source code). Okay, I'm totally new to Rust, the developers at my side might've told me before but I forgot about it I guess. > The current Guix master cargo build system doesn't build anything. Ah! > On the other hand, my WIP v2 build system does build the libraries. But > that's not because it installs the stuff it built (it shouldn't - it should > install the source code), it's just to make it easily possible to find out > what dependencies are missing and whether it's the correct version of the > library (e.g. working in Rust stable). Is builds in the "build" phase AND in > the "check" phase in order to find out which dependencies are required for > the library and which are required for the tests only. > >> Are all of these libraries? rust-openssl-sys contained nothing, but >> rust-openssl-sys:src had all the data. > > Yeah, David added a "src" output that contains the source code. The openssl-sys (or rather, the openssl crate) was one of these crates which required an external, non-rust dependency to be present. I was wondering how rust dealt with this, as I was just following upstream description and saw that something must be missing/broken at our end (or somewhere down my graph). > My WIP v2 build system doesn't do that yet since I have ~5000 lines of > existing rust package recipes I'd have to update - and I'd rather get > everything to work first and then drop all my Rust packages and recreate > them. Keep in mind that if you want to use my WIP v2 build system you also > have to either (1) patch guix/build-system/cargo.scm not to do the src output > or (2) patch guix/build/cargo-build-system.scm to do the src output. > Please be advised that it's not finished yet - that's why it's marked WIP and > not merged :) > > This is all very much work-in-progress. But your Rust package recipes without > dependencies probably won't change later. So those are fine. The others - > ehhh don't know yet. For example it's not clear yet to me whether we need > propagated-inputs for source libs. I think we do. I could create a branch with just those 0 dependency crates (see below for a list including but not limited to them) and send them to be merged? That's a base which can't be (very) wrong. In addition I could also split off the fluid and roboto patch to be merged. > Right now I'm working on untangling circular dependencies manually - probably > have to talk to upstream some more. > > (Sigh... why do people so often create circular dependencies? Don't they feel > icky about it?) > I have rebased with the latest master since David just pushed the cargo updates. Building rust-serde gave me the VM error you mentioned earlier. David Craven wrote: > > Both can happen at the same time and help each other. I/we need > > these patches, and I think keeping this branch around on our > > (my) > > side will help finding bugs in the cargo build-system. I just > > hope it won't be another year until it all works well enough. > Well if you'd like to help there, I think that finding and > packaging a > simple package that contains a Cargo.lock file and a dependency > tree > <= 10 crates would be a more useful test scenario. A test base > this > large would be useful at a later stage. I'm having difficulties > finding a crate that fits those requirements, maybe I'll package > some > old rust project of mine... I have most dependency trees listed in our gitlab. I can paste the ones where I found less than 10 dependencies. Keep in mind I could be wrong about some as later I started to add not all dependencies for those I already packaged before. No names except the name itself means that it has 0 dependencies, and "~" was a visual guideline for myself when I still used gitlab's limited markdown to track what I already packaged: futures log bitflags cfg-if void scoped-tls slab tempdir openssl-probe pkg-config rustc-serialize tempdir time -> needs: (advapi32-sys), ~log~, (kernel32-sys), ~libc~, rustc-serialize, (winapi) bitflags libc glob rand nix-test bytes -> needs: rand spin lazy-static qml (will require DOtherSide and crates: lazy-static, libc -- ♥Ⓐ ng0 PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
> Both can happen at the same time and help each other. I/we need > these patches, and I think keeping this branch around on our (my) > side will help finding bugs in the cargo build-system. I just > hope it won't be another year until it all works well enough. Well if you'd like to help there, I think that finding and packaging a simple package that contains a Cargo.lock file and a dependency tree <= 10 crates would be a more useful test scenario. A test base this large would be useful at a later stage. I'm having difficulties finding a crate that fits those requirements, maybe I'll package some old rust project of mine...
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
David Craven writes: >> One more addition: I will only send a rebased patch series after >> all patches have been reviewed. > >> I think running “guix lint” is essential before submitting patches to >> the mailing list, because it avoids wasted effort for a lot of common >> mistakes. > > It is my opinion that it is to early to be merging hundreds of rust > packages, and that the cargo-build-system has some "deep" issues that > need fixing first. Both can happen at the same time and help each other. I/we need these patches, and I think keeping this branch around on our (my) side will help finding bugs in the cargo build-system. I just hope it won't be another year until it all works well enough. I see that new rust related patches landed now, so I have to rebase now. I'm waiting for more reviews and will send a rebased version then. -- ♥Ⓐ ng0 PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
Hi ng0, > > For those I checked (like rust-openssl-sys) left me in confusion. Does our > > cargo build-system just build nothing? Most Rust libraries are installed as source code (similar to C++ templates which are also installed as source code). The current Guix master cargo build system doesn't build anything. On the other hand, my WIP v2 build system does build the libraries. But that's not because it installs the stuff it built (it shouldn't - it should install the source code), it's just to make it easily possible to find out what dependencies are missing and whether it's the correct version of the library (e.g. working in Rust stable). Is builds in the "build" phase AND in the "check" phase in order to find out which dependencies are required for the library and which are required for the tests only. > Are all of these libraries? rust-openssl-sys contained nothing, but > rust-openssl-sys:src had all the data. Yeah, David added a "src" output that contains the source code. My WIP v2 build system doesn't do that yet since I have ~5000 lines of existing rust package recipes I'd have to update - and I'd rather get everything to work first and then drop all my Rust packages and recreate them. Keep in mind that if you want to use my WIP v2 build system you also have to either (1) patch guix/build-system/cargo.scm not to do the src output or (2) patch guix/build/cargo-build-system.scm to do the src output. Please be advised that it's not finished yet - that's why it's marked WIP and not merged :) This is all very much work-in-progress. But your Rust package recipes without dependencies probably won't change later. So those are fine. The others - ehhh don't know yet. For example it's not clear yet to me whether we need propagated-inputs for source libs. I think we do. Right now I'm working on untangling circular dependencies manually - probably have to talk to upstream some more. (Sigh... why do people so often create circular dependencies? Don't they feel icky about it?)
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
> One more addition: I will only send a rebased patch series after > all patches have been reviewed. > I think running “guix lint” is essential before submitting patches to > the mailing list, because it avoids wasted effort for a lot of common > mistakes. It is my opinion that it is to early to be merging hundreds of rust packages, and that the cargo-build-system has some "deep" issues that need fixing first.
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
ng0 writes: > ... and I still am not sure wether the build system works or not (see bottom > of this email) because I still have to learn more about this language. Our > prototype will soon switch to fluid (it's currently using qml-material but I > refrained from adding that as its upstream looks like it moved on to fluid), > so apart from the prototype the only "real" application I can check this > chain on can only be added later after this has been added. Remember > panopticon, the package which got me to poke at the rust/cargo in winter of > 2015 and spring of last year? This can be made functional (finally) once I > package some more crates in addition to all the ones I send with this. > > Please provide constructive critique and review. Read the crate repositories > and their own descriptions etc. It was very hard and not intentional to end > up with tautological descriptions/synopsis. Remember that we are starting at > zero and that some (most?) of the now following crates add very basic > functionalities. > … > > > > I have not run any of the 81 packages through guix lint, I would if I would > know a way to mass lint a list of packages. > For those I checked (like rust-openssl-sys) left me in confusion. Does our > cargo build-system just build nothing? Are all of these libraries? > rust-openssl-sys contained nothing, but rust-openssl-sys:src had all the > data. Same goes for the libpsyc-rust > > One more addition: I will only send a rebased patch series after all patches have been reviewed. As a proof of concept I can go on and add more dependencies and package panopticon (https://panopticon.re) if that's what's needed for acceptance? As written above, I will do it anyway after the patches are in. it would just add 12 - 40 (depending on how much dependencies of dependencies I'd need) more patches, so this current patch series is more friendly. -- ♥Ⓐ ng0 PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
ng0 writes: > I have not run any of the 81 packages through guix lint, I would if I > would know a way to mass lint a list of packages. “guix lint” accepts more than one argument so you can lint all of the packages with one command. I do the same for R packages. Using Emacs or some other editor with macro features (or even just the shell) you can also easily condense your recent commit messages into a list of package names to be passed to “guix lint”. I think running “guix lint” is essential before submitting patches to the mailing list, because it avoids wasted effort for a lot of common mistakes.
Re: 77 Rust Crates, fluid, roboto-font, libpsyc rust bindings
Lotsa packages, nice. On 04/01/17 09:35, ng0 wrote: [...] I have not run any of the 81 packages through guix lint, I would if I would know a way to mass lint a list of packages. How about using parallel? $ cat list_of_packages |parallel -j1 guix lint ben
