Re: Improving importers best investment for growing gnu/packages/

2022-05-02 Thread zimoun
Hi,

On Tue, 29 Mar 2022 at 10:50, Brendan Tildesley  wrote:

> Therefore I'm working on making use of
> https://github.com/rust-lang/crates.io-index  to fully import base definitions
> all required crates rather than pulling metadata from the internet with each 
> refresh.

>From my point of view, the issue is that it requires external Rust tools
and I think such tool will never be part of the Guix dependencies.  Guix
itself cannot depends on Rust-thing, GHC-thing, OCaml-thing,
Julia-thing, etc. because they ease the importers.

And that’s why the current importers parse metadata and somehow
“reimplement” the logic behind the package manager of each language.  I
agree that it is a lot of work and hard to maintain but doing so allow
Guix to have a small footprint (dependency, speaking).

That’s said, how to improve the importers?

Well, Guix provides now GUIX_EXTENSIONS_PATH which allows to extend by
other subcommands.  “guix workflow” is an example, I do not remember how
“guix home” did initially, anyway!

My point is: this mechanism allows to have a package (with many
dependencies) extending Guix itself.  Somehow, it reads

guix install guix-rusty
guix rusty 

where ’guix-rusty’ would be a Guile package (program) using whatever it
needs from the Rust ecosystem (say parse system calls of cargo or
whatever).

Guix is Scheme library after all, let use it! :-)


Cheers,
simon



Re: Improving importers best investment for growing gnu/packages/

2022-03-29 Thread Liliana Marie Prikler
Am Dienstag, dem 29.03.2022 um 10:50 +1100 schrieb Brendan Tildesley:
> On 25/3/22 18:05, Paul Alesius wrote:
> > Rust analyzer is a language server for the Rust programming
> > language.
> > 
> > I've tried to produce a patch for the latest version, and it has
> > probably hundreds of dependencies that need to be updated.
> [...]
> > I'd suggest that a Guix package for rust-analyzer is not needed,
> > especially due to the excessive time required to update its package
> > definition and all of the vendored dependency crates, and focus
> > should instead be on rust and rust-cargo.
> 
> I understand how defeating it feels when looking at the countless
> hundreds of dependencies that get pulled in, however, I believe this
> can be solved by improving the importers in Guix.  I think it
> requires too much manual labour to maintain packages curretly,
> particularly when refresh -u will only update the
> package version and not dependencies.
> Therefore I'm working on making use of
> https://github.com/rust-lang/crates.io-index to fully import base
> definitions all required crates rather than pulling metadata from the
> internet with each refresh.
> Maintaining rust packages then becomes mostly
> 1. Refresh generated package definitions
> 2. In a separate file edit any hacks/fix needed argument fields etc
> 3. Edit cargo-build-system to account for the most common hacks like
> setting
> LIBCLANG_PATH.
I'm not sure how this is for Rust packages, but in general, I'd
consider over-reliance on `guix refresh' more harmful than not.  There
has recently been a paper published, which shows this to be the case
for spreadsheet debugging – give people a smell checker without
appropriate hints where it might be flawed and they will be worse at
finding flaws than the control group.  I make a claim, that it's the
same for packaging software.  Our importers should come with a warning
sign attached that they perform the best possible approximation based
on available metadata, but that they might be missing important
information or contain bugs, and that the output needs human
verification.

In general, even though one might claim that this conclusion has been
largely forgone in the case of cargo, npm, et al., human intelligence
is a requirement in human computer interaction.  Trying to take too
much of it away leads to adverse effects rather than "smart systems".

> It should be easy for a guix user to get all the latest crates that
> they would get using cargo.
> I think importers can be taken to the next level by going beyond
> metadata and actually inspecting the contents of source files. For
> example grepping build files it can be determined if pkg-config is
> used and add it to inputs.
That'd probably fine for pkg-config as a single package, but to get
packages to a working state, you'll have to do so for more than just
pkg-config (you list dependencies referenced by pkg-config below, for
example).  Speaking more generally, short of actually running the build
(repeatedly) against all possible combinations of inputs, you will not
obtain a complete or accurate input set (particularly also considering
that some of said inputs might not even be packaged).  And even if you
do, all your machine could then tell you, is that it builds; not
whether it functions as intended.  This also assumes that the source
won't need to be patched to remove icky stuff, no phase needs to be
adjusted, ...

> An index of .pc files can automatically detect and add the inputs a
> package is most likely looking for. One complaint about Guix is that
> typically just running ./configure; make on a project will not work,
> but if a guix can have a Universal Importer where running guix wave-
> magic-wand in a directory can inspect files, determine the build
> system, required inputs etc and build a program, would that
> not be wonderful for programmers? Working on KDE made me feel like I
> was wasting too much time manually adding inputs and would be better
> of writing an importer.
To be fair, scanning pkg-config files sounds like a good idea until you
consider it is but a heuristic and pkg-config names do not easily map
to package names.

> Using rust in a hypothetical, imagine if a rust developer that is
> otherwise uninterested in Guix, they just want to build their
> software and distribute actually chooses to use Guix not due to
> idealism but because it:
> A. Handles their rust dependecies at least as well as cargo.
> B. Handles their non-rust dependencies even better.
> C. Improved Guix pack/build can produce distributable packages,
> stripped of unneeded files that actually runs on other distributions.
> 
> Would that not be awesome?
Hypothetically speaking, apart from the fact that rust inputs are not
inputs because the Rust ecosystem is awful, maintaining a guix.scm on
the side of your Rust package should not be that much more difficult
than maintaining a cargo.toml.  And better yet, you don't need those
god-awful lock files, you could just 

Re: Improving importers best investment for growing gnu/packages/

2022-03-29 Thread Brendan Tildesley

On 29/3/22 19:01, Maxime Devos wrote:


Brendan Tildesley schreef op di 29-03-2022 om 10:50 [+1100]:

For example grepping build files it can be determined if pkg-config is used
and add it to inputs. an index of .pc files can automatically detect and add
the inputs a package is most likely looking for

Not sure what the cargo equivalent of this would be, but this looks
nice and feasible.


That was meant as an example of what Guix could provide that cargo doesn't.


Greetings,
Maxime.

Re: Improving importers best investment for growing gnu/packages/

2022-03-29 Thread Maxime Devos
Brendan Tildesley schreef op di 29-03-2022 om 10:50 [+1100]:
> For example grepping build files it can be determined if pkg-config is used
> and add it to inputs. an index of .pc files can automatically detect and add
> the inputs a package is most likely looking for

Not sure what the cargo equivalent of this would be, but this looks
nice and feasible.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part