Re: [Haskell-community] Standard package file format

[Herbert Valerio Riedel]

On 2016-09-16 at 08:20:15 +0200, Harendra Kumar wrote:

[...]

> * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A
> significant chunk of developer community is already familiar with it. It is
> being used by stack and by hpack as an alternative to cabal format. The
> complaint against it is that the specification/implementation is overly
> complex.

I'm not sure if this has been pointed out already, but beyond turning a
proper grammar into a stringly-typed one, shoehorning some features of
.cabal files into YAML syntax really appear like a case of the "Genius
Tailor"[1], e.g. consider the `hpack` example

   when:
 - condition: flag(fast)
   then:
 ghc-options: -O2
   else:
 ghc-options: -O0

besides looking quite awkward IMHO (just as an exercise, try inserting a
nested if/then/else in that example above), the prospect that a standard
format like YAML would allow to reuse standard tooling/libraries for
YAML seems quite weak to me; if, for instance, you run the above through
a YAML pretty-printer, you easily end up with something like

   when:
   - else:
   ghc-options: -O0
 then:
   ghc-options: -O2
 condition: flag(fast)

or any other ordering depending on how the keys are sorted/hashed.

Besides, many YAML (& JSON) parsers silently drop duplicate keys, so if
by accident you place a 2nd `else:` branch somewhere, you end up with an
ambiguous .yaml file which may either result in an error, in the first
key getting dropped (most likely variant), or in the 2nd key getting
dropped. Which one you get depends on the YAML parser implementation.


I really don't understand the appeal of applying the golden hammer of
YAML, if `.cabal`'s grammar is already self-evident and concise with its
syntax:

  if flag(fast)
ghc-options: -O2
  else
ghc-options: -O0

where this if/then/else construct is encoded in the grammar proper
rather than being merely a semantic interpretation after decoding a
general grammar designed for simpler typed data-representations which
isn't even accurate enough (since it has additional symmetries/freedoms)
to capture the desired grammar faithfully, which make YAML quite
error-prone for this specific application.



 [1]: The "Genius Tailor" was mentioned recently in a related discussion here:
  https://mail.haskell.org/pipermail/haskell-cafe/2016-September/124868.html

-- hvr
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] Standard package file format

[Harendra Kumar]

On 17 September 2016 at 03:43, Herbert Valerio Riedel 
wrote:

>
> I'm not sure if this has been pointed out already, but beyond turning a
> proper grammar into a stringly-typed one, shoehorning some features of
> .cabal files into YAML syntax really appear like a case of the "Genius
> Tailor"[1], e.g. consider the `hpack` example
>
>when:
>  - condition: flag(fast)
>then:
>  ghc-options: -O2
>else:
>  ghc-options: -O0
>
>
I agree. Supporting conditionals with YAML looks hacky!

-harendra
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Harendra Kumar]

On 16 September 2016 at 16:51, Paolo Giarrusso 
wrote:

>
> I agree "full-fledged build system" is not a possible immediate goal.
> But an EDSL for expressing cabal projects (as they are today) would
> still be in scope of your proposal—and I thought you liked the idea
> (see quote below). Using the earlier options: option 3 is not in scope
> of this thread, but option 2 is, with the only danger that the design
> space is so big to present a challenge.
>

Yeah I like the idea of using Haskell for configs but perhaps in a
different problem space e.g. in a build spec. See the quote from my earlier
quote below, sorry for the confusion :-) Yes, maybe option 2 might work for
package specifications but sounds pretty hairy to explore for this use case
alone, unless we have other motivations.


> Quoting from Harendra Kumar's earlier mail:
>
>  If we have to express not just a package specification but a
> sophisticated build configuration, we need a real language. Expressing
> conditionals, reuse etc becomes a compromise in a purely declarative
> language.
>

-harendra
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Paolo Giarrusso]

On 16 September 2016 at 12:13, Patrick Pelletier
 wrote:
> On 9/16/16 2:36 AM, Paolo Giarrusso wrote:
>>
>> (Resending from right address)
>>
>> We're talking about *three* options:
>> 1. syntax for pure Haskell values, which I'll call HSON (Haskell
>> jSON). That's just an alternative to YAML/TOML/... That would need
>> extensions to allow omitting optional fields entirely.
>> 2. a pure Haskell embedded domain-specific language (EDSL) that simply
>> generates cabal description records (GenericPackageDescription
>> values). That would allow abstraction over some patterns but not much
>> more. But that alone is already an argument for EDSLs—the one Harendra
>> already presented.
>> 3. a Haskell embedded domain-specific language (EDSL) designed for an
>> extensible build tool, like Clojure's (apparently), SBT for Scala or
>> many others. That would potentially be a rabbit hole leading to a
>> rather *different* tool—with a different package format to boot. That
>> can't work as long as all libraries have to be built using the same
>> tool. But stack and cabal are really about how to manage package
>> databases/GHC/external environments, while extensible build tools are
>> about (a more powerful form) of writing custom setup scripts. I
>> suspect some extensions might be easier if more of the actual building
>> was done by the setup script, but I'm not sure.
>
>
> Options 2 and 3 both require running Haskell code at build time.

> But if all packages had to use the new EDSL, then cross-compilation would 
> essentially become impossible.

"All packages migrate to new format" doesn't seem really a plausible
option, as I already hinted in the text you quote.
There are multiple JVM build tools because they're interoperable (like
cabal-install and Stack): each library picks its own build tool, but
they can still be linked together.
Hpack generates cabal files, stack reuses cabal or hpack files.

In principle, option 2 just needs a non-cross-compiled program to
produce a package description—say by producing a cabal file. You just
need to runghc it, either via ghci or by compiling and running a
binary. Option 3 can be trickier depending on details, but the as long
as you account for cross-compilation in the design it should be
doable. For Template Haskell the problem is deeper (see
http://blog.ezyang.com/2016/07/what-template-haskell-gets-wrong-and-racket-gets-right/),
so let's *not* use it here.
-- 
Paolo G. Giarrusso - Ph.D. Student, Tübingen University
http://ps.informatik.uni-tuebingen.de/team/giarrusso/
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Imants Cekusins]

this may be one of the 3 points on Paolo's list. In case it is not, here is
another option (4?):


   - define .hs data records for project config, package configs


   - write export tools to export config records to existing formats:

cabal
stack yaml

...

​
this way, there is no need to revise the current workflow or modify tools.

However we define a common standard content structure, most users do not
need to worry about .cabal, .yaml syntax
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Chris Smith]

I guess the overriding question I have here is: what is the PROBLEM being
solved?  I know of basically no beginners who were confused or intimidated
by the syntax of Cabal's file format.  It's fairly commonplace for
beginners to be confused by the *semantics*: which fields are needed and
what they mean, how package version bounds work, what flags are and how
they interact with dependencies, the relationship between libraries and
executables defined in the same file, etc.  But the syntax?  It's just not
an issue.  I'm not sure what it means to say that people have to "learn"
it, because in introducing dozens of people to building things in Haskell,
I've never seen that learning process even be noticeable, much less an
impediment.

With this in mind, a lot of the statements about these various languages
are not entirely convincing.  That it's a superset of JSON?  It's not clear
why this matters.  A psychological impression of complexity?  Just not
anything I've seen evidence of.  Indeed, aside from the rather painful
many-years-long migration, the *cost* (though certainly not a prohibitive
one) of moving to something like YAML or TOML is that they have a bit
louder syntax, that demands more attention and feels more complex.

There is one substantial disadvantage I'd point out to the Cabal file
format as it stands, and that's that it's pretty non-obvious how to parse
it, so we will always struggle to interact with it from automated tools,
unless those tools are also written in Haskell and can use the Cabal
library.  That's a real concern; pragmatic large-scale build environments
are not tied to specific languages, and include a variety of ad-hoc
third-party tooling that needs to be integrated, and Cabal remains opaque
to them.  But that doesn't seem to be what's motivating this conversation.

On Thu, Sep 15, 2016 at 11:20 PM, Harendra Kumar 
wrote:

> I am starting a new thread for the package file format related discussion.
>
> From a developer's perspective, the major benefit of a standard and widely
> adopted format and is that people can utilize their knowledge acquired from
> elsewhere, they do not have to go through and learn differently looking and
> incomplete documentation of different tools. The benefit of a common config
> specification is that developers can choose tools freely without worrying
> about learning the same concepts presented in different ways.
>
> Multiple formats flying around also create a psychological impression of
> complexity in the ecosystem for newcomers. If we have consistency there are
> better chances of attracting more people to the language ecosystem.
>
> I gather the following from the discussion till now:
>
> * We have cabal, YAML and TOML as potential candidates for a common
> package format which can additionally incorporate the concept of
> snapshots/package collections and potentially more extensions useful across
> build tools.
>
> * cabal has the benefit of incumbency and backward compatibility, it has
> shortcomings which are being addressed but it is still a format which is
> very specific to Haskell ecosystem. It is not a standard and not going to
> become one. We have to always deal with it ourselves and everyone coming to
> Haskell will have to learn it.
>
> * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A
> significant chunk of developer community is already familiar with it. It is
> being used by stack and by hpack as an alternative to cabal format. The
> complaint against it is that the specification/implementation is overly
> complex.
>
> * TOML (https://github.com/toml-lang/toml) is promising, simpler than
> YAML and is being used by a few important projects but is still evolving
> and is not completely stable. On a first glance it looks pretty simple and
> a lot of other tools use a similar config format. It is aiming to become a
> standard and aiming for a wider adoption.
>
> As a next step we can perhaps do an hpack like experiment using the TOML
> format. That way we will have some experience with that as well and get to
> know if there are any potential problems expressing the existing cabal
> files.
>
> More thoughts, opinions on the topic will help create a better
> understanding about it.
>
> -harendra
>
> ___
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
>
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Imants Cekusins]

> So if you can execute arbitrary code in a new-gen cabal file, apart from
the security aspects, ...
​
well config files could use different (not .hs) extensions. They could use
their own Prelude and not allow importing other modules.

The main benefit is to reuse existing parsers and simplify code-config sync.
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

Re: [Haskell-community] [Haskell-cafe] Standard package file format

[Harendra Kumar]

On 16 September 2016 at 12:35, Imants Cekusins  wrote:

> Why not adopt (a subset of) .hs AST file format to structure both project
> and package files?
>

Aha, that's my preferred choice. If there is a way to restrict features and
we can allow just a subset we can have a nice configuration language which
is a real language. In fact, I have been toying around this. If we have to
express not just a package specification but a sophisticated build
configuration, we need a real language. Expressing conditionals, reuse etc
becomes a compromise in a purely declarative language.

For example make has so many built-in functions in it that it has become a
full fledged language by itself. The google bazel build uses python as the
build config language. Haskell will make a much better choice for such use
cases. Pure declarative is a pain for such use cases.

-harendra
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community

[Haskell-community] Standard package file format

[Harendra Kumar]

I am starting a new thread for the package file format related discussion.

>From a developer's perspective, the major benefit of a standard and widely
adopted format and is that people can utilize their knowledge acquired from
elsewhere, they do not have to go through and learn differently looking and
incomplete documentation of different tools. The benefit of a common config
specification is that developers can choose tools freely without worrying
about learning the same concepts presented in different ways.

Multiple formats flying around also create a psychological impression of
complexity in the ecosystem for newcomers. If we have consistency there are
better chances of attracting more people to the language ecosystem.

I gather the following from the discussion till now:

* We have cabal, YAML and TOML as potential candidates for a common package
format which can additionally incorporate the concept of snapshots/package
collections and potentially more extensions useful across build tools.

* cabal has the benefit of incumbency and backward compatibility, it has
shortcomings which are being addressed but it is still a format which is
very specific to Haskell ecosystem. It is not a standard and not going to
become one. We have to always deal with it ourselves and everyone coming to
Haskell will have to learn it.

* YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A
significant chunk of developer community is already familiar with it. It is
being used by stack and by hpack as an alternative to cabal format. The
complaint against it is that the specification/implementation is overly
complex.

* TOML (https://github.com/toml-lang/toml) is promising, simpler than YAML
and is being used by a few important projects but is still evolving and is
not completely stable. On a first glance it looks pretty simple and a lot
of other tools use a similar config format. It is aiming to become a
standard and aiming for a wider adoption.

As a next step we can perhaps do an hpack like experiment using the TOML
format. That way we will have some experience with that as well and get to
know if there are any potential problems expressing the existing cabal
files.

More thoughts, opinions on the topic will help create a better
understanding about it.

-harendra
___
Haskell-community mailing list
Haskell-community@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community