Re: [Haskell-community] [Haskell-cafe] Standard package file format
I haven't totally followed this whole thread, so apologies if this isn't entirely relevant, but I use shake for building, and cabal for dependencies. The shakefile has the list of packages and required versions, and generates the .cabal file, which is used with --only-dependencies to get dependencies. I think it works well. I can't do builds in cabal anyway since it can't handle anything complicated, but even if I had a simple build I'd prefer shake since it's so much nicer. Since it's in haskell, it's flexible but can't be analyzed, though I can't think of why you'd want to analyze it. Meanwhile, cabal is just fine at expressing packages and versions, and is basically just a way to tell cabal-install what to download and install. Since I generate it, I don't care much about the format, but the existing one seems perfectly adequate. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
Another factor in favor of YAML is that it is a superset of JSON, which eases the learning curve even more (with JSON being a de facto lingua franca for cross-platform untyped data structures), and offers some extra possibilities, although I admit that I can't think of any practical uses. The fact that both Yaml and JSON can be represented as Aeson Values would also make things (arguably) easier for tool writers. On Sep 16, 2016 8:20 AM, "Harendra Kumar" wrote: > I am starting a new thread for the package file format related discussion. > > From a developer's perspective, the major benefit of a standard and widely > adopted format and is that people can utilize their knowledge acquired from > elsewhere, they do not have to go through and learn differently looking and > incomplete documentation of different tools. The benefit of a common config > specification is that developers can choose tools freely without worrying > about learning the same concepts presented in different ways. > > Multiple formats flying around also create a psychological impression of > complexity in the ecosystem for newcomers. If we have consistency there are > better chances of attracting more people to the language ecosystem. > > I gather the following from the discussion till now: > > * We have cabal, YAML and TOML as potential candidates for a common > package format which can additionally incorporate the concept of > snapshots/package collections and potentially more extensions useful across > build tools. > > * cabal has the benefit of incumbency and backward compatibility, it has > shortcomings which are being addressed but it is still a format which is > very specific to Haskell ecosystem. It is not a standard and not going to > become one. We have to always deal with it ourselves and everyone coming to > Haskell will have to learn it. > > * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A > significant chunk of developer community is already familiar with it. It is > being used by stack and by hpack as an alternative to cabal format. The > complaint against it is that the specification/implementation is overly > complex. > > * TOML (https://github.com/toml-lang/toml) is promising, simpler than > YAML and is being used by a few important projects but is still evolving > and is not completely stable. On a first glance it looks pretty simple and > a lot of other tools use a similar config format. It is aiming to become a > standard and aiming for a wider adoption. > > As a next step we can perhaps do an hpack like experiment using the TOML > format. That way we will have some experience with that as well and get to > know if there are any potential problems expressing the existing cabal > files. > > More thoughts, opinions on the topic will help create a better > understanding about it. > > -harendra > > ___ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
Sbt seems to be doing rather well, using full Scala in configurations. I think package descriptions should be limited, but not syntactically. Using some specific monad might work OK. > On 16 Sep 2016, at 09:22, Alan & Kim Zimmerman wrote: > > The more power you put into the package file description, the harder it is > for the surrounding ecosystem to reason about it. > > So if you can execute arbitrary code in a new-gen cabal file, apart from the > security aspects, it becomes difficult to be sure what is actually being > specified, if you do not reproduce the original environment when evaluating > the file. > > Alan > > On Fri, Sep 16, 2016 at 9:18 AM, Harendra Kumar > wrote: > On 16 September 2016 at 12:35, Imants Cekusins wrote: > Why not adopt (a subset of) .hs AST file format to structure both project and > package files? > > Aha, that's my preferred choice. If there is a way to restrict features and > we can allow just a subset we can have a nice configuration language which is > a real language. In fact, I have been toying around this. If we have to > express not just a package specification but a sophisticated build > configuration, we need a real language. Expressing conditionals, reuse etc > becomes a compromise in a purely declarative language. > > For example make has so many built-in functions in it that it has become a > full fledged language by itself. The google bazel build uses python as the > build config language. Haskell will make a much better choice for such use > cases. Pure declarative is a pain for such use cases. > > -harendra > > ___ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > > ___ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
> On 16 Sep 2016, at 09:24, Chris Smith wrote: > > I guess the overriding question I have here is: what is the PROBLEM being > solved? I know of basically no beginners who were confused or intimidated by > the syntax of Cabal's file format. As a "beginner"(*), I fully agree. However having more than one language in the mix can be confusing and complicating... > It's fairly commonplace for beginners to be confused by the *semantics*: > which fields are needed and what they mean, how package version bounds work, > what flags are and how they interact with dependencies, the relationship > between libraries and executables defined in the same file, etc. It's all about the semantics - it should preferably be formalised, and ideally the relevant library/package system should be able to check/enforce rules. > But the syntax? It's just not an issue. I'm not sure what it means to say > that people have to "learn" it, because in introducing dozens of people to > building things in Haskell, I've never seen that learning process even be > noticeable, much less an impediment. I quite agree > Andrew Butterfield School of Computer Science & Statistics Trinity College Dublin 2, Ireland (*) I've only started to use cabal recently, because a TA of mine built a cabal-based coursework grading system for me - I generally do application devpt in Haskell and the only build command I need is ghc --make Currently moving quickly onto stack this year ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
We're talking about *three* options: 1. syntax for pure Haskell values, which I'll call HSON (Haskell jSON). That's just an alternative to YAML/TOML/... That would need extensions to allow omitting optional fields entirely. 2. a pure Haskell embedded domain-specific language (EDSL) that simply generates cabal description records (GenericPackageDescription values). That would allow abstraction over some patterns but not much more. But that alone is already an argument for EDSLs—the one Harendra already presented. 3. a Haskell embedded domain-specific language (EDSL) designed for an extensible build tool, like Clojure's (apparently), SBT for Scala or many others. That would potentially be a rabbit hole leading to a rather *different* tool—with a different package format to boot. That can't work as long as all libraries have to be built using the same tool. But stack and cabal are really about how to manage package databases/GHC/external environments, while extensible build tools are about (a more powerful form) of writing custom setup scripts. I suspect some extensions might be easier if more of the actual building was done by the setup script, but I'm not sure. On 16 September 2016 at 10:57, yogsototh wrote: > >> I guess the overriding question I have here is: what is the PROBLEM being >> solved? > > > Let me share my experience with Clojure and lein. They use a clojure > hash-map for their configuration. So yes arbitrary code could be executed > and I believe this is a _very good thing_. > > Why? Because it makes it very easy to add sub-configuration that can be used > by third party plugin. For example: > > - a plugin that help the use of environment variables (lein-environ) which > is really helpful for application development (not so much for library > development) > - a plugin that use S3 for our private dependencies (not supported by > default by lein) > > > For deployment: we were able to add request to our API server that provide > not only the written version but also the git commit hash. So we could be > certain of the version of the server. Too much time there were sys/admin > deployment errors. And that could only be achieved because we were able to > run arbitrary command in the project description file. > > I certainly forget many other advantages of having a package description > format which is simply a data structure in the hosted language. But this has > by far my preference. > > - cabal is ok, but very imperfect, I generally need to have a lot of > copy/paste, I need to change it very often while writing application with > many dependencies > - JSON/YAML/TOML are simply not powerful enough to match all semantics we > might need to configure a project. For example we might want to have Set > instead of List for some properties. Or I don't know maybe ternary tree > structures. > > The point is: we pay a price by adding a step between the semantic and the > syntax. > While if our configuration format was in Haskell we could express the > semantic more directly. > > ___ > Haskell-community mailing list > Haskell-community@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community > -- Paolo G. Giarrusso - Ph.D. Student, Tübingen University http://ps.informatik.uni-tuebingen.de/team/giarrusso/ ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
On 16 September 2016 at 16:51, Paolo Giarrusso wrote: > > I agree "full-fledged build system" is not a possible immediate goal. > But an EDSL for expressing cabal projects (as they are today) would > still be in scope of your proposal—and I thought you liked the idea > (see quote below). Using the earlier options: option 3 is not in scope > of this thread, but option 2 is, with the only danger that the design > space is so big to present a challenge. > Yeah I like the idea of using Haskell for configs but perhaps in a different problem space e.g. in a build spec. See the quote from my earlier quote below, sorry for the confusion :-) Yes, maybe option 2 might work for package specifications but sounds pretty hairy to explore for this use case alone, unless we have other motivations. > Quoting from Harendra Kumar's earlier mail: > > If we have to express not just a package specification but a > sophisticated build configuration, we need a real language. Expressing > conditionals, reuse etc becomes a compromise in a purely declarative > language. > -harendra ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
On 16 September 2016 at 13:05, Harendra Kumar wrote: > This seems to have gone into a different direction. The original point was > about the package specification format and not expressing a full fledged > build system. That is an entirely different ballgame. The main point of the > thread was whether it makes sense to use a single specification format for > both stack and cabal install (YAML vs .cabal and then TOML came into > picture). Haskell does not seem to be a choice for a package specification > format unless we have a very different goal in mind. I agree "full-fledged build system" is not a possible immediate goal. But an EDSL for expressing cabal projects (as they are today) would still be in scope of your proposal—and I thought you liked the idea (see quote below). Using the earlier options: option 3 is not in scope of this thread, but option 2 is, with the only danger that the design space is so big to present a challenge. Quoting from Harendra Kumar's earlier mail: >> Why not adopt (a subset of) .hs AST file format to structure both project >> and package files? > Aha, that's my preferred choice. If there is a way to restrict features and > we can allow just a subset we can have a nice configuration language which is > a real language. In fact, I have been toying around this. If we have to > express not just a package specification but a sophisticated build > configuration, we need a real language. Expressing conditionals, reuse etc > becomes a compromise in a purely declarative language. > For example make has so many built-in functions in it that it has become a > full fledged language by itself. The google bazel build uses python as the > build config language. Haskell will make a much better choice for such use > cases. Pure declarative is a pain for such use cases. > On 16 September 2016 at 16:08, Paolo Giarrusso > wrote: >> >> On 16 September 2016 at 12:13, Patrick Pelletier >> wrote: >> > On 9/16/16 2:36 AM, Paolo Giarrusso wrote: >> >> We're talking about *three* options: >> >> 1. syntax for pure Haskell values, which I'll call HSON (Haskell >> >> jSON). That's just an alternative to YAML/TOML/... That would need >> >> extensions to allow omitting optional fields entirely. >> >> 2. a pure Haskell embedded domain-specific language (EDSL) that simply >> >> generates cabal description records (GenericPackageDescription >> >> values). That would allow abstraction over some patterns but not much >> >> more. But that alone is already an argument for EDSLs—the one Harendra >> >> already presented. >> >> 3. a Haskell embedded domain-specific language (EDSL) designed for an >> >> extensible build tool, like Clojure's (apparently), SBT for Scala or >> >> many others. That would potentially be a rabbit hole leading to a >> >> rather *different* tool—with a different package format to boot. That >> >> can't work as long as all libraries have to be built using the same >> >> tool. But stack and cabal are really about how to manage package >> >> databases/GHC/external environments, while extensible build tools are >> >> about (a more powerful form) of writing custom setup scripts. I >> >> suspect some extensions might be easier if more of the actual building >> >> was done by the setup script, but I'm not sure. -- Paolo G. Giarrusso - Ph.D. Student, Tübingen University http://ps.informatik.uni-tuebingen.de/team/giarrusso/ ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
This seems to have gone into a different direction. The original point was about the package specification format and not expressing a full fledged build system. That is an entirely different ballgame. The main point of the thread was whether it makes sense to use a single specification format for both stack and cabal install (YAML vs .cabal and then TOML came into picture). Haskell does not seem to be a choice for a package specification format unless we have a very different goal in mind. -harendra On 16 September 2016 at 16:08, Paolo Giarrusso wrote: > On 16 September 2016 at 12:13, Patrick Pelletier > wrote: > > On 9/16/16 2:36 AM, Paolo Giarrusso wrote: > >> > >> (Resending from right address) > >> > >> We're talking about *three* options: > >> 1. syntax for pure Haskell values, which I'll call HSON (Haskell > >> jSON). That's just an alternative to YAML/TOML/... That would need > >> extensions to allow omitting optional fields entirely. > >> 2. a pure Haskell embedded domain-specific language (EDSL) that simply > >> generates cabal description records (GenericPackageDescription > >> values). That would allow abstraction over some patterns but not much > >> more. But that alone is already an argument for EDSLs—the one Harendra > >> already presented. > >> 3. a Haskell embedded domain-specific language (EDSL) designed for an > >> extensible build tool, like Clojure's (apparently), SBT for Scala or > >> many others. That would potentially be a rabbit hole leading to a > >> rather *different* tool—with a different package format to boot. That > >> can't work as long as all libraries have to be built using the same > >> tool. But stack and cabal are really about how to manage package > >> databases/GHC/external environments, while extensible build tools are > >> about (a more powerful form) of writing custom setup scripts. I > >> suspect some extensions might be easier if more of the actual building > >> was done by the setup script, but I'm not sure. > > > > > > Options 2 and 3 both require running Haskell code at build time. > > > But if all packages had to use the new EDSL, then cross-compilation > would essentially become impossible. > > "All packages migrate to new format" doesn't seem really a plausible > option, as I already hinted in the text you quote. > There are multiple JVM build tools because they're interoperable (like > cabal-install and Stack): each library picks its own build tool, but > they can still be linked together. > Hpack generates cabal files, stack reuses cabal or hpack files. > > In principle, option 2 just needs a non-cross-compiled program to > produce a package description—say by producing a cabal file. You just > need to runghc it, either via ghci or by compiling and running a > binary. Option 3 can be trickier depending on details, but the as long > as you account for cross-compilation in the design it should be > doable. For Template Haskell the problem is deeper (see > http://blog.ezyang.com/2016/07/what-template-haskell-gets- > wrong-and-racket-gets-right/), > so let's *not* use it here. > -- > Paolo G. Giarrusso - Ph.D. Student, Tübingen University > http://ps.informatik.uni-tuebingen.de/team/giarrusso/ > ___ > Haskell-community mailing list > Haskell-community@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community > ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
On 16 September 2016 at 12:13, Patrick Pelletier wrote: > On 9/16/16 2:36 AM, Paolo Giarrusso wrote: >> >> (Resending from right address) >> >> We're talking about *three* options: >> 1. syntax for pure Haskell values, which I'll call HSON (Haskell >> jSON). That's just an alternative to YAML/TOML/... That would need >> extensions to allow omitting optional fields entirely. >> 2. a pure Haskell embedded domain-specific language (EDSL) that simply >> generates cabal description records (GenericPackageDescription >> values). That would allow abstraction over some patterns but not much >> more. But that alone is already an argument for EDSLs—the one Harendra >> already presented. >> 3. a Haskell embedded domain-specific language (EDSL) designed for an >> extensible build tool, like Clojure's (apparently), SBT for Scala or >> many others. That would potentially be a rabbit hole leading to a >> rather *different* tool—with a different package format to boot. That >> can't work as long as all libraries have to be built using the same >> tool. But stack and cabal are really about how to manage package >> databases/GHC/external environments, while extensible build tools are >> about (a more powerful form) of writing custom setup scripts. I >> suspect some extensions might be easier if more of the actual building >> was done by the setup script, but I'm not sure. > > > Options 2 and 3 both require running Haskell code at build time. > But if all packages had to use the new EDSL, then cross-compilation would > essentially become impossible. "All packages migrate to new format" doesn't seem really a plausible option, as I already hinted in the text you quote. There are multiple JVM build tools because they're interoperable (like cabal-install and Stack): each library picks its own build tool, but they can still be linked together. Hpack generates cabal files, stack reuses cabal or hpack files. In principle, option 2 just needs a non-cross-compiled program to produce a package description—say by producing a cabal file. You just need to runghc it, either via ghci or by compiling and running a binary. Option 3 can be trickier depending on details, but the as long as you account for cross-compilation in the design it should be doable. For Template Haskell the problem is deeper (see http://blog.ezyang.com/2016/07/what-template-haskell-gets-wrong-and-racket-gets-right/), so let's *not* use it here. -- Paolo G. Giarrusso - Ph.D. Student, Tübingen University http://ps.informatik.uni-tuebingen.de/team/giarrusso/ ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
On 9/16/16 2:36 AM, Paolo Giarrusso wrote: (Resending from right address) We're talking about *three* options: 1. syntax for pure Haskell values, which I'll call HSON (Haskell jSON). That's just an alternative to YAML/TOML/... That would need extensions to allow omitting optional fields entirely. 2. a pure Haskell embedded domain-specific language (EDSL) that simply generates cabal description records (GenericPackageDescription values). That would allow abstraction over some patterns but not much more. But that alone is already an argument for EDSLs—the one Harendra already presented. 3. a Haskell embedded domain-specific language (EDSL) designed for an extensible build tool, like Clojure's (apparently), SBT for Scala or many others. That would potentially be a rabbit hole leading to a rather *different* tool—with a different package format to boot. That can't work as long as all libraries have to be built using the same tool. But stack and cabal are really about how to manage package databases/GHC/external environments, while extensible build tools are about (a more powerful form) of writing custom setup scripts. I suspect some extensions might be easier if more of the actual building was done by the setup script, but I'm not sure. Options 2 and 3 both require running Haskell code at build time. This presents problems in a couple of cases: Cross compilation: There are already a couple of cases where we need to run Haskell code at build time: Template Haskell, and custom Setup.hs. Neither of these are supported in cross-compilation. (The former is a ghc issue, while the latter is a Cabal issue.) So I'm assuming that the new Haskell-based EDSL wouldn't work in cross-compilation, either. The difference is that Template Haskell and custom Setup.hs are only used by some packages. But if all packages had to use the new EDSL, then cross-compilation would essentially become impossible. Platforms without ghci: Even when not cross-compiling, some platforms don't support ghci. (This is usually the less popular platforms. Not too long ago, it even included ARM.) ghci support is necessary for Template Haskell, and I assume the EDSL would work the same way. So then less popular platforms would be left out in the cold. --Patrick ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
this may be one of the 3 points on Paolo's list. In case it is not, here is another option (4?): - define .hs data records for project config, package configs - write export tools to export config records to existing formats: cabal stack yaml ... this way, there is no need to revise the current workflow or modify tools. However we define a common standard content structure, most users do not need to worry about .cabal, .yaml syntax ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
[Haskell-community] [Haskell-cafe] Standard package file format
(Resending from right address) We're talking about *three* options: 1. syntax for pure Haskell values, which I'll call HSON (Haskell jSON). That's just an alternative to YAML/TOML/... That would need extensions to allow omitting optional fields entirely. 2. a pure Haskell embedded domain-specific language (EDSL) that simply generates cabal description records (GenericPackageDescription values). That would allow abstraction over some patterns but not much more. But that alone is already an argument for EDSLs—the one Harendra already presented. 3. a Haskell embedded domain-specific language (EDSL) designed for an extensible build tool, like Clojure's (apparently), SBT for Scala or many others. That would potentially be a rabbit hole leading to a rather *different* tool—with a different package format to boot. That can't work as long as all libraries have to be built using the same tool. But stack and cabal are really about how to manage package databases/GHC/external environments, while extensible build tools are about (a more powerful form) of writing custom setup scripts. I suspect some extensions might be easier if more of the actual building was done by the setup script, but I'm not sure. On 16 September 2016 at 10:57, yogsototh wrote: > >> I guess the overriding question I have here is: what is the PROBLEM being >> solved? > > > Let me share my experience with Clojure and lein. They use a clojure > hash-map for their configuration. So yes arbitrary code could be executed > and I believe this is a _very good thing_. > > Why? Because it makes it very easy to add sub-configuration that can be used > by third party plugin. For example: > > - a plugin that help the use of environment variables (lein-environ) which > is really helpful for application development (not so much for library > development) > - a plugin that use S3 for our private dependencies (not supported by > default by lein) > > > For deployment: we were able to add request to our API server that provide > not only the written version but also the git commit hash. So we could be > certain of the version of the server. Too much time there were sys/admin > deployment errors. And that could only be achieved because we were able to > run arbitrary command in the project description file. > > I certainly forget many other advantages of having a package description > format which is simply a data structure in the hosted language. But this has > by far my preference. > > - cabal is ok, but very imperfect, I generally need to have a lot of > copy/paste, I need to change it very often while writing application with > many dependencies > - JSON/YAML/TOML are simply not powerful enough to match all semantics we > might need to configure a project. For example we might want to have Set > instead of List for some properties. Or I don't know maybe ternary tree > structures. > > The point is: we pay a price by adding a step between the semantic and the > syntax. > While if our configuration format was in Haskell we could express the > semantic more directly. > > ___ > Haskell-community mailing list > Haskell-community@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community > -- Paolo G. Giarrusso - Ph.D. Student, Tübingen University http://ps.informatik.uni-tuebingen.de/team/giarrusso/ ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
.. for interop with other packagers / builders, .hs compatible config content could be transformed / exported to other formats. .hs -> YAML, JSON, ... is likely to be possible and easier than the other way around. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
> I guess the overriding question I have here is: what is the PROBLEM being > solved? > Let me share my experience with Clojure and lein. They use a clojure hash-map for their configuration. So yes arbitrary code could be executed and I believe this is a _very good thing_. Why? Because it makes it very easy to add sub-configuration that can be used by third party plugin. For example: - a plugin that help the use of environment variables (lein-environ) which is really helpful for application development (not so much for library development) - a plugin that use S3 for our private dependencies (not supported by default by lein) For deployment: we were able to add request to our API server that provide not only the written version but also the git commit hash. So we could be certain of the version of the server. Too much time there were sys/admin deployment errors. And that could only be achieved because we were able to run arbitrary command in the project description file. I certainly forget many other advantages of having a package description format which is simply a data structure in the hosted language. But this has by far my preference. - cabal is ok, but very imperfect, I generally need to have a lot of copy/paste, I need to change it very often while writing application with many dependencies - JSON/YAML/TOML are simply not powerful enough to match all semantics we might need to configure a project. For example we might want to have Set instead of List for some properties. Or I don't know maybe ternary tree structures. The point is: we pay a price by adding a step between the semantic and the syntax. While if our configuration format was in Haskell we could express the semantic more directly. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
The discussion originated in an earlier thread from a question about the possibility of using the same format across different tools, cabal and stack which currently use different file formats. If they have to use the same format what that format should be. On 16 September 2016 at 13:54, Chris Smith wrote: > I guess the overriding question I have here is: what is the PROBLEM being > solved? I know of basically no beginners who were confused or intimidated > by the syntax of Cabal's file format. It's fairly commonplace for > beginners to be confused by the *semantics*: which fields are needed and > what they mean, how package version bounds work, what flags are and how > they interact with dependencies, the relationship between libraries and > executables defined in the same file, etc. But the syntax? It's just not > an issue. I'm not sure what it means to say that people have to "learn" > it, because in introducing dozens of people to building things in Haskell, > I've never seen that learning process even be noticeable, much less an > impediment. > > With this in mind, a lot of the statements about these various languages > are not entirely convincing. That it's a superset of JSON? It's not clear > why this matters. A psychological impression of complexity? Just not > anything I've seen evidence of. Indeed, aside from the rather painful > many-years-long migration, the *cost* (though certainly not a prohibitive > one) of moving to something like YAML or TOML is that they have a bit > louder syntax, that demands more attention and feels more complex. > > There is one substantial disadvantage I'd point out to the Cabal file > format as it stands, and that's that it's pretty non-obvious how to parse > it, so we will always struggle to interact with it from automated tools, > unless those tools are also written in Haskell and can use the Cabal > library. That's a real concern; pragmatic large-scale build environments > are not tied to specific languages, and include a variety of ad-hoc > third-party tooling that needs to be integrated, and Cabal remains opaque > to them. But that doesn't seem to be what's motivating this conversation. > > On Thu, Sep 15, 2016 at 11:20 PM, Harendra Kumar > wrote: > >> I am starting a new thread for the package file format related discussion. >> >> From a developer's perspective, the major benefit of a standard and >> widely adopted format and is that people can utilize their knowledge >> acquired from elsewhere, they do not have to go through and learn >> differently looking and incomplete documentation of different tools. The >> benefit of a common config specification is that developers can choose >> tools freely without worrying about learning the same concepts presented in >> different ways. >> >> Multiple formats flying around also create a psychological impression of >> complexity in the ecosystem for newcomers. If we have consistency there are >> better chances of attracting more people to the language ecosystem. >> >> I gather the following from the discussion till now: >> >> * We have cabal, YAML and TOML as potential candidates for a common >> package format which can additionally incorporate the concept of >> snapshots/package collections and potentially more extensions useful across >> build tools. >> >> * cabal has the benefit of incumbency and backward compatibility, it has >> shortcomings which are being addressed but it is still a format which is >> very specific to Haskell ecosystem. It is not a standard and not going to >> become one. We have to always deal with it ourselves and everyone coming to >> Haskell will have to learn it. >> >> * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A >> significant chunk of developer community is already familiar with it. It is >> being used by stack and by hpack as an alternative to cabal format. The >> complaint against it is that the specification/implementation is overly >> complex. >> >> * TOML (https://github.com/toml-lang/toml) is promising, simpler than >> YAML and is being used by a few important projects but is still evolving >> and is not completely stable. On a first glance it looks pretty simple and >> a lot of other tools use a similar config format. It is aiming to become a >> standard and aiming for a wider adoption. >> >> As a next step we can perhaps do an hpack like experiment using the TOML >> format. That way we will have some experience with that as well and get to >> know if there are any potential problems expressing the existing cabal >> files. >> >> More thoughts, opinions on the topic will help create a better >> understanding about it. >> >> -harendra >> >> ___ >> Haskell-Cafe mailing list >> To (un)subscribe, modify options or view archives go to: >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe >> Only members subscribed via the mailman list are allowed to post. >> > > >
Re: [Haskell-community] [Haskell-cafe] Standard package file format
> what is the PROBLEM being solved? by making config files follow .hs syntax, cabal file structure may be defined as a data record. This would make it clear, which fields are compulsory, which are optional. Enums may be used. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
I guess the overriding question I have here is: what is the PROBLEM being solved? I know of basically no beginners who were confused or intimidated by the syntax of Cabal's file format. It's fairly commonplace for beginners to be confused by the *semantics*: which fields are needed and what they mean, how package version bounds work, what flags are and how they interact with dependencies, the relationship between libraries and executables defined in the same file, etc. But the syntax? It's just not an issue. I'm not sure what it means to say that people have to "learn" it, because in introducing dozens of people to building things in Haskell, I've never seen that learning process even be noticeable, much less an impediment. With this in mind, a lot of the statements about these various languages are not entirely convincing. That it's a superset of JSON? It's not clear why this matters. A psychological impression of complexity? Just not anything I've seen evidence of. Indeed, aside from the rather painful many-years-long migration, the *cost* (though certainly not a prohibitive one) of moving to something like YAML or TOML is that they have a bit louder syntax, that demands more attention and feels more complex. There is one substantial disadvantage I'd point out to the Cabal file format as it stands, and that's that it's pretty non-obvious how to parse it, so we will always struggle to interact with it from automated tools, unless those tools are also written in Haskell and can use the Cabal library. That's a real concern; pragmatic large-scale build environments are not tied to specific languages, and include a variety of ad-hoc third-party tooling that needs to be integrated, and Cabal remains opaque to them. But that doesn't seem to be what's motivating this conversation. On Thu, Sep 15, 2016 at 11:20 PM, Harendra Kumar wrote: > I am starting a new thread for the package file format related discussion. > > From a developer's perspective, the major benefit of a standard and widely > adopted format and is that people can utilize their knowledge acquired from > elsewhere, they do not have to go through and learn differently looking and > incomplete documentation of different tools. The benefit of a common config > specification is that developers can choose tools freely without worrying > about learning the same concepts presented in different ways. > > Multiple formats flying around also create a psychological impression of > complexity in the ecosystem for newcomers. If we have consistency there are > better chances of attracting more people to the language ecosystem. > > I gather the following from the discussion till now: > > * We have cabal, YAML and TOML as potential candidates for a common > package format which can additionally incorporate the concept of > snapshots/package collections and potentially more extensions useful across > build tools. > > * cabal has the benefit of incumbency and backward compatibility, it has > shortcomings which are being addressed but it is still a format which is > very specific to Haskell ecosystem. It is not a standard and not going to > become one. We have to always deal with it ourselves and everyone coming to > Haskell will have to learn it. > > * YAML (http://yaml.org/spec/1.2/spec.html) is standard and popular. A > significant chunk of developer community is already familiar with it. It is > being used by stack and by hpack as an alternative to cabal format. The > complaint against it is that the specification/implementation is overly > complex. > > * TOML (https://github.com/toml-lang/toml) is promising, simpler than > YAML and is being used by a few important projects but is still evolving > and is not completely stable. On a first glance it looks pretty simple and > a lot of other tools use a similar config format. It is aiming to become a > standard and aiming for a wider adoption. > > As a next step we can perhaps do an hpack like experiment using the TOML > format. That way we will have some experience with that as well and get to > know if there are any potential problems expressing the existing cabal > files. > > More thoughts, opinions on the topic will help create a better > understanding about it. > > -harendra > > ___ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
> So if you can execute arbitrary code in a new-gen cabal file, apart from the security aspects, ... well config files could use different (not .hs) extensions. They could use their own Prelude and not allow importing other modules. The main benefit is to reuse existing parsers and simplify code-config sync. ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
The more power you put into the package file description, the harder it is for the surrounding ecosystem to reason about it. So if you can execute arbitrary code in a new-gen cabal file, apart from the security aspects, it becomes difficult to be sure what is actually being specified, if you do not reproduce the original environment when evaluating the file. Alan On Fri, Sep 16, 2016 at 9:18 AM, Harendra Kumar wrote: > On 16 September 2016 at 12:35, Imants Cekusins wrote: > >> Why not adopt (a subset of) .hs AST file format to structure both project >> and package files? >> > > Aha, that's my preferred choice. If there is a way to restrict features > and we can allow just a subset we can have a nice configuration language > which is a real language. In fact, I have been toying around this. If we > have to express not just a package specification but a sophisticated build > configuration, we need a real language. Expressing conditionals, reuse etc > becomes a compromise in a purely declarative language. > > For example make has so many built-in functions in it that it has become a > full fledged language by itself. The google bazel build uses python as the > build config language. Haskell will make a much better choice for such use > cases. Pure declarative is a pain for such use cases. > > -harendra > > ___ > Haskell-Cafe mailing list > To (un)subscribe, modify options or view archives go to: > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe > Only members subscribed via the mailman list are allowed to post. > ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
On 16 September 2016 at 12:35, Imants Cekusins wrote: > Why not adopt (a subset of) .hs AST file format to structure both project > and package files? > Aha, that's my preferred choice. If there is a way to restrict features and we can allow just a subset we can have a nice configuration language which is a real language. In fact, I have been toying around this. If we have to express not just a package specification but a sophisticated build configuration, we need a real language. Expressing conditionals, reuse etc becomes a compromise in a purely declarative language. For example make has so many built-in functions in it that it has become a full fledged language by itself. The google bazel build uses python as the build config language. Haskell will make a much better choice for such use cases. Pure declarative is a pain for such use cases. -harendra ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
Re: [Haskell-community] [Haskell-cafe] Standard package file format
Why not adopt (a subset of) .hs AST file format to structure both project and package files? This would simplify parsing config files as well as syncing code and config files in IDEs. To draw an analogy, JSON derives from JavaScript. Isn't this a precedent? ___ Haskell-community mailing list Haskell-community@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-community
