Re: base package (was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

Am Mittwoch, den 13.02.2013, 11:34 +0200 schrieb Roman Cheplyaka:
> It would be great to have a portable base, without any GHC-specific
> stuff in it. After all, modules like Control.Monad or Data.Foldable
> are pure Haskell2010.

while you are considering to split base, please also consider separating
IO out. We can expect compiling Haskell to, say JavaScript or other
targets that are not processes in the usual sense. For these IO might
not make sense.

Having something below base that provides the pure stuff (common data
structures etc.) would enable libraries to easily say: „My algorithm can
be used in normal programs as well as in programs that are compiled to
JS“ by not depending on base, but on, say, pure-base.

Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (was: GHC 7.8 release?)

[Stephen Paul Weber]

Somebody claiming to be Roman Cheplyaka wrote:

* Simon Marlow  [2013-02-13 09:00:15+]

It's feasible to split base, but to a first approximation what you
end up with is base renamed to ghc-base, and then the new base
contains just stub modules that re-export stuff from ghc-base.


It would be great to have a portable base, without any GHC-specific
stuff in it. After all, modules like Control.Monad or Data.Foldable are
pure Haskell2010.


+1

--
Stephen Paul Weber, @singpolyma
See  for how I prefer to be contacted
edition right joseph


signature.asc
Description: Digital signature
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Ian Lynagh]

On Wed, Feb 13, 2013 at 06:28:22PM +0100, Joachim Breitner wrote:
> 
> Am Mittwoch, den 13.02.2013, 13:58 + schrieb Ian Lynagh:
> > If we go this route, then we would probably want to end up without a
> > package called 'base', and then to make a new package called 'base'
> > that just re-exports modules from all the new packages.
> 
> can you transparently re-export a module from another package? I.e. if
> base depends on io, IO provides System.IO, is there a way for base to
> tell ghc to pretend that System.IO is in base, but that there is no
> conflict if io happens to be un-hidden as well.

No. But there are currently no packages that depend on both base and io,
and anyone adding a dependency on io would remove the base dependency at
the same time.

> It seems that something like this would be required to move modules from
> base to something below it without breaking existing code.

I don't see why that's necessary. base would end up containing a load of
modules that look something like

{-# LANGUAGE PackageImports #-}
module System.IO (module X) where

import "io" System.IO as X

> Also, if it works that smooth, this would not have to be one big
> reorganization, but could be done piece by piece.

It's tricky to do it piece by piece. It's hard to remove individual
sensible pieces in the first place, and it means that you can't
subsequently move modules between packages later without breaking code
depending on the new packages.

> > The disadvantage is that, at some point between the first release and
> > the release that removes base, each package will have to have its
> > dependencies updated.
> 
> Why remove base? If it is just a list of dependencies and list of
> modules to be re-exported, then keeping it (but advocate that it should
> not be used) should not be too much a burden.

* Any package using it doesn't benefit from the reduced version bumps,
  so we do actually want packages to move away from it

* Even though base (probably) wouldn't require a lot of work at any one
  time, it would require a little work every now and again, and that
  adds up to a lot of work

* Any time a module is added to one of the new packages, either we'd
  have to spend time adding it to base too, or packages continuing to
  use base wouldn't (easily) be able to use that new module.

> (This is assuming that the reorganizing should not change existing
> module names. If your plan was to give the modules new names, this
> problem does not exist, but I’d rather prefer the less intrusive
> approach.)

The odd module might be renamed, and there will probably be a handful of
definitions that move from one module to another, but for the most part
I was envisaging that we'd end up with the same modules exporting the
same things.


Thanks
Ian


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

I have started a wikipage with the list of all modules from base, for a
first round of shuffling, grouping and brainstorming:

http://hackage.haskell.org/trac/ghc/wiki/SplitBase


Am Mittwoch, den 13.02.2013, 18:09 + schrieb Ian Lynagh:
> On Wed, Feb 13, 2013 at 06:28:22PM +0100, Joachim Breitner wrote:
> > Am Mittwoch, den 13.02.2013, 13:58 + schrieb Ian Lynagh:
> > > If we go this route, then we would probably want to end up without a
> > > package called 'base', and then to make a new package called 'base'
> > > that just re-exports modules from all the new packages.
> > 
> > can you transparently re-export a module from another package? I.e. if
> > base depends on io, IO provides System.IO, is there a way for base to
> > tell ghc to pretend that System.IO is in base, but that there is no
> > conflict if io happens to be un-hidden as well.
> 
> No. But there are currently no packages that depend on both base and io,
> and anyone adding a dependency on io would remove the base dependency at
> the same time.

hmm, that reminds me of how haskell98 was handled, and it was slightly
annoying when haskell98 and base eventually were made to conflict, and
we had to patch some unmaintained packages.

Ok, in this case io would be introduced with the intention of being used
exclusive from base. So as long as we make sure that the set of modules
exported by base is always the union of all modules provided by package
that have any module in common with base this would be fine.

(Why this condition? Imagine io adding IO.GreatModule without base also
providing the module. Then a program that still uses base cannot use
IO.GreatModule without fixing the dependencies _now_ (or using package
imports everywhere). It would be nice if library authors allowed to do
the change whenever convenient.)

> > Also, if it works that smooth, this would not have to be one big
> > reorganization, but could be done piece by piece.
> 
> It's tricky to do it piece by piece. It's hard to remove individual
> sensible pieces in the first place, and it means that you can't
> subsequently move modules between packages later without breaking code
> depending on the new packages.

Agreed.

> > > The disadvantage is that, at some point between the first release and
> > > the release that removes base, each package will have to have its
> > > dependencies updated.
> > 
> > Why remove base? If it is just a list of dependencies and list of
> > modules to be re-exported, then keeping it (but advocate that it should
> > not be used) should not be too much a burden.
> 
> * Any package using it doesn't benefit from the reduced version bumps,
>   so we do actually want packages to move away from it

We want them to do so. We should not force them (most surely will...)

> * Even though base (probably) wouldn't require a lot of work at any one
>   time, it would require a little work every now and again, and that
>   adds up to a lot of work

Hopefully it is just updating the set of modules to be exported, sounds
like it could be automated, given a list of packages.

> * Any time a module is added to one of the new packages, either we'd
>   have to spend time adding it to base too, or packages continuing to
>   use base wouldn't (easily) be able to use that new module.

Hence we should add them; shouldn’t be too much work.


After every larger change to base I am forced to touch old, hardly
maintained code that I do not know, to get the packages working in
Debian again. Hence my plea for staying compatible as long as feasible.

Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Stephen Paul Weber]

Somebody signing messages as Joachim Breitner wrote:

I have started a wikipage with the list of all modules from base, for a
first round of shuffling, grouping and brainstorming:

http://hackage.haskell.org/trac/ghc/wiki/SplitBase


Looks like a good start!

Here's an idea: why not use the `haskell2010` package as one of the 
groupings?  It seems like this sort of reorganisation could help solve the 
problem we currently have where one cannot using any of the features of 
`base` along with the `haskell2010` modules.


--
Stephen Paul Weber, @singpolyma
See  for how I prefer to be contacted
edition right joseph


signature.asc
Description: Digital signature
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Felipe Almeida Lessa]

On Wed, Feb 13, 2013 at 4:32 PM, Joachim Breitner
 wrote:
>> No. But there are currently no packages that depend on both base and io,
>> and anyone adding a dependency on io would remove the base dependency at
>> the same time.
>
> hmm, that reminds me of how haskell98 was handled, and it was slightly
> annoying when haskell98 and base eventually were made to conflict, and
> we had to patch some unmaintained packages.
>
> Ok, in this case io would be introduced with the intention of being used
> exclusive from base. So as long as we make sure that the set of modules
> exported by base is always the union of all modules provided by package
> that have any module in common with base this would be fine.
>
> (Why this condition? Imagine io adding IO.GreatModule without base also
> providing the module. Then a program that still uses base cannot use
> IO.GreatModule without fixing the dependencies _now_ (or using package
> imports everywhere). It would be nice if library authors allowed to do
> the change whenever convenient.)

There should also be a condition stating that base should only
re-export modules, and that those re-exports need to have the same
name on another package.  This condition guarantees that the only
thing you need to change is the import list, and even this change
could be (at least partially) automated via a tool take took all your
imports and decided which new packages export them.

-- 
Felipe.

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Ian Lynagh]

On Wed, Feb 13, 2013 at 07:32:06PM +0100, Joachim Breitner wrote:
> 
> I have started a wikipage with the list of all modules from base, for a
> first round of shuffling, grouping and brainstorming:
> 
> http://hackage.haskell.org/trac/ghc/wiki/SplitBase

Great, thanks for taking the lead on this!

> > > > The disadvantage is that, at some point between the first release and
> > > > the release that removes base, each package will have to have its
> > > > dependencies updated.
> > > 
> > > Why remove base? If it is just a list of dependencies and list of
> > > modules to be re-exported, then keeping it (but advocate that it should
> > > not be used) should not be too much a burden.
> > 
> > * Any package using it doesn't benefit from the reduced version bumps,
> >   so we do actually want packages to move away from it
> 
> We want them to do so. We should not force them (most surely will...)

A lot of packages won't react until something actually breaks.

(and I suspect many are unmaintained and unused, and won't react even
once it does break).

> > * Even though base (probably) wouldn't require a lot of work at any one
> >   time, it would require a little work every now and again, and that
> >   adds up to a lot of work
> 
> Hopefully it is just updating the set of modules to be exported, sounds
> like it could be automated, given a list of packages.
> 
> > * Any time a module is added to one of the new packages, either we'd
> >   have to spend time adding it to base too, or packages continuing to
> >   use base wouldn't (easily) be able to use that new module.
> 
> Hence we should add them; shouldn’t be too much work.

I realised that there's actually no reason that the new 'base' package
has to come with GHC (even immediately after the break-up); it can just
be a package on Hackage (and, if desired, in the Haskell Platform).

So it could easily be maintained by someone else, and thus be not much
work for you, and 0 work for me  :-)


Thanks
Ian


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

Am Donnerstag, den 14.02.2013, 02:21 + schrieb Ian Lynagh:
> On Wed, Feb 13, 2013 at 07:32:06PM +0100, Joachim Breitner wrote:
> > 
> > I have started a wikipage with the list of all modules from base,
> for a
> > first round of shuffling, grouping and brainstorming:
> > 
> > http://hackage.haskell.org/trac/ghc/wiki/SplitBase
> 
> Great, thanks for taking the lead on this!

lets see how far that leads goes...

Yesterday, I experimented a bit with base’s code, first beginning with
as few modules as possible and adding what’s required; then starting
with the whole thing and trying to remove e.g. IO.

But clearly it is not easy:
 1. Int requires throw DivideByZero; Monad requires error. That
pulls in Exceptions.
 2. Exceptions require Typeable.
 3. Typeable is implemented with GHC.Fingerprint.
 4. GHC.Fingerprint is implemented with Foreign and IO.
 5. Foreign clearly needs Int and the like.

so it is not clear how to pull out a pure base without IO and Foreign.
Are there any good tricks how to break these interdependencies?

Maybe it is possible to have a pure base without the Monad class and
without some of the operations on Int that throw exceptions, but that
wold hardly be useable.


There are other issues, some avoidable (such as the hard-coded base:Num
constraint on literals); I collected a list on
http://hackage.haskell.org/trac/ghc/wiki/SplitBase



Maybe the proper is to reverse the whole approach: Leave base as it is,
and then build re-exporting smaller packages (e.g. a base-pure) on top
of it. The advantage is:
  * No need to rewrite the tightly intertwined base.
  * Libraries still have the option to have tighter dependencies.
  * Base can evolve with lots of breaking changes, as long as they
do not affect the API by the smaller packages.
  * Development of this collection can happen outside the GHC tree.
  * Alternative implementations for (some of) these packages can be
created, if the reason why they could not be moved out of base
is one of implementation, not of API

How does that sound?

Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Ian Lynagh]

On Thu, Feb 14, 2013 at 03:48:51PM +0100, Joachim Breitner wrote:
> 
> Yesterday, I experimented a bit with base’s code, first beginning with
> as few modules as possible and adding what’s required; then starting
> with the whole thing and trying to remove e.g. IO.
> 
> But clearly it is not easy:
>  1. Int requires throw DivideByZero; Monad requires error. That
> pulls in Exceptions.
>  2. Exceptions require Typeable.
>  3. Typeable is implemented with GHC.Fingerprint.
>  4. GHC.Fingerprint is implemented with Foreign and IO.
>  5. Foreign clearly needs Int and the like.
> 
> so it is not clear how to pull out a pure base without IO and Foreign.
> Are there any good tricks how to break these interdependencies?

We'll probably end up with a package (let's say ghc-bottom for now)
right at the bottom of the hierarchy which contains a minimal set of
definitions for Typeable, throw, etc, in modules like GHC.Typeable,
GHC.IO (or, if it's small and the dependencies are circular, might be
easier to have it all in a single module).

It might be possible to merge this into ghc-prim later, but don't worry
about that for now.

These definitions would then be re-exported by Data.Typeable,
Control.Exception etc higher up. Other libraries would be discouraged
from depending on ghc-bottom, either socially or by needing to jump
through extra hoops in a .cabal file fo allow it.



However, this is the hard end to start from, because the modules right
at the "bottom" depend on things much higher up, with cyclic imports.
It'll be a lot easier to pull modules off the top instead, as nothing
imports them. They also tend to be less magical (e.g. don't have
wired-in names).

Also, don't worry about necessarily pulling out /all/ the modules you
want for a package all at once. Once you have smaller chunks it'll be
easier to see what changes are necessary to move modules up/down. You
may also need to leave e.g. some of IO lower down than the io package in
a GHC.* module, and then to re-export it from io.

Once the top has been pulled at as much as possible, it'll be a lot
easier to see what's going on with the remainder, and work out what
needs to be done to break the biggest loops.

> There are other issues, some avoidable (such as the hard-coded base:Num
> constraint on literals); I collected a list on
> http://hackage.haskell.org/trac/ghc/wiki/SplitBase

Right, for things with built-in syntax you'll have to update GHC's
wired-in names as you go. This will mostly just mean changing the
modules, e.g.
gHC_ENUM = mkBaseModule (fsLit "GHC.Enum")
in compiler/prelude/PrelNames.lhs might become
gHC_ENUM = mkGhcPureModule (fsLit "GHC.Enum")
with mkGhcPureModule defined analogously to mkBaseModule, but
occasionally you might want to move a definition to another module. If
you get stuck, just yell.

> Maybe the proper is to reverse the whole approach: Leave base as it is,
> and then build re-exporting smaller packages (e.g. a base-pure) on top
> of it. The advantage is:
>   * No need to rewrite the tightly intertwined base.
>   * Libraries still have the option to have tighter dependencies.
>   * Base can evolve with lots of breaking changes, as long as they
> do not affect the API by the smaller packages.
>   * Development of this collection can happen outside the GHC tree.
>   * Alternative implementations for (some of) these packages can be
> created, if the reason why they could not be moved out of base
> is one of implementation, not of API

Disadvantages:

* No-one would use the new packages unless they come with GHC;
  e.g. not a perfect analogy, but compare the number of rev-deps
  according to http://packdeps.haskellers.com/reverse of the various
  *prelude* packages vs base:
  4831 base
 6 basic-prelude
 8 classy-prelude
 4 classy-prelude-conduit
 2 custom-prelude
 1 general-prelude
 1 modular-prelude
17 numeric-prelude
 2 prelude-extras
* If it comes with GHC, it would mean us permanently maintaining the two
  levels
* base would still be an opaque blob, with too many modules and cyclic
  imports, which makes development tricky


Thanks
Ian


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

I made a little progress after crippling GHC.Fingerprint:

The package at
https://github.com/nomeata/packages-base/tree/base-pure
(Branch base-pure) builds and contains just these modules:

./Control/Applicative.hs
./Control/Arrow.hs
./Control/Category.hs
./Control/Monad/Fix.hs
./Control/Monad.hs
./Data/Bits.hs
./Data/Bool.hs
./Data/Either.hs
./Data/Eq.hs
./Data/Foldable.hs
./Data/Function.hs
./Data/Functor.hs
./Data/Int.hs
./Data/List.hs
./Data/Maybe.hs
./Data/Monoid.hs
./Data/Ord.hs
./Data/Ratio.hs
./Data/Traversable.hs
./Data/Tuple.hs
./Data/Typeable.hs
./Data/Typeable.hs-boot
./Data/Typeable/Internal.hs
./Data/Typeable/Internal.hs-boot
./dist/build/autogen/Paths_base_pure.hs
./GHC/Base.lhs
./GHC/Char.hs
./GHC/Enum.lhs
./GHC/Err.lhs
./GHC/Err.lhs-boot
./GHC/Exception.lhs
./GHC/Fingerprint.hs
./GHC/Fingerprint/Type.hs
./GHC/Int.hs
./GHC/List.lhs
./GHC/Num.lhs
./GHC/Real.lhs
./GHC/Show.lhs
./GHC/Word.hs
./Prelude.hs (contains just $!)
./Unsafe/Coerce.hs

The crippling can be seen here:
https://github.com/nomeata/packages-base/blob/base-pure/GHC/Fingerprint.hs

So if there were a magic way of getting a working GHC.Fingerprint module
without pulling in Foreign, this would be a good start for a base free
of any trace of
 * Foreign
 * IO
 * Floating point arithmetic
 * Read
 * ST
 * Array

Alternative, a magic way of providing the functions error and
divZeroError without having to define Exceptions would do well.

I guess it is not possible to define the "data ErrorCall" without the
Exception class and somehow call the primop raise# in a way that the
error can be caught, as catch will expect something of type
"SomeException" that has the Exception dictionary bundled.

Any idea how to achieve that, without having resort to ghc-bottom as
suggested by Ian (which would be ok, but not as nice as genuine small
packages to start with)?

Greetings,
Joachim


-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Johan Tibell]

On Thu, Feb 14, 2013 at 8:45 AM, Joachim Breitner
wrote:

> ./Control/Applicative.hs
> ./Control/Arrow.hs
> ./Control/Category.hs
> ./Control/Monad/Fix.hs
> ./Control/Monad.hs
> ./Data/Bits.hs
> ./Data/Bool.hs
> ./Data/Either.hs
> ./Data/Eq.hs
> ./Data/Foldable.hs
> ./Data/Function.hs
> ./Data/Functor.hs
> ./Data/Int.hs
> ./Data/List.hs
> ./Data/Maybe.hs
> ./Data/Monoid.hs
> ./Data/Ord.hs
> ./Data/Ratio.hs
> ./Data/Traversable.hs
> ./Data/Tuple.hs
> ./Data/Typeable.hs
> ./Data/Typeable.hs-boot
> ./Data/Typeable/Internal.hs
> ./Data/Typeable/Internal.hs-boot
> ./dist/build/autogen/Paths_base_pure.hs
> ./GHC/Base.lhs
> ./GHC/Char.hs
> ./GHC/Enum.lhs
> ./GHC/Err.lhs
> ./GHC/Err.lhs-boot
> ./GHC/Exception.lhs
> ./GHC/Fingerprint.hs
> ./GHC/Fingerprint/Type.hs
> ./GHC/Int.hs
> ./GHC/List.lhs
> ./GHC/Num.lhs
> ./GHC/Real.lhs
> ./GHC/Show.lhs
> ./GHC/Word.hs
> ./Prelude.hs (contains just $!)
> ./Unsafe/Coerce.hs
>

That's great. I'm curious  I was under the impression that it was hard to
split out a pure subset as functions might call 'error' (e.g. due to
incomplete pattern matches) and that would pull in the whole I/O subsystem.
How did you avoid that?

-- Johan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

Am Donnerstag, den 14.02.2013, 13:19 -0800 schrieb Johan Tibell:
> That's great. I'm curious  I was under the impression that it was hard
> to split out a pure subset as functions might call 'error' (e.g. due
> to incomplete pattern matches) and that would pull in the whole I/O
> subsystem. How did you avoid that?

as mentioned before: By crippling GHC.Fingerprint. error foo just calls
raise (ErrorCall foo), which calls "raise# (SomeException (ErrorCall
foo)". The problem is that the definition of SomeException pulls in the
Exception type class, which pulls in Typeable, which pulls in
GHC.Fingerprint, which uses FFI and IO code to to fingerprinting...

Looking at the code it seems that the FFI is only required for MD5.
Maybe a pure implementation there is worth it, if it is not
performance-critical code. Or, another work-around, would be primops for
these commands that can be used without the FFI types and IO.

I have also removed GHC.Unicode and functions like lines that require
the FFI calls there. These seem to be the largest open issues.

Greetings,
Joachim


-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Johan Tibell]

On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner
wrote:

> Hi,
>
> Am Donnerstag, den 14.02.2013, 13:19 -0800 schrieb Johan Tibell:
> > That's great. I'm curious  I was under the impression that it was hard
> > to split out a pure subset as functions might call 'error' (e.g. due
> > to incomplete pattern matches) and that would pull in the whole I/O
> > subsystem. How did you avoid that?
>
> as mentioned before: By crippling GHC.Fingerprint. error foo just calls
> raise (ErrorCall foo), which calls "raise# (SomeException (ErrorCall
> foo)". The problem is that the definition of SomeException pulls in the
> Exception type class, which pulls in Typeable, which pulls in
> GHC.Fingerprint, which uses FFI and IO code to to fingerprinting...


I don't think having FFI far down the stack is a problem. There are lots of
pure data types we'd like in the "pure data" layer (e.g. bytestring) that
uses FFI. As long as the I/O layer itself (System.IO, the I/O manager, etc)
doesn't get pulled in there's no real problem in depending on the FFI.
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[brandon s allbery kf8nh]

On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote:
> On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner  (mailto:m...@joachim-breitner.de)> wrote:
> I don't think having FFI far down the stack is a problem. There are lots of 
> pure data types we'd like in the "pure data" layer (e.g. bytestring) that 
> uses FFI. As long as the I/O layer itself (System.IO, the I/O manager, etc) 
> doesn't get pulled in there's no real problem in depending on the FFI. 
Doesn't the FFI pull in some part of the I/O layer, though?  In particular 
threaded programs are going to end up using forkOS?

-- 
brandon s allbery kf8nh
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery
kf8nh:
> On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote:
> > On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner
> >  wrote:
> > I don't think having FFI far down the stack is a problem. There are
> > lots of pure data types we'd like in the "pure data" layer (e.g.
> > bytestring) that uses FFI. As long as the I/O layer itself
> > (System.IO, the I/O manager, etc) doesn't get pulled in there's no
> > real problem in depending on the FFI. 

I think it would be nice, also to other Haskell implementations that
might have not have FFI, to separate the really basic stuff from
pure-but-impurely-implemented stuff. At least as long as it does not
caues trouble.

GHC.Fingerprint does not need to be crippled when it is going to use a
pure hashing; I quickly added some simple fingerpriting found via
Wikipedia that was easier than MD5.
https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc
The choice is of course not final, maybe something with more bits is
desirable.

> Doesn't the FFI pull in some part of the I/O layer, though?  In
> particular threaded programs are going to end up using forkOS?

Another good reason to try to have a pure ground library.

Based on base-pure, the next step would be to check if FFI can be
provided without IO (but I doubt that is difficult), so there would be
an base-io package on top of base-pure, and then bytestring can depend
on that base-io and base-pure, while users of bytestring of course don’t
have to depend on base-io (as long as they are not using the IO-related
functions of bytestring).

Questions:
 * Does anyone have a tool to compare package APIs? It would be
interesting to diff base’s API with the combined APIs of the package we
are creating right now.
 * Currently, base-pure exports lots of modules that should not be part
of its “pure” API (all the GHC.* packages). But I assume they need to be
exported for the benefit of packages like base-io built on top. Should
we provide another package that re-exports those that are for public
consumption and is likely to have a more stable API? Again I feel the
need for packages re-exporting modules without incurring a conflict.
 * What to do with Prelude. Should it be in base-io (which is
potentially the lowest package containing everything needed) or rather
in a package of its own? Or should it live in a package of its own? Or
can we use the haskell2010 package for that somehow?
 * Should base-io provide just the IO monad and all, say, file-related
stuff in a separate package or is this going too far?


Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

RE: base package (Was: GHC 7.8 release?)

[Simon Peyton-Jones]

| > Doesn't the FFI pull in some part of the I/O layer, though?  In
| > particular threaded programs are going to end up using forkOS?
| 
| Another good reason to try to have a pure ground library.

Remember that we have UNSAFE ffi calls and SAFE ones.  

The SAFE ones may block, cause GC etc.  They involve a lot of jiggery pokery 
and I would not be surprised if that affected the I/O manager.

But UNSAFE ones are, by design, no more than "fat machine instructions" that 
are implemented by taking an out-of-line call.  They should not block.  They 
should not cause GC.  Nothing.  Think of 'sin' and 'cos' for example.

Fingerprinting is a classic example, I would have thought.

So my guess is that it should not be hard to allow UNSAFE ffi calls in the core 
(non-IO-ish) bits, leaving SAFE calls for higher up the stack.

Simon
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,

more progress: On top of base-pure, I created base-io involving GHC/IO
and everything required to build it (which pulled in ST, some of Foreign
and unfortunately some stuff related to Handles and Devices, because it
is mentioned in IOException). This is the list of modules:

Foreign.C.Types,
Foreign.ForeignPtr,
Foreign.ForeignPtr.Imp,
Foreign.ForeignPtr.Safe,
Foreign.ForeignPtr.Unsafe,
Foreign.Ptr,
Foreign.Storable,
GHC.ForeignPtr,
GHC.IO.BufferedIO,
GHC.IO.Buffer,
GHC.IO.Device,
GHC.IO.Encoding.Types,
GHC.IO.Exception,
GHC.IO.Handle.Types,
GHC.IO,
GHC.IORef,
GHC.MVar,
GHC.Ptr,
GHC.Stable,
GHC.ST,
GHC.Storable,
GHC.STRef

It is on a different branch on my github repo:
https://github.com/nomeata/packages-base/tree/base-io

GHC would complain that the CInt type is not valid in a ffi call
(probably due to the different package name), so I replaced foreign
declarations by regular ones defined using “undefined” – after all I’m
just trying to discover how things can be split at all and just work
towards building stuff.

ST can probably be pulled below this package, after all it is quite
pure. Either a package of its own, or in base-pure.

Greetings,
Joachim

-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Simon Marlow]

On 15/02/13 09:36, Simon Peyton-Jones wrote:

| > Doesn't the FFI pull in some part of the I/O layer, though?  In
| > particular threaded programs are going to end up using forkOS?
|
| Another good reason to try to have a pure ground library.

Remember that we have UNSAFE ffi calls and SAFE ones.

The SAFE ones may block, cause GC etc.  They involve a lot of jiggery pokery 
and I would not be surprised if that affected the I/O manager.

But UNSAFE ones are, by design, no more than "fat machine instructions" that 
are implemented by taking an out-of-line call.  They should not block.  They should not 
cause GC.  Nothing.  Think of 'sin' and 'cos' for example.

Fingerprinting is a classic example, I would have thought.

So my guess is that it should not be hard to allow UNSAFE ffi calls in the core 
(non-IO-ish) bits, leaving SAFE calls for higher up the stack.


Actually as far as the Haskell-level API goes, there's no difference 
between safe and unsafe FFI calls, the difference is all in the codegen. 
 I don't think safe calls cause any more difficulties for splitting up 
the base.


Cheers,
Simon





___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Simon Marlow]

On 15/02/13 08:36, Joachim Breitner wrote:

Hi,

Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery
kf8nh:

On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote:

On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner
 wrote:
I don't think having FFI far down the stack is a problem. There are
lots of pure data types we'd like in the "pure data" layer (e.g.
bytestring) that uses FFI. As long as the I/O layer itself
(System.IO, the I/O manager, etc) doesn't get pulled in there's no
real problem in depending on the FFI.


I think it would be nice, also to other Haskell implementations that
might have not have FFI, to separate the really basic stuff from
pure-but-impurely-implemented stuff. At least as long as it does not
caues trouble.

GHC.Fingerprint does not need to be crippled when it is going to use a
pure hashing; I quickly added some simple fingerpriting found via
Wikipedia that was easier than MD5.
https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc
The choice is of course not final, maybe something with more bits is
desirable.


Remember that fingerprinting is not hashing.  For fingerprinting we need 
to have a realistic expectation of no collisions.  I don't think FNV is 
suitable.


I'm sure it would be possible to replace the C md5 code with some 
Haskell.  Performance *is* important here though - Typeable is in the 
inner loop of certain generic programming libraries, like SYB.


Cheers,
Simon


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Simon Marlow]

On 15/02/13 12:22, Joachim Breitner wrote:

Hi,

more progress: On top of base-pure, I created base-io involving GHC/IO
and everything required to build it (which pulled in ST, some of Foreign
and unfortunately some stuff related to Handles and Devices, because it
is mentioned in IOException). This is the list of modules:

 Foreign.C.Types,
 Foreign.ForeignPtr,
 Foreign.ForeignPtr.Imp,
 Foreign.ForeignPtr.Safe,
 Foreign.ForeignPtr.Unsafe,
 Foreign.Ptr,
 Foreign.Storable,
 GHC.ForeignPtr,
 GHC.IO.BufferedIO,
 GHC.IO.Buffer,
 GHC.IO.Device,
 GHC.IO.Encoding.Types,
 GHC.IO.Exception,
 GHC.IO.Handle.Types,
 GHC.IO,
 GHC.IORef,
 GHC.MVar,
 GHC.Ptr,
 GHC.Stable,
 GHC.ST,
 GHC.Storable,
 GHC.STRef


You have a random collection of modules here :)

I think you want to have the IO *monad* (GHC.IO) live in a lower layer, 
separate from the IO *library* (GHC.IO.Device and so on).  Every Haskell 
implementation will need the IO monad, but they might want to replace 
the IO library with something else.


Things like GHC.IORef, GHC.MVar can all live in a low-down layer because 
they're just wrappers over the primops.


Cheers,
Simon


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Wolfram Kahl]

On Thu, Feb 14, 2013 at 03:48:51PM +0100, Joachim Breitner wrote:
> 
> Yesterday, I experimented a bit with base’s code, [...]
>
> Maybe the proper is to reverse the whole approach: Leave base as it is,
> and then build re-exporting smaller packages (e.g. a base-pure) on top
> of it. The advantage is:
>   * No need to rewrite the tightly intertwined base.
>   * Libraries still have the option to have tighter dependencies.
>   * Base can evolve with lots of breaking changes, as long as they
> do not affect the API by the smaller packages.
>   * Development of this collection can happen outside the GHC tree.
>   * Alternative implementations for (some of) these packages can be
> created, if the reason why they could not be moved out of base
> is one of implementation, not of API
> 
> How does that sound?

Essentially good to me...


One might consider instead (as has been proposed before, I believe),
to rename the current ``base'' to something like ``ghc-base''
which is not intended to be depended on by packages not shipped with GHC
(that is, by default ``hidden'' in ghc-pkg), and instead export:
  base   with a very stable interface
  io with a very stable interface
  GHCwith a probably rapidly evolving interface.
  *  possibly other packages giving access to internals

Most packages that currently depend on ``base'' would then depend
only on ``base'' and possibly ``io'', and by virtue of the stability
of these two interfaces would therefore not be affected
by most GHC releases.

This would effectively be
   ``splitting the interfaces GHC and io out from base''
instead of
   ``deprecating base and replacing it with the three new interfaces
 base-pure, io, and GHC''.

That choice is possibly mostly a matter of taste ---
I think that the name ``base'' is good for a user-facing interface,
and the name ``ghc-base'' more indicative of its
implementation-dependent character.


Wolfram

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Johan Tibell]

On Thu, Feb 14, 2013 at 6:48 AM, Joachim Breitner
wrote:

> Maybe the proper is to reverse the whole approach: Leave base as it is,
> and then build re-exporting smaller packages (e.g. a base-pure) on top
> of it. The advantage is:
>   * No need to rewrite the tightly intertwined base.
>   * Libraries still have the option to have tighter dependencies.
>   * Base can evolve with lots of breaking changes, as long as they
> do not affect the API by the smaller packages.
>   * Development of this collection can happen outside the GHC tree.
>   * Alternative implementations for (some of) these packages can be
> created, if the reason why they could not be moved out of base
> is one of implementation, not of API
>
> How does that sound?


I'm not in favor of this approach as it precludes pushing any data types
down the stack. In particular, we want text and bytestring to be below the
I/O layer, so we can defined Handles that work with those in base itself.
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Edward A Kmett]

As a super obvious aside, we could just switch code paths based on 
platform/environment. That lets you keep the fast code path and have a pure 
fallback for the javascripters. Just propagate an FFI_AVAILABLE flag into the 
compilation unit. We're going to have a number of these points which force a 
tension between generality And speed as we go, and it'd nice to have the 
ability to gracefully fall back.

On Feb 15, 2013, at 9:45 AM, Simon Marlow  wrote:

> On 15/02/13 08:36, Joachim Breitner wrote:
>> Hi,
>> 
>> Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery
>> kf8nh:
>>> On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote:
 On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner
  wrote:
 I don't think having FFI far down the stack is a problem. There are
 lots of pure data types we'd like in the "pure data" layer (e.g.
 bytestring) that uses FFI. As long as the I/O layer itself
 (System.IO, the I/O manager, etc) doesn't get pulled in there's no
 real problem in depending on the FFI.
>> 
>> I think it would be nice, also to other Haskell implementations that
>> might have not have FFI, to separate the really basic stuff from
>> pure-but-impurely-implemented stuff. At least as long as it does not
>> caues trouble.
>> 
>> GHC.Fingerprint does not need to be crippled when it is going to use a
>> pure hashing; I quickly added some simple fingerpriting found via
>> Wikipedia that was easier than MD5.
>> https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc
>> The choice is of course not final, maybe something with more bits is
>> desirable.
> 
> Remember that fingerprinting is not hashing.  For fingerprinting we need to 
> have a realistic expectation of no collisions.  I don't think FNV is suitable.
> 
> I'm sure it would be possible to replace the C md5 code with some Haskell.  
> Performance *is* important here though - Typeable is in the inner loop of 
> certain generic programming libraries, like SYB.
> 
> Cheers,
>Simon
> 
> 
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Joachim Breitner]

Hi,


Am Freitag, den 15.02.2013, 14:50 + schrieb Simon Marlow:
> On 15/02/13 12:22, Joachim Breitner wrote:
> > Hi,
> >
> > more progress: On top of base-pure, I created base-io involving GHC/IO
> > and everything required to build it (which pulled in ST, some of Foreign
> > and unfortunately some stuff related to Handles and Devices, because it
> > is mentioned in IOException). This is the list of modules:
> >
>
> You have a random collection of modules here :)
> 
> I think you want to have the IO *monad* (GHC.IO) live in a lower layer, 
> separate from the IO *library* (GHC.IO.Device and so on).  Every Haskell 
> implementation will need the IO monad, but they might want to replace 
> the IO library with something else.
> 
> Things like GHC.IORef, GHC.MVar can all live in a low-down layer because 
> they're just wrappers over the primops.

Right, that is my aim, and I started with GHC.IO. But unfortunately, the
IO monad calls failIO, which is an IOError which has a field of type
ioe_handle :: Maybe Handle (and one of type CInt) which pulls in all the
rest there, and so far I did not have a good idea how to untangle that.

What would break if fail would not raise an IOError, but a separate
exception type, e.g. IOFailError? Probably too much, as users expect to
catch the exception raised by fail with an exception handler that
matches IOError.


Greetings,
Joachim


-- 
Joachim "nomeata" Breitner
Debian Developer
  nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata



signature.asc
Description: This is a digitally signed message part
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Ian Lynagh]

On Fri, Feb 15, 2013 at 02:45:19PM +, Simon Marlow wrote:
> 
> Remember that fingerprinting is not hashing.  For fingerprinting we
> need to have a realistic expectation of no collisions.  I don't
> think FNV is suitable.
> 
> I'm sure it would be possible to replace the C md5 code with some
> Haskell.  Performance *is* important here though - Typeable is in
> the inner loop of certain generic programming libraries, like SYB.

We currently just compare
hash(str)
for equality, right? Could we instead compare
(hash str, str)
? That would be even more correct, even if a bad/cheap hash function is
used, and would only be slower for the case where the types match
(unless you're unlucky and get a hash collision).

In fact, we may be able to arrange it so that in the equal case the
strings are normally exactly the same string, so we can do a cheap
pointer equality test (like ByteString already does) to make the equal
case fast too (falling back to actually checking the strings are equal,
if they aren't the same string).


Thanks
Ian


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Edward Kmett]

Comparing hash, ptr, str gives you a pretty good acceptance/rejection test.
hash for the quick rejection, ptr for quick acceptance, str for accuracy.
Especially since the particular fingerprints for Typeable at least are
usually made up of 3 bytestrings that were just stuffed in and forgotten
about.

That said, this would seem to bring ByteString or at least Ptr in at a
pretty low level for the level of generality folks seem to be suddenly
seeking.

-Edward

On Wed, Feb 20, 2013 at 12:12 PM, Ian Lynagh  wrote:

> On Fri, Feb 15, 2013 at 02:45:19PM +, Simon Marlow wrote:
> >
> > Remember that fingerprinting is not hashing.  For fingerprinting we
> > need to have a realistic expectation of no collisions.  I don't
> > think FNV is suitable.
> >
> > I'm sure it would be possible to replace the C md5 code with some
> > Haskell.  Performance *is* important here though - Typeable is in
> > the inner loop of certain generic programming libraries, like SYB.
>
> We currently just compare
> hash(str)
> for equality, right? Could we instead compare
> (hash str, str)
> ? That would be even more correct, even if a bad/cheap hash function is
> used, and would only be slower for the case where the types match
> (unless you're unlucky and get a hash collision).
>
> In fact, we may be able to arrange it so that in the equal case the
> strings are normally exactly the same string, so we can do a cheap
> pointer equality test (like ByteString already does) to make the equal
> case fast too (falling back to actually checking the strings are equal,
> if they aren't the same string).
>
>
> Thanks
> Ian
>
>
> ___
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users@haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

Re: base package (Was: GHC 7.8 release?)

[Simon Marlow]

On 20/02/13 17:12, Ian Lynagh wrote:

On Fri, Feb 15, 2013 at 02:45:19PM +, Simon Marlow wrote:


Remember that fingerprinting is not hashing.  For fingerprinting we
need to have a realistic expectation of no collisions.  I don't
think FNV is suitable.

I'm sure it would be possible to replace the C md5 code with some
Haskell.  Performance *is* important here though - Typeable is in
the inner loop of certain generic programming libraries, like SYB.


We currently just compare
 hash(str)
for equality, right? Could we instead compare
 (hash str, str)
? That would be even more correct, even if a bad/cheap hash function is
used, and would only be slower for the case where the types match
(unless you're unlucky and get a hash collision).

>

In fact, we may be able to arrange it so that in the equal case the
strings are normally exactly the same string, so we can do a cheap
pointer equality test (like ByteString already does) to make the equal
case fast too (falling back to actually checking the strings are equal,
if they aren't the same string).


So it's not a single string, a TypeRep consists of a TyCon applied to 
some arguments, which themselves are TypeReps etc.


You could do pointer equality, and maybe that would work for the common 
cases.  But I don't see why we have to do that when fingerprinting works 
well and we already have it.  Why add a potential performance pitfall 
when we don't have to?


One other thing: it's useful to be able to use the fingerprint as an 
identifier for the contents, e.g. when sending Typeable values across 
the network.  If you can't do this with the fingerprint, then you need 
another unique Id, which is the situation we used to have before 
fingerprints.


Cheers,
Simon


___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users