Re: [racket-dev] Splitting the Racket repository

[Greg Hendershott]

I believe I've fixed everything shown on
 for which I'm
responsible:

aws: Commits merged to master and pushed.

http: Fixed a buggy test mishandling 301/302 redirects.

sha: Nothing needed to be fixed.

I'll keep an eye on DrDr; Sam said he'd add me to the email alerts.

On Sun, Nov 30, 2014 at 4:13 PM, Greg Hendershott
 wrote:
> Sam gave me a heads-up on IRC.
>
>> Also, DrDr now builds several packages that it didn't before: the
>> s3-sync, sha, http, and aws packages. Currently, these packages have
>> some test failures, but we hope to fix that soon.
>
> I'm responsible for aws, http, and sha. I'm working on this.
>
> I'm close to merging some commits[^1] to aws/master. Will look at http
> and sha after that.
>
> Also this is an opportunity to set up aws to be built by Travis
> CI[^2]. If tests sneak back in that would fail on DrDr, they should
> fail for the same reason on Travis as an early warning for me.  (The
> basic issue is that I have tests that actually interact with AWS, but
> these need credentials and cost $ so they need to be skipped in
> scenarios like DrDr and Travis CI. Some other tests can still be run,
> and will be.)
>
> [^1]: https://github.com/greghendershott/aws/commits/drdr
>
> [^2]: https://travis-ci.org/greghendershott/aws/builds/42570181
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Splitting the Racket repository

[Matthew Flatt]

At Sat, 29 Nov 2014 22:00:44 -0500, Eli Barzilay wrote:
> On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt
>  wrote:
> > On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay  wrote:
> >> On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
> >>  wrote:
> >>> To clone individual repositories, use the new `--clone` option for
> >>> `raco pkg`, such as:
> >>> [...]
> >>
> >> This is very obscure.  Is there a compact description of what to do
> >> when you want to make a change in a file that is not on the main
> >> repo?
> >
> > You just need to edit and change the relevant repository -- nothing
> > else is required. You don't have to use --clone or any other
> > mechanism.  [...]
> 
> This doesn't answer my question -- so I think that I wasn't clear
> enough.  Say that I have some change to a specific file.  I want to know
> how to do it in the following two ways:
> 
> [...]
> 
> 2. The thing that is more relevant for others (and that's the important
>one): I have a normal installation, I find a bug -- what do I do now?
>[...]

A strategy is explained in the "Developing Packages with Git" section
of the package-manager documentation. To elaborate for the running
example:


If you have "remote-shell" installed via "pkgs.racket-lang.org" and you
want to modify it, then

 raco pkg update --clone <...>/remote-shell

creates a clone at `<...>/remote-shell'. You can then use just
`remote-shell` to make a subdirectory of the current directory.

If you do that with the version of `raco pkg` as of this morning, it
will suggest that you use `--multi-clone ask`. (As of just now,
`--multi-clone ask` is the default.) When you're in `--multi-clone ask`
mode, `raco pkg` will suggest that "remote-shell", "remote-shell-lib",
and "remote-shell-doc", are all cloned together.


The argument after `--clone` is a directory path, but if no other
arguments are provided to `raco pkg update`, a package name is inferred
from the directory path (and a Git repo is inferred from that package
name).

In the example above, if you want to clone "remote-shell" to the
current directory instead of a subdirectory of the current directory,
supply the package name explicitly after "." as the directory path:

 raco pkg update --clone . remote-shell

That is, a clone of the package "remote-shell" is made in the current
directory as linked as the package installation.



If you installed a Git-based package via "pkgs.racket-lang.org" using a
version of `raco pkg` that's older than a week or so, then `raco pkg`
didn't record that the package came from a Git-repo URL, and so the
simple `--clone` form above doesn't work. More significantly, if you
install a snapshot of Racket, then the simple `--clone` form won't work
with a package that is provided in built form by the snapshot's
catalog. I'm working on a refinement to better address those
situations.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Splitting the Racket repository

[Greg Hendershott]

Sam gave me a heads-up on IRC.

> Also, DrDr now builds several packages that it didn't before: the
> s3-sync, sha, http, and aws packages. Currently, these packages have
> some test failures, but we hope to fix that soon.

I'm responsible for aws, http, and sha. I'm working on this.

I'm close to merging some commits[^1] to aws/master. Will look at http
and sha after that.

Also this is an opportunity to set up aws to be built by Travis
CI[^2]. If tests sneak back in that would fail on DrDr, they should
fail for the same reason on Travis as an early warning for me.  (The
basic issue is that I have tests that actually interact with AWS, but
these need credentials and cost $ so they need to be skipped in
scenarios like DrDr and Travis CI. Some other tests can still be run,
and will be.)

[^1]: https://github.com/greghendershott/aws/commits/drdr

[^2]: https://travis-ci.org/greghendershott/aws/builds/42570181
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Splitting the Racket repository

[Sam Tobin-Hochstadt]

On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
 wrote:
>
> # Changes for git users
>
> If you build Racket from source from Git, that build now contains
> fewer packages. There is not yet an single-step way to get all of the
> split pkgs as git repositories; we plan to write a script for this
> soon.

I've now written a first draft of this script. You can get it at
https://github.com/samth/clone-all

If you run:
$ racket clone-all.rkt 

it will create clones of all the packages split out of the repository in .

We'll integrate this into the main repository soon.

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Jay McCarthy]

On Sun, Nov 30, 2014 at 12:40 PM, Neil Van Dyke  wrote:
>
> Jay McCarthy wrote on 11/30/2014 12:30 PM:
>>
>> On Sunday, November 30, 2014, Neil Van Dyke > > wrote:
>>
>>
>> Jay McCarthy wrote on 11/30/2014 12:13 PM:
>>
>> The documentation cited is making clear that there is NO
>> connection between the name of a package and the provided
>> modules. There is no such thing as a package namespace.
>>
>>
>> I'd really like there to be.  For third-party packages.
>>
>>
>> I do not know what a third party package is.
>
>
> I'm thinking of *non*-core-Racket-developers.  Who have different needs than
> core Racket developers.  It's good that Racket is blurring the distinction,
> but the needs still aren't necessarily the same.  I say "third-party" as
> shorthand familiar to me, but perhaps there's a better term.

I don't understand what you think the different needs are. I agree
that there can be a different commitment to quality from the two
groups, such as the Racket core policy for backwards compatibility
that can't be enforced on anyone else and isn't practiced by core
developers on other stuff (for instance, I don't practice this on my
experimental game libraries.) But I can't think of any other
technically different needs.

>
>>
>>
>> Packages may find it convenient to build and provide reusable
>> functionality with many organizational names. This is
>> particularly true of "data", as many packages may have useful
>> data structures.
>>
>> Of course, as such support code becomes very useful and
>> developed, it makes sense to sprin it off into its own package.
>>
>>
>> Are you saying that `data` is some kind of classification of "what
>> this module is about", and in this case specifically, "this
>> module, which is part of some more specific package, happens to be
>> regarding general-purpose data structures, so we're putting it
>> over here in the `data` area of a shared namespace hierarchy"?
>>
>> Yes, although this is just for the benefit of search and reading docs; it
>> has no technical enforcement.
>>
>> If so, I don't understand why that would be considered a good idea.
>>
>>
>> It is a principle to create general purpose reusable code in the package
>> ecosystem rather than little archipelagos with lots of private code that
>> gets duplicated and has clever names.
>
>
>
> I keep wanting to encourage lots-and-lots of packages, and they can be
> packaged and released in a very lightweight yet well-conceived manner.

Yes. The package system is optimized for this case: put a directory
online or put code inside of a github repo --- boom, you have a
package!

> To go back to your example, if it's a useful general-purpose data structures
> library, then that's another package, which should be very easy to do.  If
> the package author isn't ready to split it out, then they're not ready to
> split it out, so no sense talking about it.  As far as I can tell, in no
> case is this `data` namespace relevant.

We are not enforcing through technology either outcome of splitting it
out or not. We are are enabling the ability to split it out if they
want, however. I don't understand what "talking about it" means... do
you mean writing documentation or literally talking about by sending
emails/etc? In either case, different authors may want to do different
things and the package system doesn't enforce one decision or another.

> Side note: I really don't think ontological classification of package topics
> belongs in package names.  Topical classifications evolve over time, and
> independent of the objects themselves. Classifications belong in
> directories, not in names.

I don't understand this.

--

It sounds like there is a problem that you are searching for a
solution for and aren't finding it in what the package system
delivers. Rather than asking, "Does the package system provide
solution A?", can you ask "How does the package system solve problem
X?" That would make it much more useful for me to give a useful reply.

Jay



-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Jay McCarthy]

Just to be clear. I don't think that "tic-tac-toe" and "data/matrix"
are the important thing in this example. The important thing is that
the package name and the module don't share a prefix. Another good
example would be something like "graphs" and "data/fib-heap" or
something where it may to be easier to imagine the connection between
the purpose of the package and a particular module provided.

Jay

On Sun, Nov 30, 2014 at 12:16 PM, Jay McCarthy  wrote:
> I do not think we should change the example. I do not want people to falsely
> believe that they can any expectation of the modules provided by a package
> from its name. In particular, your comment about this being "strange" is not
> supported by the package system's promises.
>
> Jay
>
>
> On Sunday, November 30, 2014, Matthew Flatt  wrote:
>>
>> We should change that example. It would indeed be strange for package
>> named "tic-tac-toe" would introduce a `data/matrix` module, and the
>> documentation really shouldn't suggest that it makes sense for a
>> package to introduce overlaps that are not reasonably expected from the
>> package name.
>>
>>
>> There are plenty of real examples where it's sensible for different
>> packages to introduce modules in overlapping collections, though.
>> Sometimes, it's because different packages implement different facets
>> of a conceptual whole, such as "math-lib", "math-doc", and "math-test"
>> all extending the "math" collection. Sometimes, it's more a mater of
>> history, such as "compatibility-lib" extending several different
>> collections. And then there are example that you might classify as
>> either of those or as a third kind, depending on how you look at it:
>> "draw-lib" adds `racket/draw`, and "gui-package-manager" adds
>> `pkg/gui`.
>>
>>
>> The "data" collection is among those that were intended to be extended
>> by multiple packages. Whether that's really a good idea remains to be
>> seen, but that's why "data" shows up as an example in the manual.
>> (Again, though, no one would expect a "tic-tac-toe" package to
>> introduce a new module in "data".)
>>
>>
>> At Sun, 30 Nov 2014 10:22:49 -0500, Neil Van Dyke wrote:
>> > Given the example from the documentation, of the `tic-tac-toe` package
>> > and "conflicts" (quoted at end of this email), instead, why isn't the
>> > norm to do:
>> >
>> >  (require tic-tac-toe)
>> >
>> > Or, when necessary:
>> >
>> >  (require tic-tac-toe/matrix)
>> >
>> > Why, when one installs a package named `tic-tac-toe`, would one be
>> > referencing modules of the package as `data/matrix`?
>> >
>> > I don't recall ever seeing a situation in which I wanted different
>> > third-party Racket packages to be stomping over each other's
>> > module-naming namespaces.
>> >
>> > For people who want to release lots of well-behaved reusable packages,
>> > using good practices, with minimal effort and head-scratching, things
>> > like this are a problem.
>> >
>> > > 2 Package Concepts
>> > >
>> > > A package is a set of modules in some number of collections. Modules
>> > > installed using the Racket package manager are required like any other
>> > > modules. For example, if the package tic-tac-toe contains the module
>> > > "matrix.rkt" in a "data" collection, then after tic-tac-toe is
>> > > installed,
>> > >
>> > > (require data/matrix)
>> > [...]
>> > > 2.5 Package Conflicts
>> > >
>> > > Two packages are in conflict if they contain the same module. For
>> > > example, if the package tic-tac-toe contains the module file
>> > > "data/matrix.rkt" and the package factory-optimize contains the module
>> > > file "data/matrix.rkt", then tic-tac-toe and factory-optimize are in
>> > > conflict.
>> > >
>> > > A package may also be in conflict with Racket itself, if it contains a
>> > > module file that is part of the base Racket implementation. For
>> > > example, any package that contains "racket/list.rkt" is in conflict
>> > > with Racket.
>> > >
>> > > For the purposes of conflicts, a module is a file that ends in ".rkt",
>> > > ".ss", or ".scrbl".
>> >
>> > Neil V.
>> >
>> > _
>> >   Racket Developers list:
>> >   http://lists.racket-lang.org/dev
>> _
>>   Racket Developers list:
>>   http://lists.racket-lang.org/dev
>
>
>
> --
> Jay McCarthy
> http://jeapostrophe.github.io
>
>"Wherefore, be not weary in well-doing,
>   for ye are laying the foundation of a great work.
> And out of small things proceedeth that which is great."
>   - D&C 64:33



-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Neil Van Dyke]

Sam Tobin-Hochstadt wrote on 11/30/2014 12:52 PM:

Are you saying that `data` is some kind of classification of "what this
module is about", and in this case specifically, "this module, which is part
of some more specific package, happens to be regarding general-purpose data
structures, so we're putting it over here in the `data` area of a shared
namespace hierarchy"?

Yes, the idea is that it makes sense for `data/hamt` to come from the
"hamt" package, and for `data/fector` to come from the "fector"
package (this is currently the case on pkgs.racket-lang.org).


  If so, I don't understand why that would be
considered a good idea.

Because the client of these modules shouldn't have to think about
which package they belong in to use them. This gives the package
developer freedom to combine or split packages without requiring code
changes from clients.


I think this might be centralized, core-Racket-developer, thinking. I 
think that some people might be trying to herd decentralized development 
to build what looks then like a tidy, centralized library.


This would also explain why the new package system originally advised to 
use generic names for packages, which I think is not a great match for 
organic, decentralized development.


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Sam Tobin-Hochstadt]

On Sun, Nov 30, 2014 at 12:23 PM, Neil Van Dyke  wrote:
>
>>
>> Packages may find it convenient to build and provide reusable
>> functionality with many organizational names. This is particularly true of
>> "data", as many packages may have useful data structures.
>>
>> Of course, as such support code becomes very useful and developed, it
>> makes sense to sprin it off into its own package.
>
> Are you saying that `data` is some kind of classification of "what this
> module is about", and in this case specifically, "this module, which is part
> of some more specific package, happens to be regarding general-purpose data
> structures, so we're putting it over here in the `data` area of a shared
> namespace hierarchy"?

Yes, the idea is that it makes sense for `data/hamt` to come from the
"hamt" package, and for `data/fector` to come from the "fector"
package (this is currently the case on pkgs.racket-lang.org).

>  If so, I don't understand why that would be
> considered a good idea.

Because the client of these modules shouldn't have to think about
which package they belong in to use them. This gives the package
developer freedom to combine or split packages without requiring code
changes from clients. In general, one of the assumptions of the new
package system is that metadata changes, such as dependencies, are
easier to manage than code changes. This assumption is fairly
pervasive in successful package systems, including extra-linguistic
ones such as dpkg as well as in-language ones such as Cabal and npm.

The particular mode of use, where module names splice, has been
successful in a number of other communities, such as Haskell, where
they have just such a Data hierarchy, and where, like in Racket,
package names and module names aren't connected. This is also the case
in Java packages, although the naming scheme is somewhat different.

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Neil Van Dyke]

Jay McCarthy wrote on 11/30/2014 12:30 PM:
On Sunday, November 30, 2014, Neil Van Dyke > wrote:



Jay McCarthy wrote on 11/30/2014 12:13 PM:

The documentation cited is making clear that there is NO
connection between the name of a package and the provided
modules. There is no such thing as a package namespace.


I'd really like there to be.  For third-party packages.


I do not know what a third party package is.


I'm thinking of *non*-core-Racket-developers.  Who have different needs 
than core Racket developers.  It's good that Racket is blurring the 
distinction, but the needs still aren't necessarily the same.  I say 
"third-party" as shorthand familiar to me, but perhaps there's a better 
term.






Packages may find it convenient to build and provide reusable
functionality with many organizational names. This is
particularly true of "data", as many packages may have useful
data structures.

Of course, as such support code becomes very useful and
developed, it makes sense to sprin it off into its own package.


Are you saying that `data` is some kind of classification of "what
this module is about", and in this case specifically, "this
module, which is part of some more specific package, happens to be
regarding general-purpose data structures, so we're putting it
over here in the `data` area of a shared namespace hierarchy"? 



Yes, although this is just for the benefit of search and reading docs; 
it has no technical enforcement.


If so, I don't understand why that would be considered a good idea.


It is a principle to create general purpose reusable code in the 
package ecosystem rather than little archipelagos with lots of private 
code that gets duplicated and has clever names.



I keep wanting to encourage lots-and-lots of packages, and they can be 
packaged and released in a very lightweight yet well-conceived manner.


To go back to your example, if it's a useful general-purpose data 
structures library, then that's another package, which should be very 
easy to do.  If the package author isn't ready to split it out, then 
they're not ready to split it out, so no sense talking about it.  As far 
as I can tell, in no case is this `data` namespace relevant.


Side note: I really don't think ontological classification of package 
topics belongs in package names.  Topical classifications evolve over 
time, and independent of the objects themselves. Classifications belong 
in directories, not in names.


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Jay McCarthy]

On Sunday, November 30, 2014, Neil Van Dyke  wrote:

>
> Jay McCarthy wrote on 11/30/2014 12:13 PM:
>
>> The documentation cited is making clear that there is NO connection
>> between the name of a package and the provided modules. There is no such
>> thing as a package namespace.
>>
>
> I'd really like there to be.  For third-party packages.


I do not know what a third party package is.


>
>
>> Packages may find it convenient to build and provide reusable
>> functionality with many organizational names. This is particularly true of
>> "data", as many packages may have useful data structures.
>>
>> Of course, as such support code becomes very useful and developed, it
>> makes sense to sprin it off into its own package.
>>
>
> Are you saying that `data` is some kind of classification of "what this
> module is about", and in this case specifically, "this module, which is
> part of some more specific package, happens to be regarding general-purpose
> data structures, so we're putting it over here in the `data` area of a
> shared namespace hierarchy"?


Yes, although this is just for the benefit of search and reading docs; it
has no technical enforcement.


> If so, I don't understand why that would be considered a good idea.


It is a principle to create general purpose reusable code in the package
ecosystem rather than little archipelagos with lots of private code that
gets duplicated and has clever names.

Jay


>
> Neil V.
>
>

-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system and versions

[Jay McCarthy]

I don't understand this question. The package system doesn't really have a
concept of version like you are referring to.

The package X is versionless. It is possible to know that your copy of X is
different than another's, hence checksums and updating (which should have
been called sync, in retrospect.) And you can know that your copy of X
isn't the copy that package Y expects, hence the (poorly named) version
metadata in the info file. But there is always only one package X in your
installation (and no promise that X anywhere else has anything to do with
your X.)

If you want to make a totally new package that has nothing to do with X, as
non-backwards compatibility means. Then you are free to name it X and make
it difficult for people to maintain old programs using the original X. If
you do so, the system will not stop you, but you are going against the
advice and experience of the Racket developers wrt quality customer
service.

Jay

On Sunday, November 30, 2014, Neil Van Dyke  wrote:

> Any chance of revisiting the new package system's stances on versions --
> specifically, on the two issues:
> 1. Can subsequent versions of a named package (which has an identity)  be
> non-backward-compatible?
> 2. Can a Racket setup (and even an individual program) have multiple
> versions of a package at the same time?
>
> Regarding #1, the requirement to never make a non-backward-compatible
> version of a package without giving up package identity is a burden, and a
> deterrent to third-party package releases.
>
> For a real-world example, see "http://planet.racket-lang.org/";, where it
> looks like most of the familiar names (who were going to a good amount of
> trouble to release already), managed to release packages marked as
> non-backward-compatible (i.e., PLaneT major version number other than 1).
> Wouldn't requiring them to never break backward compatibility be a
> deterrent to releasing at all? Or, if they still released, would `dherman`,
> following the instructions of Racket to create a new package identity for
> any backward-incompatible version, have given us packages `javascript`,
> `javascript2`, ... `javascript9`, rather than `javascript` versions `1.x`
> through `9.x`.
>
> No-backward-incompatibility might make sense for core Racket (although
> even core Racket has broken this multiple times), but it's a big deterrent
> to researchers, and to industry developers who are willing to open source
> components of their larger systems in lightweight ways.
>
> Regarding #2, I suggest that should be revisited if #1 is revisited.
>
> I think PLaneT got both of these a lot closer to right, at least for
> third-party packages.
>
> Neil V.
>
> _
>  Racket Developers list:
>  http://lists.racket-lang.org/dev
>


-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Neil Van Dyke]

Jay McCarthy wrote on 11/30/2014 12:13 PM:
The documentation cited is making clear that there is NO connection 
between the name of a package and the provided modules. There is no 
such thing as a package namespace.


I'd really like there to be.  For third-party packages.



Packages may find it convenient to build and provide reusable 
functionality with many organizational names. This is particularly 
true of "data", as many packages may have useful data structures.


Of course, as such support code becomes very useful and developed, it 
makes sense to sprin it off into its own package.


Are you saying that `data` is some kind of classification of "what this 
module is about", and in this case specifically, "this module, which is 
part of some more specific package, happens to be regarding 
general-purpose data structures, so we're putting it over here in the 
`data` area of a shared namespace hierarchy"?  If so, I don't understand 
why that would be considered a good idea.


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev

Re: [racket-dev] new package system and versions

[Matthew Flatt]

Although I object to some of your characterizations of the difference
between PLaneT and the new package system, it's fair to say that PLaneT
provides better support to package authors for creating new APIs that
are intended to replace (but coexist with) old APIs.

I think the answer to that problem in the new system is better tools
for package authors, along with better catalog-display mechanisms for
package users. That is, I think "javascript" through "javascript9" are
fine as package names (and even collection names), but I think those
packages should be grouped together as a listing in the catalog.
Meanwhile, we need tools to help the author of "javascript3" introduce
"javascript4", and so on.

Creating an external tool might not be necessary if the notion of
major-version-compatibility were built into the package system in a
different way. Unfortunately, it's not easy to add more version support
into the current design and implementation. It's the same as designing
a more traditional programming language: choices interact with all
sorts of trade-offs, both conceptually and in the implementation.

So, if the question is "will the core package system's treatment of
versions change?", then I think the answer is "no". If the question is
"will the package versioning experience improve for package authors and
users?", then I hope the answer is "yes" --- but I intend to leave the
actual answer to others (yes, you!), for now. Until we find otherwise,
I can't help feeling that the current design and implementation has all
the primitives that we need[*] for building better things on top.

 [*] Of course, it has way too many primitives. Again, like any design,
 it's a process to figure out what the true core could be.

At Sun, 30 Nov 2014 10:43:19 -0500, Neil Van Dyke wrote:
> Any chance of revisiting the new package system's stances on versions -- 
> specifically, on the two issues:
> 1. Can subsequent versions of a named package (which has an identity)  
> be non-backward-compatible?
> 2. Can a Racket setup (and even an individual program) have multiple 
> versions of a package at the same time?
> 
> Regarding #1, the requirement to never make a non-backward-compatible 
> version of a package without giving up package identity is a burden, and 
> a deterrent to third-party package releases.
> 
> For a real-world example, see "http://planet.racket-lang.org/";, where it 
> looks like most of the familiar names (who were going to a good amount 
> of trouble to release already), managed to release packages marked as 
> non-backward-compatible (i.e., PLaneT major version number other than 
> 1).  Wouldn't requiring them to never break backward compatibility be a 
> deterrent to releasing at all? Or, if they still released, would 
> `dherman`, following the instructions of Racket to create a new package 
> identity for any backward-incompatible version, have given us packages 
> `javascript`, `javascript2`, ... `javascript9`, rather than `javascript` 
> versions `1.x` through `9.x`.
> 
> No-backward-incompatibility might make sense for core Racket (although 
> even core Racket has broken this multiple times), but it's a big 
> deterrent to researchers, and to industry developers who are willing to 
> open source components of their larger systems in lightweight ways.
> 
> Regarding #2, I suggest that should be revisited if #1 is revisited.
> 
> I think PLaneT got both of these a lot closer to right, at least for 
> third-party packages.
> 
> Neil V.
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Jay McCarthy]

I do not think we should change the example. I do not want people to
falsely believe that they can any expectation of the modules provided by a
package from its name. In particular, your comment about this being
"strange" is not supported by the package system's promises.

Jay

On Sunday, November 30, 2014, Matthew Flatt  wrote:

> We should change that example. It would indeed be strange for package
> named "tic-tac-toe" would introduce a `data/matrix` module, and the
> documentation really shouldn't suggest that it makes sense for a
> package to introduce overlaps that are not reasonably expected from the
> package name.
>
>
> There are plenty of real examples where it's sensible for different
> packages to introduce modules in overlapping collections, though.
> Sometimes, it's because different packages implement different facets
> of a conceptual whole, such as "math-lib", "math-doc", and "math-test"
> all extending the "math" collection. Sometimes, it's more a mater of
> history, such as "compatibility-lib" extending several different
> collections. And then there are example that you might classify as
> either of those or as a third kind, depending on how you look at it:
> "draw-lib" adds `racket/draw`, and "gui-package-manager" adds
> `pkg/gui`.
>
>
> The "data" collection is among those that were intended to be extended
> by multiple packages. Whether that's really a good idea remains to be
> seen, but that's why "data" shows up as an example in the manual.
> (Again, though, no one would expect a "tic-tac-toe" package to
> introduce a new module in "data".)
>
>
> At Sun, 30 Nov 2014 10:22:49 -0500, Neil Van Dyke wrote:
> > Given the example from the documentation, of the `tic-tac-toe` package
> > and "conflicts" (quoted at end of this email), instead, why isn't the
> > norm to do:
> >
> >  (require tic-tac-toe)
> >
> > Or, when necessary:
> >
> >  (require tic-tac-toe/matrix)
> >
> > Why, when one installs a package named `tic-tac-toe`, would one be
> > referencing modules of the package as `data/matrix`?
> >
> > I don't recall ever seeing a situation in which I wanted different
> > third-party Racket packages to be stomping over each other's
> > module-naming namespaces.
> >
> > For people who want to release lots of well-behaved reusable packages,
> > using good practices, with minimal effort and head-scratching, things
> > like this are a problem.
> >
> > > 2 Package Concepts
> > >
> > > A package is a set of modules in some number of collections. Modules
> > > installed using the Racket package manager are required like any other
> > > modules. For example, if the package tic-tac-toe contains the module
> > > "matrix.rkt" in a "data" collection, then after tic-tac-toe is
> installed,
> > >
> > > (require data/matrix)
> > [...]
> > > 2.5 Package Conflicts
> > >
> > > Two packages are in conflict if they contain the same module. For
> > > example, if the package tic-tac-toe contains the module file
> > > "data/matrix.rkt" and the package factory-optimize contains the module
> > > file "data/matrix.rkt", then tic-tac-toe and factory-optimize are in
> > > conflict.
> > >
> > > A package may also be in conflict with Racket itself, if it contains a
> > > module file that is part of the base Racket implementation. For
> > > example, any package that contains "racket/list.rkt" is in conflict
> > > with Racket.
> > >
> > > For the purposes of conflicts, a module is a file that ends in ".rkt",
> > > ".ss", or ".scrbl".
> >
> > Neil V.
> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
>


-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Splitting the Racket repository

[Sam Tobin-Hochstadt]

On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay  wrote:
> On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
>  wrote:
>>
>> All the history for the code has been preserved, and for code that
>> dates back before 2005, the history is extended back to the original
>> CVS repository. See https://github.com/racket/games/ for an example of
>> this.
>
> There's a failure in the import -- if you look at the commits of this
> repo (https://github.com/racket/games/commits/master), then starting
> from Robby's commit from Jul 17 ("add a contract to make-card...") and
> going back, the "original commit" references are all bogus.

I've now fixed this (with an assist from Matthew). If you look at that
commit again, it should reference an "original commit" that you can
find in `plt/racket`.

Furthermore, I've renamed the repository with the history. It's now here:
  github.com/samth/plt-history

It has the original commit for everything that ends up in a split. It
also has some instructions on how to see the history including all the
original commits.

Thanks for noticing the error.

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Jay McCarthy]

The documentation cited is making clear that there is NO connection between
the name of a package and the provided modules. There is no such thing as a
package namespace.

Packages may find it convenient to build and provide reusable functionality
with many organizational names. This is particularly true of "data", as
many packages may have useful data structures.

Of course, as such support code becomes very useful and developed, it makes
sense to sprin it off into its own package.

Jay

On Sunday, November 30, 2014, Neil Van Dyke  wrote:

> Given the example from the documentation, of the `tic-tac-toe` package and
> "conflicts" (quoted at end of this email), instead, why isn't the norm to
> do:
>
> (require tic-tac-toe)
>
> Or, when necessary:
>
> (require tic-tac-toe/matrix)
>
> Why, when one installs a package named `tic-tac-toe`, would one be
> referencing modules of the package as `data/matrix`?
>
> I don't recall ever seeing a situation in which I wanted different
> third-party Racket packages to be stomping over each other's module-naming
> namespaces.
>
> For people who want to release lots of well-behaved reusable packages,
> using good practices, with minimal effort and head-scratching, things like
> this are a problem.
>
>  2 Package Concepts
>>
>> A package is a set of modules in some number of collections. Modules
>> installed using the Racket package manager are required like any other
>> modules. For example, if the package tic-tac-toe contains the module
>> "matrix.rkt" in a "data" collection, then after tic-tac-toe is installed,
>>
>> (require data/matrix)
>>
> [...]
>
>> 2.5 Package Conflicts
>>
>> Two packages are in conflict if they contain the same module. For
>> example, if the package tic-tac-toe contains the module file
>> "data/matrix.rkt" and the package factory-optimize contains the module file
>> "data/matrix.rkt", then tic-tac-toe and factory-optimize are in conflict.
>>
>> A package may also be in conflict with Racket itself, if it contains a
>> module file that is part of the base Racket implementation. For example,
>> any package that contains "racket/list.rkt" is in conflict with Racket.
>>
>> For the purposes of conflicts, a module is a file that ends in ".rkt",
>> ".ss", or ".scrbl".
>>
>
> Neil V.
>
> _
>  Racket Developers list:
>  http://lists.racket-lang.org/dev
>


-- 
Jay McCarthy
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D&C 64:33
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Neil Van Dyke]

Sam Tobin-Hochstadt wrote on 11/30/2014 10:55 AM:

Another example where this happens (and in a way that couldn't be
fixed by combining packages) is with typed versions of libraries. If I
release a package with the collection `foo`, and then someone else
produces a typed version of it, that will probably be named either
`foo/typed` (and thus be in the `foo` namespace) or `typed/foo` (and
thus be in the `typed` namespace).


Personally, I'd much rather have packages respect namespace boundaries, 
and have a simpler model, and let people call that TR library 
`foo-typed` or `typed-foo`.  (Or, TR comes up with a better scheme for 
providing TR and non-TR interfaces to a module, and reduces DRY problems 
in specifying APIs at the same time.)


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Sam Tobin-Hochstadt]

On Sun, Nov 30, 2014 at 10:44 AM, Matthew Flatt  wrote:
>
> There are plenty of real examples where it's sensible for different
> packages to introduce modules in overlapping collections, though.
> Sometimes, it's because different packages implement different facets
> of a conceptual whole, such as "math-lib", "math-doc", and "math-test"
> all extending the "math" collection.

Another example where this happens (and in a way that couldn't be
fixed by combining packages) is with typed versions of libraries. If I
release a package with the collection `foo`, and then someone else
produces a typed version of it, that will probably be named either
`foo/typed` (and thus be in the `foo` namespace) or `typed/foo` (and
thus be in the `typed` namespace).

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Matthew Flatt]

We should change that example. It would indeed be strange for package
named "tic-tac-toe" would introduce a `data/matrix` module, and the
documentation really shouldn't suggest that it makes sense for a
package to introduce overlaps that are not reasonably expected from the
package name.


There are plenty of real examples where it's sensible for different
packages to introduce modules in overlapping collections, though.
Sometimes, it's because different packages implement different facets
of a conceptual whole, such as "math-lib", "math-doc", and "math-test"
all extending the "math" collection. Sometimes, it's more a mater of
history, such as "compatibility-lib" extending several different
collections. And then there are example that you might classify as
either of those or as a third kind, depending on how you look at it:
"draw-lib" adds `racket/draw`, and "gui-package-manager" adds
`pkg/gui`.


The "data" collection is among those that were intended to be extended
by multiple packages. Whether that's really a good idea remains to be
seen, but that's why "data" shows up as an example in the manual.
(Again, though, no one would expect a "tic-tac-toe" package to
introduce a new module in "data".)


At Sun, 30 Nov 2014 10:22:49 -0500, Neil Van Dyke wrote:
> Given the example from the documentation, of the `tic-tac-toe` package 
> and "conflicts" (quoted at end of this email), instead, why isn't the 
> norm to do:
> 
>  (require tic-tac-toe)
> 
> Or, when necessary:
> 
>  (require tic-tac-toe/matrix)
> 
> Why, when one installs a package named `tic-tac-toe`, would one be 
> referencing modules of the package as `data/matrix`?
> 
> I don't recall ever seeing a situation in which I wanted different 
> third-party Racket packages to be stomping over each other's 
> module-naming namespaces.
> 
> For people who want to release lots of well-behaved reusable packages, 
> using good practices, with minimal effort and head-scratching, things 
> like this are a problem.
> 
> > 2 Package Concepts
> >
> > A package is a set of modules in some number of collections. Modules 
> > installed using the Racket package manager are required like any other 
> > modules. For example, if the package tic-tac-toe contains the module 
> > "matrix.rkt" in a "data" collection, then after tic-tac-toe is installed,
> >
> > (require data/matrix)
> [...]
> > 2.5 Package Conflicts
> >
> > Two packages are in conflict if they contain the same module. For 
> > example, if the package tic-tac-toe contains the module file 
> > "data/matrix.rkt" and the package factory-optimize contains the module 
> > file "data/matrix.rkt", then tic-tac-toe and factory-optimize are in 
> > conflict.
> >
> > A package may also be in conflict with Racket itself, if it contains a 
> > module file that is part of the base Racket implementation. For 
> > example, any package that contains "racket/list.rkt" is in conflict 
> > with Racket.
> >
> > For the purposes of conflicts, a module is a file that ends in ".rkt", 
> > ".ss", or ".scrbl".
> 
> Neil V.
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

[racket-dev] new package system and versions

[Neil Van Dyke]

Any chance of revisiting the new package system's stances on versions -- 
specifically, on the two issues:
1. Can subsequent versions of a named package (which has an identity)  
be non-backward-compatible?
2. Can a Racket setup (and even an individual program) have multiple 
versions of a package at the same time?


Regarding #1, the requirement to never make a non-backward-compatible 
version of a package without giving up package identity is a burden, and 
a deterrent to third-party package releases.


For a real-world example, see "http://planet.racket-lang.org/";, where it 
looks like most of the familiar names (who were going to a good amount 
of trouble to release already), managed to release packages marked as 
non-backward-compatible (i.e., PLaneT major version number other than 
1).  Wouldn't requiring them to never break backward compatibility be a 
deterrent to releasing at all? Or, if they still released, would 
`dherman`, following the instructions of Racket to create a new package 
identity for any backward-incompatible version, have given us packages 
`javascript`, `javascript2`, ... `javascript9`, rather than `javascript` 
versions `1.x` through `9.x`.


No-backward-incompatibility might make sense for core Racket (although 
even core Racket has broken this multiple times), but it's a big 
deterrent to researchers, and to industry developers who are willing to 
open source components of their larger systems in lightweight ways.


Regarding #2, I suggest that should be revisited if #1 is revisited.

I think PLaneT got both of these a lot closer to right, at least for 
third-party packages.


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev

[racket-dev] new package system collections and conflicts

[Neil Van Dyke]

Given the example from the documentation, of the `tic-tac-toe` package 
and "conflicts" (quoted at end of this email), instead, why isn't the 
norm to do:


(require tic-tac-toe)

Or, when necessary:

(require tic-tac-toe/matrix)

Why, when one installs a package named `tic-tac-toe`, would one be 
referencing modules of the package as `data/matrix`?


I don't recall ever seeing a situation in which I wanted different 
third-party Racket packages to be stomping over each other's 
module-naming namespaces.


For people who want to release lots of well-behaved reusable packages, 
using good practices, with minimal effort and head-scratching, things 
like this are a problem.



2 Package Concepts

A package is a set of modules in some number of collections. Modules 
installed using the Racket package manager are required like any other 
modules. For example, if the package tic-tac-toe contains the module 
"matrix.rkt" in a "data" collection, then after tic-tac-toe is installed,


(require data/matrix)

[...]

2.5 Package Conflicts

Two packages are in conflict if they contain the same module. For 
example, if the package tic-tac-toe contains the module file 
"data/matrix.rkt" and the package factory-optimize contains the module 
file "data/matrix.rkt", then tic-tac-toe and factory-optimize are in 
conflict.


A package may also be in conflict with Racket itself, if it contains a 
module file that is part of the base Racket implementation. For 
example, any package that contains "racket/list.rkt" is in conflict 
with Racket.


For the purposes of conflicts, a module is a file that ends in ".rkt", 
".ss", or ".scrbl".


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev