Re: [Distutils] it's happened - wheels without sdists (flit)

[Barry Warsaw]

On Mar 31, 2015, at 09:14 AM, Nick Coghlan wrote:

>We need to build from source not just to ensure our binaries match the
>published source code, but also because our build systems are designed to
>let us *patch* the packages before we build them. This is what lets us
>backport security updates, bug fixes, and sometimes even entire features
>without needing to rebase a package on a new upstream release of a project.

For Debian, all this, plus it's required by the Debian Social Contract and
Debian Free Software Guidelines.

>(AFAIK Debian still favours invoking setup.py directly, hopefully Barry
>will correct me if I'm wrong).

Correct.  For any setup.py/distutils/setuptools-based project, we have one
preferred, recommended, and non-deprecated "build system", and though it's
fairly flexible, it mostly relies on a working setup.py.  As Debian Jessie is
frozen right now, I don't expect that to change for now.

We have sort of a love-hate relationship with pip, but it would be interesting
to sit down at Pycon and discuss how we might like to see downstream distro
support improve or change as it relates to common upstream packaging
standards.

Cheers,
-Barry


pgp4mKza0LaGV.pgp
Description: OpenPGP digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 8:33 PM, Alex Clark  wrote:
> 
> On 3/30/15 11:56 AM, Donald Stufft wrote:
>> 
>>> On Mar 30, 2015, at 11:52 AM, Paul Moore  wrote:
>>> 
>>> On 30 March 2015 at 16:45, Donald Stufft  wrote:
 Well, parts of it are turing complete, since it pulls the version number
 out of the module itself and that’s just Python too.
>>> 
>>> Sorry, I wasn't specifically looking at flit there. But I'm in the
>>> camp that says just put the version in your ini file and in your
>>> module, and don't worry that you have it in 2 places. If managing
>>> version numbers is the biggest showstopper in moving to declarative
>>> metadata, then we've won :-)
>>> 
>>> Paul
>> 
>> Honestly, I don’t think that setup.py as a development interface is that
>> bad. It gets really bad when we start sticking it inside of a sdist and
>> using that as part of the installation metadata.
>> 
>> It’s not unusual for me to want (or need) to do something a little bit
>> different in a project, or something that the original authors didn’t
>> quite intend to do. This is perfectly valid and fine inside of a file
>> that only ever gets executed on a developer machine. However it *needs*
>> to be “compiled” down to a static file when creating a sdist.
> 
> 
> Right, that is my understanding: setup.py is fine except when it is executed 
> on installation.
> 
> But I think there is a slight cognitive advantage to setup.ini vs. setup.py. 
> You can never execute an ini file, even in development. So the same file can 
> (somehow) be used in development and production without "compiling down" 
> first. In other words: maybe switching to ini is the right thing to do long 
> term.
> 
> However, the practicality of doing so may be so small (due to 
> disutils/setuptools baggage and/or inability to overcome setup.py momentum) 
> that "compiling down" (setup.py) becomes a more attractive first step, at 
> least.
> 

I think trying to use the same file is an attractive nuisance. Separating them 
lets you do a lot more for developer convenience without hurting the ability of 
the installation side of things. For instance, it’s somewhat common to want to 
import the module in order to pull the version out of it. That’s not something 
we’d want to do outside of the developer side of things, but it’s something 
that’s completely reasonable to do in the developer toolchain (either 
implicitly as in flit, or explicitly as in setup.py).

IOW, splitting these into the “developer side” tooling and the “format that 
automated installers consume” lets you define both of them to be the best at 
what they are doing instead of trying to put something together that satisfies 
two competing use cases.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Alex Clark]

On 3/30/15 11:56 AM, Donald Stufft wrote:



On Mar 30, 2015, at 11:52 AM, Paul Moore  wrote:

On 30 March 2015 at 16:45, Donald Stufft  wrote:

Well, parts of it are turing complete, since it pulls the version number
out of the module itself and that’s just Python too.


Sorry, I wasn't specifically looking at flit there. But I'm in the
camp that says just put the version in your ini file and in your
module, and don't worry that you have it in 2 places. If managing
version numbers is the biggest showstopper in moving to declarative
metadata, then we've won :-)

Paul


Honestly, I don’t think that setup.py as a development interface is that
bad. It gets really bad when we start sticking it inside of a sdist and
using that as part of the installation metadata.

It’s not unusual for me to want (or need) to do something a little bit
different in a project, or something that the original authors didn’t
quite intend to do. This is perfectly valid and fine inside of a file
that only ever gets executed on a developer machine. However it *needs*
to be “compiled” down to a static file when creating a sdist.



Right, that is my understanding: setup.py is fine except when it is 
executed on installation.


But I think there is a slight cognitive advantage to setup.ini vs. 
setup.py. You can never execute an ini file, even in development. So the 
same file can (somehow) be used in development and production without 
"compiling down" first. In other words: maybe switching to ini is the 
right thing to do long term.


However, the practicality of doing so may be so small (due to 
disutils/setuptools baggage and/or inability to overcome setup.py 
momentum) that "compiling down" (setup.py) becomes a more attractive 
first step, at least.





---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig




--
Alex Clark · http://about.me/alex.clark

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] d2to1 setup.cfg schema

[Erik Bray]

On Mon, Mar 30, 2015 at 7:07 PM, Robert Collins
 wrote:
> On 31 March 2015 at 12:03, Erik Bray  wrote:
>
>> I haven't followed this whole discussion (I started to in the
>> beginning, but haven't kept up), but I'm not really sure what's being
>> said here.  d2to1 *does* support declaring setup-requires dependencies
>> in setup.cfg, and those dependencies should be loaded before any hook
>> scripts are used.  Everything in d2to1 is done via various hook
>> points, and the hook functions can be either shipped with the package,
>> or come from external requirements installed via setup-requires.  It
>> works pretty well in most cases.
>
> Oh, it does!? I was looking through the source and couldn't figure it
> out. What key is looked for for setup-requires? Also does it define a
> schema for extra-requires?

Yeah, sorry about that.  That's one of those things that was never
actually supported in distutils2 by the time it went poof, and that I
added later.

You can use:

[metadata]
setup-requires-dist = foo

So say, for example you have some package called "versionutils" that's
used to generate the package's version number (by reading it from
another file, tacking on VCS info, etc.)  You can use:

[metadata]
setup-requires-dist = versionutils

[global]
setup-hooks = versionutils.version_hook

or something to that effect.  It will ensure versionutils is
importable (this uses easy_install just like the normal setup_requires
feature in setuptools; I would like to change this one day to instead
use something like Daniel's setup-requires [1] trick).
It will then, fairly early in the setup process, hand the package
metadata over to versionutils.version_hook, and let it insert a
version string.

Erik

[1] https://bitbucket.org/dholth/setup-requires
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Nick Coghlan]

On 31 Mar 2015 01:17, "Xavier Fernandez"  wrote:
>
> Fair enough, I didn't think of compiled wheels :)
> And having a clean way to run tests for the provided wheel is indeed an
other good point.

To elaborate on Donald's answer, one of our general requirements downstream
in distro land is the ability to rebuild the entire distro from source
without human intervention (for example:
https://beaker-project.org/docs-develop/user-guide/beaker-provided-tasks.html#distribution-rebuild
for Fedora & derivatives)

We need to build from source not just to ensure our binaries match the
published source code, but also because our build systems are designed to
let us *patch* the packages before we build them. This is what lets us
backport security updates, bug fixes, and sometimes even entire features
without needing to rebase a package on a new upstream release of a project.

We also like to be able to automatically *generate* package build
instructions from upstream metadata. Those tools aren't perfect today (too
much info is missing from current generation upstream metadata), but
they're a lot better than nothing. Historically those tools have relied on
"setup.py install" working on an unpacked sdist.

These days in Fedora we aim to rely on "pip install" working instead of
invoking setup.py directly, but there are over 15000 Python packages in
Fedora (and a similar number in Debian), and Fedora's policy only switched
to preferring indirection through pip for Python package builds last year
(AFAIK Debian still favours invoking setup.py directly, hopefully Barry
will correct me if I'm wrong).

But as long as those source package builds keep working downstream, we
don't really care what's happening behind the scenes in the upstream build
systems. It's mildly irritating when packaging a new release of an already
included project turns into a yak shaving expedition to package a whole new
build system as a dependency, but that's just one of the consequences of
running an open source integration project rather than writing everything
from scratch yourself.

It's also one of the reasons why d2to1 is such a neat hack - that presents
as plain distutils at sdist build time, while allowing the use of
distutils2 features at package definition time upstream.

Cheers,
Nick.

>
> On Mon, Mar 30, 2015 at 5:04 PM, Donald Stufft  wrote:
>>
>>
>>> On Mar 30, 2015, at 10:58 AM, Xavier Fernandez 
wrote:
>>>
 Wheels without sdists are likely a generally bad idea, downstream
 redistributors are not going to like them.
>>>
>>>
>>> Why do you think that ? Wheels seem way simpler/saner than all the
possible things setup.py can do.
>>
>>
>> Wheels are simpler than a setup.py, because wheels are a binary format
and Wheels don’t need to
>> handle things like build software because it’s already been built.
However downstream redistributors
>> will not accept a Wheel as the source for a package because it is a
binary format. It doesn’t matter
>> if you can unzip it and there is pure python there, it is still a binary
format. So if you release only Wheels
>> you’re essentially saying that downstream redistributors will never
package your software (or any
>> software that depends on it).
>>
>> A few issues that Wheel only has:
>>
>> * If your project has a C extension, downstream redistributes need
access to the source code not the
>>   compiled code (as does anyone wanting to use the project on platform
you didn’t release for).
>>
>> * If your project has tests that don’t get installed, they should get
shipped as part of the sdist so that
>>   downstream can run them as part of their packaging activities to
ensure they didn’t break anything
>>   in the test suite. However if you’re installing from Wheel you can’t
do that.
>>
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] d2to1 setup.cfg schema

[Robert Collins]

On 31 March 2015 at 12:03, Erik Bray  wrote:

> I haven't followed this whole discussion (I started to in the
> beginning, but haven't kept up), but I'm not really sure what's being
> said here.  d2to1 *does* support declaring setup-requires dependencies
> in setup.cfg, and those dependencies should be loaded before any hook
> scripts are used.  Everything in d2to1 is done via various hook
> points, and the hook functions can be either shipped with the package,
> or come from external requirements installed via setup-requires.  It
> works pretty well in most cases.

Oh, it does!? I was looking through the source and couldn't figure it
out. What key is looked for for setup-requires? Also does it define a
schema for extra-requires?

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] d2to1 setup.cfg schema

[Erik Bray]

On Tue, Mar 24, 2015 at 6:51 PM, Nick Coghlan  wrote:
>
> On 25 Mar 2015 07:35, "Robert Collins"  wrote:
>>
>> This is a break-out thread from the centi-thread that spawned about
>> setup-requires.
>>
>> d2to1 defined some metadata keys in setup.cfg,in particular 'name' and
>> 'requires-dist'. Confusing 'requires-dist' contains the
>> 'install_requires' one might use with setuptools' setup() function.
>
> That particular name comes from PEP 345:
> https://www.python.org/dev/peps/pep-0345/
>
> Extending d2to1 to accept "install-requires" as meaning the same thing as
> the existing "requires-dist" (and complaining if a setup.cfg file contains
> both) would make sense to me, as it provides a more obvious migration path
> from setuptools, and pairs up nicely with a new "setup-requires" section for
> setup.py dependencies.

I would be fine with that, and other similar changes.  Better
documentation of the format is needed too--it used to just rely on the
distutils2 documentation, but since distutils2 is dead d2to1 deserves
documentation in its own right.

> (It also occurs to me that we should probably ask the d2to1 folks if they'd
> be interested in bringing the project under the PyPA banner as happened with
> setuptools, distlib, etc. It's emerged as a key piece of the transition from
> Turing complete build process customisation to static build metadata
> configuration)

As "the d2to1 folks", more or less, (not counting the pbr folks who've
done their own thing and might have opinions), I would be fine with
this.  It has been on my agenda for over a year to release an update
to d2to1 under a new name--something less tied to the failed
distutils2 project (along with its own documentation, see above).

The new name I've been working under cheekily called "setup.cfg", that
is, the actual Python package is named "setup.cfg".  You import
setup.cfg in your setup.py and it basically does the rest.  But if
that ends up being deemed too confusing/silly that would be
understandable--I'm open to other ideas.

>> Since the declarative setup-requires concept also involves putting
>> dependencies in setup.cfg (but setup_requires rather than
>> install_requires), I followed the naming convention d2to1 had started.
>> But - all the reviewers (and I agree) think this is confusing and
>> non-obvious.
>>
>> Since d2to1 is strictly a build-time thing - it reflects the keys into
>> the metadata and thus your egg-info/requires.txt is unaltered in
>> output, I think its reasonable to argue that we don't need to be
>> compatible with it.
>>
>> OTOH folk using d2to1 would not gain the benefits that declarative
>> setup-requires may offer in setuptools // pip.

I haven't followed this whole discussion (I started to in the
beginning, but haven't kept up), but I'm not really sure what's being
said here.  d2to1 *does* support declaring setup-requires dependencies
in setup.cfg, and those dependencies should be loaded before any hook
scripts are used.  Everything in d2to1 is done via various hook
points, and the hook functions can be either shipped with the package,
or come from external requirements installed via setup-requires.  It
works pretty well in most cases.

Erik
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Isn't it neat that pip-installable packages can be built without distutils?

[Nick Coghlan]

On 31 Mar 2015 04:11, "Donald Stufft"  wrote:
>
>
> > On Mar 30, 2015, at 2:03 PM, Daniel Holth  wrote:
> >
> > What else can we do to make the experience better for those who would
> > try to replace distutils?
>
> Continue work on the interoperability standards that exist for that very
> purpose.

One of the other key missing pieces is a tool independent regression test
suite that defines the expected setup.py CLI. The existing setuptools,
distutils & distutils2 test suites do a lot of their testing via Python
level APIs rather than being written as external functional tests for the
setup.py CLI they provide.

So one place to start would be to work on creating such an independent test
suite (from a technical perspective, I'd suggest basing it on pytest &
tox), focusing initially on the commands that pip invokes, and using the
pip/virtualenv/setuptools/distutils/distutils2 test suites as guides for
options and environment variable settings to check.

After that, figuring out how to scan the Debian & Fedora repos to extract
the distro level build commands used for various projects would provide an
additional source of good test suite fodder.

The advantage of such a test suite project is that it would start to
meaningfully quantify what "replacing distutils" actually means from the
practical perspective of "calling setup.py at the command line".

Regards,
Nick.


>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Barry Warsaw]

On Mar 30, 2015, at 11:56 AM, Donald Stufft wrote:

>Honestly, I don’t think that setup.py as a development interface is that
>bad.

Especially when you cargo cult most of a new project's basic infrastructure[*]
from one that's already working.  Sweat out the first one, then reuse. ;)

Cheers,
-Barry

[*] For me, not just setup.py but also tox.ini, coverage.ini, MANIFEST.in,
other helpers, readmes, license files, source templates, etc.


pgp6UWM2grPTR.pgp
Description: OpenPGP digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Versioned trove classifiers for Django

[Alex Clark]

On 3/30/15 1:13 AM, Richard Jones wrote:

Added!

Framework :: Django :: 1.4
Framework :: Django :: 1.5
Framework :: Django :: 1.6
Framework :: Django :: 1.7
Framework :: Django :: 1.8



Great! In Plone's case, we also added every foreseeable future version, 
which it appears you have already done. Tha gives folks the ability to 
cut new releases of add-ons against beta and rc versions of Django, 
which is awesome.






On Mon, 30 Mar 2015 at 16:09 James Bennett mailto:ubernost...@gmail.com>> wrote:

I would be OK with including 1.5 just for completeness' sake.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig




--
Alex Clark · http://about.me/alex.clark

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Isn't it neat that pip-installable packages can be built without distutils?

[Donald Stufft]

> On Mar 30, 2015, at 2:03 PM, Daniel Holth  wrote:
> 
> What else can we do to make the experience better for those who would
> try to replace distutils?

Continue work on the interoperability standards that exist for that very
purpose.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Isn't it neat that pip-installable packages can be built without distutils?

[Daniel Holth]

What else can we do to make the experience better for those who would
try to replace distutils?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 11:33 AM, Ian Foote  wrote:
> 
> 
> On 30/03/15 16:05, Ian Cordasco wrote:
>> On Mon, Mar 30, 2015 at 9:59 AM, Daniel Holth > > wrote:
>> Yes, setup.py should die. Flit is one example, and you can understand
>> it not by copy/pasting, but by spending half an hour reading its
>> complete source code.
>> 
>> In other words, no one should read the docs because that's a waste of time? 
>> Because a lot of time has been poured into the packaging docs and if they're 
>> not sufficient, then instead of improving them, people should write 
>> undocumented tools that force people to read the source? I'm not sure how 
>> that's better than what we already have.
> 
> You're attacking a strawman. Flit does have documentation. What Daniel was 
> trying to say is that flit is small enough to understand by just reading the 
> source code.



Meh, comparisons by SLOC are silly in either direction.

The size of the code base isn’t an interesting discriminator in judging 
quality. Python does not try to be a language that minimizes the number of 
lines of code and when looking at straight SLOC (in abstract of these two 
projects) it’s entirely possible to have something harder to understand with 
less SLOC and easier to understand with more. In particular looking at these 
two pieces of software, trying to compare SLOC is even more silly given the two 
toolchains don’t even come close to the same feature set.

Ian C, is pointing out that being able to understand it “just by reading the 
source code” (whatever that means, what software can’t you understand by 
reading the source code?) isn’t an extremely useful discriminator either 
because it’s premised on the fact that there is going to be a common situation 
where you *need* to read the source code to understand it. So the statement is 
in itself not particularly useful because it’s premised on something that is 
fundamentally user hostile. An end user shouldn’t care if it’s written in 600 
lines, 6000 lines, or 6 million lines. What matters to them is the interface it 
provides and the documentation around it and often the “ease” of reading the 
code is used to excuse poor (or non existent) documentation.

IOW, using SLOC or size or anything of the like as a discriminator is akin to 
saying “well it can fit on a floppy” as a measure of quality.

None of this is particularly centered around flit (or distutils) so I’m not 
speaking about either project there.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Foote]

On 30/03/15 16:05, Ian Cordasco wrote:
On Mon, Mar 30, 2015 at 9:59 AM, Daniel Holth > wrote:


Yes, setup.py should die. Flit is one example, and you can understand
it not by copy/pasting, but by spending half an hour reading its
complete source code.


In other words, no one should read the docs because that's a waste of 
time? Because a lot of time has been poured into the packaging docs 
and if they're not sufficient, then instead of improving them, people 
should write undocumented tools that force people to read the source? 
I'm not sure how that's better than what we already have.


You're attacking a strawman. Flit does have documentation. What Daniel 
was trying to say is that flit is small enough to understand by just 
reading the source code.


Regards,
Ian F
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 11:15 AM, Ionel Cristian Mărieș 
wrote:

>
>
> On Mon, Mar 30, 2015 at 6:55 PM, Ian Cordasco 
> wrote:
>
>> Then, when the "new user" goes to publish it, there's tons of prior
>> documentation on how to do it. If they run into problems using flit they
>> have the skimpy documentation or the source.
>
>
> ​Now it might have skimpy docs and no users, but that's largely a product
> of time. I think `flit` should be judged on what it can be in the future,
> not all what it's right now. To put it in picture, the argument you're
> making is like comparing the amazon rainforest to a banana milkshake
> recipe.​
>

Well to make a better comparison, we're discussing a healthy diet to a
sugar laced treat. The healthy diet is sustainable because it is well
documented and has tons of users with experience with it, but has pitfalls
in that it can be expensive at times, while the sugar laced treat is good
for a quick blood sugar spike but will leave you thoroughly unsatisfied and
eventually wanting for the healthy diet.

New users, (like some people who prefer high sugar diets) may prefer the
initial simplicity, but at the cost of having to do a lot more work up
front. Those who follow a healthy diet (and ostensibly exercise regiment)
will have tooling to not have to worry about how to construct a healthy
diet. New users who find and work with those already on a healthy diet will
learn the tools that help them avoid the pitfalls of starting on a healthy
diet (e.g., tools that generate setup.py for you and maintain the 98% use
case, which is inevitably where new users fall*) and will be better off for
the long term.

* Note, new users still has yet to be defined by anyone advocating that
this is better for new users because of mystical reasons (like being able
to read the source code).
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 11:14 AM, Daniel Holth  wrote:

> On Mon, Mar 30, 2015 at 11:55 AM, Ian Cordasco
>  wrote:
> >
> >
> > On Mon, Mar 30, 2015 at 10:47 AM, Ionel Cristian Mărieș <
> cont...@ionelmc.ro>
> > wrote:
> >>
> >> On Mon, Mar 30, 2015 at 6:39 PM, Xavier Fernandez
> >>  wrote:
> >>>
> >>> I think the point was not to say that documentation is useless (and
> there
> >>> is some: http://flit.readthedocs.org/en/latest/ ) but that the
> >>> code/implementation is much simpler than the combination of
> >>> distutils/setuptools/bdist_wheel.
> >>>
> >>> On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco
> >>>  wrote:
> 
> 
>  So for new python programmers (or newbie users in general) reading the
>  entire source of another package to understand it is a better
> experience?
> 
> >> To put that in context, flit goes for less than 600 SLOC while
> >> distutils+setuptools+wheel amount to over 2 SLOC. At that ratio
> >> arguments for distutils+setuptools+wheel documentation seem
> unreasonable.
> >
> >
> > To be clear, no one should ever be advocating to "just read the source"
> as a
> > form of documentation. This is why the Packaging guide exists (because no
> > one should ever be expected to read the distutils, setuptools, or wheel
> > source to use it).
> >
> > Code is never as self-documenting as people like to believe. And since
> we're
> > talking about new users (without defining what they're new to) reading
> the
> > source should only be for educational purposes. cookiecutter will serve
> new
> > users better than flit or anything else. cookiecutter will teach new
> users
> > good package structure and take care of the (possibly hard parts) of a
> > setup.py. Then, when the "new user" goes to publish it, there's tons of
> > prior documentation on how to do it. If they run into problems using flit
> > they have the skimpy documentation or the source.
> >
> > Yeah, it's "easy" to read 600 SLOC for you, but what about for some "new
> > user"? Are they new to python? Why do they have to care about reading the
> > source if something else will "just work" as documented for their
> "simple"
> > use case?
>
> No one has advocated reading the source code instead of reading the
> documentation.
>

Thankfully this is a publicly archived list. Quoting yourself:

> Flit is one example, and you can understand it not by copy/pasting,
> but by spending half an hour reading its complete source code.

In which you advocate reading the source of a tool over using setup.py
which has countless resources written about it on the internet.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ionel Cristian Mărieș]

On Mon, Mar 30, 2015 at 6:55 PM, Ian Cordasco 
wrote:

> Then, when the "new user" goes to publish it, there's tons of prior
> documentation on how to do it. If they run into problems using flit they
> have the skimpy documentation or the source.


​Now it might have skimpy docs and no users, but that's largely a product
of time. I think `flit` should be judged on what it can be in the future,
not all what it's right now. To put it in picture, the argument you're
making is like comparing the amazon rainforest to a banana milkshake
recipe.​



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

On Mon, Mar 30, 2015 at 11:55 AM, Ian Cordasco
 wrote:
>
>
> On Mon, Mar 30, 2015 at 10:47 AM, Ionel Cristian Mărieș 
> wrote:
>>
>> On Mon, Mar 30, 2015 at 6:39 PM, Xavier Fernandez
>>  wrote:
>>>
>>> I think the point was not to say that documentation is useless (and there
>>> is some: http://flit.readthedocs.org/en/latest/ ) but that the
>>> code/implementation is much simpler than the combination of
>>> distutils/setuptools/bdist_wheel.
>>>
>>> On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco
>>>  wrote:


 So for new python programmers (or newbie users in general) reading the
 entire source of another package to understand it is a better experience?

>> To put that in context, flit goes for less than 600 SLOC while
>> distutils+setuptools+wheel amount to over 2 SLOC. At that ratio
>> arguments for distutils+setuptools+wheel documentation seem unreasonable.
>
>
> To be clear, no one should ever be advocating to "just read the source" as a
> form of documentation. This is why the Packaging guide exists (because no
> one should ever be expected to read the distutils, setuptools, or wheel
> source to use it).
>
> Code is never as self-documenting as people like to believe. And since we're
> talking about new users (without defining what they're new to) reading the
> source should only be for educational purposes. cookiecutter will serve new
> users better than flit or anything else. cookiecutter will teach new users
> good package structure and take care of the (possibly hard parts) of a
> setup.py. Then, when the "new user" goes to publish it, there's tons of
> prior documentation on how to do it. If they run into problems using flit
> they have the skimpy documentation or the source.
>
> Yeah, it's "easy" to read 600 SLOC for you, but what about for some "new
> user"? Are they new to python? Why do they have to care about reading the
> source if something else will "just work" as documented for their "simple"
> use case?

No one has advocated reading the source code instead of reading the
documentation.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 11:52 AM, Paul Moore  wrote:
> 
> On 30 March 2015 at 16:45, Donald Stufft  wrote:
>> Well, parts of it are turing complete, since it pulls the version number
>> out of the module itself and that’s just Python too.
> 
> Sorry, I wasn't specifically looking at flit there. But I'm in the
> camp that says just put the version in your ini file and in your
> module, and don't worry that you have it in 2 places. If managing
> version numbers is the biggest showstopper in moving to declarative
> metadata, then we've won :-)
> 
> Paul

Honestly, I don’t think that setup.py as a development interface is that
bad. It gets really bad when we start sticking it inside of a sdist and
using that as part of the installation metadata.

It’s not unusual for me to want (or need) to do something a little bit
different in a project, or something that the original authors didn’t
quite intend to do. This is perfectly valid and fine inside of a file
that only ever gets executed on a developer machine. However it *needs*
to be “compiled” down to a static file when creating a sdist.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 10:47 AM, Ionel Cristian Mărieș 
wrote:

> On Mon, Mar 30, 2015 at 6:39 PM, Xavier Fernandez  > wrote:
>
>> I think the point was not to say that documentation is useless (and there
>> is some: http://flit.readthedocs.org/en/latest/ ) but that the
>> code/implementation is much simpler than the combination of
>> distutils/setuptools/bdist_wheel.
>>
>> On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco > > wrote:
>>
>>>
>>> So for new python programmers (or newbie users in general) reading the
>>> entire source of another package to understand it is a better experience?
>>>
>>> ​
> ​To put that in context, flit goes for less than 600 SLOC while
> distutils+setuptools+wheel amount to​ over 2 SLOC. At that ratio
> arguments for distutils+setuptools+wheel documentation seem unreasonable.
>

To be clear, no one should ever be advocating to "just read the source" as
a form of documentation. This is why the Packaging guide exists (because no
one should ever be expected to read the distutils, setuptools, or wheel
source to use it).

Code is never as self-documenting as people like to believe. And since
we're talking about new users (without defining what they're new to)
reading the source should only be for educational purposes. cookiecutter
will serve new users better than flit or anything else. cookiecutter will
teach new users good package structure and take care of the (possibly hard
parts) of a setup.py. Then, when the "new user" goes to publish it, there's
tons of prior documentation on how to do it. If they run into problems
using flit they have the skimpy documentation or the source.

Yeah, it's "easy" to read 600 SLOC for you, but what about for some "new
user"? Are they new to python? Why do they have to care about reading the
source if something else will "just work" as documented for their "simple"
use case?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

This is also a great time to be getting working setup-requires. A
pip-compatible flit sdist could declare flit as a setup requirement,
and its setup.py could translate install/bdist_wheel to the
appropriate flit calls.

On Mon, Mar 30, 2015 at 11:47 AM, Ionel Cristian Mărieș
 wrote:
> On Mon, Mar 30, 2015 at 6:39 PM, Xavier Fernandez 
> wrote:
>>
>> I think the point was not to say that documentation is useless (and there
>> is some: http://flit.readthedocs.org/en/latest/ ) but that the
>> code/implementation is much simpler than the combination of
>> distutils/setuptools/bdist_wheel.
>>
>> On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco 
>> wrote:
>>>
>>>
>>> So for new python programmers (or newbie users in general) reading the
>>> entire source of another package to understand it is a better experience?
>>>
> To put that in context, flit goes for less than 600 SLOC while
> distutils+setuptools+wheel amount to over 2 SLOC. At that ratio
> arguments for distutils+setuptools+wheel documentation seem unreasonable.
>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Paul Moore]

On 30 March 2015 at 16:45, Donald Stufft  wrote:
> Well, parts of it are turing complete, since it pulls the version number
> out of the module itself and that’s just Python too.

Sorry, I wasn't specifically looking at flit there. But I'm in the
camp that says just put the version in your ini file and in your
module, and don't worry that you have it in 2 places. If managing
version numbers is the biggest showstopper in moving to declarative
metadata, then we've won :-)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ionel Cristian Mărieș]

On Mon, Mar 30, 2015 at 6:39 PM, Xavier Fernandez 
wrote:

> I think the point was not to say that documentation is useless (and there
> is some: http://flit.readthedocs.org/en/latest/ ) but that the
> code/implementation is much simpler than the combination of
> distutils/setuptools/bdist_wheel.
>
> On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco 
> wrote:
>
>>
>> So for new python programmers (or newbie users in general) reading the
>> entire source of another package to understand it is a better experience?
>>
>> ​
​To put that in context, flit goes for less than 600 SLOC while
distutils+setuptools+wheel amount to​ over 2 SLOC. At that ratio
arguments for distutils+setuptools+wheel documentation seem unreasonable.

​
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 11:41 AM, Paul Moore  wrote:
> 
> On 30 March 2015 at 16:34, Donald Stufft  wrote:
>> The current toolchain absolutely has some problems, but I’m not convinced
>> that shuffling around the same data into different locations is the answer
>> to those problems.
> 
> Moving the data to a location that isn't Turing-complete would be a
> benefit. Changing the processing backend to something different once
> the declarative UI is in common use would be a lot easier than where
> we are now. Even a 90% solution would be worthwhile here - we don't
> *have* to cater for every use case in one go.
> 
> Paul

Well, parts of it are turing complete, since it pulls the version number
out of the module itself and that’s just Python too.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Paul Moore]

On 30 March 2015 at 16:34, Donald Stufft  wrote:
> The current toolchain absolutely has some problems, but I’m not convinced
> that shuffling around the same data into different locations is the answer
> to those problems.

Moving the data to a location that isn't Turing-complete would be a
benefit. Changing the processing backend to something different once
the declarative UI is in common use would be a lot easier than where
we are now. Even a 90% solution would be worthwhile here - we don't
*have* to cater for every use case in one go.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Xavier Fernandez]

I think the point was not to say that documentation is useless (and there
is some: http://flit.readthedocs.org/en/latest/ ) but that the
code/implementation is much simpler than the combination of
distutils/setuptools/bdist_wheel.

On Mon, Mar 30, 2015 at 5:26 PM, Ian Cordasco 
wrote:

>
>
> On Mon, Mar 30, 2015 at 10:23 AM, Ionel Cristian Mărieș <
> cont...@ionelmc.ro> wrote:
>
>>
>> On Mon, Mar 30, 2015 at 6:05 PM, Ian Cordasco > > wrote:
>>
>>> In other words, no one should read the docs because that's a waste of
>>> time? Because a lot of time has been poured into the packaging docs and if
>>> they're not sufficient, then instead of improving them, people should write
>>> undocumented tools that force people to read the source? I'm not sure how
>>> that's better than what we already have.
>>
>>
>> ​A waste of time? No. But it sure is poor investment of time for newbie
>> users. It's a futile struggle to document (or read about) all the features
>> of distutils and setuptools, not because it can't be documented, but
>> because there's too much ground to cover and PyPA's packaging.python.org
>> approach is to ​just give an overview of what's available and avoid giving
>> any real best practice recommendations as much as possible. There may be
>> good reasons for that but that's not a sensible approach to giving users a
>> "pitfall free" learning path.
>>
>
> So for new python programmers (or newbie users in general) reading the
> entire source of another package to understand it is a better experience?
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 11:36 AM, Daniel Holth  wrote:
> 
> On Mon, Mar 30, 2015 at 11:34 AM, Donald Stufft  wrote:
>> 
>>> On Mar 30, 2015, at 11:18 AM, Daniel Holth  wrote:
>>> 
>>> setup.py as implemented with distutils/setuptools has a bit of a
>>> Goldilocks problem: it's just right for a medium-complexity project
>>> but when your project is very simple it's too hard, and when you get
>>> to the point where you are trying to extend distutils by writing a
>>> 10,000 line extension, yikes. So it's fantastic to be able to just
>>> avoid distutils entirely if it isn't the right size for your project.
>>> This example, flit, does not invoke any code from distutils,
>>> setuptools or bdist_wheel to do its thing.
>>> 
>>> A source release could just be an archive of the repository.
>>> 
>> 
>> An archive of the repository is not the same thing as a source release.
>> 
>> Honestly, most of my setup.py’s look basically the same as a flit ini
>> file, just inside of python instead of ini. For example, I’m not sure
>> how something like https://github.com/pypa/packaging/blob/master/setup.py
>> or https://github.com/pypa/warehouse/blob/master/setup.py or
>> https://github.com/pypa/twine/blob/master/setup.py or
>> https://github.com/pypa/readme/blob/master/setup.py would be improved by
>> moving it to a ini file instead of a python file.
>> 
>> The current toolchain absolutely has some problems, but I’m not convinced
>> that shuffling around the same data into different locations is the answer
>> to those problems.
> 
> The way to solve the problems is to allow anyone to try by providing
> good hooks that do not require extending distutils.

Sure. Which is why that’s what we’ve essentially been doing. Like I said I
have no problems with flit itself other than I think it needs to produce
sdists to be a reasonable solution. I don’t personally think it’s much
easier to work with than a simple setup.py (and in my experience, I often
want a simple.py plus a little extra) but I have no problem with its
existence as long as it learns how to produce sdists.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

On Mon, Mar 30, 2015 at 11:34 AM, Donald Stufft  wrote:
>
>> On Mar 30, 2015, at 11:18 AM, Daniel Holth  wrote:
>>
>> setup.py as implemented with distutils/setuptools has a bit of a
>> Goldilocks problem: it's just right for a medium-complexity project
>> but when your project is very simple it's too hard, and when you get
>> to the point where you are trying to extend distutils by writing a
>> 10,000 line extension, yikes. So it's fantastic to be able to just
>> avoid distutils entirely if it isn't the right size for your project.
>> This example, flit, does not invoke any code from distutils,
>> setuptools or bdist_wheel to do its thing.
>>
>> A source release could just be an archive of the repository.
>>
>
> An archive of the repository is not the same thing as a source release.
>
> Honestly, most of my setup.py’s look basically the same as a flit ini
> file, just inside of python instead of ini. For example, I’m not sure
> how something like https://github.com/pypa/packaging/blob/master/setup.py
> or https://github.com/pypa/warehouse/blob/master/setup.py or
> https://github.com/pypa/twine/blob/master/setup.py or
> https://github.com/pypa/readme/blob/master/setup.py would be improved by
> moving it to a ini file instead of a python file.
>
> The current toolchain absolutely has some problems, but I’m not convinced
> that shuffling around the same data into different locations is the answer
> to those problems.

The way to solve the problems is to allow anyone to try by providing
good hooks that do not require extending distutils.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 11:18 AM, Daniel Holth  wrote:
> 
> setup.py as implemented with distutils/setuptools has a bit of a
> Goldilocks problem: it's just right for a medium-complexity project
> but when your project is very simple it's too hard, and when you get
> to the point where you are trying to extend distutils by writing a
> 10,000 line extension, yikes. So it's fantastic to be able to just
> avoid distutils entirely if it isn't the right size for your project.
> This example, flit, does not invoke any code from distutils,
> setuptools or bdist_wheel to do its thing.
> 
> A source release could just be an archive of the repository.
> 

An archive of the repository is not the same thing as a source release.

Honestly, most of my setup.py’s look basically the same as a flit ini
file, just inside of python instead of ini. For example, I’m not sure
how something like https://github.com/pypa/packaging/blob/master/setup.py
or https://github.com/pypa/warehouse/blob/master/setup.py or
https://github.com/pypa/twine/blob/master/setup.py or
https://github.com/pypa/readme/blob/master/setup.py would be improved by
moving it to a ini file instead of a python file.

The current toolchain absolutely has some problems, but I’m not convinced
that shuffling around the same data into different locations is the answer
to those problems.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Paul Moore]

On 30 March 2015 at 16:26, Ian Cordasco  wrote:
>> A waste of time? No. But it sure is poor investment of time for newbie
>> users. It's a futile struggle to document (or read about) all the features
>> of distutils and setuptools, not because it can't be documented, but because
>> there's too much ground to cover and PyPA's packaging.python.org approach is
>> to just give an overview of what's available and avoid giving any real best
>> practice recommendations as much as possible. There may be good reasons for
>> that but that's not a sensible approach to giving users a "pitfall free"
>> learning path.
>
> So for new python programmers (or newbie users in general) reading the
> entire source of another package to understand it is a better experience?

Note that if you want a copy-and-paste level "quick start" setup.py,
https://github.com/pypa/sampleproject is basically what you want. The
packaging user guide uses it as its recommended starting point. If you
don't like it, that probably means you already have enough knowledge
to write your own version :-)

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

On Mon, Mar 30, 2015 at 11:22 AM, Ian Cordasco
 wrote:
> On Mon, Mar 30, 2015 at 10:18 AM, Daniel Holth  wrote:
>>
>> setup.py as implemented with distutils/setuptools has a bit of a
>> Goldilocks problem: it's just right for a medium-complexity project
>> but when your project is very simple it's too hard, and when you get
>> to the point where you are trying to extend distutils by writing a
>> 10,000 line extension, yikes. So it's fantastic to be able to just
>> avoid distutils entirely if it isn't the right size for your project.
>> This example, flit, does not invoke any code from distutils,
>> setuptools or bdist_wheel to do its thing.
>>
>> A source release could just be an archive of the repository.
>
>
> You still have not answered how reading flit's source code to get it working
> is better than using cookiecutter to generate a project, and using `python
> setup.py bdist_wheel sdist` (which is well-documented, and has tons of
> answered questions on sites like StackOverflow to help in case of a
> problem).

It's simple. distutils, setuptools, and bdist_wheel are all terrible!
They solve a 57,000 package strong backwards compatibility problem,
and bdist_wheel specifically helps correct the tightly coupled
build-install design flaw in distutils, but if you can avoid them
great!

You probably would not have to actually read the source code to flit
in order to use it, but if it came down to that then it would not take
very long :)
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ionel Cristian Mărieș]

On Mon, Mar 30, 2015 at 6:05 PM, Ian Cordasco 
wrote:

> In other words, no one should read the docs because that's a waste of
> time? Because a lot of time has been poured into the packaging docs and if
> they're not sufficient, then instead of improving them, people should write
> undocumented tools that force people to read the source? I'm not sure how
> that's better than what we already have.


​A waste of time? No. But it sure is poor investment of time for newbie
users. It's a futile struggle to document (or read about) all the features
of distutils and setuptools, not because it can't be documented, but
because there's too much ground to cover and PyPA's packaging.python.org
approach is to ​just give an overview of what's available and avoid giving
any real best practice recommendations as much as possible. There may be
good reasons for that but that's not a sensible approach to giving users a
"pitfall free" learning path.



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 10:23 AM, Ionel Cristian Mărieș 
wrote:

>
> On Mon, Mar 30, 2015 at 6:05 PM, Ian Cordasco 
> wrote:
>
>> In other words, no one should read the docs because that's a waste of
>> time? Because a lot of time has been poured into the packaging docs and if
>> they're not sufficient, then instead of improving them, people should write
>> undocumented tools that force people to read the source? I'm not sure how
>> that's better than what we already have.
>
>
> ​A waste of time? No. But it sure is poor investment of time for newbie
> users. It's a futile struggle to document (or read about) all the features
> of distutils and setuptools, not because it can't be documented, but
> because there's too much ground to cover and PyPA's packaging.python.org
> approach is to ​just give an overview of what's available and avoid giving
> any real best practice recommendations as much as possible. There may be
> good reasons for that but that's not a sensible approach to giving users a
> "pitfall free" learning path.
>

So for new python programmers (or newbie users in general) reading the
entire source of another package to understand it is a better experience?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Paul Moore]

On 30 March 2015 at 16:06, Donald Stufft  wrote:
>> On Mar 30, 2015, at 10:59 AM, Daniel Holth  wrote:
>>
>> Yes, setup.py should die. Flit is one example, and you can understand
>> it not by copy/pasting, but by spending half an hour reading its
>> complete source code.
>
> I don’t have a problem with flit (although I’m not sure it’s easier, it
> appears it took the setup.py keyword args and turned them into ini file
> directives). I do have a problem with any solution which doesn’t include
> sdist support.

Personally, I could see a benefit to something that allowed me to
write my setup.py as

import fancytool
fancytool.setup()

and got everything from a static file. But otherwise worked just like
distutils/setuptools (i.e. fancytool.setup() calls setuptools.setup()
behind the scenes). I'd be happy if it only handled pure-python
packages, and if it didn't cover complicated things. I wouldn't be
worried that I had to manually install fancytool before my setup.py
worked (no setup_requires stuff here, thanks!)

I don't see "don't use setuptools behind the scenes" as a necessary
goal. I *do* see "make the UI simple for 90% of projects" as a
worthwhile goal.

I like the idea and goal behind flit. I'm not sure the implementation
strategy is sufficiently compatible with existing practices, though.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 10:18 AM, Daniel Holth  wrote:

> setup.py as implemented with distutils/setuptools has a bit of a
> Goldilocks problem: it's just right for a medium-complexity project
> but when your project is very simple it's too hard, and when you get
> to the point where you are trying to extend distutils by writing a
> 10,000 line extension, yikes. So it's fantastic to be able to just
> avoid distutils entirely if it isn't the right size for your project.
> This example, flit, does not invoke any code from distutils,
> setuptools or bdist_wheel to do its thing.
>
> A source release could just be an archive of the repository.
>

You still have not answered how reading flit's source code to get it
working is better than using cookiecutter to generate a project, and using
`python setup.py bdist_wheel sdist` (which is well-documented, and has tons
of answered questions on sites like StackOverflow to help in case of a
problem).
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

setup.py as implemented with distutils/setuptools has a bit of a
Goldilocks problem: it's just right for a medium-complexity project
but when your project is very simple it's too hard, and when you get
to the point where you are trying to extend distutils by writing a
10,000 line extension, yikes. So it's fantastic to be able to just
avoid distutils entirely if it isn't the right size for your project.
This example, flit, does not invoke any code from distutils,
setuptools or bdist_wheel to do its thing.

A source release could just be an archive of the repository.


On Mon, Mar 30, 2015 at 10:59 AM, Daniel Holth  wrote:
> Yes, setup.py should die. Flit is one example, and you can understand
> it not by copy/pasting, but by spending half an hour reading its
> complete source code.
>
> On Mon, Mar 30, 2015 at 10:50 AM, Ian Cordasco
>  wrote:
>> On Mon, Mar 30, 2015 at 9:46 AM, Daniel Holth  wrote:
>>>
>>> The approach doesn't exclude the possibility of a source distribution,
>>> it's only a few weeks old. I would suggest that if you have to choose
>>> between having a setup.py, not having a setup.py, and not having the
>>> package on pypi at all because the packager can't figure out setup.py,
>>> prefer the second option.
>>
>>
>> Is figuring out setup.py still a thing? Between cookiecutter laying out your
>> project with a setup.py and the Packaging Guide, how many people still have
>> trouble setting up a setup.py for a package? It almost seems like this is a
>> solution wanting for a problem.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Xavier Fernandez]

Fair enough, I didn't think of compiled wheels :)
And having a clean way to run tests for the provided wheel is indeed an
other good point.

On Mon, Mar 30, 2015 at 5:04 PM, Donald Stufft  wrote:

>
> On Mar 30, 2015, at 10:58 AM, Xavier Fernandez 
> wrote:
>
> Wheels without sdists are likely a generally bad idea, downstream
>> redistributors are not going to like them.
>>
>
> Why do you think that ? Wheels seem way simpler/saner than all the
> possible things setup.py can do.
>
>
> Wheels are simpler than a setup.py, because wheels are a binary format and
> Wheels don’t need to
> handle things like build software because it’s already been built. However
> downstream redistributors
> will not accept a Wheel as the source for a package because it is a binary
> format. It doesn’t matter
> if you can unzip it and there is pure python there, it is still a binary
> format. So if you release only Wheels
> you’re essentially saying that downstream redistributors will never
> package your software (or any
> software that depends on it).
>
> A few issues that Wheel only has:
>
> * If your project has a C extension, downstream redistributes need access
> to the source code not the
>   compiled code (as does anyone wanting to use the project on platform you
> didn’t release for).
>
> * If your project has tests that don’t get installed, they should get
> shipped as part of the sdist so that
>   downstream can run them as part of their packaging activities to ensure
> they didn’t break anything
>   in the test suite. However if you’re installing from Wheel you can’t do
> that.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 10:59 AM, Daniel Holth  wrote:
> 
> Yes, setup.py should die. Flit is one example, and you can understand
> it not by copy/pasting, but by spending half an hour reading its
> complete source code.

I don’t have a problem with flit (although I’m not sure it’s easier, it
appears it took the setup.py keyword args and turned them into ini file
directives). I do have a problem with any solution which doesn’t include
sdist support.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 9:59 AM, Daniel Holth  wrote:

> Yes, setup.py should die. Flit is one example, and you can understand
> it not by copy/pasting, but by spending half an hour reading its
> complete source code.
>

In other words, no one should read the docs because that's a waste of time?
Because a lot of time has been poured into the packaging docs and if
they're not sufficient, then instead of improving them, people should write
undocumented tools that force people to read the source? I'm not sure how
that's better than what we already have.

Further, how does flit deal with C-extensions? How does flit deal with
C-extensions distributed on Linux? How does it generate the appropriate
wheels for that?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 10:58 AM, Xavier Fernandez  
> wrote:
> 
> Wheels without sdists are likely a generally bad idea, downstream
> redistributors are not going to like them.
> 
> Why do you think that ? Wheels seem way simpler/saner than all the possible 
> things setup.py can do.


Wheels are simpler than a setup.py, because wheels are a binary format and 
Wheels don’t need to
handle things like build software because it’s already been built. However 
downstream redistributors
will not accept a Wheel as the source for a package because it is a binary 
format. It doesn’t matter
if you can unzip it and there is pure python there, it is still a binary 
format. So if you release only Wheels
you’re essentially saying that downstream redistributors will never package 
your software (or any
software that depends on it).

A few issues that Wheel only has:

* If your project has a C extension, downstream redistributes need access to 
the source code not the
  compiled code (as does anyone wanting to use the project on platform you 
didn’t release for).

* If your project has tests that don’t get installed, they should get shipped 
as part of the sdist so that
  downstream can run them as part of their packaging activities to ensure they 
didn’t break anything
  in the test suite. However if you’re installing from Wheel you can’t do that.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

Yes, setup.py should die. Flit is one example, and you can understand
it not by copy/pasting, but by spending half an hour reading its
complete source code.

On Mon, Mar 30, 2015 at 10:50 AM, Ian Cordasco
 wrote:
> On Mon, Mar 30, 2015 at 9:46 AM, Daniel Holth  wrote:
>>
>> The approach doesn't exclude the possibility of a source distribution,
>> it's only a few weeks old. I would suggest that if you have to choose
>> between having a setup.py, not having a setup.py, and not having the
>> package on pypi at all because the packager can't figure out setup.py,
>> prefer the second option.
>
>
> Is figuring out setup.py still a thing? Between cookiecutter laying out your
> project with a setup.py and the Packaging Guide, how many people still have
> trouble setting up a setup.py for a package? It almost seems like this is a
> solution wanting for a problem.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Xavier Fernandez]

>
> Wheels without sdists are likely a generally bad idea, downstream
> redistributors are not going to like them.
>

Why do you think that ? Wheels seem way simpler/saner than all the possible
things setup.py can do.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Ian Cordasco]

On Mon, Mar 30, 2015 at 9:46 AM, Daniel Holth  wrote:

> The approach doesn't exclude the possibility of a source distribution,
> it's only a few weeks old. I would suggest that if you have to choose
> between having a setup.py, not having a setup.py, and not having the
> package on pypi at all because the packager can't figure out setup.py,
> prefer the second option.
>

Is figuring out setup.py still a thing? Between cookiecutter laying out
your project with a setup.py and the Packaging Guide, how many people still
have trouble setting up a setup.py for a package? It almost seems like this
is a solution wanting for a problem.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

The approach doesn't exclude the possibility of a source distribution,
it's only a few weeks old. I would suggest that if you have to choose
between having a setup.py, not having a setup.py, and not having the
package on pypi at all because the packager can't figure out setup.py,
prefer the second option.

On Mon, Mar 30, 2015 at 10:32 AM, Donald Stufft  wrote:
>
>> On Mar 30, 2015, at 10:19 AM, Daniel Holth  wrote:
>>
>> I was wondering when someone would attempt this. Simple .ini file and
>> some conventions, and you have a pip-installable wheel. No
>> bdist_wheel, setup.py, MANIFEST or setuptools involved. I like it.
>> Still alpha/beta.
>>
>> https://github.com/takluyver/flit
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>
> Wheels without sdists are likely a generally bad idea, downstream
> redistributors are not going to like them.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] it's happened - wheels without sdists (flit)

[Donald Stufft]

> On Mar 30, 2015, at 10:19 AM, Daniel Holth  wrote:
> 
> I was wondering when someone would attempt this. Simple .ini file and
> some conventions, and you have a pip-installable wheel. No
> bdist_wheel, setup.py, MANIFEST or setuptools involved. I like it.
> Still alpha/beta.
> 
> https://github.com/takluyver/flit
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

Wheels without sdists are likely a generally bad idea, downstream
redistributors are not going to like them.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] it's happened - wheels without sdists (flit)

[Daniel Holth]

I was wondering when someone would attempt this. Simple .ini file and
some conventions, and you have a pip-installable wheel. No
bdist_wheel, setup.py, MANIFEST or setuptools involved. I like it.
Still alpha/beta.

https://github.com/takluyver/flit
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig