[Distutils] PEP 426: proposed change to extension fields + entry points
I've decided I'm really not happy with the current approach to
extension fields in PEP 426. It's ugly, clunky, inflexible and is hard
to cleanly convert to more structured metadata.
Here's the current example from the PEP:
Extension: Chili
Chili/Type: Poblano
Chili/Heat: Mild
Here's what I'm thinking of changing it to:
Extension: Chili
Chili/json: {
type: Poblano,
heat: Mild
}
So rather than allowing *arbitrary* additional headers as it does now,
an Extension declaration would allow only one new header:
extension-name/json
That would then allow arbitrary embedded JSON (so long as you still
follow the no blank lines, because that 's the end of the headers
rule)
When converting the entire metadata definition to JSON for subsequent
processing, the extensions fields would then cleanly convert from
multiple extension entries to something like:
extensions: {
Chili: {
type: Poblano,
heat: Mild
}
}
The other area where I think such an embedded JSON approach could
work is coming up with a clean format for an Entry-Points field.
Specifically, I am thinking of proposing the setuptools.setup
inspired:
Entry-Points: {
console_scripts: {
foo: my_package.some_module:main_func,
bar: other_module:some_func
},
gui_scripts: {
baz: my_package_gui.start_func
}
}
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
I'm probably the only one but I'm not a fan of JSON with all the extra
marks compared to the venerable, lovely, flatter and much easier to edit
Key: value format. The METADATA file needs to represent Name, Version, and
Requirements and the rest is fluff that no one will ever use. It's very
healthy to put other kinds of metadata in different files in the .dist-info
directory (no need to bump the metadata version number) and our package
indices can learn how to index those too.
For entry-points it would be more than sufficient to say this package
defines console_scripts and gui_scripts and put the details elsewhere.
On Mon, Feb 25, 2013 at 9:19 AM, Nick Coghlan ncogh...@gmail.com wrote:
I've decided I'm really not happy with the current approach to
extension fields in PEP 426. It's ugly, clunky, inflexible and is hard
to cleanly convert to more structured metadata.
Here's the current example from the PEP:
Extension: Chili
Chili/Type: Poblano
Chili/Heat: Mild
Here's what I'm thinking of changing it to:
Extension: Chili
Chili/json: {
type: Poblano,
heat: Mild
}
So rather than allowing *arbitrary* additional headers as it does now,
an Extension declaration would allow only one new header:
extension-name/json
That would then allow arbitrary embedded JSON (so long as you still
follow the no blank lines, because that 's the end of the headers
rule)
When converting the entire metadata definition to JSON for subsequent
processing, the extensions fields would then cleanly convert from
multiple extension entries to something like:
extensions: {
Chili: {
type: Poblano,
heat: Mild
}
}
The other area where I think such an embedded JSON approach could
work is coming up with a clean format for an Entry-Points field.
Specifically, I am thinking of proposing the setuptools.setup
inspired:
Entry-Points: {
console_scripts: {
foo: my_package.some_module:main_func,
bar: other_module:some_func
},
gui_scripts: {
baz: my_package_gui.start_func
}
}
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
On Tue, Feb 26, 2013 at 12:38 AM, Daniel Holth dho...@gmail.com wrote: I'm probably the only one but I'm not a fan of JSON with all the extra marks compared to the venerable, lovely, flatter and much easier to edit Key: value format. I don't really care that much about human readability of the raw metadata files, I care a lot more about ease of interoperability for automated tools, and ease of display for generic tools that may not understand the semantics. I also care about the standard format being amenable to the publication of metadata through TUF's JSON based infrastructure. The METADATA file needs to represent Name, Version, and Requirements and the rest is fluff that no one will ever use. It's very healthy to put other kinds of metadata in different files in the .dist-info directory (no need to bump the metadata version number) and our package indices can learn how to index those too. No, it's not healthy at all. It means you have an ad hoc collection of formats, that some tools may or may not understand, so you can't publish them with confidence, because you're relying on consumers using particular tools. The whole point of wheel is to decouple builders from installers, which means you can't rely on ad hoc formats for communicating between them reliably. Furthermore, handling arbitrary files would make life a lot harder than it needs to be when we finally make the installation database work with import hooks, whereas exposing JSON-compatible structured metadata through an API is relatively trivial (you can just return a metadata dictionary, without needing to care how the import hook stores that data). Should people be able to add arbitrary files to .dist-info directories and be confident that standards compliant tools will preserve them correctly through the deployment process? Absolutely. Will the standard library ever meaningfully interpret anything other than WHEEL and the four files already named in PEP 376? Most likely not. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
On Mon, Feb 25, 2013 at 9:54 AM, Nick Coghlan ncogh...@gmail.com wrote: On Tue, Feb 26, 2013 at 12:38 AM, Daniel Holth dho...@gmail.com wrote: I'm probably the only one but I'm not a fan of JSON with all the extra marks compared to the venerable, lovely, flatter and much easier to edit Key: value format. I don't really care that much about human readability of the raw metadata files, I care a lot more about ease of interoperability for automated tools, and ease of display for generic tools that may not understand the semantics. I also care about the standard format being amenable to the publication of metadata through TUF's JSON based infrastructure. When I see that file I've always thought why aren't we writing these by hand instead of through setup.py and then validating that they are correct with a tool? Bento's file sortof follows this logic. I can accept JSON. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
On Tue, Feb 26, 2013 at 12:59 AM, Daniel Holth dho...@gmail.com wrote: On Mon, Feb 25, 2013 at 9:54 AM, Nick Coghlan ncogh...@gmail.com wrote: On Tue, Feb 26, 2013 at 12:38 AM, Daniel Holth dho...@gmail.com wrote: I'm probably the only one but I'm not a fan of JSON with all the extra marks compared to the venerable, lovely, flatter and much easier to edit Key: value format. I don't really care that much about human readability of the raw metadata files, I care a lot more about ease of interoperability for automated tools, and ease of display for generic tools that may not understand the semantics. I also care about the standard format being amenable to the publication of metadata through TUF's JSON based infrastructure. When I see that file I've always thought why aren't we writing these by hand instead of through setup.py and then validating that they are correct with a tool? Bento's file sortof follows this logic. I can accept JSON. That's a similar question to the one the distutils2 folks asked before coming up with setup.cfg. The reason I think we need to assume a generated file rather than a hand-edited one is because, for the vast majority of Python users, for the foreseeable future, their static metadata is going to be generated by running a setup.py file. For most others, it's going to be generated from either setup.cfg or from bento.info. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
On Tue, Feb 26, 2013 at 1:29 AM, Daniel Holth dho...@gmail.com wrote: Well this is a rabbit hole. setup.cfg is what you get when the metadata devolves into the arguments passed to setup(). Perhaps that is the real reason I don't like JSON that much; the temptation would be to make it nothing more than the argument dict to setup() and that ties us to distutils. Sure, it's a temptation, but that doesn't make it a good idea. Note that the format I'm suggesting for Entry-Points is *inspired* by the way you specify entry points when calling setuptools, but it's true structured data rather than embedding the foo = entry-point-def mini language that setuptools uses. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
Nick Coghlan ncoghlan at gmail.com writes:
The other area where I think such an embedded JSON approach could
work is coming up with a clean format for an Entry-Points field.
Specifically, I am thinking of proposing the setuptools.setup
inspired:
Entry-Points: {
console_scripts: {
foo: my_package.some_module:main_func,
bar: other_module:some_func
},
gui_scripts: {
baz: my_package_gui.start_func
}
}
What is the reason for this half-way-house, rather than going straight to JSON?
I suppose one could argue backward compatibility, but since we are moving quite
a distance from a plain old key-value here, we may as well embrace JSON
properly.
Perhaps we should embrace the dict rather than any specific file format, as I
did with dictConfig in logging.
Note that distlib already supports entry points, though these are called
exports because they may or may not be callable code. (The name exports
was suggested by PJE and, IMO, is better/more general than entry points).
For example, for Babel-0.9.6:
exports: {
babel.extractors: [
ignore = babel.messages.extract:extract_nothing,
python = babel.messages.extract:extract_python,
javascript = babel.messages.extract:extract_javascript
],
distutils.commands: [
compile_catalog = babel.messages.frontend:compile_catalog,
extract_messages = babel.messages.frontend:extract_messages,
init_catalog = babel.messages.frontend:init_catalog,
update_catalog = babel.messages.frontend:update_catalog
],
babel.checkers: [
num_plurals = babel.messages.checkers:num_plurals,
python_format = babel.messages.checkers:python_format
],
distutils.setup_keywords: [
message_extractors = babel.messages.frontend:check_message_extractors
],
scripts: {
console: [
pybabel = babel.messages.frontend:main
]
}
}
This is currently used, for example, in distlib's script generation logic (for
example, to generate the pybabel script for Babel).
Regards,
Vinay Sajip
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
I can accept a rename but there is no way to avoid having 2 names not 1 new
name for the feature.
We go halfway now. The next version can go any other way.
On Feb 25, 2013 12:23 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
Nick Coghlan ncoghlan at gmail.com writes:
The other area where I think such an embedded JSON approach could
work is coming up with a clean format for an Entry-Points field.
Specifically, I am thinking of proposing the setuptools.setup
inspired:
Entry-Points: {
console_scripts: {
foo: my_package.some_module:main_func,
bar: other_module:some_func
},
gui_scripts: {
baz: my_package_gui.start_func
}
}
What is the reason for this half-way-house, rather than going straight to
JSON?
I suppose one could argue backward compatibility, but since we are moving
quite
a distance from a plain old key-value here, we may as well embrace JSON
properly.
Perhaps we should embrace the dict rather than any specific file format,
as I
did with dictConfig in logging.
Note that distlib already supports entry points, though these are called
exports because they may or may not be callable code. (The name exports
was suggested by PJE and, IMO, is better/more general than entry points).
For example, for Babel-0.9.6:
exports: {
babel.extractors: [
ignore = babel.messages.extract:extract_nothing,
python = babel.messages.extract:extract_python,
javascript = babel.messages.extract:extract_javascript
],
distutils.commands: [
compile_catalog = babel.messages.frontend:compile_catalog,
extract_messages = babel.messages.frontend:extract_messages,
init_catalog = babel.messages.frontend:init_catalog,
update_catalog = babel.messages.frontend:update_catalog
],
babel.checkers: [
num_plurals = babel.messages.checkers:num_plurals,
python_format = babel.messages.checkers:python_format
],
distutils.setup_keywords: [
message_extractors =
babel.messages.frontend:check_message_extractors
],
scripts: {
console: [
pybabel = babel.messages.frontend:main
]
}
}
This is currently used, for example, in distlib's script generation logic
(for
example, to generate the pybabel script for Babel).
Regards,
Vinay Sajip
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
Daniel Holth dholth at gmail.com writes: I can accept a rename but there is no way to avoid having 2 names not 1 new name for the feature. We go halfway now. The next version can go any other way. Just to be clear, the naming of exports vs. entry points was not the main thrust of my point - I mentioned it purely by way of explaining that part of the JSON snippet I posted. What I meant to say is that that parsing or writing a JSON file called entry_points.txt (or whatever) is just as easy as parsing or writing an ini-format file. However, the benefit of JSON is clear when composing a larger set of data from separate files (see the separate thread about metadata caching). One can more easily compose the data in entry_points.txt, METADATA, requires-dist.txt and anything else if they are just JSON files. That is what I do in my JSON metadata format - the dependency information is extracted from the metadata for a release into an aggregated form which is easier to use when doing dependency resolution. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
Daniel Holth dholth at gmail.com writes: We all must realize that incremental improvements are not harmful. Delay is harmful; there has been no obvious way to make a Python package this decade based on the idea that something better might be just around the corner or that the current way will be deprecated. The goal of this version of the PEP is to better represent important setuptools metadata statically while imposing as near to zero cost as possible on the actual setuptools users. They will not welcome an unproven alternative that requires work on their part. Instead, we change things a little bit, support setuptools / distribute OR something else for 5 years, until something else is obviously, compellingly better. I'm not sure what work we're asking setuptools *users* to do: IIUC, the setuptools files in .egg-info and the corresponding ones in .dist-info are not edited by hand by users. As far as I can see, having JSON-format files will not adversely impact the workload for anyone, compared with the suggestion that Nick made (embedding JSON in specific parts of a key-value format) and which I was responding to. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
All I'm trying to say is do not add anything else to pep 426. There will be other versions. This version can be consumed by distutils as of last July. Daniel Holth dholth at gmail.com writes: We all must realize that incremental improvements are not harmful. Delay is harmful; there has been no obvious way to make a Python package this decade based on the idea that something better might be just around the corner or that the current way will be deprecated. The goal of this version of the PEP is to better represent important setuptools metadata statically while imposing as near to zero cost as possible on the actual setuptools users. They will not welcome an unproven alternative that requires work on their part. Instead, we change things a little bit, support setuptools / distribute OR something else for 5 years, until something else is obviously, compellingly better. I'm not sure what work we're asking setuptools *users* to do: IIUC, the setuptools files in .egg-info and the corresponding ones in .dist-info are not edited by hand by users. As far as I can see, having JSON-format files will not adversely impact the workload for anyone, compared with the suggestion that Nick made (embedding JSON in specific parts of a key-value format) and which I was responding to. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 426: proposed change to extension fields + entry points
by which I mean distribute On Feb 25, 2013 5:55 PM, Daniel Holth dho...@gmail.com wrote: All I'm trying to say is do not add anything else to pep 426. There will be other versions. This version can be consumed by distutils as of last July. Daniel Holth dholth at gmail.com writes: We all must realize that incremental improvements are not harmful. Delay is harmful; there has been no obvious way to make a Python package this decade based on the idea that something better might be just around the corner or that the current way will be deprecated. The goal of this version of the PEP is to better represent important setuptools metadata statically while imposing as near to zero cost as possible on the actual setuptools users. They will not welcome an unproven alternative that requires work on their part. Instead, we change things a little bit, support setuptools / distribute OR something else for 5 years, until something else is obviously, compellingly better. I'm not sure what work we're asking setuptools *users* to do: IIUC, the setuptools files in .egg-info and the corresponding ones in .dist-info are not edited by hand by users. As far as I can see, having JSON-format files will not adversely impact the workload for anyone, compared with the suggestion that Nick made (embedding JSON in specific parts of a key-value format) and which I was responding to. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
