[gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

Hi all,

The way how we currently specify the EAPI in ebuilds has some
problems. For example, there is no sane way to allow usage of features
of a new bash version in a new EAPI. So we are currently stuck with
bash 3.2. Also changes of global scope behaviour, like addition of new
global scope functions (similar to "inherit") are not possible.

These flaws are outlined in GLEP 55 [1]:
| In order to get the EAPI the package manager needs to source the
| ebuild, which itself needs the EAPI in the first place. Otherwise it
| imposes a serious limitation, namely every ebuild, using any of the
| future EAPIs, will have to be source'able by old package managers
| [...]

The council has voted down GLEP 55 more than a year ago, but at the
same time requested that a solution for the mentioned issues should be
found. [2] However, there was no progress since then.

The issue arose again in bug 402167 [3] where several solutions have
been discussed. Below, I try to summarise the possible options
resulting from that discussion.


*** Proposal 1: "Parse the EAPI assignment statement" ***

This first proposal would require that the syntax of the EAPI
assignment statement in ebuilds matches a well defined regular
expression. A scan of the Portage tree shows that the statement only
occurs in the following variations (using EAPI 4 as example):

   EAPI=4
   EAPI="4"
   EAPI='4'

Sometimes this is followed by whitespace or a comment (starting with
a # sign). Also, with very few exceptions the EAPI assignment occurs
within the first few lines of the ebuild. For the vast majority of
ebuilds it is in line 5.

Written in a more formal way, appropriate for a specification:
- Ebuilds must contain at most one EAPI assignment statement.
- It must occur within the first N lines of the ebuild (N=10 and N=30
  have been suggested).
- The statement must match the following regular expression (extended
  regexp syntax):
  ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$

Note: The first and the third point are already fulfilled by all
ebuilds in the Portage tree. The second point will require very few
ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).

The package manager would determine the EAPI by parsing the assignment
with above regular expression. A sanity check would be added. Citing
Zac Medico in [3]: "The fact that we can compare the probed EAPI to
the actual EAPI variable after the ebuild is sourced seems like a
perfect sanity check. We could easily detect inconsistencies and flag
such ebuilds as invalid, providing a reliable feedback mechanism to
ebuild developers."

This proposal comes in two variants:
1a) The change is applied retroactively for all EAPIs.
1b) It is only applied for EAPI 5 and later (which means that the
result of the EAPI parsing would be discarded for earlier EAPIs).


*** Proposal 2: "EAPI in header comment" ***

A different approach would be to specify the EAPI in a specially
formatted comment in the ebuild's header. No syntax has been suggested
yet, but I believe that the following would work as a specification:
- The EAPI must be declared in a special comment in the first line of
  the ebuild's header, as follows:
- The first line of the ebuild must contain the word "ebuild",
  followed by whitespace, followed by the EAPI, followed by
  end-of-line or whitespace.

Again, the proposal comes in two variants:
2a) It is combined with a one time change of the file extension, like
.ebuild -> .eb.
2b) The usual EAPI assignment statement in the ebuild is still
required, at least for a transition period.

In the 2a case, the EAPI variable could be made read-only in bash
before sourcing the ebuild. In the 2b case, a sanity check similar to
the one mentioned above would be added.


What do you think?

(I really hope for a constructive discussion here. So, if you want
to comment that all of the above proposals suck and GLEP 55 is much
superior, then please open a new thread for it.)

Ulrich


[1] 
[2] 
[3] 

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller  wrote:
> (I really hope for a constructive discussion here. So, if you want
> to comment that all of the above proposals suck and GLEP 55 is much
> superior, then please open a new thread for it.)

I think if you want a constructive discussion, then you should ignore
the previous Council's decision and reconsider all the solutions that
have been presented.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexis Ballier]

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller  wrote:

> Hi all,
> 
> The way how we currently specify the EAPI in ebuilds has some
> problems. For example, there is no sane way to allow usage of features
> of a new bash version in a new EAPI. So we are currently stuck with
> bash 3.2. Also changes of global scope behaviour, like addition of new
> global scope functions (similar to "inherit") are not possible.
> 
> These flaws are outlined in GLEP 55 [1]:
> | In order to get the EAPI the package manager needs to source the
> | ebuild, which itself needs the EAPI in the first place. Otherwise it
> | imposes a serious limitation, namely every ebuild, using any of the
> | future EAPIs, will have to be source'able by old package managers
> | [...]
> 
> The council has voted down GLEP 55 more than a year ago, but at the
> same time requested that a solution for the mentioned issues should be
> found. [2] However, there was no progress since then.
> 
> The issue arose again in bug 402167 [3] where several solutions have
> been discussed. Below, I try to summarise the possible options
> resulting from that discussion.
> 
> 
> *** Proposal 1: "Parse the EAPI assignment statement" ***
> 
> This first proposal would require that the syntax of the EAPI
> assignment statement in ebuilds matches a well defined regular
> expression. A scan of the Portage tree shows that the statement only
> occurs in the following variations (using EAPI 4 as example):
> 
>EAPI=4
>EAPI="4"
>EAPI='4'
> 
> Sometimes this is followed by whitespace or a comment (starting with
> a # sign). Also, with very few exceptions the EAPI assignment occurs
> within the first few lines of the ebuild. For the vast majority of
> ebuilds it is in line 5.
> 
> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
>   have been suggested).
> - The statement must match the following regular expression (extended
>   regexp syntax):
>   ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
> 
> Note: The first and the third point are already fulfilled by all
> ebuilds in the Portage tree. The second point will require very few
> ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
> 
> The package manager would determine the EAPI by parsing the assignment
> with above regular expression. A sanity check would be added. Citing
> Zac Medico in [3]: "The fact that we can compare the probed EAPI to
> the actual EAPI variable after the ebuild is sourced seems like a
> perfect sanity check. We could easily detect inconsistencies and flag
> such ebuilds as invalid, providing a reliable feedback mechanism to
> ebuild developers."
> 
> This proposal comes in two variants:
> 1a) The change is applied retroactively for all EAPIs.
> 1b) It is only applied for EAPI 5 and later (which means that the
> result of the EAPI parsing would be discarded for earlier EAPIs).

+1 for the whole proposal, this seems to be solving the problem and is
the less intrusive change

As i understand it, $PM will need to try the regexp tingy on any ebuild
anyway, guess the EAPI then source the ebuild with the right sourcer
to get the real EAPI and compare it. So, for me, the proposal is
retroactive. The check whether guessed and actual EAPI are the same can
be made a warning, or just ignored, in EAPI<5 and fatal in >=5 though.

A.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[David Leverton]

On 7 March 2012 21:07, Alexis Ballier  wrote:
> As i understand it, $PM will need to try the regexp tingy on any ebuild
> anyway, guess the EAPI then source the ebuild with the right sourcer
> to get the real EAPI and compare it.

Not exactly... the idea with proposal 2) is that the header comment
becomes the One True Way to set the EAPI, so it wouldn't be "guessing"
as such, and ebuilds wouldn't be allowed to change it during sourcing
any more than they can redefine PV or the like.  As mentioned, 2b)
still risks a mismatch between the header and the assignment, but
hopefully that would be temporary and the assignments could be dropped
eventually.  (And I'd suggest that the header EAPI is still considered
authoritative if present - the mismatch check should probably be a
hard error, or if not it should generate a warning and use the header
value.  There should be minimal if any risk of this changing behaviour
for any existing ebuild, since existing ebuilds almost certainly won't
match the chosen header syntax.)

As for my opinion, I would prefer 2a), or 2b) as a second choice.
IMHO it's much simpler and cleaner to come up with a strict,
well-defined header format than to try and work within (an arbitrarily
restricted version of) bash syntax.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/07/2012 03:41 PM, Ulrich Mueller wrote:


*** Proposal 2: "EAPI in header comment" ***

A different approach would be to specify the EAPI in a specially
formatted comment in the ebuild's header. No syntax has been suggested
yet, but I believe that the following would work as a specification:
- The EAPI must be declared in a special comment in the first line of
   the ebuild's header, as follows:
- The first line of the ebuild must contain the word "ebuild",
   followed by whitespace, followed by the EAPI, followed by
   end-of-line or whitespace.



Someone suggested using a standard shebang the last time this came up, 
and if I remember correctly it was one of the least-disagreeable 
solutions proposed. We could of course define our own custom format, but 
I think something like,


  #!/usr/bin/eapi5

would be perfect if we could hand off the interpretation of the ebuild 
to that program. That solves the problem with new bash features, too, 
since you could point that command at a specific version.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexandre Rostovtsev]

On Wed, 2012-03-07 at 21:41 +0100, Ulrich Mueller wrote:
> .ebuild -> .eb

FYI, any Russian speaker is *guaranteed* to read the name ".eb" as a
very common obscenity.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Wed, 07 Mar 2012, Michael Orlitzky wrote:

> Someone suggested using a standard shebang the last time this came
> up, and if I remember correctly it was one of the least-disagreeable
> solutions proposed. We could of course define our own custom format,
> but I think something like,

>#!/usr/bin/eapi5

> would be perfect if we could hand off the interpretation of the
> ebuild to that program. That solves the problem with new bash
> features, too, since you could point that command at a specific
> version.

Please note that I've formulated the specification in a way that
accounts for this:

>> - The first line of the ebuild must contain the word "ebuild",
>>   followed by whitespace, followed by the EAPI, followed by
>>   end-of-line or whitespace.

This would also match the line "#!/usr/bin/ebuild 5" (if the magic
word should be "ebuild" or "eapi" can be discussed of course).
So even if we would introduce the shebang only in a later EAPI,
previous portage versions would still recognise the new header.

Ulrich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Jeroen Roovers]

On Wed, 07 Mar 2012 17:36:05 -0500
Alexandre Rostovtsev  wrote:

> FYI, any Russian speaker is *guaranteed* to read the name ".eb" as a
> very common obscenity.

In Dutch it means the low tide, and as a verb, it means "becoming low"
or "decreasing" as in the tide or some other fluid.

In English it means something very similar as in the Dutch verb, only
without the tidal reference. Other Germanic languages will probably
have their own variants.

In Crimean Tatar it apparently[1] references "easiness" or "simplicity",
in Hungarian it's the word for "dog", while in Nauruan it means
"country."

But that's not important right now (see also [2]).

The important thing to remember is, the bike shed protects our bicycles
from the elements.


 jer


[1] http://en.wiktionary.org/wiki/eb
[2] http://en.wikipedia.org/wiki/bytesex

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alec Warner]

On Wed, Mar 7, 2012 at 12:41 PM, Ulrich Mueller  wrote:
> Hi all,
>
> The way how we currently specify the EAPI in ebuilds has some
> problems. For example, there is no sane way to allow usage of features
> of a new bash version in a new EAPI. So we are currently stuck with
> bash 3.2. Also changes of global scope behaviour, like addition of new
> global scope functions (similar to "inherit") are not possible.
>
> These flaws are outlined in GLEP 55 [1]:
> | In order to get the EAPI the package manager needs to source the
> | ebuild, which itself needs the EAPI in the first place. Otherwise it
> | imposes a serious limitation, namely every ebuild, using any of the
> | future EAPIs, will have to be source'able by old package managers
> | [...]
>
> The council has voted down GLEP 55 more than a year ago, but at the
> same time requested that a solution for the mentioned issues should be
> found. [2] However, there was no progress since then.
>
> The issue arose again in bug 402167 [3] where several solutions have
> been discussed. Below, I try to summarise the possible options
> resulting from that discussion.
>
>
> *** Proposal 1: "Parse the EAPI assignment statement" ***
>
> This first proposal would require that the syntax of the EAPI
> assignment statement in ebuilds matches a well defined regular
> expression. A scan of the Portage tree shows that the statement only
> occurs in the following variations (using EAPI 4 as example):
>
>   EAPI=4
>   EAPI="4"
>   EAPI='4'
>
> Sometimes this is followed by whitespace or a comment (starting with
> a # sign). Also, with very few exceptions the EAPI assignment occurs
> within the first few lines of the ebuild. For the vast majority of
> ebuilds it is in line 5.
>
> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
>  have been suggested).
> - The statement must match the following regular expression (extended
>  regexp syntax):
>  ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
>
> Note: The first and the third point are already fulfilled by all
> ebuilds in the Portage tree. The second point will require very few
> ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
>
> The package manager would determine the EAPI by parsing the assignment
> with above regular expression. A sanity check would be added. Citing
> Zac Medico in [3]: "The fact that we can compare the probed EAPI to
> the actual EAPI variable after the ebuild is sourced seems like a
> perfect sanity check. We could easily detect inconsistencies and flag
> such ebuilds as invalid, providing a reliable feedback mechanism to
> ebuild developers."
>
> This proposal comes in two variants:
> 1a) The change is applied retroactively for all EAPIs.
> 1b) It is only applied for EAPI 5 and later (which means that the
>    result of the EAPI parsing would be discarded for earlier EAPIs).

I don't like this idea because the sane way should be easy and straightforward.
Mixing a constant declaration with bash assignment just confuses users
who think the assignment is full bash when in fact it is not.

EAPI=$(somefunc)
EAPI=${SOMEVAR%%-*}
and so forth all don't meet the regex (and would be flagged invalid.)
However a naive author might think they work.

I don't think any naive author would think either would work in a comment

# EAPI=$(somefunc)
# EAPI=${SOEMVAR%%-*}

Bash doesn't parse comments, so confusion is unlikely.

>
>
> *** Proposal 2: "EAPI in header comment" ***
>
> A different approach would be to specify the EAPI in a specially
> formatted comment in the ebuild's header. No syntax has been suggested
> yet, but I believe that the following would work as a specification:
> - The EAPI must be declared in a special comment in the first line of
>  the ebuild's header, as follows:
> - The first line of the ebuild must contain the word "ebuild",
>  followed by whitespace, followed by the EAPI, followed by
>  end-of-line or whitespace.
>
> Again, the proposal comes in two variants:
> 2a) It is combined with a one time change of the file extension, like
>    .ebuild -> .eb.
> 2b) The usual EAPI assignment statement in the ebuild is still
>    required, at least for a transition period.
>
> In the 2a case, the EAPI variable could be made read-only in bash
> before sourcing the ebuild. In the 2b case, a sanity check similar to
> the one mentioned above would be added.
>
>
> What do you think?

Overloading is bad.

There is no real difference between:
#!/usr/bin/ebuild --eapi 5
# EAPI=5

The problem is that the former is also the way to specify how to
execute the ebuild; so unless you plan to make ebuilds executable and
having /usr/bin/ebuild provide the ebuild environment, using that
syntax is confusing to users.

>
> (I really hope for a constructive discussion here. So, if you want
> to comment that all of the above proposals suck and GLEP 55 

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Wed, 7 Mar 2012, Alec Warner wrote:

>> *** Proposal 1: "Parse the EAPI assignment statement" ***
>> [...]

> I don't like this idea because the sane way should be easy and
> straightforward. Mixing a constant declaration with bash assignment
> just confuses users who think the assignment is full bash when in
> fact it is not.

> EAPI=$(somefunc)
> EAPI=${SOMEVAR%%-*}
> and so forth all don't meet the regex (and would be flagged
> invalid.) However a naive author might think they work.

Such constructs also cannot be used with any of the other proposed
solutions. And in fact, nobody is using such things in practice.
_All_ ebuilds in the Portage tree can be successfully parsed with the
regexp proposed.

The obvious sanity check, i.e. comparing the EAPI obtained by parsing
and by sourcing, could be added to repoman, which would prevent such
non-conforming ebuilds from being committed to the tree.

>> *** Proposal 2: "EAPI in header comment" ***
>> [...]

> Overloading is bad.

> There is no real difference between:
> #!/usr/bin/ebuild --eapi 5
> # EAPI=5

> The problem is that the former is also the way to specify how to
> execute the ebuild; so unless you plan to make ebuilds executable and
> having /usr/bin/ebuild provide the ebuild environment, using that
> syntax is confusing to users.

I agree with this point.

Many file formats are identifying themselves with a magic token (as
it is used by sys-apps/file), but it is not necessarily a shebang.

Ulrich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alec Warner]

On Wed, Mar 7, 2012 at 11:27 PM, Ulrich Mueller  wrote:
>> On Wed, 7 Mar 2012, Alec Warner wrote:
>
>>> *** Proposal 1: "Parse the EAPI assignment statement" ***
>>> [...]
>
>> I don't like this idea because the sane way should be easy and
>> straightforward. Mixing a constant declaration with bash assignment
>> just confuses users who think the assignment is full bash when in
>> fact it is not.
>
>> EAPI=$(somefunc)
>> EAPI=${SOMEVAR%%-*}
>> and so forth all don't meet the regex (and would be flagged
>> invalid.) However a naive author might think they work.
>
> Such constructs also cannot be used with any of the other proposed
> solutions. And in fact, nobody is using such things in practice.
> _All_ ebuilds in the Portage tree can be successfully parsed with the
> regexp proposed.

I'm not saying they are valid EAPI syntax; but they are all valid
bash. I tend to assume all authors are as...ignorant as myself. Lets
not give them the rope to hang themselves.

>
> The obvious sanity check, i.e. comparing the EAPI obtained by parsing
> and by sourcing, could be added to repoman, which would prevent such
> non-conforming ebuilds from being committed to the tree.
>
>>> *** Proposal 2: "EAPI in header comment" ***
>>> [...]
>
>> Overloading is bad.
>
>> There is no real difference between:
>> #!/usr/bin/ebuild --eapi 5
>> # EAPI=5
>
>> The problem is that the former is also the way to specify how to
>> execute the ebuild; so unless you plan to make ebuilds executable and
>> having /usr/bin/ebuild provide the ebuild environment, using that
>> syntax is confusing to users.
>
> I agree with this point.
>
> Many file formats are identifying themselves with a magic token (as
> it is used by sys-apps/file), but it is not necessarily a shebang.
>
> Ulrich
>

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Marc Schiffbauer]

* Ulrich Mueller schrieb am 08.03.12 um 08:27 Uhr:
> > On Wed, 7 Mar 2012, Alec Warner wrote:
> 
> >> *** Proposal 1: "Parse the EAPI assignment statement" ***
> >> [...]
> 
> > I don't like this idea because the sane way should be easy and
> > straightforward. Mixing a constant declaration with bash assignment
> > just confuses users who think the assignment is full bash when in
> > fact it is not.
> 
> > EAPI=$(somefunc)
> > EAPI=${SOMEVAR%%-*}
> > and so forth all don't meet the regex (and would be flagged
> > invalid.) However a naive author might think they work.
> 
> Such constructs also cannot be used with any of the other proposed
> solutions. And in fact, nobody is using such things in practice.
> _All_ ebuilds in the Portage tree can be successfully parsed with the
> regexp proposed.

Ebuilds are bash scripts. I think introducing exceptions or
constraints here is not straightforward.

I think the only relevant part whether EAPI is set correctly or not
should be the outcome of $EAPI.

I would vote for a solution in a bash comment where repoman would
have to check for its existance and equality to $EAPI.

-Marc
-- 
8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134


pgpiOirLF4AWG.pgp
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Wed, 07 Mar 2012 17:14:13 -0500
Michael Orlitzky  wrote:

> On 03/07/2012 03:41 PM, Ulrich Mueller wrote:
> >
> > *** Proposal 2: "EAPI in header comment" ***
> >
> > A different approach would be to specify the EAPI in a specially
> > formatted comment in the ebuild's header. No syntax has been
> > suggested yet, but I believe that the following would work as a
> > specification:
> > - The EAPI must be declared in a special comment in the first line
> > of the ebuild's header, as follows:
> > - The first line of the ebuild must contain the word "ebuild",
> >followed by whitespace, followed by the EAPI, followed by
> >end-of-line or whitespace.
> >
> 
> Someone suggested using a standard shebang the last time this came
> up, and if I remember correctly it was one of the least-disagreeable 
> solutions proposed. We could of course define our own custom format,
> but I think something like,
> 
>#!/usr/bin/eapi5
> 
> would be perfect if we could hand off the interpretation of the
> ebuild to that program. That solves the problem with new bash
> features, too, since you could point that command at a specific
> version.

And what would /usr/bin/eapi5 do? Are you suggesting misusing shebang
or making ebuilds PM-centric?

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Wed, 7 Mar 2012 20:12:25 -0800
Alec Warner  wrote:

> On Wed, Mar 7, 2012 at 12:41 PM, Ulrich Mueller 
> wrote:
> > Hi all,
> >
> > The way how we currently specify the EAPI in ebuilds has some
> > problems. For example, there is no sane way to allow usage of
> > features of a new bash version in a new EAPI. So we are currently
> > stuck with bash 3.2. Also changes of global scope behaviour, like
> > addition of new global scope functions (similar to "inherit") are
> > not possible.
> >
> > These flaws are outlined in GLEP 55 [1]:
> > | In order to get the EAPI the package manager needs to source the
> > | ebuild, which itself needs the EAPI in the first place. Otherwise
> > it | imposes a serious limitation, namely every ebuild, using any
> > of the | future EAPIs, will have to be source'able by old package
> > managers | [...]
> >
> > The council has voted down GLEP 55 more than a year ago, but at the
> > same time requested that a solution for the mentioned issues should
> > be found. [2] However, there was no progress since then.
> >
> > The issue arose again in bug 402167 [3] where several solutions have
> > been discussed. Below, I try to summarise the possible options
> > resulting from that discussion.
> >
> >
> > *** Proposal 1: "Parse the EAPI assignment statement" ***
> >
> > This first proposal would require that the syntax of the EAPI
> > assignment statement in ebuilds matches a well defined regular
> > expression. A scan of the Portage tree shows that the statement only
> > occurs in the following variations (using EAPI 4 as example):
> >
> >   EAPI=4
> >   EAPI="4"
> >   EAPI='4'
> >
> > Sometimes this is followed by whitespace or a comment (starting with
> > a # sign). Also, with very few exceptions the EAPI assignment occurs
> > within the first few lines of the ebuild. For the vast majority of
> > ebuilds it is in line 5.
> >
> > Written in a more formal way, appropriate for a specification:
> > - Ebuilds must contain at most one EAPI assignment statement.
> > - It must occur within the first N lines of the ebuild (N=10 and
> > N=30 have been suggested).
> > - The statement must match the following regular expression
> > (extended regexp syntax):
> >  ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
> >
> > Note: The first and the third point are already fulfilled by all
> > ebuilds in the Portage tree. The second point will require very few
> > ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
> >
> > The package manager would determine the EAPI by parsing the
> > assignment with above regular expression. A sanity check would be
> > added. Citing Zac Medico in [3]: "The fact that we can compare the
> > probed EAPI to the actual EAPI variable after the ebuild is sourced
> > seems like a perfect sanity check. We could easily detect
> > inconsistencies and flag such ebuilds as invalid, providing a
> > reliable feedback mechanism to ebuild developers."
> >
> > This proposal comes in two variants:
> > 1a) The change is applied retroactively for all EAPIs.
> > 1b) It is only applied for EAPI 5 and later (which means that the
> >    result of the EAPI parsing would be discarded for earlier EAPIs).
> 
> I don't like this idea because the sane way should be easy and
> straightforward. Mixing a constant declaration with bash assignment
> just confuses users who think the assignment is full bash when in
> fact it is not.
> 
> EAPI=$(somefunc)
> EAPI=${SOMEVAR%%-*}
> and so forth all don't meet the regex (and would be flagged invalid.)
> However a naive author might think they work.

And they all should be invalid due to our policies. The most important
ebuild variables like EAPI should be readable on sight, without having
to lookup random variables, functions etc.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 12:13 AM, Alec Warner wrote:
> On Wed, Mar 7, 2012 at 11:27 PM, Ulrich Mueller  wrote:
>> Such constructs also cannot be used with any of the other proposed
>> solutions. And in fact, nobody is using such things in practice.
>> _All_ ebuilds in the Portage tree can be successfully parsed with the
>> regexp proposed.
> 
> I'm not saying they are valid EAPI syntax; but they are all valid
> bash. I tend to assume all authors are as...ignorant as myself. Lets
> not give them the rope to hang themselves.

Something like DEPEND="foo bar" is also valid bash, and yet we don't
allow that either because "foo bar" does not contain valid dependency
atoms. Also, keep in mind that we're not allowing people to "hang
themselves" with invalid EAPI assignments. We'll simply give them a
reasonable error message so that they can quickly and easily correct
their mistake.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/08/2012 07:03 AM, Michał Górny wrote:


Someone suggested using a standard shebang the last time this came
up, and if I remember correctly it was one of the least-disagreeable
solutions proposed. We could of course define our own custom format,
but I think something like,

#!/usr/bin/eapi5

would be perfect if we could hand off the interpretation of the
ebuild to that program. That solves the problem with new bash
features, too, since you could point that command at a specific
version.


And what would /usr/bin/eapi5 do? Are you suggesting misusing shebang
or making ebuilds PM-centric?



I was saying that I'd prefer a more-standard use of the shebang (if 
possible), rather than defining our own header comment syntax. Either 
way I think the second option is cleaner than regular expressions.


Right now, we're guaranteed the features of bash-3.2. I guess we 
actually use whatever is executing ebuild.sh to source them. But we need 
to interpret the ebuild file with something: we might as well put *that* 
in the shebang, since that's what it's for.


So if we were to do this with an ebuild right now, we'd add,

  #!/usr/bin/eapi4

to the header, and instead of sourcing the ebuild with whatever 
ebuild.sh is using, we would run it with 'eapi4' and pass whatever we 
need back and forth. Or maybe ebuild.sh would reload itself using 
'eapi4'. If any of that makes sense, the PMS would just need to specify 
some requirements on the shebang command.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/07/2012 03:41 PM, Ulrich Mueller wrote:


*** Proposal 1: "Parse the EAPI assignment statement" ***



There's also libbash now:

  http://www.gentoo.org/proj/en/libbash/index.xml

Anyone know how close we are to being able to use it to parse the EAPI?

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[David Leverton]

On Mar 8, 2012 3:29 PM, "Zac Medico"  wrote:
> Something like DEPEND="foo bar" is also valid bash, and yet we don't
> allow that either because "foo bar" does not contain valid dependency
> atoms.

There's a bit of a difference between caring about the value of a
variable and caring about what syntax was used to assign it

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 08:11 AM, David Leverton wrote:

On Mar 8, 2012 3:29 PM, "Zac Medico"  wrote:

Something like DEPEND="foo bar" is also valid bash, and yet we don't
allow that either because "foo bar" does not contain valid dependency
atoms.


There's a bit of a difference between caring about the value of a
variable and caring about what syntax was used to assign it


Maybe that sort of distinction truly makes a difference to some people, 
but to me it just seems like hair-splitting [1].


[1] http://en.wikipedia.org/wiki/Trivial_objections
--
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 08:21:53 -0800
Zac Medico  wrote:
> Maybe that sort of distinction truly makes a difference to some
> people, but to me it just seems like hair-splitting [1].

So just to get this straight, you think that the following two
restrictions are effectively equivalent?

* The variable DEPEND's value must be set according to the following
  rules:

* The EAPI variable assignment must not use full bash syntax. Instead,
  it must be assigned according to the following rules:

And you believe that having exactly one place inside ebuild text where
there are different whitespace, quoting and indenting rules for
something that otherwise looks exactly like any other metadata variable
isn't going to cause problems?

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 01:42 AM, Marc Schiffbauer wrote:

* Ulrich Mueller schrieb am 08.03.12 um 08:27 Uhr:

Such constructs also cannot be used with any of the other proposed
solutions. And in fact, nobody is using such things in practice.
_All_ ebuilds in the Portage tree can be successfully parsed with the
regexp proposed.


Ebuilds are bash scripts. I think introducing exceptions or
constraints here is not straightforward.


Given that ebuilds already have to conform to a vast number of 
constraints that ordinary bash scripts do not. I think that it's 
perfectly reasonable for ebuilds to have a constrained syntax for EAPI 
assignments.

--
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 08:30:57 -0800
Zac Medico  wrote:
> On 03/08/2012 01:42 AM, Marc Schiffbauer wrote:
> > * Ulrich Mueller schrieb am 08.03.12 um 08:27 Uhr:
> >> Such constructs also cannot be used with any of the other proposed
> >> solutions. And in fact, nobody is using such things in practice.
> >> _All_ ebuilds in the Portage tree can be successfully parsed with
> >> the regexp proposed.
> >
> > Ebuilds are bash scripts. I think introducing exceptions or
> > constraints here is not straightforward.
> 
> Given that ebuilds already have to conform to a vast number of 
> constraints that ordinary bash scripts do not. I think that it's 
> perfectly reasonable for ebuilds to have a constrained syntax for
> EAPI assignments.

...and only EAPI assignments? Not for any other metadata variable?

Doesn't that sort of suggest that EAPI shouldn't be a metadata variable?

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Mike Gilbert]

On Wed, Mar 7, 2012 at 3:41 PM, Ulrich Mueller  wrote:
> Again, the proposal comes in two variants:
> 2a) It is combined with a one time change of the file extension, like
>    .ebuild -> .eb.
> 2b) The usual EAPI assignment statement in the ebuild is still
>    required, at least for a transition period.
>

Just throwing my opinion in:

I like proposal 2 better than proposal 1. Placing a regex-based
constraint on a bash variable assignment doesn't feel right to me.

I slightly prefer 2a over 2b because it feels cleaner. I'm sure it
will break some tools, but I don't have a good feel for the scope of
that breakage.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 08:29 AM, Ciaran McCreesh wrote:

On Thu, 08 Mar 2012 08:21:53 -0800
Zac Medico  wrote:

Maybe that sort of distinction truly makes a difference to some
people, but to me it just seems like hair-splitting [1].


So just to get this straight, you think that the following two
restrictions are effectively equivalent?

* The variable DEPEND's value must be set according to the following
   rules:

* The EAPI variable assignment must not use full bash syntax. Instead,
   it must be assigned according to the following rules:


Yeah, I think they're reasonably/roughly equivalent for how we use them 
in the current context.



And you believe that having exactly one place inside ebuild text where
there are different whitespace, quoting and indenting rules for
something that otherwise looks exactly like any other metadata variable
isn't going to cause problems?


Yes, with the stipulation that we implement a sanity check / feedback 
mechanism which guarantees that the probed EAPI is identical with the 
result obtained from bash [1].


[1] https://bugs.gentoo.org/show_bug.cgi?id=402167#c18
--
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Thu, 08 Mar 2012, Michael Orlitzky wrote:

> On 03/07/2012 03:41 PM, Ulrich Mueller wrote:
>> 
>> *** Proposal 1: "Parse the EAPI assignment statement" ***

> There's also libbash now:

>http://www.gentoo.org/proj/en/libbash/index.xml

Looks like complete overkill to me, considering the simple task at
hand.

Ulrich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexandre Rostovtsev]

On Thu, 2012-03-08 at 16:29 +, Ciaran McCreesh wrote:
> And you believe that having exactly one place inside ebuild text where
> there are different whitespace, quoting and indenting rules for
> something that otherwise looks exactly like any other metadata variable
> isn't going to cause problems?

In light of the fact that all 29758 ebuilds in portage already satisfy
the proposed whitespace, quoting, and indenting constrains on EAPI
assignment, the probability of problems appears to be vanishingly small.
And "vanishingly small" and can be reduced to zero by simply adding a
check to repoman.

Can you come up with a sane scenario under which an ebuild writer would
want to use a different syntax for setting EAPI?

-Alexandre

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Rich Freeman]

On Thu, Mar 8, 2012 at 11:51 AM, Ulrich Mueller  wrote:
>> On Thu, 08 Mar 2012, Michael Orlitzky wrote:
>
>> There's also libbash now:
>
> Looks like complete overkill to me, considering the simple task at
> hand.
>

Plus, wasn't the whole point that we can't guarantee that the bash
installed on a user's system can parse the ebuild until we've checked
the EAPI?  If we use libbash doesn't that just keep the same
constraint but on a different package?  Is the current stable libbash
guaranteed to be able to parse a bash v12 script?

If you just parse the file with a defined set of rules without regard
to how bash might parse the file, then you can determine the EAPI and
then decide how to source it.  For all we know EAPI G will turn
ebuilds into python scripts so trying to read the thing with bash or
libbash will be doomed to failure.

Rich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 11:59:33 -0500
Alexandre Rostovtsev  wrote:
> On Thu, 2012-03-08 at 16:29 +, Ciaran McCreesh wrote:
> > And you believe that having exactly one place inside ebuild text
> > where there are different whitespace, quoting and indenting rules
> > for something that otherwise looks exactly like any other metadata
> > variable isn't going to cause problems?
> 
> In light of the fact that all 29758 ebuilds in portage already satisfy
> the proposed whitespace, quoting, and indenting constrains on EAPI
> assignment, the probability of problems appears to be vanishingly
> small. And "vanishingly small" and can be reduced to zero by simply
> adding a check to repoman.

Because they were recently changed, presumably...

https://bugs.gentoo.org/show_bug.cgi?id=402167#c36

We had this discussion the last time around too, and people were told
to assign in a particular way. As you can see, it didn't work.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 08:35 AM, Ciaran McCreesh wrote:

On Thu, 08 Mar 2012 08:30:57 -0800
Zac Medico  wrote:

On 03/08/2012 01:42 AM, Marc Schiffbauer wrote:

* Ulrich Mueller schrieb am 08.03.12 um 08:27 Uhr:

Such constructs also cannot be used with any of the other proposed
solutions. And in fact, nobody is using such things in practice.
_All_ ebuilds in the Portage tree can be successfully parsed with
the regexp proposed.


Ebuilds are bash scripts. I think introducing exceptions or
constraints here is not straightforward.


Given that ebuilds already have to conform to a vast number of
constraints that ordinary bash scripts do not. I think that it's
perfectly reasonable for ebuilds to have a constrained syntax for
EAPI assignments.


...and only EAPI assignments? Not for any other metadata variable?


It's only needed for the EAPI, since that's the only value defined by 
the ebuild that we intend to use to control how the global environment 
is initialized prior to sourcing of the ebuild.



Doesn't that sort of suggest that EAPI shouldn't be a metadata variable?


It's a very special metadata variable. Of course, it could also be 
implemented in many different ways that do not involve bash variable 
assingments. Maybe the differences between the various possible ways 
truly make a difference to some people, but to me it's just 
hair-splitting [1].


[1] http://en.wikipedia.org/wiki/Trivial_objections
--
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 09:07:18 -0800
Zac Medico  wrote:
> It's a very special metadata variable. Of course, it could also be 
> implemented in many different ways that do not involve bash variable 
> assingments. Maybe the differences between the various possible ways 
> truly make a difference to some people, but to me it's just 
> hair-splitting [1].
> 
> [1] http://en.wikipedia.org/wiki/Trivial_objections

Having a different, special rule for something that looks exactly like
lots of other things that do not have that different, special rule is
hardly hair splitting. This rule would have to be documented and have
special code to carefully enforce it. That's a big deal.

Having something break because you add an unrelated comment to the top
of a file is weird.

Having something break because you indent it, where nothing else breaks
if you do the same thing, is weird.

Having something break because you make full use of bash syntax, where
nothing else breaks if you do the same thing, is weird.

There are already a whole pile of subtle traps for ebuild writers and
complications for people learning the system. We should be aiming to
reduce these, not add to them.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Thu, 08 Mar 2012 10:56:21 -0500
Michael Orlitzky  wrote:

> On 03/08/2012 07:03 AM, Michał Górny wrote:
> >>
> >> Someone suggested using a standard shebang the last time this came
> >> up, and if I remember correctly it was one of the
> >> least-disagreeable solutions proposed. We could of course define
> >> our own custom format, but I think something like,
> >>
> >> #!/usr/bin/eapi5
> >>
> >> would be perfect if we could hand off the interpretation of the
> >> ebuild to that program. That solves the problem with new bash
> >> features, too, since you could point that command at a specific
> >> version.
> >
> > And what would /usr/bin/eapi5 do? Are you suggesting misusing
> > shebang or making ebuilds PM-centric?
> >
> 
> I was saying that I'd prefer a more-standard use of the shebang (if 
> possible), rather than defining our own header comment syntax. Either 
> way I think the second option is cleaner than regular expressions.
> 
> Right now, we're guaranteed the features of bash-3.2. I guess we 
> actually use whatever is executing ebuild.sh to source them. But we
> need to interpret the ebuild file with something: we might as well
> put *that* in the shebang, since that's what it's for.
> 
> So if we were to do this with an ebuild right now, we'd add,
> 
>#!/usr/bin/eapi4
> 
> to the header, and instead of sourcing the ebuild with whatever 
> ebuild.sh is using, we would run it with 'eapi4' and pass whatever we 
> need back and forth. Or maybe ebuild.sh would reload itself using 
> 'eapi4'. If any of that makes sense, the PMS would just need to
> specify some requirements on the shebang command.

And something will need to provide that /usr/bin/eapi4 thing. And that
introduces new problems:

1) how are we going to support multiple package managers? will we need
to install eapi4 thing as a smart wrapper choosing the right PM?

2) what about Prefix? #!/usr/bin/env eapi4 then, or proactive updating
of shebangs in synced ebuilds? and then regenerating the whole cache
(guess how long does it take to update it),

3) what should happen if user executes ebuild? the ebuild should merge
itself? with dependencies or without?

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Jeroen Roovers]

On Thu, 8 Mar 2012 17:14:58 +
Ciaran McCreesh  wrote:

> Having a different, special rule for something that looks exactly like
> lots of other things that do not have that different, special rule is
> hardly hair splitting. This rule would have to be documented and have
> special code to carefully enforce it. That's a big deal.

Exactly like HOMEPAGE is a big deal?

HOMEPAGE Package's homepage. If you are unable to locate an
official one, try to provide a link to freshmeat.net or a similar
package tracking site. Never refer to a variable name in the string;
include only raw text.

http://devmanual.gentoo.org/ebuild-writing/variables/index.html


   jer

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 8 Mar 2012 18:30:47 +0100
Jeroen Roovers  wrote:
> On Thu, 8 Mar 2012 17:14:58 +
> Ciaran McCreesh  wrote:
> > Having a different, special rule for something that looks exactly
> > like lots of other things that do not have that different, special
> > rule is hardly hair splitting. This rule would have to be
> > documented and have special code to carefully enforce it. That's a
> > big deal.
> 
> Exactly like HOMEPAGE is a big deal?
> 
> HOMEPAGE   Package's homepage. If you are unable to locate an
> official one, try to provide a link to freshmeat.net or a similar
> package tracking site. Never refer to a variable name in the string;
> include only raw text.
> 
> http://devmanual.gentoo.org/ebuild-writing/variables/index.html

That's a QA thing, not a spec thing, and it's there because some
old-school developers don't know how to use their package manager
properly, and want to copy-paste a string from an ebuild into their
browser.

It's a silly rule. And if you take a count of how many ebuilds and
eclasses break that rule, you'll see exactly why such rules are a
problem.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/08/2012 12:28 PM, Michał Górny wrote:


And something will need to provide that /usr/bin/eapi4 thing. And that
introduces new problems:


I'm just parroting someone else's suggestion; I don't really know enough 
about the details to answer these properly. Not that that will stop me.




1) how are we going to support multiple package managers? will we need
to install eapi4 thing as a smart wrapper choosing the right PM?

2) what about Prefix? #!/usr/bin/env eapi4 then, or proactive updating
of shebangs in synced ebuilds? and then regenerating the whole cache
(guess how long does it take to update it),


Wouldn't #!/use/bin/env eapi4 handle both (1) and (2)? You might have to 
eselect package-manager or something first if you want to use two PMs at 
once.




3) what should happen if user executes ebuild? the ebuild should merge
itself? with dependencies or without?


If a user marks the ebuild executable and does ./foo-x.y.ebuild, the 
eapi4 wrapper can decide what to do. Nothing at all, print larry the 
cow, or crash (what we do now) are all fine with me =)

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller  wrote:

> *** Proposal 1: "Parse the EAPI assignment statement" ***
>
> [...]
> 
> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
>   have been suggested).
> - The statement must match the following regular expression (extended
>   regexp syntax):
>   ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$

I'd make the regexp less strict -- at least allow whitespace around '='.
If the intent is to not rely on a specific bash version for a global
scope, why should we limit it to the (current) bash syntax?

And it may be also a good idea to not rely on a specific line format,
so e.g. 'dnl EAPI=4' will work as well.

> 1b) It is only applied for EAPI 5 and later (which means that the
> result of the EAPI parsing would be discarded for earlier EAPIs).

Err... so what happens if 'new parsing' detects EAPI 4 and 'old
parsing' detects EAPI 5? Or more exactly, how does it know when
an older EAPI is used if it is supposed to not use the value it uses to
detect EAPI?

> *** Proposal 2: "EAPI in header comment" ***
> 
> A different approach would be to specify the EAPI in a specially
> formatted comment in the ebuild's header. No syntax has been suggested
> yet, but I believe that the following would work as a specification:
> - The EAPI must be declared in a special comment in the first line of
>   the ebuild's header, as follows:
> - The first line of the ebuild must contain the word "ebuild",
>   followed by whitespace, followed by the EAPI, followed by
>   end-of-line or whitespace.

What if we ever decide to use a language which would would have another
requirements for first line?

> Again, the proposal comes in two variants:
> 2a) It is combined with a one time change of the file extension, like
> .ebuild -> .eb.

And we're going to retroactively migrate the tree or have random file
suffixes intermixed? Not to mention we're either keeping two different
variants for a longer while, or disregarding backwards compatibility
with older package managers for no actual benefit.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 12:48:51 -0500
Michael Orlitzky  wrote:
> On 03/08/2012 12:28 PM, Michał Górny wrote:
> > And something will need to provide that /usr/bin/eapi4 thing. And
> > that introduces new problems:
> 
> I'm just parroting someone else's suggestion; I don't really know
> enough about the details to answer these properly. Not that that will
> stop me.

It probably should. Although in the early days the model for ebuilds
was that they were scripts that were "executed", nowadays there's so
much support required that it's better to think of ebuilds as being
data. If you did have a /usr/bin/eapi5, it would have to be implemented
as something that invoked the package manager, not as a direct
interpreter.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 8 Mar 2012 18:52:13 +0100
Michał Górny  wrote:
> > Again, the proposal comes in two variants:
> > 2a) It is combined with a one time change of the file extension,
> > like .ebuild -> .eb.
> 
> And we're going to retroactively migrate the tree or have random file
> suffixes intermixed? Not to mention we're either keeping two different
> variants for a longer while, or disregarding backwards compatibility
> with older package managers for no actual benefit.

No, you'd just switch extensions for EAPI 5 onwards, and keep the old
rules for EAPI 4. Same idea and impact as GLEP 55, except without as
much future proofing.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 09:52 AM, Michał Górny wrote:

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller  wrote:

1b) It is only applied for EAPI 5 and later (which means that the
 result of the EAPI parsing would be discarded for earlier EAPIs).


Err... so what happens if 'new parsing' detects EAPI 4 and 'old
parsing' detects EAPI 5? Or more exactly, how does it know when
an older EAPI is used if it is supposed to not use the value it uses to
detect EAPI?


We can simply detect this case and treat it as an error. The purpose of 
the "discarded for earlier EAPIs" part is to allow more variance for 
older EAPIs, and treating this case as an error will probably affect 
zero or a negligible number of older ebuilds in practice.

--
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/08/2012 12:53 PM, Ciaran McCreesh wrote:

On Thu, 08 Mar 2012 12:48:51 -0500
Michael Orlitzky  wrote:

On 03/08/2012 12:28 PM, Michał Górny wrote:

And something will need to provide that /usr/bin/eapi4 thing. And
that introduces new problems:


I'm just parroting someone else's suggestion; I don't really know
enough about the details to answer these properly. Not that that will
stop me.


It probably should. Although in the early days the model for ebuilds
was that they were scripts that were "executed", nowadays there's so
much support required that it's better to think of ebuilds as being
data. If you did have a /usr/bin/eapi5, it would have to be implemented
as something that invoked the package manager, not as a direct
interpreter.


Fair enough, but aren't you arguing the opposite point with Zac? If 
ebuilds are data, fine, we write EAPI=4 somewhere and be done with it. 
Anything not having that format is out-of-spec.


If they're code, they're code, and we need to execute them somehow. And 
the reason for the proposal in the first place was that the way we do it 
now ain't so great, eh?

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 08 Mar 2012 13:37:09 -0500
Michael Orlitzky  wrote:
> > It probably should. Although in the early days the model for ebuilds
> > was that they were scripts that were "executed", nowadays there's so
> > much support required that it's better to think of ebuilds as being
> > data. If you did have a /usr/bin/eapi5, it would have to be
> > implemented as something that invoked the package manager, not as a
> > direct interpreter.
> 
> Fair enough, but aren't you arguing the opposite point with Zac? If
> ebuilds are data, fine, we write EAPI=4 somewhere and be done with
> it. Anything not having that format is out-of-spec.

The problem is that right now there's no way to determine the format of
the data until you already know the format of the data. We hack around
this by not allowing "drastic" format changes, where "drastic" includes
"using things in newer versions of bash" and "not adding new global
scope commands".

The question under discussion is whether we a) keep "what format the
data is in" as being part of the data, but impose some strange and
arbitrary conditions on it, b) make a one-time change to have some kind
of 'header' inside the file describing its format that isn't really part
of the data itself, or c) admit that GLEP 55 already solved the problem
and we might as well just fix the issue properly once and for all, even
if GLEP 55's author is considered by some to be one of Satan's little
minions.

> If they're code, they're code, and we need to execute them somehow.

The notion of "execute them somehow" that's used doesn't fit in with
the #! interpreter model. You aren't executing ebuilds via an
interpreter. You're performing an action that involves using the data
and code in an ebuild multiple times and in multiple different ways,
and that may also involve doing the same to an installed package that
is being replaced.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Thu, 8 Mar 2012, Michał Górny wrote:

>> *** Proposal 1: "Parse the EAPI assignment statement" ***
>> 
>> [...]
>> 
>> Written in a more formal way, appropriate for a specification:
>> - Ebuilds must contain at most one EAPI assignment statement.
>> - It must occur within the first N lines of the ebuild (N=10 and N=30
>> have been suggested).
>> - The statement must match the following regular expression (extended
>> regexp syntax):
>> ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$

> I'd make the regexp less strict -- at least allow whitespace around '='.
> If the intent is to not rely on a specific bash version for a global
> scope, why should we limit it to the (current) bash syntax?

This could certainly be done ...

> And it may be also a good idea to not rely on a specific line format,
> so e.g. 'dnl EAPI=4' will work as well.

... but loosening it too much looks a little dangerous, because it
would probably match comments and could produce unintended matches.

If we allow for a general syntax, we should restrict it to the first
line of the ebuild. But that is proposal 2 then.

>> 1b) It is only applied for EAPI 5 and later (which means that the
>> result of the EAPI parsing would be discarded for earlier EAPIs).

> Err... so what happens if 'new parsing' detects EAPI 4 and 'old
> parsing' detects EAPI 5?

This cannot happen for a legal ebuild:
- If the ebuild is EAPI 4, then sourcing it ("old parsing") must
  detect EAPI 4.
- If the ebuild is EAPI 5, then "new parsing" must detect EAPI 5.

> Or more exactly, how does it know when an older EAPI is used if it
> is supposed to not use the value it uses to detect EAPI?

If new parsing detects EAPI 5 or later, then use this value (and do a
sanity check after the ebuild has been sourced). Otherwise, discard
the value determined by new parsing and detect the EAPI in the
traditional way by sourcing the ebuild.

Ulrich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Thu, 8 Mar 2012, Ciaran McCreesh wrote:

> On Thu, 08 Mar 2012 11:59:33 -0500
> Alexandre Rostovtsev  wrote:
>> In light of the fact that all 29758 ebuilds in portage already
>> satisfy the proposed whitespace, quoting, and indenting constrains
>> on EAPI assignment, the probability of problems appears to be
>> vanishingly small. And "vanishingly small" and can be reduced to
>> zero by simply adding a check to repoman.

> Because they were recently changed, presumably...

> https://bugs.gentoo.org/show_bug.cgi?id=402167#c36

> We had this discussion the last time around too, and people were
> told to assign in a particular way. As you can see, it didn't work.

Sorry, but this is nonsense (or rather FUD). Indeed we had 3 ebuilds
(0.01%) in the Portage tree where parsing resulted in an EAPI
different from the one in metadata.

In one of them, removal of the old assignment statement had simply
been forgotten [1]. For the other two, the EAPI had been assigned by
an eclass [2], which we consider illegal anyway.

Ulrich


[1] 

[2] 

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Thu, 8 Mar 2012 20:17:41 +0100
Ulrich Mueller  wrote:
> In one of them, removal of the old assignment statement had simply
> been forgotten [1]. For the other two, the EAPI had been assigned by
> an eclass [2], which we consider illegal anyway.

...and yet people do it. That and the violations of the HOMEPAGE rule
tell you a lot about what happens when something is made syntactically
valid but supposedly not legal.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexis Ballier]

On Thu, 8 Mar 2012 20:04:55 +0100
Ulrich Mueller  wrote:
> > Err... so what happens if 'new parsing' detects EAPI 4 and 'old
> > parsing' detects EAPI 5?
> 
> This cannot happen for a legal ebuild:
> - If the ebuild is EAPI 4, then sourcing it ("old parsing") must
>   detect EAPI 4.

the problem here may be when "old parsing" will wrongly parse a
malformed EAPI 42 assignment (so that new parsing doesnt detect an
EAPI and assumes its 0) as EAPI 4.
this is a purely hypothetical scenario that will be easily detected if
it ever happens, though :)

anyway, "new parsing" != "old parsing" should trigger a repoman
warning for old EAPIs.

A.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexis Ballier]

On Thu, 8 Mar 2012 19:31:16 +
Ciaran McCreesh  wrote:

> On Thu, 8 Mar 2012 20:17:41 +0100
> Ulrich Mueller  wrote:
> > In one of them, removal of the old assignment statement had simply
> > been forgotten [1]. For the other two, the EAPI had been assigned by
> > an eclass [2], which we consider illegal anyway.
> 
> ...and yet people do it. That and the violations of the HOMEPAGE rule
> tell you a lot about what happens when something is made syntactically
> valid but supposedly not legal.
> 

... and this is where repoman helps.
broken deps are syntactically valid but not legal either.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/08/2012 01:48 PM, Ciaran McCreesh wrote:



If they're code, they're code, and we need to execute them somehow.


The notion of "execute them somehow" that's used doesn't fit in with
the #! interpreter model. You aren't executing ebuilds via an
interpreter. You're performing an action that involves using the data
and code in an ebuild multiple times and in multiple different ways,
and that may also involve doing the same to an installed package that
is being replaced.



I do understand that; but the fact that the data are computed in an ugly 
turing-complete language complicates things.


Did someone already propose replacing EAPI=foo with a function call akin 
to inherit?


  eapi 4
  inherit whatever
  ...

the call to eapi() would then set $EAPI accordingly. If the ebuild is 
being executed directly, it could exit $EAPI; otherwise, it would 
continue normally. That would give us an interface to the variable, and 
we wouldn't need to know the EAPI ahead of time to do it as long as it's 
the first function called in the ebuild.


This is of course isomorphic to requiring a specific EAPI=4 format, but 
does allow you to do stupid things like x=`seq 4 4`; eapi $x; if you want.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alec Warner]

On Thu, Mar 8, 2012 at 1:35 PM, Michael Orlitzky  wrote:
> On 03/08/2012 01:48 PM, Ciaran McCreesh wrote:
>>
>>
>>> If they're code, they're code, and we need to execute them somehow.
>>
>>
>> The notion of "execute them somehow" that's used doesn't fit in with
>> the #! interpreter model. You aren't executing ebuilds via an
>> interpreter. You're performing an action that involves using the data
>> and code in an ebuild multiple times and in multiple different ways,
>> and that may also involve doing the same to an installed package that
>> is being replaced.
>>
>
> I do understand that; but the fact that the data are computed in an ugly
> turing-complete language complicates things.
>
> Did someone already propose replacing EAPI=foo with a function call akin to
> inherit?

the eapi function doesn't exist in earlier eapis, so you end up doing
the rename thing. Otherwise old package managers barf on the new
syntax. I still think going with a comment is better than this.

>
>  eapi 4
>  inherit whatever
>  ...
>
> the call to eapi() would then set $EAPI accordingly. If the ebuild is being
> executed directly, it could exit $EAPI; otherwise, it would continue
> normally. That would give us an interface to the variable, and we wouldn't
> need to know the EAPI ahead of time to do it as long as it's the first
> function called in the ebuild.
>
> This is of course isomorphic to requiring a specific EAPI=4 format, but does
> allow you to do stupid things like x=`seq 4 4`; eapi $x; if you want.
>

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Walter Dnes]

On Wed, Mar 07, 2012 at 09:41:02PM +0100, Ulrich Mueller wrote

> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
>   have been suggested).
> - The statement must match the following regular expression (extended
>   regexp syntax):
>   ^[ \t]*EAPI=(['"]?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
> 
> Note: The first and the third point are already fulfilled by all
> ebuilds in the Portage tree. The second point will require very few
> ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).

  The second point could be rendered moot with a little sanity checking
like so...

#!/bin/bash
counter=`grep -c "^EAPI=" ${1}`
if [ ${counter} -eq 0 ]; then
#  
   exit
elif [ ${counter} -gt 1 ]; then
   echo "***ERROR*** only 1 line allowed beginning with EAPI="
   exit
else
#  
   exit
fi

  This treats the EAPI statement as a declaration, and doesn't care
where it shows up in the file.  I'm assuming no leading blanks in front
of "EAPI".

-- 
Walter Dnes 

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Thu, 08 Mar 2012 16:35:14 -0500
Michael Orlitzky  wrote:

> On 03/08/2012 01:48 PM, Ciaran McCreesh wrote:
> >
> >> If they're code, they're code, and we need to execute them somehow.
> >
> > The notion of "execute them somehow" that's used doesn't fit in with
> > the #! interpreter model. You aren't executing ebuilds via an
> > interpreter. You're performing an action that involves using the
> > data and code in an ebuild multiple times and in multiple different
> > ways, and that may also involve doing the same to an installed
> > package that is being replaced.
> >
> 
> I do understand that; but the fact that the data are computed in an
> ugly turing-complete language complicates things.
> 
> Did someone already propose replacing EAPI=foo with a function call
> akin to inherit?
> 
>eapi 4
>inherit whatever
>...
> 
> the call to eapi() would then set $EAPI accordingly. If the ebuild is 
> being executed directly, it could exit $EAPI; otherwise, it would 
> continue normally. That would give us an interface to the variable,
> and we wouldn't need to know the EAPI ahead of time to do it as long
> as it's the first function called in the ebuild.
> 
> This is of course isomorphic to requiring a specific EAPI=4 format,
> but does allow you to do stupid things like x=`seq 4 4`; eapi $x; if
> you want.

What advantage does it give us? We still can't change ebuild syntax in
global scope because bash will barf.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/2012 12:04 AM, Michał Górny wrote:


This is of course isomorphic to requiring a specific EAPI=4 format,
but does allow you to do stupid things like x=`seq 4 4`; eapi $x; if
you want.


What advantage does it give us? We still can't change ebuild syntax in
global scope because bash will barf.



Not in EAPI=5, no, but once all PMs are using the eapi function we could.

The function can do any crazy thing you want. Right now, we need to 
source the entire ebuild to get at its environment. Before we source it 
for real, we just want to know the value of $EAPI. Since eapi() will be 
the first function called, it can be our interface to this variable. 
Here's a stupid but hopefully clear example:


  $ cat test.ebuild
  eapi 4
  HOMEPAGE="http://www.example.com/";
  echo "test.ebuild, current EAPI=${EAPI}"


  $ cat test-sourcer.sh
  #!/bin/bash

  function eapi() {
  if [ "$TELL_ME_YOUR_EAPI" == 1 ]; then
  exit $1
  fi
  export EAPI=$1
  }

  export TELL_ME_YOUR_EAPI=1
  `eval 'source test.ebuild'`
  echo "Found EAPI: $?"

  export TELL_ME_YOUR_EAPI=0
  source test.ebuild


This sources it once, which short-circuits at the eapi() call returning 
'4'. Then we source it a second time with EAPI=4, and see that it makes 
it past the call to eapi() where any incompatible features would be.


  $ ./test-sourcer.sh
  Found EAPI: 4
  test.ebuild, current EAPI=4

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/08/2012 09:35 PM, Michael Orlitzky wrote:
> The function can do any crazy thing you want.

We don't need a function. We need to know the EAPI before we source the
ebuild, and a function doesn't give us that.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Marc Schiffbauer]

* Zac Medico schrieb am 08.03.12 um 17:30 Uhr:
> On 03/08/2012 01:42 AM, Marc Schiffbauer wrote:
> > * Ulrich Mueller schrieb am 08.03.12 um 08:27 Uhr:
> >> Such constructs also cannot be used with any of the other proposed
> >> solutions. And in fact, nobody is using such things in practice.
> >> _All_ ebuilds in the Portage tree can be successfully parsed with the
> >> regexp proposed.
> >
> > Ebuilds are bash scripts. I think introducing exceptions or
> > constraints here is not straightforward.
> 
> Given that ebuilds already have to conform to a vast number of 
> constraints that ordinary bash scripts do not. I think that it's 
> perfectly reasonable for ebuilds to have a constrained syntax for EAPI 
> assignments.

There are constraints in ebuilds, right. But its an *ebuild* in the
end, not an ordinary shell script. Thats true.

So if EAPI is very special, and I am now convinced it is, then well, 
this might be the most important contraint in an ebuild at all.

If that is true I would vote to keep this as simple as possible:

* EAPI *must* *be* the first non-Argument / shell code in an ebuild
* The value of EAPI in the assignment *MUST* *NOT* contain any
  other variables or other shell substitutions.

Period. Done.

* That would be the least invasive change.
* Could easily be checked by repoman
* Is easy parsable by other programs (python code)

-Marc
-- 
8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134


pgpFVfsBDxzOc.pgp
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 00:51, Zac Medico wrote:
> On 03/08/2012 09:35 PM, Michael Orlitzky wrote:
>> The function can do any crazy thing you want.
> 
> We don't need a function. We need to know the EAPI before we source the
> ebuild, and a function doesn't give us that.

Surely we can source one or two lines of the ebuild safely, like the
example shows?

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 06:42 AM, Michael Orlitzky wrote:
> On 03/09/12 00:51, Zac Medico wrote:
>> On 03/08/2012 09:35 PM, Michael Orlitzky wrote:
>>> The function can do any crazy thing you want.
>>
>> We don't need a function. We need to know the EAPI before we source the
>> ebuild, and a function doesn't give us that.
> 
> Surely we can source one or two lines of the ebuild safely, like the
> example shows?

Why would we though, when sourcing is a relatively costly operation, and
there are much more efficient ways to get the EAPI?

Also, you've got a chicken and egg problem, since you're assuming that
your eapi() function is going to be available in the global environment
when the ebuild is sourced. The whole point of knowing the EAPI in
advance is so that we will be able to make changes to the global
environment, such as add, remove, or change the behavior of _any_ function.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 10:05, Zac Medico wrote:
>>
>> Surely we can source one or two lines of the ebuild safely, like the
>> example shows?
> 
> Why would we though, when sourcing is a relatively costly operation, and
> there are much more efficient ways to get the EAPI?

There do not seem to be any that people agree on =)

I mean, it's /easy/ to get the EAPI. There's a list of ways on the GLEP,
but all of them have trade-offs.

The header comment seems to be the preference in this thread. The
advantage that the eapi function has over a comment is that it's not
magic -- it's just normal bash syntax. So we've addressed that issue at
a small performance cost (we're really only sourcing the ebuild up to
'exit').

Whether or not that trade-off is worth it, who knows. I would prefer to
make it work before we make it fast; but new ideas can't hurt
considering nobody has agreed on an old one.


> Also, you've got a chicken and egg problem, since you're assuming that
> your eapi() function is going to be available in the global environment
> when the ebuild is sourced. The whole point of knowing the EAPI in
> advance is so that we will be able to make changes to the global
> environment, such as add, remove, or change the behavior of _any_ function.

Nah, at the top of the ebuild, you would have,

 EAPI=5
 eapi 5

until package managers catch up, just like we would have to do with the
comment. Once they've caught up, we can drop the EAPI= assignment, and
begin using new features after the call to 'eapi'.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
> The advantage that the eapi function has over a comment is that it's not
> magic -- it's just normal bash syntax. So we've addressed that issue at
> a small performance cost (we're really only sourcing the ebuild up to
> 'exit').

Also consider the case where a user syncs after not having updated for a
couple of months, and the tree contains some ebuilds with EAPIs that are
not supported by the currently installed package manager.

In this case, when resolving dependencies and filtering ebuilds based on
whether or not their EAPI is supported, spawning bash once per ebuild is
much more costly than the alternatives.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexis Ballier]

On Fri, 09 Mar 2012 07:41:09 -0800
Zac Medico  wrote:

> On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
> > The advantage that the eapi function has over a comment is that
> > it's not magic -- it's just normal bash syntax. So we've addressed
> > that issue at a small performance cost (we're really only sourcing
> > the ebuild up to 'exit').
> 
> Also consider the case where a user syncs after not having updated
> for a couple of months, and the tree contains some ebuilds with EAPIs
> that are not supported by the currently installed package manager.
> 
> In this case, when resolving dependencies and filtering ebuilds based
> on whether or not their EAPI is supported, spawning bash once per
> ebuild is much more costly than the alternatives.

isnt the whole point of the proposal to get eapi without sourcing ?

so that we can use new bash features at local or global scope without
risking that people with an old bash get syntax errors trying to get
the eapi

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ian Stakenvicius]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 09/03/12 10:41 AM, Zac Medico wrote:
> On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
>> The advantage that the eapi function has over a comment is that
>> it's not magic -- it's just normal bash syntax. So we've
>> addressed that issue at a small performance cost (we're really
>> only sourcing the ebuild up to 'exit').
> 
> Also consider the case where a user syncs after not having updated
> for a couple of months, and the tree contains some ebuilds with
> EAPIs that are not supported by the currently installed package
> manager.


IIRC we get this already, when the EAPI isn't supported by the version
of portage installed -- upgrading really old systems won't allow an
emerge of python-2.7 due to a too-new EAPI, and python-2.7 is needed
to upgrade to the newer portage.

I don't see how the EAPI check itself failing and thereby excluding an
ebuild is much different than the specified EAPI excluding it..?
Either way, the end user is going to have issues if they don't keep
their portage up to date.



-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)

iF4EAREIAAYFAk9aJ0QACgkQAJxUfCtlWe2tTAEA7iUgDOCaGoQhz1dXukQ/a3lY
rsdqewd2DYZWtsv+3XoA/iRVe+qf4HXdkWTchFRHlolaTJechz6AZCzKY/sNdu4w
=1e/8
-END PGP SIGNATURE-

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 07:51 AM, Alexis Ballier wrote:
> On Fri, 09 Mar 2012 07:41:09 -0800
> Zac Medico  wrote:
> 
>> On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
>>> The advantage that the eapi function has over a comment is that
>>> it's not magic -- it's just normal bash syntax. So we've addressed
>>> that issue at a small performance cost (we're really only sourcing
>>> the ebuild up to 'exit').
>>
>> Also consider the case where a user syncs after not having updated
>> for a couple of months, and the tree contains some ebuilds with EAPIs
>> that are not supported by the currently installed package manager.
>>
>> In this case, when resolving dependencies and filtering ebuilds based
>> on whether or not their EAPI is supported, spawning bash once per
>> ebuild is much more costly than the alternatives.
> 
> isnt the whole point of the proposal to get eapi without sourcing ?
> 
> so that we can use new bash features at local or global scope without
> risking that people with an old bash get syntax errors trying to get
> the eapi

Right. Michael has lost sight of the goal and is moving off on a tangent.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 07:52 AM, Ian Stakenvicius wrote:
> On 09/03/12 10:41 AM, Zac Medico wrote:
>> On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
>>> The advantage that the eapi function has over a comment is that
>>> it's not magic -- it's just normal bash syntax. So we've
>>> addressed that issue at a small performance cost (we're really
>>> only sourcing the ebuild up to 'exit').
> 
>> Also consider the case where a user syncs after not having updated
>> for a couple of months, and the tree contains some ebuilds with
>> EAPIs that are not supported by the currently installed package
>> manager.
> 
> 
> IIRC we get this already, when the EAPI isn't supported by the version
> of portage installed -- upgrading really old systems won't allow an
> emerge of python-2.7 due to a too-new EAPI, and python-2.7 is needed
> to upgrade to the newer portage.

You're talking about a very specific (unfortunate) upgrade path
breakage. We should keep the discussion focused on more generic cases
that apply to future upgrade scenarios, rather than focusing on past
breakages (though it's obvious that we should also try to avoid such
upgrade path breakages in the future).

Assuming that there is still a viable upgrade path, the package manager
needs resolve dependencies and filter ebuilds based on whether or not
their EAPI is supported. There are basically two ways to do the EAPI
filtering:

1) Access the EAPI from the metadata cache. This assumes that the
package manager is able to validate metadata cache entries for EAPIs
that it does not support, which is a somewhat fragile assumption given
the complexities of cache validation, which involves verification of
ebuilds and additional sources of metadata which may include eclasses
and possibly other files in the future.

2) Parse the ebuild from the ebuild using a pre-defined protocol (parse
EAPI assignment, parse comment, or parse file name as in GLEP 55...).

> I don't see how the EAPI check itself failing and thereby excluding an
> ebuild is much different than the specified EAPI excluding it..?
> Either way, the end user is going to have issues if they don't keep
> their portage up to date.

They may or may not have issues. Our goal is to minimize our
vulnerability to these kinds of issues as much as possible. Being able
to obtain the ebuild EAPI without the expense of sourcing it is one
small step toward this goal.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Fri, 09 Mar 2012 00:35:14 -0500
Michael Orlitzky  wrote:

> On 03/09/2012 12:04 AM, Michał Górny wrote:
> >>
> >> This is of course isomorphic to requiring a specific EAPI=4 format,
> >> but does allow you to do stupid things like x=`seq 4 4`; eapi $x;
> >> if you want.
> >
> > What advantage does it give us? We still can't change ebuild syntax
> > in global scope because bash will barf.
> >
> 
> Not in EAPI=5, no, but once all PMs are using the eapi function we
> could.
> 
> The function can do any crazy thing you want. Right now, we need to 
> source the entire ebuild to get at its environment. Before we source
> it for real, we just want to know the value of $EAPI. Since eapi()
> will be the first function called, it can be our interface to this
> variable.

What if bash starts to parse the script completely and barfs at 'syntax
error' before it starts executing stuff?

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Eray Aslan]

On Fri, Mar 09, 2012 at 08:15:11AM -0800, Zac Medico wrote:
> They may or may not have issues. Our goal is to minimize our
> vulnerability to these kinds of issues as much as possible. Being able
> to obtain the ebuild EAPI without the expense of sourcing it is one
> small step toward this goal.

EAPI is metadata and is best treated as such.  In other words, history
aside, it does not belong inside an ebuild.  Making EAPI info part of
the filename does look like a reasonable solution - similar to
seen/replied flags in the filenames in maildir directories.  Heck, even
version numbers in an ebuild filename is similar.

I don't understand why there is a strong objection to it.

But anyway, it is Friday night and I am out of here.  Have fun.

-- 
Eray Aslan 


pgpgziRXSe880.pgp
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 08:33 AM, Eray Aslan wrote:
> On Fri, Mar 09, 2012 at 08:15:11AM -0800, Zac Medico wrote:
>> They may or may not have issues. Our goal is to minimize our
>> vulnerability to these kinds of issues as much as possible. Being able
>> to obtain the ebuild EAPI without the expense of sourcing it is one
>> small step toward this goal.
> 
> EAPI is metadata and is best treated as such.  In other words, history
> aside, it does not belong inside an ebuild.  Making EAPI info part of
> the filename does look like a reasonable solution - similar to
> seen/replied flags in the filenames in maildir directories.  Heck, even
> version numbers in an ebuild filename is similar.
> 
> I don't understand why there is a strong objection to it.

I have a mild preference for the "parse EAPI assignment" approach,
simply because it's the least invasive. That said, ultimately it doesn't
make much difference to me whether we parse it from inside the ebuild or
from its file name. From my perspective, arguing between these
approaches boils down to hair-splitting [1].

[1] http://en.wikipedia.org/wiki/Trivial_objections
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 10:58, Zac Medico wrote:
> On 03/09/2012 07:51 AM, Alexis Ballier wrote:
>> On Fri, 09 Mar 2012 07:41:09 -0800
>> Zac Medico  wrote:
>>
>>> On 03/09/2012 07:21 AM, Michael Orlitzky wrote:
 The advantage that the eapi function has over a comment is that
 it's not magic -- it's just normal bash syntax. So we've addressed
 that issue at a small performance cost (we're really only sourcing
 the ebuild up to 'exit').
>>>
>>> Also consider the case where a user syncs after not having updated
>>> for a couple of months, and the tree contains some ebuilds with EAPIs
>>> that are not supported by the currently installed package manager.
>>>
>>> In this case, when resolving dependencies and filtering ebuilds based
>>> on whether or not their EAPI is supported, spawning bash once per
>>> ebuild is much more costly than the alternatives.
>>
>> isnt the whole point of the proposal to get eapi without sourcing ?
>>
>> so that we can use new bash features at local or global scope without
>> risking that people with an old bash get syntax errors trying to get
>> the eapi
> 
> Right. Michael has lost sight of the goal and is moving off on a tangent.

The point was to be able to get the EAPI without crashing if the ebuild
uses newer features. If you can get the EAPI without sourcing, that
obviously accomplishes the goal. But there are other goals, too, like
not limiting the syntax of the EAPI assignment. I was just trying to
think up something that addresses them all.

In any case, yeah, it would crash and burn if someone synced his tree
with an ancient version of portage. But so would the comment solution.
If you want to fix that, we either have to rename everything (and hope
we get it right this time) or reconsider GLEP 55.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 11:29, Michał Górny wrote:
> 
> What if bash starts to parse the script completely and barfs at 'syntax
> error' before it starts executing stuff?
> 

It doesn't parse the script completely, it executes line-by-line, so we
can bail out early.

This returns 1:

  exit 1

  QWE*$)@#$%IT@$KTRGV{PO#J$%$M@#$_)TVI!%K@B)$YJMV$%())@K#@)

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 08:49 AM, Michael Orlitzky wrote:
> The point was to be able to get the EAPI without crashing if the ebuild
> uses newer features. If you can get the EAPI without sourcing, that
> obviously accomplishes the goal. But there are other goals, too, like
> not limiting the syntax of the EAPI assignment. I was just trying to
> think up something that addresses them all.

Yeah, but you compromised on what can be considered our primary goal,
which is to obtain the EAPI without sourcing the ebuild.

> In any case, yeah, it would crash and burn if someone synced his tree
> with an ancient version of portage. But so would the comment solution.

Don't forget the possibility of bash syntax extensions triggering a
syntax error [1]. The comment and bash assignment solutions are immune
to that because bash is exceedingly unlikely to break backward
compatibility in the comment and bash assignment syntax.

[1]
http://archives.gentoo.org/gentoo-dev/msg_2c1fbbb32f5bc47756303ca18055bee4.xml
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ulrich Mueller]

> On Fri, 09 Mar 2012, Michael Orlitzky wrote:

>> What if bash starts to parse the script completely and barfs at
>> 'syntax error' before it starts executing stuff?

> It doesn't parse the script completely, it executes line-by-line, so
> we can bail out early.

How can you tell that this behaviour won't be changed in a future bash
version?

Ulrich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 12:11, Ulrich Mueller wrote:
>> On Fri, 09 Mar 2012, Michael Orlitzky wrote:
> 
>>> What if bash starts to parse the script completely and barfs at
>>> 'syntax error' before it starts executing stuff?
> 
>> It doesn't parse the script completely, it executes line-by-line, so
>> we can bail out early.
> 
> How can you tell that this behaviour won't be changed in a future bash
> version?
> 

Who's to say that in the future my computer won't be made out of
delicious ice cream, eliminating the need for EAPIs entirely?

Chances are, this would break thousands of scripts, so we hope they
wouldn't do it. If it does happen, we either deal with it then, or don't
upgrade to that version of bash -- the same as we would do with any
other massive breaking change.

At any rate, I'm now convinced that we all want GLEP 55, but with a
different name.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 09:31 AM, Michael Orlitzky wrote:
> On 03/09/12 12:11, Ulrich Mueller wrote:
>>> On Fri, 09 Mar 2012, Michael Orlitzky wrote:
>>
 What if bash starts to parse the script completely and barfs at
 'syntax error' before it starts executing stuff?
>>
>>> It doesn't parse the script completely, it executes line-by-line, so
>>> we can bail out early.
>>
>> How can you tell that this behaviour won't be changed in a future bash
>> version?
>>
> 
> Who's to say that in the future my computer won't be made out of
> delicious ice cream, eliminating the need for EAPIs entirely?
> 
> Chances are, this would break thousands of scripts, so we hope they
> wouldn't do it. If it does happen, we either deal with it then, or don't
> upgrade to that version of bash -- the same as we would do with any
> other massive breaking change.

Ulrich is talking about extensions which require a newer version of
bash. These kinds of extensions are quite common and don't cause
"massive breaking" because people simply have to upgrade bash in order
to use the new extensions, and their old scripts continue to run because
the new extensions don't interfere with backward compatibility.

Your eapi() function proposal is especially fragile in this context
because it assumes that the installed version of bash will be able to
execute a script that may target a newer version of bash. This is a
special case that is typically not a problem, although it is a major
problem under the specific conditions that your eapi() function approach
induces.

Anyway, lets focus on our main goal, which is to decide on a way to
obtain the EAPI _without_ sourcing the ebuild.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Fri, 09 Mar 2012 12:31:24 -0500
Michael Orlitzky  wrote:

> On 03/09/12 12:11, Ulrich Mueller wrote:
> >> On Fri, 09 Mar 2012, Michael Orlitzky wrote:
> > 
> >>> What if bash starts to parse the script completely and barfs at
> >>> 'syntax error' before it starts executing stuff?
> > 
> >> It doesn't parse the script completely, it executes line-by-line,
> >> so we can bail out early.
> > 
> > How can you tell that this behaviour won't be changed in a future
> > bash version?
> > 
> 
> Who's to say that in the future my computer won't be made out of
> delicious ice cream, eliminating the need for EAPIs entirely?
> 
> Chances are, this would break thousands of scripts, so we hope they
> wouldn't do it. If it does happen, we either deal with it then, or
> don't upgrade to that version of bash -- the same as we would do with
> any other massive breaking change.

Thousands of scripts? So... you're saying that people actually use
thousands of scripts which have invalid syntax...

Well, one thing I can think of now is makeself and similar. Those are
indeed a quite good argument.

But the main point here is that at some point someone may want to use
a non-bash syntax for ebuilds. Or some kind of optional bash extension.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[James Broadhead]

On 9 March 2012 17:31, Michael Orlitzky  wrote:
> At any rate, I'm now convinced that we all want GLEP 55, but with a
> different name.

I think that moving the data to the filename is probably a better
approach than semi- or repeat parsing, but I prefer preserving the
.ebuild extension, and think that eapi should be specified similarly
to ebuild revision, as a suffix. for instance:

app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
new schema)
app-foo/bar-1.0.0-r1-e1.ebuild
app-foo/bar-1.0.0-r1-e99.ebuild

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 12:47, Zac Medico wrote:
> 
> Ulrich is talking about extensions which require a newer version of
> bash. These kinds of extensions are quite common and don't cause
> "massive breaking" because people simply have to upgrade bash in order
> to use the new extensions, and their old scripts continue to run because
> the new extensions don't interfere with backward compatibility.
> 
> Your eapi() function proposal is especially fragile in this context
> because it assumes that the installed version of bash will be able to
> execute a script that may target a newer version of bash. This is a
> special case that is typically not a problem, although it is a major
> problem under the specific conditions that your eapi() function approach
> induces.

Well, you wouldn't need to execute the script targeting a newer version
of bash. You would need to source it, which involves maybe setting
EAPI=5, and then calling the 'eapi' function which will immediately exit.

The new features aren't a problem because you never get to them. (If you
try to use some new bash extension to compute your EAPI value, well, you
shouldn't have done that.)


> Anyway, lets focus on our main goal, which is to decide on a way to
> obtain the EAPI _without_ sourcing the ebuild.

Agreed. I'm forced to agree with Ciaran: can we just ignore the previous
council's decision, and reconsider the merits of GLEP 55?

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alexis Ballier]

On Fri, 9 Mar 2012 18:02:51 +
James Broadhead  wrote:

> On 9 March 2012 17:31, Michael Orlitzky  wrote:
> > At any rate, I'm now convinced that we all want GLEP 55, but with a
> > different name.
> 
> I think that moving the data to the filename is probably a better
> approach than semi- or repeat parsing, but I prefer preserving the
> .ebuild extension, and think that eapi should be specified similarly
> to ebuild revision, as a suffix. for instance:
> 
> app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
> new schema)
> app-foo/bar-1.0.0-r1-e1.ebuild
> app-foo/bar-1.0.0-r1-e99.ebuild
> 

if you want to keep .ebuild you need to keep current naming, afaik
package managers fail on invalid names

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 10:24 AM, Alexis Ballier wrote:
> On Fri, 9 Mar 2012 18:02:51 +
> James Broadhead  wrote:
> 
>> On 9 March 2012 17:31, Michael Orlitzky  wrote:
>>> At any rate, I'm now convinced that we all want GLEP 55, but with a
>>> different name.
>>
>> I think that moving the data to the filename is probably a better
>> approach than semi- or repeat parsing, but I prefer preserving the
>> .ebuild extension, and think that eapi should be specified similarly
>> to ebuild revision, as a suffix. for instance:
>>
>> app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
>> new schema)
>> app-foo/bar-1.0.0-r1-e1.ebuild
>> app-foo/bar-1.0.0-r1-e99.ebuild
>>
> 
> if you want to keep .ebuild you need to keep current naming, afaik
> package managers fail on invalid names

Invalid names like those should only trigger warnings.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 13:02, James Broadhead wrote:
> On 9 March 2012 17:31, Michael Orlitzky  wrote:
>> At any rate, I'm now convinced that we all want GLEP 55, but with a
>> different name.
> 
> I think that moving the data to the filename is probably a better
> approach than semi- or repeat parsing, but I prefer preserving the
> .ebuild extension, and think that eapi should be specified similarly
> to ebuild revision, as a suffix. for instance:
> 
> app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
> new schema)
> app-foo/bar-1.0.0-r1-e1.ebuild
> app-foo/bar-1.0.0-r1-e99.ebuild
> 

One of the benefits of GLEP 55 naming is that old package managers won't
try to parse them. So, for example, if we put new features in,

  app-foo/bar-1.0.0.ebuild-5

portage from 2003 won't try to source it.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 10:33 AM, Michael Orlitzky wrote:
> On 03/09/12 13:02, James Broadhead wrote:
>> On 9 March 2012 17:31, Michael Orlitzky  wrote:
>>> At any rate, I'm now convinced that we all want GLEP 55, but with a
>>> different name.
>>
>> I think that moving the data to the filename is probably a better
>> approach than semi- or repeat parsing, but I prefer preserving the
>> .ebuild extension, and think that eapi should be specified similarly
>> to ebuild revision, as a suffix. for instance:
>>
>> app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
>> new schema)
>> app-foo/bar-1.0.0-r1-e1.ebuild
>> app-foo/bar-1.0.0-r1-e99.ebuild
>>
> 
> One of the benefits of GLEP 55 naming is that old package managers won't
> try to parse them. So, for example, if we put new features in,
> 
>   app-foo/bar-1.0.0.ebuild-5
> 
> portage from 2003 won't try to source it.

Every software product has an end of life. I think if a system hasn't
been updated in the last 2 years or so, then it's fair to assume that it
will never be updated. So, all relevant versions of portage should
simply show a warning message if the encounter an ebuild name such as
"app-foo/bar-1.0.0-r1-e99.ebuild".
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Rich Freeman]

On Fri, Mar 9, 2012 at 12:47 PM, Zac Medico  wrote:
> Anyway, lets focus on our main goal, which is to decide on a way to
> obtain the EAPI _without_ sourcing the ebuild.

Agreed.  Plus, an approach that either uses the filename or something
like a comment line is also going to be much more flexible if bash
syntax changes substantially, or if we want to support
ebuilds-in-python or some other approach.

That is the main merit I see to the shebang approach - you could
define some kind of API that actually involves executing the ebuild.
Sticking the EAPI in the filename is a little less complicated and it
gives you the same flexibility - I'm not aware of any file format that
is fussy about the content of the filename.

Even approaches like putting EAPI=5 in the file (even inside a
comment) might break if the file is meant to be generally interpreted
by some program that has rigid syntax rules.Most scripting
languages could probably handle this somehow, but if you ever wanted
ELF ebuilds that wouldn't be so likely to fly unless you could embed
\nEAPI=ELF8\n in something near the start of the file.

Sure, I doubt we'll ever want ELF ebuilds (I'd consider
non-programatic ebuilds for more trivial packages more likely), but
the point is that we should prefer options that offer more flexibility
down the road over ones that solve the immediate need but just leave
us with the same debate two years from now.

Rich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Fri, 09 Mar 2012 11:49:44 -0500
Michael Orlitzky  wrote:
> >> isnt the whole point of the proposal to get eapi without sourcing ?
> >>
> >> so that we can use new bash features at local or global scope
> >> without risking that people with an old bash get syntax errors
> >> trying to get the eapi
> > 
> > Right. Michael has lost sight of the goal and is moving off on a
> > tangent.
> 
> The point was to be able to get the EAPI without crashing if the
> ebuild uses newer features.

No, it's not. There's more to it than that.

Some EAPIs really require defining certain environment variables, shell
options, sandbox things etc *before* the sourcing starts. It's a massive
pain in the ass to try to handle setting that kind of thing on the fly
once the sourcing has already started. Knowing the EAPI before having
to spawn a bash process isn't just about performance, it's also about
making ebuilds much less difficult to deal with.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Fri, 09 Mar 2012 10:56:03 -0800
Zac Medico  wrote:
> Every software product has an end of life. I think if a system hasn't
> been updated in the last 2 years or so, then it's fair to assume that
> it will never be updated. So, all relevant versions of portage should
> simply show a warning message if the encounter an ebuild name such as
> "app-foo/bar-1.0.0-r1-e99.ebuild".

It's worth noting that "showing a warning message" will result in
zillions of freaking out users, and that we want to be able to
introduce new EAPIs without having a stable Portage version that
handles it around.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michael Orlitzky]

On 03/09/12 13:56, Zac Medico wrote:
> On 03/09/2012 10:33 AM, Michael Orlitzky wrote:
>> On 03/09/12 13:02, James Broadhead wrote:
>>> On 9 March 2012 17:31, Michael Orlitzky  wrote:
 At any rate, I'm now convinced that we all want GLEP 55, but with a
 different name.
>>>
>>> I think that moving the data to the filename is probably a better
>>> approach than semi- or repeat parsing, but I prefer preserving the
>>> .ebuild extension, and think that eapi should be specified similarly
>>> to ebuild revision, as a suffix. for instance:
>>>
>>> app-foo/bar-1.0.0-r1.ebuild # EAPI0 (or the highest EAPI prior to the
>>> new schema)
>>> app-foo/bar-1.0.0-r1-e1.ebuild
>>> app-foo/bar-1.0.0-r1-e99.ebuild
>>>
>>
>> One of the benefits of GLEP 55 naming is that old package managers won't
>> try to parse them. So, for example, if we put new features in,
>>
>>   app-foo/bar-1.0.0.ebuild-5
>>
>> portage from 2003 won't try to source it.
> 
> Every software product has an end of life. I think if a system hasn't
> been updated in the last 2 years or so, then it's fair to assume that it
> will never be updated. So, all relevant versions of portage should
> simply show a warning message if the encounter an ebuild name such as
> "app-foo/bar-1.0.0-r1-e99.ebuild".

I was just repeating your point against the eapi function =)

And there is a non-zero benefit to it, I've had to rescue systems that
haven't seen an update in years.

The best reason I can think of for using ebuild-EAPI is simply semantic.
The basename of the ebuild refers to the package, the extension decides
what interprets it. That is at least consistent with every other file
extension.

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Kent Fredric]

On 9 March 2012 06:30, Jeroen Roovers  wrote:
> On Thu, 8 Mar 2012 17:14:58 +
> Ciaran McCreesh  wrote:
>
>> Having a different, special rule for something that looks exactly like
>> lots of other things that do not have that different, special rule is
>> hardly hair splitting. This rule would have to be documented and have
>> special code to carefully enforce it. That's a big deal.
>
> Exactly like HOMEPAGE is a big deal?
>
> HOMEPAGE         Package's homepage. If you are unable to locate an
> official one, try to provide a link to freshmeat.net or a similar
> package tracking site. Never refer to a variable name in the string;
> include only raw text.

I'm not sure if that is a good example of "things that are weird like EAPI"

You can set HOMEPAGE in an eclass for instance, using anything you can
do with bash and build it out of substrings.

perl-module.eclass does just that.

[[ -z "${HOMEPAGE}" ]] && \
HOMEPAGE="http://search.cpan.org/dist/${MY_PN:-${PN}}/";

Not really comparable in entirety to EAPI.

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Sat, 10 Mar 2012 14:06:49 +1300
Kent Fredric  wrote:
> > HOMEPAGE         Package's homepage. If you are unable to locate an
> > official one, try to provide a link to freshmeat.net or a similar
> > package tracking site. Never refer to a variable name in the string;
> > include only raw text.
> 
> I'm not sure if that is a good example of "things that are weird like
> EAPI"
> 
> You can set HOMEPAGE in an eclass for instance, using anything you can
> do with bash and build it out of substrings.
> 
> perl-module.eclass does just that.
> 
> [[ -z "${HOMEPAGE}" ]] && \
> HOMEPAGE="http://search.cpan.org/dist/${MY_PN:-${PN}}/";
> 
> Not really comparable in entirety to EAPI.

The point is that there's a silly QA policy saying not to do that. It's
not a PMS rule.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/09/2012 11:20 AM, Ciaran McCreesh wrote:
> On Fri, 09 Mar 2012 11:49:44 -0500
> Michael Orlitzky  wrote:
 isnt the whole point of the proposal to get eapi without sourcing ?

 so that we can use new bash features at local or global scope
 without risking that people with an old bash get syntax errors
 trying to get the eapi
>>>
>>> Right. Michael has lost sight of the goal and is moving off on a
>>> tangent.
>>
>> The point was to be able to get the EAPI without crashing if the
>> ebuild uses newer features.
> 
> No, it's not. There's more to it than that.
> 
> Some EAPIs really require defining certain environment variables, shell
> options, sandbox things etc *before* the sourcing starts. It's a massive
> pain in the ass to try to handle setting that kind of thing on the fly
> once the sourcing has already started. Knowing the EAPI before having
> to spawn a bash process isn't just about performance, it's also about
> making ebuilds much less difficult to deal with.

Yeah. Another way of putting it is that the requirement to spawn a bash
process and source the ebuild adds a ridiculous amount of unnecessary
complexity, in violation of the KISS principle [1].

[1] http://en.wikipedia.org/wiki/KISS_principle
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Brian Harring]

On Fri, Mar 09, 2012 at 06:52:40PM +0100, Micha?? G??rny wrote:
> On Fri, 09 Mar 2012 12:31:24 -0500
> Michael Orlitzky  wrote:
> 
> > On 03/09/12 12:11, Ulrich Mueller wrote:
> > >> On Fri, 09 Mar 2012, Michael Orlitzky wrote:
> > > 
> > >>> What if bash starts to parse the script completely and barfs at
> > >>> 'syntax error' before it starts executing stuff?
> > > 
> > >> It doesn't parse the script completely, it executes line-by-line,
> > >> so we can bail out early.
> > > 
> > > How can you tell that this behaviour won't be changed in a future
> > > bash version?
> > > 
> > 
> > Who's to say that in the future my computer won't be made out of
> > delicious ice cream, eliminating the need for EAPIs entirely?
> > 
> > Chances are, this would break thousands of scripts, so we hope they
> > wouldn't do it. If it does happen, we either deal with it then, or
> > don't upgrade to that version of bash -- the same as we would do with
> > any other massive breaking change.
> 
> Thousands of scripts? So... you're saying that people actually use
> thousands of scripts which have invalid syntax...

Just a note; you need to look into how aliases work.  That right there 
unfortunately means bash isn't going to pre-parse, not as long as 
aliases are supported.

Back to your arguing...
~brian

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Brian Harring]

On Sat, Mar 10, 2012 at 08:06:50AM -0800, Zac Medico wrote:
> On 03/09/2012 11:20 AM, Ciaran McCreesh wrote:
> > On Fri, 09 Mar 2012 11:49:44 -0500
> > Michael Orlitzky  wrote:
>  isnt the whole point of the proposal to get eapi without sourcing ?
> 
>  so that we can use new bash features at local or global scope
>  without risking that people with an old bash get syntax errors
>  trying to get the eapi
> >>>
> >>> Right. Michael has lost sight of the goal and is moving off on a
> >>> tangent.
> >>
> >> The point was to be able to get the EAPI without crashing if the
> >> ebuild uses newer features.
> > 
> > No, it's not. There's more to it than that.
> > 
> > Some EAPIs really require defining certain environment variables, shell
> > options, sandbox things etc *before* the sourcing starts. It's a massive
> > pain in the ass to try to handle setting that kind of thing on the fly
> > once the sourcing has already started. Knowing the EAPI before having
> > to spawn a bash process isn't just about performance, it's also about
> > making ebuilds much less difficult to deal with.
> 
> Yeah. Another way of putting it is that the requirement to spawn a bash
> process and source the ebuild adds a ridiculous amount of unnecessary
> complexity, in violation of the KISS principle [1].

This statement is incorrect.

Even if EAPI could be parsed via some non sourcing approach, we 
*still* have to source the ebuild to get the metadata for when the 
EAPI is supported (the vast majority of usage).  That complexity is 
there one way or another- we wouldn't be trying to extract the EAPI 
from the ebuild unless the cache was invalid/missing.

Phrasing it more bluntly: you can only avoid the sourcing step if you 
can isolate that the EAPI is unsupported (which is extremely rare in 
terms of actual user experience).  For the rest of the time (well past 
the 99% mark) sourcing is the immediate step following.


Also, stop referencing wikipedia.  People know what "trivial 
objection" and "KISS" is.  Pointing at random wikipedia links when
people object is just a form of fallacious argument from authority 
( http://en.wikipedia.org/wiki/Argument_from_authority ). :P

~brian

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Brian Harring]

On Fri, Mar 09, 2012 at 09:47:50AM -0800, Zac Medico wrote:
> On 03/09/2012 09:31 AM, Michael Orlitzky wrote:
> > On 03/09/12 12:11, Ulrich Mueller wrote:
> >>> On Fri, 09 Mar 2012, Michael Orlitzky wrote:
> >>
>  What if bash starts to parse the script completely and barfs at
>  'syntax error' before it starts executing stuff?
> >>
> >>> It doesn't parse the script completely, it executes line-by-line, so
> >>> we can bail out early.
> >>
> >> How can you tell that this behaviour won't be changed in a future bash
> >> version?
> >>
> > 
> > Who's to say that in the future my computer won't be made out of
> > delicious ice cream, eliminating the need for EAPIs entirely?
> > 
> > Chances are, this would break thousands of scripts, so we hope they
> > wouldn't do it. If it does happen, we either deal with it then, or don't
> > upgrade to that version of bash -- the same as we would do with any
> > other massive breaking change.
> 
> Ulrich is talking about extensions which require a newer version of
> bash. These kinds of extensions are quite common and don't cause
> "massive breaking" because people simply have to upgrade bash in order
> to use the new extensions, and their old scripts continue to run because
> the new extensions don't interfere with backward compatibility.
> 
> Your eapi() function proposal is especially fragile in this context
> because it assumes that the installed version of bash will be able to
> execute a script that may target a newer version of bash. This is a
> special case that is typically not a problem, although it is a major
> problem under the specific conditions that your eapi() function approach
> induces.

Pragmatic reality, the eapi function actually would work fine.  As 
pointed out elsewhere, bash parses as it goes- which isn't going to 
change.

If someone invokes 'eapi happy-meal' and it's not supported, 
the sourcing is stopped immediately, cache gets -happy-meal for the 
EAPI, and the pm continues to ignore the ebuild till either the 
ebuilds mtime changes (which case it redoes the metadata regen) or the 
PM now supports that EAPI, and... you guessed it, redoes the metadata 
regen.  The cache behaviour is basically the same regardless of the 
EAPI mechanism.

Now, this isn't to say everyone views the function as *optimal*.  
People can argue about that as much as they'd like.

The point however is that it *would* work.  Anyone claiming 
fragility/otherwise needs to put forth actual code examples.


> Anyway, lets focus on our main goal, which is to decide on a way to
> obtain the EAPI _without_ sourcing the ebuild.

Nitpicking, but the point needs be made; this is *your* requirement.  
The requirement is to be able to deploy new globals/bash 
requirements/whatever.

There is a difference.

~brian

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Rich Freeman]

On Sun, Mar 11, 2012 at 10:03 PM, Brian Harring  wrote:
> Pragmatic reality, the eapi function actually would work fine.  As
> pointed out elsewhere, bash parses as it goes- which isn't going to
> change.

Unless the ebuild isn't written in bash...

How do you source the ebuild if you don't know what to use to source
it?  How do you know what to use to source it if you don't know the
EAPI?  Right now all the existing EAPIs use bash, but there is no
reason the file couldn't be xml, or python, or just about anything
else.

If we want to allow for that kind of flexibility, then it might make
sense to go ahead and state that our convention is to stick EAPI=5 in
one of the first few lines of the ebuild, or inside a comment, but
also go a step further and state that the text "EAPI=" cannot appear
elsewhere in the ebuild (or perhaps within the first 10 lines).  Just
about any file format we might use would allow us to make "EAPI="
appear in it, but not all could guarantee that it would occur at the
start of a line, or at the start of a line immediately after a #.

In any case, I can really see the KISS value in a very rigid syntax
that is trivial to parse.  Stuff like this almost makes me wish our
ebuilds already were xml files or such, with bash embedded inside
sections.  Finding a particular tag in an xml file is trivial as the
fundamentals haven't changed in 15 years.

Rich

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Alec Warner]

On Sun, Mar 11, 2012 at 7:20 PM, Rich Freeman  wrote:
> On Sun, Mar 11, 2012 at 10:03 PM, Brian Harring  wrote:
>> Pragmatic reality, the eapi function actually would work fine.  As
>> pointed out elsewhere, bash parses as it goes- which isn't going to
>> change.
>
> Unless the ebuild isn't written in bash...

I'd opt for a different extension in that case actually.

>
> How do you source the ebuild if you don't know what to use to source
> it?  How do you know what to use to source it if you don't know the
> EAPI?  Right now all the existing EAPIs use bash, but there is no
> reason the file couldn't be xml, or python, or just about anything
> else.
>
> If we want to allow for that kind of flexibility, then it might make
> sense to go ahead and state that our convention is to stick EAPI=5 in
> one of the first few lines of the ebuild, or inside a comment, but
> also go a step further and state that the text "EAPI=" cannot appear
> elsewhere in the ebuild (or perhaps within the first 10 lines).  Just
> about any file format we might use would allow us to make "EAPI="
> appear in it, but not all could guarantee that it would occur at the
> start of a line, or at the start of a line immediately after a #.
>
> In any case, I can really see the KISS value in a very rigid syntax
> that is trivial to parse.  Stuff like this almost makes me wish our
> ebuilds already were xml files or such, with bash embedded inside
> sections.  Finding a particular tag in an xml file is trivial as the
> fundamentals haven't changed in 15 years.

I will stab the next person who suggests 'xml-like ebuilds.'

-A

>
> Rich
>

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/11/2012 07:03 PM, Brian Harring wrote:
> Pragmatic reality, the eapi function actually would work fine.  As 
> pointed out elsewhere, bash parses as it goes- which isn't going to 
> change.
> 
> If someone invokes 'eapi happy-meal' and it's not supported, 
> the sourcing is stopped immediately, cache gets -happy-meal for the 
> EAPI, and the pm continues to ignore the ebuild till either the 
> ebuilds mtime changes (which case it redoes the metadata regen) or the 
> PM now supports that EAPI, and... you guessed it, redoes the metadata 
> regen.  The cache behaviour is basically the same regardless of the 
> EAPI mechanism.
> 
> Now, this isn't to say everyone views the function as *optimal*.  
> People can argue about that as much as they'd like.
> 
> The point however is that it *would* work.  Anyone claiming 
> fragility/otherwise needs to put forth actual code examples.

Suppose that EAPI 5 requires bash-5. There may be bash-5 syntax in the
ebuild prior to your eapi() function call, causing a fatal syntax error.

>> Anyway, lets focus on our main goal, which is to decide on a way to
>> obtain the EAPI _without_ sourcing the ebuild.
> 
> Nitpicking, but the point needs be made; this is *your* requirement.  
> The requirement is to be able to deploy new globals/bash 
> requirements/whatever.

Well, I think most people would tend to accept my requirement, and that
it falls within the realm of common-sense if avoiding unnecessary
complexity is one of our goals.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/11/2012 06:55 PM, Brian Harring wrote:
> On Sat, Mar 10, 2012 at 08:06:50AM -0800, Zac Medico wrote:
>> Yeah. Another way of putting it is that the requirement to spawn a bash
>> process and source the ebuild adds a ridiculous amount of unnecessary
>> complexity, in violation of the KISS principle [1].
> 
> This statement is incorrect.
> 
> Even if EAPI could be parsed via some non sourcing approach, we 
> *still* have to source the ebuild to get the metadata for when the 
> EAPI is supported (the vast majority of usage).  That complexity is 
> there one way or another- we wouldn't be trying to extract the EAPI 
> from the ebuild unless the cache was invalid/missing.

There are a couple of other cases worth considering:

1) User downloads an overlay that doesn't provide cache. We want the
package manager to give a pretty "EAPI unsupported" message, rather than
spit out some bash syntax errors.

2) You're assuming that a package manager can validate cache for an EAPI
that it doesn't necessarily support. That's a somewhat fragile
assumption, given the complexities of cache validation, which involve
verification all files that affect metadata and those files may vary
depending on the EAPI.

> Phrasing it more bluntly: you can only avoid the sourcing step if you 
> can isolate that the EAPI is unsupported (which is extremely rare in 
> terms of actual user experience).  For the rest of the time (well past 
> the 99% mark) sourcing is the immediate step following.

For the sake of being robust in all possible cases, it's just a lot
simpler if we can obtain the EAPI simply and reliably, without spawning
bash to source the ebuild.

> Also, stop referencing wikipedia.  People know what "trivial 
> objection" and "KISS" is.

You can't assume that. On this list we've got potential to have readers
and responders with lots of different backgrounds. Your insistence on
using bash to obtain the EAPI would make me wonder if you understood the
KISS principle, if I didn't know you better.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Kent Fredric]

On 12 March 2012 15:20, Rich Freeman  wrote:
> On Sun, Mar 11, 2012 at 10:03 PM, Brian Harring  wrote:
>> Pragmatic reality, the eapi function actually would work fine.  As
>> pointed out elsewhere, bash parses as it goes- which isn't going to
>> change.
>
> Unless the ebuild isn't written in bash...
>
> How do you source the ebuild if you don't know what to use to source
> it?  How do you know what to use to source it if you don't know the
> EAPI?  Right now all the existing EAPIs use bash, but there is no
> reason the file couldn't be xml, or python, or just about anything
> else.


A convention that is often used in this scenario is to combine a
hashbang call with that specific sourcer stripping that hashbang prior
to the parse.

ie:

foo.ebuild:

#!/usr/bin/env eapi-xml-5

would send the current file ( foo.ebuild ) to the process "eapi-xml-5"
( as defined by the current $PATH setting )

This process can easily then strip the #! stanza before sending the
content to an XML parser.

( ps. while I wouldn't actually use XML, its a good example case
because standard shell-script style commenting is illegal syntax in
XML )

The benefits of this approach seem obvious, but there are also obvious
downsides.

1. PRO: Unlike /usr/bin/eapi  , this style is PMS Agnostic, and only
requires the PMS have a process named "eapi-xml-5" *somewhere* in the
system under $PATH

2. CON: Unlike /usr/bin/eapi , you're limited by what you can send it
, /usr/bin/env  eapi xml-5 would be preferable syntax imo, but it
doesn't work, because its parsed as [ '/usr/bin/env' , 'eapi xml-5' ]
which then yeilds a "permission denied" due to no  command with that
name existing.

3. PRO: There's not /much/ risk of a user trying to run it directly,
mostly because there's no +x bit

4. CON: This syntax is going to conflict with whatever language we are
proceeding, which while this caveat is dealt with by the command that
the file is dispatched to, its a lot of work getting everything else
to play ball ( editors won't understand, linters wont understand,
various other tools that work strictly with the native forms of that
language wont understand ) and this problem exists for *every* case
where the coding system we're targeting doesn't natively support
whatever "magic" indicators we're trying to stuff in the file.

5. PRO: detecting this notation being used in a file is cheap-ish ,
you only have to read 2 characters into the file to know its using
this notation.


As a result of all these, it seems to me if we ever want to support
non-bash or non-scripting sources ( ie: YAML, JSON, XML , etc ) or any
other language which doesn't support "# is a comment" , we're going to
be pretty much forced to declare the EAPI outside the file somehow.

Whether this is via the filename ( yes, I've seen that debate ) or via
some other metadata file ( which seems more poison than cure ) would
be the real issue.

Having to store this in a metadata file that maps *.ebuild to EAPI
would seem like a solution that would only increase the filesize of
portage, and slow down runtime substantially, unless of course you had
a per-repository index that sped this up and was generated during the
metadata/ generation phase.

Then $PMS could just load that file from each repository , and
assuming the cache was still valid, it could easily know the EAPI for
any .ebuild it would source in advance.

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Kent Fredric]

On 12 March 2012 15:24, Alec Warner  wrote:

> I will stab the next person who suggests 'xml-like ebuilds.'

State-fully coded ebuilds, while perhaps not to your liking, for some
code-types can be incredibly useful.

For example, 9/10 perl-module ebuilds don't need any code at all in
the ebuild itself, it could do most of the work via structured data,
with the  rest of the work done by an e-class.

However, bash has serious problems when you want to represent hierarchical data.


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Zac Medico]

On 03/11/2012 11:50 PM, Kent Fredric wrote:
> #!/usr/bin/env eapi-xml-5
> 
> would send the current file ( foo.ebuild ) to the process "eapi-xml-5"
> ( as defined by the current $PATH setting )

All we need is a way for the package manager to probe the EAPI. Spawning
a process to do that is just overkill.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Kent Fredric]

On 12 March 2012 20:08, Zac Medico  wrote:
> On 03/11/2012 11:50 PM, Kent Fredric wrote:
>> #!/usr/bin/env eapi-xml-5
>>
>> would send the current file ( foo.ebuild ) to the process "eapi-xml-5"
>> ( as defined by the current $PATH setting )
>
> All we need is a way for the package manager to probe the EAPI. Spawning
> a process to do that is just overkill.

Yeah, but if you read the rest of my message, you'll see my argument
that hinges around the fact most proposals so far tend to focus around
the fact the current format for ebuild is bash, and bash only, and  I
think it very hard to implement a language agnostic way to specify the
EAPI in a way that will work on languages where you

a) Can't use # to mark comments
b) May not even be working with the file directly, and perhaps
generating a .json file from a data-structure being serialised, and
the key named "EAPI" may be unpredictably placed near the end of the
.json file.

Case b is entirely plausible, and even realistic, because
non-script-based data-structures often don't bear importance on the
order of elements.


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Mon, 12 Mar 2012 00:08:46 -0700
Zac Medico  wrote:

> On 03/11/2012 11:50 PM, Kent Fredric wrote:
> > #!/usr/bin/env eapi-xml-5
> > 
> > would send the current file ( foo.ebuild ) to the process
> > "eapi-xml-5" ( as defined by the current $PATH setting )
> 
> All we need is a way for the package manager to probe the EAPI.
> Spawning a process to do that is just overkill.

And we could just use a good regex for that instead.

Something like: [eE][aA][pP][iI]  [a-z0-9-+]+

and just require users for this to be the first thing declared in
an ebuild. Of course, this could make problems with stuff like:

# EAPI 4 because of foobarbaz
EAPI=4

(on the other hand, in this particular case it will fetch '4' anyway).

And this will work as well with:

15-xml

and

- eapi: 15-yaml

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Ciaran McCreesh]

On Mon, 12 Mar 2012 09:27:11 +0100
Michał Górny  wrote:
> 15-xml
> 
> and
> 
> - eapi: 15-yaml

You're carefully concocting your examples to make it look like it
should work. If you go the XML route, though, the EAPI would either be
in a DTD or as .

Part of the point of all of this is that we shouldn't have to guess
what future EAPIs will look like.

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Brian Harring]

On Sun, Mar 11, 2012 at 09:08:24PM -0700, Zac Medico wrote:
> On 03/11/2012 06:55 PM, Brian Harring wrote:
> > On Sat, Mar 10, 2012 at 08:06:50AM -0800, Zac Medico wrote:
> >> Yeah. Another way of putting it is that the requirement to spawn a bash
> >> process and source the ebuild adds a ridiculous amount of unnecessary
> >> complexity, in violation of the KISS principle [1].
> > 
> > This statement is incorrect.
> > 
> > Even if EAPI could be parsed via some non sourcing approach, we 
> > *still* have to source the ebuild to get the metadata for when the 
> > EAPI is supported (the vast majority of usage).  That complexity is 
> > there one way or another- we wouldn't be trying to extract the EAPI 
> > from the ebuild unless the cache was invalid/missing.
> 
> There are a couple of other cases worth considering:
> 
> 1) User downloads an overlay that doesn't provide cache. We want the
> package manager to give a pretty "EAPI unsupported" message, rather than
> spit out some bash syntax errors.

This criticsm pretty much applies *strictly to the existing 
implementation*.  It's disenguous busting it in this fashion.

EAPI as a function explicitly gives it an out before hitting any of 
that, eliminating your entire critique.  Same goes for parsing it out 
of the ebuild, or renaming the extension.


> 2) You're assuming that a package manager can validate cache for an EAPI
> that it doesn't necessarily support.

Actually, I'm not.

> That's a somewhat fragile
> assumption, given the complexities of cache validation, which involve
> verification all files that affect metadata and those files may vary
> depending on the EAPI.

This is a fair bit of handwavey bullshit.  The same "complexities of 
cache validation" we have to by default deal with for valid/supported 
EAPIs; the case for unsupported EAPIs is actually simpler than 
for supported EAPis, and up until 7ddb7d30, was likely working fine in 
portage.  Tweaking portage to restore that support is pretty easy.

Also note that with the sole exception of g55, every implementation 
has to deal with this specific issue, even g55 isn't fully exempt from 
it since in cleansing a cache, there is the open question of pruning 
cache entries for ebuilds known, but not understood by the local PM.

Either way, the algo is as follows: PM pulls EAPI from the ebuild 
(regardless of method); PM recognizes that it can't handle it- thus 
it stores EAPI:-$EAPI, and the normal cache ebuild checksumming; 
whether it be mtime, md5 for md5-dict, or whatever form new caches 
choose to use.

For attempts to load, the cache subsystem checks that validation 
before trusting the cache entry- this is in place and has been for a 
very, very long time.  This isn't anything new.

So... the entry point is the ebuild; we have checksums for it, and a 
prefixed versions of it's EAPI stored.  The sole deal breaker here is
if we were to decide to allow eclasses to set EAPI (rather than the
current "you're not supposed to, although we don't explicitly block 
it"- which could be tightened to be disallowed at the implementation 
level for EAPI>4).

The following scenarios exist:

1) PM still doesn't support that EAPI, looks at the cache/ebuild: 
checksums are the same, thus the stored EAPI is trustable, leading to 
the PM knowing it still can't process that ebuild and masking it 
appropriately.

2) PM still doesn't support that EAPI, looks, sees that the checksums
no longer match the ebuild.  Re-pulls metadata/EAPI; if the ebuild has 
been changed to a supported EAPI, continues on its way.  If not, 
stores the checksum/negated EAPI, and masks appropriately.

3) PM now supports that EAPI, looks, sees the negated eapi and 
recognizes it as one it knowns, and forces regeneration.

All of those scenarios are there, and easy enough to handle- this 
applies for the existing EAPI implementation additionally.


Now, this isn't to say that someone can't go and try to structure new 
features explicitly to defeat the existing validations.  They could.

The question here is whether or not new features would actually 
*break* that w/out intentionally going out of their way to do so (aka, 
skip the strawman arguments, actual examples required).


> > Phrasing it more bluntly: you can only avoid the sourcing step if you 
> > can isolate that the EAPI is unsupported (which is extremely rare in 
> > terms of actual user experience).  For the rest of the time (well past 
> > the 99% mark) sourcing is the immediate step following.
> 
> For the sake of being robust in all possible cases, it's just a lot
> simpler if we can obtain the EAPI simply and reliably, without spawning
> bash to source the ebuild.

This still is arguable; any performant PM is going to have to do the 
cache dance I mentioned above, or maintain an in memory list of 
ebuilds it knows it can't handle (which if you want to talk robust, 
fails miserably if the API isn't designed for sync invalidation of 
that global list).  Complexity, oh my.

Tha

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Kent Fredric]

On 12 March 2012 21:27, Michał Górny  wrote:
> And we could just use a good regex for that instead.
>
> Something like: [eE][aA][pP][iI]  [a-z0-9-+]+
>
> and just require users for this to be the first thing declared in
> an ebuild. Of course, this could make problems with stuff like:
>
> # EAPI 4 because of foobarbaz
> EAPI=4
>
> (on the other hand, in this particular case it will fetch '4' anyway).
>
> And this will work as well with:
>
> 15-xml
>
> and
>
> - eapi: 15-yaml
>

Also, remember the proposal is to read it only from the first 10-30
lines of the file, and if you're *generating* YAML/XML , then this is
not necessarily guaranteed.

Some generation tools emit keys in alphanumeric ordering, others
psuedo-randomly, and the EAPI declaration line in some formats could
easily be on the very last line of the file.

And then the regexp could falsely detect something in another key the
original author had not intended as an EAPI definition, but merely a
comment, ...

ie:


  This is a user comment, hurr, I hate EAPI 5 




-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Mon, 12 Mar 2012 08:30:19 +
Ciaran McCreesh  wrote:

> On Mon, 12 Mar 2012 09:27:11 +0100
> Michał Górny  wrote:
> > 15-xml
> > 
> > and
> > 
> > - eapi: 15-yaml
> 
> You're carefully concocting your examples to make it look like it
> should work.

Or I am just printing the first thing that comes into my head.

> If you go the XML route, though, the EAPI would either be in a DTD

Like .../gentoo/eapi/15-xml.dtd ? That would match my regex as well,
unless we allow dots in EAPI.

> or as .

No, definitely not. That's not the XML style.

> Part of the point of all of this is that we shouldn't have to guess
> what future EAPIs will look like.

I'm just suggesting a way which will support a little more than
bash-based solutions. We could also assume that if a file doesn't match
the regexp at all, it's a unsupported EAPI.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] RFD: EAPI specification in ebuilds

[Michał Górny]

On Mon, 12 Mar 2012 21:39:52 +1300
Kent Fredric  wrote:

> On 12 March 2012 21:27, Michał Górny  wrote:
> > And we could just use a good regex for that instead.
> >
> > Something like: [eE][aA][pP][iI]  [a-z0-9-+]+
> >
> > and just require users for this to be the first thing declared in
> > an ebuild. Of course, this could make problems with stuff like:
> >
> > # EAPI 4 because of foobarbaz
> > EAPI=4
> >
> > (on the other hand, in this particular case it will fetch '4'
> > anyway).
> >
> > And this will work as well with:
> >
> > 15-xml
> >
> > and
> >
> > - eapi: 15-yaml
> >
> 
> Also, remember the proposal is to read it only from the first 10-30
> lines of the file, and if you're *generating* YAML/XML , then this is
> not necessarily guaranteed.
> 
> Some generation tools emit keys in alphanumeric ordering, others
> psuedo-randomly, and the EAPI declaration line in some formats could
> easily be on the very last line of the file.

You should ensure your tool does the right thing, I'd say. XML with DTD
often requires specific element order, so such a tool is simply bound
to fail with XML.

For yaml, it's a fair point.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature