Re: Determine before installation whether a port can be installed

[Mojca Miklavec]

On 27 July 2016 at 06:09, Ryan Schmidt wrote:
>
> One solution that occurs to me is to define a new "preflight" phase, to be 
> run before dependencies are computed. Ports can override that phase and do 
> whatever checks they need and exit with "return -code error" if needed. This 
> seems like the most straightforward and flexible solution.

Independent of whether or not this phase would be used for this
purpose, I would find it useful.

At the moment there is no way to "just do something (not even useful)"
with the port. I can run "port fetch", but then this would fetch the
distfile even if the binary package already exists and fetching could
be skipped, so even fetching is "wasting resources".

The problem (that could be solved on both buildbot and user
installation if we had such a phase) is that currently running "sudo
port install " would happily install all
dependencies even if the port itself cannot be installed.

On the buildbot we could run
sudo port preflight portname
# then install dependencies
sudo port install portname
and if the first one fails, there's no need to continue.

For regular users this step could also prevent installing dependencies
if it could be executed "cheap enough".


If I want to bring the port to a state when main.log is created, so
that it could be retrieved with "port logfile " without
actually trying to fetch/extract/configure anything, this is probably
not possible at the moment.

Given how much trickery is currently already present in the buildbot
setup, a failing preflight phase could be sufficient to filter out
false negatives and avoid sending emails about (knowingly) broken
builds.

I'm not saying that we shouldn't implement better ways to signal
obsolete and known-to-be-broken ports. I'm just saying that
implementing some initial phase (that doesn't do anything) would be
helpful.

Mojca
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

> On Aug 4, 2016, at 2:36 PM, Clemens Lang wrote:
> 
> Hi,
> 
> On Thu, Jul 28, 2016 at 03:21:56AM -0500, Ryan Schmidt wrote:
>> However, as I've said, there are other reasons why a port might not be
>> able to install. Should we really modify MacPorts base and create new
>> variables to accommodate each of those reasons? Is there a specific
>> objection to what I consider the easiest and most flexible of the
>> options I suggested: implement a new preflight phase, in which the
>> port can run any logic to determine if the port can be installed?
> 
> I have a specific objection: If this is a phase and does not use
> declarative syntax, you absolutely must open and run the code in the
> Portfile to determine whether a port will build on your OS.
> 
> Opening a running Portfiles is (a) already where our dependency
> management spends most time, (b) hard to optimize in the absence of a
> interpreter-fork-like behavior in Tcl with our current approach, (c)
> completely breaks the idea of using SAT-solving for dependency
> resolution or any other modern method, because those methods depend on
> having accurrate information about *all* available ports (which would
> mean we'd have to run all ports).
> 
> So in short: The phase cannot be put in the port index, which is a bad
> idea in my opinion. Declarative approaches like the platform statement
> can.

Thanks for the explanation! These are good points that I hadn't considered.


> On Thu, Jul 28, 2016 at 01:02:15PM +0200, Rainer Müller wrote:
>> On 2016-07-28 09:06, Artur Szostak wrote:
>>>  macosx >= 10.9
>>>  macosx < 10.10
>> 
>> https://trac.macports.org/ticket/15712
> 
> I like this approach. Remember the brew uses a similar approach to
> blacklisting, which is basically
>  platforms >= :lion
> (please excuse any butchering of Ruby syntax). This is very simple to
> get right, and I'd prefer a similar approach.
> 
> Ryan: Which other cases can you think of? I think we can handle most of
> them with a similar approach. Why not have a field that contains
> identifiers which reference well-known tests for a feature? Here's an
> idea:
> 
>> constraints java-1.0
> 
> uses _resources/port1.0/constraints/java-1.0.tcl to determine whether
> the constraint is fulfilled, but does this once for all ports, rather
> than once per port. Similarly, we could have
> 
>> constraints {platform-1.0 >= lion}
> 
> which could load _resources/port1.0/constraints/platform-1.0.tcl and run
> a proc from there with the ">=" "lion" arguments (again, without running
> the complete Portfile and loading all files in port1.0).

This could work. Does this mean that we drop all pretense of compiling on other 
operating systems? Or, if we continue to allow it, do we need a way to 
distinguish between 1) a port that works on any OS but if built on macOS needs 
a particular version, 2) a port that only works on macOS and need a particular 
macOS version, and 3) a port that only works on macOS but works on any version?


> That would also allow you to implement manual download checks:
> 
>> constraints {manual-download-1.0 "${targetfile}" "${downloadurl}"}
> 
> Whether that field is now called "platform" or "constraints" doesn't
> really matter. We could make them aliases.

Very interesting idea for declaring the need for manual downloads.



> What do you think?


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Clemens Lang]

Hi,

On Thu, Jul 28, 2016 at 03:21:56AM -0500, Ryan Schmidt wrote:
> However, as I've said, there are other reasons why a port might not be
> able to install. Should we really modify MacPorts base and create new
> variables to accommodate each of those reasons? Is there a specific
> objection to what I consider the easiest and most flexible of the
> options I suggested: implement a new preflight phase, in which the
> port can run any logic to determine if the port can be installed?

I have a specific objection: If this is a phase and does not use
declarative syntax, you absolutely must open and run the code in the
Portfile to determine whether a port will build on your OS.

Opening a running Portfiles is (a) already where our dependency
management spends most time, (b) hard to optimize in the absence of a
interpreter-fork-like behavior in Tcl with our current approach, (c)
completely breaks the idea of using SAT-solving for dependency
resolution or any other modern method, because those methods depend on
having accurrate information about *all* available ports (which would
mean we'd have to run all ports).

So in short: The phase cannot be put in the port index, which is a bad
idea in my opinion. Declarative approaches like the platform statement
can.


On Thu, Jul 28, 2016 at 01:02:15PM +0200, Rainer Müller wrote:
> On 2016-07-28 09:06, Artur Szostak wrote:
> >   macosx >= 10.9
> >   macosx < 10.10
> 
> https://trac.macports.org/ticket/15712

I like this approach. Remember the brew uses a similar approach to
blacklisting, which is basically
  platforms >= :lion
(please excuse any butchering of Ruby syntax). This is very simple to
get right, and I'd prefer a similar approach.

Ryan: Which other cases can you think of? I think we can handle most of
them with a similar approach. Why not have a field that contains
identifiers which reference well-known tests for a feature? Here's an
idea:

 > constraints java-1.0

uses _resources/port1.0/constraints/java-1.0.tcl to determine whether
the constraint is fulfilled, but does this once for all ports, rather
than once per port. Similarly, we could have

 > constraints {platform-1.0 >= lion}

which could load _resources/port1.0/constraints/platform-1.0.tcl and run
a proc from there with the ">=" "lion" arguments (again, without running
the complete Portfile and loading all files in port1.0).

That would also allow you to implement manual download checks:

 > constraints {manual-download-1.0 "${targetfile}" "${downloadurl}"}

Whether that field is now called "platform" or "constraints" doesn't
really matter. We could make them aliases.

What do you think?
-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

> On Jul 28, 2016, at 8:36 AM, m...@macports.org wrote:
> 
> 
>> On Jul 27, 2016, at 6:19 PM, Ryan Schmidt  wrote:
>> 
>> Your suggestion seems like a variation of my "third possibility", which has 
>> the disadvantages that every new reason why a port might not be installable 
>> would need a new variable introduced in MacPorts base and a new release of 
>> MacPorts base.
> 
> We already need to release a new version of base for Sierra. I am suggesting 
> we deploy these changes then. 
> 
> 
>> What about ports that need the user to install Java first? What about ports 
>> that need the user to download a file manually first (for example 
>> oracle-instantclient which requires users to agree to a license agreement 
>> before downloading distfiles)?
> 
> Yes, so some kind of pre-fetch argument or new variables are needed for those 
> cases. Given your argument about clear intent (which I completely agree with 
> in general), I think that new keywords should be created for each case where 
> possible. 
> 
> requires_java yes
> 
> That has the added complication of having base then determine if java is 
> indeed installed which can differ per version of MacOS. 
> 
> The oracle-instantclient example is singular and thus should require custom 
> code in the Portfile such as in a pre-fetch phase as you suggest. 

But I'm specifically not advocating for code in a pre-fetch phase, which is 
what we already do now in many places, because I want to know before the fetch 
phase runs and before dependencies are installed whether the port can be 
installed or not. I am advocating for a new preflight phase that runs before 
dependencies are computed into which a port could put logic to decide whether 
it is installable or not.

I'm concerned about expending development effort on building new variables such 
as "requires_java" when a generic solution that can be used by ports having 
unusual requirements is still needed. Why not use the generic solution 
everywhere?


>> Your idea for the p5-graveyard and similar ports to clear the platforms 
>> variable to indicate the port is not installable is clever, but wouldn't it 
>> be clearer to just call the variable "installable", which the port could set 
>> using any logic it requires, or to put such logic in a preflight phase?
> 
> I don’t have a problem with a new “installable” keyword. However clearing the 
> platforms variable and adding a comment for the few ports like p5-graveyard 
> seems reasonable to me and does not require more work. 

It requires work to change to MacPorts base to do something with the platforms 
variable, which it currently does not do.


>> Your idea to use string names for each version of macOS has the problem that 
>> it means a ton of updates have to happen each time a new version of macOS is 
>> released, which is annually now. Most ports that have macOS version 
>> restrictions either build only on old systems (your suggestion would work 
>> well for them), or only on new systems (your suggestion works less well 
>> there because e.g. when Sierra comes out, all those ports need "sierra" 
>> added). A better implementation would let the port indicate that a port 
>> builds on macOS 10.X or older, or macOS 10.Y or newer. This is what those 
>> ports currently do, by checking os.major.
> 
> Only ports that manually override the platforms variable would be required to 
> be updated for any new OS.

Yes, I understand. The default would be to be installable on all platforms.

> However, to be clear, one should only do that for ports that only run on an 
> old version of MacOS. So mostly Portfiles (that need to use this 
> functionality) should use platforms_remove which would then automatically 
> include the new OS in base. Usually we are removing old/unsupported OS 
> versions. 
> 
> platforms_remove tiger leopard snowleopard
> 
> So any correctly written Portfile would not need any changes for every new 
> version of MacOS. 

That's a good point, if portfile developers remember to do it that way. That 
resolves my objection.

The command (which already exists) is "platforms-delete". All "options" (the 
special variables MacPorts knows about that you can set in portfiles without 
using the "set" command) support the -append, -delete, -replace and -stdsed 
modifiers.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

On Jul 28, 2016, at 9:18 AM, Ken Cunningham wrote:

> OK -- So - I am absolutely nobody at all on this list, so please take this 
> following idea gently.
> 
> This work you're embarking on (specifying what systems a given port will 
> build on, and which it won't) would seem in fact to be very similar to what I 
> was getting at when I asked about the 'distributions' idea in that ticket I 
> opened a few weeks ago.
> 
> The only extra part of this that I was hoping for would the ability to peek 
> back in the port file history and find (and install) the last version of a 
> port file (apache2, say) that was able to be installed on a given system 
> (like SL). 
> 
> This ability would solve a lot of troubles for older systems, although there 
> could be the same dependencies issues that this idea will generate as well, 
> perhaps.
> 
> As macports moves forward, the caboose of which machines are left behind will 
> continue to move forward as well. But there are working portfiles back there 
> that are still good...
> 
> My $0.02.

These are interesting ideas, but would require major changes* to MacPorts and 
don't relate to the issue I'm trying to discuss in this thread.



* MacPorts currently has no idea that Portfiles have a "history"; it has no 
idea that Portfiles live in a Subversion repository and that earlier versions 
of a Portfile exist.

* MacPorts currently does not support users who install older versions of 
Portfiles. It may work, but if it does not, the user is on their own.

* The MacPorts project currently makes no guarantees that an old version of a 
Portfile is installable on the current version of MacPorts, and there have been 
many situations in the past where indeed old Portfiles cannot be used on 
current versions of MacPorts without manual changes.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ken Cunningham]

OK -- So - I am absolutely nobody at all on this list, so please take this 
following idea gently.

This work you're embarking on (specifying what systems a given port will build 
on, and which it won't) would seem in fact to be very similar to what I was 
getting at when I asked about the 'distributions' idea in that ticket I opened 
a few weeks ago.

The only extra part of this that I was hoping for would the ability to peek 
back in the port file history and find (and install) the last version of a port 
file (apache2, say) that was able to be installed on a given system (like SL). 

This ability would solve a lot of troubles for older systems, although there 
could be the same dependencies issues that this idea will generate as well, 
perhaps.

As macports moves forward, the caboose of which machines are left behind will 
continue to move forward as well. But there are working portfiles back there 
that are still good...

My $0.02.

Ken
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[mf2k]

> On Jul 27, 2016, at 6:19 PM, Ryan Schmidt  wrote:
> 
> Your suggestion seems like a variation of my "third possibility", which has 
> the disadvantages that every new reason why a port might not be installable 
> would need a new variable introduced in MacPorts base and a new release of 
> MacPorts base.

We already need to release a new version of base for Sierra. I am suggesting we 
deploy these changes then. 


> What about ports that need the user to install Java first? What about ports 
> that need the user to download a file manually first (for example 
> oracle-instantclient which requires users to agree to a license agreement 
> before downloading distfiles)?

Yes, so some kind of pre-fetch argument or new variables are needed for those 
cases. Given your argument about clear intent (which I completely agree with in 
general), I think that new keywords should be created for each case where 
possible. 

requires_java yes

That has the added complication of having base then determine if java is indeed 
installed which can differ per version of MacOS. 

The oracle-instantclient example is singular and thus should require custom 
code in the Portfile such as in a pre-fetch phase as you suggest. 


> Your idea for the p5-graveyard and similar ports to clear the platforms 
> variable to indicate the port is not installable is clever, but wouldn't it 
> be clearer to just call the variable "installable", which the port could set 
> using any logic it requires, or to put such logic in a preflight phase?

I don’t have a problem with a new “installable” keyword. However clearing the 
platforms variable and adding a comment for the few ports like p5-graveyard 
seems reasonable to me and does not require more work. 


> Your idea to use string names for each version of macOS has the problem that 
> it means a ton of updates have to happen each time a new version of macOS is 
> released, which is annually now. Most ports that have macOS version 
> restrictions either build only on old systems (your suggestion would work 
> well for them), or only on new systems (your suggestion works less well there 
> because e.g. when Sierra comes out, all those ports need "sierra" added). A 
> better implementation would let the port indicate that a port builds on macOS 
> 10.X or older, or macOS 10.Y or newer. This is what those ports currently do, 
> by checking os.major.

Only ports that manually override the platforms variable would be required to 
be updated for any new OS. However, to be clear, one should only do that for 
ports that only run on an old version of MacOS. So mostly Portfiles (that need 
to use this functionality) should use platforms_remove which would then 
automatically include the new OS in base. Usually we are removing 
old/unsupported OS versions. 

platforms_remove tiger leopard snowleopard

So any correctly written Portfile would not need any changes for every new 
version of MacOS. 


Cheers!
Frank

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Rainer Müller]

On 2016-07-28 09:06, Artur Szostak wrote:
> This sounds like there is a need to allow encoding of version ranges. In 
> which case I would suggest to solve this problem generically, to allow 
> encoding version ranges in port dependencies also. Similar to what is 
> possible with RPMs or Debian packages. If the logic is implemented for 
> dependencies then one could surely easily have things like the following 
> encoded in the Portfile:
> 
>   macosx >= 10.9
>   macosx < 10.10

https://trac.macports.org/ticket/15712

Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

On Jul 27, 2016, at 8:24 PM, Craig Treleaven wrote:

> On Jul 27, 2016, at 12:09 AM, Ryan Schmidt wrote:
> 
>> I would like MacPorts to be extended so that it can be determined in 
>> advance, from the command line, before running "port install", whether a 
>> port can be installed (barring unexpected build failures and bugs). 
>> Currently, MacPorts assumes any port can be installed on any system, which 
>> is not the case. Many ports cannot be built on certain versions of macOS. 
>> Some require libc++. Some require dependencies to be installed with a 
>> nondefault variant. Some have other requirements. Some ports, like 
>> p5-graveyard or other ports that serve as placeholders designed to inform 
>> the user of the discontinuation of a port, are designed to fail. Being able 
>> to determine in advance whether a port is supposed to be installable will 
>> let us skip those ports when triggering builds on the buildbot. This will 
>> cut down on unnecessary work performed by the buildbot, and will avoid 
>> unnecessary emails sent to maintainers who already know the port will fail 
>> in those circumstances.
>> 
>> How can we accomplish this? We currently use "return -code error" to trigger 
>> these kinds of error messages from ports, but we do so within a phase, such 
>> as pre-configure or pre-fetch, but that means that code doesn't run until 
>> those phases are running, and I want to know before those phases run, indeed 
>> even before dependencies are calculated and installed.
>> 
>> One solution that occurs to me is to define a new "preflight" phase, to be 
>> run before dependencies are computed. Ports can override that phase and do 
>> whatever checks they need and exit with "return -code error" if needed. This 
>> seems like the most straightforward and flexible solution.
>> 
>> Another possible solution could be to define a new Boolean variable 
>> "installable" to indicate if the port is installable, which would default to 
>> "yes". If a port sets this to "no", MacPorts could print a generic failure 
>> message. There could be a second variable which the port developer could set 
>> to a custom failure message.
>> 
>> A third possibility could be to codify each of the reasons why a build might 
>> fail, and introduce new variables for each reason. For example, a variable 
>> to indicate the supported C++ libraries, or a variable to indicate the 
>> supported macOS versions. There might be some advantage to this, in that it 
>> could be used to programmatically answer questions like what C++ libraries 
>> or macOS versions a port supports, but it is the least flexible and most 
>> complicated solution.
> 
> Could I submit that there may be two issues being conflated?
> 
> 1) Some ports are known to crap out on our buildbots.
> 
> 2) It is known that some ports cannot be installed on particular system 
> configurations.
> 
> For 1), I think we should simply add a keyword to the portfile that tells the 
> buildbot not to attempt to build this port.  “mp_buildbot_skip” or some such. 
>  Add this to the p5-graveyard and all other ports that we know the buildbot 
> cannot or should not attempt to build.
> 
> For 2), we already have a series of tools that provide information about the 
> environment and let the port do the right thing.  I like the idea of a 
> preflight phase—a number of ports ‘abuse’ the pre-fetch phase to essentially 
> do this.  However I’m not sure how prevalent or serious the problem is in 2) 
> that we’re trying to solve.

The buildbot should be a tool that helps developers identify when a port fails 
to build, so that the developers can fix it. I don't want to inundate 
developers with emails about build failures that they already know will happen. 
This will cause developers to ignore buildbot emails, and I don't want that.

I oppose making changes to portfiles that are specific to the buildbot. The 
buildbot is not special. It is just another computer with a MacPorts 
installation. Anything the buildbot needs, users need too.


> (And there is the -y flag to ‘dry-run’ the installation.)

Yes... Is there a way I'm not seeing for the -y flag to be used to accomplish 
what I want?

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

On Jul 28, 2016, at 2:06 AM, Artur Szostak wrote:

> "... A better implementation would let the port indicate that a port builds 
> on macOS 10.X or older, or macOS 10.Y or newer. ..."
> 
> This sounds like there is a need to allow encoding of version ranges.

Right, *if* we make a MacPorts base feature that allows or disallows building 
on some macOS versions, it should allow ranges. We could use the same syntax we 
use in the compiler_blacklist_versions portgroup for blacklisting ranges of 
clang compiler versions.

However, as I've said, there are other reasons why a port might not be able to 
install. Should we really modify MacPorts base and create new variables to 
accommodate each of those reasons? Is there a specific objection to what I 
consider the easiest and most flexible of the options I suggested: implement a 
new preflight phase, in which the port can run any logic to determine if the 
port can be installed?

> In which case I would suggest to solve this problem generically, to allow 
> encoding version ranges in port dependencies also. Similar to what is 
> possible with RPMs or Debian packages. If the logic is implemented for 
> dependencies then one could surely easily have things like the following 
> encoded in the Portfile:
> 
>  macosx >= 10.9
>  macosx < 10.10

Allowing version numbers or ranges in port dependencies is completely unrelated 
to this thread, and is completely outside the realm of possibility, because 
MacPorts does not have the capability to install a specific version of a port, 
because there is only one version of a portfile in existence at a time, so that 
is the only version MacPorts can install. I don't see this ever changing.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

RE: Determine before installation whether a port can be installed

[Artur Szostak]

"... A better implementation would let the port indicate that a port builds on 
macOS 10.X or older, or macOS 10.Y or newer. ..."

This sounds like there is a need to allow encoding of version ranges. In which 
case I would suggest to solve this problem generically, to allow encoding 
version ranges in port dependencies also. Similar to what is possible with RPMs 
or Debian packages. If the logic is implemented for dependencies then one could 
surely easily have things like the following encoded in the Portfile:

  macosx >= 10.9
  macosx < 10.10

Kind regards.

Artur
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Craig Treleaven]

> On Jul 27, 2016, at 12:09 AM, Ryan Schmidt  wrote:
> 
> I would like MacPorts to be extended so that it can be determined in advance, 
> from the command line, before running "port install", whether a port can be 
> installed (barring unexpected build failures and bugs). Currently, MacPorts 
> assumes any port can be installed on any system, which is not the case. Many 
> ports cannot be built on certain versions of macOS. Some require libc++. Some 
> require dependencies to be installed with a nondefault variant. Some have 
> other requirements. Some ports, like p5-graveyard or other ports that serve 
> as placeholders designed to inform the user of the discontinuation of a port, 
> are designed to fail. Being able to determine in advance whether a port is 
> supposed to be installable will let us skip those ports when triggering 
> builds on the buildbot. This will cut down on unnecessary work performed by 
> the buildbot, and will avoid unnecessary emails sent to maintainers who 
> already know the port will fail in those circumstances.
> 
> How can we accomplish this? We currently use "return -code error" to trigger 
> these kinds of error messages from ports, but we do so within a phase, such 
> as pre-configure or pre-fetch, but that means that code doesn't run until 
> those phases are running, and I want to know before those phases run, indeed 
> even before dependencies are calculated and installed.
> 
> One solution that occurs to me is to define a new "preflight" phase, to be 
> run before dependencies are computed. Ports can override that phase and do 
> whatever checks they need and exit with "return -code error" if needed. This 
> seems like the most straightforward and flexible solution.
> 
> Another possible solution could be to define a new Boolean variable 
> "installable" to indicate if the port is installable, which would default to 
> "yes". If a port sets this to "no", MacPorts could print a generic failure 
> message. There could be a second variable which the port developer could set 
> to a custom failure message.
> 
> A third possibility could be to codify each of the reasons why a build might 
> fail, and introduce new variables for each reason. For example, a variable to 
> indicate the supported C++ libraries, or a variable to indicate the supported 
> macOS versions. There might be some advantage to this, in that it could be 
> used to programmatically answer questions like what C++ libraries or macOS 
> versions a port supports, but it is the least flexible and most complicated 
> solution.

Could I submit that there may be two issues being conflated?

1) Some ports are known to crap out on our buildbots.

2) It is known that some ports cannot be installed on particular system 
configurations.

For 1), I think we should simply add a keyword to the portfile that tells the 
buildbot not to attempt to build this port.  “mp_buildbot_skip” or some such.  
Add this to the p5-graveyard and all other ports that we know the buildbot 
cannot or should not attempt to build.

For 2), we already have a series of tools that provide information about the 
environment and let the port do the right thing.  I like the idea of a 
preflight phase—a number of ports ‘abuse’ the pre-fetch phase to essentially do 
this.  However I’m not sure how prevalent or serious the problem is in 2) that 
we’re trying to solve.

Craig
(And there is the -y flag to ‘dry-run’ the installation.)
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Ryan Schmidt]

> On Jul 27, 2016, at 8:26 AM, m...@macports.org wrote:
> 
> Hi Ryan,
> 
> 
>> On Jul 26, 2016, at 10:09 PM, Ryan Schmidt  wrote:
>> 
>> I would like MacPorts to be extended so that it can be determined in 
>> advance, from the command line, before running "port install", whether a 
>> port can be installed (barring unexpected build failures and bugs). 
>> Currently, MacPorts assumes any port can be installed on any system, which 
>> is not the case. Many ports cannot be built on certain versions of macOS. 
>> Some require libc++. Some require dependencies to be installed with a 
>> nondefault variant. Some have other requirements. Some ports, like 
>> p5-graveyard or other ports that serve as placeholders designed to inform 
>> the user of the discontinuation of a port, are designed to fail. Being able 
>> to determine in advance whether a port is supposed to be installable will 
>> let us skip those ports when triggering builds on the buildbot. This will 
>> cut down on unnecessary work performed by the buildbot, and will avoid 
>> unnecessary emails sent to maintainers who already know the port will fail 
>> in those circumstances.
>> 
>> How can we accomplish this? We currently use "return -code error" to trigger 
>> these kinds of error messages from ports, but we do so within a phase, such 
>> as pre-configure or pre-fetch, but that means that code doesn't run until 
>> those phases are running, and I want to know before those phases run, indeed 
>> even before dependencies are calculated and installed.
>> 
>> One solution that occurs to me is to define a new "preflight" phase, to be 
>> run before dependencies are computed. Ports can override that phase and do 
>> whatever checks they need and exit with "return -code error" if needed. This 
>> seems like the most straightforward and flexible solution.
>> 
>> Another possible solution could be to define a new Boolean variable 
>> "installable" to indicate if the port is installable, which would default to 
>> "yes". If a port sets this to "no", MacPorts could print a generic failure 
>> message. There could be a second variable which the port developer could set 
>> to a custom failure message.
>> 
>> A third possibility could be to codify each of the reasons why a build might 
>> fail, and introduce new variables for each reason. For example, a variable 
>> to indicate the supported C++ libraries, or a variable to indicate the 
>> supported macOS versions. There might be some advantage to this, in that it 
>> could be used to programmatically answer questions like what C++ libraries 
>> or macOS versions a port supports, but it is the least flexible and most 
>> complicated solution.
> 
> In my mind, the existing and largely useless “platform" variable should be 
> re-purposed and used for this purpose. It currently usually says “darwin” but 
> that could be expanded for each version of MacOS. 
> 
> To implement this, I think the default value in base should be changed to the 
> following for the next version of Macports for Sierra:
> 
> platforms tiger leopard snowleopard lion mountainlion mavericks yosemite 
> elcapitan sierra
> 
> Then we remove the “platform” variable from all Portfiles when we release the 
> new version of base. For ports that cannot build on, for example, lion the 
> Portfile’s platforms variable can either be overridden as such:
> 
> platforms tiger leopard snowleopard mountainlion mavericks yosemite elcapitan 
> sierra
> 
> or we expand base to allow a construct like:
> 
> platforms-remove lion
> 
> Or maybe this functionality already exists?
> 
> For each new version of MacOS, we need to release a new version of base 
> anyway. That version of base will get the new MacOS version added to the 
> default “platforms” variable…
> 
> platforms tiger leopard snowleopard mountainlion mavericks yosemite elcapitan 
> sierra sequoia
> 
> For the libc++ issue, we simply add a keyword:
> 
> requires_libc++ yes
> 
> The p5-graveyard port(s) can set the platforms variable to blank. 

Your suggestion seems like a variation of my "third possibility", which has the 
disadvantages that every new reason why a port might not be installable would 
need a new variable introduced in MacPorts base and a new release of MacPorts 
base. What about ports that need the user to install Java first? What about 
ports that need the user to download a file manually first (for example 
oracle-instantclient which requires users to agree to a license agreement 
before downloading distfiles)? Your idea for the p5-graveyard and similar ports 
to clear the platforms variable to indicate the port is not installable is 
clever, but wouldn't it be clearer to just call the variable "installable", 
which the port could set using any logic it requires, or to put such logic in a 
preflight phase?

Your idea to use string names for each version of macOS has the problem that it 
means a ton of updates have to happen each time a new version of 

Re: Determine before installation whether a port can be installed

[mf2k]

Hi Ryan,


> On Jul 26, 2016, at 10:09 PM, Ryan Schmidt  wrote:
> 
> I would like MacPorts to be extended so that it can be determined in advance, 
> from the command line, before running "port install", whether a port can be 
> installed (barring unexpected build failures and bugs). Currently, MacPorts 
> assumes any port can be installed on any system, which is not the case. Many 
> ports cannot be built on certain versions of macOS. Some require libc++. Some 
> require dependencies to be installed with a nondefault variant. Some have 
> other requirements. Some ports, like p5-graveyard or other ports that serve 
> as placeholders designed to inform the user of the discontinuation of a port, 
> are designed to fail. Being able to determine in advance whether a port is 
> supposed to be installable will let us skip those ports when triggering 
> builds on the buildbot. This will cut down on unnecessary work performed by 
> the buildbot, and will avoid unnecessary emails sent to maintainers who 
> already know the port will fail in those circumstances.
> 
> How can we accomplish this? We currently use "return -code error" to trigger 
> these kinds of error messages from ports, but we do so within a phase, such 
> as pre-configure or pre-fetch, but that means that code doesn't run until 
> those phases are running, and I want to know before those phases run, indeed 
> even before dependencies are calculated and installed.
> 
> One solution that occurs to me is to define a new "preflight" phase, to be 
> run before dependencies are computed. Ports can override that phase and do 
> whatever checks they need and exit with "return -code error" if needed. This 
> seems like the most straightforward and flexible solution.
> 
> Another possible solution could be to define a new Boolean variable 
> "installable" to indicate if the port is installable, which would default to 
> "yes". If a port sets this to "no", MacPorts could print a generic failure 
> message. There could be a second variable which the port developer could set 
> to a custom failure message.
> 
> A third possibility could be to codify each of the reasons why a build might 
> fail, and introduce new variables for each reason. For example, a variable to 
> indicate the supported C++ libraries, or a variable to indicate the supported 
> macOS versions. There might be some advantage to this, in that it could be 
> used to programmatically answer questions like what C++ libraries or macOS 
> versions a port supports, but it is the least flexible and most complicated 
> solution.

In my mind, the existing and largely useless “platform" variable should be 
re-purposed and used for this purpose. It currently usually says “darwin” but 
that could be expanded for each version of MacOS. 

To implement this, I think the default value in base should be changed to the 
following for the next version of Macports for Sierra:

platforms tiger leopard snowleopard lion mountainlion mavericks yosemite 
elcapitan sierra

Then we remove the “platform” variable from all Portfiles when we release the 
new version of base. For ports that cannot build on, for example, lion the 
Portfile’s platforms variable can either be overridden as such:

platforms tiger leopard snowleopard mountainlion mavericks yosemite elcapitan 
sierra

or we expand base to allow a construct like:

platforms-remove lion

Or maybe this functionality already exists?

For each new version of MacOS, we need to release a new version of base anyway. 
That version of base will get the new MacOS version added to the default 
“platforms” variable…

platforms tiger leopard snowleopard mountainlion mavericks yosemite elcapitan 
sierra sequoia

For the libc++ issue, we simply add a keyword:

requires_libc++ yes

The p5-graveyard port(s) can set the platforms variable to blank. 


Cheers!
Frank

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev