Re: mpi

[Ryan Schmidt]

On Oct 1, 2014, at 12:23 AM, Sean Farley wrote:

> Ryan Schmidt writes:
> 
>> On Sep 30, 2014, at 11:25 PM, Sean Farley wrote:
>>> 
>>> Ryan Schmidt writes:
>>> 
 On Sep 30, 2014, at 10:48 PM, Sean Farley wrote:
> 
> Variants create a new node in the dependency DAG. Subports create a new
> node in the dependency DAG. There is no topological difference. The only
> difference is in how they were implemented.
 
 As far as I understand, variants are not involved in the dependency DAG in 
 MacPorts at this time. Only ports/subports are.
>>> 
>>> Variants *in MacPorts* are not involved in the DAG *due to
>>> implementation*. They do indeed change the graph.
>> 
>> I guess I'm not familiar with what graph you're talking about. I'm only 
>> concerned with how dependencies work in MacPorts, not some other real or 
>> hypothetical package manager.
> 
> I'm not talking about a package manager, just the dependency graph. What
> I'm trying to point out is that variants change the graph. For example,
> look at the wonky code of the dependents of cairo, checking for
> cairo+quartz vs. cairo+x11.

Agreed, quartz and x11 variants are another problem.


> It seems that you want to define variants as things that do not change
> the graph. If so, MacPorts would (and should) have to ensure that
> variant blocks don't change the dependencies (but then how to check the
> dependents of a port?).
> 
> You would have a simpler definition of a variant (something that enables
> an option but doesn't change the graph) but would end up with more
> (sub)ports.

A variant that adds a dependency on another port is not a problem. We do this 
all the time.

A variant that adds a dependency on a variant of another port is a problem, in 
that it's not possible, and the require_active_variants workaround is not ideal.


> It is understandable to not want to implement the greater complexity.
> Your proposal is to get rid of gcc for compiling C/C++. If that's the
> direction MacPorts wants to go, then I'd request that it be uniform: no
> port has any variant to change the C/C++ compilers.
 
 As I understand it, it is indeed our goal for ports not to have variants 
 that change the C/C++ compilers. However, many ports have gcc variants 
 from years ago before we fully understood the C++ standard library 
 mismatch issues and they have not been updated. For each port that has 
 compiler variants, we would need to examine them to see why they do that, 
 and how to safely remove them.
>>> 
>>> The thorny ports (perhaps more) will be ATLAS and boost. A good place to
>>> start would be to simplify the ports that use the compilers port group
>>> and refactor that to make it use the configure.compiler logic. After
>>> that, there are approximately 118 ports that use gcc as a compiler.
>> 
>> Yes, atlas has a bunch of compiler variants. I would love for most or all of 
>> them to go away. But part of atlas needs fortran so it needs to deal with 
>> that at least.
> 
> I would love for fortran to go away, too, but certain packages will need
> different compilers.
> 
>> I already "fixed" boost by removing the use of the mpi portgroup. If mpi 
>> support needs to be brought back, then a general solution needs to be found 
>> for the mpi portgroup, which based on what you said earlier might be as 
>> simple as deferring its work until the pre-configure block.
> 
> MPI support is definitely needed. I did not say earlier that would
> work. I said that would only work if you and MacPorts decide to get rid
> of gcc C/C++ compilers (and unify compilers-group with
> configure.compilers).
> 
> I am a little worried that you glossed over the amount of work needed to
> make a general solution. There are two big choices:
> 
> 1) does the MacPorts community decide to tackle variants, subports,
>   graphs, etc.?
> 
> 2) does the MacPorts community drop support for gcc as a C/C++ compiler?
> 
> (2) is certainly less work but would also lose some advantage over other
> package managers.

The moment OS X 10.9 came around, we basically lost the ability to effectively 
use G++ (and even before then there were some issues). It's out of our control; 
we're at the mercy of the C++ library Apple uses, and the fact that G++ does 
not use it.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 11:25 PM, Sean Farley wrote:
>> 
>> Ryan Schmidt writes:
>> 
>>> On Sep 30, 2014, at 10:48 PM, Sean Farley wrote:
 
 Variants create a new node in the dependency DAG. Subports create a new
 node in the dependency DAG. There is no topological difference. The only
 difference is in how they were implemented.
>>> 
>>> As far as I understand, variants are not involved in the dependency DAG in 
>>> MacPorts at this time. Only ports/subports are.
>> 
>> Variants *in MacPorts* are not involved in the DAG *due to
>> implementation*. They do indeed change the graph.
>
> I guess I'm not familiar with what graph you're talking about. I'm only 
> concerned with how dependencies work in MacPorts, not some other real or 
> hypothetical package manager.

I'm not talking about a package manager, just the dependency graph. What
I'm trying to point out is that variants change the graph. For example,
look at the wonky code of the dependents of cairo, checking for
cairo+quartz vs. cairo+x11.

It seems that you want to define variants as things that do not change
the graph. If so, MacPorts would (and should) have to ensure that
variant blocks don't change the dependencies (but then how to check the
dependents of a port?).

You would have a simpler definition of a variant (something that enables
an option but doesn't change the graph) but would end up with more
(sub)ports.

 It is understandable to not want to implement the greater complexity.
 Your proposal is to get rid of gcc for compiling C/C++. If that's the
 direction MacPorts wants to go, then I'd request that it be uniform: no
 port has any variant to change the C/C++ compilers.
>>> 
>>> As I understand it, it is indeed our goal for ports not to have variants 
>>> that change the C/C++ compilers. However, many ports have gcc variants from 
>>> years ago before we fully understood the C++ standard library mismatch 
>>> issues and they have not been updated. For each port that has compiler 
>>> variants, we would need to examine them to see why they do that, and how to 
>>> safely remove them.
>> 
>> The thorny ports (perhaps more) will be ATLAS and boost. A good place to
>> start would be to simplify the ports that use the compilers port group
>> and refactor that to make it use the configure.compiler logic. After
>> that, there are approximately 118 ports that use gcc as a compiler.
>
> Yes, atlas has a bunch of compiler variants. I would love for most or all of 
> them to go away. But part of atlas needs fortran so it needs to deal with 
> that at least.

I would love for fortran to go away, too, but certain packages will need
different compilers.

> I already "fixed" boost by removing the use of the mpi portgroup. If mpi 
> support needs to be brought back, then a general solution needs to be found 
> for the mpi portgroup, which based on what you said earlier might be as 
> simple as deferring its work until the pre-configure block.

MPI support is definitely needed. I did not say earlier that would
work. I said that would only work if you and MacPorts decide to get rid
of gcc C/C++ compilers (and unify compilers-group with
configure.compilers).

I am a little worried that you glossed over the amount of work needed to
make a general solution. There are two big choices:

1) does the MacPorts community decide to tackle variants, subports,
   graphs, etc.?

2) does the MacPorts community drop support for gcc as a C/C++ compiler?

(2) is certainly less work but would also lose some advantage over other
package managers.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 11:25 PM, Sean Farley wrote:
> 
> Ryan Schmidt writes:
> 
>> On Sep 30, 2014, at 10:48 PM, Sean Farley wrote:
>>> 
>>> Variants create a new node in the dependency DAG. Subports create a new
>>> node in the dependency DAG. There is no topological difference. The only
>>> difference is in how they were implemented.
>> 
>> As far as I understand, variants are not involved in the dependency DAG in 
>> MacPorts at this time. Only ports/subports are.
> 
> Variants *in MacPorts* are not involved in the DAG *due to
> implementation*. They do indeed change the graph.

I guess I'm not familiar with what graph you're talking about. I'm only 
concerned with how dependencies work in MacPorts, not some other real or 
hypothetical package manager.


>>> It is understandable to not want to implement the greater complexity.
>>> Your proposal is to get rid of gcc for compiling C/C++. If that's the
>>> direction MacPorts wants to go, then I'd request that it be uniform: no
>>> port has any variant to change the C/C++ compilers.
>> 
>> As I understand it, it is indeed our goal for ports not to have variants 
>> that change the C/C++ compilers. However, many ports have gcc variants from 
>> years ago before we fully understood the C++ standard library mismatch 
>> issues and they have not been updated. For each port that has compiler 
>> variants, we would need to examine them to see why they do that, and how to 
>> safely remove them.
> 
> The thorny ports (perhaps more) will be ATLAS and boost. A good place to
> start would be to simplify the ports that use the compilers port group
> and refactor that to make it use the configure.compiler logic. After
> that, there are approximately 118 ports that use gcc as a compiler.

Yes, atlas has a bunch of compiler variants. I would love for most or all of 
them to go away. But part of atlas needs fortran so it needs to deal with that 
at least. And I believe Vince argued that for atlas and other 
performance-critical scientific software it is beneficial to be able to try 
different compilers to get the absolute best performance. I'm not sure who 
actually has time to do that however. Also, atlas already builds itself 
multiple times with different compiler options to get best performance, which 
is why building atlas takes so long.

I already "fixed" boost by removing the use of the mpi portgroup. If mpi 
support needs to be brought back, then a general solution needs to be found for 
the mpi portgroup, which based on what you said earlier might be as simple as 
deferring its work until the pre-configure block.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 10:48 PM, Sean Farley wrote:
>> 
>> Ryan Schmidt writes:
>> 
>>> On Sep 30, 2014, at 9:25 PM, Sean Farley wrote:
 
> That's not what variants are for. That's what subports are for.
 
 Subports are trying to solve this but expose the implementation level
 too far up. Only one solution should exist to depending on a
 variant. Subports, unfortunately, put all the burden on the portfile
 author and bloat the output of the list of ports.
 
 $ port echo mpich*
 mpich
 mpich-clang
 mpich-clang31
 mpich-clang32
 mpich-clang33
 mpich-clang34
 mpich-clang35
 mpich-default
 mpich-devel
 mpich-devel-clang
 mpich-devel-clang31
 mpich-devel-clang32
 mpich-devel-clang33
 mpich-devel-clang34
 mpich-devel-clang35
 mpich-devel-default
 mpich-devel-dragonegg31
 mpich-devel-dragonegg32
 mpich-devel-dragonegg33
 mpich-devel-gcc43
 mpich-devel-gcc44
 mpich-devel-gcc45
 mpich-devel-gcc46
 mpich-devel-gcc47
 mpich-devel-gcc48
 mpich-devel-gcc49
 mpich-devel-llvm
 mpich-dragonegg31
 mpich-dragonegg32
 mpich-dragonegg33
 mpich-gcc43
 mpich-gcc44
 mpich-gcc45
 mpich-gcc46
 mpich-gcc47
 mpich-gcc48
 mpich-gcc49
 mpich-llvm
 
 Why should this be exposed to the user? Imagine, now, if there were no
 such thing as a variant. This would solve all the dependence issues on a
 port's variant.
>>> 
>>> I'm trying to explain that this is not what variants were made for.
>>> 
>>> Variants are supposed to be for adding optional functionality, or choosing 
>>> between options.
>> 
>> Variants create a new node in the dependency DAG. Subports create a new
>> node in the dependency DAG. There is no topological difference. The only
>> difference is in how they were implemented.
>
> As far as I understand, variants are not involved in the dependency DAG in 
> MacPorts at this time. Only ports/subports are.

Variants *in MacPorts* are not involved in the DAG *due to
implementation*. They do indeed change the graph.

>> It is understandable to not want to implement the greater complexity.
>> Your proposal is to get rid of gcc for compiling C/C++. If that's the
>> direction MacPorts wants to go, then I'd request that it be uniform: no
>> port has any variant to change the C/C++ compilers.
>
> As I understand it, it is indeed our goal for ports not to have variants that 
> change the C/C++ compilers. However, many ports have gcc variants from years 
> ago before we fully understood the C++ standard library mismatch issues and 
> they have not been updated. For each port that has compiler variants, we 
> would need to examine them to see why they do that, and how to safely remove 
> them.

The thorny ports (perhaps more) will be ATLAS and boost. A good place to
start would be to simplify the ports that use the compilers port group
and refactor that to make it use the configure.compiler logic. After
that, there are approximately 118 ports that use gcc as a compiler.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Brandon Allbery]

On Wed, Oct 1, 2014 at 12:16 AM, Ryan Schmidt 
wrote:

> > Said mismatch issues didn't exist until Apple switched the default
> compiler to clang and default runtime to libc++, in 10.9. In particular,
> while clang was the default compiler on 10.8, libstdc++ was still the
> default runtime; it was 10.9 that switched runtimes and thereby broke most
> of MacPorts' C++ ecosystem with g++.
>
> The mismatch issues existed before 10.9 in some cases, they were just more
> difficult to spot because they resulted in unusual runtime behavior, rather
> than a clear compile-time error.


Ah, I was unaware of this.

(And this is why I'm uncomfortable with being the apparent "expert" on the
whole thing. I have this nasty suspicion that the only people involved with
MacPorts who really understand the details are afraid they'd have to run
any comment past Apple's lawyers first.)

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 11:10 PM, Brandon Allbery wrote:
> 
> On Wed, Oct 1, 2014 at 12:04 AM, Ryan Schmidt wrote:
>> However, many ports have gcc variants from years ago before we fully 
>> understood the C++ standard library mismatch issues
> 
> Said mismatch issues didn't exist until Apple switched the default compiler 
> to clang and default runtime to libc++, in 10.9. In particular, while clang 
> was the default compiler on 10.8, libstdc++ was still the default runtime; it 
> was 10.9 that switched runtimes and thereby broke most of MacPorts' C++ 
> ecosystem with g++.

The mismatch issues existed before 10.9 in some cases, they were just more 
difficult to spot because they resulted in unusual runtime behavior, rather 
than a clear compile-time error.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Brandon Allbery]

On Wed, Oct 1, 2014 at 12:04 AM, Ryan Schmidt 
wrote:

> However, many ports have gcc variants from years ago before we fully
> understood the C++ standard library mismatch issues


Said mismatch issues didn't exist until Apple switched the default compiler
to clang and default runtime to libc++, in 10.9. In particular, while clang
was the default compiler on 10.8, libstdc++ was still the default runtime;
it was 10.9 that switched runtimes and thereby broke most of MacPorts' C++
ecosystem with g++.

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 10:35 PM, Sean Farley wrote:
>
>> Lawrence Velázquez writes:
>> 
>>> On Sep 30, 2014, at 10:52 PM, Ryan Schmidt wrote:
>>> 
 I understand the proposal, but I don't agree that we should implement it.
>>> 
>>> It does seem like it would introduce a great deal of complexity to address 
>>> what is clearly (to me, at least) an antipattern (using variants to select 
>>> build options).
>> 
>> The complexity is much greater, indeed. And not something I wish to
>> implement in tcl.
>
> It's not just the complexity of modifying base. You're proposing that instead 
> of each port being installed to the same /opt/local prefix, each port would 
> be installed to its own prefix. This would probably require rebuilding all 
> ports, not to mention manually editing most of the portfiles (to change how 
> they find their dependencies), of which there are about 10,000 at this point. 
> I just don't see how anybody's ever going to have the time to do that.

Yes, might as well start over at that point :-)
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Sean Farley]

Ryan Schmidt writes:

>> That sounds right. What I'm attempting to improve (or, rather, stir up
>> awareness) are the many steps that are manual. Being that stealth
>> updates are infrequent, I usually forgot to check for them. I think 
>> 
>> A workflow I am trying to strive for (which maybe should be another email):
>> 
>> Task: update outdated ports
>> 
>> $ port echo maintainer:sean | parallel --trim lr port livecheck {}
>> mercurial seems to have been updated (port version: 3.0, new version: 3.1)
>
> For this I use my portmylivecheck script:
>
> https://svn.macports.org/repository/macports/users/ryandesign/scripts/
>
>> $ port maintainerupdate mercurial
>>  # this would replace the previous version string with the new one and
>>  # accept the checksum changes
>
> For this I use my portcheckup script:
>
> https://svn.macports.org/repository/macports/users/ryandesign/scripts/

I have my own scripts (some based of yours) that help me but I would
suggest something more or less built-in (or at the very least installed
in /opt/local somewhere and easy to find).

I believe that would help us guide ticket reporters to take a more
active role in contributing patches.

>> $ hg ci -m "mercurial: update to 3.1"
>> $ hg push try-server
>
> If you're suggesting this just as a personal workflow for you, ok. But if 
> you're suggesting this would be an infrastructure available to everyone, then 
> we're either back at the old discussion of switching the MacPorts repository 
> from Subversion to git or mercurial or something else, which we agreed before 
> was too much effort, or we're talking about introducing a second version 
> control system into the mix somehow, which sounds really confusing to me.

I mean more of a having a fleet of buildbot(-ish) servers to help
developers run a suite of tests. I personally do not have enough
machines to test every OS. I am willing to pay for some virtual private
servers but I think there would need to be some improvement of MacPorts
infrastructure to allow easy deploy on test machines.

>> # this is where a buildbot would test the change and tell me a few
>> # common issues:
>> #   - was there a stealth update
>
> We could perhaps enhance the buildbot scripts to re-fetch the upstream 
> distfile every time you commit the port and to let you know if its checksums 
> no longer match those in the portfile. But if so, only a human can determine 
> if that was because of a stealth update or because of something else. It 
> feels like this would be a waste of bandwidth and server resources, however.

This would at least save you the trouble of emailing the port author.

>> #   - did I "Use The Right Compiler"
>
> You can of course use the setup documented at the bottom of the 
> UsingTheRightCompiler wiki page to determine this on your local machine prior 
> to committing.

This is manual and therefore missed by some developers (including me
when I got a new machine).

> If you're wanting the buildbot to automate testing this for you, then I 
> suppose instead of the current placeholder script that prints an error and 
> exits (which can sometimes be missed because the output is discarded or 
> hidden on a config.log or other log file), we could have the placeholder 
> script create a log file of its own, followed by actually invoking the 
> compiler so that the build succeeds anyway, then do something with the 
> logfile later.

Yes, this would be a great addition. Having better output, in general,
would be helpful for tickets, too.

>> #   - run "port test" (or something similar)
>
> Yup, we've discussed previously that it would be nice if tests could be 
> automatically run, however many ports' tests do currently fail, and some 
> others take a very very very long time.

My hope would be for these failures to get better coverage (or in some
cases not tested, e.g. ATLAS).
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 10:48 PM, Sean Farley wrote:
> 
> Ryan Schmidt writes:
> 
>> On Sep 30, 2014, at 9:25 PM, Sean Farley wrote:
>>> 
 That's not what variants are for. That's what subports are for.
>>> 
>>> Subports are trying to solve this but expose the implementation level
>>> too far up. Only one solution should exist to depending on a
>>> variant. Subports, unfortunately, put all the burden on the portfile
>>> author and bloat the output of the list of ports.
>>> 
>>> $ port echo mpich*
>>> mpich
>>> mpich-clang
>>> mpich-clang31
>>> mpich-clang32
>>> mpich-clang33
>>> mpich-clang34
>>> mpich-clang35
>>> mpich-default
>>> mpich-devel
>>> mpich-devel-clang
>>> mpich-devel-clang31
>>> mpich-devel-clang32
>>> mpich-devel-clang33
>>> mpich-devel-clang34
>>> mpich-devel-clang35
>>> mpich-devel-default
>>> mpich-devel-dragonegg31
>>> mpich-devel-dragonegg32
>>> mpich-devel-dragonegg33
>>> mpich-devel-gcc43
>>> mpich-devel-gcc44
>>> mpich-devel-gcc45
>>> mpich-devel-gcc46
>>> mpich-devel-gcc47
>>> mpich-devel-gcc48
>>> mpich-devel-gcc49
>>> mpich-devel-llvm
>>> mpich-dragonegg31
>>> mpich-dragonegg32
>>> mpich-dragonegg33
>>> mpich-gcc43
>>> mpich-gcc44
>>> mpich-gcc45
>>> mpich-gcc46
>>> mpich-gcc47
>>> mpich-gcc48
>>> mpich-gcc49
>>> mpich-llvm
>>> 
>>> Why should this be exposed to the user? Imagine, now, if there were no
>>> such thing as a variant. This would solve all the dependence issues on a
>>> port's variant.
>> 
>> I'm trying to explain that this is not what variants were made for.
>> 
>> Variants are supposed to be for adding optional functionality, or choosing 
>> between options.
> 
> Variants create a new node in the dependency DAG. Subports create a new
> node in the dependency DAG. There is no topological difference. The only
> difference is in how they were implemented.

As far as I understand, variants are not involved in the dependency DAG in 
MacPorts at this time. Only ports/subports are.


> It is understandable to not want to implement the greater complexity.
> Your proposal is to get rid of gcc for compiling C/C++. If that's the
> direction MacPorts wants to go, then I'd request that it be uniform: no
> port has any variant to change the C/C++ compilers.

As I understand it, it is indeed our goal for ports not to have variants that 
change the C/C++ compilers. However, many ports have gcc variants from years 
ago before we fully understood the C++ standard library mismatch issues and 
they have not been updated. For each port that has compiler variants, we would 
need to examine them to see why they do that, and how to safely remove them.




___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 10:35 PM, Sean Farley wrote:

> Lawrence Velázquez writes:
> 
>> On Sep 30, 2014, at 10:52 PM, Ryan Schmidt wrote:
>> 
>>> I understand the proposal, but I don't agree that we should implement it.
>> 
>> It does seem like it would introduce a great deal of complexity to address 
>> what is clearly (to me, at least) an antipattern (using variants to select 
>> build options).
> 
> The complexity is much greater, indeed. And not something I wish to
> implement in tcl.

It's not just the complexity of modifying base. You're proposing that instead 
of each port being installed to the same /opt/local prefix, each port would be 
installed to its own prefix. This would probably require rebuilding all ports, 
not to mention manually editing most of the portfiles (to change how they find 
their dependencies), of which there are about 10,000 at this point. I just 
don't see how anybody's ever going to have the time to do that.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 9:25 PM, Sean Farley wrote:
>> 
>>> That's not what variants are for. That's what subports are for.
>> 
>> Subports are trying to solve this but expose the implementation level
>> too far up. Only one solution should exist to depending on a
>> variant. Subports, unfortunately, put all the burden on the portfile
>> author and bloat the output of the list of ports.
>> 
>> $ port echo mpich*
>> mpich
>> mpich-clang
>> mpich-clang31
>> mpich-clang32
>> mpich-clang33
>> mpich-clang34
>> mpich-clang35
>> mpich-default
>> mpich-devel
>> mpich-devel-clang
>> mpich-devel-clang31
>> mpich-devel-clang32
>> mpich-devel-clang33
>> mpich-devel-clang34
>> mpich-devel-clang35
>> mpich-devel-default
>> mpich-devel-dragonegg31
>> mpich-devel-dragonegg32
>> mpich-devel-dragonegg33
>> mpich-devel-gcc43
>> mpich-devel-gcc44
>> mpich-devel-gcc45
>> mpich-devel-gcc46
>> mpich-devel-gcc47
>> mpich-devel-gcc48
>> mpich-devel-gcc49
>> mpich-devel-llvm
>> mpich-dragonegg31
>> mpich-dragonegg32
>> mpich-dragonegg33
>> mpich-gcc43
>> mpich-gcc44
>> mpich-gcc45
>> mpich-gcc46
>> mpich-gcc47
>> mpich-gcc48
>> mpich-gcc49
>> mpich-llvm
>> 
>> Why should this be exposed to the user? Imagine, now, if there were no
>> such thing as a variant. This would solve all the dependence issues on a
>> port's variant.
>
> I'm trying to explain that this is not what variants were made for.
>
> Variants are supposed to be for adding optional functionality, or choosing 
> between options.

Variants create a new node in the dependency DAG. Subports create a new
node in the dependency DAG. There is no topological difference. The only
difference is in how they were implemented.

It is understandable to not want to implement the greater complexity.
Your proposal is to get rid of gcc for compiling C/C++. If that's the
direction MacPorts wants to go, then I'd request that it be uniform: no
port has any variant to change the C/C++ compilers.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Lawrence Velázquez writes:

> On Sep 30, 2014, at 10:52 PM, Ryan Schmidt  wrote:
>
>> I understand the proposal, but I don't agree that we should implement it.
>
> It does seem like it would introduce a great deal of complexity to address 
> what is clearly (to me, at least) an antipattern (using variants to select 
> build options).

The complexity is much greater, indeed. And not something I wish to
implement in tcl.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Ryan Schmidt]

> On Sep 30, 2014, at 8:55 PM, Sean Farley wrote:
> 
> 
> Ryan Schmidt writes:
> 
 * It assumes the port's revision will be increased. That's not always 
 necessary. Sometimes the only change in the distfile is the name of (or 
 presence of) the enclosing directory, or there are only changes in 
 comments or documentation files or other files that don't affect the 
 build, and in those cases there would be zero benefit to increasing the 
 port's revision and forcing everyone to rebuild or redownload.
>>> 
>>> That's true. In this case, we should borrow an idea from DVCSes: use a
>>> hash to see if something has changed (probably something like a Merkle
>>> tree).
>> 
>> I'm not familiar with a Merkle tree. I skimmed the wikipedia page but I'm 
>> not sure what you're suggesting here. What should be hashed, by whom, at 
>> what time?
> 
> It's just a fancy way to help detect if a tree has changed. Might not be
> helpful here, exactly.
> 
>>> This would be a nontrivial amount of work but would help mitigate
>>> the whole 'epoch' business.
>> 
>> I don't see what the epoch field has to do with this discussion at all.
> 
> It's an orthogonal discussion which I'll save for another time.
> 
 * The old distfile would get removed from our mirrors, making it 
 impossible for anyone to later determine what exactly changed in the 
 stealth update. Many maintainers handle stealth updates incorrectly. It is 
 nice for others to be able to investigate after the fact.
>>> 
>>> If we used hashes in the distfile naming, we wouldn't be overwriting
>>> things.
>> 
>> Ok, that's another good suggestion. I've seen that approach used by debian 
>> which puts each distfile in a directory whose name is the hash of the 
>> distfile.
>> 
>> ~
>> 
>> 
>> Let me just once again make sure that I understand the situation that we're 
>> discussing here. Here are the facts as I understand them. Let me know if 
>> I've misunderstood our goal in this discussion or if there are any other 
>> aspects.
>> 
>> 
>> 1. It is the maintainer's responsibility to notice when a stealth update has 
>> occurred.
>> 
>> 2. It is the maintainer's responsibility to obtain the new distfile and 
>> place its new checksums into the portfile, after verifying that the new 
>> distfile is correct and is not malicious.
>> 
>> 3. It is the maintainer's responsibility to decide whether this change 
>> requires users to rebuild their ports, and on the basis of this 
>> determination to increase the port's revision or not.
>> 
>> 4. It is considered too onerous for the maintainer to need to insert the 
>> line "dist_subdir ${name}/${version}_1" into the portfile when a stealth 
>> update occurs, and we would like to make this step unnecessary.
> 
> That sounds right. What I'm attempting to improve (or, rather, stir up
> awareness) are the many steps that are manual. Being that stealth
> updates are infrequent, I usually forgot to check for them. I think 
> 
> A workflow I am trying to strive for (which maybe should be another email):
> 
> Task: update outdated ports
> 
> $ port echo maintainer:sean | parallel --trim lr port livecheck {}
> mercurial seems to have been updated (port version: 3.0, new version: 3.1)

For this I use my portmylivecheck script:

https://svn.macports.org/repository/macports/users/ryandesign/scripts/

> $ port maintainerupdate mercurial
>  # this would replace the previous version string with the new one and
>  # accept the checksum changes

For this I use my portcheckup script:

https://svn.macports.org/repository/macports/users/ryandesign/scripts/

> $ hg ci -m "mercurial: update to 3.1"
> $ hg push try-server

If you're suggesting this just as a personal workflow for you, ok. But if 
you're suggesting this would be an infrastructure available to everyone, then 
we're either back at the old discussion of switching the MacPorts repository 
from Subversion to git or mercurial or something else, which we agreed before 
was too much effort, or we're talking about introducing a second version 
control system into the mix somehow, which sounds really confusing to me.


> # this is where a buildbot would test the change and tell me a few
> # common issues:
> #   - was there a stealth update

We could perhaps enhance the buildbot scripts to re-fetch the upstream distfile 
every time you commit the port and to let you know if its checksums no longer 
match those in the portfile. But if so, only a human can determine if that was 
because of a stealth update or because of something else. It feels like this 
would be a waste of bandwidth and server resources, however.

> #   - did I "Use The Right Compiler"

You can of course use the setup documented at the bottom of the 
UsingTheRightCompiler wiki page to determine this on your local machine prior 
to committing.

If you're wanting the buildbot to automate testing this for you, then I suppose 
instead of the current placeholder script that prints an error and 

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 9:25 PM, Sean Farley wrote:
> 
>> That's not what variants are for. That's what subports are for.
> 
> Subports are trying to solve this but expose the implementation level
> too far up. Only one solution should exist to depending on a
> variant. Subports, unfortunately, put all the burden on the portfile
> author and bloat the output of the list of ports.
> 
> $ port echo mpich*
> mpich
> mpich-clang
> mpich-clang31
> mpich-clang32
> mpich-clang33
> mpich-clang34
> mpich-clang35
> mpich-default
> mpich-devel
> mpich-devel-clang
> mpich-devel-clang31
> mpich-devel-clang32
> mpich-devel-clang33
> mpich-devel-clang34
> mpich-devel-clang35
> mpich-devel-default
> mpich-devel-dragonegg31
> mpich-devel-dragonegg32
> mpich-devel-dragonegg33
> mpich-devel-gcc43
> mpich-devel-gcc44
> mpich-devel-gcc45
> mpich-devel-gcc46
> mpich-devel-gcc47
> mpich-devel-gcc48
> mpich-devel-gcc49
> mpich-devel-llvm
> mpich-dragonegg31
> mpich-dragonegg32
> mpich-dragonegg33
> mpich-gcc43
> mpich-gcc44
> mpich-gcc45
> mpich-gcc46
> mpich-gcc47
> mpich-gcc48
> mpich-gcc49
> mpich-llvm
> 
> Why should this be exposed to the user? Imagine, now, if there were no
> such thing as a variant. This would solve all the dependence issues on a
> port's variant.

I'm trying to explain that this is not what variants were made for.

Variants are supposed to be for adding optional functionality, or choosing 
between options.

For example, the php56-mysql port offers variants for selecting which mysql 
library to use to communicate with a mysql server. Ports that depend on 
php56-mysql don't care which of those variants is used, and it only makes sense 
to use one of them at a time. So it is perfectly reasonable that the user can 
only install one instance of the port.

It is, however, conceivable that the user may wish to install the php mysql 
module for more than one version of php simultaneously, and this is 
accomplished through subports: php53-mysql, php54-mysql, php55-mysql, 
php56-mysql.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Lawrence Velázquez]

On Sep 30, 2014, at 10:52 PM, Ryan Schmidt  wrote:

> I understand the proposal, but I don't agree that we should implement it.

It does seem like it would introduce a great deal of complexity to address what 
is clearly (to me, at least) an antipattern (using variants to select build 
options).

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

> On Sep 30, 2014, at 9:44 PM, Lawrence Velázquez  wrote:
> 
> On Sep 30, 2014, at 10:09 PM, Ryan Schmidt  wrote:
> 
>> On Sep 30, 2014, at 9:08 PM, Sean Farley wrote:
>> 
>>> Ryan Schmidt writes:
>>> 
> The question, to me, is: why is it still not
> possible to distinguish foo+gcc and foo+clang in MacPorts?
 
 I'm not sure what you mean.
>>> 
>>> Why can't all a port's variants be installed at the same time?
>>> 
>>> $ port install boost
>>> $ port install boost +gcc48
>>> 
>>> Every port could have its own custom prefix and only the active one
>>> would be a symlink in /opt/local.
>> 
>> That's not what variants are for. That's what subports are for.
> 
> Subports don't install files differently from regular ports, though.

That is correct.

> Sean's proposal (as I understand it) is more or less how Homebrew installs 
> things. You'd end up with something like this:
> 
> /opt/local/
> ├── bin/
> │   └── foo -> ../dports/foo+bar/bin/foo
> ├── etc/
> ├── lib/
> │   └── libfoo.dylib -> ../dports/foo+bar/lib/libfoo.dylib
> ├── dports/
> │   ├── foo+bar/
> │   │   ├── bin/
> │   │   │   └── foo
> │   │   ├── etc/
> │   │   ├── lib/
> │   │   │   └── libfoo.dylib
> │   │   ├── share/
> │   │   └── var/
> │   └── foo+baz/
> │   ├── bin/
> │   │   └── foo
> │   ├── etc/
> │   ├── lib/
> │   │   └── libfoo.dylib
> │   ├── share/
> │   └── var/
> ├── share/
> └── var/
> 
> MacPorts keeps simultaneously-installed (but not active) ports in archives; 
> Homebrew keeps them in subdirectories.

I understand the proposal, but I don't agree that we should implement it.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Lawrence Velázquez]

On Sep 30, 2014, at 10:09 PM, Ryan Schmidt  wrote:

> On Sep 30, 2014, at 9:08 PM, Sean Farley wrote:
> 
>> Ryan Schmidt writes:
>> 
 The question, to me, is: why is it still not
 possible to distinguish foo+gcc and foo+clang in MacPorts?
>>> 
>>> I'm not sure what you mean.
>> 
>> Why can't all a port's variants be installed at the same time?
>> 
>> $ port install boost
>> $ port install boost +gcc48
>> 
>> Every port could have its own custom prefix and only the active one
>> would be a symlink in /opt/local.
> 
> That's not what variants are for. That's what subports are for.

Subports don't install files differently from regular ports, though. Sean's 
proposal (as I understand it) is more or less how Homebrew installs things. 
You'd end up with something like this:

/opt/local/
├── bin/
│   └── foo -> ../dports/foo+bar/bin/foo
├── etc/
├── lib/
│   └── libfoo.dylib -> ../dports/foo+bar/lib/libfoo.dylib
├── dports/
│   ├── foo+bar/
│   │   ├── bin/
│   │   │   └── foo
│   │   ├── etc/
│   │   ├── lib/
│   │   │   └── libfoo.dylib
│   │   ├── share/
│   │   └── var/
│   └── foo+baz/
│   ├── bin/
│   │   └── foo
│   ├── etc/
│   ├── lib/
│   │   └── libfoo.dylib
│   ├── share/
│   └── var/
├── share/
└── var/

MacPorts keeps simultaneously-installed (but not active) ports in archives; 
Homebrew keeps them in subdirectories.

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Brandon Allbery writes:

> On Tue, Sep 30, 2014 at 10:08 PM, Sean Farley  wrote:
>
>> Why can't all a port's variants be installed at the same time?
>
>
> This wouldn't solve problems like, if you get anywhere near any of the
> Apple C++ frameworks, you just lost the ability to use g++ in any way,
> shape, or form on 10.9.

I agree but for the sake of simplicity, I'd like to keep the domain
restricted to MacPorts-only software (e.g. mpi -> boost -> dolfin
problems).
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 9:08 PM, Sean Farley wrote:
>
>> Ryan Schmidt writes:
>> 
 The question, to me, is: why is it still not
 possible to distinguish foo+gcc and foo+clang in MacPorts?
>>> 
>>> I'm not sure what you mean.
>> 
>> Why can't all a port's variants be installed at the same time?
>> 
>> $ port install boost
>> $ port install boost +gcc48
>> 
>> Every port could have its own custom prefix and only the active one
>> would be a symlink in /opt/local.
>
> That's not what variants are for. That's what subports are for.

Subports are trying to solve this but expose the implementation level
too far up. Only one solution should exist to depending on a
variant. Subports, unfortunately, put all the burden on the portfile
author and bloat the output of the list of ports.

$ port echo mpich*
mpich
mpich-clang
mpich-clang31
mpich-clang32
mpich-clang33
mpich-clang34
mpich-clang35
mpich-default
mpich-devel
mpich-devel-clang
mpich-devel-clang31
mpich-devel-clang32
mpich-devel-clang33
mpich-devel-clang34
mpich-devel-clang35
mpich-devel-default
mpich-devel-dragonegg31
mpich-devel-dragonegg32
mpich-devel-dragonegg33
mpich-devel-gcc43
mpich-devel-gcc44
mpich-devel-gcc45
mpich-devel-gcc46
mpich-devel-gcc47
mpich-devel-gcc48
mpich-devel-gcc49
mpich-devel-llvm
mpich-dragonegg31
mpich-dragonegg32
mpich-dragonegg33
mpich-gcc43
mpich-gcc44
mpich-gcc45
mpich-gcc46
mpich-gcc47
mpich-gcc48
mpich-gcc49
mpich-llvm

Why should this be exposed to the user? Imagine, now, if there were no
such thing as a variant. This would solve all the dependence issues on a
port's variant.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: OT: sed/awk pointers needed

[Craig Treleaven]

At 2:27 PM -0400 9/30/14, Lawrence Velázquez wrote:

On Sep 30, 2014, at 12:00 PM, Brandon Allbery  wrote:
 > On Tue, Sep 30, 2014 at 11:44 AM, Craig 
Treleaven  wrote:
 >> I have only rudimentary acquaintance with 
sed and less with awk.  I'd like to learn more 
and this seemed like an opportunity to do so. 
[...]

 >
 > Very, very painfully. I think I could do it, 
with a LOT of fiddling, but would very much 
prefer to use a more appropriate tool. awk is 
somewhat better, but I'd reach for 
perl/python/ruby first.


[vq's sed version deleted ...]


As I said, it was a learning exercise and I've 
got a working version with gawk.  (Awk lacks the 
sub()/gsub() string functions.)


My version is:

-cut from here---
# identify port name lines by the phrase " has the variants"
/ has the variants/ {
 portname = $1
 next
}

# deal with the default variant lines, they start with [+]
# (which is a crazy string to match with a regex)
# note that the regex in the pattern match and the sub() function are
# (must be) wildly different!
/^\[+./ {
  sub("\\[\\+.", "   ")
  print portname ", "  $1 ", " "Default"
  next
}

# All the other variant lines start with 3 spaces, followed by a lower case
# variant name
/^   [a-z]/ {
  print portname ", "  $1 ", " "N"
  next
}
-to here

And it works as follows:
$ port variants php53-mysql php54-mysql |grep -v 
conflicts |grep -e mariadb -e mysql -e percona | 
cut -d : -f 1 | gawk -f rejig.awk -

php53-mysql, mariadb, N
php53-mysql, mysql4, N
php53-mysql, mysql5, N
php53-mysql, mysql51, N
php53-mysql, mysql55, N
php53-mysql, mysql56, N
php53-mysql, mysqlnd, Default
php53-mysql, percona, N
php54-mysql, mariadb, N
php54-mysql, mysql4, N
php54-mysql, mysql5, N
php54-mysql, mysql51, N
php54-mysql, mysql55, N
php54-mysql, mysql56, N
php54-mysql, mysqlnd, Default
php54-mysql, percona, N

Now to see if I can actually run this against the 
89 ports that appear to have mysql-related 
variants...


Craig
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Brandon Allbery]

On Tue, Sep 30, 2014 at 10:08 PM, Sean Farley  wrote:

> Why can't all a port's variants be installed at the same time?


This wouldn't solve problems like, if you get anywhere near any of the
Apple C++ frameworks, you just lost the ability to use g++ in any way,
shape, or form on 10.9.

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 9:08 PM, Sean Farley wrote:

> Ryan Schmidt writes:
> 
>>> The question, to me, is: why is it still not
>>> possible to distinguish foo+gcc and foo+clang in MacPorts?
>> 
>> I'm not sure what you mean.
> 
> Why can't all a port's variants be installed at the same time?
> 
> $ port install boost
> $ port install boost +gcc48
> 
> Every port could have its own custom prefix and only the active one
> would be a symlink in /opt/local.

That's not what variants are for. That's what subports are for.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 8:08 PM, Sean Farley wrote:
>> 
>> Ryan Schmidt writes:
>> 
>>> On Sep 30, 2014, at 7:04 PM, Sean Farley wrote:
>>> 
 This is pretty much was is done (modulo the blacklist). If I understand
 correctly, though, you want to remove gcc from being able to be used as
 a compiler? This would be a serious limitation.
>>> 
>>> Removed as a C/C++/ObjC/ObjC++ compiler, yes. In what situations do you 
>>> find that using gcc as the C/C++/ObjC/ObjC++ compiler is still necessary?
>> 
>> Of course there are many situations. We're talking about numerical code
>> here, so these projects are sometimes pushing the boundaries of what the
>> compilers can optimize. It is a great help if one can test both of these
>> cases out.
>> 
>> Flat out removing all gcc compilers is dead on arrival and will
>> ultimately drive people to ditch MacPorts.
>
> What also causes people to ditch MacPorts, or at least to file bug reports 
> about it, is when ports like boost offer variants, which the user rightly 
> assumes they can use, but whose use causes the port to either fail to compile 
> or causes other ports using that port to fail to compile.
>
> Of course we'll keep the gcc ports in MacPorts, we just can't really offer 
> the option of compiling arbitrary ports with g++ at the user's whim.
>
>
>> Using gcc for C and fortran, is not a problem. The only issue, of
>> course, is C++.
>
> True.
>
>> Even then, using C++ is only a problem when trying to
>> mix libc++ and libstdc++.
>
> Yup.
>
>> The question, to me, is: why is it still not
>> possible to distinguish foo+gcc and foo+clang in MacPorts?
>
> I'm not sure what you mean.

Why can't all a port's variants be installed at the same time?

$ port install boost
$ port install boost +gcc48

Every port could have its own custom prefix and only the active one
would be a symlink in /opt/local.

My point is that these issues all relate to depending on a port's
variant. This would be a powerful and very rich improvement to
MacPorts. A hacky alternative would be to provide a 'boost-stdlib' (or
whatever name) port that could be installed into a custom prefix for use
in this situation.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Sean Farley]

Ryan Schmidt writes:

>>> * It assumes the port's revision will be increased. That's not always 
>>> necessary. Sometimes the only change in the distfile is the name of (or 
>>> presence of) the enclosing directory, or there are only changes in comments 
>>> or documentation files or other files that don't affect the build, and in 
>>> those cases there would be zero benefit to increasing the port's revision 
>>> and forcing everyone to rebuild or redownload.
>> 
>> That's true. In this case, we should borrow an idea from DVCSes: use a
>> hash to see if something has changed (probably something like a Merkle
>> tree).
>
> I'm not familiar with a Merkle tree. I skimmed the wikipedia page but I'm not 
> sure what you're suggesting here. What should be hashed, by whom, at what 
> time?

It's just a fancy way to help detect if a tree has changed. Might not be
helpful here, exactly.

>> This would be a nontrivial amount of work but would help mitigate
>> the whole 'epoch' business.
>
> I don't see what the epoch field has to do with this discussion at all.

It's an orthogonal discussion which I'll save for another time.

>>> * The old distfile would get removed from our mirrors, making it impossible 
>>> for anyone to later determine what exactly changed in the stealth update. 
>>> Many maintainers handle stealth updates incorrectly. It is nice for others 
>>> to be able to investigate after the fact.
>> 
>> If we used hashes in the distfile naming, we wouldn't be overwriting
>> things.
>
> Ok, that's another good suggestion. I've seen that approach used by debian 
> which puts each distfile in a directory whose name is the hash of the 
> distfile.
>
> ~
>
>
> Let me just once again make sure that I understand the situation that we're 
> discussing here. Here are the facts as I understand them. Let me know if I've 
> misunderstood our goal in this discussion or if there are any other aspects.
>
>
> 1. It is the maintainer's responsibility to notice when a stealth update has 
> occurred.
>
> 2. It is the maintainer's responsibility to obtain the new distfile and place 
> its new checksums into the portfile, after verifying that the new distfile is 
> correct and is not malicious.
>
> 3. It is the maintainer's responsibility to decide whether this change 
> requires users to rebuild their ports, and on the basis of this determination 
> to increase the port's revision or not.
>
> 4. It is considered too onerous for the maintainer to need to insert the line 
> "dist_subdir ${name}/${version}_1" into the portfile when a stealth update 
> occurs, and we would like to make this step unnecessary.

That sounds right. What I'm attempting to improve (or, rather, stir up
awareness) are the many steps that are manual. Being that stealth
updates are infrequent, I usually forgot to check for them. I think 

A workflow I am trying to strive for (which maybe should be another email):

Task: update outdated ports

$ port echo maintainer:sean | parallel --trim lr port livecheck {}
mercurial seems to have been updated (port version: 3.0, new version: 3.1)

$ port maintainerupdate mercurial
  # this would replace the previous version string with the new one and
  # accept the checksum changes

$ hg ci -m "mercurial: update to 3.1"
$ hg push try-server

# this is where a buildbot would test the change and tell me a few
# common issues:
#   - was there a stealth update
#   - did I "Use The Right Compiler"
#   - run "port test" (or something similar)

If all the tests pass (and perhaps all the dependent ports pass) then it
would be allowed to be pushed to the central repository.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: OT: sed/awk pointers needed

[Lawrence Velázquez]

On Sep 30, 2014, at 4:02 PM, Craig Treleaven  wrote:

> you've given me a complete solution!

I just realized that you're probably using "N" to signify that a variant is not 
selected, so "Y" should be used if a variant *is* selected.


# Save port to hold space.
/ has the variants/ {
s/ has .*$//
h
d
}

# Remove noise.
/conflicts with/d
/^[^:]*mariadb[^:]*:/b db_variant
/^[^:]*mysql[^:]*:/b db_variant
/^[^:]*percona[^:]*:/b db_variant
d

:db_variant

# Append desired suffix to variant name.
/^\[+\]/s/.*]\(.*\):.*$/\1  Default/
/^[[:blank:]]*+/s/[[:blank:]]*+\(.*\):.*$/\1Y/
/^[[:blank:]]*[^+]/s/[[:blank:]]*\(.*\):.*$/\1  N/

# Extract port name from hold space and append variant name and suffix.
x
s/[[:blank:]].*$//
G
s/\n/   /

# Print pattern space and save to hold space for subsequent lines.
p
h


vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 8:08 PM, Sean Farley wrote:
> 
> Ryan Schmidt writes:
> 
>> On Sep 30, 2014, at 7:04 PM, Sean Farley wrote:
>> 
>>> This is pretty much was is done (modulo the blacklist). If I understand
>>> correctly, though, you want to remove gcc from being able to be used as
>>> a compiler? This would be a serious limitation.
>> 
>> Removed as a C/C++/ObjC/ObjC++ compiler, yes. In what situations do you find 
>> that using gcc as the C/C++/ObjC/ObjC++ compiler is still necessary?
> 
> Of course there are many situations. We're talking about numerical code
> here, so these projects are sometimes pushing the boundaries of what the
> compilers can optimize. It is a great help if one can test both of these
> cases out.
> 
> Flat out removing all gcc compilers is dead on arrival and will
> ultimately drive people to ditch MacPorts.

What also causes people to ditch MacPorts, or at least to file bug reports 
about it, is when ports like boost offer variants, which the user rightly 
assumes they can use, but whose use causes the port to either fail to compile 
or causes other ports using that port to fail to compile.

Of course we'll keep the gcc ports in MacPorts, we just can't really offer the 
option of compiling arbitrary ports with g++ at the user's whim.


> Using gcc for C and fortran, is not a problem. The only issue, of
> course, is C++.

True.

> Even then, using C++ is only a problem when trying to
> mix libc++ and libstdc++.

Yup.

> The question, to me, is: why is it still not
> possible to distinguish foo+gcc and foo+clang in MacPorts?

I'm not sure what you mean.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 7:04 PM, Sean Farley wrote:
>
>> This is pretty much was is done (modulo the blacklist). If I understand
>> correctly, though, you want to remove gcc from being able to be used as
>> a compiler? This would be a serious limitation.
>
> Removed as a C/C++/ObjC/ObjC++ compiler, yes. In what situations do you find 
> that using gcc as the C/C++/ObjC/ObjC++ compiler is still necessary?

Of course there are many situations. We're talking about numerical code
here, so these projects are sometimes pushing the boundaries of what the
compilers can optimize. It is a great help if one can test both of these
cases out.

Flat out removing all gcc compilers is dead on arrival and will
ultimately drive people to ditch MacPorts.

Using gcc for C and fortran, is not a problem. The only issue, of
course, is C++. Even then, using C++ is only a problem when trying to
mix libc++ and libstdc++. The question, to me, is: why is it still not
possible to distinguish foo+gcc and foo+clang in MacPorts?
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Ryan Schmidt]

On Sep 30, 2014, at 7:26 PM, Sean Farley wrote:
> 
> Ryan Schmidt writes:
> 
>> On Sep 30, 2014, at 4:55 PM, Sean Farley wrote:
>> 
>>> As for this exact issue, it could be something as simple as a fallback
>>> test:
>>> 
>>> if checksum doesn't match and revision is new:
>>> try downloading again
>>> write state so we don't download in an infinite loop
>> 
>> That's a conceivable change. But let's look at what problems that particular 
>> change would involve:
>> 
>> * MacPorts would need to gain the capability to go back to a previous phase 
>> (fetch) after it has already been completed. Or perhaps simpler, MacPorts 
>> would need the ability to automatically clean a port and try it from the 
>> beginning based on some criteria. This would be quite useful for a number of 
>> issues MacPorts users currently experience, but it's not a capability that 
>> exists today.
> 
> This is a very real issue. Difficult to do in today's code base.
> 
>> * It assumes the port's revision will be increased. That's not always 
>> necessary. Sometimes the only change in the distfile is the name of (or 
>> presence of) the enclosing directory, or there are only changes in comments 
>> or documentation files or other files that don't affect the build, and in 
>> those cases there would be zero benefit to increasing the port's revision 
>> and forcing everyone to rebuild or redownload.
> 
> That's true. In this case, we should borrow an idea from DVCSes: use a
> hash to see if something has changed (probably something like a Merkle
> tree).

I'm not familiar with a Merkle tree. I skimmed the wikipedia page but I'm not 
sure what you're suggesting here. What should be hashed, by whom, at what time?


> This would be a nontrivial amount of work but would help mitigate
> the whole 'epoch' business.

I don't see what the epoch field has to do with this discussion at all.


>> * The old distfile would get removed from our mirrors, making it impossible 
>> for anyone to later determine what exactly changed in the stealth update. 
>> Many maintainers handle stealth updates incorrectly. It is nice for others 
>> to be able to investigate after the fact.
> 
> If we used hashes in the distfile naming, we wouldn't be overwriting
> things.

Ok, that's another good suggestion. I've seen that approach used by debian 
which puts each distfile in a directory whose name is the hash of the distfile.


~


Let me just once again make sure that I understand the situation that we're 
discussing here. Here are the facts as I understand them. Let me know if I've 
misunderstood our goal in this discussion or if there are any other aspects.


1. It is the maintainer's responsibility to notice when a stealth update has 
occurred.

2. It is the maintainer's responsibility to obtain the new distfile and place 
its new checksums into the portfile, after verifying that the new distfile is 
correct and is not malicious.

3. It is the maintainer's responsibility to decide whether this change requires 
users to rebuild their ports, and on the basis of this determination to 
increase the port's revision or not.

4. It is considered too onerous for the maintainer to need to insert the line 
"dist_subdir ${name}/${version}_1" into the portfile when a stealth update 
occurs, and we would like to make this step unnecessary.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 7:04 PM, Sean Farley wrote:

> This is pretty much was is done (modulo the blacklist). If I understand
> correctly, though, you want to remove gcc from being able to be used as
> a compiler? This would be a serious limitation.

Removed as a C/C++/ObjC/ObjC++ compiler, yes. In what situations do you find 
that using gcc as the C/C++/ObjC/ObjC++ compiler is still necessary?


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Sean Farley]

Ryan Schmidt writes:

> On Sep 30, 2014, at 4:55 PM, Sean Farley wrote:
>
>> As for this exact issue, it could be something as simple as a fallback
>> test:
>> 
>> if checksum doesn't match and revision is new:
>>  try downloading again
>>  write state so we don't download in an infinite loop
>
> That's a conceivable change. But let's look at what problems that particular 
> change would involve:
>
> * MacPorts would need to gain the capability to go back to a previous phase 
> (fetch) after it has already been completed. Or perhaps simpler, MacPorts 
> would need the ability to automatically clean a port and try it from the 
> beginning based on some criteria. This would be quite useful for a number of 
> issues MacPorts users currently experience, but it's not a capability that 
> exists today.

This is a very real issue. Difficult to do in today's code base.

> * It assumes the port's revision will be increased. That's not always 
> necessary. Sometimes the only change in the distfile is the name of (or 
> presence of) the enclosing directory, or there are only changes in comments 
> or documentation files or other files that don't affect the build, and in 
> those cases there would be zero benefit to increasing the port's revision and 
> forcing everyone to rebuild or redownload.

That's true. In this case, we should borrow an idea from DVCSes: use a
hash to see if something has changed (probably something like a Merkle
tree). This would be a nontrivial amount of work but would help mitigate
the whole 'epoch' business.

> * The old distfile would get removed from our mirrors, making it impossible 
> for anyone to later determine what exactly changed in the stealth update. 
> Many maintainers handle stealth updates incorrectly. It is nice for others to 
> be able to investigate after the fact.

If we used hashes in the distfile naming, we wouldn't be overwriting
things.

These are all just ideas / vague shots in the dark. I'm not volunteering
to do this work :-P
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

>> This creates the scenario of: does the port want the full suite of, for
>> example, gcc (meaning both C and fortran) or just the fortran
>> compiler.
>
> And which of these two strategies does the mpi portgroup employ?

Both, that's why there is a 'require_fortran' variable.

>> This led to the creation of the 'require_fortran' monstrosity.
>
> With the release of OS X 10.9 and its use of libc++ we seem to have found the 
> use of gcc ports as values for configure.compiler untenable, giving rise to 
> the fortran recipe which just sets the fortran compiler and leaves the 
> C/C++/ObjC/ObjC++ compilers alone:
>
> https://trac.macports.org/wiki/PortfileRecipes#fortran
>
> I'm not sure if or how this relates to the require_fortran that you mentioned 
> above.

The portfile recipe would be the equivalent of something like +clang
+gfortran which would give you the clang compilers for C and gfortran
4.8 for fortran code. There is also the choice of +g95 in the compilers
portgroup but that doesn't work as well in my experience.

>> -
>> Complications
>> -
>> 
>> There are two complications with this implementation. The first is that
>> there is no knowledge of configure.compiler (used for blacklisting).
>
> I don't understand the situation in depth, but if mpi is requested, shouldn't 
> we just inspect the value of configure.cc / configure.cxx / configure.obcj / 
> configure.objcxx / configure.fc etc. (in a pre-configure block, to give a 
> port time to change compiler.blacklist / compiler.whitelist / 
> compiler.fallback), and replace them with their corresponding mpi wrappers? 
> If a port needs a Fortran compiler then it would also need to provide gcc 
> variants which would select only the fortran compiler, a la the fortran 
> recipe.

This is pretty much was is done (modulo the blacklist). If I understand
correctly, though, you want to remove gcc from being able to be used as
a compiler? This would be a serious limitation.

Instead, I'd like to make it easier to say: "take the port 'mpich' and
make all of its dependents use the same compilers". That's what the mpi
portgroup tries to mitigate.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Ryan Schmidt]

On Sep 30, 2014, at 4:10 PM, Sean Farley wrote:

> ---
> Fortran
> ---
> 
> While the above matrix was relatively straight-forward to implement
> there was one glaring problem: fortran. LLVM / clang does not provide
> it. Wonderful.

What portion of ports that want mpi also need a fortran compiler? 95%? 5%?

> This creates the scenario of: does the port want the full suite of, for
> example, gcc (meaning both C and fortran) or just the fortran
> compiler.

And which of these two strategies does the mpi portgroup employ?

> This led to the creation of the 'require_fortran' monstrosity.

With the release of OS X 10.9 and its use of libc++ we seem to have found the 
use of gcc ports as values for configure.compiler untenable, giving rise to the 
fortran recipe which just sets the fortran compiler and leaves the 
C/C++/ObjC/ObjC++ compilers alone:

https://trac.macports.org/wiki/PortfileRecipes#fortran

I'm not sure if or how this relates to the require_fortran that you mentioned 
above.

> -
> Complications
> -
> 
> There are two complications with this implementation. The first is that
> there is no knowledge of configure.compiler (used for blacklisting).

I don't understand the situation in depth, but if mpi is requested, shouldn't 
we just inspect the value of configure.cc / configure.cxx / configure.obcj / 
configure.objcxx / configure.fc etc. (in a pre-configure block, to give a port 
time to change compiler.blacklist / compiler.whitelist / compiler.fallback), 
and replace them with their corresponding mpi wrappers? If a port needs a 
Fortran compiler then it would also need to provide gcc variants which would 
select only the fortran compiler, a la the fortran recipe.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Ryan Schmidt]

On Sep 30, 2014, at 4:55 PM, Sean Farley wrote:

> As for this exact issue, it could be something as simple as a fallback
> test:
> 
> if checksum doesn't match and revision is new:
>  try downloading again
>  write state so we don't download in an infinite loop

That's a conceivable change. But let's look at what problems that particular 
change would involve:

* MacPorts would need to gain the capability to go back to a previous phase 
(fetch) after it has already been completed. Or perhaps simpler, MacPorts would 
need the ability to automatically clean a port and try it from the beginning 
based on some criteria. This would be quite useful for a number of issues 
MacPorts users currently experience, but it's not a capability that exists 
today.

* It assumes the port's revision will be increased. That's not always 
necessary. Sometimes the only change in the distfile is the name of (or 
presence of) the enclosing directory, or there are only changes in comments or 
documentation files or other files that don't affect the build, and in those 
cases there would be zero benefit to increasing the port's revision and forcing 
everyone to rebuild or redownload.

* The old distfile would get removed from our mirrors, making it impossible for 
anyone to later determine what exactly changed in the stealth update. Many 
maintainers handle stealth updates incorrectly. It is nice for others to be 
able to investigate after the fact.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [123602] trunk/dports/python/py-astropy/Portfile

[Sean Farley]

Daniel J. Luke writes:

> On Aug 12, 2014, at 11:11 AM, Ryan Schmidt  wrote:
>> 
>> It's still not totally automatic: the developer still has to update the 
>> checksums, so if the developer is sufficiently aware that the checksums need 
>> to be updated, why can't the developer also follow the stealth update 
>> recipe? This hasn't been a problem before.
>
> I don't necessarily disagree, but I also don't think it's a bad idea to at 
> least think about possible ways to reduce friction for people contributing to 
> MacPorts.

Sorry for the delay; I just moved across the country and started a new
job (Hello, Bay Area people!).

Daniel did a great job of reading my mind in this thread :-) I am trying
to brainstorm about possible ways to help people contribute.

As for this exact issue, it could be something as simple as a fallback
test:

if checksum doesn't match and revision is new:
  try downloading again
  write state so we don't download in an infinite loop
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: mpi

[Sean Farley]

Ryan Schmidt writes:

> We've had multiple bug reports from users who had installed boost with one of 
> the gcc variants. Either the build failed when using those variants, or the 
> build of boost succeeded but then the build of another port that depends on 
> boost failed. We attributed this to the mixing of C++ runtimes on 10.9+. I 
> removed the compiler variants from the boost port, but this has the effect of 
> making it impossible to install boost with mpi support, which apparently some 
> ports like dolphin require.
>
> I have to say I have no understanding of what mpi is or what the mpi 
> portgroup is doing. Sean, could you explain?


Introduction


MPI is a way to do parallel programming and stands for 'message passing
interface.' There are two major implementations: OpenMPI and
MPICH. Simply put, they are just libraries. A (IMHO, bad) decision was
made long ago to provide a compiler wrapper that would link the MPI
libraries so that a user could type:

mpicc test.c

--
First step
--

This made the first step of trying to build MPI ports in MacPorts
apparent: provide a way to supply the MPI compiler wrapper to a port.

That means providing a way to obtain values from the matrix of compilers
and MPI libraries: (fixed width font below)

 clang  clang  ...   gcc   gcc   ...   gcc   ...  dragonegg  ...
  3.13.2 4.4   4.5 4.9   3.1
+--+-+-+-+-+-+-+-+-+-+
MPICH   |  | | | | | | | | | |
+--+-+-+-+-+-+-+-+-+-+
OpenMPI |  | | | | | | | | | |
+--+-+-+-+-+-+-+-+-+-+

---
Fortran
---

While the above matrix was relatively straight-forward to implement
there was one glaring problem: fortran. LLVM / clang does not provide
it. Wonderful.

This creates the scenario of: does the port want the full suite of, for
example, gcc (meaning both C and fortran) or just the fortran
compiler. This led to the creation of the 'require_fortran' monstrosity.

-
Complications
-

There are two complications with this implementation. The first is that
there is no knowledge of configure.compiler (used for blacklisting).

The second, more difficult, complication is that this implementation
uses variants meaning only one version can be active at a time. This is
the very core of issue 126: https://trac.macports.org/ticket/126

Some package managers treat variants as subports. This would mean
installing each version / variant of a port into its own unique
path. For example,

foo-1.0: /opt/local/foo-1.0
foo-1.0+variantA: /opt/local/foo-1.0+variantA
foo-1.1+variantB: /opt/local/foo-1.1+variantB

For the convenience of building software outside the MacPorts tree, you
could set the active version of a port by symlinking its files into
/opt/local.

Changing MacPorts to this model would be no easy task.

Hopefully, now we can see the reason for the original problem: variant
dependence. If a user wants dolfin with gcc49:

$ port install dolfin +mpich +gcc49

Then this installs boost with +mpich +gcc49, forcing everything linking
to this version of boost to be compiled (and linked) with gcc's C++
compiler.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: MacPorts-MySQL Dictator wanted, prefer benevolent

[Brandon Allbery]

On Tue, Sep 30, 2014 at 5:02 PM, Bradley Giesbrecht 
wrote:

> > That's where the dictator comes in.  At the moment, I would say the
> obvious choices for a default MySQL variant would be mysql55, mariadb55 or,
> maybe, mysql56.  AFAICT there is no clear-cut, compelling reason to choose
> one over the others (technical, legal, religious, whatever).
>
> Momentum appears to be on the side of MariaDB
>

Agreed. In particular, most Linux distributions have adopted MariaDB as
their standard. (No doubt to the annoyance of the Percona adherents)

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: MacPorts-MySQL Dictator wanted, prefer benevolent

[Bradley Giesbrecht]

On Sep 30, 2014, at 8:37 AM, Craig Treleaven  wrote:

> By way of background, my MythTV ports depend on p5.16-dbd-mysql, py27-mysql 
> and php-5-mysql.  Previously, all of these defaulted to the now-obsolete 
> mysql5 variant.  Now, py-mysql defaults to mariadb55.  p516-dbd-mysql offers 
> a "mariadb" variant; not "mariadb55".  php5-mysql defaults to mysqlind 
> ("MySQL native driver") and offers a "mariadb" variant; again not 
> "mariadb55". p5.16-dbd-mysql still defaults to mysql5.
> 
> The subtle variance in naming the variants creates confusion.  The range of 
> defaults can lead to bloat and confusion.
> 
> Initially, I found there was a ticket related to py-mysql database variants 
> [1] and I filed additional tickets for php5-mysql [2] and p5-dbd-mysql [3].  
> Since then, I thought to search for other ports that offer variants related 
> to MySQL ('port echo variant:mysql*')--a total of 89 ports, of which about 20 
> are subports.
> 
> [1] https://trac.macports.org/ticket/39068
> [2] https://trac.macports.org/ticket/44481
> [3] https://trac.macports.org/ticket/44484
> 
> The tickets have been open for 6 weeks or more with no movement. I've started 
> to do some analysis on this (with another thread asking for help) to see if 
> we can standardize on a list of MySQL-related variant names and maybe even 
> agree on a common default.
> 
> That's where the dictator comes in.  At the moment, I would say the obvious 
> choices for a default MySQL variant would be mysql55, mariadb55 or, maybe, 
> mysql56.  AFAICT there is no clear-cut, compelling reason to choose one over 
> the others (technical, legal, religious, whatever).

Momentum appears to be on the side of MariaDB
MariaDB 10.0 is the current release branch and is tracked by the mariadb-10.0 
port.


Regards,
Bradley Giesbrecht (pixilla)



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: MacPorts-MySQL Dictator wanted, prefer benevolent

[Ryan Schmidt]

On Sep 30, 2014, at 10:37 AM, Craig Treleaven wrote:

> By way of background, my MythTV ports depend on p5.16-dbd-mysql, py27-mysql 
> and php-5-mysql.  Previously, all of these defaulted to the now-obsolete 
> mysql5 variant.  Now, py-mysql defaults to mariadb55.  p516-dbd-mysql offers 
> a "mariadb" variant; not "mariadb55".  php5-mysql defaults to mysqlind 
> ("MySQL native driver") and offers a "mariadb" variant; again not 
> "mariadb55". p5.16-dbd-mysql still defaults to mysql5.
> 
> The subtle variance in naming the variants creates confusion.  The range of 
> defaults can lead to bloat and confusion.
> 
> Initially, I found there was a ticket related to py-mysql database variants 
> [1] and I filed additional tickets for php5-mysql [2] and p5-dbd-mysql [3].  
> Since then, I thought to search for other ports that offer variants related 
> to MySQL ('port echo variant:mysql*')--a total of 89 ports, of which about 20 
> are subports.
> 
> [1] https://trac.macports.org/ticket/39068
> [2] https://trac.macports.org/ticket/44481
> [3] https://trac.macports.org/ticket/44484
> 
> The tickets have been open for 6 weeks or more with no movement. I've started 
> to do some analysis on this (with another thread asking for help) to see if 
> we can standardize on a list of MySQL-related variant names and maybe even 
> agree on a common default.
> 
> That's where the dictator comes in.  At the moment, I would say the obvious 
> choices for a default MySQL variant would be mysql55, mariadb55 or, maybe, 
> mysql56.  AFAICT there is no clear-cut, compelling reason to choose one over 
> the others (technical, legal, religious, whatever).  Other threads have 
> touched on this issue, recently, with no clear consensus developed.  
> Nonetheless, I think the MacPorts project would be better served by blessing 
> one that can hold us for a few years rather than have various ports going in 
> all different directions.
> 
> Is this something that the PortMgr group could take the lead on?

Indeed we have discussed this before, and it was proposed that the latest 
stable MariaDB version should be the default variant of ports that have MySQL 
support.

I don't think we can or should select a default to last us for years. We don't 
want to stagnate like we did with the +mysql5 variant. Rather, we should let 
the latest stable version be the default, and update ports as needed.

Currently, the latest stable version of MariaDB is 10.0.

The question is what to name the variants. This has also been brought up for 
discussion before. My most recent thread on the topic from 2 weeks ago got no 
replies on the list; perhaps I was too wordy.

My suggestion was that using dots in version numbers, but no underscores, would 
be the cleanest and most informative, which would make the complete list of 
proposed MySQL variant today: +mariadb5.5, +mariadb10.0, +mariadb10.1, 
+mysql5.1, +mysql5.5, +mysql5.6, and +percona5.6.

Currently, "port lint" says that a dot is not a valid character in a variant 
name, but I suspect lint is just being overly cautious, and perhaps someone 
with a little time could do some tests and see if variant names containing dots 
do in fact work correctly.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: OT: sed/awk pointers needed

[Craig Treleaven]

At 2:27 PM -0400 9/30/14, Lawrence Velázquez wrote:

On Sep 30, 2014, at 12:00 PM, Brandon Allbery  wrote:

 On Tue, Sep 30, 2014 at 11:44 AM, Craig 
Treleaven  wrote:
 I have only rudimentary acquaintance with sed 
and less with awk.  I'd like to learn more and 
this seemed like an opportunity to do so. From 
some reading [1], I think sed's Hold buffer is 
the way to extract the port name and insert it 
onto the line with each of the variants.  But 
the tutorial only uses the hold buffer for 
full lines; not just, say, the first word of 
the line.  Is this possible?  How does one do 
that?


 Very, very painfully. I think I could do it, 
with a LOT of fiddling, but would very much 
prefer to use a more appropriate tool. awk is 
somewhat better, but I'd reach for 
perl/python/ruby first.


Yup. Sed is not a great tool for this sort of 
thing; the functionality it provides is 
ill-suited for complex conditionals and 
branching. You'll regret writing the script as 
soon as you go to read/edit it in a week or 
month.


vq

-->8--

That being said, I have masochistic tendencies.

% port variants php53-mysql php54-mysql | sed -nf variants.sed
php53-mysql mariadb N
php53-mysql mysql4  N
php53-mysql mysql5  N
php53-mysql mysql51 N
php53-mysql mysql55 N
php53-mysql mysql56 N
php53-mysql mysqlnd Default
php53-mysql percona N
php54-mysql mariadb N
php54-mysql mysql4  N
php54-mysql mysql5  N
php54-mysql mysql51 N
php54-mysql mysql55 N
php54-mysql mysql56 N
php54-mysql mysqlnd Default
php54-mysql percona N
%


Attachment converted: Macintosh HD:variants.sed (/) (0342ACEF)


Thank you!  Off list, I was pointed to a sample 
Awk program* that reformats a 'billing report'. 
That helped; you've given me a complete solution!


* http://www.programmingforums.org/post226750.html

APL used to have the reputation as a "write-only" 
language.  This script is pretty close to that 
ideal.  ;)


Craig
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [125934] trunk/dports/mail

[Ryan Schmidt]

On Sep 30, 2014, at 11:13 AM, Bradley Giesbrecht wrote:

> On Sep 29, 2014, at 7:38 PM, Ryan Schmidt wrote:
> 
>> I presume this is noarch?
> 
> Yes. While I was writing this Portfile I wished that we had a Portfile 
> creation script that would prompt for input of the most important Portfile 
> keywords.
> I'll add supported_archs noarch.
> 
>> Remember you'll have to use the unversioned distfile recipe eventually, if 
>> the README ever changes.
> 
> Ah, the MP dist mirrors README fails checksums, I guess while writing the 
> Portfile I must have grabbed an invalid README and now the file is on the 
> dist mirrors. Is this possible?
> 
> I'll add a dist_subdir.

The server fetches the files from the URLs you specify. And even after your fix 
it seems not to be working:


--->  Attempting to fetch README from ftp://ftp.wl0.org/ftp.wl0.org/postfinger/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100   7810   7810 0  17244  0 --:--:-- --:--:-- --:--:-- 17355
DEBUG: Privilege de-escalation not attempted as not running as root.
DEBUG: checksum phase started at Tue Sep 30 09:13:46 PDT 2014
DEBUG: Executing org.macports.checksum (postfinger)

--->  Verifying checksums for postfinger
--->  Checksumming postfinger-1.30

DEBUG: Calculated (rmd160) is 62f8e8ce2888bf3d9e615072e9f25284d3f4155e
DEBUG: Correct (rmd160) checksum for postfinger-1.30
DEBUG: Calculated (sha256) is 
2c993b87ca3063eb4b81f7f652d6292a72d777be500bda214d54ce485aee2fd0
DEBUG: Correct (sha256) checksum for postfinger-1.30

--->  Checksumming README

DEBUG: Calculated (rmd160) is 764fbf6ce5dd7162f588001d15bca477fb50fd5e
Error: Checksum (rmd160) mismatch for README

Portfile checksum: README rmd160 7f08ecd13c073d81ca11ef097370fbdd361adb3d
Distfile checksum: README rmd160 764fbf6ce5dd7162f588001d15bca477fb50fd5e

DEBUG: Calculated (sha256) is 
d2cc2e326ec7b214414359c6e1429171eda8095008f74e7287617fbd24428e1d
Error: Checksum (sha256) mismatch for README

Portfile checksum: README sha256 
1e842f569b1d7878adc6bb38a1f5e80d8d3d25de051b820527f7f9fb9a8a4d95
Distfile checksum: README sha256 
d2cc2e326ec7b214414359c6e1429171eda8095008f74e7287617fbd24428e1d

Error: org.macports.checksum for port postfinger returned: Unable to verify 
file checksums





___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: nginx port

[Clemens Lang]

Hi,

- On 30 Sep, 2014, at 11:16, Ivan Larionov xeron.os...@gmail.com wrote:

> https://trac.macports.org/changeset/125927
> 
> How you can say it’s not “unstable” if 2 modules doesn’t work at all?

Upstream no longer calls it unstable and recommends using mainline rather
than stable:

"Note that stable does not mean ‘more reliable or more bug-free’. In fact,
 mainline is generally regarded as more reliable because only critical
 fixes are merged to stable."

The packaging of stable was a remnant from a time when what now is
mainline was still called "unstable" (and obviously unstable isn't
something we package in MacPorts).

The modules don't affect whether the main port is stable or not. These
are 3rd party modules, and apparently these 3rd parties are currently
lagging behind nginx development. If you want to help out getting them
compatible with 1.7.5 I'm sure they'd welcome some help. Most of these
changes are simple renames anyway and easily fixed. If you can provide a
patch that fixes the build for one of the modules, I'd happily apply it.


> May be it’s better to have 2 different ports, like nginx and nginx-devel?

No. This would mean more effort for me which I am not willing to spend on
the already pretty time-intensive nginx port. If somebody else wants to do
the extra work he/she is very welcome to provide a patch.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: OT: sed/awk pointers needed

[Lawrence Velázquez]

On Sep 30, 2014, at 12:00 PM, Brandon Allbery  wrote:

> On Tue, Sep 30, 2014 at 11:44 AM, Craig Treleaven  
> wrote:
>> I have only rudimentary acquaintance with sed and less with awk.  I'd like 
>> to learn more and this seemed like an opportunity to do so. From some 
>> reading [1], I think sed's Hold buffer is the way to extract the port name 
>> and insert it onto the line with each of the variants.  But the tutorial 
>> only uses the hold buffer for full lines; not just, say, the first word of 
>> the line.  Is this possible?  How does one do that?
> 
> Very, very painfully. I think I could do it, with a LOT of fiddling, but 
> would very much prefer to use a more appropriate tool. awk is somewhat 
> better, but I'd reach for perl/python/ruby first.

Yup. Sed is not a great tool for this sort of thing; the functionality it 
provides is ill-suited for complex conditionals and branching. You'll regret 
writing the script as soon as you go to read/edit it in a week or month.

vq

-->8--

That being said, I have masochistic tendencies.

% port variants php53-mysql php54-mysql | sed -nf variants.sed
php53-mysql mariadb N
php53-mysql mysql4  N
php53-mysql mysql5  N
php53-mysql mysql51 N
php53-mysql mysql55 N
php53-mysql mysql56 N
php53-mysql mysqlnd Default
php53-mysql percona N
php54-mysql mariadb N
php54-mysql mysql4  N
php54-mysql mysql5  N
php54-mysql mysql51 N
php54-mysql mysql55 N
php54-mysql mysql56 N
php54-mysql mysqlnd Default
php54-mysql percona N
%



variants.sed
Description: Binary data
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [125934] trunk/dports/mail

[Bradley Giesbrecht]

On Sep 29, 2014, at 7:38 PM, Ryan Schmidt  wrote:

> I presume this is noarch?

Yes. While I was writing this Portfile I wished that we had a Portfile creation 
script that would prompt for input of the most important Portfile keywords.
I'll add supported_archs noarch.

> Remember you'll have to use the unversioned distfile recipe eventually, if 
> the README ever changes.

Ah, the MP dist mirrors README fails checksums, I guess while writing the 
Portfile I must have grabbed an invalid README and now the file is on the dist 
mirrors. Is this possible?

I'll add a dist_subdir.


Regards,
Bradley Giesbrecht (pixilla)



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: OT: sed/awk pointers needed

[Brandon Allbery]

On Tue, Sep 30, 2014 at 11:44 AM, Craig Treleaven 
wrote:

> I have only rudimentary acquaintance with sed and less with awk.  I'd like
> to learn more and this seemed like an opportunity to do so. From some
> reading [1], I think sed's Hold buffer is the way to extract the port name
> and insert it onto the line with each of the variants.  But the tutorial
> only uses the hold buffer for full lines; not just, say, the first word of
> the line.  Is this possible?  How does one do that?


Very, very painfully. I think I could do it, with a LOT of fiddling, but
would very much prefer to use a more appropriate tool. awk is somewhat
better, but I'd reach for perl/python/ruby first.

Very roughly, the trick is to chop the line down to the first word, save it
to the hold buffer, then on subsequent lines append hold space to pattern
space, swap the line around the embedded newline that gets inserted before
the appendage, then replace that newline with a space. Use branch testing
to detect the difference between the two kinds of lines:

s/^  //
t dataline
s/ .*$//
h
b
: dataline
G
s/^\(.*\)\n\(.*\)$/\2 \1/
# now starts with the package name and a space; put the rest of the
processing here...

You can see how painful it gets. (Also the above is untested.) Better to
use a real language.

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

OT: sed/awk pointers needed

[Craig Treleaven]

tl;dr version:
How do I transform a listing like this:

$ port variants php53-mysql php54-mysql |grep -v conflicts |grep -e 
mariadb -e mysql -e percona | cut -d : -f 1

php53-mysql has the variants
   mariadb
   mysql4
   mysql5
   mysql51
   mysql55
   mysql56
[+]mysqlnd
   percona
php54-mysql has the variants
   mariadb
   mysql4
   mysql5
   mysql51
   mysql55
   mysql56
[+]mysqlnd
   percona

into:

php53-mysql   mariadb N
php53-mysql   mysql4 N
php53-mysql   mysql5 N
php53-mysql   mysql51 N
php53-mysql   mysql55 N
php53-mysql   mysql56 N
php53-mysql   mysqlnd Default
php53-mysql   percona N
php54-mysql   mariadb N
php54-mysql   mysql4 N
php54-mysql   mysql5 N
php54-mysql   mysql51 N
php54-mysql   mysql55 N
php54-mysql   mysql56 N
php54-mysql   mysqlnd Default
php54-mysql   percona N


I have only rudimentary acquaintance with sed and less with awk.  I'd 
like to learn more and this seemed like an opportunity to do so. 
From some reading [1], I think sed's Hold buffer is the way to 
extract the port name and insert it onto the line with each of the 
variants.  But the tutorial only uses the hold buffer for full lines; 
not just, say, the first word of the line.  Is this possible?  How 
does one do that?


[1] http://www.grymoire.com/Unix/Sed.html#uh-52

Or maybe I'm barking up the wrong tree?  Is there another 
tool/technique that is better suited?


Thanks in advance,

Craig
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

MacPorts-MySQL Dictator wanted, prefer benevolent

[Craig Treleaven]

By way of background, my MythTV ports depend on p5.16-dbd-mysql, 
py27-mysql and php-5-mysql.  Previously, all of these defaulted to 
the now-obsolete mysql5 variant.  Now, py-mysql defaults to 
mariadb55.  p516-dbd-mysql offers a "mariadb" variant; not 
"mariadb55".  php5-mysql defaults to mysqlind ("MySQL native driver") 
and offers a "mariadb" variant; again not "mariadb55". 
p5.16-dbd-mysql still defaults to mysql5.


The subtle variance in naming the variants creates confusion.  The 
range of defaults can lead to bloat and confusion.


Initially, I found there was a ticket related to py-mysql database 
variants [1] and I filed additional tickets for php5-mysql [2] and 
p5-dbd-mysql [3].  Since then, I thought to search for other ports 
that offer variants related to MySQL ('port echo variant:mysql*')--a 
total of 89 ports, of which about 20 are subports.


[1] https://trac.macports.org/ticket/39068
[2] https://trac.macports.org/ticket/44481
[3] https://trac.macports.org/ticket/44484

The tickets have been open for 6 weeks or more with no movement. 
I've started to do some analysis on this (with another thread asking 
for help) to see if we can standardize on a list of MySQL-related 
variant names and maybe even agree on a common default.


That's where the dictator comes in.  At the moment, I would say the 
obvious choices for a default MySQL variant would be mysql55, 
mariadb55 or, maybe, mysql56.  AFAICT there is no clear-cut, 
compelling reason to choose one over the others (technical, legal, 
religious, whatever).  Other threads have touched on this issue, 
recently, with no clear consensus developed.  Nonetheless, I think 
the MacPorts project would be better served by blessing one that can 
hold us for a few years rather than have various ports going in all 
different directions.


Is this something that the PortMgr group could take the lead on?

Craig
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [125841] trunk/dports/games/crafty/Portfile

[Kurt Hindenburg]

On 9/27/14, 11:07 PM, Joshua Root wrote:

Revision: 125841
   https://trac.macports.org/changeset/125841
Author:   khindenburg at macports.org
Date: 2014-09-27 07:32:31 -0700 (Sat, 27 Sep 2014)
Log Message:
---
crafty: add license (Unknown)

Modified Paths:
--
 trunk/dports/games/crafty/Portfile

Modified: trunk/dports/games/crafty/Portfile
===
--- trunk/dports/games/crafty/Portfile  2014-09-27 14:30:59 UTC (rev 125840)
+++ trunk/dports/games/crafty/Portfile  2014-09-27 14:32:31 UTC (rev 125841)
@@ -7,6 +7,7 @@
  categories  games
  version 24.1
  platforms   darwin
+license Unknown

Unknown means that nobody has checked what the license is. If there is
no mention of a license anywhere in the source or docs, use "none". Note
that this means that default copyright restrictions apply, i.e. we can't
distribute the software because we lack the copyright holder's
permission. This means that the port must be excluded from mirroring.

Please get clarification from upstream about terms for redistribution if
at all possible.
Thanks for the info - If you have time, can you look at #45217 and see 
what license would work for macports?

  Kurt

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Calligra and Koffice [was: imake]

[Brandon Allbery]

On Tue, Sep 30, 2014 at 5:46 AM, Nicolas Pavillon 
wrote:

> Apart from the possibility of announcing calligra as a replacement of
> koffice, I am nevertheless hesitating about committing the existing
> portfile. It can build, so that it would be a good starting point to
> improve on, but on the other hand, as there are some issues, that may lead
> to complaints.
> I was thinking of committing it with a warning note about the possible
> issues. Thoughts?
>

calligra-devel maybe? koffice2-devel was in there until the old maintainer
gave up on all the problems our kde4 had at the time.

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Calligra and Koffice [was: imake]

[Nicolas Pavillon]

Hello, 

> As Ryan said, it is not really the duty of MacPorts developers to keep
> ports alive if they are not supported upstream.  KDE 3 is long since
> "dead" (unmaintained) and all KDE apps dependent on KDE 3 with it.
> So, do not worry about KOffice… :-(

You have a fair point here. I’ll proceed on that.

> When we have time, we should see what can be done to
> make Calligra more workable in KDE 4 on Apple OS X.

Apart from the possibility of announcing calligra as a replacement of koffice, 
I am nevertheless hesitating about committing the existing portfile. It can 
build, so that it would be a good starting point to improve on, but on the 
other hand, as there are some issues, that may lead to complaints. 
I was thinking of committing it with a warning note about the possible issues. 
Thoughts?

Cheers, 

Nicolas
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

nginx port

[Ivan Larionov]

https://trac.macports.org/changeset/125927 


How you can say it’s not “unstable” if 2 modules doesn’t work at all?
May be it’s better to have 2 different ports, like nginx and nginx-devel?

--
With best regards, Ivan Larionov.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

mpi

[Ryan Schmidt]

We've had multiple bug reports from users who had installed boost with one of 
the gcc variants. Either the build failed when using those variants, or the 
build of boost succeeded but then the build of another port that depends on 
boost failed. We attributed this to the mixing of C++ runtimes on 10.9+. I 
removed the compiler variants from the boost port, but this has the effect of 
making it impossible to install boost with mpi support, which apparently some 
ports like dolphin require.

I have to say I have no understanding of what mpi is or what the mpi portgroup 
is doing. Sean, could you explain?

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev