buildbot build time (was: establish installed dependency not in portfile)

[db]

On 25 Oct 2017, at 07:35, Mojca Miklavec  wrote:
> On 23 October 2017 at 10:40, db wrote:
>> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
>>> That info is easy to read and collect from the buildbot. The keyword is: 
>>> "to collect".
>> I checked https://build.macports.org/json/help, but couldn't find it.
> For example:
> - 
> https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
> which corresponds to
> - https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
> 
> The idea would be to write a script to iterate through all numbers
> from 1 on and store results somewhere.
> 
> If you want the time estimate specifically, here might be the relevant
> information:
> 
>  "name": "install-port",
> ...
>  "times": [
>1508901749.648058,
>1508901850.387383
>  ]
> 
> I guess the difference represents the seconds spent building the port.

Thanks. I was searching for "to collect" there and in base, to no avail.

Those are the start/end epochs that I need to subtract, for a single port.

How can I correlate portname to buildnumber?

"mod_perl.so missing" errors when scanning binaries for linking errors

[Murray Eisenberg]

Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo port 
upgrade…” or “sudo port install…”, at the end I see:

  --->  Updating database of binaries
--->  Scanning binaries for linking errors
Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so: Error 
opening or reading file
Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so: Error 
opening or reading file
--->  No broken files found.   

fromWith apache 2.4, there seems to be no directory /op/local/apache2 at all 
(unless it should still be there and I inadvertently removed it).

What’s wrong?

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

Re: "mod_perl.so missing" errors when scanning binaries for linking errors

[Murray Eisenberg]

> On 25 Oct2017, at 10:57 AM, Murray Eisenberg  
> wrote:
> 
> Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo port 
> upgrade…” or “sudo port install…”, at the end I see:
> 
>   --->  Updating database of binaries
> --->  Scanning binaries for linking errors
> Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so: Error 
> opening or reading file
> Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so: Error 
> opening or reading file
> --->  No broken files found.   
> 
> fromWith apache 2.4, there seems to be no directory /op/local/apache2 at all 
> (unless it should still be there and I inadvertently removed it).
> 
> What’s wrong?


P.S. mod_perl.so does exist in:

   /opt/local/lib/apache2/modules/mod_perl.so

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

Re: "mod_perl.so missing" errors when scanning binaries for linking errors

[Rainer Müller]

On 2017-10-25 16:57, Murray Eisenberg wrote:
> Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo
> port upgrade…” or “sudo port install…”, at the end I see:
> 
>   --->  Updating database of binaries
> --->  Scanning binaries for linking errors
> Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so:
> Error opening or reading file
> Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so:
> Error opening or reading file
> --->  No broken files found.           
> 
> fromWith apache 2.4, there seems to be no directory /op/local/apache2 at
> all (unless it should still be there and I inadvertently removed it).
> 
> What’s wrong?

This looks like /opt/local/apache2/modules/mod_perl.so is in the
registry as installed and active, but it does not exist in the filesystem.

Does 'port provides /opt/local/apache2/modules/mod_perl.so' show which
port it is meant to belong to? And which port does the other mone at
   /opt/local/lib/apache2/modules/mod_perl.so belong to?

I assume something went wrong with mod_perl2 when it was updated for
apache2 @2.4.x.

Rainer

Re: "mod_perl.so missing" errors when scanning binaries for linking errors

[Marius Schamschula]

I just checked the mod_perl2 Portfile and discovered a typo. However, that only 
affected the +apache22 variant.

I wonder how it cleanly installed on my machine. I guess because there still 
might have been an /opt/local/apache2 directory, of mod_perl2 created it for 
just one file...

Marius
--
Marius Schamschula



> On Oct 25, 2017, at 12:18 PM, Rainer Müller  wrote:
> 
> On 2017-10-25 16:57, Murray Eisenberg wrote:
>> Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo
>> port upgrade…” or “sudo port install…”, at the end I see:
>> 
>>   --->  Updating database of binaries
>> --->  Scanning binaries for linking errors
>> Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so:
>> Error opening or reading file
>> Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so:
>> Error opening or reading file
>> --->  No broken files found.   
>> 
>> fromWith apache 2.4, there seems to be no directory /op/local/apache2 at
>> all (unless it should still be there and I inadvertently removed it).
>> 
>> What’s wrong?
> 
> This looks like /opt/local/apache2/modules/mod_perl.so is in the
> registry as installed and active, but it does not exist in the filesystem.
> 
> Does 'port provides /opt/local/apache2/modules/mod_perl.so' show which
> port it is meant to belong to? And which port does the other mone at
>   /opt/local/lib/apache2/modules/mod_perl.so belong to?
> 
> I assume something went wrong with mod_perl2 when it was updated for
> apache2 @2.4.x.
> 
> Rainer

Re: "mod_perl.so missing" errors when scanning binaries for linking errors

[Murray Eisenberg]

On 25 Oct2017, at 1:18 PM, Rainer Müller  wrote:
> 
> On 2017-10-25 16:57, Murray Eisenberg wrote:
>> Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo
>> port upgrade…” or “sudo port install…”, at the end I see:
>> 
>>   --->  Updating database of binaries
>> --->  Scanning binaries for linking errors
>> Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so:
>> Error opening or reading file
>> Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so:
>> Error opening or reading file
>> --->  No broken files found.   
>> 
>> fromWith apache 2.4, there seems to be no directory /op/local/apache2 at
>> all (unless it should still be there and I inadvertently removed it).
>> 
>> What’s wrong?
> 
> This looks like /opt/local/apache2/modules/mod_perl.so is in the
> registry as installed and active, but it does not exist in the filesystem.
> 
> Does 'port provides /opt/local/apache2/modules/mod_perl.so' show which
> port it is meant to belong to? And which port does the other mone at
>   /opt/local/lib/apache2/modules/mod_perl.so belong to?
> 
> I assume something went wrong with mod_perl2 when it was updated for
> apache2 @2.4.x.
> 
> Rainer

Output from

   port provides /opt/local/apache2/modules/mod_perl.so

is:
 /opt/local/apache2/modules/mod_perl.so does not exist.

Output from

 port provides /opt/local/lib/apache2/modules/mod_perl.so

is:

/opt/local/lib/apache2/modules/mod_perl.so is not provided by a MacPorts 
port.

Yet 

/opt/local/lib/apache2/modules/mod_perl.so

exists.

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

Re: "mod_perl.so missing" errors when scanning binaries for linking errors

[Murray Eisenberg]

Just upgraded 

   mod_perl2 2.0.10_0 < 2.0.10_1

and some other ports (after first doing a “clean mod_perl2”), and at the end I 
no longer got any Waringing messages
about mod_perl.so.

> On 25 Oct2017, at 1:42 PM, Marius Schamschula  wrote:
> 
> I just checked the mod_perl2 Portfile and discovered a typo. However, that 
> only affected the +apache22 variant.
> 
> I wonder how it cleanly installed on my machine. I guess because there still 
> might have been an /opt/local/apache2 directory, of mod_perl2 created it for 
> just one file...
> 
> Marius
> --
> Marius Schamschula
> 
> 
> 
>> On Oct 25, 2017, at 12:18 PM, Rainer Müller > > wrote:
>> 
>> On 2017-10-25 16:57, Murray Eisenberg wrote:
>>> Since upgrading from apache 2.2 to apache 2.4, whenever I do a “sudo
>>> port upgrade…” or “sudo port install…”, at the end I see:
>>> 
>>>   --->  Updating database of binaries
>>> --->  Scanning binaries for linking errors
>>> Warning: Error parsing file /opt/local/apache2/libexec/mod_perl.so:
>>> Error opening or reading file
>>> Warning: Error parsing file /opt/local/apache2/modules/mod_perl.so:
>>> Error opening or reading file
>>> --->  No broken files found.   
>>> 
>>> fromWith apache 2.4, there seems to be no directory /op/local/apache2 at
>>> all (unless it should still be there and I inadvertently removed it).
>>> 
>>> What’s wrong?
>> 
>> This looks like /opt/local/apache2/modules/mod_perl.so is in the
>> registry as installed and active, but it does not exist in the filesystem.
>> 
>> Does 'port provides /opt/local/apache2/modules/mod_perl.so' show which
>> port it is meant to belong to? And which port does the other mone at
>>   /opt/local/lib/apache2/modules/mod_perl.so belong to?
>> 
>> I assume something went wrong with mod_perl2 when it was updated for
>> apache2 @2.4.x.
>> 
>> Rainer
> 

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

Re: buildbot build time

[Ryan Schmidt]

On Oct 25, 2017, at 03:28, db wrote:
> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote:
>> On 23 October 2017 at 10:40, db wrote:
>>> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
 That info is easy to read and collect from the buildbot. The keyword is: 
 "to collect".
>>> I checked https://build.macports.org/json/help, but couldn't find it.
>> For example:
>> - 
>> https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
>> which corresponds to
>> - https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
>> 
>> The idea would be to write a script to iterate through all numbers
>> from 1 on and store results somewhere.
>> 
>> If you want the time estimate specifically, here might be the relevant
>> information:
>> 
>> "name": "install-port",
>> ...
>> "times": [
>>   1508901749.648058,
>>   1508901850.387383
>> ]
>> 
>> I guess the difference represents the seconds spent building the port.
> 
> Thanks. I was searching for "to collect" there and in base, to no avail.
> 
> Those are the start/end epochs that I need to subtract, for a single port.
> 
> How can I correlate portname to buildnumber?

I don't think scraping information from buildbot json files is the right way to 
collect this data. If we want to collect it, the buildbot scripts could be 
modified to do so.

But as I said before, I don't think the information is useful to collect. 
Besides the fact that the buildbot worker machines have different hardware 
specs than user hardware, not all of the buildbot workers run on the same 
hardware. A build will go faster on our High Sierra worker than on the Sierra 
worker, because the High Sierra worker has a faster processor. In addition, the 
buildbot workers are virtual machines, with multiple VMs running on a single 
host. So a build may be faster or slower depending on whether the other VMs 
that run on the same host are busy or not.

If all you want to do is display on a web site somewhere that a particular 
build of a port took a particular amount of time on our buildbot workers, ok. 
But I wouldn't try to use this information to, for example, modify MacPorts 
base to have it offer the user a prediction of how long the port will take to 
build on their system, because I don't think we can accurately predict that.

Re: buildbot build time

[Mojca Miklavec]

On 25 October 2017 at 16:23, Ryan Schmidt wrote:
> On Oct 25, 2017, at 03:28, db wrote:
>> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote:
>>> On 23 October 2017 at 10:40, db wrote:
 On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
> That info is easy to read and collect from the buildbot. The keyword is: 
> "to collect".
 I checked https://build.macports.org/json/help, but couldn't find it.
>>> For example:
>>> - 
>>> https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
>>> which corresponds to
>>> - 
>>> https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
>>>
>>> The idea would be to write a script to iterate through all numbers
>>> from 1 on and store results somewhere.
>>>
>>> If you want the time estimate specifically, here might be the relevant
>>> information:
>>>
>>> "name": "install-port",
>>> ...
>>> "times": [
>>>   1508901749.648058,
>>>   1508901850.387383
>>> ]
>>>
>>> I guess the difference represents the seconds spent building the port.
>>
>> Thanks. I was searching for "to collect" there and in base, to no avail.

"To collect" was not meant as a literate strings to search for, but as
the main action that needs to be done.

>> Those are the start/end epochs that I need to subtract, for a single port.
>>
>> How can I correlate portname to buildnumber?

You cannot at the moment.

What I envision is a script that would run once from number 1 on and
then either daily or hourly to update for new builds. The script would
then construct a database of builds and do some simple visualisation.
The most important part would be to ask some simple app to return data
about one particular port. I would expect it to return the list of all
builds (including URLs) for that particular port, but most
importantly, for each buildslave:
- whether the last build succeeded or failed
  - (if failed, whether it was a dependency or the port installation
that failed)
- commit timestamp / shasum & version of the port
- whatever else seems relevant, potentially including the build time
and size of the package

> I don't think scraping information from buildbot json files is the right way 
> to collect this data. If we want to collect it, the buildbot scripts could be 
> modified to do so.

I don't see any big difference. The important part would be to do it
in the first place. How to do it most efficiently can be a subject for
discussion, but parsing json files with a library seems the easiest
approach to me, in particular because we are currently unable to
modify the buildbot script to return us data about packages built in
the past. We can always modify the build scripts later.

> But as I said before, I don't think the information is useful to collect.

Build time doesn't seem *the most* interesting part of information,
but it is useful to know in any case.

> Besides the fact that the buildbot worker machines have different hardware 
> specs than user hardware, not all of the buildbot workers run on the same 
> hardware. A build will go faster on our High Sierra worker than on the Sierra 
> worker, because the High Sierra worker has a faster processor. In addition, 
> the buildbot workers are virtual machines, with multiple VMs running on a 
> single host. So a build may be faster or slower depending on whether the 
> other VMs that run on the same host are busy or not.
>
> If all you want to do is display on a web site somewhere that a particular 
> build of a port took a particular amount of time on our buildbot workers, ok. 
> But I wouldn't try to use this information to, for example, modify MacPorts 
> base to have it offer the user a prediction of how long the port will take to 
> build on their system, because I don't think we can accurately predict that.

I would definitely not use it for that purpose. But having *some
statistics* about which ports built and failed on which builder would
be super super useful. What I miss most is quickly finding the build
where a particular port failed to build. At the moment it's nearly
impossible to find it.

Mojca

Re: buildbot build time

[Ryan Schmidt]

> On Oct 25, 2017, at 18:45, Mojca Miklavec  wrote:
> 
> On 25 October 2017 at 16:23, Ryan Schmidt wrote:
>> On Oct 25, 2017, at 03:28, db wrote:
>>> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote:
 On 23 October 2017 at 10:40, db wrote:
> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
>> That info is easy to read and collect from the buildbot. The keyword is: 
>> "to collect".
> I checked https://build.macports.org/json/help, but couldn't find it.
 For example:
 - 
 https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
 which corresponds to
 - 
 https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
 
 The idea would be to write a script to iterate through all numbers
 from 1 on and store results somewhere.
 
 If you want the time estimate specifically, here might be the relevant
 information:
 
"name": "install-port",
 ...
"times": [
  1508901749.648058,
  1508901850.387383
]
 
 I guess the difference represents the seconds spent building the port.
>>> 
>>> Thanks. I was searching for "to collect" there and in base, to no avail.
> 
> "To collect" was not meant as a literate strings to search for, but as
> the main action that needs to be done.
> 
>>> Those are the start/end epochs that I need to subtract, for a single port.
>>> 
>>> How can I correlate portname to buildnumber?
> 
> You cannot at the moment.
> 
> What I envision is a script that would run once from number 1 on and
> then either daily or hourly to update for new builds. The script would
> then construct a database of builds and do some simple visualisation.
> The most important part would be to ask some simple app to return data
> about one particular port. I would expect it to return the list of all
> builds (including URLs) for that particular port, but most
> importantly, for each buildslave:
> - whether the last build succeeded or failed
>  - (if failed, whether it was a dependency or the port installation
> that failed)
> - commit timestamp / shasum & version of the port
> - whatever else seems relevant, potentially including the build time
> and size of the package
> 
>> I don't think scraping information from buildbot json files is the right way 
>> to collect this data. If we want to collect it, the buildbot scripts could 
>> be modified to do so.
> 
> I don't see any big difference. The important part would be to do it
> in the first place. How to do it most efficiently can be a subject for
> discussion, but parsing json files with a library seems the easiest
> approach to me, in particular because we are currently unable to
> modify the buildbot script to return us data about packages built in
> the past. We can always modify the build scripts later.

True, modifying the build scripts wouldn't help collect data about past builds. 
We would need to process past builds separately.


>> But as I said before, I don't think the information is useful to collect.
> 
> Build time doesn't seem *the most* interesting part of information,
> but it is useful to know in any case.
> 
>> Besides the fact that the buildbot worker machines have different hardware 
>> specs than user hardware, not all of the buildbot workers run on the same 
>> hardware. A build will go faster on our High Sierra worker than on the 
>> Sierra worker, because the High Sierra worker has a faster processor. In 
>> addition, the buildbot workers are virtual machines, with multiple VMs 
>> running on a single host. So a build may be faster or slower depending on 
>> whether the other VMs that run on the same host are busy or not.
>> 
>> If all you want to do is display on a web site somewhere that a particular 
>> build of a port took a particular amount of time on our buildbot workers, 
>> ok. But I wouldn't try to use this information to, for example, modify 
>> MacPorts base to have it offer the user a prediction of how long the port 
>> will take to build on their system, because I don't think we can accurately 
>> predict that.
> 
> I would definitely not use it for that purpose. But having *some
> statistics* about which ports built and failed on which builder would
> be super super useful. What I miss most is quickly finding the build
> where a particular port failed to build. At the moment it's nearly
> impossible to find it.

I agree with all of that. I had not considered that those issues might also be 
addressed by this process.

p5.26-libapreq2 and mod_perl2 morass

[Murray Eisenberg]

I’m trying to upgrade perl-related ports from perl 5.24 to perl 5.26.  

When I executed...

sudo port install p5.26-libapreq2

...I got message...

   The following dependencies will be installed: 
  mod_perl2
  p5.26-extutils-xsbuilder

…and I proceeded. Got messages:

Error: Failed to activate mod_perl2: Image error: 
/opt/local/lib/apache2/modules/mod_perl.so already exists and does not belong 
to a registered   
  port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' to 
force the activation.

Proceeded, I tried:

   sudo port -f activate mod_perl2

Got message:

The following versions of mod_perl2 are currently installed:
 1) mod_perl2 @2.0.10_0+perl5_24
2) mod_perl2 @2.0.10_1+perl5_24

GotBut I didn’t want the+perl5_24 variant of mod_perl2 now, since the idea is 
to upgrade to perl 5.26.
So I aborted.

Next, tried...

   sudo port uninstall mod_perl2 @2.0.10_0+perl5_24

 …with resulting message steps:

Note: It is not recommended to uninstall/deactivate a port that has 
dependents as it breaks the dependents.
The following ports will break: p5.24-libapreq2 @2.130.0_5
Continue? [y/N]: y
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Uninstalling mod_perl2 @2.0.10_0+perl5_24
--->  Cleaning mod_perl2

Now I tried again...

   sudo port install p5.26-libapreq2

… with result:

--->  Computing dependencies for p5.26-libapreq2
  The following dependencies will be installed: 
  mod_perl2
  p5.26-extutils-xsbuilder
  Continue? [Y/n]: y
   …
   --->  Activating mod_perl2 @2.0.10_1+perl5_24
  Error: Failed to activate mod_perl2: Image error: 
/opt/local/lib/apache2/modules/mod_perl.so already exists and does not belong 
to a registered   
  port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' to 
force the activation.
  …
  Error: Processing of port p5.26-libapreq2 failed

How do I proceed??

Here’s what the end of the … mod_perl2/main.log :

:error:install Failed to install mod_perl2: no destroot found at: 
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
:debug:install Error code: NONE
:debug:install Backtrace: no destroot found at: 
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
:debug:install while executing
:debug:install "create_archive $location $portarchivetype"
:debug:install (procedure "portinstall::install_main" line 22)
:debug:install invoked from within
:debug:install "$procedure $targetname"

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

Re: p5.26-libapreq2 and mod_perl2 morass

[Marius Schamschula]

Murray,

mod_perl2 defaults to building with perl 5.24, as this currently the default 
branch. You have to explicitly build with perl 5.26 by specifying the variant:

sudo port install mod_perl2 +perl5_26

Once you do this, I think the rest of your install should proceed.

> On Oct 25, 2017, at 7:41 PM, Murray Eisenberg  
> wrote:
> 
> I’m trying to upgrade perl-related ports from perl 5.24 to perl 5.26.  
> 
> When I executed...
> 
> sudo port install p5.26-libapreq2
> 
> ...I got message...
> 
>The following dependencies will be installed: 
>   mod_perl2
>   p5.26-extutils-xsbuilder
> 
> …and I proceeded. Got messages:
> 
> Error: Failed to activate mod_perl2: Image error: 
> /opt/local/lib/apache2/modules/mod_perl.so already exists and does not belong 
> to a registered   
>   port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' 
> to force the activation.
> 
> Proceeded, I tried:
> 
>sudo port -f activate mod_perl2
> 
> Got message:
> 
> The following versions of mod_perl2 are currently installed:
>  1) mod_perl2 @2.0.10_0+perl5_24
> 2) mod_perl2 @2.0.10_1+perl5_24
> 
> GotBut I didn’t want the+perl5_24 variant of mod_perl2 now, since the idea is 
> to upgrade to perl 5.26.
> So I aborted.
> 
> Next, tried...
> 
>sudo port uninstall mod_perl2 @2.0.10_0+perl5_24
> 
>  …with resulting message steps:
> 
> Note: It is not recommended to uninstall/deactivate a port that has 
> dependents as it breaks the dependents.
> The following ports will break: p5.24-libapreq2 @2.130.0_5
> Continue? [y/N]: y
> Warning: Uninstall forced.  Proceeding despite dependencies.
> --->  Uninstalling mod_perl2 @2.0.10_0+perl5_24
> --->  Cleaning mod_perl2
> 
> Now I tried again...
> 
>sudo port install p5.26-libapreq2
> 
> … with result:
> 
> --->  Computing dependencies for p5.26-libapreq2
>   The following dependencies will be installed: 
>   mod_perl2
>   p5.26-extutils-xsbuilder
>   Continue? [Y/n]: y
>…
>--->  Activating mod_perl2 @2.0.10_1+perl5_24
>   Error: Failed to activate mod_perl2: Image error: 
> /opt/local/lib/apache2/modules/mod_perl.so already exists and does not belong 
> to a registered   
>   port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' 
> to force the activation.
>   …
>   Error: Processing of port p5.26-libapreq2 failed
> 
> How do I proceed??
> 
> Here’s what the end of the … mod_perl2/main.log :
> 
> :error:install Failed to install mod_perl2: no destroot found at: 
> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
> :debug:install Error code: NONE
> :debug:install Backtrace: no destroot found at: 
> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
> :debug:install while executing
> :debug:install "create_archive $location $portarchivetype"
> :debug:install (procedure "portinstall::install_main" line 22)
> :debug:install invoked from within
> :debug:install "$procedure $targetname"
> 
> ---
> Murray Eisenberg  murrayeisenb...@gmail.com 
> 
> 503 King Farm Blvd #101   Home (240)-246-7240
> Rockville, MD 20850-6667  Mobile (413)-427-5334
> 
> 



Marius
--
Marius Schamschula

Re: buildbot build time

[Craig Treleaven]

> On Oct 25, 2017, at 7:45 PM, Mojca Miklavec  wrote:
> 
> On 25 October 2017 at 16:23, Ryan Schmidt wrote:
>> On Oct 25, 2017, at 03:28, db wrote:
>>> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote:
 On 23 October 2017 at 10:40, db wrote:
> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote:
>> That info is easy to read and collect from the buildbot. The keyword is: 
>> "to collect".
> I checked https://build.macports.org/json/help, but couldn't find it.
 For example:
 - 
 https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1
 which corresponds to
 - 
 https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627
 
 The idea would be to write a script to iterate through all numbers
 from 1 on and store results somewhere.
 
 If you want the time estimate specifically, here might be the relevant
 information:
 
"name": "install-port",
 ...
"times": [
  1508901749.648058,
  1508901850.387383
]
 
 I guess the difference represents the seconds spent building the port.
>>> 
>>> Thanks. I was searching for "to collect" there and in base, to no avail.
> 
> "To collect" was not meant as a literate strings to search for, but as
> the main action that needs to be done.
> 
>>> Those are the start/end epochs that I need to subtract, for a single port.
>>> 
>>> How can I correlate portname to buildnumber?
> 
> You cannot at the moment.
> 
> What I envision is a script that would run once from number 1 on and
> then either daily or hourly to update for new builds. The script would
> then construct a database of builds and do some simple visualisation.
> The most important part would be to ask some simple app to return data
> about one particular port. I would expect it to return the list of all
> builds (including URLs) for that particular port, but most
> importantly, for each buildslave:
> - whether the last build succeeded or failed
>  - (if failed, whether it was a dependency or the port installation
> that failed)
> - commit timestamp / shasum & version of the port
> - whatever else seems relevant, potentially including the build time
> and size of the package
> 
>> I don't think scraping information from buildbot json files is the right way 
>> to collect this data. If we want to collect it, the buildbot scripts could 
>> be modified to do so.
> 
> I don't see any big difference. The important part would be to do it
> in the first place. How to do it most efficiently can be a subject for
> discussion, but parsing json files with a library seems the easiest
> approach to me, in particular because we are currently unable to
> modify the buildbot script to return us data about packages built in
> the past. We can always modify the build scripts later.
> 
>> But as I said before, I don't think the information is useful to collect.
> 
> Build time doesn't seem *the most* interesting part of information,
> but it is useful to know in any case.
> 
>> Besides the fact that the buildbot worker machines have different hardware 
>> specs than user hardware, not all of the buildbot workers run on the same 
>> hardware. A build will go faster on our High Sierra worker than on the 
>> Sierra worker, because the High Sierra worker has a faster processor. In 
>> addition, the buildbot workers are virtual machines, with multiple VMs 
>> running on a single host. So a build may be faster or slower depending on 
>> whether the other VMs that run on the same host are busy or not.
>> 
>> If all you want to do is display on a web site somewhere that a particular 
>> build of a port took a particular amount of time on our buildbot workers, 
>> ok. But I wouldn't try to use this information to, for example, modify 
>> MacPorts base to have it offer the user a prediction of how long the port 
>> will take to build on their system, because I don't think we can accurately 
>> predict that.
> 
> I would definitely not use it for that purpose. But having *some
> statistics* about which ports built and failed on which builder would
> be super super useful. What I miss most is quickly finding the build
> where a particular port failed to build. At the moment it's nearly
> impossible to find it.

If we’re going to do this, can we be a little more precise about “failed” 
builds?  

1) Any port that relies on a non-default variant “fails” on the buildbots.  All 
of my myth* ports will ALWAYS show a “failed” status on ALL of the buildbots 
for this reason.   I would rather that users were told that the build was “not 
attempted”.  Telling them that build “failed" isn’t helpful.

2) Any port that needs C++11 “fails” on the existing 10.7 and earlier buildbots.

3) Various ports have a pre-fetch block that checks the OS version and “fails” 
if it is not supported.

Assuming the above can be fixed, is the following a mock-up of the data we’d 
collect?  Each commit to the ports tree tri

Re: buildbot build time

[Ryan Schmidt]

On Oct 25, 2017, at 20:09, Craig Treleaven wrote:

> If we’re going to do this, can we be a little more precise about “failed” 
> builds?  
> 
> 1) Any port that relies on a non-default variant “fails” on the buildbots.  
> All of my myth* ports will ALWAYS show a “failed” status on ALL of the 
> buildbots for this reason.   I would rather that users were told that the 
> build was “not attempted”.  Telling them that build “failed" isn’t helpful.
> 
> 2) Any port that needs C++11 “fails” on the existing 10.7 and earlier 
> buildbots.
> 
> 3) Various ports have a pre-fetch block that checks the OS version and 
> “fails” if it is not supported.

As I've said many times before, this is a big project, involving changes to 
MacPorts base to provide a mechanism whereby ports can indicate that a build is 
expected to fail in certain circumstances. I don't think any work has begun on 
defining or implementing that mechanism.

Re: p5.26-libapreq2 and mod_perl2 morass

[Murray Eisenberg]

That did it, thanks! 

Just a note that then I had to deactivate p5.24-libapreq2 before going ahead to 
install p5.26-libapreq2 .


> On 25 Oct2017, at 8:55 PM, Marius Schamschula  wrote:
> 
> Murray,
> 
> mod_perl2 defaults to building with perl 5.24, as this currently the default 
> branch. You have to explicitly build with perl 5.26 by specifying the variant:
> 
> sudo port install mod_perl2 +perl5_26
> 
> Once you do this, I think the rest of your install should proceed.
> 
>> On Oct 25, 2017, at 7:41 PM, Murray Eisenberg > > wrote:
>> 
>> I’m trying to upgrade perl-related ports from perl 5.24 to perl 5.26.  
>> 
>> When I executed...
>> 
>> sudo port install p5.26-libapreq2
>> 
>> ...I got message...
>> 
>>The following dependencies will be installed: 
>>   mod_perl2
>>   p5.26-extutils-xsbuilder
>> 
>> …and I proceeded. Got messages:
>> 
>> Error: Failed to activate mod_perl2: Image error: 
>> /opt/local/lib/apache2/modules/mod_perl.so already exists and does not 
>> belong to a registered   
>>   port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' 
>> to force the activation.
>> 
>> Proceeded, I tried:
>> 
>>sudo port -f activate mod_perl2
>> 
>> Got message:
>> 
>> The following versions of mod_perl2 are currently installed:
>>  1) mod_perl2 @2.0.10_0+perl5_24
>> 2) mod_perl2 @2.0.10_1+perl5_24
>> 
>> GotBut I didn’t want the+perl5_24 variant of mod_perl2 now, since the idea 
>> is to upgrade to perl 5.26.
>> So I aborted.
>> 
>> Next, tried...
>> 
>>sudo port uninstall mod_perl2 @2.0.10_0+perl5_24
>> 
>>  …with resulting message steps:
>> 
>> Note: It is not recommended to uninstall/deactivate a port that has 
>> dependents as it breaks the dependents.
>> The following ports will break: p5.24-libapreq2 @2.130.0_5
>> Continue? [y/N]: y
>> Warning: Uninstall forced.  Proceeding despite dependencies.
>> --->  Uninstalling mod_perl2 @2.0.10_0+perl5_24
>> --->  Cleaning mod_perl2
>> 
>> Now I tried again...
>> 
>>sudo port install p5.26-libapreq2
>> 
>> … with result:
>> 
>> --->  Computing dependencies for p5.26-libapreq2
>>   The following dependencies will be installed: 
>>   mod_perl2
>>   p5.26-extutils-xsbuilder
>>   Continue? [Y/n]: y
>>…
>>--->  Activating mod_perl2 @2.0.10_1+perl5_24
>>   Error: Failed to activate mod_perl2: Image error: 
>> /opt/local/lib/apache2/modules/mod_perl.so already exists and does not 
>> belong to a registered   
>>   port.  Unable to activate port mod_perl2. Use 'port -f activate mod_perl2' 
>> to force the activation.
>>   …
>>   Error: Processing of port p5.26-libapreq2 failed
>> 
>> How do I proceed??
>> 
>> Here’s what the end of the … mod_perl2/main.log :
>> 
>> :error:install Failed to install mod_perl2: no destroot found at: 
>> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
>> :debug:install Error code: NONE
>> :debug:install Backtrace: no destroot found at: 
>> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_www_mod_perl2/mod_perl2/work/destroot
>> :debug:install while executing
>> :debug:install "create_archive $location $portarchivetype"
>> :debug:install (procedure "portinstall::install_main" line 22)
>> :debug:install invoked from within
>> :debug:install "$procedure $targetname"
>> 
>> ---
>> Murray Eisenberg murrayeisenb...@gmail.com 
>> 
>> 503 King Farm Blvd #101  Home (240)-246-7240
>> Rockville, MD 20850-6667 Mobile (413)-427-5334
>> 
>> 
> 
> 
> 
> Marius
> --
> Marius Schamschula
> 
> 
> 
> 

---
Murray Eisenbergmurrayeisenb...@gmail.com
503 King Farm Blvd #101 Home (240)-246-7240
Rockville, MD 20850-6667Mobile (413)-427-5334

how to select gcc7

[pagani laurent via macports-users]

Hi,

I have gcc5, gcc6 and gcc7 installed, in particular :

  gcc7 @7.2.0_0 (active)

but

>gcc --version
gcc (MacPorts gcc5 5.5.0_0) 5.5.0

I try to select gcc7 but fail (did not find any help on the webpage 
https://guide.macports.org/#using.port about port select. Is there any ?)

>sudo port select gcc7 gcc
Selecting 'gcc' for 'gcc7' failed: The specified group 'gcc7' does not exist.

I tried the other way round, no better :

>sudo port select gcc gcc7
Selecting 'gcc7' for 'gcc' failed: The specified version 'gcc7' is not valid.

How should I do that ?

Thanks,
L.

"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" (devise Shadok)

Re: buildbot build time

[Dave Horsfall]

On Wed, 25 Oct 2017, Ryan Schmidt wrote:

As I've said many times before, this is a big project, involving changes 
to MacPorts base to provide a mechanism whereby ports can indicate that 
a build is expected to fail in certain circumstances. I don't think any 
work has begun on defining or implementing that mechanism.


Besides, it could be regarded as being equivalent to solving the halting 
problem...


--
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

Re: how to select gcc7

[Ken Cunningham]

$ port select --list gcc
Available versions for gcc:
gcc40
gcc42 (active)
llvm-gcc42
mp-gcc5
mp-gcc6
mp-gcc7
none

so

sudo port select gcc mp-gcc7 

should do the trick

On 2017-10-25, at 7:06 PM, pagani laurent via macports-users wrote:

> Hi,
> 
> I have gcc5, gcc6 and gcc7 installed, in particular :
> 
>  gcc7 @7.2.0_0 (active)
> 
> but
> 
>> gcc --version
> gcc (MacPorts gcc5 5.5.0_0) 5.5.0
> 
> I try to select gcc7 but fail (did not find any help on the webpage 
> https://guide.macports.org/#using.port about port select. Is there any ?)
> 
>> sudo port select gcc7 gcc
> Selecting 'gcc' for 'gcc7' failed: The specified group 'gcc7' does not exist.
> 
> I tried the other way round, no better :
> 
>> sudo port select gcc gcc7
> Selecting 'gcc7' for 'gcc' failed: The specified version 'gcc7' is not valid.
> 
> How should I do that ?
> 
> Thanks,
> L.
> 
> "S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" (devise 
> Shadok)
> 

Re: buildbot build time

[Mojca Miklavec]

On 25 October 2017 at 18:09, Craig Treleaven wrote:
>
> If we’re going to do this, can we be a little more precise about “failed” 
> builds?

Not yet at this stage unless/until someone comes up and implements the
needed functionality in the core. (Like Ryan said.)

Ideally the buildbot would not even attempt to build a port that's not
compatible, so we wouldn't even get any failure reports. We might
still want to record "skipped" or something similar in that case, but
that's a different / parallel problem.

> 1) Any port that relies on a non-default variant “fails” on the buildbots.  
> All of my myth* ports will ALWAYS show a “failed” status on ALL of the 
> buildbots for this reason.   I would rather that users were told that the 
> build was “not attempted”.  Telling them that build “failed" isn’t helpful.

That's yet another problem that could be fixed in a different way,
probably once the dependency engine implementation (a past GSOC
project) gets deployed. Once we fix this properly, it should no longer
be an issue.

> 2) Any port that needs C++11 “fails” on the existing 10.7 and earlier 
> buildbots.

That's not entirely true. The cxx11 1.1 portgroup often works fine.
But I would not worry about this case for now. We are not attempting
to go for 100% success rate, just collect statistics for now.

> 3) Various ports have a pre-fetch block that checks the OS version and 
> “fails” if it is not supported.

See the first point above.

> Assuming the above can be fixed,

Independent of this ...

> is the following a mock-up of the data we’d collect?  Each commit to the 
> ports tree triggers builds on the then-current fleet of builders:
>
> Port:blurfl
> Verson:  2.54.1_0
> Commmit: 
> https://github.com/macports/macports-ports/commit/6b120d678245ba6f14a5a364f83c3c5970f762af
>
> Buildbots:
> Builder Build IDBuild (minutes.seconds)
> X86-10.13   1.20
> X86-10.12   2.20
> X86-10.11   2.00
> X86-10.10   4.50
> X86-10.92.40
> X86-10.8fail
> X86-10.7not supported
> X86-10.6not supported
> PPC-10.5not supported

Yes, something in that sense. For now we would not have the "not
supported" flag until the core functionality changes. For the failed
case it would be helpful to know whether any earlier version
successfully built. And some builders might not have the latest
version built yet (probably still waiting somewhere in the queue). I
would probably collect some more simple parameters, but
success/failure/version is the most important one anyway.

Mojca

port select problem

[pagani laurent via macports-users]

Hi,

Lorenzo>sudo port select --set ipython2 py27-ipython
Selecting 'py27-ipython' for 'ipython2' succeeded. 'py27-ipython' is now active.
Lorenzo>ipython2
ipython2: Command not found.

Why ?

If I type ipython, it will work instead while I did not select it to be 
py27-ipython

L.

"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" (devise Shadok)

Re: port select problem

[Dave Horsfall]

On Thu, 26 Oct 2017, pagani laurent via macports-users wrote:


Lorenzo>ipython2
ipython2: Command not found.

Why ?


Depending upon your shell, you might need to "rehash".

--
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

Re: port select problem

[pagani laurent via macports-users]

In principle you don’t need to rehash with Macports. But I checked and it makes 
no difference.
I could select ipython3 to py35-ipython and it works. But not ipython2.

L.

> Le 26 oct. 2017 à 06:08, Dave Horsfall  a écrit :
> 
> On Thu, 26 Oct 2017, pagani laurent via macports-users wrote:
> 
>> Lorenzo>ipython2
>> ipython2: Command not found.
>> 
>> Why ?
> 
> Depending upon your shell, you might need to "rehash".
> 
> -- 
> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" (devise Shadok)

Re: port select problem

[Ryan Schmidt]

On Oct 25, 2017, at 22:42, pagani laurent wrote:

> Lorenzo>sudo port select --set ipython2 py27-ipython
> Selecting 'py27-ipython' for 'ipython2' succeeded. 'py27-ipython' is now 
> active.
> Lorenzo>ipython2
> ipython2: Command not found.
> 
> Why ?

I can't explain that. It works for me.