Re: Reminders about revisions and when to increase them
On Jun 28, 2020, at 00:38, Blair Zajac wrote: > Is this on the wiki? The guide is where things should be documented. I don't know if all of this is already there. I'm guessing not, since there seems to have been so much confusion about it. I don't spend as much time as maybe I should in updating the guide. For one thing, it's written in docbook XML, which is a pain to modify. Rainer worked on converting it to asciidoc which should be easier to edit once that conversion is finalized. The other problem is that I don't like most of the guide content and its organization so it's very hard to go modify or add just one part since in the process of doing that I see everything else that's wrong and become overwhelmed. > Also, adding if a port: dependency is changed requires a revision bump would > be good to list for completeness, e.g. > https://github.com/macports/macports-ports/commit/9e9d40fea3f205523d842ff78e5ca8063cffbaf0 > . Well yeah. Changing a dependency is the same as adding a dependency and removing another, which change the registry, which requires a revbump.
Re: Reminders about revisions and when to increase them
Is this on the wiki?
Also, adding if a port: dependency is changed requires a revision bump would be
good to list for completeness, e.g.
https://github.com/macports/macports-ports/commit/9e9d40fea3f205523d842ff78e5ca8063cffbaf0
.
> On Jun 25, 2020, at 10:00 PM, Ryan Schmidt wrote:
>
> Hi all,
>
> I want to remind everyone about when you should increase a port's revision
> and when you should not. Please don't increase revisions unnecessarily
> because it wastes time on our build servers and on users systems.
>
>
> You SHOULD increase a port's revision if a user who already has the port
> installed should be prompted to reinstall the port with those changes. The
> most frequent reason is if you are changing the files installed by the port.
> Examples:
>
> * Adding a dependency on a library that the port would use opportunistically
> * Linking with a new version of a dependency's library
> * Adding a patchfile that changes files that will be installed
> * Adding documentation files
> * Renaming or relocating files or directories that are installed
> * Removing a variant
> * Adding a variant to the port's default_variants
>
>
> You SHOULD increase a port's revision if you are adding or removing library
> or runtime dependencies, because those are recorded in the registry. Examples:
>
> * A port X requires libpng but this was not declared in X's portfile. If the
> user happened to have libpng installed already then X builds fine but in the
> registry MacPorts has not recorded that X uses libpng because it doesn't know
> that. As a result, the user would be able to uninstall libpng and MacPorts
> would not warn the user that this would break X. Increase the revision when
> adding the dependency on libpng so that the user must reinstall the port so
> that the correct dependencies get into the registry.
>
>
> You SHOULD NOT increase a port's revision if your change will not change
> anything for users who already have the port installed. Examples:
>
> * Setting or changing the port's license
> * Fixing a build failure, for example adding a build dependency like
> pkgconfig if the port would fail to build without it
> * Removing an unneeded build dependency
> * Adding a new non-default variant
> * Removing a variant from the port's default_variants (variants are preserved
> on upgrades)
> * Fixing a typo in a comment in the Portfile
> * Changing the whitespace of the Portfile
>
>
> Subports are a complication. A single Portfile might define several subports.
> Think carefully when you change revisions which subports the change should
> apply to.
>
> For example, a simple python module port has subports for several versions of
> python but they are all for the same version of that python module. It makes
> sense to have a single version line and a single revision line beneath that
> and to increase the revision for all of the subports at the same time. But
> other python module ports may be more complicated, offering a newer version
> of the python module on newer python versions and an older version of the
> module on older python versions. In that case there are two version lines and
> two revision lines, and you must think carefully about which of the two
> revision lines, or both, should be increased for any particular change. For
> example if you are adding a missing dependency to the new version of the
> python module but that dependency is not used by the old version, you would
> only be increasing the revision of the new version.
>
> In "regular" ports that use subports (i.e. not python/php/perl modules),
> don't forget that the main port is a subport too. In a portfile that has
> subports, just because there's a version line at the top of the portfile
> doesn't mean you should necessarily add a revision line right below it.
> Instead, define separate revision lines for each subport, including the main
> port:
>
> if {${subport} eq ${name} {
>revision0
>...
> }
>
> subport foo {
>revision0
>...
>
> }
>
> subport bar {
>revision0
>...
>
> }
>
> Do include a "revision 0" line, even though that is the default. That way it
> is easy for you and other developers who might have need to modify your port
> to see exactly where revisions could be set. It also makes it easier for
> tools that automatically bump revisions to do so correctly.
>
>
> Many developers seem to have been under the impression that it is necessary
> to increase the revision in order to get the buildbot to retry a build. That
> is not the case.
>
> Every commit sends a notification to the buildbot. The buildbot doesn't look
> at the content of the commit except to determine which ports' source
> directories were modified. It updates to the latest version of macports-ports
> and then checks if an archive already exists on the right server for the
> those ports. If not, it builds and uploads them. The archive name contains
>
Re: shellescape proc
On Jun 27, 2020, at 23:18, Fred Wright wrote: > On Fri, 26 Jun 2020, Ryan Schmidt wrote: > >> Somehow I missed that we have a shellescape proc. This is very exciting! >> I've wanted this function for a long time, and I'm just realizing now that >> it exists and that we've had it for 6 years. >> >> https://github.com/macports/macports-base/commit/494a1feafa2ad46c66f9714b760fb5b50ff5dd48 >> >> Thank you, Clemens! >> >> We should be using this everywhere, such as in portextract.tcl. >> >> It looks like it is available for use in portfiles too. Portfiles should >> switch to using this when specifying filenames or paths in a system call >> instead of manually quoting them with 'quotes' (or, more usually, not doing >> anything). >> >> Looks like neither shellescape nor quotemeta are documented in the guide. > > Perhaps with good reason: > > MacML @21:03:49.192:: Error: Failed to extract curl-ca-bundle: invalid > command name "shellescape" Whoops, didn't realize that it had only just been made available in the not yet released MacPorts 2.7.0 for Portfiles to use. Fixed: https://github.com/macports/macports-ports/commit/6c5276d315e64e99f0a7e57b6cf9940b83e9eab6 > This happens on 10.5, 10.8, and 10.12. The missing binary archives are > consistent with this. That's unrelated. We don't make binaries for 10.5 right now. We used to make binaries for 10.5 ppc but the Power Mac's power supply died and I haven't gotten the replacement Power Mac online yet to take over. The 10.8 and 10.12 buildbot workers were offline for a time because the SSDs they were stored on died. They're back up and running using hard disks for now but there is a large backlog of builds.
Re: shellescape proc
On Fri, 26 Jun 2020, Ryan Schmidt wrote: Somehow I missed that we have a shellescape proc. This is very exciting! I've wanted this function for a long time, and I'm just realizing now that it exists and that we've had it for 6 years. https://github.com/macports/macports-base/commit/494a1feafa2ad46c66f9714b760fb5b50ff5dd48 Thank you, Clemens! We should be using this everywhere, such as in portextract.tcl. It looks like it is available for use in portfiles too. Portfiles should switch to using this when specifying filenames or paths in a system call instead of manually quoting them with 'quotes' (or, more usually, not doing anything). Looks like neither shellescape nor quotemeta are documented in the guide. Perhaps with good reason: MacML @21:03:49.192:: Error: Failed to extract curl-ca-bundle: invalid command name "shellescape" This happens on 10.5, 10.8, and 10.12. The missing binary archives are consistent with this. Fred Wright
Re: randomly bumping things to require perl 5.30 vs 5.28 requires everyone to have both installed ...
> On Jun 23, 2020, at 12:48 AM, Ryan Schmidt wrote: > > My ports are slightly out of date but I don't see a choking amount of > dependencies for curl on High Sierra: > > $ port rdeps curl > The following ports are dependencies of curl @7.70.0_0+ssl: > xz >lbzip2 >libiconv > gperf >gettext > ncurses > pkgconfig > libidn2 >autoconf >automake >libtool > xattr >unzip >libunistring > perl5 >perl5.28 > db48 > gdbm >readline > texinfo >help2man > p5.28-locale-gettext >perl5.30 > libpsl >python38 > bzip2 > expat > libedit > libffi > openssl >zlib > sqlite3 > python_select > python3_select >glib2 > libxml2 >icu > pcre > curl-ca-bundle > > > If you are on an older system you may have additional dependencies in the > chain, likely compilers, but that's not curl's fault; it doesn't impose any > compiler restrictions. Its dependencies might. > some trimming of the curl portfile, removing the deps on libidn and libpsl (which macports-base does not appear to need in curl to run), and thanks for to you for removing perl from curl-ca-bundle recently, and I now have a very lean built-from-os-roots curl sufficient to support macports-base and many other purposes down to this most set of ports, even on older systems: $ sudo port -v install curl ---> Computing dependencies for curl.. The following dependencies will be installed: curl-ca-bundle gperf libiconv openssl pkgconfig pkgconfig calls in gperf and libiconv, otherwise it’s just openssl and curl/curl-ca-bundle. I suppose I could even do without pkgconfig as well, and just pass in the paths, but that’s lean enough for now. No perls, no pythons. I’ll call it skip-the-dishes-curl or something ... Ken
Re: Cached distfile with the same name
Awesome, thank you Ryan. On Sat, Jun 27, 2020 at 1:30 PM Ryan Schmidt wrote: > > > On Jun 27, 2020, at 11:32, Herby G wrote: > > > I have a situation where a particular port is using a distfile of the > same name from version to version. > > > > So there's currently an older version of the port's distfile cached in > the MacPorts mirrors. > > > > Unfortunately, trying to update the port is failing since the distfile > is first fetched from the MacPorts mirror. > > > > Since the old and new version of the distfile have the exact same name, > checksums won't compute for the new version of the Portfile. > > > > How is this normally handled? > > https://trac.macports.org/wiki/PortfileRecipes#unversioned-distfiles > > >
Re: Cached distfile with the same name
On Jun 27, 2020, at 11:32, Herby G wrote: > I have a situation where a particular port is using a distfile of the same > name from version to version. > > So there's currently an older version of the port's distfile cached in the > MacPorts mirrors. > > Unfortunately, trying to update the port is failing since the distfile is > first fetched from the MacPorts mirror. > > Since the old and new version of the distfile have the exact same name, > checksums won't compute for the new version of the Portfile. > > How is this normally handled? https://trac.macports.org/wiki/PortfileRecipes#unversioned-distfiles
Re: randomly bumping things to require perl 5.30 vs 5.28 requires everyone to have both installed ...
On 2020-06-27, at 6:27 AM, Daniel J. Luke wrote: > On Jun 22, 2020, at 11:20 PM, Ryan Schmidt wrote: >> On Jun 22, 2020, at 14:34, Daniel J. Luke wrote: >>> We should just have one perl5 port that tracks the current release. >> >> You say this every year (or at least often). > > I say this every time we run into the set of problems that we would solve by > moving to this method (that we continually avoid solving just kicking the can > down the road with the current solution - which is a huge amount of work that > we just have to repeat every time there's a perl release). > >> Are you volunteering to be the one to ensure that every port that uses perl >> is compatible with the new perl release when it comes out? Without someone >> to do that, blindly upgrading everything from say perl 5.28 to 5.30 will >> likely break ports. > > Every time we upgrade a library we 'blindly break ports' since we don't (and > can't) comprehensively test every combination of things. > > It sounds like your argument against doing this is "we want the ports tree to > be stable" which I don't think has been the case in the past (and if we /do/ > want it to be a stable tree, we shouldn't be doing may of the updates that we > do now). > >> Do you remember perl 5.26? It broke a lot of ports, requiring a lot of fixes >> like this to be added: >> https://github.com/macports/macports-ports/commit/454eb2b0608266ab7bdf51a82d690be0f97610fe > > I do, part of the pain with perl5.26 was that we waited a long time to update > things because everyone was afraid of fixing the things that were broken. > > I'll note that upstream already does test CPAN modules with new versions of > perl (and notifies their version of maintainers) - so the things that remain > broken are things that aren't actively maintained upstream (and if they > remain broken in our ports tree, aren't being actively maintained by us > either). > >> The strategy currently being employed by those volunteers who are >> maintaining the perl ports is to keep everything defaulted to 5.28 for now, >> add 5.30 ports and fix problems in them as they're found, and once >> everything is building then switch the default to 5.30. This seems sensible >> to me. > > Things get fixed faster when they're broken, I'd bet perl 7 (which is what > perl5.32 is going to be) will be out before we're moved to perl5.30 (of > course, perl 7 is going to break some backwards compatibility). I suspect the > fundamental disagreement here is that I'm more comfortable with breaking > things in the ports tree (and then fixing them) than you are. > > -- > Daniel J. Luke > So in the end, are we setting texinfo back to perl5.28 ? K
Cached distfile with the same name
Hello all, I have a situation where a particular port is using a distfile of the same name from version to version. So there's currently an older version of the port's distfile cached in the MacPorts mirrors. Unfortunately, trying to update the port is failing since the distfile is first fetched from the MacPorts mirror. Since the old and new version of the distfile have the exact same name, checksums won't compute for the new version of the Portfile. How is this normally handled? Thanks in advance.
Re: Best practices for port testing environment?
A separate macOS install in Parallels or Fusion would be another option, I guess. > Am 27.06.2020 um 16:31 schrieb Joshua Root : > > On 2020-6-28 00:23 , David Richmond wrote: >> New here to hacking on MacPorts; please talk to me like I'm dumb. Can >> anyone point me to a resource (or offer their own thoughts) on setting >> up test environments for ports? >> >> The MacPorts Guide obviously describes setting up a local repository, >> but if I am, say, chasing a bug down during build, what is the >> best/easiest way to create a "clean" test environment? That is to say, >> say I have successfully built port_foo on my machine, but now a user or >> bug report says it's broken. (Perhaps a dependency is out of whack.) How >> can I build port_foo from "scratch," as if no ports at all are >> installed? (Without breaking my own, active macports tree, that is.) >> >> I could just rename the port in the Portfile in my local repository, but >> macports will find my local installed dependencies and it seems to me I >> want a "clean" environment where it won't. >> >> And please do redirect me to a different approach altogether if I am >> barking up the wrong tree. > > You can use trace mode (-t option) to hide any ports that are not > declared as dependencies of the port being built. There are other > possible approaches like installing in a different prefix or temporarily > deactivating everything, but trace mode is the easiest and works well in > most cases, the main drawback being reduced performance. > > - Josh
Re: Best practices for port testing environment?
On 2020-6-28 00:23 , David Richmond wrote: > New here to hacking on MacPorts; please talk to me like I'm dumb. Can > anyone point me to a resource (or offer their own thoughts) on setting > up test environments for ports? > > The MacPorts Guide obviously describes setting up a local repository, > but if I am, say, chasing a bug down during build, what is the > best/easiest way to create a "clean" test environment? That is to say, > say I have successfully built port_foo on my machine, but now a user or > bug report says it's broken. (Perhaps a dependency is out of whack.) How > can I build port_foo from "scratch," as if no ports at all are > installed? (Without breaking my own, active macports tree, that is.) > > I could just rename the port in the Portfile in my local repository, but > macports will find my local installed dependencies and it seems to me I > want a "clean" environment where it won't. > > And please do redirect me to a different approach altogether if I am > barking up the wrong tree. You can use trace mode (-t option) to hide any ports that are not declared as dependencies of the port being built. There are other possible approaches like installing in a different prefix or temporarily deactivating everything, but trace mode is the easiest and works well in most cases, the main drawback being reduced performance. - Josh
Best practices for port testing environment?
New here to hacking on MacPorts; please talk to me like I'm dumb. Can anyone point me to a resource (or offer their own thoughts) on setting up test environments for ports? The MacPorts Guide obviously describes setting up a local repository, but if I am, say, chasing a bug down during build, what is the best/easiest way to create a "clean" test environment? That is to say, say I have successfully built port_foo on my machine, but now a user or bug report says it's broken. (Perhaps a dependency is out of whack.) How can I build port_foo from "scratch," as if no ports at all are installed? (Without breaking my own, active macports tree, that is.) I could just rename the port in the Portfile in my local repository, but macports will find my local installed dependencies and it seems to me I want a "clean" environment where it won't. And please do redirect me to a different approach altogether if I am barking up the wrong tree. Thanks, David
Re: randomly bumping things to require perl 5.30 vs 5.28 requires everyone to have both installed ...
On Jun 22, 2020, at 11:20 PM, Ryan Schmidt wrote: > On Jun 22, 2020, at 14:34, Daniel J. Luke wrote: >> We should just have one perl5 port that tracks the current release. > > You say this every year (or at least often). I say this every time we run into the set of problems that we would solve by moving to this method (that we continually avoid solving just kicking the can down the road with the current solution - which is a huge amount of work that we just have to repeat every time there's a perl release). > Are you volunteering to be the one to ensure that every port that uses perl > is compatible with the new perl release when it comes out? Without someone to > do that, blindly upgrading everything from say perl 5.28 to 5.30 will likely > break ports. Every time we upgrade a library we 'blindly break ports' since we don't (and can't) comprehensively test every combination of things. It sounds like your argument against doing this is "we want the ports tree to be stable" which I don't think has been the case in the past (and if we /do/ want it to be a stable tree, we shouldn't be doing may of the updates that we do now). > Do you remember perl 5.26? It broke a lot of ports, requiring a lot of fixes > like this to be added: > https://github.com/macports/macports-ports/commit/454eb2b0608266ab7bdf51a82d690be0f97610fe I do, part of the pain with perl5.26 was that we waited a long time to update things because everyone was afraid of fixing the things that were broken. I'll note that upstream already does test CPAN modules with new versions of perl (and notifies their version of maintainers) - so the things that remain broken are things that aren't actively maintained upstream (and if they remain broken in our ports tree, aren't being actively maintained by us either). > The strategy currently being employed by those volunteers who are maintaining > the perl ports is to keep everything defaulted to 5.28 for now, add 5.30 > ports and fix problems in them as they're found, and once everything is > building then switch the default to 5.30. This seems sensible to me. Things get fixed faster when they're broken, I'd bet perl 7 (which is what perl5.32 is going to be) will be out before we're moved to perl5.30 (of course, perl 7 is going to break some backwards compatibility). I suspect the fundamental disagreement here is that I'm more comfortable with breaking things in the ports tree (and then fixing them) than you are. -- Daniel J. Luke
