Re: [macports-ports] 01/02: New port for wallet @1.3

[Clemens Lang]

Hi Ryan,

On Sat, Nov 05, 2016 at 03:40:48AM -0500, Ryan Schmidt wrote:
> You should not use newlines in notes, unless you actually want the
> user to see a newline there, for example if you are starting a new
> paragraph or are constructing a list of items.

Thanks, fixed:
 https://github.com/macports/macports-ports/commit/f601b79

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PR usage by people with commit access

[Clemens Lang]

Hi,

On Fri, Nov 04, 2016 at 10:10:44AM -0700, Ivan Larionov wrote:
> I think this is a good practice for port maintainers with write access
> to repository still use PRs instead of direct commits to master.

I agree, with some limitations:

> * Better visibility of changes. Instead of cloning full repository and
> digging through history I can search PRs by port name and see changes
> which were done by maintainers and contributors.

It's more work. We're all doing this in our free time and clicking
through GitHub's Web UI a couple of times just for the searchability is
not worth the extra effort. If you want the full picture, you'll have to
search the history anyway, which luckily is now much faster than it used
to be with Subversion.

> * Using the same change methods as outside contributors may help to
> develop better PR flow. Currently I see some lack of the standard
> flow, maintainers commit contributors' changes in different ways, PRs
> marked as rejected while they're actually merged, people ask to enable
> "Allow edits from maintainers" for PR, etc.

Absolutely agree.

> * Ability to get a feedback / review from other project members.

Agree for the cases where other people interested in the ports and
capable of reviewing exist, and those people have the time to review in
a timely fashion.

> We use private github setup on my work and we have a rule that you
> shouldn't commit directly to master in a project with multiple
> contributors until it's very small change like fixing typo. Open PR,
> ask for review, merge. Or fix issues and merge if you got any useful
> comments on PR.

We do the same thing at my workplace, but this only works because there
are other people with an interest to review your changes (being paid to
do it). I'm not sure whether we have the manpower to review every commit
pushed into MacPorts.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Using a different bugtracker [Was: Re: Poll: should Trac send email notifications when adding or replacing an attachment?]

[Clemens Lang]

Hi,

- On 3 Nov, 2016, at 10:58, René J.V. Bertin rjvber...@gmail.com wrote:

> Not to open another can of worms, but just how married are we to trac? Of 
> course
> it's never evident to migrate a web service, but has it never been considered 
> to
> investigate other popular bug trackers (e.g. bugzilla) and possibly switch 
> over
> by redirecting all new ticket requests to the new service?
> Github also has an issue tracker, for instance. I'm not familiar enough with 
> its
> advanced features. I couldn't say if it does attachments, but it does have 1
> rare feature I like very much: replying by email.

We actually investigated other solutions for the GitHub move. Some of us would
have preferred using the GitHub issue tracker, and we even had a conversion of
our tickets to GitHub already, but the longer we looked at it, the more problems
and limitations we actually found, which is why we ended up staying with Trac.

We also looked into other bugtrackers, and there was no clear benefit over Trac
to justify switching.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Proposal for a MacPorts'ish commit message template

[Clemens Lang]

On Wed, Nov 02, 2016 at 05:46:16PM -0500, Ryan Schmidt wrote:
> Well, you mentioned vim. Very occasionally I use that. I don't
> remember seeing any particular syntax highlighting in it when editing
> a commit message.

You need syntax highlighting (:syntax on) and ideally color enabled.
Additionally, the filetype must be set to 'gitcommit' (:setfiletype
gitcommit). I have "syntax on" in my ~/.vimrc and the filetype is
automatically set when I get prompted for a git commit message. I don't
know how that happens.

To enforce a certain textwidth, you can just :set textwidth=72. Text
typed after you set this will automatically wrap. Other text can be
reflowed, for example by selecting it in visual mode (shift+v) and
pressing gq.

> Usually I use TextWrangler or TextMate.

TextMate has a file type for git commit messages. Unfortunately, it only
seems to highlight overlong summary lines and nothing else (e.g. it does
not warn you about text in the second line). You can get a wrap column
in TextMate using View > Show Wrap Column and set the position of the
line in View > Wrap Column...

I do not know TextWrangler.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub usernames as maintainers

[Clemens Lang]

On Wed, Nov 02, 2016 at 06:45:21PM -0400, Brandon Allbery wrote:
> For me, it looks too much like an email address when combined with
> other things (e.g. "maintainers openmaintainer @foo").

I think that's a formatting problem. We could just as well print
https://github.com/neverpanic in the output of port info --maintainer,
which is (a) not ambiguous, and (b) clickable.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Proposal for a MacPorts'ish commit message template

[Clemens Lang]

On Wed, Nov 02, 2016 at 05:33:28PM -0500, Ryan Schmidt wrote:
> > vim does that with syntax highlighting automatically nowadays when
> > it notices you are writing a commit message. If you need an
> > indication on your line width, may I suggest you configure your
> > editor appropriately?
> 
> Can we document in the WorkingWithGit page how to do that? I have no
> idea.

Which editor are you using? It differs by editor.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub usernames as maintainers

[Clemens Lang]

On Wed, Nov 02, 2016 at 05:27:30PM -0500, Ryan Schmidt wrote:
> On Nov 2, 2016, at 1:49 AM, Mojca Miklavec wrote:
> > It would be nice if "port info" would also print the github
> > usernames of maintainers.
> 
> When we were originally planning the transition to GitHub, I suggested
> that we allow an "@" syntax in the maintainers line for GitHub
> usernames (i.e. "maintainers @neverpanic" for Clemens). I don't
> remember right now why we then decided that wasn't a good idea.

I think we should re-visit that decision. Maybe I'll find a couple of
minutes to implement this in base. If so, I'll send a PR for discussion.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [macports-ports] branch master updated: libcxx: Bump to 3.9.0 (#52666)

[Clemens Lang]

Hi,

On Wed, Nov 02, 2016 at 09:57:37AM -0700, Bradley Giesbrecht wrote:
>  Can you pass multiple tickets (closes x y z)?

Yes, space-separated, comma-separated, or because the ticket references
are quite long, just with multiple Closes $ticket lines.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Proposal for a MacPorts'ish commit message template

[Clemens Lang]

Hi,

On Tue, Nov 01, 2016 at 11:51:52PM +0100, Marko Käning wrote:
> I know that many of you weren't in favour of a commit message
> template, but I propose one anyway, which I derived from KDE’s neat
> one,

Developers are free to use your template if they want to. We don't want
to mandate using it though (since we couldn't enforce it anyway).

> as I find it on the console quite handy to know when 50 or 72
> characters are reached in a line:

vim does that with syntax highlighting automatically nowadays when it
notices you are writing a commit message. If you need an indication on
your line width, may I suggest you configure your editor appropriately?

> # --[ Links to issues on MacPorts' trac ]--|
> #ISSUE:  
> #RESOLVES:   
> #BLOCKED BY: 

You could have taken the time to actually adjust this to what MacPorts'
Trac instance accepts. See
 https://trac.edgewall.org/wiki/CommitTicketUpdater
for the description of the plugins that does this for us and
 https://github.com/macports/trac.macports.org/blob/master/conf/trac.ini#L251
for the configuration we use.

> # --[ Links to other pull requests at GitHub ]—|
> #PR:  
> #
> ##EXAMPLE:
> # PR:  #123

Could use the documented keywords GitHub accepts to handle pull requests
(you can close pull requests from commit messages!)

Additionally, I don't like the upper case keywords.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

GitHub usernames as maintainers [Was: Re: [macports-ports] branch master updated: berkeleygw: Remove svn $Id$ line.]

[Clemens Lang]

On Tue, Nov 01, 2016 at 07:42:23PM +0100, Mojca Miklavec wrote:
> If someone submits a pull request for a maintained port, the
> maintainer should probably be notified explicitly with @username. The
> problem is that currently only Trac maintainers have access to that
> mapping.

It's not a set of data I'd like to keep forever. We're requesting
people's non-public email addresses from GitHub to do the migration, not
to implement arbitrary other features on top of that data.


> At some point (rather sooner than later) we'll have to start adding
> GitHub usernames to Portfiles (we might have to extend the base to
> properly support that).

I agree, especially because the GitHub username is now also valid in
Trac. At the moment, you'd get new tickets sent to your email address,
not your Trac account (unless a user notices the equivalence in the
auto-completion box).

> And quite honestly I would also be in favour of asking committers to
> use the same MacPorts handle as their GitHub account name (and only
> keep the old ones as aliases).

I like my handle, but I can see how it could lead to confusion. On the
other hand, the maintainer handles are bound to become less important in
the future anyway. If you want to reach a committer, there usually are
contact methods on their GitHub profile.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: rev-upgrade question

[Clemens Lang]

On Tue, Nov 01, 2016 at 07:09:48PM +0100, Marko Käning wrote:
> how can I convince rev-upgrade to actually rebuild broken ports if
> 'revupgrade_mode' is set to ‘report' in macports.conf? 

You can specifically rebuild them manually using port -nf upgrade $port.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Removing "$Id$" lines

[Clemens Lang]

Hi,

On Tue, Nov 01, 2016 at 10:38:42AM -0700, Bradley Giesbrecht wrote:
> Would it work as well to have the buildbots ignore commits with
> unchanged epoch_version_revision.

That's https://trac.macports.org/ticket/52765. It would still take the
time to process the change and figure out that a port has already built
successfully before, and it would still trigger builds for ports that
previously failed to build (because that particular commit could be the
one that fixes the build, which you don't know unless you try).

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: querying the registry for reverse dependencies

[Clemens Lang]

Hi,

On Tue, Nov 01, 2016 at 03:10:30PM +0100, René J.V. Bertin wrote:
> A variant of my earlier question: is there a magic formula to query
> the registry which files (and/or ports) depend on a given binary file?
> IOW, an elegant alternative to something like

No, there's no such thing in the registry. The registry only has
per-port information, so the best you can do is a combination of 'port
provides' and 'port dependents'.

> %> sudo bzip2 -v foo
> %> sudo port -v rev-upgrade
> %> sudo bunzip2 foo.bz2

I don't understand how bzip2 and port rev-upgrade relate.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [macports-ports] branch master updated: jasper: needs a revbump

[Clemens Lang]

On Mon, Oct 31, 2016 at 08:08:26PM +0100, Marko Käning wrote:
> Doesn’t the second build of jasper justify the revbump, or should we
> simply ignore this and let rev-upgrade take care of it??

The revbump is justified. Your commit message should have mentioned why
it was necessary, though.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hi,

On Mon, Oct 31, 2016 at 10:25:33AM +0100, Davide Liessi wrote:
> I have just noticed in one of my GitHub repositories that in Settings
> -> Options there is a "Merge button" section that lets you
> enable/disable the behaviours of the merge button described in the
> above link.
> Maybe you could leave only "Allow rebase merging" enabled and disable
> the rest.

We already did this, but thanks for the pointer.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Clemens Lang]

Hi,

On Mon, Oct 31, 2016 at 10:18:42AM +0100, René J.  V. Bertin wrote:
> I may have overlooked this, but does github have any provisions that
> would allow the PortIndex files to be generated on the server and
> served with the actual repo contents? That would probably give a very
> significant reduction in the resources spent collectively to
> regenerate those files...

Just as with Subversion, the answer is no. Remember that the PortIndex
is specific to the macOS version you are running, so a server-generated
PortIndex could only generate all of them into different files.

Additionally, git does not preserve timestamps from the repository on
checkout, so you might actually end up re-generating the index locally
anyway.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

GitHub migration complete

[Clemens Lang]

Good day MacPorts developers and users,

We are pleased to announce that MacPorts has moved to GitHub.

Our Subversion repository has been split into several repositories on
GitHub . The buildbot, email notifications, and Trac are now triggered
by changes made on GitHub.

MacPorts developers should now have commit access to the GitHub
repositories. If you are a MacPorts developer and have not yet joined
the MacPorts GitHub organization, please follow instructions in the FAQ
linked below.

If you have further questions, please refer to this new FAQ page:
https://trac.macports.org/wiki/FAQ/GitHubMigration

If your question is not yet answered, ask on the mailing lists so it can
be added.

We have tested the changes thoroughly, but in a system this complex
there might still be undiscovered problems. If you notice any, please do
not hesitate to ask on the mailing lists or file a ticket in Trac
against the "server/hosting" component.


On behalf of the MacPorts migration team,
Lawrence Velázquez
Rainer Müller
Ryan Schmidt
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Subversion switched to read-only

[Clemens Lang]

Hi everybody,

we are making Subversion read-only in a couple of seconds. If you get an
error message when comitting that is expected.

We will re-configure a couple of hooks and tools to the new GitHub-based
setup and then enable write access on GitHub. We try to keep the
downtime short and will keep you updated.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Compiling dnsmasq with dnssec support?

[Clemens Lang]

Hi,

On Sun, Oct 30, 2016 at 04:50:33PM +0100, Johannes Kastl wrote:
> Thanks Rainer, I'll try that.

I'd suggest to just enable it without a variant. If it's configurable at
runtime, I don't see the need to make users jump through hoops.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: git fetch fails because of gnutls_handshake

[Clemens Lang]

Hi,

On Sat, Oct 29, 2016 at 08:21:23PM -0700, Fred Wright wrote:
> Though if the new "Cc me" button were any smaller, it would be
> considered a web bug. :-)

That's
  https://trac.macports.org/ticket/52675
If you have time, read through the ticket and submit a pull request to
fix it.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: git fetch fails because of gnutls_handshake

[Clemens Lang]

Hi,

On Sat, Oct 29, 2016 at 04:38:37PM +0200, René J.V. Bertin wrote:
> OK, I must admit this is on Linux; I omitted that to avoid too quick
> "we don't support that replies" O:-)

We don't support that. And the fact that you've just wanted three
people's time for looking into this even tells us why that's a good
policy.

> My system gnutls is kind of old though:
> %> /usr/bin/gnutls-cli -version
> /usr/bin/gnutls-cli (GnuTLS) 2.12.23
> Packaged by Debian (2.12.23-12ubuntu2.5)

So go shout at whoever gave you that package, instead of wasting our
time here.

> but from what I understand gnutls itself isn't the likely culprit?

Considering the error output I have a hard time following how you come
to that conclusion.

> I may investigate the workaround I proposed on trac, falling back to
> using an external curl command.

I will veto this code for the reasons outlined in Trac.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: git fetch fails because of gnutls_handshake

[Clemens Lang]

Hi,

On Sat, Oct 29, 2016 at 02:48:32PM +0200, René J.V. Bertin wrote:
> Hi,
> 
> I'm seeing this on one of my systems (haven't been able to try others yet):
> 
> {{{
> DEBUG: Executing org.macports.fetch (git)
> --->  git-2.10.1.tar.gz doesn't seem to exist in 
> /opt/local/var/macports/distfiles/git
> --->  Attempting to fetch git-2.10.1.tar.gz from 
> https://www.kernel.org/pub/software/scm/git/
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   178  100   1780 0419  0 --:--:-- --:--:-- --:--:--   418
>   0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
> DEBUG: Fetching distfile failed: gnutls_handshake() failed: Handshake failed
> }}}
> 
> As far as I can tell the fetch procedure uses a curl function provided
> by Pextlib, which in turn uses the host's gnutls libraries. This might
> suggest that those are outdated w.r.t. www.kernel.org, but when I use
> the system curl command the fetch completes without problems.

That's https://trac.macports.org/ticket/51516.


> What could this be due to?

Your system libcurl is too old to talk modern SSL, or you have modified
your system libcurl to use a different library, or you have built
MacPorts against a custom copy of libcurl that uses a SSL library that
cannot talk to kernel.org.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: querying the registry for installed files

[Clemens Lang]

On Fri, Oct 28, 2016 at 08:41:58PM +0200, René J.V. Bertin wrote:
> I think the registry keeps track of all installed files, whether they
> belong to active or inactive ports, right?
> If so, is there a way to query the registry with a filename pattern,
> to obtain the full paths of known matching filesand the port(s) which
> provide them?

Isn't this essentially what 'port provides' does?

But yes, that's possible:
 sqlite> .load macports.sqlext
 sqlite> select
  ports.name
, files.path
 from files
 join ports
 on files.id = ports.id
 where
files.active = 1
and ports.name = 'apr'
and files.path like '%/include/%';


-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Trac down again

[Clemens Lang]

On Wed, Oct 26, 2016 at 09:13:33PM +0200, Marko Käning wrote:
> Where is our trac again?

I've kicked it. Not exactly sure what causes these outages, it's
probably some crawler that fills a queue with requests and causes all of
them to be dropped due to timeout after a while.

We're keeping an eye on it.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: "Failed to open portfile from registry" while reinstalling because of configure.optflags

[Clemens Lang]

Hi,

On Wed, Oct 26, 2016 at 02:17:53AM +0200, René J.V. Bertin wrote:
> Yeah. There could be a check if the Portfile exists to catch the 1st
> possibility

That would introduce a race condition, wouldn't it? Checking whether a
file exists before doing something with it is a classic mistake.

> and some more information about the error (if there was one) could be
> useful too; the catch command can return a string with that
> information.

That information already exists, and you'll see it in debug mode.

> > Additional files are not saved alongside the port. You should
> > probably avoid depending on external files for the pre- and
> > post-deactivate phases and running the Portfile.
> 
> Or doing anything with them in the shared parts of the Portfile...

That's what I meant when I said "running the Portfile".

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: "Failed to open portfile from registry" while reinstalling because of configure.optflags

[Clemens Lang]

Hi,

On Wed, Oct 26, 2016 at 12:29:10AM +0200, René J.V. Bertin wrote:
> The issue I mentioned that I fixed was such an example of an
> oversight. I'd imposed the condition of reading a "companion"
> PortGroup from the same directory as the main PortGroup, and that
> condition cannot hold for copies in the registry which all live in
> their own directory.

So the behavior was actually correct.


> But the configure.optflags issue is not due to an issue in a Portfile
> or PortGroup. As far as I can tell it happens with every port; it just
> goes unnoticed because most of the time people (presumably) don't
> specify additional compiler flags.

While the possiblity to set these flags on the command line exists, this
is for development purposes only and unsupported. If you have a patch to
fix this, I'm happy to apply it.


> BTW: what is ${filespath} set to for ports executed from the registry?
> If it doesn't point to the usual location there's probably room for a
> mechanism to specify additional files to store in the registry port
> dir.

Additional files are not saved alongside the port. You should probably
avoid depending on external files for the pre- and post-deactivate
phases and running the Portfile.

We could argue that's a bug, but this has worked quite fell so far.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: "Failed to open portfile from registry" while reinstalling because of configure.optflags

[Clemens Lang]

Hi,

On Tue, Oct 25, 2016 at 05:06:27PM -0500, Ryan Schmidt wrote:
> Ah. So the error message is just wrong? There isn't actually a problem
> opening the portfile from the registry; there's a problem running the
> portfile that was successfully opened and read?

Yes, unless of course you randomly deleted files in the path where
MacPorts keeps those historic copies of Portfiles, or there's a bug in
the code that maintains these files.

The problem is that we don't see the difference. The code that procudes
the error is more or less
  "source $filename"
which can fail either because the file isn't there, or because the code
in the file failed.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: "Failed to open portfile from registry" while reinstalling because of configure.optflags

[Clemens Lang]

Hi,

On Tue, Oct 25, 2016 at 08:34:31AM -0500, Ryan Schmidt wrote:
> I don't know the answers to your questions, but I wanted to mention
> that I also have noticed the "Failed to open portfile from registry"
> error far more often than seems normal to me. ("Never" would seem
> normal to me.)

Interesting. The last time I've seen any of these messages is for the
last GHC upgrade, and they were indeed expected and could not be avoided
there.

On Tue, Oct 25, 2016 at 07:17:23PM +0200, René J.V. Bertin wrote:
> Yes, never would seem normal, esp. if the Portfile is there when you
> check :)

When a port is opened from registry, it does not use the state of the
Portfile that you currently have in your ports tree. It uses the state
of the Portfile as it was when you installed the port. This also applies
to PortGroups.

As such, Portfiles or PortGroups that haven't been written carefully can
trigger this warning from time to time. A common problem is checking for
the presence of a file, or running a tool, and not catching the errors.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hi,

On Tue, Oct 25, 2016 at 12:17:57AM +0200, Marko Käning wrote:
> A description of how exactly one would rebase (potentially squash and
> history-rewrite) a submitted PR onto current master should be on our
> WorkingWithGit wiki page.

the easiest approach is just clicking the button that does this on
GitHub. Of course there's also a way to do it from the command line.

> At the moment it is not very clear to me how a MacPorts committer
> would actually deal with a pull request submitted by a port maintainer
> to the central repository. But I’ve got to admit that I haven’t read
> much more than our wiki page up to now. There are surely more details
> in GitHub’s help...

Part of the problem here is that we're not exactly sure either. We'll
just see how it goes for the first few PRs and then define the rules as
we see what works for us.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Git tools for managing patchsets

[Clemens Lang]

On Mon, Oct 24, 2016 at 11:57:51PM +0200, Marko Käning wrote:
> The only question mark I have there is:
> 
>   Will we ask the committers to squash their changesets or prefer to
>   clutter the main repo with potentially many tiny iterations for the
>   changed ports??

I think "clutter" is not the appropriate word here. We will also not ask
committers to generally squash all their changes.

> Personally I don’t like history rewriting, but a squash every now and
> then seems fine to me, as an update to a port sometimes requires a few
> iterations until it is ready for pushing to the central repo and it is
> usually one logical unit deserving an atomic commit.

It will probably be up to you as a MacPorts developer. I agree that
logical units should be committed atomically, but that doesn't mean that
you cannot commit multiple atomic changes in one push or pull request.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

On Mon, Oct 24, 2016 at 08:22:38PM +0200, Mojca Miklavec wrote:
> Even if the method of achieving this is not prescribed***, I wouldn't
> mind a bit of testing before screwing up the real repository.
> 
> *** But having some cheatsheet would help.

Sure, feel free to create a fork, play around, add me to the review and
I'll find stuff that you can change so you can play around ;-)

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hi,

On Mon, Oct 24, 2016 at 08:00:04PM +0200, Vincent Habchi wrote:
> I’ve bought Coda 2 when I use to do a bit of HTML development. Can I
> use it to check out - tinker with the new MacPorts GIT repository?

You should check with the developers of Coda on their Git support. I
don't think a tool built especially for website editing will be the best
choice, but maybe it works for you.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

On Mon, Oct 24, 2016 at 07:47:18PM +0200, Mojca Miklavec wrote:
> I can send you a screenshot comparing the version I opened two hours
> ago and the same page reloaded just now. The result changed in the
> meantime. In any case I can no longer provide you any broken example
> (there are still some for other developers, but I cannot judge about
> whether other developers did the migration already or not).

So that's probably because the conversion does not happen instantly, but
is processed in a delayed cronjob and wasn't finished when you initially
looked at it.

> The problem I have right now though is how to list the tickets owned /
> reported by me. The query I used on the old trac no longer works.
> 
> For example:
> [[TicketQuery(status=assigned|new|reopened&owner~=mo...@macports.org)]]

Use your GitHub username as owner instead of the email address.

> The idea is not to play with it on my own. I know how trivial git
> commands work. What is not yet clear to me is whether we would be
> clicking the gui buttons to accept pull requests or do some
> non-conventional steps of merging multiple commits, adding our changes
> on top, rebasing to master etc.

I would leave that up to the developers. Previously, GitHub did not
support rebasing pull requests, but that was fixed a while ago, so now
you can also merge PRs by rebasing them on top of master.

I don't think we should mandate a complex "run these magic git commands"
workflow. Making things complicated will just make them go wrong.

> (My preference would be to keep linear history for master and not to
> keep ten broken revisions of a Portfile resulting from stepwise
> improvements in a pull request, but it would be nice to do some
> testing first.)

Yes, that's also my preference. So we can agree on:
 - rebase when merging PRs
 - rewrite history on PR branches until it looks good

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Git tools for managing patchsets

[Clemens Lang]

Hi,

On Mon, Oct 24, 2016 at 10:29:19AM -0700, Michael wrote:
> My understanding -- and maybe this is my error here -- is that your
> patches have to be constantly rebased onto the current version every
> time the upstream releases a new version.

I think our understanding of what "upstream" is in this sentence
differs. We are *not* talking about patches to be applied to packaged
software, we are talking about changes to the port definitions, i.e. the
Portfiles themselves. "Upstream" in this case means the master branch of
github.com/macports/macports-ports, as opposed to a branch in a fork of
your own that you may be using to prepare changes for a pull request.

> When you rebase, you have new commits, and a new history. So the
> history of how your patchset has changed over time resets at each
> rebase at each new upstream release.

There is no "upstream release", just the continous stream of development
happening in macports-ports. We request that all changes to Portfiles
should cleanly rebase on top of our current HEAD of master, so that we
don't get at least two commits (the change & a merge commit) for each
pull request. Yes, the history of that change while it is developed may
be altered by the rebase, but these changes should ideally not be
long-running patches.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hi,

On Mon, Oct 24, 2016 at 06:50:47PM +0200, Mojca Miklavec wrote:
> Is that true also for any other email we used prior to becoming
> committers?

Yes.

> Can new emails be added later?

Yes, but you'll have to relogin.

> How exactly does it work when people enter multiple emails? (Judging
> from, say #37017, I guess that whenever I was assigned as the owner of
> the ticket with my old pre-committer-email, that would now be replaced
> with my github handle.)

All email addresses you add will be replaced with your GitHub account.

> One thing I'm confused about is that whenever I'm listed as a reporter
> or in CC, my name would be replaced by all the three data (my macports
> handle, my full email, my github account), while the tickets where I'm
> the owner only contain my macports email. Tickets where the owner has
> been assigned later contain full info about that owner. I didn't
> investigate too closely yet.

I could not find any tickets assigned to your email addresses. Maybe you
checked before the migration ran?

> May I suggest creating a clone of the port repository on GitHub (in
> whatever state it is now) and let it serve as a playground for testing
> different strategies of pushing, using pull requests, merging,
> properly rebasing, merging several commits of a pull request together,
> editing pull requests by non-committers (when a pull request needs
> just a tiny bit of fixing: will that be one commit or original commit
> + edits by "committer"), ...

You can do all of that in a fork of the repository.

> It would help a lot if we had at least some idea how we want to
> proceed after the official switch. And some "grace period" to test
> what we want to do and what we want to avoid. Some playground for
> people that are new to git & and github wouldn't hurt either.

Proposals for rules are very welcome. Feel free to start a wiki page so
we can have a discussion. Also note that any testing can easily be done
in a fork of the repository.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Trac & Subversion available again

[Clemens Lang]

Hi,

On Mon, Oct 24, 2016 at 12:54:23PM -0400, Daniel J. Luke wrote:
> This looks like it's true for me as well (cannot edit tickets beyond
> making comments).

Try clearing your cache (a force-reload should do that). It seems the
problem can also be that the edit form is there, but hidden for you due
to some CSS rules.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Git tools for managing patchsets

[Clemens Lang]

Hi,

On Mon, Oct 24, 2016 at 09:37:37AM -0700, Michael wrote:
> So since MacPorts is moving to git, and from what I saw in the "how to
> use git" docs you mentioned, you apparently want people to work with
> patchsets rebased onto the current head from upstream.
> 
> As I was thinking about that, I realized that you lose your history of
> the patchset in the process.

If I understand this correctly, this is a problem that applies to
repositories of source code, not repositories of build description
files. The way we currently keep patches for our ports is putting them
next to the Portfiles in what will be the macports-ports Git repository.
Consequently, we already have the history of these patchfiles.

As an example, let's consider the yubico-c-client port. It's defined in
MacPorts in
  security/yubico-c-client/Portfile
The patches to be applied to the source code of yubico-c-client are
under version control in
  security/yubico-c-client/files
which already gives us a history of the patches.

If I understand the tools you proposed correctly, they work on top of
the source code of yubico-c-client, which would be a clone of
  https://github.com/Yubico/yubico-c-client


Or am I misunderstanding things here?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: port:qt5 and (proposed) port:qt5-kde cohabitation

[Clemens Lang]

On Sun, Oct 23, 2016 at 08:00:25PM +0200, René J.V. Bertin wrote:
> I meant my local copy evidently which I'll most like keep in
> sources.conf and update from the new git working copy. If the svn
> history is migrated to github I'll be able to delete the .svn
> directory.

Sorry, I was assuming the things you sent to the list were relevant for
the subscribers. What you are doing locally on your machine is, of
course, your business.

> > The Referer header is used, see
> >   
> > https://www.edgewall.org/docs/tags-trac-1.0/epydoc/trac.web.auth-pysrc.html#LoginModule._redirect_back
> 
> I'm surely not the only one using privoxy who might want to use trac.

Possibly. In any case it's a problem in Trac that should be solved
together with Trac upstream.

> Do you know if we can add trac.macports.org to a whitelist for the
> corresponding filter rule, or some other host?

I do not know how privoxy works. If this uses some global filter rules,
please get in touch with the maintainers of these filter rules yourself.
Otherwise, adding an exception for trac.macports.org should be
sufficient.

> > What makes you think this is running on GitHub?
> 
> The Github login. Of course I noticed that the site name didn't
> change, but that doesn't mean it isn't hosted on a github server
> nowadays. 

GitHub allows 3rd party applications to use their login service, which
is what we are doing. See https://developer.github.com/v3/oauth/. Our
servers do not serve the GitHub login page, and we never see your GitHub
password.

In a nutshell, we redirect a user to a special URL at GitHub and pass an
application ID and a nonce. That user logs in to GitHub and GitHub sends
them back to our webserver with a HMAC using a previously shared key
matching the application ID so we know the login was successful. It's a
little more complicated than that, but the OAuth standard should answer
all your remaining questions.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: port:qt5 and (proposed) port:qt5-kde cohabitation

[Clemens Lang]

Hi,

On Sun, Oct 23, 2016 at 06:01:13PM +0200, René J.V. Bertin wrote:
> Thanks for the archive links. A quick scan suggests that my "shadow"
> tree I referred to will be a new working copy of the git repo, from
> where I'll filter things into the current svn tree.

There will be no svn tree in a week. We'll keep the old repository
around, but it will be read only.

> Will the migration conserve the full history?

Yes.

> Is there anything in browser settings that could influence this? Do
> you know if the feature depends on referral information, for instance.
> I'm running privoxy which filters that out, though that shouldn't
> happen on https if the info is encrypted too

The Referer header is used, see
  
https://www.edgewall.org/docs/tags-trac-1.0/epydoc/trac.web.auth-pysrc.html#LoginModule._redirect_back

> and also most other sites redirect me properly.

I cannot comment on what "most other sites" do.

> Will see, thanks - will depend on the language used there...

CSS for stylesheet changes.

> That wasn't what I had in mind. I think of this as a feature (like
> reply) checking whether the user is logged in, and serving a login
> dialog rather than an error if that is not the case.

That's not easily possible due to the limitations of the OAuth2
protocol, unless you want to run the complete authentication in an
iframe or something equally awkward.

> I presume you do get an error with the current implementation if you
> click reply after your session timed out?

Yes. A session lasts 30 days, though.

> Clearly neither did I. I did notice the resemblance to trac, but with
> it running on github I thought you adapted something provided there to
> look and work as closely as possibly to Trac.

What makes you think this is running on GitHub?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: port:qt5 and (proposed) port:qt5-kde cohabitation

[Clemens Lang]

Hi,

On Sun, Oct 23, 2016 at 05:01:08PM +0200, René J.V. Bertin wrote:
> Apparently not, were they sent on this ML? If so I must have applied
> too strict filters (basically "don't send list messages and don't
> filter copies sent to my own address" ...)

https://lists.macosforge.org/pipermail/macports-dev/2016-August/033405.html
https://lists.macosforge.org/pipermail/macports-dev/2016-October/034232.html

> > Re-login after the migration couldn't be avoided, since our
> 
> It's not the re-login that annoys me, it's the fact that each time I
> have to log in I have to take additional actions to go back to where I
> wanted to do something that required the login. And the login link is
> tiny, too ^^

Can't reproduce. If I open a ticket and click the login button it takes
a second or two until all the redirection associated with OAuth has
happened and drops my right back where I was.

The size of the login link is the same size as all other links in the
top bar. If you think those should be larger, feel free to suggest a
patch to https://github.com/macports/trac.macports.org/.


> Ideally, forums and ticket trackers provide a login-on-demand feature
> that is triggered when required as part of an operation that requires
> authentication.

We're limited by Trac's options here. While we can modify the
authentication scheme with plugins, making every page that requires
authentication a login page is somewhat more complicated. Additionally,
a lot of pages exist both without and with authentication with different
results, so this wouldn't apply.


> I don't know how much control you have over the software that replaced
> trac (it's something I've never seen before on github) but if you do
> this might be something to keep in mind for a future revision.

The software that replaced Trac is Trac. I'm not sure what you are
talking about here.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: port:qt5 and (proposed) port:qt5-kde cohabitation

[Clemens Lang]

On Sun, Oct 23, 2016 at 11:25:52AM +0200, René J.V. Bertin wrote:
> SVN is being retired completely?

Yes, SVN is being retired completely. Have you not read the announcement
emails?

> I noticed it for Trac (which isn't a complete improvement *). I'm not
> aware that git allows pulling only changes to a single subdirectory
> like svn does (svn up in a port dir. to update only that port, for
> instance). Does it?

No it doesn't. It's something you'll have to live with.

> *) open a ticket via an email alert, notice you're not logged in
> (anymore), click on login, use the back button to go back to the
> ticket page, *reload*, and only then can you reply.

Re-login after the migration couldn't be avoided, since our
authentication scheme changed. We've increased cookie lifetime already,
though, so you shouln't have to click the log-in button too often.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hi,

On Sat, Oct 22, 2016 at 02:31:32PM +0200, Marko Käning wrote:
> When reading the WorkingWithGit wiki page [1] I saw how port
> contributors can post their suggestions to MacPorts via "Pull
> Requests”, yet it does not get clear from that text how those PRs will
> then actually be included into the official repo…

Developers will merge them, either using the command line client, or the
GitHub UI. We haven't decided and documented which merge method to use,
although I'd prefer the rebase.


> Just this moment I got invited to MacPorts' "Developer team" on GitHub
> and saw that I now have "pull access” to the MacPorts ports
> repository, which is probably enabling me in the future to push
> changes into it?
> 
>   But the term “pull access” confuses me here!
> 
> I mean, I can pull in changes into my clone or fork from the MacPorts
> repo any time…

Pull access means that we haven't given the team write access to the
repository yet, so at the moment being a member of the GitHub org only
gives you added privileges in Trac. This will change as soon as we're
done with the conversion next weekend.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Trac & Subversion available again

[Clemens Lang]

Hello MacPorts users and developers,

On Fri, Oct 21, 2016 at 08:12:59PM +0200, Clemens Lang wrote:
> Due to the current SVN and Trac downtime, we are also discussing to
> make the move of Trac sooner if that helps us restore service earlier.
> We will keep you informed on this.

Due to the current unplanned downtime, we brought the migration of our
Trac installation forward and finished it today. Pending DNS
propagation, you should be able to reach Trac at trac.macports.org
again. Note that you will need a GitHub account to log into Trac now.

Please make sure that you have added the email address you used in Trac
to your GitHub account. A cronjob will automatically associate your
tickets and other contributions with the GitHub login a few minutes
after your login.


The Subversion server svn.macports.org has also been moved and
committing works again. Please commit your pending work before October
29th.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Moving to GitHub: Status Update, Action Required

[Clemens Lang]

Hello MacPorts users and developers,

MacPorts will be moving to GitHub soon. We're sending this email to
inform you about changes in how you access the MacPorts repositories and
bug tracker. Additionally, this email contains information on planned
downtimes. 

Action Required: GitHub Accounts

Our new Trac installation will use GitHub for login. If you do not have
a GitHub account yet, please create one now at
 https://github.com/join

To help us match your previous contributions and Trac tickets to your
GitHub account, please go to
 https://github.com/settings/emails
and ensure that you have added and verified all email addresses you have
used for MacPorts Trac. There is no need for any of these addresses to
be the primary one, or even public. They just need to be listed and
verified.

If you are a MacPorts committer, please make sure that you have added
your @macports.org address.

Action Required by MacPorts Developers: Joining the GitHub Organization
===
If you are a MacPorts developer and have commit access, please send the
following mail to macports-in...@lists.macports.org:
 Subject: Please invite me to MacPorts on GitHub
 Content:
  Handle: 
We will send an invite to join the MacPorts organization on GitHub to
your MacPorts email address. Follow the steps in this invitation email
to get commit access to the MacPorts Git repositories and privileges in
Trac.

Migration Timeline
==
The switch to Git will happen on the weekend of October 29th/30th. We
will disable committing to the Subversion repository, run a last
incremental export to Git and push the changes to GitHub. If you have
commit access, please do not commit to the repositories at GitHub until
a mail to the list indicates the conversion is done. Please read through
 https://trac.macports.org/wiki/WorkingWithGit
which contains a number of guidelines for working with the MacPorts Git
repositories.

We will also place our old Trac instance in read-only mode and move the
tickets to our new instance. Note that the new Trac instance uses GitHub
for login. We recommend that you have added and verified all email
addresses that you used for filing tickets in Trac to your GitHub
account before you log in to the new Trac. For MacPorts developers, this
includes your MacPorts email address. This will allow us to
automatically transfer your tickets to your new user account.

Due to the current SVN and Trac downtime, we are also discussing to make
the move of Trac sooner if that helps us restore service earlier. We
will keep you informed on this.


On behalf of the MacPorts migration team,
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: 2nd MacPorts Meeting & Online Meeting

[Clemens Lang]

Hi Marko,

On Mon, Oct 17, 2016 at 12:38:20AM +0200, Marko Käning wrote:
> now that you say it… Weren’t you aiming at the Wiesn earlier this
> year? ;-)

Well, as you noticed, that didn't happen ;)


> I noticed that there is eventually an official MacPorts ports git repo
> [1] online since a few days now, which Ryan seems to be administering…
> Cools stuff! It looks like MacPorts will soon work through git. I
> think I was the first who forked this. ;)

There will be an official announcement email with more instructions in a
couple of days, too. We're still hammering out the details of the
process and the draft.

For now, note that the repositories are subject to complete history
rewrite and force push on all branches at any time, because we're fixing
the last few flaws in the conversion scripts, so your fork might soon
been completely out of touch with the main repo.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: 2nd MacPorts Meeting & Online Meeting

[Clemens Lang]

Hi,

On Mon, Oct 10, 2016 at 07:53:13AM +0200, Mojca Miklavec wrote:
> Me and Aljaž would be willing to host the MacPorts meeting in 2017
> again.

Thanks, and sorry for not putting a plan together to do it in Munich in
the fall. Seems I need to improve my time management :)


> In addition to the face-to-face meeting, I would like to propose
> trying out some online meeting some time in mid-November (or any other
> time, according to your wishes).

A good idea, we should try that.


> It would be nice to prepare proposals / agenda for discussion in
> advance though. I would limit the meeting to 2 hours max and repeat it
> if necessary.

I think this might also be a good time to give a quick status update on
where we are with the conversion to Git and the move to GitHub, and
potentially answer questions that any of the attendees might have.

There's quite a bit of work going on at the moment "behind the scenes"
that does not make it to the mailing lists, and I have a feeling it
seems to the list like there is no progress, which is not the case.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: advanced bash-ing for long compiles

[Clemens Lang]

Hi,

- On 15 Oct, 2016, at 01:05, Fred Wright f...@fwright.net wrote:

> By default, this isn't possible via the GUI, due to the macports
> directory's being hidden.

Use cmd+shift+g in the file chooser.


> One can also accomplish this via:
> 
>   sudo touch /opt/local/var/macports/build/.metadata_never_index
> 
> Though I don't know if this works across all OS versions.

Do you have documentation on this? The only documentation I could find
back then suggested that Spotlight will ignore hidden directories, so
that's why we made it hidden.


> Hiding the macports directory also gets in the way of attaching build logs
> to tickets, since the browser's dialog box can't navigate to them.

See above, you can use cmd+shift+g.

> I've unhid this directory in the past for that reason, but every so often
> something "fixes" it.  If preventing indexing is the sole reason for
> hiding the directory, wouldn't using ".metadata_never_index" be better?

MacPorts re-hides the directory. Indexing is the only reason to hide it.
If you have a link to documentation that shows that .metadata_never_index
is supported, I'll happily switch to that.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: MP Cert Revoked?

[Clemens Lang]

Hi,

- On 13 Oct, 2016, at 17:00, Michael Dickens michae...@macports.org wrote:

> Your connection is not private
> Attackers might be trying to steal your information from
> trac.macports.org (for example, passwords, messages, or credit cards).
> NET::ERR_CERT_REVOKED

See https://twitter.com/globalsign/status/786505261842247680
See also http://apple.stackexchange.com/a/257082

The problem is with the intermediate, not with our certificate, it seems.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Documentation overhaul

[Clemens Lang]

Hi,

On Mon, Oct 10, 2016 at 09:44:17PM +0200, Rainer Müller wrote:
> Now here is Marcel offering his help with the documentation. I do not
> want to turn down his offer or stop him from contributing in any way.
> 
> [...]
>
> If Marcel has time right now, I think he should just start with the
> documentation.

I fully agree. Marcel, since I've touched most of our documentation at
some point (even though only for minor changes), feel free to reach out
to me if you have any questions. I likely won't have a lot of time to
contribute text on my own, but I can certainly point you to existing
documentation or clarify behavior that isn't documented yet.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: reasons for `port provides` not working?

[Clemens Lang]

Hi,

- On 10 Oct, 2016, at 16:49, René J.V. Bertin rjvber...@gmail.com wrote:

> Either way, this shouldn't have any incidence on file-to-port lookups.
> Either:
> - /opt/local/foo/bar is registered as such and a look-up of that path succeeds
> - /opt/local/foo/bar is actually /what/ever/foo/bar, registered as such, and
> thus a look-up of the resolved path succeeds.

It's debatable whether it shouldn't. It does.


> Good thing I didn't ask you specifically then...

You're asking the list, and I'm sure others may have the same impression.


> In fact, in absence of proof that my use of a symlinked $prefix explains
> everything I won't assume that that is the culprit.

Well, let me lawyer you with your own mail from 2014 then:
  https://lists.macosforge.org/pipermail/macports-users/2014-July/035965.html
Your attitude of "unless you can proof it's me, it's you" isn't helpful. I've
had enough of it.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: reasons for `port provides` not working?

[Clemens Lang]

Hi,

- On 10 Oct, 2016, at 15:11, René J.V. Bertin rjvber...@gmail.com wrote:

> I've been bitten once too often by an oversight that I c/should have seen if
> `port provides` worked. It's not been working for me for a long time, and
> mostly I don't really miss it, but now I'd like to figure out if there's a way
> to get this feature online again.
> 
> Without reinstalling from scratch, of course.
> 
> What could this be related to?

IIRC the problem was that your installation mangled paths somehow, either by
making /opt/local a symlink or some other uncommon configuration.

> FWIW, I do get a proper error when a port tries to install a file already
> installed by another file, so a priori the information is all there.

You'll forgive me when I just tell you that problems caused by unsupported
modifications made by you locally are not supported by MacPorts. I won't spend
any time diagnosing this unless you can show that it's broken in a fresh
install.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Issues with oudated ports / GitHub

[Clemens Lang]

Hi,

On Fri, Oct 07, 2016 at 09:13:11AM -0500, Ryan Schmidt wrote:
> Regarding updating pandoc, see https://trac.macports.org/ticket/48324
> Regarding updating ghc, see https://trac.macports.org/ticket/48899
> Regarding old llvm on Sierra, see https://trac.macports.org/ticket/52424

As a follow-up on the whole GHC vs. LLVM (and thus Pandoc) topic: In the
past, we packaged a subset of GHC and Haskell ports that did not match
the versions in haskell-platform. Feedback back then was that we should
stay with the haskell platform, so we did.

For this reason, I cannot simply update GHC without updating
haskell-platform (unless we want to re-visit this decision). I have the
update of GHC and the platform ports to version 8.0.1 [1] done, expect
for stack [2]. Haskell Platform without Stack seems less useful, since
Stack is supposed to become a de-facto dev environment management tool,
if I understand things correctly. Unfortunately, stack has a gazillion
dependencies we haven't packaged yet, at which point it makes little
sense for me to attempt to write all those Portfiles by hand. So either:
 - a group of people splits the work
 - generation of haskell portfiles needs to be automated
I'd welcome help with both. Let me know if you know a Haskell or two and
want to help out with the latter.

[1] https://www.haskell.org/platform/contents.html
[2] http://hackage.haskell.org/package/stack

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: CMake issue: binary (needed during build) links againts /opt/local/lib/foo.dylib

[Clemens Lang]

On Sat, Oct 08, 2016 at 04:10:34PM +0200, Mojca Miklavec wrote:
> >> The command "otool -L cfg" reports
> >>@rpath/libMiKTeX209-core.1.dylib
> >
> > That looks correct, but really depends on what "cfg"'s rpath is.
> 
> "cfg" is a binary. I'm not sure whether it gets installed, but it
> does, then the path is definitely wrong.

The settings I gave you tell CMake to re-link the binary without an
rpath on 'make install', so if you ran this in the build tree (which I
assumed because you mentioned destroot was broken) it was correct.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: CMake issue: binary (needed during build) links againts /opt/local/lib/foo.dylib

[Clemens Lang]

Hi Mojca,

- On 4 Oct, 2016, at 00:51, Mojca Miklavec mo...@macports.org wrote:

>-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=FALSE

I think this flag only affects installation (i.e. what happens on
make install), so you may want to check its effect once you have the
destroot phase working.


> The command "otool -L cfg" reports
>@rpath/libMiKTeX209-core.1.dylib

That looks correct, but really depends on what "cfg"'s rpath is.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Working with Git

[Clemens Lang]

Hi Ryan,

- On 6 Oct, 2016, at 04:53, Ryan Schmidt ryandes...@macports.org wrote:

> With Subversion, the user would submit a patch in a Trac ticket. To test it, I
> would download the patch and apply it to my local Subversion working copy. If 
> I
> like it, I commit it. If I don't like it, I give feedback to the user in the
> ticket, or I edit the Portfile further and commit it, then tell the user in 
> the
> ticket what changes I made.
> 
> How will this work on GitHub?

You add the pull request as a remote, fetch the changes, check them out in a
branch and test locally. If you like them, merge/cherry-pick/rebase them into 
the
master and push. This automatically closes the pull request.

Alternatively, you can start amending the changes locally by adding new commits 
on
top, or editing existing commits. If you like, you can push those back to the 
pull
request and submit them using GitHub's interface, but there's really not that 
much
difference between committing them directly and clicking the merge submit in the
pull request UI.

If a user updates his pull request that works the same way as a normal update 
from
git would: You fetch the changes and update your local copy; see our existing 
docs
on working with git.


> Clemens, in your repository here, you committed something I had previously
> committed in a fork, but had not submitted a pull request for:
> 
> https://github.com/neverpanic/macports-svn2git-rules/commit/bfeed37956f72bf996865e414a375128186d1adf
> 
> The GitHub interface says "ryandesign committed with neverpanic". How did you
> cause that notation to appear, and is that something we should be using in our
> MacPorts git workflow?

I added your repository as remote and used git cherry-pick $your_commit_id. I 
then
edited your change using git commit --amend and git rebase --interactive and
pushed. The Author field of the commit was not modified, but the Committer field
was changed to me, which is why you see "ryandesign committed with neverpanic".

HTH,
-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: ${prefix} as a disk-image (was: #49559: nepomuk-core can't be installed on a case-sensitive system)

[Clemens Lang]

Hi,

On Sat, Oct 01, 2016 at 10:33:08PM +0200, René J.V. Bertin wrote:
> I coined this admittedly a bit wild idea the other day that all of
> ${prefix} could be hosted on a R/W dynamic disk image, aka a sparse
> image or sparse bundle.

Sparse Bundles have the huge downside that they don't shrink
automatically. Yes, there are methods to shrink them, but that's extra
work and extra code.

> That would make it possible to install the more "unixy" ports to a
> case-sensitive filesystem without need for guiding the user through
> the steps required to create a case-sensitive volume.

Most "unixy" ports install just fine on case-insensitive filesystems,
and of the few that don't, most can be fixed with little effort. This
whole idea is overkill.

Additionally, I'd leave such a fundamental decision to Apple. It seems
we'll end up with a case-sensitive default anyway in a couple of years.

> It would also provide an alternative to those of us who want to use
> the default /opt/local prefix but still install MacPorts on a
> different volume. Rather than replacing /opt/local with a symlink
> after the initial install, we'd simply give the desired location for
> the image, and keep /opt/local as its mount point.

People can already do that. People can also mount a different filesystem
at /opt/local. I just don't think that should be our default setup.

> I also like the idea of being able to take the whole MacPorts tree
> offline with a simple command. 

A pretty uncommon use-case.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: CMake issue: binary (needed during build) links againts /opt/local/lib/foo.dylib

[Clemens Lang]

Hi Mojca,

- On 30 Sep, 2016, at 03:30, Mojca Miklavec mo...@macports.org wrote:

> Does anyone with a deeper understanding of CMake build system have any
> idea what to fix?

CMake should get this right automatically. You've already been pointed to
 https://cmake.org/Wiki/CMake_RPATH_handling

Try changing
 - CMAKE_BUILD_WITH_INSTALL_RPATH (it should be false),
 - CMAKE_SKIP_BUILD_RPATH (should be false) and
 - CMAKE_INSTALL_RPATH_USE_LINK_PATH (try both?)


HTH,
-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [153120] trunk/dports/fuse/sshfs

[Clemens Lang]

Hi,

On Mon, Sep 26, 2016 at 11:23:17AM -0700, Dan Ports wrote:
> For me, the main benefit is that it's immediately obvious where the
> patch comes from, and, since it's github, how it relates to the
> release version. (In this case, it's a backported patch from an
> unreleased version.)

This is something we should track anyway, regardless of where a patch is
or whether it is copied. I've made it a habit to follow
  
http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations
in patchfiles for my port, i.e. include

  Upstream-Status: Backport [$commit-id]
or
  Upstream-Status: Submitted [$url-to-pull-request-or-mailing-list]
or
  Upstream-Status: Inappropriate [configuration]

I encourage others to do the same.

> Obviously, there are other ways to accomplish this, but this one seems
> as good as any...

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] WorkingWithGit modified

[Clemens Lang]

Hi,

On Fri, Sep 23, 2016 at 02:14:25PM -0500, Ryan Schmidt wrote:
> True but I'd like to default to assuming a user will make a valuable
> contribution and will want to make additional contributions in the
> future. And our track record of accepting patches in Trac in a timely
> fashion is not good. Hopefully it will improve once we go to pull
> requests instead of patches but I don't want our failure to timely
> accept a pull request to prevent a user from submitting a second
> unrelated pull request. 

That's a good point.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] WorkingWithGit modified

[Clemens Lang]

Hi,

On Fri, Sep 23, 2016 at 11:56:19AM -0700, Sterling Smith wrote:
> Being a co-maintainer of a software project hosted on GitHub, where
> the main developer does not use branches leads to inconsistencies in
> how his contributions are handled vs other developers.

What would those inconsistencies be?

What is the difference between the master branch in a fork and an
arbitrarily named branch based off master in a fork? Can't both be
rebased against upstream/master in exactly the same way?

> I highly recommend that Clemens move to putting logically separate
> changes in separate topical branches, and avoid developing on master,
> except very tiny changes, e.g. typos in docs.

My point was (as explained in the parallel mail) that I seldom have
multiple logically separate changes at the same time, and just make my
fork's master branch my topic branch for the duration of the pull
request.

> There should also be a decision on the recommended way to get updates
> from the latest master, whether that is by merging or rebasing.  I
> personally like rebasing, but there is a stigma associated with it.
> Note the possibility of a safer forced push after a rebase with
> --force-with-lease.  (I didn't look to see if there was already a
> recommendation.)

See https://trac.macports.org/wiki/WorkingWithGit#updating. The text
currently recommends a rebase.

What's the stigma? The need to force push to your fork?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] WorkingWithGit modified

[Clemens Lang]

Hi,

On Fri, Sep 23, 2016 at 01:38:43PM -0500, Ryan Schmidt wrote:
> > On Sep 23, 2016, at 13:33, Clemens Lang  wrote:
> > 
> > I didn't include this because it's not how I normally work. I
> > usually only create branches for larger changes. I wouldn't be
> > opposed to include it, but I'm probably not the best person to write
> > these docs.
> 
> If you commit directly to master of your fork, then submit a pull
> request, you can't do anything else on master until your pull request
> is accepted. If you make further changes on master they will be
> included in the pull request, which you wouldn't want if they are
> unrelated changes.

For most of the pull requests I've sent on GitHub so far I wanted a
specific feature or bugfix to be integrated and did not want to make
other changes, so the branch was just not necessary.

If you still want to make other changes, you can always work locally,
too. You can see your local repository as yet another branch.

> And once your pull request is merged, then what? GitHub will give you
> a nice "you can now delete your master branch because it's been
> merged" button...

I usually deleted the entire repository after the PR was merged.

Maybe this approach will change for MacPorts, but most of our users
probably don't update 12 different ports in 6 different PRs at the same
time.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] WorkingWithGit modified

[Clemens Lang]

On Fri, Sep 23, 2016 at 01:08:38PM -0500, Ryan Schmidt wrote:
> Something I did not see discussed on this page was that after you
> fork, and before you start making changes, you should create a branch
> specific to the work you are doing.

I didn't include this because it's not how I normally work. I usually
only create branches for larger changes. I wouldn't be opposed to
include it, but I'm probably not the best person to write these docs.

In general, I think our "how to pull request" docs aren't very fleshed
out yet; this is intentional, because we don't yet know how we will deal
with pull requests and whether we want to request a specific workflow.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Goodbye Mac OS Forge, hello GitHub

[Clemens Lang]

Hi Craig,

On Fri, Sep 16, 2016 at 11:01:25AM -0400, Craig Treleaven wrote:
> Where does the migration stand now?

Buildbots and mirror infrastructure is moved, the GitHub org is mostly
configured as far as I can see.

We have a new Trac instance with GitHub integration set up, but do not
have the ticket database to test whether we need to do any migration on
the data. Rainer and I are waiting for Ryan to give us a dump for
testing.

We have a copy of Guide & Website running, but we do not have automatic
updates for those yet. We want to use a buildbot job to do this so
failures are easier to debug and more visible. The buildslave that is
supposed to run this job isn't available yet.

We need a converted Git repository set up to automatically import new
SVN commits so that our infrastructure that relies on input from
repositories (e.g. the buildbots) can be switched to Git. I think the
conversion rules should be good to go, but IIRC Ryan wanted to do some
commit message rewriting for ticket references. I'm not sure what the
state on this is.

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

RFC: mp-buildbot failcache

[Clemens Lang]

'${depvariants}' failed, aborting." >&2
 echo "[FAIL]" >> "$log_status_dependencies"
 echo "Building '$port' ... [ERROR] (failed to install dependency 
'${depname}') maintainers: $(get-maintainers "$port" "${depname}")." >> 
"$log_subports_progress"
+
+# Update failcache
+# $depvariants isn't quoted on purpose
+# shellcheck disable=SC2086
+failcache_failure "$depname" $depvariants
 return 1
 else
 echo "[OK]" >> "$log_status_dependencies"
+# Remove failcache if it exists
+# $depvariants isn't quoted on purpose
+# shellcheck disable=SC2086
+failcache_success "$depname" $depvariants
 dependencies_counter=$((dependencies_counter + 1))
 fi
 done
Index: mpbb-install-port
=======
--- mpbb-install-port   (revision 152544)
+++ mpbb-install-port   (working copy)
@@ -55,10 +55,15 @@
 time_start=$(date +%s)
 # $option_prefix is set in mpbb
 # shellcheck disable=SC2154
-if ! "${option_prefix}/bin/port" -dk install "$port"; then
+if "${option_prefix}/bin/port" -dk install "$@"; then
+# Remove failcache if it exists
+failcache_success "$@"
+else
 echo "Build of '$port' failed."
 # log: summary for the portwatcher
 echo "Building '$port' ... [ERROR] maintainers: $(get-maintainers 
"$port")." >> "$log_subports_progress"
+# update failcache
+failcache_failure "$@"
 return 1
 fi
 time_stop=$(date +%s)
Index: tools/canonical_variants.tcl
===
--- tools/canonical_variants.tcl(nonexistent)
+++ tools/canonical_variants.tcl(working copy)
@@ -0,0 +1,97 @@
+#!/usr/bin/env port-tclsh
+# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; 
c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
+# Writes the canonical list of variants as it appears in binary archive names
+# to stdout.
+#
+# Copyright (c) 2016 The MacPorts Project.
+# Copyright (c) 2016 Clemens Lang 
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in
+#the documentation and/or other materials provided with the
+#distribution.
+# 3. Neither the name of the MacPorts project, nor the names of any 
contributors
+#may be used to endorse or promote products derived from this software
+#without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+# OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+# IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+package require macports
+
+if {[llength $::argv] == 0} {
+puts stderr "Usage: $argv0  \[(+|-)variant...\]"
+exit 1
+}
+
+# initialize macports
+if {[catch {mportinit "" "" ""} result]} {
+   ui_error "$errorInfo"
+   ui_error "Failed to initialize ports sytem: $result"
+   exit 1
+}
+
+# look up the path of the Portfile for the given port
+set portname [lindex $::argv 0]
+#try -pass_signal {...}
+try {
+set result [mportlookup $portname]
+if {[llength $result] < 2} {
+ui_error "No such port: $portname"
+exit 1
+}
+} catch {{*} eCode eMessage} {
+ui_error "mportlookup $portname failed: $eMessage"
+exit 1
+}
+
+# parse the given variants from the command line
+array set variants {}
+foreach item [lrange $::argv 1 end] {
+foreach {_ sign variant} [regexp -all -inline -- 
{([-+])([[:alpha:]_]+[\w\.]*)} $item] {
+set variants($variant) $sign
+}
+}
+
+# open the port to get its active variants
+array set portinfo [lindex $result 1]
+#try -pass_signal {...}
+try {
+set mport [mportopen $portinfo(porturl) [list subport $portname] [array 
get variants]]
+} catch {{*} eCode eMessage} {
+ui_error "mportopen ${portinfo(porturl)} failed: $eMessage"
+exit 1
+}
+
+array set info [mportinfo $mport]
+puts $info(canonical_active_variants)
+
+#try -pass_signal {...}
+try {
+mportclose $mport
+} catch {{*} eCode eMessage} {
+ui_warn "mportclose $portname failed: $eMessage"
+}
+
+# shut down MacPorts
+#try -pass_signal {...}
+try {
+mportshutdown
+} catch {{*} eCode eMessage} {
+ui_error "mportshutdown failed: $eMessage"
+exit 1
+}

Property changes on: tools/canonical_variants.tcl
___
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svn:keywords
## -0,0 +1 ##
+Id
\ No newline at end of property
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [152507] contrib/mp-buildbot/mpbb-cleanup

[Clemens Lang]

Hi,

On Mon, Sep 12, 2016 at 10:53:04AM +0200, Rainer Müller wrote:
> The 'return' just skips everything else in this step, so that would be
> covered.

Yes, sorry, I haven't had my coffee yet.

> Another solution would be to run selfupdate before the cleanup step to
> ensure we always cleanup with a working port command. Although such a
> situation should only happen when the slaveprefix was deleted
> manually.

I don't think that's necessary. If MacPorts isn't installed, cleanup
wouldn't do anything anyway.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [152507] contrib/mp-buildbot/mpbb-cleanup

[Clemens Lang]

Hi,

On Sun, Sep 11, 2016 at 11:47:58AM -0700, rai...@macports.org wrote:
> Revision: 152507
>   https://trac.macports.org/changeset/152507
> Author:   rai...@macports.org
> Date: 2016-09-11 11:47:58 -0700 (Sun, 11 Sep 2016)
> Log Message:
> ---
> buildbot: skip cleanup if port is not installed
> [...]
> Modified: contrib/mp-buildbot/mpbb-cleanup
> ===
> --- contrib/mp-buildbot/mpbb-cleanup  2016-09-11 18:41:15 UTC (rev 152506)
> +++ contrib/mp-buildbot/mpbb-cleanup  2016-09-11 18:47:58 UTC (rev 152507)
> @@ -10,6 +10,15 @@
>  }
>  
>  cleanup() {
> +# if this is the very first build, selfupdate did not install port yet
> +# $option_prefix is set by mpbb
> +# shellcheck disable=SC2154
> +if [ ! -e "${option_prefix}/bin/port" ]; then
> +echo "---> Skipping cleanup"
> +echo "port not installed at ${option_prefix}/bin/port"
> +return
> +fi
> +
>  echo "> Deactivating ports"
>  # $option_prefix is set by mpbb
>  # shellcheck disable=SC2154

If $prefix/bin/port does not exist, $prefix/bin/port-tclsh will not
exist either, and the invocation of "tools/uninstall-old-ports.tcl"
below will fail as well.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Fakeroot destrooting [Was: Re: lldb ...]

[Clemens Lang]

Hi,

On Sat, Sep 10, 2016 at 09:14:16AM -0700, Jeremy Huddleston Sequoia wrote:
> No, the DYLD_INSERT_LIBRARIES approach is the right one here.
> Interested users would need to disable SIP.

"Interested users" would be everybody who uses MacPorts. I'd vote
against telling all our users to disable SIP. It's a useful
security/safety feature and it even helps us because users can no longer
mess up their /usr/bin.

I don't see why the kernel, dyld, or whoever strips the flags could not
just behave like running a copy of the binary at hand when it sees a
DYLD variable, i.e. do the workaround we're doing manually at the
moment.


> It would be nice if a mechanism were in place to determine trust of
> certain libraries in DYLD_INSERT_LIBRARIES.

So you're suggesting DYLD_INSERT_LIBRARIES on SIP-protected binaries
should only work if the inserted library is signed? How would that
improve anything? Are you suggesting every open source project out there
that uses library preloading now pays for a certificate and regularly
builds and releases binaries for macOS? Frankly, I don't see that
happening.


> Please file radars and point me to them, so I can make sure they get
> routed to the right place (likely as dupes, but dupes are very useful
> "votes" for bugs).

Those tickets have been filed when SIP was introduced and
DYLD_INSERT_LIBRARIES stopped working. Evidently, it wasn't important
enough to get fixed, so you'll forgive me if I have better things to do
with my time.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Fakeroot destrooting [Was: Re: lldb ...]

[Clemens Lang]

Hi,

On Sat, Sep 10, 2016 at 03:45:25PM +0200, René J.V. Bertin wrote:
> But I see quite a few items on there that are unlikely to call other
> applications. If this is only about preserving DYLD_LIBRARY_PRELOAD,
> why do you need to treat utilities like cat or rm?

cat and rm deal with files, consequently trace mode needs its code to be
injected into them (as would fakeroot). Not copying them would mean
their unmodified copies run without the "sandbox" enabled, since
DYLD_INSERT_LIBRARIES wouldn't have any effect on them.

> And what happens when you (re)set one of the tainted env. variables in
> a shell or interpreter with the SIP bit set? Is it unset or filtered
> out when you call another executable, even if that exec doesn't have
> the SIP bit set?

It's kept, but how would you do that? You cannot inject code into that
executable, and a port's build system specifies what shell scripts (for
example) get run.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Fakeroot destrooting [Was: Re: lldb ...]

[Clemens Lang]

Hi,

On Sat, Sep 10, 2016 at 02:42:36PM +0200, René J.V. Bertin wrote:
> >Except that fakeroot uses library preloading, a technique that's more
> >or less dead on modern OS X thanks to Apple's changes related to SIP:
> >DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit
> >set.
> 
> Fakeroot uses library preloading on Linux, but that doesn't mean other
> solutions aren't possible. Evidently that would require an official
> Apple fakeroot implementation.

There are no other solutions that I'm aware of on OS X. We discussed
using the sandboxing mechanism, but it doesn't support our use case.
Short of writing a kernel module or writing your own loader, there are
no other options.

> > It can still be done with utter hacks (copying the binary into a
> > file without the SIP bit and executing it from there, which we do
> > for trace mode), 
> 
> Do you mean something along the lines of `cat /bin/sh > /tmp/sh`? If
> so, why not simply use port:bash for trace mode?

Doing this only for the Shell isn't enough. Next affected tool would be
make(1), then install(1), cp(1), sandbox-exec(1), gzip(1), etc. pp. Just
FYI, I'm including a list of the files that are currently affected on my
system by the last 200-or-so runs of MacPorts at the end of this mail.

> BTW, how do you get calls like system() to use something other than
> /bin/sh ?

Hook execve(2) and posix_spawn(2) to transparently run a different
binary. That requires parsing the shebang line, too.

> > but I have neither seen any other library preloading utility that
> > used to work on OS X implement these changes nor convinced any of
> > their developers to do so.
> 
> They simpy advise to disable SIP, then?

They simply stopped caring. For most of them, it has always been
additional effort to support macOS for little gain. It's no secret that
macOS isn't the platform for Unix devs that it used to be.



Here's the list:
.
├── System
│   └── Library
│   └── Frameworks
│   ├── Python.framework
│   │   └── Versions
│   │   ├── 2.6
│   │   │   └── Resources
│   │   │   └── Python.app
│   │   │   └── Contents
│   │   │   └── MacOS
│   │   │   └── Python
│   │   └── 2.7
│   │   └── Resources
│   │   └── Python.app
│   │   └── Contents
│   │   └── MacOS
│   │   └── Python
│   └── Ruby.framework
│   └── Versions
│   └── Current
│   └── usr
│   └── bin
│   └── ruby
├── bin
│   ├── bash
│   ├── cat
│   ├── chmod
│   ├── cp
│   ├── csh
│   ├── date
│   ├── dd
│   ├── echo
│   ├── ed
│   ├── expr
│   ├── hostname
│   ├── ksh
│   ├── launchctl
│   ├── ln
│   ├── ls
│   ├── mkdir
│   ├── mv
│   ├── ps
│   ├── pwd
│   ├── rm
│   ├── rmdir
│   ├── sh
│   ├── sleep
│   └── test
├── sbin
│   └── md5
└── usr
├── bin
│   ├── Rez
│   ├── SetFile
│   ├── ar
│   ├── arch
│   ├── awk
│   ├── basename
│   ├── bc
│   ├── bison
│   ├── bzip2
│   ├── c++
│   ├── c++filt
│   ├── cc
│   ├── clang
│   ├── clang++
│   ├── cmp
│   ├── codesign
│   ├── codesign_allocate
│   ├── comm
│   ├── cpio
│   ├── cpp
│   ├── ctags
│   ├── curl
│   ├── cut
│   ├── diff
│   ├── dirname
│   ├── dsymutil
│   ├── egrep
│   ├── env
│   ├── etags
│   ├── expand
│   ├── false
│   ├── fgrep
│   ├── file
│   ├── find
│   ├── flex
│   ├── g++
│   ├── gcc
│   ├── getconf
│   ├── git
│   ├── gm4
│   ├── gnumake
│   ├── gperf
│   ├── grep
│   ├── groff
│   ├── grotty
│   ├── gzip
│   ├── head
│   ├── hiutil
│   ├── hostinfo
│   ├── ibtool
│   ├── id
│   ├── indent
│   ├── install
│   ├── install_name_tool
│   ├── jar
│   ├── java
│   ├── javac
│   ├── javadoc
│   ├── join
│   ├── ld
│   ├── less
│   ├── libtool
│   ├── lipo
│   ├── llvm-gcc
│   ├── locale
│   ├── logname
│   ├── m4
│   ├── machine
│   ├── make
│   ├── makeinfo
│   ├── man
│   ├── mig
│   ├── mkfifo
│   ├── mktemp
│   ├── nm
│   ├── nmedit
│   ├── od
│   ├── otool
│   ├── patch
│   ├── perl
│   ├── perl5.18
│   ├── pr
│   ├── python
│   ├── python2.6
│   ├── python2.7
│   ├── ranlib
│   ├── readlink
│   ├── rpcgen
│   ├── rsync
│   ├── ruby
│   ├── sandbox-exec
│   ├── sed
│   ├── size
│   ├── sort
│   ├── split
│   ├── stat
│   ├── strings
│   ├── strip
│   ├── svn
│   ├── svnversion
│   ├── sw_vers
│   ├── tail
│   ├── tar
│   ├── tbl
│   ├── tclsh
 

Fakeroot destrooting [Was: Re: lldb ...]

[Clemens Lang]

Hi,

On Fri, Sep 09, 2016 at 01:59:50PM -0700, Jeremy Huddleston Sequoia wrote:
> > As an aside, I'd be in favour of setting up MacPorts such that
> > ${prefix} is owned by a ${macports_operator} who's got admin rights
> > (= myself) and reserve use of actual root privilege to those few
> > ports that require setting up SETUID/GETUID executables or that need
> > to create users or groups.
> 
> YES!  We should not be needing to do such things as root.  That is
> 100% true, and I am in full support of moving away from that and only
> using root for activate.  We should be able to use fakeroot
> (https://wiki.debian.org/FakeRoot) for destdir.

Except that fakeroot uses library preloading, a technique that's more or
less dead on modern OS X thanks to Apple's changes related to SIP:
DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit set.
Combine that with every binary in /usr/bin and /bin having the bit, and
you'll end up the variable being removed on the first invocation of a
shell (which is basically everywhere in the build systems of our ports).

It can still be done with utter hacks (copying the binary into a file
without the SIP bit and executing it from there, which we do for trace
mode), but I have neither seen any other library preloading utility that
used to work on OS X implement these changes nor convinced any of their
developers to do so.

Other approaches that would allow simulating permissions, such as Linux'
user namespaces don't exist on OS X at all. I think it's pretty obvious
that implementing new code that relies on library preloading is riding a
dead horse on macOS.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: how about a 'snowleopardfixes' library port?

[Clemens Lang]

Hi,

On Tue, Aug 23, 2016 at 10:00:05AM -0700, Ken Cunningham wrote:
> clang -dynamiclib -std=gnu99 strnlen.c getline.c -current_version 1.0 
> -compatibility_version 1.0 -o libsnowleopardfixes.a.dylib

You need -install_name ${prefix}/lib/libsnowleopardfixes.a.dylib here.

> configure:3663: ./conftest
> dyld: Library not loaded: libsnowleopardfixes.a.dylib
>   Referenced from: 
> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_sysutils_lnav/lnav/work/lnav-0.8.1/./conftest
>   Reason: image not found
> ./configure: line 3665: 50401 Trace/BPT trap  ./conftest$ac_cv_exeext
> configure:3667: $? = 133
> configure:3674: error: in 
> `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_sysutils_lnav/lnav/work/lnav-0.8.1':
> configure:3676: error: cannot run C++ compiled programs.
> If you meant to cross compile, use `--host'.

This check just tests whether you can run compiled programs. Because
your library does not have a correct install name it is not found by the
loader, which causes the test program to fail.

The configure script just happens to assume that you might be
cross-compiling if you cannot run the generated binaries.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] WorkingWithGit modified

[Clemens Lang]

Hi Marko,

On Sun, Aug 21, 2016 at 06:55:25AM +, MacPorts wrote:
> Page "WorkingWithGit" was changed by m...@macports.org
> Diff URL: 
> 
> Revision 20
> Comment: Add basic setup info based on KDE's configuration hints
> Changes:
> ---8<--8<--8<--8<--8<--8<--8<--8<
> Index: WorkingWithGit
> =
> --- WorkingWithGit (version: 19)
> +++ WorkingWithGit (version: 20)

Thanks for the section on setting up commiter name and email address, I
had forgot about this.

> +Additionally one should define a few (not necessarily global) presets for 
> working with your clone of the MacPorts repository:
> +{{{
> +$ git config --global push.default nothing
> +$ git config --global branch.autosetuprebase always
> +$ git config --global core.excludesfile ~/.gitignore_global
> +$ git config --global commit.template ~/.git-commit-template
> +}}}

I do not think we should recommend setting these variables for the
following reasons:

 - Users don't understand what they're doing here. I want our developers
   to know what and why things are happening with Git, not serve them a
   "here's how you ought to do it" recipe.
 - This is also the reason why the "Fetching the latest changes" section
   I wrote yesterday is so long: Developers should understand what
   happens when they run these commands, not just be told "run this". I
   think we should move branch.autosetuprebase to the end of the
   "Fetching the latest changes" section where we tell them that instead
   of the --rebase flag, they can also set this option. Also, setting
   this option globally is bad; other projects might not want this
   behavior.
 - push.default nothing may be a good idea, but I would only put it as a
   recommendation in a section on committing and pushing changes for
   people that want to make sure they're not accidentially pushing
   something they don't want to push.
 - I do not think we need .gitignore_global; We already converted our
   svn:excludes into .gitignores in the repositories and we've always
   left the configuration of svn clients to our developers. If they
   think this is necessary, they can figure it out on their own.
 - I also do not think we should recommend a git commit template for the
   same reason. Developers with Subversion have always been free to
   configure their own client and while a commit template is possible
   with Subversion, we did not recommend it there either.

What do you think?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [151267] contrib/mp-buildbot

[Clemens Lang]

Hi Mojca,

On Thu, Aug 11, 2016 at 01:03:06AM -0700, mo...@macports.org wrote:
> Revision: 151267
>   https://trac.macports.org/changeset/151267
> Author:   mo...@macports.org
> Date: 2016-08-11 01:03:06 -0700 (Thu, 11 Aug 2016)
> Log Message:
> ---
> mp-buildbot: add (not too helpful) progress for the port watcher
> 
> Modified Paths:
> --
> contrib/mp-buildbot/mpbb-install-dependencies
> contrib/mp-buildbot/mpbb-install-port
> contrib/mp-buildbot/mpbb-list-subports
> 
> Modified: contrib/mp-buildbot/mpbb-install-dependencies
> ===
> --- contrib/mp-buildbot/mpbb-install-dependencies 2016-08-11 06:56:46 UTC 
> (rev 151266)
> +++ contrib/mp-buildbot/mpbb-install-dependencies 2016-08-11 08:03:06 UTC 
> (rev 151267)
> [...]
>
> Modified: contrib/mp-buildbot/mpbb-install-port
> ===
> --- contrib/mp-buildbot/mpbb-install-port 2016-08-11 06:56:46 UTC (rev 
> 151266)
> +++ contrib/mp-buildbot/mpbb-install-port 2016-08-11 08:03:06 UTC (rev 
> 151267)
> [...]
>  
> Modified: contrib/mp-buildbot/mpbb-list-subports
> ===
> --- contrib/mp-buildbot/mpbb-list-subports2016-08-11 06:56:46 UTC (rev 
> 151266)
> +++ contrib/mp-buildbot/mpbb-list-subports2016-08-11 08:03:06 UTC (rev 
> 151267)

I don't think the changes in mpbb-list-subports are very helpful. The
new file is basically the same as the stdout of the task and does not
provide a lot of additional value, unless you want to somehow extend or
automatically parse it?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [151264] contrib/mp-buildbot/mpbb-install-port

[Clemens Lang]

Hi Mojca,

On Wed, Aug 10, 2016 at 11:48:51PM -0700, mo...@macports.org wrote:
> Revision: 151264
>   https://trac.macports.org/changeset/151264
> Author:   mo...@macports.org
> Date: 2016-08-10 23:48:51 -0700 (Wed, 10 Aug 2016)
> Log Message:
> ---
> mp-buildbot/install-port: add basic statistics and main.log (not sure if 
> useful, port -d is more verbose)
> 
> +# TODO: printing statistics (and installing the port + dependencies)
> +#   only makes sense when the port hasn't been installed previously
> +# log: statistics
> +echo "time:$(($time_stop - $time_start))s" >> "$log_port_stats"

Inside $(( )), dollar signs are not required for variables.
$((time_stop - time_start)) works just fine.

> +# log: main.log
> +local port_mainlog=$("${option_prefix}/bin/port" logfile 
> "${option_port}")
> +if [ -f $port_mainlog ]; then

$port_mainlog should be quoted here.


-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: How to discontinue ports completely (py26 deprecation) ...

[Clemens Lang]

On Wed, Aug 10, 2016 at 04:47:25PM -0500, Ryan Schmidt wrote:
> On Aug 10, 2016, at 13:18, Peter Danecek  wrote:
> > 
> > However, how would we procede in this case, when we have no explicit
> > replacement to indicate?
> 
> I believe the obsolete 1.0 portgroup can accommodate this: include the
> portgroup but don't set replaced_by. 
> 
> However, do first consider whether this needs to be done, or whether
> the port can instead be left as is. 

What's the benefit of replacing a port we might no longer want to keep
with an explicit error message on upgrade? Instead, we could just
outright delete the port, which will leave it installed on the systems
of those users that had it installed (i.e. not break their system), but
also no longer allow fresh installations of it.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: our buildslaves' Python and bash

[Clemens Lang]

Hi,

On Wed, Aug 10, 2016 at 04:52:02PM -0400, Lawrence Velázquez wrote:
> Are the buildslaves using the OS-provided bash (for mpbb) and
> MacPorts' python27 (for Buildbot), or what?

It's very likely the OS-provided bash, since mpbb's shebang says
/bin/bash. As for Python, I think there's a separate MacPorts
installation in a non-default path that installs buildbot from our
buildbot port, which should also come with MacPorts Python.

We could make mpbb use MacPorts bash, but I'm not sure we need any Bash
4 features at the moment.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

ports.tar update frequency on new mirror

[Clemens Lang]

Hi Ryan,

I'm wondering what update frequency I can expect for ports.tar on the
new main mirror.

I have a cronjob set up that regularly checks whether ports.tar has not
been updated in a while; it came in handy a couple of times because most
developers usually sync using SVN and would not notice the rsync mirror
becoming stale. For the last few days, it has been sending me a regular
stream of emails warning me about old tarballs, even though eventually a
new update was pushed.

The top spot is a mail from around noon today, where ports.tar had not
been updated for 12.5 hours. Is this normal? Should I change my warning
level?

- Forwarded message from Cron Daemon  -

Date: Wed, 10 Aug 2016 13:37:01 +0200
From: Cron Daemon 
To: dadad...@xoxoxo.neverpanic.de
Subject: Cron  nice -n20 ionice -c3 
/var/www/virtual/macports/home/portsynccheck.sh

rsync://rsync.macports.org/release/tarballs/ports.tar is outdated:
  last updated 12h 35m 17s ago
  at 2016-08-10 01:01:44.0 +0200

- End forwarded message -

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [151215] contrib/mp-buildbot

[Clemens Lang]

Hi Mojca,

a couple of review comments inline below.

On Wed, Aug 10, 2016 at 11:09:30AM -0700, mo...@macports.org wrote:
> Revision: 151215
>   https://trac.macports.org/changeset/151215
> Author:   mo...@macports.org
> Date: 2016-08-10 11:09:29 -0700 (Wed, 10 Aug 2016)
> Log Message:
> ---
> mp-buildbot: create a log file with progress of installed dependencies
> 
> +# prepare the log file
> +if ! [[ -d "${option_logdir}" ]] ; then
> +mkdir -p "${option_logdir}"
> +fi

The double brackets aren't necessary here. Additionally, testing and
then creating a directory has an implicit race condition (somebody else
could create it in between), so it's good that you use -p here, but then
you could just do mkdir -p directly and avoid the if completely.

> +log_status_dependencies=${option_logdir}/dependencies-progress.txt
> +# make ure to start with an empty file

Typo, should be make *s*ure.

> +echo -n "" > $log_status_dependencies

Instead of explicitly printing nothing into a file, you can just run
 > "$log_status_dependencies"
which does the same thing. Note that you should quote the redirect
target, just in case $option_logdir contains spaces. This applies to all
other uses of $log_status_dependencies as well.

> +echo "" >> $log_status_dependencies

echo "" is the same as just using echo without argument.

> +text="Installing dependency ($dependencies_counter of 
> $dependencies_count) '${depname}', variants: '${depvariants}'"
> +echo "> ${text}"
> +echo -n "${text}' ... " >> $log_status_dependencies

You have an additional closing single quote after ${text} here, which
leads to
 Installing dependency (41 of 205) 'xorg-xcb-proto', variants: '+python27'' ... 
 [OK]


Thanks for the work on this, though; it's much easier to read!

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Determine before installation whether a port can be installed

[Clemens Lang]

Hi,

On Thu, Jul 28, 2016 at 03:21:56AM -0500, Ryan Schmidt wrote:
> However, as I've said, there are other reasons why a port might not be
> able to install. Should we really modify MacPorts base and create new
> variables to accommodate each of those reasons? Is there a specific
> objection to what I consider the easiest and most flexible of the
> options I suggested: implement a new preflight phase, in which the
> port can run any logic to determine if the port can be installed?

I have a specific objection: If this is a phase and does not use
declarative syntax, you absolutely must open and run the code in the
Portfile to determine whether a port will build on your OS.

Opening a running Portfiles is (a) already where our dependency
management spends most time, (b) hard to optimize in the absence of a
interpreter-fork-like behavior in Tcl with our current approach, (c)
completely breaks the idea of using SAT-solving for dependency
resolution or any other modern method, because those methods depend on
having accurrate information about *all* available ports (which would
mean we'd have to run all ports).

So in short: The phase cannot be put in the port index, which is a bad
idea in my opinion. Declarative approaches like the platform statement
can.


On Thu, Jul 28, 2016 at 01:02:15PM +0200, Rainer Müller wrote:
> On 2016-07-28 09:06, Artur Szostak wrote:
> >   macosx >= 10.9
> >   macosx < 10.10
> 
> https://trac.macports.org/ticket/15712

I like this approach. Remember the brew uses a similar approach to
blacklisting, which is basically
  platforms >= :lion
(please excuse any butchering of Ruby syntax). This is very simple to
get right, and I'd prefer a similar approach.

Ryan: Which other cases can you think of? I think we can handle most of
them with a similar approach. Why not have a field that contains
identifiers which reference well-known tests for a feature? Here's an
idea:

 > constraints java-1.0

uses _resources/port1.0/constraints/java-1.0.tcl to determine whether
the constraint is fulfilled, but does this once for all ports, rather
than once per port. Similarly, we could have

 > constraints {platform-1.0 >= lion}

which could load _resources/port1.0/constraints/platform-1.0.tcl and run
a proc from there with the ">=" "lion" arguments (again, without running
the complete Portfile and loading all files in port1.0).

That would also allow you to implement manual download checks:

 > constraints {manual-download-1.0 "${targetfile}" "${downloadurl}"}

Whether that field is now called "platform" or "constraints" doesn't
really matter. We could make them aliases.

What do you think?
-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [150874] trunk/dports/lang/python32

[Clemens Lang]

FWIW, I agree with Lawrence here. I think we should improve our patch
naming, especially since it doesn't really matter *where* a patch comes
from, either. Why do we really need to know whether a patch-*.diff file
comes from MacPorts when compared to a different patch? We should be
judging patches by their content, not their origin.

What I've seen in other systems and would also recommend as a guideline
for patches in MacPorts:
 - Write a commit message into the patch file. Patch headers can include
   arbitrary text, so type a message explaining what the patch does and
   why it is needed. Add references to any upstream tickets (I've
   started doing this for my ports according to OpenEmbedded's
   Upstream-Status tag [1]).
 - Use the summary line of the commit message as patch name, like git
   format-patch does it.

I have in the past also ignored our patch naming guidelines when I
thought it made sense; for example, I generally backport patches from
upstream git repositories using .patch, because that's what
GitHub generates.

[1] 
http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations

-- 
Clemens


signature.asc
Description: PGP signature
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Reason why default rsync URL was changed to a tarball

[Clemens Lang]

Hi,

- On 19 Jul, 2016, at 17:09, Ryan Schmidt ryandes...@macports.org wrote:

> The commit message says "make sync and selfupdate via signed tarballs the
> default"
> 
> Was signing support the only reason for this change? On one of my systems with
> an evidently slow disk, "sudo port sync" takes several minutes to complete,
> spent mostly on decompressing the tarball, even, I think, if the tarball is
> unchanged, whereas using the old rsync URL, "sudo port sync" is very quick
> since it only has to write the changes to disk and not the entire ports
> collection each time.

I've noticed the same thing, but I don't see an alternative. We don't provide
a signature for the non-tarball variant and rsync isn't encrypted or otherwise
authenticated, so the non-tar version is significantly worse.

May there is a flag we could pass to tar to only extract new or changed files
that would speed up this process?

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [150004] trunk/dports/python/py-ipython/Portfile

[Clemens Lang]

Hi Sean,

- On 8 Jul, 2016, at 07:59,  s...@macports.org wrote:

> [150004] trunk/dports/python/py-ipython/Portfile
> Revision 150004
> Author s...@macports.org
> Date 2016-07-07 22:59:39 -0700 (Thu, 07 Jul 2016)
> Log Message py-ipython: update to 5.0.0

I'm seeing a checksum failure for this change when fetched from 
https://github.com/ipython/ipython/tarball/5.0.0:

--->  Checksumming ipython-5.0.0.tar.gz
Error: Checksum (rmd160) mismatch for ipython-5.0.0.tar.gz
Portfile checksum: ipython-5.0.0.tar.gz rmd160 
9160a6f37e8364d0dce6fc7a41ed4e1c1b81dbb3
Distfile checksum: ipython-5.0.0.tar.gz rmd160 
2bd45ecded8dd0425db4604675f8ef709ad7189a
Error: Checksum (sha256) mismatch for ipython-5.0.0.tar.gz
Portfile checksum: ipython-5.0.0.tar.gz sha256 
7ec0737169c74056c7fc8298246db5478a2d6c90cfd19c3253222112357545df
Distfile checksum: ipython-5.0.0.tar.gz sha256 
7e0023896ceca69d3150cef76276b1c0e825e84eee287c7d0717703ceb3c8818
The correct checksum line may be:
checksums   rmd160  2bd45ecded8dd0425db4604675f8ef709ad7189a \
sha256  
7e0023896ceca69d3150cef76276b1c0e825e84eee287c7d0717703ceb3c8818
Error: Failed to checksum py27-ipython: Unable to verify file checksums

-- 
Clemens Lang
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Debian/Ubuntu-style -dev ports

[Clemens Lang]

Hi,

On Thu, May 19, 2016 at 12:45:48AM +0200, René J.V. Bertin wrote:
> On Wednesday May 18 2016 17:50:00 Brandon Allbery wrote:
> 
> > The problem with this is that if a MacPorts user can't get something from a
> > prebuilt package, you have to force both the "user" and "developer" ports
> > to be installed because any dependents will necessarily be doing linking.
> 
> Yes, but is that really a problem? The overhead should be minimal
> since the -dev port only contains symlinks, and if there's a way to
> record the fact when a port has a -dev port a depends_build
> port:foo-dev  could be implied (added automatically) by a simple
> depends_lib port:foo statement.

This is really something that should either be done completely and
correctly, or not. In theory, it's actually not all that hard. You only
need the concept of one Portfile generating multiple different packages,
allow separate installation of those packages, and introduce a phase
that splits the files installed by a port into those packages (much of
which can be automated using a set of rules, as you mention).

Yes, it's something we could do for library symlinks, but then you'd
leave the pkg-config files around that reference libraries that don't
exist where build systems would expect them. Same for CMake modules,
header files and a bunch of other stuff.

tl;dr: It can be done, but it should be done right.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: file/dir name case (in)sensitivity

[Clemens Lang]

Hi,

On Fri, May 06, 2016 at 12:51:57PM +0200, René J.V. Bertin wrote:
> Would it be worthwhile to figure out and put up an advanced topic
> describing how to set up MacPorts to use a case-sensitive ${prefix}
> while still being able to benefit from binary packages?

Our buildbots use case-sensitive filesystems, so you should be able to
use binary packages for both case-insensitive machines (the
configuration most people are probably using) and case-sensitive
environments (because that already works fine on the buildbots).

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: [MacPorts] MacPortsDevelopers modified

[Clemens Lang]

Hi Marius,

On Wed, Apr 27, 2016 at 12:39:18AM +, MacPorts wrote:
> Page "MacPortsDevelopers" was changed by m...@macports.org
> Diff URL: 
> 
> Revision 293
> Changes:
> ---8<--8<--8<--8<--8<--8<--8<--8<
> Index: MacPortsDevelopers
> =
> --- MacPortsDevelopers (version: 292)
> +++ MacPortsDevelopers (version: 293)
> @@ -115,6 +115,7 @@
> +||[wiki:mps] || Marius Schamschula || mschamschula ||
> ---8<--8<--8<--8<--8<--8<--8<--8<

Welcome aboard! Nice to have you among us.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Getting new port comitted: asciidoctor

[Clemens Lang]

Hello Guido,

On Tue, Apr 19, 2016 at 08:49:28AM +0200, Guido Kollerie wrote:
> Can the new port `asciidoctor` be committed by someone? Suggested
> changes were made. All that remains is for someone to add it to the
> ports collection.
> 
> https://trac.macports.org/ticket/51087

Committed in https://trac.macports.org/changeset/147968. Thank you for
your port.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Errors should probably come after notes?

[Clemens Lang]

Hi,

On Fri, Apr 08, 2016 at 09:29:35PM +0200, Mojca Miklavec wrote:
> What I don't like so much is that the notes come *after* the error
> report. I ended up with about 10 notes following the error, so I could
> easily have missed the fact that there was an error at all (I had to
> scroll a few screens back to actually see it).

That's a tough question -- you wouldn't want users to miss the notes,
either.

> Shouldn't the order be reversed? (I'm not sure how difficult it is to
> change it though.)

Implementation is in port/port.tcl. The call that prints the notes is in
process_cmd, line 4685: portclient::notifications::display.
Unfortunately that part that prints the error message is in $action_proc
called a few lines before, so it's not trivial to switch.


HTH,
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

Hi,

On Wed, Apr 06, 2016 at 03:08:09PM +0200, René J. V. Bertin wrote:
> That doesn't teach me much new
> %> bunzip2 < /opt/local/var/macports/registry/registry-corrupt.db.bz2 > kk.db
> %> /opt/local/bin/sqlite3 kk.db
> %> .load MacPorts-svn/src/cregistry/macports.sqlext
> %> pragma integrity_check;
> ok

So your registry wasn't broken, apparently?

> I don't know if it helps, but here are 2 backtraces I got after
> sending a SIGKILL (not SIGQUIT as the log pretends). They're pasted in
> chronological order, but they were preceded by an unknown number of
> other attempts to "get back in", for which I didn't get a crash
> report.

I think I have previously had situations where the database seemed to be
rather slow and re-creating fixed it. I did not investigate further,
though, but I suspect you may have hit a similar case.

> I wonder if the "DeleteColumnNames" function above doesn't mean that
> somehow all invalid data has been stripped from the "corrupt"
> database, which would make it valid but probably useless.

I doubt that. SQLite would rather throw an error than loose data.
Loosing data is just about the worst thing a database can do.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PortGroup directory hierarchy/priority

[Clemens Lang]

On Wed, Apr 06, 2016 at 10:48:00AM +0200, René J. V. Bertin wrote:
> > Builds can be made bit-by-bit reproducible even when building from
> > source. It's not even particularly difficult for a lot of ports, as
> > long as the toolchain stays the same, timestamp issues are handled
> > and the environment the software builds in is under close control
> > (e.g. a chroot).
> 
> The problem is that you don't have that close control over a user's
> machine, if not only because MacPorts "base" is fully open source.

We don't need to have control over each minor detail, the control we
currently have is more than enough to get builds reproducible (even
though we may need one hack or the other, like DYLD_INSERT_LIBRARIES).

> > No. Modifications of PortGroups do not trigger re-indexing unless
> > you also touch the Portfiles that use it.
> 
> That reminds me of a discussion a while back on how Portfile changes
> are detected (and where the files are expected). Would it be very
> costly to replace the ascii portindex files with sqlite3 databases
> that store more information, including references to included
> PortGroups?

No. Some problems are still to be solved, like how to sync a
sqlite3-based pre-generated portindex from a server. I had even started
an sqlite3-based portindex a while back, but the project died. It's
possible, but needs manpower and refactoring.

> I mean runtime cost, implementation of something like that seems like
> a nice summer project for someone.

Unfortunately due to the number of places where the portindex is used
(without abstractions) across MacPorts base, I fear this may be a bigger
project than a summer.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PortGroup directory hierarchy/priority

[Clemens Lang]

Hi,

On Mon, Apr 04, 2016 at 08:17:55PM +0200, René J.V. Bertin wrote:
> > > There's only one way to enforce the reproducible build principle
> > > all the way, and that's to remove the whole possibility of
> > > building from source so that users can only install official,
> > > binary packages.

> > No, that's (a) not true and (b) actually prevents one of the goals

> I can see (b) but not (a)?

Builds can be made bit-by-bit reproducible even when building from
source. It's not even particularly difficult for a lot of ports, as long
as the toolchain stays the same, timestamp issues are handled and the
environment the software builds in is under close control (e.g. a
chroot).

> > Sure, let's assume you change the github 1.0 PortGroup, because you
> > need an additional feature… that could change how other ports fetch,
> > and thus the source that is being built be other ports.
> 
> […]
> However, my argument still stands that the developer of that port and
> PortGroup would be subject to the same thing. There's indeed a trust
> issue there, but one that goes both ways. 

Yes, developers of separate port trees implicitly trust the main port
tree. I don't think that's the same situation, though, because the same
developers also wrote MacPorts base, which they also implicitly trust.

> > The whole installation idea does not work with the current codebase.
> > If a PortGroup changes, the information in the PortIndex needs to be
> > updated, but the PortIndex uses the modification date of Portfiles
> > to determine whether the information is current.
> 
> So no PortGroup is taken into account?

No. Modifications of PortGroups do not trigger re-indexing unless you
also touch the Portfiles that use it.

> > You can not change how a port behaves without touching the Portfile.
> > See the above rationale on the PortIndex.
> 
> And that doesn't apply to copying PortGroups manually into _resources?

That also applies to copying PortGroups manually into _resources.

> > I think most of this discussion could be solved by merging your
> > changes into the main port tree, couldn't it? I think that's what we
> > should be
> 
> Yes, that's true. There is testing underway to that end, but only 2 or
> 3 port developers have stepped up to help here, and there's barely any
> activity on the corresponding trac tickets.

Yes, manpower is one of our general problems. We should try to improve
our buildbot setup to provide more automation (e.g. Continous
Integration) -- that would at least simplify test coverage. As you may
have seen, we have actually started looking into this at MacPorts
Meeting 2016 in Slovenia last month.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

Hi,

On Mon, Apr 04, 2016 at 07:04:46PM +0200, René J.V. Bertin wrote:
> How would one check that (without putting my whole install at risk)?

http://serverfault.com/questions/8048/how-can-i-verify-that-a-sqlite-db3-file-is-valid-consistent

See also
https://trac.macports.org/browser/trunk/base/src/cregistry/README.sqlext

HTH,
-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

On Mon, Apr 04, 2016 at 06:42:24PM +0200, René J. V. Bertin wrote:
> Thanks for the suggestion. Curiously that went just fine, but it also
> seems to contradict the previous claim that any interaction with an
> sqlite3 db modifies it.

No, any writing interaction modifies it (that should be obvious).
Reading does not change the database (unless you lock it exclusively
while reading). My point with the modifications was that watching for
the file to trigger a copy would potentially trigger a copy on every
fsync(), which might be a lot.

However, now I'm interested why the backup worked flawlessly, but
MacPorts failed to open the registry. Maybe there was something wrong
with one of the indexes in the database (which wouldn't be used when
doing a full dump or backup, but are being used by MacPorts in standard
usage).

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PortGroup directory hierarchy/priority

[Clemens Lang]

Hi,

On Mon, Apr 04, 2016 at 10:42:16AM +0200, René J.V. Bertin wrote:
> Typically paths defined through variables indeed, or configure
> settings,

Paths and flags should be part of configuration files installed by a
port, e.g. CMake config files or pkg-config files.

> plus the occasional pre- or post- block.

I agree that this is a valid use case.

> >That would potentially cause ports to build different depending on
> >which packages you have installed in your environment, which is not
> >reproducible.
> 
> Yes, but that is already the case when a user replaces a mainstream
> port with one from an alternative source.

Yes, but then the transitive dependency tree of the changed port was
also changed. A overwritten PortGroup could change the behavior of ports
that do not have the provider of the PortGroup in their dependencies.

> There's only one way to enforce the reproducible build principle all
> the way, and that's to remove the whole possibility of building from
> source so that users can only install official, binary packages.

No, that's (a) not true and (b) actually prevents one of the goals of
reproducibility, namely that the average user can verify that the
binaries served by us are exactly what they should be and we haven't
backdoored them.

> > The behavior of a port might even be affected by ports that are not
> > in its transitive dependency tree. 
> 
> Can you think of an example other than the one below?

Sure, let's assume you change the github 1.0 PortGroup, because you need
an additional feature… that could change how other ports fetch, and thus
the source that is being built be other ports.

> > The more I think about this, the more I get the impression a
> > PortGroup should only be affecting its port tree, and nothing else.
> 
> To draw an analogy, you prefer a Prohibition-like approach to a more
> permissive one that provides the tools to achieve do certain things in
> a controlled way?

I prefer an approach that minimizes the probability to break things.
Yes, this limits the developers of Portfiles somewhat, but it doesn't
make things impossible -- the PortGroup and affected ports can always be
copied.


On Mon, Apr 04, 2016 at 05:13:25PM +0200, René J.V. Bertin wrote:
> On Monday April 04 2016 16:22:27 Rainer Müller wrote:
> 
> >A user already installed port A with the official port group foo-1.0.
> >Then they add a new source, overriding the port group foo-1.0, which
> >redefines some paths. This will not cause A to be outdated and no
> >rebuild of A will be triggered. Now installing B, which depends on A,
> >will use the new port group. The result is unspecified and the build
> >would also most likely fail.
> 
> Yes, I already pointed that out in a previous message.
> 
> This should *not* be the case if the current lookup rules for
> _resources continue to be applied, but a higher priority central
> location (var/macports/sources/PortGroups) is added where copies of
> (symlinks to) PortGroups are installed.
> In that scenario, foo-1.0 will only be replaced when the user installs
> the custom port A. Nothing will change if the custom source contains a
> rogue foo-1.0 PortGroup that does not belong to a variant of port A,
> or if that custom port A is never installed.

The whole installation idea does not work with the current codebase. If
a PortGroup changes, the information in the PortIndex needs to be
updated, but the PortIndex uses the modification date of Portfiles to
determine whether the information is current.

That's also why enabling a source cannot silently change the behavior of
a port: it's index has already been generated, and there is currently no
way of knowing that it should be re-generated with the new PortGroup.
Adding code to track this would add significant complexity to achieve
something that is already possible by copying PortGroups and ports.

> > needed, or the version of the port group could be changed.
> 
> Changing a PortGroup version is a no-go as far as I'm concerned, as it
> makes it impossible to build existing ports against an alternative
> PortGroup without modification.

You can not change how a port behaves without touching the Portfile. See
the above rationale on the PortIndex.

> > As soon as a port group was overridden for a port we would have to
> > disable binary archives completely for this port. 
> 
> Not necessarily, and in fact that depends more on the associated
> *port* than on the PortGroup. My custom cmake PortGroup for instance
> doesn't introduce any incompatible changes (except possibly for ports
> that happen not to be built as intended with the current PortGroup ...
> but those ports are buggy anyway).

Well, and exactly the exception you mentioned there means that for the
general case, this must be disabled. We should not provide a solution
that "works mostly" and breaks in weird ways in corner cases. We want a
solution that works always.

> > However, if you copy the port into the customized ports tree with
> > the new port

Re: PortGroup directory hierarchy/priority

[Clemens Lang]

On Mon, Apr 04, 2016 at 12:37:05AM +0200, René J.V. Bertin wrote:
> Take a custom version of a port "foo" which has a corresponding
> PortGroup "foo". That PortGroup contains information foo's dependents
> require, and that is not identical between the custom and default
> port:foo.

What kind of information are we talking about here? Compiler flags?
Paths?

> A user can install the port tree containing that port,PortGroup
> combination for other reasons, and not install the custom port:foo. In
> that case, the custom foo PortGroup should be ignored, but when the
> custom port:foo is installed any port from the main port tree that
> depends on "foo" should use the appropriate PortGroup.

That would potentially cause ports to build different depending on which
packages you have installed in your environment, which is not
reproducible. The behavior of a port might even be affected by ports
that are not in its transitive dependency tree. Additionally, this could
cause a support burden for us…

"""
 Reporter: Here's this log, the build of A fails.
 Debugger: Cannot reproduce. Your build passes -DFOO, do you have local
   modifications of the port or an overlay of it in the ports tree?
 Reporter: No, the Portfile is pristine and my overlays don't have the
   port… (but a random other port installs an override for a
   PortGroup that is used by this port).
 Debugger: Closed Won't Fix: Please don't fiddle with our ports and
   report bugs about that.
"""

You might not even be able to get such a bug fixed with the developers
of such a PortGroup, e.g. because the conflict might be between two
non-standard port trees. The more I think about this, the more I get the
impression a PortGroup should only be affecting its port tree, and
nothing else.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

Hi,

On Sun, Apr 03, 2016 at 05:56:49PM +0200, René J.V. Bertin wrote:
> >How did you presume that? Are you sure your assumption is correct?
> 
> The only sensible feedback I got was from the CrashReporter after I
> sent a SIGTERM (^]) which showed a backtrace into sqlite fsck
> functions. I also think I had something similar before, and like the
> previous time, restoring the registry.db file from backup unblocked
> the situation.
> Now, maybe the file wasn't "corrupt" but in some other state, but as
> far as I'm concerned the net result was the same ;)

Sounds like SQLite was doing its job and failed at it. There's nothing
we can do from a MacPorts PoV on SQLite failing to recover. Is there
anything odd about your configuration that could cause problems with
SQLite?

> > That would still leave a time window where your registry doesn't
> > match what's on disk. You wouldn't loose all data, but your data
> > would now be faulty. I'd argue that's not much of an improvement.
> 
> Does it? If the operation that would introduce new data fails and
> leaves the file in some unknown state, can you really say new data has
> been recorded?

You suggested having a cron job that makes a backup. If an operation was
successful after said backup, the state has already changed and going
back to the backup will loose data. The only other alternative is making
a backup copy before each and every operation that modifies the
database.

> It's not unlike what journaled and COW filesystems do (as far as I
> understand those). Commit the new state to some transitional registry,
> and make that current when you know the commit has succeeded. If that
> operation fails, you only lose whatever you were trying to change,
> just like in the scenario I had in mind - and in our case that should
> typically be an operation that'll be easy enough to repeat.

We're using a databse precisely to get this behavior. We shouldn't
second-guess the database but fix the reason that causes the database to
not provide this behavior.

> Yeah, I was thinking of that too. The problem is of course that's
> something you think about *after* hitting ^C when you realise you just
> launched something that also carries the risk of borking your whole
> install. Not easy to time that ^C (maybe I should get used to using ^Z
> instead).

Which is why we're adding signal handling to fix that.

> I was very specifically thinking of a mechanism that maintains a
> backup of the last known valid state. I know I mentioned an offline
> procedure, but that was with launchd's file-watching feature in mind
> which would trigger the copy after each change to the file.

It's an SQLite database, each transaction will trigger a change to the
file. That would cause a lot of backups to be created. Additionally, you
cannot safely copy a consistent representation of a database without
using the database access layer (on standard OS X filesystems), so this
achieves nothing: How would you ensure that the database isn't being
modified while you copy it?

> > I agree, upgrade --force should imply only rebuilding the given
> > ports. A patch would be nice.
> 
> Any idea if that's going to require a lot of changes or rather
> something that's as trivial as it sounds like it should be?

Josh's answer sounds like it's not that much effort. I don't know,
however, I'm not very familiar with that part of MacPorts base.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

On Sun, Apr 03, 2016 at 11:19:33PM +1000, Joshua Root wrote:
> Not if they have outdated dependencies. Upgrade without -n is a
> recursive operation.

Good point.

> Not passing the --force flag on to the deps by default would be ok I
> guess, but what if the user wants that?

Doesn't the global -f option achieve the same, then?

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PortGroup directory hierarchy/priority

[Clemens Lang]

Hi,

On Sun, Apr 03, 2016 at 01:26:01PM +0200, René J.V. Bertin wrote:
> I tend to view PortGroups as header files, that is files where you
> move variables and code that should be available to more than just the
> port that "goes with" the PortGroup.

PortGroups are not headers, they are just mere include files. You should
see them as code that you would otherwise copy/paste into multiple
Portfiles, nothing more.

> In fact, you could (almost) imagine an implementation where, say, Qt5
> installs its PortGroup in some appropriate place, just like it
> installs Qt headerfiles.

PortGroups cannot be "installed". They need to be there at all times in
the port tree.

> You could even think of such an implementation where the port decides
> whether it installs any PortGroup(s). That should allow for custom
> ports with custom PortGroups that shouldn't apply outside of their
> port tree. I think it would also allow for backward compatibility if
> it keeps a fallback to the current lookup strategy.

Frankly, I don't see the problem with the current method. Just because
you don't like to use SVN during your development doesn't mean that we
should change the implementation.

In fact, I'd argue that a PortGroup should only ever be allowed to
influence the ports in its port tree. The rationale for this is trust:
Enabling a third-party port tree should not give the developers of this
port tree to modify how the main port tree behaves (other than
overriding ports, possibly, but depending on the user's choice in
sources.conf).

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: registry backup (and upgrade --force)

[Clemens Lang]

Hello,

On Sun, Apr 03, 2016 at 11:31:47AM +0200, René J.V. Bertin wrote:
> Subsequent port commands hung without even a line of debug output,
> which I finally presumed due to registry corruption.

How did you presume that? Are you sure your assumption is correct?

> I think something like this already came up not that long ago:
> wouldn't it be possible and wise to keep a backup of the registry.db
> file, possibly compressed? The file does become quite large, so making
> a compressed copy before every operation that changes the file (or
> after every successful change) might be too costly, but the
> compression could be handled "offline" by a launchctl script.

That would still leave a time window where your registry doesn't match
what's on disk. You wouldn't loose all data, but your data would now be
faulty. I'd argue that's not much of an improvement.

We do, however, have signal handling code on trunk that should
gracefully handle interruptions in the critical phases that touch the
registry and avoid these problems in the future. Until then:
Interrupting MacPorts during activation/deactivation isn't safe. Don't
do it.

> I don't know if there's a way to check an sqlite3 db's integrity
> without handing

There is a way to check a sqlite3 database's integrity.

> but if it is, an existing backup could be used to replace a corrupt
> registry more or less transparently.

Only if the contents are the same, which isn't the case in general when
the backup is a day old. Doing this kind of recovery automatically would
silently introduce corrupted data -- a very bad idea.

> In addition, what do you think of using the existing prompting
> mechanisms to warn the user that `upgrade --force` without -n will
> reinstall not only the requested port(s) but all its/their
> dependencies?

I agree, upgrade --force should imply only rebuilding the given ports. A
patch would be nice.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: PortIndex2MySQL run failure on Sunday 2016-03-06 at 07:00:01

[Clemens Lang]

Hi,

On Sun, Mar 06, 2016 at 07:15:29PM +0100, Mojca Miklavec wrote:
> Whooops, I'm sorry, I didn't think about the possibility of this
> scenariot.
> 
> Are you able to help me out to come up with a proper patch or should
> we simply replace
> 
> if {[active_variants gtk3 quartz ""]} {
> default_variants-append +quartz
> } else {
> default_variants-append +x11
> }
> 
> by
> 
> default_variants-append +x11
> 
> ?

See the comments at the top of the active_variants-1.1 PortGroup for an
example that covers this case. Starting at line 34, it explains the
general case and then simplifies down to the special cases. (You are
currently using the second case, but should be using the first one.)

HTH,
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Distinguishing between deactivated ports than I want to keep and outdated port I no longer care about

[Clemens Lang]

Hi,

On Wed, Feb 24, 2016 at 01:56:55AM -0600, Ryan Schmidt wrote:
> > Is there any equivalent of "port installed" that also shows whether
> > a port was requested
> 
> No, but you can show installed requested ports with "port installed
> requested" and installed unrequested ports with "port installed
> unrequested". If you want a combined list you could use:
> 
> (port -q installed requested | sed 's/$/ (requested)/' && port -q installed 
> unrequested | sed 's/$/ (unrequested)/') | sort

port installed \
inactive and \
unrequested and not \
( $(port -q echo requested | sed -E 's/^/rdependentof:/') \)

might give you exactly the list you're looking for? Because I'm lacking
a test setup for this I cannot verify that this works as I think it
should, though.

> > and the size of .tar.bz2 with binaries?
> 
> To see the disk space used by the installed files of a port, you can use:
> 
> port space name-of-port
> 
> I'm not sure if that includes the size of the .tbz2 archive file. To get that 
> separately, you can look at:
> 
> ls -l /opt/local/var/macports/software/name-of-port/

Or rather: ls -l "$(port -q location "$portname")"

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev