Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-13 Thread Matthew D. Fuller
On Wed, Mar 12, 2014 at 07:01:20AM -0500 I heard the voice of
Bryan Drewery, and lo! it spake thus:
 
 Take openssl for example, [...]

From the user perspective, this in particular has been an occasional
but steady low-level grumble of mine for years, and I've definitely
come to wish it were hidden away.  The few times I've wanted a newer
version for something and tried using the ports openssl, it's just
turned into a giant mess of conflicts between the two and I've had to
give up and undertake a big cleanup process.  Maybe if you set the
make.conf flag and go only ports openssl straight from the first
server setup it would work right, but I'm not even sure about _that_.
Wasn't there just a discussion earlier this week about something not
honoring it right?

So, yeah; if you ask me, openssl can't possibly be given the
lib/private or libbsdssl or whatnot treatment too soon   :)


-- 
Matthew Fuller (MF4839)   |  fulle...@over-yonder.net
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
   On the Internet, nobody can hear you scream.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-13 Thread Scot Hetzel
On Thu, Mar 13, 2014 at 3:42 AM, Matthew D. Fuller
fulle...@over-yonder.net wrote:
 On Wed, Mar 12, 2014 at 07:01:20AM -0500 I heard the voice of
 Bryan Drewery, and lo! it spake thus:

 Take openssl for example, [...]

 From the user perspective, this in particular has been an occasional
 but steady low-level grumble of mine for years, and I've definitely
 come to wish it were hidden away.  The few times I've wanted a newer
 version for something and tried using the ports openssl, it's just
 turned into a giant mess of conflicts between the two and I've had to
 give up and undertake a big cleanup process.  Maybe if you set the
 make.conf flag and go only ports openssl straight from the first
 server setup it would work right, but I'm not even sure about _that_.
 Wasn't there just a discussion earlier this week about something not
 honoring it right?

This was caused by the port maintainer deciding to force the port to
always us the base version of OpenSSL on FreeBSD = 10.  Instead of
letting the sysadmin decide on which to use (port / base).
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-13 Thread Matthew D. Fuller
On Thu, Mar 13, 2014 at 08:22:08AM -0500 I heard the voice of
Scot Hetzel, and lo! it spake thus:

 This was caused by the port maintainer deciding to force the port to
 always us the base version of OpenSSL on FreeBSD = 10.  Instead of
 letting the sysadmin decide on which to use (port / base).

That imputes rather more specific intent than I think is really called
for; it's just as easy to read as didn't consider the case of
somebody on = 10 wanting to use port openssl instead at all rather
than chose to disallow $PREV.  But either way, I think it makes my
point; it's a fringe-enough option that there are liable to be
landmines waiting.  And I'm confident that Murphy has me on
speed-dial, so they're sure to be in places I'm going to need on 5
minutes notice sometime.

Whereas is there _is_ no /usr/lib/lib{ssl,crypto}, it can't create a
messy conflict   8-}


-- 
Matthew Fuller (MF4839)   |  fulle...@over-yonder.net
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
   On the Internet, nobody can hear you scream.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-12 Thread Bryan Drewery

 On Mar 11, 2014, at 23:48, Alexey Dokuchaev da...@freebsd.org wrote:
 
 On Tue, Mar 11, 2014 at 07:50:37PM -0500, Bryan Drewery wrote:
 This goes against our plans to have all ports depend only on ports. I
 admit this has not been communicated well. libexecinfo should probably
 be moved to /usr/lib/private on head to prevent ports from using it.
 
 [ Taking this to ports@ as it deems important on its own ]
 
 What's wrong with depending on system libraries?  OSVERSION check does
 indeed make it a bit hackish; I would use !exists(/usr/include/execinfo.h)
 instead, but the change itself is fine, I also do so (cf. biology/ugene).

You conveniently trimmed out a lot of context here. This thread was not
'Re: ports/187468' on this list.

 
 Any port depends on our libc.  Shall we package it as well?  Many ports
 depend on zlib, bzip2 (all in the base), etc.  Does this plan of yours say
 that soon we would have to add archivers/bzip2 in LIB_DEPENDS for any port
 that wants to link against libbz2?
 
 Was it discussed somewhere, in public?  Because it is certainly news for
 me; moreover, I *like* to depend on base for simple things, just as I like
 to use system compiler whenever possible.
 
 ./danfe

Obviously we will use libc from base right now. As for compiler, we were
in very public plans last year to use a ports compiler but that fell
through.

This project will take a long time.

Having ports mostly depend on port libraries only became more easily
possible and part of the plans with recent PRIVLATELIB support in base.

PRIVLATELIB is a way to mark a library in base as private, it installs
into /usr/lib/private. This prevents any port or user-application from
using it. Removing the support burden from the project and allowing us
to better understand the scope of issues (EN/SA) that come up from that
library.

Problems with depending on base:

1. Creates a security advisory burden. Take openssl for example, any
issue there requires an SA (security advisory), freebsd-update, and if
there are ABI changes then requires rebuilding of ports and base tools.
The ports have no way to automatically rebuild. This is a manual item in
a document to do so.
2. Head packages are impossible. Every week lately there has been some
userland change that required clearing and rebuilding head packages.
Take the recent libelf.so.1 bump to libelf.so.2. Had that library been
private and only ports depending on the port version, the port build
tools would have detected the version bump and rebuilt the ports.
However because it was not in ports, ports had to manually be rebuilt by
reading /usr/src/UPDATING.
3. By depending on base we can only support the oldest release, or
depend on the port version anyway on the oldest releases.
4. Ports is a very fast moving target and src is a very slow moving
target. We have to support 5+ year old systems. This gets complex that
as a maintainer your port may be using 3 or 4 different versions of a
library. If it depended on the port version then it would be much
simpler. We have *5* official releases right now and head that all
require packages right now.

Take for example libc++. The C++ standard is changing every 4-5 years
now. If at FreeBSD 13,14 we are still supporting 10 with its C++11-only
features, we'll need the ports to use the port version of libc++ on
those systems to get (final) C++-14 or C++-17. Might as well use the
port version on all.

Currently converted libraries in head:

libssh
libucl
libunbound
libheimipcs


Recently we are working to move libreadline to private as well and fix
up all the ports. We also plan to do libdialog and libncurses soon. If
we could actually use a recent or custom libdialog, the dialog4ports
could expand a lot. Instead we are stuck supporting only the oldest
release, or bundling libraries which is very wasteful on space.

Base has way too much in it. Many libraries are imported only for some
tool that is imported. The maintainer never intended for that library to
suddenly be a promised library from the project. But many have become that.

Most of all consider a goal of packaging base. The final implementation
here is not decided. However, we definitely don't want 2 versions of
openssl and all the other libraries. We only want 1 true version for all
base and ports packages. The SA need for base goes away once all base
tools also use the port library versions as well, for sensitive
libraries such as openssl. This merges into the pkg audit/vuxml scheme
and greatly reduces workload for secteam.

Lastly ports should be fully self-contained as much as possible.
Supporting 6 different releases/versions right now is very difficult
(8.3,8.4,9.1,9.2,10.0,head). I am even working on bootstrapping bmake
into the build to allow us to remove all of the compatibility code and
simplify dev/test. The recent QT5 import did not work on 8.3 as it was
using newer make :tl which 8.3's fmake did not have. This broke 8.3
packages. An exp-run would not have caught it and other 

Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-12 Thread Alexey Dokuchaev
On Wed, Mar 12, 2014 at 07:01:20AM -0500, Bryan Drewery wrote:
  On Mar 11, 2014, at 23:48, Alexey Dokuchaev da...@freebsd.org wrote:
  On Tue, Mar 11, 2014 at 07:50:37PM -0500, Bryan Drewery wrote:
  This goes against our plans to have all ports depend only on ports. I
  admit this has not been communicated well. libexecinfo should probably
  be moved to /usr/lib/private on head to prevent ports from using it.
  
  [ Taking this to ports@ as it deems important on its own ]
  
  What's wrong with depending on system libraries?  OSVERSION check does
  indeed make it a bit hackish; I would use !exists(/usr/include/execinfo.h)
  instead, but the change itself is fine, I also do so (cf. biology/ugene).
 
 You conveniently trimmed out a lot of context here. This thread was not
 'Re: ports/187468' on this list.

Taking this to ports@ implies that this thread did not originate on ports@.
I could've simply omit reference to PR altogether; what context from the PR
changes the meaning of plans to have all ports depend only on ports?  IMHO
leaving a PR number is enough for anyone who's interested to find the origin
of the discussion, but I'm not that worried about PR rather than the problem
with base dependencies.

 Problems with depending on base: [...]

Thanks for an in-depth answer; most (if not all) of this makes sense.  Sorry
if it was discussed earlier and my question caused you quite a deal of extra
typing; all I can say in my defence that I really appreciated it.

./danfe
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-12 Thread Bryan Drewery

On 2014-03-12 09:36, Alexey Dokuchaev wrote:

On Wed, Mar 12, 2014 at 07:01:20AM -0500, Bryan Drewery wrote:

 On Mar 11, 2014, at 23:48, Alexey Dokuchaev da...@freebsd.org wrote:
 On Tue, Mar 11, 2014 at 07:50:37PM -0500, Bryan Drewery wrote:
 This goes against our plans to have all ports depend only on ports. I
 admit this has not been communicated well. libexecinfo should probably
 be moved to /usr/lib/private on head to prevent ports from using it.

 [ Taking this to ports@ as it deems important on its own ]

 What's wrong with depending on system libraries?  OSVERSION check does
 indeed make it a bit hackish; I would use !exists(/usr/include/execinfo.h)
 instead, but the change itself is fine, I also do so (cf. biology/ugene).

You conveniently trimmed out a lot of context here. This thread was 
not

'Re: ports/187468' on this list.


Taking this to ports@ implies that this thread did not originate on 
ports@.
I could've simply omit reference to PR altogether; what context from 
the PR
changes the meaning of plans to have all ports depend only on ports?  
IMHO
leaving a PR number is enough for anyone who's interested to find the 
origin
of the discussion, but I'm not that worried about PR rather than the 
problem

with base dependencies.


Problems with depending on base: [...]


Thanks for an in-depth answer; most (if not all) of this makes sense.  
Sorry
if it was discussed earlier and my question caused you quite a deal of 
extra

typing; all I can say in my defence that I really appreciated it.

./danfe


No, I do appreciate it. We need to communicate more. Bapt and I had 
discussed
this with Des briefly and had pretty much taken on this task privately. 
These

things do need to be discussed in public more.

--
Regards,
Bryan Drewery
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-12 Thread Kevin Oberman
On Wed, Mar 12, 2014 at 8:16 AM, Bryan Drewery bdrew...@freebsd.org wrote:

 On 2014-03-12 09:36, Alexey Dokuchaev wrote:

 On Wed, Mar 12, 2014 at 07:01:20AM -0500, Bryan Drewery wrote:

  On Mar 11, 2014, at 23:48, Alexey Dokuchaev da...@freebsd.org wrote:
  On Tue, Mar 11, 2014 at 07:50:37PM -0500, Bryan Drewery wrote:
  This goes against our plans to have all ports depend only on ports. I
  admit this has not been communicated well. libexecinfo should probably
  be moved to /usr/lib/private on head to prevent ports from using it.
 
  [ Taking this to ports@ as it deems important on its own ]
 
  What's wrong with depending on system libraries?  OSVERSION check does
  indeed make it a bit hackish; I would use
 !exists(/usr/include/execinfo.h)
  instead, but the change itself is fine, I also do so (cf.
 biology/ugene).

 You conveniently trimmed out a lot of context here. This thread was not
 'Re: ports/187468' on this list.


 Taking this to ports@ implies that this thread did not originate on
 ports@.
 I could've simply omit reference to PR altogether; what context from the
 PR
 changes the meaning of plans to have all ports depend only on ports?
  IMHO
 leaving a PR number is enough for anyone who's interested to find the
 origin
 of the discussion, but I'm not that worried about PR rather than the
 problem
 with base dependencies.

  Problems with depending on base: [...]


 Thanks for an in-depth answer; most (if not all) of this makes sense.
  Sorry
 if it was discussed earlier and my question caused you quite a deal of
 extra
 typing; all I can say in my defence that I really appreciated it.

 ./danfe


 No, I do appreciate it. We need to communicate more. Bapt and I had
 discussed
 this with Des briefly and had pretty much taken on this task privately.
 These
 things do need to be discussed in public more.


Thanks, danfe, for bringing this to everyone's attention.

Too many changes that have had a significant impact on users have been
discussed in near (if unintended) privacy and then suddenly were
implemented. Most of these changes has been for the better and I'd not want
to see them rolled back, but on a couple of cases a bit more notice would
have been nice. (The big exception was pulling BIND from the base system
with fairly short notice and a major downgrade in the security of the
default port installation vs. the old ports or the base install. All on the
basis of a mistaken belief of the time that BIND9 would be supported. Had
any of us who worked closely with BIND known of this, the mistake could
have been rectified and a lot of work as well as confusion avoided. OTOH, I
am happy to see BIND out of the base OS.)

This one is really, really big. It is a fairly fundamental change in how
ports does things. It will mean more installed ports and may be a bit of a
double-edged sword for maintainability. I like the concept and I suspect
that, once I have given it enough though, I will agree that it's a good
idea. I'm just happy that a lot of people a lot smarter than I am will be
looking at it, too, for potential issues. And it should not be a sudden
implementation as it should be done one facility at a time.

And, just to be clear, I do NOT believe that those in portsmgr were
deliberately trying to hide things. (Conspiracy theorists will, of course,
disagree.) It's just that it is easy to forget how limited some audiences
are compared to the number of people who can do valid evaluation of a major
proposal and make useful suggestions.
-- 
R. Kevin Oberman, Network Engineer, Retired
E-mail: rkober...@gmail.com
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Dependencies: base vs. ports (Was: Re: ports/187468)

2014-03-11 Thread Alexey Dokuchaev
On Tue, Mar 11, 2014 at 07:50:37PM -0500, Bryan Drewery wrote:
 This goes against our plans to have all ports depend only on ports. I
 admit this has not been communicated well. libexecinfo should probably
 be moved to /usr/lib/private on head to prevent ports from using it.

[ Taking this to ports@ as it deems important on its own ]

What's wrong with depending on system libraries?  OSVERSION check does
indeed make it a bit hackish; I would use !exists(/usr/include/execinfo.h)
instead, but the change itself is fine, I also do so (cf. biology/ugene).

Any port depends on our libc.  Shall we package it as well?  Many ports
depend on zlib, bzip2 (all in the base), etc.  Does this plan of yours say
that soon we would have to add archivers/bzip2 in LIB_DEPENDS for any port
that wants to link against libbz2?

Was it discussed somewhere, in public?  Because it is certainly news for
me; moreover, I *like* to depend on base for simple things, just as I like
to use system compiler whenever possible.

./danfe
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org