Re: squealing/whistling audio

2012-10-01 Thread Doug Barton
On 09/18/2012 23:17, matt wrote:
 On 09/18/12 23:00, Doug Barton wrote:
 On 9/18/2012 10:56 PM, matt wrote:
 On 09/18/12 18:01, Doug Barton wrote:
 Sometime in the last couple of months an old problem has resurfaced on
 HEAD, a sort of squealing/whistling sound in the audio, even without
 anything playing. The sound is similar to the wind whistling through
 something.

 Before I blindly go off on a bisecting spree, does anyone have a
 suggestion as to where I might look?

 Doug


 Electrically that's usually oscillation (too high gain or too much
 electrical feedback) or an unshielded input.
 Sorry, I should have been more clear. This problem doesn't occur in
 windows, or linux, and last time I checked, didn't occur in freebsd 8. I
 have everything irrlevant zeroed out in mixer.

 Doug

 It was worth a shot. It is possible that each OS (and version) is
 setting the associations up properly though except HEAD on your machine.
 Is it a laptop or a desktop board?

Desktop.

 Any difference in the nid configuration between freebsd 8 and HEAD?

No.

 Does changing any of the hw.snd sysctls (latency, exact rate, vpc_0db)
 have any effect on the sound?

It doesn't affect the squeal.

 http://freshbsd.org/commit/freebsd/r230551 might be worth a look.

Didn't help.

 I
 couldn't find anything newer that looked like it would have an effect.
 Their are some earlier commits around January that are dealing with
 signal gain that could also have an effect.
 
 Otherwise, I'm not sure where else to look.

Thanks anyway.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: squealing/whistling audio

2012-09-19 Thread Doug Barton
On 9/18/2012 10:56 PM, matt wrote:
 On 09/18/12 18:01, Doug Barton wrote:
 Sometime in the last couple of months an old problem has resurfaced on
 HEAD, a sort of squealing/whistling sound in the audio, even without
 anything playing. The sound is similar to the wind whistling through
 something.

 Before I blindly go off on a bisecting spree, does anyone have a
 suggestion as to where I might look?

 Doug


 Electrically that's usually oscillation (too high gain or too much
 electrical feedback) or an unshielded input.

Sorry, I should have been more clear. This problem doesn't occur in
windows, or linux, and last time I checked, didn't occur in freebsd 8. I
have everything irrlevant zeroed out in mixer.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


squealing/whistling audio

2012-09-18 Thread Doug Barton
Sometime in the last couple of months an old problem has resurfaced on
HEAD, a sort of squealing/whistling sound in the audio, even without
anything playing. The sound is similar to the wind whistling through
something.

Before I blindly go off on a bisecting spree, does anyone have a
suggestion as to where I might look?

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Shouldn't world be able to build without /usr/include?

2012-09-16 Thread Doug Barton
=== tools/build (obj,includes,depend,all,install)
grep: /usr/include/stdio.h: No such file or directory
/usr/obj/frontier/svn/head/tmp/frontier/svn/head/tools/build created for
/frontier/svn/head/tools/build
grep: /usr/include/stdio.h: No such file or directory
cd /frontier/svn/head/tools/build; make buildincludes; make installincludes
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/stdio.h: No such file or directory
grep: /usr/include/stdio.h: No such file or directory
grep -v HAVE_GETLINE
/frontier/svn/head/tools/build/../../lib/libmagic/config.h  config.h
rm -f .depend
mkdep -f .depend -a-DHAVE_CONFIG_H -I.
-I/usr/obj/frontier/svn/head/tmp/legacy/usr/include -std=gnu99
/frontier/svn/head/tools/build/../../contrib/file/getline.c
In file included from
/frontier/svn/head/tools/build/../../contrib/file/getline.c:32:
/frontier/svn/head/tools/build/../../contrib/file/file.h:52:74: error:
stdio.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:53:19: error:
errno.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:54:44: error:
fcntl.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:59:20: error:
stdint.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:62:22: error:
inttypes.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:64:19: error:
regex.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:65:23: error:
sys/types.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:66:23: error:
sys/param.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:68:22: error:
sys/stat.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/file.h:69:20: error:
stdarg.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/getline.c:34:20:
error: stdlib.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/getline.c:36:20:
error: unistd.h: No such file or directory
/frontier/svn/head/tools/build/../../contrib/file/getline.c:38:20:
error: string.h: No such file or directory
mkdep: compile failed
*** [.depend] Error code 1



-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: underexposed snapshots

2012-09-16 Thread Doug Barton
On 09/14/2012 23:43, Randy Bush wrote:
 ftp://ftp.FreeBSD.org/pub/FreeBSD/ISO-IMAGES-arch is a bit empty.  i
 guess things are moving around.  any idea where i can get the latest
 tag=.

I and others have brought up this issue repeatedly over the last couple
of years, and the PTB have decided that since allbsd is doing it for us,
we don't need to put any effort into making it happen ourselves. That in
spite of the fact that numerous volunteers have come forward willing to
help.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-12 Thread Doug Barton
On 09/11/2012 02:52 AM, Erik Cederstrand wrote:
 So can we do a sweep on the ports tree and mark the 2232 ports with 
 USE_GCC=4.2 until they can actually build with clang?

Unfortunately it isn't that simple. We already have a statistically
significant number of ports that don't even compile with gcc 4.2.1. How
many compilers do we expect the users to install? :)

What we need to do is what I and others have been asking to do for
years. We need to designate a modern version of gcc (no less than 4.6)
as the official default ports compiler, and rework whatever is needed to
support this. Fortunately, that goal is much more easily achieved than
fixing ports to build and run with clang. (It's harder than it sounds
because there are certain key libs that define some paths depending on
what compiler they were built with, but still easier than dealing with
clang in the short term.)

Once that is done, the compiler in the base is an afterthought, and we
can do away with gcc in the base altogether much more easily. Users who
want to help support building ports with clang can continue to do so.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-12 Thread Doug Barton
On 09/11/2012 11:15 PM, Mark Linimon wrote:
 On Tue, Sep 11, 2012 at 11:27:50AM +0200, Lars Engels wrote:
 At the moment the ports maintainers don't give much about if their ports
 build with CLANG or not because they're not forced to.
 
 I think this is a mis-representation.
 
 Adding the requirement your ports must work on clang is adding an
 ex-post-facto requirement.  This creates the following matrix of what
 we are implicitly asking maintainers to do:
 
 (FreeBSD 7|8|9|10) * (amd64|arm|i386|powerpc|sparc64) * (base gcc|base clang)
 
 It is completely insane to expect anyone to be able to test in all of those
 environments, or even a tiny subset of them.  This isn't what most people
 sign up for when they sign up to maintain ports.
 
 Those who don't run CURRENT won't notice, but those who do will have to
 get their butts up and fix the ports
 
 I think it's foolish to assume that maintainres don't have their butts in
 gear as it is.  Please note, we have nearly 1300 PRs, hundreds of ports with
 build errors and/or PRs, and hundreds that fail on -current only.  I try to
 advertise all these things the best I know how.  Adding the hundreds that
 fail on -clang only and then blaming the maintainers is simply going to be
 counter-productive.

Write the day on your calendars folks, I completely agree with what Mark
said above. :) This is a big part of what I meant with some of my more
colorful comments in my original post on this topic.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-12 Thread Doug Barton
On 09/11/2012 05:03 AM, Steve Kargl wrote:
 On Tue, Sep 11, 2012 at 04:10:13PM +0200, Dimitry Andric wrote:

 However, I think the majority of users can get by just fine using clang,
 right now.  Doug Barton even confirmed in this thread that 80% of our
 ports already work with it!
 
 He stated that 80% build with clang.  I doubt that he actually
 tested the functionality of some 17000 ports.

Correct.

Also, users who actually are helping with testing clang for ports
continue to report runtime problems, even with things that build fine.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-12 Thread Doug Barton
On 9/12/2012 12:40 AM, Erik Cederstrand wrote:
 Den 12/09/2012 kl. 11.29 skrev Doug Barton do...@freebsd.org:
 
 On 09/11/2012 02:52 AM, Erik Cederstrand wrote:
 So can we do a sweep on the ports tree and mark the 2232 ports
 with USE_GCC=4.2 until they can actually build with clang?
 
 Unfortunately it isn't that simple. We already have a
 statistically significant number of ports that don't even compile
 with gcc 4.2.1. How many compilers do we expect the users to
 install? :)
 
 If a port doesn't compile with the default compiler in base, I expect
 that port to add a build dependency on the compiler that it actually
 does compiles with.

Yes, they do this now. The problem is that the set is growing, and the
rate of growth is increasing.

 Of course, I hope to not have 6 different
 compilers installed on my system, but the list of build or runtime
 dependencies are at the discretion of the port (maintainer). As you
 (I think) said, we can't force port maintainers to patch their ports
 to support clang.

Those are unrelated issues. Please re-read the bits of my post that you
snipped. The overwhelming majority of problems we have with compiling
ports now would be fixed by having a modern version of gcc as the
official (i.e., supported) ports compiler. The clang efforts would be
a parallel track.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-12 Thread Doug Barton
On 9/12/2012 1:49 AM, David Chisnall wrote:
 On 12 Sep 2012, at 10:09, Doug Barton wrote:
 
 Also, users who actually are helping with testing clang for ports
 continue to report runtime problems, even with things that build fine.
 
 I hope that you are encouraging maintainers of ports that don't work as 
 expected with clang to submit bug reports upstream.  We can't fix bugs if we 
 aren't made aware of them.

I personally am not directly involved in this effort (other than for my
own ports), but from what I've seen the classical emphasis on pushing
bug reports upstream has been applied in this area as well.

hth,

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-11 Thread Doug Barton
On 09/11/2012 02:27 AM, Lars Engels wrote:
 On Mon, Sep 10, 2012 at 10:54:04PM -0700, Doug Barton wrote:
 As of last week, 4,680 ports out of 23,857 failed to build with clang on
 9-amd64. That's almost a 20% failure rate. Until we have better support
 for either building ports with clang, or have better support for the
 idea of a ports compiler, this change is premature. The ports are an
 important part of the FreeBSD Operating _System_, and pulling the
 trigger on the default compiler before the ports problems are addressed
 robustly seems like a big fat FU.

 That said, I agree that this issue needs to be addressed. In fact, 9
 months before the release of 9.0 I said on the internal committers list
 that there was no point in making a new release until we had thoroughly
 addressed both the default compiler for the base, and resolving the
 ports compiler issue. While there has been some movement on the
 former, there has been nothing done on the latter for years now, even
 though everyone agrees that it is an important issue.

 I'd like to request that rather than moving the default compiler
 prematurely that you call for volunteers to address the problems with
 the ports. Both the issues of fixing more ports to build correctly with
 clang, and the issue of defining a ports compiler version of gcc (and
 appropriate infrastructure) for those that can't. Once those issues are
 resolved there would not be any further obstacles to moving the default.
 Until they are, the change is premature.

 Doug
 
 Doug, as you can already use CLANG instead of GCC now, you will be able
 to use GCC instead of CLANG after November 4th.

There's lots of things I _can_ do, what we're discussing is what the
defaults should be.

 At the moment the ports maintainers don't give much about if their ports
 build with CLANG or not

Do you follow ports development? At all? There have been extensive
efforts over the last several years to get more ports compiling with
clang. The problem is that things like the c89 issue don't percolate
down, and we don't have a concerted effort from all of the relevant
parties to improve the issue.

Fixing the problem of getting the right eyeballs on the things that need
fixing won't be improved by switching the default before they are fixed.
In fact, it's likely to make the people who are src-centric now even
less likely to help because their work will be done.

 Those who don't run CURRENT won't notice, but those who do will have to
 get their butts up and fix the ports, so 10.0 can have 99% of all ports
 build with CLANG and even 8.x and 9.x can already profit from having the
 broken ports fixed now.

Yeah, and I'm going to get a pony out of this deal, right? :)

You completely misunderstand the nature of the problem, therefore your
proposed solution isn't going to solve it.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Clang as default compiler November 4th

2012-09-10 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

As of last week, 4,680 ports out of 23,857 failed to build with clang on
9-amd64. That's almost a 20% failure rate. Until we have better support
for either building ports with clang, or have better support for the
idea of a ports compiler, this change is premature. The ports are an
important part of the FreeBSD Operating _System_, and pulling the
trigger on the default compiler before the ports problems are addressed
robustly seems like a big fat FU.

That said, I agree that this issue needs to be addressed. In fact, 9
months before the release of 9.0 I said on the internal committers list
that there was no point in making a new release until we had thoroughly
addressed both the default compiler for the base, and resolving the
ports compiler issue. While there has been some movement on the
former, there has been nothing done on the latter for years now, even
though everyone agrees that it is an important issue.

I'd like to request that rather than moving the default compiler
prematurely that you call for volunteers to address the problems with
the ports. Both the issues of fixing more ports to build correctly with
clang, and the issue of defining a ports compiler version of gcc (and
appropriate infrastructure) for those that can't. Once those issues are
resolved there would not be any further obstacles to moving the default.
Until they are, the change is premature.

Doug


On 09/10/2012 14:12, Brooks Davis wrote:
 [Please confine your replies to toolch...@freebsd.org to keep the thread
 on the most relevant list.]
 
 For the past several years we've been working towards migrating from
 GCC to Clang/LLVM as our default compiler.  We intend to ship FreeBSD
 10.0 with Clang as the default compiler on i386 and amd64 platforms.  To
 this end, we will make WITH_CLANG_IS_CC the default on i386 and amd64
 platforms on November 4th.
 
 What does the mean to you?
 
  * When you build world after the default is changed /usr/bin/cc, cpp, and
c++ will be links to clang.
 
  * This means the initial phase of buildworld and old style kernel
compilation will use clang instead of gcc.  This is known to work.
 
  * It also means that ports will build with clang by default.  A major
of ports work, but a significant number are broken or blocked by
broken ports. For more information see:
  http://wiki.freebsd.org/PortsAndClang
 
 What issues remain?
 
  * The gcc-clang transition currently requires setting CC, CXX, and CPP
in addition to WITH_CLANG_IS_CC.  I will post a patch to toolchain@
to address this shortly.
 
  * Ports compiler selection infrastructure is still under development.
 
  * Some ports could build with clang with appropriate tweaks.
 
 What can you do to help?
 
  * Switch (some of) your systems.  Early adoption can help us find bugs.
 
  * Fix ports to build with clang.  If you don't have a clang system, you
can use the CLANG/amd64 or CLANG/i386 build environments on
redports.org.
 
 tl;dr: Clang will become the default compiler for x86 architectures on 
 2012-11-04
 
 -- Brooks
 


- -- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBCAAGBQJQTtH8AAoJEFzGhvEaGryEU3gIAJ3X2EHDCVnkC/CYTMOkceho
KS6qVcQK4OCbbG+8TKkjrHNdiBO7ZuJKxfvr/TZC1zNKc8wYBlWo3s07wCHmu8Nj
OP8UwTMKumnljnYlRanQiLO9iAZKwGfI2gdxJTb5YABN2StRMXnD17Yyic6pw090
7l+cQw3iJAI8vbO4su33HJOhru0o4XLodbazHXFc6RjabAfXfuk1W6V0PfAodVC9
ZUGbF4WA7F0sJOEVuohmSk8ICHQRzTWofpdvCTlhHc1XYTaQ9u/dLGUp1C8g/BUG
CJQua7wsBdf4VgsvlYBxTAOEpURqot0Ild7zQL+9vZtf7cGCsfalpwBWzQ9J/Wk=
=gRkN
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-09-02 Thread Doug Barton
On 09/01/2012 23:01, Matthew Seaman wrote:
 As rebuilding the repo database is something you'ld do routinely anyhow
 as part of normal maintenance

Errr ... what? Why would this be true? Doesn't pkg keep the repo
database up to date as it's making changes?

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-09-02 Thread Doug Barton
On 09/01/2012 12:59, Garrett Cooper wrote:
 Again, this is part of the reason why I suggested multiple release
 trains. Although it's more painful for bapt@, et all, it's ultimately
 what would need to be done in order for pkgng to be packaged with a
 release or set of releases.

Garrett,

I think you're seriously misunderstanding how pkg is going to work.

One thing we desperately want to get away from is tying ports stuff to
specific base releases. Can you please be more clear as to why you keep
trying to pull things back in the wrong direction?

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Can't build FreeBSD-head with CLANG

2012-09-02 Thread Doug Barton
On 08/30/2012 09:16, Dimitry Andric wrote:
 [Note that linking GPL-contaminated code into your
 kernel proper is, shall we say, ideologically impure ;-)  But that is
 not the issue here.]

Can you keep this kind of stuff to -chat please? The more we deal with
the technical aspects the better off we will all be.

I for one put ext2fs in my kernel config because I have critical stuff
on those file systems and I both do want the speed boost and don't want
to worry about what's going to happen when I boot a new kernel.

Tools, not policy.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-30 Thread Doug Barton
On 08/30/2012 07:32 AM, John Baldwin wrote:
 On Thursday, August 30, 2012 1:10:24 pm Chris Rees wrote:
 On 30 Aug 2012 18:03, John Baldwin j...@freebsd.org wrote:

 On Thursday, August 30, 2012 10:39:17 am Tijl Coosemans wrote:
 On 27-08-2012 18:24, John Baldwin wrote:
 On Sunday, August 26, 2012 4:37:53 pm Doug Barton wrote:
 The problem is that we don't really support the idea of things in the
 base magically deleting themselves.

 As I have said in previous messages, the bootstrapping problem is being
 overblown by several orders of magnitude. For newly installed systems
 where pkg is the default, /usr/local/bin/pkg will be installed. So there
 is no bootstrapping problem.

 For already-installed systems who wish to switch to pkg, they can
 install from /usr/ports, or use the pkg bootstrap tool in the base.
 Given that they will be intentionally making this change, and there will
 be instructions written up on how to do this which include the
 bootstrapping step, once again this is a non-issue.

 The whole idea of having every call to /usr/local/sbin/pkg pass through
 /usr/sbin/pkg in order to help a tiny minority of users with a one-time
 bootstrapping issue is just plain ludicrous.

 I agree.  Even if we keep /usr/sbin/pkg, we will presumably want to remove
 it from the base in a year or so via 'make delete-old', etc.  Given that,
 I'm not sure we need it there in the first place.

 What if you pkg_delete \* or rm -rf /usr/local? Do you have to reboot
 pkg then?

 Yes, if we've decided it (pkgng) should not be part of the base.  This
 doesn't strike me as that weird.  It seems similar to how one has to
 bootstrap, say, MacPorts.

 Difference is, MacPorts isn't the official Mac distribution centre.

 Leaving out pkg-bootstrap (or whatever) is marginalising ports as a
 non-integral part of the OS.
 
 *sigh*  I sadly expected an emotional red herring reply such as this.
 
 This has nothing to do with marginalising ports.  Prior to this it has been
 a key argument and point that pkg* should _not_ be tied to the base system as
 that limits flexibility in the pkg tools.  I completely agree with that
 argument and having /usr/sbin/pkg doesn't appear to be consistent with that.
 
 For example, we've already shipped a binary in 9.1 release that has a
 hardcoded URL of http://pkgbeta.FreeBSD.org;.  So now you are stuck keeping
 that URL around for the next N years, albeit pointing to the production
 (not-beta) repository.  You can't safely reuse pkgbeta.FreeBSD.org for 
 anything
 until 9.1 is EOL'd.  And you'd have to change that before 9.2 and 10.0 if you
 want to avoid being in the same boat for even longer.  That is directly 
 contrary
 to the goal of having pkg* not being tied to the base.  A much more flexible
 and scalable approach would be for each pkg repository to include a 
 binary/script
 whatever that you can make available at a URL (which is easily changeable in
 documentation on our website) that when you run self-extracts and bootstraps
 pkgng.  (The pkg-static stuff is already basically this AFAICT.)
 
 If you wish to support existing users of, say, 8.2 or 8.3 release then you 
 need
 something like this anyway.  Also, as a downstream consumer who plans to use
 a custom pkgng repository on top of a modified FreeBSD distribution, this 
 approach
 is less failure prone (i.e. if someone runs 'pkg' and it tries to download 
 something
 from some hardcoded URL that's completely wrong).

I agree with John on all counts here. Further, the idea of a
self-installing package, at least for the pkg stuff itself, addresses
the issue that someone else brought up about how to handle installation
of pkg by the installer for a new system.

For example it's pretty common in the Linux world to have a package
which is wrapped in a shell script which unpacks the tarball, verifies
it with a digital signature, then installs the bits from the tarball
where they need to go. Since pkg brings a lot of the pieces of this to
the party already, it wouldn't be hard to add the rest.

... and please feel free to insert your favorite version of my We have
to get away from the idea that something is only good/cool/really part
of FreeBSD if it's in the base rant here. :)

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/25/2012 02:49, Julien Laffaye wrote:
 True. But when you create jails without the installer, you have to
 install pkgng by hand. 

Just like all the other ports you have to install in a jail.


-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 11:37, Baptiste Daroussin wrote:
 On Sun, Aug 26, 2012 at 11:34:08AM -0700, Doug Barton wrote:
 On 08/25/2012 02:49, Julien Laffaye wrote:
 True. But when you create jails without the installer, you have
 to install pkgng by hand.
 
 Just like all the other ports you have to install in a jail.
 
 We are speaking about binary only packages, not ports.

Um, duh. I have a bad habit of using the terms interchangeably, sorry
if I caused confusion. Doesn't change my actual point though.


-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 05:58, Baptiste Daroussin wrote:

 The is the longer plan but this with also true with pkg_add -r, and the pkg
 bootstrap may it be pkg-bootstrap or /usr/sbin/pkg. We have been discussing 
 with
 Security officers and we are waiting for the plan being written and setup by
 them, so we can improved security in both pkgng and the bootstrap. This should
 have happen in BSDCan, but lack of time from everyone, didn't made it happen, 
 we
 are now aiming at Cambridge DevSummit for that.

It would be nice if this were in place before 10-current shifted to pkg
by default in order to limit the number of times that we have to start
testing over from scratch.

 Given that such a security issue is already in with the current pkg_* tools, 
 it
 was accepting that we can still go that way until the policy is written, given
 that the final goal is to have the pkgng package checked against a signature.

This isn't the security issue I was talking about by having sbin/pkg
pass every command line to local/sbin/pkg.

You keep saying that you have no objections to changing the name. I am
asking you to do that. I don't care if it is pkg-bootstrap or something
else you like better. But please change the name to not be pkg, and
limit the functionality of the tool to bootstrapping the pkg package.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 11:58, Baptiste Daroussin wrote:
 On Sun, Aug 26, 2012 at 11:39:07AM -0700, Doug Barton wrote:
 On 08/26/2012 05:58, Baptiste Daroussin wrote:

 The is the longer plan but this with also true with pkg_add -r, and the pkg
 bootstrap may it be pkg-bootstrap or /usr/sbin/pkg. We have been discussing 
 with
 Security officers and we are waiting for the plan being written and setup by
 them, so we can improved security in both pkgng and the bootstrap. This 
 should
 have happen in BSDCan, but lack of time from everyone, didn't made it 
 happen, we
 are now aiming at Cambridge DevSummit for that.

 It would be nice if this were in place before 10-current shifted to pkg
 by default in order to limit the number of times that we have to start
 testing over from scratch.

 Given that such a security issue is already in with the current pkg_* 
 tools, it
 was accepting that we can still go that way until the policy is written, 
 given
 that the final goal is to have the pkgng package checked against a 
 signature.

 This isn't the security issue I was talking about by having sbin/pkg
 pass every command line to local/sbin/pkg.

 You keep saying that you have no objections to changing the name. I am
 asking you to do that. I don't care if it is pkg-bootstrap or something
 else you like better. But please change the name to not be pkg, and
 limit the functionality of the tool to bootstrapping the pkg package.

 
 I received more feedback about keep pkg

As far as I could tell the people who responded that way don't seem to
be aware that every command to /usr/local/sbin/pkg is going to pass
through /usr/sbin/pkg. On its face, that is a bad idea for many reasons,
not the least of which is that it adds complexity where that complexity
does not need to be. The larger problem with that approach is that gives
an attacker 2 places to compromise the package installation process
instead of just 1. This becomes even more important if the pkg bootstrap
tool is the place that the public key for the digital signature is located.

 and changing it to
 pkg-bootstrap, so what should I do, changing it because you are asking for it?

A) You said you had no objections to changing it
B) I'm not the only one asking

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 12:08, Ian Lepore wrote:
 Would this get better if the bootstrap tool were named pkg and were
 installed on a fresh system at /usr/local/sbin, so that it in effect
 replaces itself with the real thing, and has no need to leave a
 forwarding stub in /usr/sbin ?
 
 Maybe it could rename itself to /usr/local/sbin/pkg-bootstrap as part of
 replacing itself, so that you could re-bootstrap your way out of a
 problem later.

That's certainly creative thinking, but I'm still queasy about 2
commands with the same name that do 2 different things. And having it
rename itself adds to the confusion down the road.

Having a simple pkg bootstrapping tool in the base is a good idea. But
the functionality needs to be extremely limited so that we don't
increase the security exposure; and so that we don't end up in a
situation where a bug fix for something in the base limits our ability
to innovate with pkg in the ports tree.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 13:02, namor wrote:
 On Thu, Aug 23, 2012 at 03:28:27PM -0700, Doug Barton wrote:
 On 8/23/2012 3:19 PM, Steve Wills wrote:
 Hi,

 It seems to me that renaming the pkg binary in /usr/sbin/pkg to 
 /usr/sbin/pkg-bootstrap would make sense. From a user standpoint, it is 
 confusing that running the command gets different results the second time 
 it is run vs. the first time. I can imagine a user saying I ran pkg, but 
 it didn't do what they said it would.  Now I run it again, and it does do 
 what it is supposed to. Also, it would enable setting up a pkg-bootstrap 
 man page separate from the pkg man page, without confusion about which one 
 you're looking at.

 So, opinions? There may still be time to fix it for 9.1 if we can decide 
 quickly.

 Yes please.

 Every time in the past that we have talked about moving the pkg_* tools
 to the ports the corresponding change for the base was to have a
 pkg_bootstrap tool that was a use once and forget kind of thing. I was
 quite surprised when sbin/pkg was added, but since people tell me I
 already comment on too much, I decided to wait and see what others thought.
 
 If I understand correctly, the main concern of the pkg-name fraction
 is to not confuse newbies. All you write is pkg install foo and pkg
 will bootstrap itself if not installed. You don't have to call
 pkg-bootstrap first (how would you know about it anyways? read pkg(8)?)
 
  - How about his: stick with /usr/sbin/pkg-boostrap
  - cat  /usr/sbin/pkg  EOF
 #!/bin/sh
 echo To use pkg you have to bootstrap the pkgng installation first,
 please call /usr/sbin/pkg-bootstrap
 EOF
 
  - pkg-debootstrap replaces/removes /usr/sbin/pkg messenger (above) after
successful installation

Again, creative thinking, so you get points for that. :)

The problem is that we don't really support the idea of things in the
base magically deleting themselves.

As I have said in previous messages, the bootstrapping problem is being
overblown by several orders of magnitude. For newly installed systems
where pkg is the default, /usr/local/bin/pkg will be installed. So there
is no bootstrapping problem.

For already-installed systems who wish to switch to pkg, they can
install from /usr/ports, or use the pkg bootstrap tool in the base.
Given that they will be intentionally making this change, and there will
be instructions written up on how to do this which include the
bootstrapping step, once again this is a non-issue.

The whole idea of having every call to /usr/local/sbin/pkg pass through
/usr/sbin/pkg in order to help a tiny minority of users with a one-time
bootstrapping issue is just plain ludicrous.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-26 Thread Doug Barton
On 08/26/2012 13:35, Warren Block wrote:
 On Sun, 26 Aug 2012, Ian Lepore wrote:
 
 On Sun, 2012-08-26 at 20:58 +0200, Baptiste Daroussin wrote:
 On Sun, Aug 26, 2012 at 11:39:07AM -0700, Doug Barton wrote:
 On 08/26/2012 05:58, Baptiste Daroussin wrote:
 This isn't the security issue I was talking about by having sbin/pkg
 pass every command line to local/sbin/pkg.

 You keep saying that you have no objections to changing the name. I am
 asking you to do that. I don't care if it is pkg-bootstrap or something
 else you like better. But please change the name to not be pkg, and
 limit the functionality of the tool to bootstrapping the pkg package.


 I received more feedback about keep pkg and changing it to
 pkg-bootstrap, so what should I do, changing it because you are
 asking for it?

 Would this get better if the bootstrap tool were named pkg and were
 installed on a fresh system at /usr/local/sbin, so that it in effect
 replaces itself with the real thing, and has no need to leave a
 forwarding stub in /usr/sbin ?

 Maybe it could rename itself to /usr/local/sbin/pkg-bootstrap as part of
 replacing itself, so that you could re-bootstrap your way out of a
 problem later.
 
 Ew.  But on a similar note, an idea I just had in IRC is to have pkgng
 overwrite the base /usr/bin/pkg with a link to /usr/local/bin/pkg.
 That effectively removes that binary.  We do have precedent for ports
 overwriting base with sendmail and openssl.

... and bind, but that's a whole different category of problems.

 Hmmm, might have to be careful that future updates don't replace the
 real thing with a newer bootstrap program.
 
 Yes.  A link could be detected by installworld and not overwritten...
 although that's a hack.

Like you said above, Ew. :)

There really is no need to be so clever here. The bootstrapping issue is
going to be a minor annoyance that affects a small percentage of our users.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-24 Thread Doug Barton
On 8/23/2012 8:03 PM, Eitan Adler wrote:
 On 23 August 2012 22:59, Doug Barton do...@freebsd.org wrote:

 I tend to agree with Steve here ... we can't be responsible for other
 people's poorly written docs.
 
 This isn't about poorly written docs. This is the user expecting a
 tool to exist, which doesn't. Take another example of a sysadmin which
 rarely installs new systems, installs FreeBSD for the third time, and
 then gets confused when pkg install vim fails.

Aren't we going to install the pkg package on new systems when they are
installed? Isn't that (shouldn't that be?) part of the project plan? It
would be insane for us not to do that, at least for the releases where
pkg is the default.

 You bring up a valid point that we should
 keep in mind for our own however. The bootstrapping issue will be the
 smallest possible annoyance on a long road of the migration process.
 
 The bootstrapping issue is a factor even after the migration :)

I think that the point I'm trying to make is that it shouldn't be.

 note that I'm not talking about the mechanism here, I'm trying to
 avoid pkg doesn't seem to be installed on my fresh system becoming a
 FAQ.

The way that we avoid that problem is not to create it for ourselves in
the first place. :)

The role of pkg-bootstrap is for those users who have already-installed
systems that need to do the conversion, or if somehow pkg becomes
corrupted on the user's system and needs to be reinstalled. That's it.

I like that you're thinking through the related issues, but in this
particular case I think you're overthinking it.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-24 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 8/24/2012 1:15 AM, Baptiste Daroussin wrote:
 BTW for people who haven't tested and want to share their opinion,
 here is how work /usr/sbin/pkg:
 
 it first checks if ${LOCALBASE}/sbin/pkg is there - if yes it
 directly execute ${LOCALBASE}/sbin/pkg with arguments passed to
 /usr/sbin/pkg

As others have already pointed out, this is a bad idea for a variety
of reasons, not the least of which is security related. It also
removes one of the primary benefits of pkg, that it be (fully) hosted
in the ports tree.

The bootstrap procedure does not need to be simple or transparent
because it's only going to exist for a very short period while users
are bringing pkg into already-installed systems where pkg is not
already the default; and they don't have an existing ports tree. The
way that you solve the bootstrap problem for systems where pkg IS the
default is to install the pkg package at system install time.

Let me rephrase that more simply ... very few users are ever going to
need the bootstrapping tool that will be in the base. Making it
mandatory for *every* user is therefore not only a bad idea, it's
contrary to one of the primary goals of the project.

Doug

- -- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (MingW32)

iQEcBAEBCAAGBQJQOAJpAAoJEFzGhvEaGryEY2QH/2Hv+cW20htODBTrFNScd7qS
NxBc7YHr3ddoyoui+Qwtl3ErjMOJr+kA3nRSsZ5ewGN5HVQ4gCAWp76bJn75BM71
q2Qmgx7HCnBMJKrmRTvDAA1nJcTKAgXFKK8hYQgiTOhWFaIjJxhDlln5llFcXwBa
VN0ErF421FkD8oyHpcQLga1BHLtldOL5itt/4Tp9cKaC5l2P9dBNbyCTxVa4XBiZ
tsK+x7XJqwj0NvXLk+b3icLIEeyopa3TJAoAtXZ27igg65norCoh2EPo7aJqP2zY
T75PKdbRJprwCpeJXulC02oZu/UERoIMLeaH1JbCZRcLAqaCJQuEGSP95as3hSY=
=BEya
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-24 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 8/24/2012 5:33 PM, Glen Barber wrote:
 On Sat, Aug 25, 2012 at 01:25:15AM +0100, Jonathan Anderson wrote:
 On 24 Aug 2012, at 23:38, Doug Barton do...@freebsd.org wrote:
 Let me rephrase that more simply ... very few users are ever going to
 need the bootstrapping tool that will be in the base.

 
 So, then they won't use it.  I fail to see the problem here.

That's because you're not paying attention. :)

Which comes first in your PATH, /usr/sbin, or /usr/local/sbin? Which
comes first in the default PATH?

What Baptiste said is that the way /usr/sbin/pkg works is to take
arguments handed to it and pass them through to /usr/local/sbin/pkg.
That means that every user who has their PATH configured in the default
manner (which is what every security text on Unix has recommended for 30
years) will be using /usr/sbin/pkg every time they type the pkg command.

 But surely the whole point of pkgng is that people *will* use pkg
 as the default method of acquiring third-party software, so they'll
 want to pkg install foo and have it Just Work. To say either you
 must download the ports tree in order to use binary packages or
 you must use pkg_add to install pkg seems to miss the point...

 
 /usr/sbin/pkg installs /usr/local/sbin/pkg without requiring the Ports
 Collection to be available locally.

It does much more than that. Go read the code.

As to the security related problems, they should be obvious. Having 1
binary that is always executed to pass arguments to another binary at
minimum doubles your attack surface. Given what /usr/sbin/pkg does, it
more than doubles it. Not to mention the flat out wrong-headed design of
having a binary that will be run as root whose primary purpose is to
pass arguments to another binary.

The reason this defeats the purpose of putting pkg in the ports tree is
that if there is a bug in /usr/sbin/pkg (which of course, there will be)
then it has to be fixed in the base, with all of the consequent drama
and delays that this will entail. If there is a bug in
/usr/local/bin/pkg, it gets fixed in the ports tree and instantly
updated, which is part of the virtue of having it in the ports tree in
the first place.

Given that if we do the rollout properly the bootstrap function will be
limited to a very small percentage of users, it makes sense to split
that function out into a separate, limited binary; and not pollute the
pkg stream with extra cruft it does not need.

 What I would like to know, is why all the anti-progress emails[1] have
 to wait until the Last Minute(tm) when information on pkgng availability
 has been available for quite some time now.

First off, I resent being told that because I'm raising legitimate
issues with something that I am being obstructionist, or
anti-progress. And my concerns are certainly not last minute. I've
been raising concerns about pkg since day 1, and given that there is
still no coherent, comprehensive project plan about the migration it's
not at all surprising that others are also starting to discover daemons
in the details.

It's also part and parcel of life in an open source project. Most people
don't pay attention about most things until they feel that it will be
affecting them. This is doubly true in open source. Given how well-known
this issue is, it should be planned for in any kind of big project such
as this. It's probably also worth mentioning that there are only so many
hours in the day, so one has to prioritize.

Doug

- -- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (MingW32)

iQEcBAEBCAAGBQJQOCwUAAoJEFzGhvEaGryEcpgH/2CAPBCldr7HlTjIzErqtbTO
S0ZaI0RabwEk85+HuFCmBLTbdKqVjGYcLqIbz7l6wOa20N1rPARtBDy5DkrMrL6s
5YAgWiZ43FyKQ4826VDVBvhPqxXMD0O+sETs2kskFUkV73u/r1/8EpfZgwCDk9F9
G8hqMVTRyoWgoh1HIaBba5/m4D7+UGPYE2w8M3QAGSULePYJLgaRdu+jd2aNBrJD
NFjY4lyLbitbIH1/fYHDR90KqlBVP6vr+bWUvdoHFJQ0W0HQw7wMtamo418SlORI
qfTaoHL4sA1sggHrlUVvxjgWbAtIcYT2F3K+u34yTaWAoqxoN9pzRy3GWXyFRzM=
=PNr3
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-23 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 8/22/2012 5:27 AM, Ivan Voras wrote:
 On 21/08/2012 22:15, Doug Barton wrote:
 
 And in this case, it doesn't matter how awesome the new tools
 are, they are a MAJOR paradigm shift for how users interact with
 ports, and we are
 
 Unless I've missed something,

Yes, you've missed quite a lot actually. You really need to follow the
discussion on ports@ if you want to stay up to speed.

 pkgng is actually *zero* paradigm shift for users familiar with
 *ports*, and here's why: people using ports can and will continue
 to use ports the way they are used to. AFAIK, the infrastructure
 which registers port installation is already there and there are
 also patches for portupgrade and portmaster which make them 
 interact nicely with the new package database.

For users who only have very limited interaction with the ports tree
this is probably true. But what we're seeing is that a lot of users
(especially those with larger installations, and re-packagers like
PC-BSD) have more than simple/limited ports interaction.

For those users the change is going to take time, sometimes
significant time to adjust to.

 The only important aspect of this is that the actual package
 database format changed (IMO, immensely for the better) and there
 are several other port management utilities which may need to be
 changed. People who got used to manually altering the old
 text-based package database will learn either not to do it anymore,
 since whole classes of errors have now become impossible to have,
 or learn how to do it with the new format.
 
 Can you explain what you mean as the paradigm shift for ports
 users here?

You just described it. And I certainly hope that the change is indeed
for the better, however that has yet to be demonstrated on a large
scale. I think shifting the default for 10 is going to give us more
data on this point, which is a good thing. But making it mandatory in
10 is premature.

 OTOH, people using *binary packages* (the very few and miserable
 users that they are since the old binary package infrastructure has
 sucked for the last decade or so), will get their world turned
 upside down, but for the better, and hopefully grow in numbers.

No argument from me on the sucking, but the number of users using the
existing packages is not few. There are more consumers of the
FreeBSD-distributed packages than you probably realize. But more
importantly there are a LOT of enterprise users who roll their own
package infrastructure.

I have been trying to get across to some of our src-centric Illuminati
for years just how valuable/important the ports are to the FreeBSD
Operating _System_. For better or worse I think that this change is
going to bear out the truth of what I (and others of course) have been
saying.

Doug

- -- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (MingW32)

iQEcBAEBCAAGBQJQNp/WAAoJEFzGhvEaGryEpIkH/AqfqLIugyLDWv6ehzaKhne8
pGCIGL6bS6naRzpvMu+hzA+eEg/ZnAp5tOjC2e0qowi50e5fF8CKEt11eZKOkyXA
FPQX00kX3KTKMyHd6SEsp6AL5FAihBASN9rVs3BGqBXge/ViI9HIDRBKpQW+11Yd
tH3wdCSfflI3UpteyJFFumIxITuTvAhYPBzSFEoThNAmf58qJWTNx8zW5jS3/lis
OnCWApouUfYOKdimbpRbguYiAnuX7o/Vrwvc9XQ6awsATDWNSPgf4kgaPvwnp9HH
eUlFtsNInlFMT9pwQhS2oQtIccx0BYsCQIXkCNQFIjddvRuUeVNjB5Vdqq7NuLk=
=kUKF
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-23 Thread Doug Barton
On 8/23/2012 3:19 PM, Steve Wills wrote:
 Hi,
 
 It seems to me that renaming the pkg binary in /usr/sbin/pkg to 
 /usr/sbin/pkg-bootstrap would make sense. From a user standpoint, it is 
 confusing that running the command gets different results the second time it 
 is run vs. the first time. I can imagine a user saying I ran pkg, but it 
 didn't do what they said it would.  Now I run it again, and it does do what 
 it is supposed to. Also, it would enable setting up a pkg-bootstrap man page 
 separate from the pkg man page, without confusion about which one you're 
 looking at.
 
 So, opinions? There may still be time to fix it for 9.1 if we can decide 
 quickly.

Yes please.

Every time in the past that we have talked about moving the pkg_* tools
to the ports the corresponding change for the base was to have a
pkg_bootstrap tool that was a use once and forget kind of thing. I was
quite surprised when sbin/pkg was added, but since people tell me I
already comment on too much, I decided to wait and see what others thought.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

2012-08-23 Thread Doug Barton
On 8/23/2012 7:23 PM, Eitan Adler wrote:
 On 23 August 2012 22:15, Steve Wills swi...@freebsd.org wrote:
 On Aug 23, 2012, at 10:08 PM, Eitan Adler wrote:

 On 23 August 2012 22:05, Steve Wills swi...@freebsd.org wrote:

 Why can't one of those steps be to run pkg-bootstrap?

 Because the how-to may not be for a new system ;)

 The possibility of bad docs somewhere outside of our control, when we can 
 (and I am actively working on) document(ing) pkgng for the handbook seems 
 kinda thin. It's not even Something's wrong on the Internet! 
 (http://xkcd.com/386/), it's Something might some day be wrong on the 
 Internet!
 
 It isn't a problem of bad docs. Its a problem of the user following
 some not-for-new-systems documentation and getting very confused when
 they see command not found.. It is practically the definition of
 POLA.

No, POLA refers to not changing long-established practices out from
under the user.

I tend to agree with Steve here ... we can't be responsible for other
people's poorly written docs. You bring up a valid point that we should
keep in mind for our own however. The bootstrapping issue will be the
smallest possible annoyance on a long road of the migration process.

OTOH this is a good use case for the prompt the user when they type a
command for something that can be installed from ports idea. :)

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-22 Thread Doug Barton
On 8/21/2012 6:58 PM, Bjoern A. Zeeb wrote:
 On Tue, 21 Aug 2012, Doug Barton wrote:
 
 I don't think we have ever done a complete replacement of major
 infrastructure in one release.
 
 You mean like sysinstall can be used as an installer on 9 that would
 do something meaningful with the current infrastructure we provide?

Given the number of users who complain when sysinstall breaks in 9, I'd
say yes. Not to mention that sysinstall is a good example of something
that we deprecated in one release and removed in the following release.

Furthermore, I don't think of the installer as nearly as critical as the
ports collection. Yes, it is important, clearly.  But it's something
that is likely to happen only once in the lifetime of a system, as
opposed to the numerous times that users will interact with the ports.
Not to mention all of the enterprise users who bypass it altogether.

Aside from the installer part of sysinstall, the post-install config
portion has been taken over by bsdconfig. So in HEAD you have 2 new
tools that are mandatory that fulfill sysinstall's old role; and in 9
you have those same 2 new tools which are the defaults, but optional.
That's exactly how it is supposed to work.

Finally, the thing that we have to keep in mind is how different the
ports tree is from anything else in the base. The infrastructure of the
ports has to support all versions of FreeBSD. So we have to be extra
cautious about deprecating things. Of course the upside of pkg is that
it (properly) lives in the ports tree itself, which will make innovation
much easier in a few years.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: r239356: does it mean, that synchronous dhcp and dhcplcinet with disabled devd gone?

2012-08-21 Thread Doug Barton
On 8/21/2012 6:34 AM, John Baldwin wrote:

 Humm.  devd is the more common case, and we explicitly don't use devd to 
 start 
 dhclient on boot even when devd is enabled (so out of the box dhcp would 
 first 
 be started by rc, but would be restarted by devd).

That sounds reasonable. People who choose not to run devd can be
responsible for restarting dhclient themselves.

 Another option is to rework dhclient to work like it does on OpenBSD where it
 renews its lease if the link bounces, but to not exit when the link goes down.

That would be preferable.

 That case would fix the currently broken case that you unplug your cable, 
 take 
 your laptop over to another network (e.g. take it home if suspend/resume 
 works), then plug it back in and are still stuck with your old IP.

I do think it's important to fix this case. However I agree with the
chorus of responders that it is more important to maintain our historic
resilience to temporary loss of connectivity.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-21 Thread Doug Barton
On 8/21/2012 6:46 AM, Baptiste Daroussin wrote:
 I would also like to just remove pkg_* tools from RELENG_10 if that fits the
 schedule.

Um, no?

Until pkg becomes mandatory (which can't happen for several years) the
pkg_* tools can't be removed altogether.

What _would_ be useful is what should have been done many years ago when
it was first suggested: Move the pkg_* tools to ports.

It's too late for 9.1 already, but if you made that change today in
HEAD, and after 9.1 (but before 8.4) you MFC it to stable/[89], then you
could theoretically make pkg mandatory after 9.1 EOLs.

To make my point more clear, the ports tree has to support the last
release to ship with pkg_* tools in the base throughout its lifetime. To
do anything else would be be a massive POLA violation.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-21 Thread Doug Barton
On 8/21/2012 11:47 AM, Garrett Cooper wrote:
 On Tue, Aug 21, 2012 at 11:17 AM, Doug Barton do...@freebsd.org wrote:
 On 8/21/2012 6:46 AM, Baptiste Daroussin wrote:
 I would also like to just remove pkg_* tools from RELENG_10 if that fits the
 schedule.

 Um, no?
 
 ...
 
 What _would_ be useful is what should have been done many years ago when
 it was first suggested: Move the pkg_* tools to ports.
 
 It already exists -- it's just out of date / crufty:

Right ... I was using move as shorthand for several different ideas,
including but not limited to the latest version of the code itself,
robust support for the code going forward, the primary supported way of
using pkg_*, etc. All of these ideas have been discussed in the past, so
I was hoping to avoid having to re-discuss them. :)

 It's too late for 9.1 already, but if you made that change today in
 HEAD, and after 9.1 (but before 8.4) you MFC it to stable/[89], then you
 could theoretically make pkg mandatory after 9.1 EOLs.

 To make my point more clear, the ports tree has to support the last
 release to ship with pkg_* tools in the base throughout its lifetime. To
 do anything else would be be a massive POLA violation.
 
 Agreed.

Great (and I saw Baptiste's response on this as well). Glad to hear that
we're on the same page about something at least. :)


-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-21 Thread Doug Barton
On 8/21/2012 12:05 PM, Baptiste Daroussin wrote:
 1/ if it fits the schedule: get rid of pkg_* tools in current to be
 able to have a fully pkgng only 10-RELEASE

I think it would fit better with historic precedents to make pkg
optional (but default on) in 10, and mandatory in 11. As stated
before, I'm fine with removing pkg_* tools from 10 if there is robust
support for them in the ports tree.

I know you're excited about this project, but let's not lose sight of
how big a change this is, and how important ports are to the project.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-21 Thread Doug Barton
On 8/21/2012 12:42 PM, Baptiste Daroussin wrote:
 On Tue, Aug 21, 2012 at 12:38:04PM -0700, Doug Barton wrote:
 On 8/21/2012 12:05 PM, Baptiste Daroussin wrote:
 1/ if it fits the schedule: get rid of pkg_* tools in current
 to be able to have a fully pkgng only 10-RELEASE
 
 I think it would fit better with historic precedents to make pkg 
 optional (but default on) in 10, and mandatory in 11. As stated 
 before, I'm fine with removing pkg_* tools from 10 if there is
 robust support for them in the ports tree.
 
 I know you're excited about this project, but let's not lose
 sight of how big a change this is, and how important ports are to
 the project.
 
 That was what if it fits the schedule was about.

I think what I'm trying to say, ever so politely, is that what you're
suggesting isn't even an option, so it shouldn't be discussed.


-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: pkgng 1.0 release schedule, and HEAD switch to pkgng by default schedule

2012-08-21 Thread Doug Barton
On 8/21/2012 1:08 PM, Warner Losh wrote:
 
 On Aug 21, 2012, at 1:51 PM, Doug Barton wrote:
 
 On 8/21/2012 12:42 PM, Baptiste Daroussin wrote:
 On Tue, Aug 21, 2012 at 12:38:04PM -0700, Doug Barton wrote:
 On 8/21/2012 12:05 PM, Baptiste Daroussin wrote:
 1/ if it fits the schedule: get rid of pkg_* tools in
 current to be able to have a fully pkgng only 10-RELEASE
 
 I think it would fit better with historic precedents to make
 pkg optional (but default on) in 10, and mandatory in 11. As
 stated before, I'm fine with removing pkg_* tools from 10 if
 there is robust support for them in the ports tree.
 
 I know you're excited about this project, but let's not lose 
 sight of how big a change this is, and how important ports are
 to the project.
 
 That was what if it fits the schedule was about.
 
 I think what I'm trying to say, ever so politely, is that what
 you're suggesting isn't even an option, so it shouldn't be
 discussed.
 
 If you are fine with removing them if there's robust support, how can
 you also be suggesting that it is impossible and shouldn't be talked
 about?

Those address different parts of the problem. Making pkg mandatory in 10
is different from where the old pkg_* tools end up. The command line
tools are just the tip of the iceberg, there are a lot of interactions
behind the scenes.

 Personally, I think we should handle this the same way that other
 replacement tools have been done, which is close to what Baptiste has
 proposed.  If the new tools are totally awesome, we have replaced old
 tools. 

I don't think we have ever done a complete replacement of major
infrastructure in one release. The traditional model has been to
deprecate in one release, remove in the next.

And in this case, it doesn't matter how awesome the new tools are, they
are a MAJOR paradigm shift for how users interact with ports, and we are
going to have a lot of users who take years to transition their
installed base. No matter how much we may want to move fast on this, it
just isn't going to be possible.

 If the new tools are good, but don't cover the older users,
 we develop along size. 

Yes, this is precisely what I'm saying. Sorry if I wasn't clear.

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck?

2012-08-20 Thread Doug Barton
On 08/15/2012 03:18, Alexander Motin wrote:
 On 15.08.2012 03:09, Doug Barton wrote:
 On 08/14/2012 12:20 PM, Adrian Chadd wrote:
 Would you be willing to compile a kernel with KTR so you can capture
 some KTR scheduler dumps?

 That way the scheduler peeps can feed this into schedgraph.py (and you
 can too!) to figure out what's going on.

 Maybe things aren't being scheduled correctly and the added latency is
 killing performance?

 You might also try switching to SCHED_ULE to see if it helps.

 Although, in the last few months as mav has been converging the 2 I've
 started to see the same problems I saw on my desktop systems previously
 re-appear even using ULE. For example, if I'm watching an AVI with VLC
 and start doing anything that generates a lot of interrupts (like moving
 large quantities of data from one disk to another) the video and sound
 start to skip. Also, various other desktop features (like menus, window
 switching, etc.) start to take measurable time to happen, sometimes
 seconds.

 ... and lest you think this is just a desktop problem, I've seen the
 same scenario on 8.x systems used as web servers. With ULE they were
 frequently getting into peak load situations that created what I called
 mini thundering herd problems where they could never quite get caught
 up. Whereas switching to 4BSD the same servers got into high-load
 situations less often, and they recovered on their own in minutes.
 
 It is quite pointless to speculate without real info like mentioned
 above KTR_SCHED traces.

I'm sorry, you're quite wrong about that. In the cases I mentioned, and
in about 2 out of 3 of the cases where users reported problems and I
suggested that they try 4BSD, the results were clear. This obviously
points out that there is a serious problem with ULE, and if I were the
one who was responsible for that code I would be looking at ways of
helping users figure out where the problems are. But that's just me.

 Main thing I've learned about schedulers, things
 there never work as you expect. There are two many factors are relations
 to predict behavior in every case.

In the web hosting case that I mentioned, I purposely kept every other
factor consistent; and changed only s/ULE/4BSD/. The results were both
clear and consistent.

 What's about playing AVIs and using other GUIs, key word here and for
 ULE in general is interactivity. ULE gives huge boost to threads it
 counts interactive.

I'm not using ULE. I haven't for over a year. Sorry if I wasn't clear.

 If somebody still wish area for experiments, there is always some:
  - if you want video player to not lag, set negative nice for it (ULE is
 not a magician to guess user wishes);

At the same time, I don't have these problems on my Linux systems, and I
don't need to adjust anything. Not to mention that given how web servers
are one of our main server implementations, the fact that we have what
seems to be a serious performance problem with out default scheduler in
that use case seems like an issue that we would want to address.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck?

2012-08-20 Thread Doug Barton
On 08/20/2012 02:59, Alexander Motin wrote:
 On 20.08.2012 11:32, Doug Barton wrote:
 On 08/15/2012 03:18, Alexander Motin wrote:
 On 15.08.2012 03:09, Doug Barton wrote:
 On 08/14/2012 12:20 PM, Adrian Chadd wrote:
 Would you be willing to compile a kernel with KTR so you can capture
 some KTR scheduler dumps?

 That way the scheduler peeps can feed this into schedgraph.py (and you
 can too!) to figure out what's going on.

 Maybe things aren't being scheduled correctly and the added latency is
 killing performance?

 You might also try switching to SCHED_ULE to see if it helps.

 Although, in the last few months as mav has been converging the 2 I've
 started to see the same problems I saw on my desktop systems previously
 re-appear even using ULE. For example, if I'm watching an AVI with VLC
 and start doing anything that generates a lot of interrupts (like
 moving
 large quantities of data from one disk to another) the video and sound
 start to skip. Also, various other desktop features (like menus, window
 switching, etc.) start to take measurable time to happen, sometimes
 seconds.

 ... and lest you think this is just a desktop problem, I've seen the
 same scenario on 8.x systems used as web servers. With ULE they were
 frequently getting into peak load situations that created what I called
 mini thundering herd problems where they could never quite get caught
 up. Whereas switching to 4BSD the same servers got into high-load
 situations less often, and they recovered on their own in minutes.

 It is quite pointless to speculate without real info like mentioned
 above KTR_SCHED traces.

 I'm sorry, you're quite wrong about that. In the cases I mentioned, and
 in about 2 out of 3 of the cases where users reported problems and I
 suggested that they try 4BSD, the results were clear. This obviously
 points out that there is a serious problem with ULE, and if I were the
 one who was responsible for that code I would be looking at ways of
 helping users figure out where the problems are. But that's just me.
 
 I am not telling anything bad about 4BSD.

Yes, I get that, but thanks for making it clear.

 Choice is provided because
 they are indeed different and none is perfect.

... which is why I'm asking you to stop making them more the same until
we get a better idea of what the issues are.

 What I would like to say is that if we want to improve situation, we
 need more detailed info then just verbal description.

And what I'm saying is that the only realistic way that you're going to
get that information that you need is to make it easier for users to
give it to you. I don't know what form that is going to need to take, I
don't know anything about schedulers.

 I am not telling
 that ULE is perfect. I went there because I've seen problems, and I am
 still fixing some pieces. I am just trying to explain described behavior
 from the point of my knowledge about it, hoping that it may help
 somebody to set up some new experiments or try some tuning/fixing.

Yes, I think it's great that you're doing this work. I'm glad to see
that someone is improving ULE. It clearly needs it. :)

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck?

2012-08-20 Thread Doug Barton
On 08/20/2012 06:32, Alexander Motin wrote:
 I have no plans to converge them. I've just found problem in ULE, that
 was replicated into 4BSD and it would be strange to fix one without
 another. But fixing it exposed another old problem specific to 4BSD,
 which I fixed reusing logically equivalent code from ULE. I saw no
 reason to reinvent a wheel there, same as to not fix obvious bug. Sure,
 it can change behavior in some way, but ULE is not guilty.

Thank you for that explanation.

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck?

2012-08-15 Thread Doug Barton
On 08/14/2012 09:18 PM, Dimitry Andric wrote:
 On 2012-08-15 02:09, Doug Barton wrote:
 On 08/14/2012 12:20 PM, Adrian Chadd wrote:
 ...
 Maybe things aren't being scheduled correctly and the added latency is
 killing performance?

 You might also try switching to SCHED_ULE to see if it helps.
 
 Most likely, s/ULE/4BSD/ here, and in the rest of your mail? :)
 

yes, thanks
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: CURRENT as gateway on not-so-fast hardware: where is a bottlneck?

2012-08-14 Thread Doug Barton
On 08/14/2012 12:20 PM, Adrian Chadd wrote:
 Hi,
 
 Would you be willing to compile a kernel with KTR so you can capture
 some KTR scheduler dumps?
 
 That way the scheduler peeps can feed this into schedgraph.py (and you
 can too!) to figure out what's going on.
 
 Maybe things aren't being scheduled correctly and the added latency is
 killing performance?

You might also try switching to SCHED_ULE to see if it helps.

Although, in the last few months as mav has been converging the 2 I've
started to see the same problems I saw on my desktop systems previously
re-appear even using ULE. For example, if I'm watching an AVI with VLC
and start doing anything that generates a lot of interrupts (like moving
large quantities of data from one disk to another) the video and sound
start to skip. Also, various other desktop features (like menus, window
switching, etc.) start to take measurable time to happen, sometimes
seconds.

... and lest you think this is just a desktop problem, I've seen the
same scenario on 8.x systems used as web servers. With ULE they were
frequently getting into peak load situations that created what I called
mini thundering herd problems where they could never quite get caught
up. Whereas switching to 4BSD the same servers got into high-load
situations less often, and they recovered on their own in minutes.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VirtualBox: Eating up 100% CPU, freezing Windows 7

2012-08-06 Thread Doug Barton
On 08/04/2012 17:56, Kevin Oberman wrote:
 On Sat, Aug 4, 2012 at 2:30 PM, Doug Barton do...@freebsd.org wrote:
 On 08/04/2012 14:26, Garrett Cooper wrote:
 On Sat, Aug 4, 2012 at 1:26 PM, Doug Barton do...@freebsd.org wrote:

 On 08/04/2012 00:40, O. Hartmann wrote:
 No, also in my case. I build world and the VBox software with each
 kernel - usually.

 You can ensure that by putting this in src.conf:

 PORTS_MODULES=  emulators/virtualbox-ose-kmod

 You can place other modules in that list as well. I use vbox so you can
 be pretty confident that this is going to keep working. :)

 That doesn't work

 I assure you that it does. I have put a non-zero amount of work into
 fixing it, I use this method, and the resulting kernel module works just
 fine.

 If you actually try it and find something is not as it should be, then
 yes; please do file a PR and feel free to cc me.

 Doug
 
 I am only aware of this because of your posts. No reference to it at
 all in src.conf(5). It would be nice to see it there.

It's in make.conf(5) for historical reasons, along with a lot of other
options that should be moved.

 It is mentioned in build(7), but only as a MAKE option and a lot of
 people are not going to realize it can be placed in src.conf.

It can also go in make.conf, so that's not a total loss I suppose.

 Also,
 for those not fairly conversant in make, it is not clear whether
 multiple ports should be space, comma, colon, or otherwise delimited
 
 This is a very nice option as it is very easy to overlook rebuilding
 kernel modules in ports when building the kernel. Thanks or working on
 it.

My pleasure ... it was one of those things on the list and I finally
got around to it.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-05 Thread Doug Barton
On 08/02/2012 12:18, David Chisnall wrote:
 Thank you for your thoughtful reply,

You too ... I let some time go by to see what others had to say. I think
it's disappointing that more people aren't concerned about this issue.

 On 2 Aug 2012, at 19:33, Doug Barton wrote:
 
 However, my point is that in spite of the fact that it's
 non-trivial, the mindset on this topic needs to change if the dev
 summits are going to continue to be significant focii of both work
 being done and decisions being made (which of course, they are).
 
 I believe that, before that decision can be made, there needs to be
 some consensus on what the purpose of the DevSummits is.  In my view,
 DevSummits do not exist to set project policy. 

And yet, that's exactly what happens. It's easy to understand why ...
you have a bunch of people together in the same place, they all agree on
a topic, and after all, since they are the ones who are there, they
should be the ones to make the decision, right?

It's not necessarily that they are trying to do something malicious,
it's just human nature. I know, I used to travel to conferences for a
living.

 They are places where:
 
 - People can talk face to face about their current and planned
 projects. - Developers can meet on a social basis, making remote
 working easier.
 
 The latter is very important - I've found in other projects that it's
 far easier to work with someone on the other side of the world when
 you've chatted with them over a few beverages-of-choice.

I agree with these points as well. (Again, been there, done that.) In
fact I'm quite confident that a lot of the issues people have with me
are related to this deficiency. :)

 Any official conversations happen on the mailing lists. 

This should be true, but it isn't. This is my point.

 DevSummits
 are for people working on similar things to meet and discuss their
 plans, and for people to have a chance to get to know what everyone
 else is doing,

... so far, so good ..

 for a limited set of 'everyone else'. 

... and this is where I call shenanigans. This is an open project.
Anything that is going to be done is going to be seen. If there are
problems with a proposal it's better to see them early. If it's a good
proposal, there is no reason not to share it.

 Slides and
 summaries so on from the more structured parts of this are available
 afterwards, which helps people who can't attend get the same benefit
 - they know what other people are working on.

In the IETF context proposals often benefit from the real-time focus of
attention, from both local and remote participants, that the meetings
provide. There is no reason to believe that this would not be true in
FreeBSD as well.

 The original complaint that spawned this long discussion was that
 decisions about the project are made behind closed doors.  This is
 obviously true in the literal sense, as code always wins over chatter
 in an open source project, and the person willing to sit down and
 write the code gets the final say on how it should work,

That's an oft-repeated maxim, especially around here. But we've already
demonstrated that it isn't true. The only time that this is true is if
the work being done is in line with what the PTB want. I've demonstrated
this time after time by volunteering to do various projects my way and
being told that my efforts weren't welcome. Not to mention having my
finished, working code reverted by a core team member for an inferior,
broken result.

So can we please stop repeating that BS and focus on the real issues?

 although
 ideally with code review, design feedback and so on from others.
 Even if we broadcast everything that happens in the official parts of
 the DevSummits, that won't necessarily fix anything because a lot of
 the most productive conversations happen over dinner or in the pub.

The community needs to not be accepting of the status quo, and demand
that things be publicized on the lists first before decisions are made.

Once again, if they are good decisions, they won't be harmed by sharing
them in advance. If they are less-good, we could be saving someone
efforts that would be otherwise wasted.

 If there is a real problem to address, then it is people making
 policy decisions at DevSummits, without adequate consultation.  I
 have not observed this happening, but I would regard it as no
 different to people making policy decisions via private email, and
 something that should be subject to the same response: revisit the
 decisions in public if there are legitimate concerns raised about it,
 subject to the usual open source rule that the person actually
 willing to do the work gets to make the final call.

Exactly.

 Finance is not the only obstacle.  In some venues, bandwidth is a
 problem (not at Cambridge hopefully - people will have stopped using
 it all to stream the olympics by then), but in other venues we only
 have WiFi, which is shared with a room full of developers.  If we buy
 some equipment (decent

Re: VirtualBox: Eating up 100% CPU, freezing Windows 7

2012-08-04 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 08/04/2012 00:40, O. Hartmann wrote:
 No, also in my case. I build world and the VBox software with each
 kernel - usually.

You can ensure that by putting this in src.conf:

PORTS_MODULES=  emulators/virtualbox-ose-kmod

You can place other modules in that list as well. I use vbox so you can
be pretty confident that this is going to keep working. :)

Doug

- -- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBCAAGBQJQHYWJAAoJEFzGhvEaGryEyLUIALgi1n65I8oBaFYJEcIkDB6P
W3f5PMZa72DN4r2lI3A3XXdPUJsNRmNy/X0HYyrrIwvfD3Z3m8bReYCd7DHAKOX4
pBsLA/73cwns9c3+zsUe4i9TZOsJM8fNVsRp/BSvdtBgv61ZZUurvt43H+Ek0E0B
h5ttGaIanxLqrkwP2FC/q30t0pmauJYu3jDTGiugOh98o/3oNT+25etyJBNgvg4c
VxBs/5aCSc5VHAcLXRN6Y0BGGbeimpPqEFlq3FEFGLkC7LGjqoSBUaJVz1cgDP+t
RIK9g0V+XIfyirgZ2VMeK3tfQ0Q17zfyl0+Iyzl2IxZptU67OBV/9LMqyhRaBOc=
=KbES
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: VirtualBox: Eating up 100% CPU, freezing Windows 7

2012-08-04 Thread Doug Barton
On 08/04/2012 14:26, Garrett Cooper wrote:
 On Sat, Aug 4, 2012 at 1:26 PM, Doug Barton do...@freebsd.org wrote:

 On 08/04/2012 00:40, O. Hartmann wrote:
 No, also in my case. I build world and the VBox software with each
 kernel - usually.

 You can ensure that by putting this in src.conf:

 PORTS_MODULES=  emulators/virtualbox-ose-kmod

 You can place other modules in that list as well. I use vbox so you can
 be pretty confident that this is going to keep working. :)
 
 That doesn't work

I assure you that it does. I have put a non-zero amount of work into
fixing it, I use this method, and the resulting kernel module works just
fine.

If you actually try it and find something is not as it should be, then
yes; please do file a PR and feel free to cc me.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 09:20, Scott Long wrote:
 
 On Aug 2, 2012, at 12:23 AM, Kevin Oberman kob6...@gmail.com
 wrote:
 
 Doug makes some good points.
 
 No, he doesn't. 

Yes I do! (So there)

 He and Arnould being argumentative and accusatory
 where none of that is warranted.
 
 I used to run the devsummits, and we did tele-conference lines for
 remote people to participate. 

I singled out BSDCAN specifically since that's where the action is for
the last several years. I do recall your efforts in this regard, but it
so happened that I was able to attend most of them in person back then.
No slight towards what you did was intended.

 After I stepped down, others took it
 up and did the same thing.  Usually, the lines were unused.  I
 suspect that organizers simply stopped thinking about them after a
 while because of poor interest.  There is no conspiracy of exclusion
 here, just simple human apathy.

Here I have to disagree with you. Once again, speaking specifically
about BSDCAN dev summits, I repeatedly asked the organizers to provide
some sort of audio stream (phone, Internet, anything) and was repeatedly
told it wasn't possible. This was not a case of lack of interest. This
was a case of We understand that it is something people want, but it
isn't going to happen.

 The invite system for the devsummit was, and still is, purely about
 providing some order to the process.  It ensures that people
 attending are willing to demonstrate a minimum amount of interest,
 more than just wondering by a room one day and dropping in for free
 food and wifi. 

I specifically made allowances for this issue in my post.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 05:54, David Chisnall wrote:
 On 2 Aug 2012, at 05:30, Doug Barton wrote:
 
 I used to ask the PTB to provide *some* form of remote
 participation for even a fraction of the events at the dev summit.
 I don't bother asking anymore because year after year my requests
 were met with any of: indifference, hostility, shrugged shoulders
 (that's a hard problem that we can't solve), or embarrassment.
 Since if the right people around here want something to happen, it
 happens; I finally came to the conclusion that they didn't want
 remote participation to happen, so it won't. That's a shame.
 
 You haven't asked for this for the Cambridge DevSummit,

You did read the part where I gave up, right?

 but others
 have and so we have arranged for cameras and microphones to be
 available for two of the sessions (the DocSummit and the ARM working
 group) to allow those who can not attend in person for various
 reasons to participate.

Well that's a start. :) And where was this availability announced? If I
missed it, that's on me. But providing remote access that you don't tell
people about isn't really any better than not providing it at all.

 I don't know how useful it will be (hopefully everything will work,
 but my experience with video conferencing is that it stops working as
 soon as you try to do something important with it),

If I can offer some advice from the trenches ... focus on making the
audio robust, and put efforts into the video as resources permit. The
combination of solid audio, making presentations available on line, and
a chat room (IRC, jabber, whatever) allows for a great deal of remote
participation. Video is nice, but if the video going down takes the
audio with it, you're no better off than when you started.

 but there is
 certainly no active attempt to exclude people who can't attend.

... and here is where I need to push back. No active attempt to exclude
people is not the same thing as actively encouraging remote
participation. It's the latter that we're after.

 After each DevSummit, the results seem to appear on the wiki quite
 promptly - often during the sessions.  At BSDCan this year, two of
 the working groups that I attended used OpenEtherPad to take minutes,
 so they were available in real time for non-attendees and people
 outside of the room were able to add things to them.  There are
 usually people in the room on IRC as well, who are willing to relay
 things from people outside.

Those all sound like nice steps forward, thank you for pointing them
out. Nothing would make me happier than to be proven wrong in this area.
What would be nice I think would be if these steps were formalized, and
shared more openly. Having things on the wiki is nice, but reporting
things in detail on the mailing lists puts it in the archives for future
reference, as well as making it more broadly available to start with.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 09:44, Garrett Cooper wrote:

 The Watson/Losh connection worked really well in BSDCan 2010 :). 

I wasn't going to mention that, since I didn't want to tell tales out of
school. But the fact that remote participation actually was provided for
the right people, even though I was told repeatedly that it wasn't
possible, actually highlights a big part of the problem.

 Advertising the teleconferencing lines might be an issue (I would
 have loved to have joined in some of the remote conferences, if for
 nothing more than be a fly on the wall, this year), but that's a
 separate thing aside.

At various points when I was asking for remote participation at BSDCAN
different people offered to provide this through their company's
teleconferencing solutions, providing that the organizers could put a
phone line in the room(s). They were told that it wasn't possible to do
that.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 10:13, David Chisnall wrote:
 On 2 Aug 2012, at 17:46, Doug Barton wrote:
 
 Well that's a start. :) And where was this availability announced?
 If I missed it, that's on me. But providing remote access that you
 don't tell people about isn't really any better than not providing
 it at all.
 
 It's not widely advertised, because we're likely to be able to
 support a limited number of remote participants (10 seems like the
 upper limit for the technology that we're looking at, and I wouldn't
 be surprised if it degrades before then). 

Welcome to the 21st Century. :) There are widely available audio and
video conferencing solutions that easily scale into the thousands of
users, at minimal cost.

 As with all other things
 in the project, we welcome people who are willing to make an effort
 to engage.  We provide it when people ask, not spontaneously, because
 organising cameras and decent microphones requires effort on the bart
 of the organisers. 

Yes, It takes effort. I get that. I've been part of the effort to
provide remote participation for other groups, on a much larger scale
than anything FreeBSD can dream of.

My point, and I cannot emphasize this highly enough, is that your entire
mindset about this is all wrong. It needs to shift from We'll do this
on a small scale, for those who ask to We'll make providing robust
remote participation a top priority, built into the planning from day
1. It's as simple as that.

 The FreeBSD Foundation has also offered to fund new contributors who
 want to attend but are unable to afford to do so on their own.  In
 spite of the fact that I spent some effort encouraging people to
 apply for this, only one person actually did.

It isn't just the financial cost of attending the summit. Often (as in
my case) it's the lack of ability to take time away from personal, work,
and/or family commitments. For others it may be the difficulty of doing
the traveling at all. The fact that only 1 person took you up on this
offer (and IIRC there have been similar results in the past) pretty
clearly indicates that you're trying to solve the wrong problem.

Given that the foundation has money to spend here, why not put 2 and 2
together and have the foundation invest in providing remote
participation? That would benefit far more people, and almost certainly
at less cost.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
BTW, for those who'd like to get a flavor of what the IETF model looks
like, the Vancouver meeting is in process now:

https://datatracker.ietf.org/meeting/84/agenda.html

Feel free to join in as a lurker.

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 10:34, Doug Barton wrote:
 BTW, for those who'd like to get a flavor of what the IETF model looks
 like, the Vancouver meeting is in process now:
 
 https://datatracker.ietf.org/meeting/84/agenda.html
 
 Feel free to join in as a lurker.

Sorry, this agenda makes it easier to see the remote participation stuff:

https://tools.ietf.org/agenda/84/


-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 10:37, David Chisnall wrote:

 Thank you for volunteering to organise this.  It's good to see people with 
 both the motivation and experience required to do something well actively 
 contributing to the project.

Cheap copout. And quite sad, especially coming from a newly elected core
team member.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 10:40, Warner Losh wrote:
 One thing to remember about the IETF.  There's many vendors that devote 
 significant resources to the IETF.  While I was at Cisco, for example, I know 
 that we provided audio and video bridges to IEFT meetings to facilitate 
 remote attendance at the meetings.  Cisco dedicated several engineers to 
 ensure that the audio and video quality remained good during the meetings and 
 were able to use facilities cisco normally used for things like WebEx and 
 MeetingPlace.  With a global presence and infrastructure, they were able to 
 pull it off.  I'm not aware of similar resources within the project.

I'm not suggesting we need anything at the full WebEx
audio/video/presentation/chat level. And apparently the Foundation has
money to spend on dev summits. As I suggested in a previous message,
this would be a good long-term investment that would benefit a lot of
people, especially in comparison to the money set aside for travel
grants which is now going begging.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 05:39, John Baldwin wrote:
 I find this a bit ironic from you given that I've met you in person at
 USENIX ATC which is an order of magnitude more expensive than BSDCan (and
 in fact, one of the reasons the US-based BSDCon died and was effectively
 supplanted by BSDCan was that BSDCan is far cheaper).

Yep, back in 2004 when traveling to conferences was part of my job, and
before my daughter was born. My life now is quite different.

... not to mention that this thread isn't about me. It's about the
importance of remote participation to the FreeBSD community as a whole.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-02 Thread Doug Barton
On 08/02/2012 11:12, David Chisnall wrote:
 FreeBSD is a volunteer project.

Yeah, I get that. I've been around quite a bit longer than you have, in
case you didn't notice. :)

I understand what you're saying, it's going to take work to change this
mindset, and to provide these resources. If you read my posts on a
factual basis, I'm not suggesting that the dev summits provide remote
participation at the same level as groups like the IETF or ICANN do, and
your point (and Warner's) that these groups have significant financial
backing is well taken.

However, my point is that in spite of the fact that it's non-trivial,
the mindset on this topic needs to change if the dev summits are going
to continue to be significant focii of both work being done and
decisions being made (which of course, they are).

What I'm *not* doing is demanding that any one person, or even any one
group take responsibility for solving the whole problem on their own.
Unfortunately, due to my inability to actually attend these meetings, I
won't be able to provide the kind of hands-on assistance that I'd like
to be able to. However it sounds like there may be financial resources
available through the foundation, which would go a long way towards
making a solution possible.

The next step would be for people to agree that this is a problem that
*needs* to be solved, followed by willingness on the part of dev summit
organizers to support these efforts, which will hopefully lead to people
who are willing and interested to step up and actually provide
solutions. It's already been true in the past that various companies
have volunteered to do this. There is no reason to believe that it
wouldn't happen again if organizers are willing to be supportive.

What I'm hearing so far is defensiveness, and an attempt to focus the
discussion on me. Neither is helpful. :) Acknowledging that this is a
problem that needs to be solved does not imply that by not solving it
you personally have failed in some way. I apologize if anything I've
written so far has implied otherwise.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: On cooperative work [Was: Re: newbus' ivar's limitation..]

2012-08-01 Thread Doug Barton
On 8/1/2012 8:36 PM, Warner Losh wrote:
 I think this proves the point everybody has been saying: you are being 
 needlessly contrary and confrontational.

Actually if you take a step back and look at what Arnaud is saying
objectively, he's right. If anyone can attend the meeting by simply
getting an invitation from a committer, the only purpose the invitation
serves is to force the mere-mortal user to kiss someone's ring. That's
precisely the kind of elitist crap that I've been railing against for so
many years now.

OTOH, currently the dev summits generally take place with limited
resources, so it's not really possible to have everyone attend. And
(TMK) the invitation process is really  more like a restaurant with a
sign that says we reserve the right to refuse service to anyone.

But on the _other_ other hand, the problem of things being discussed
and/or decisions being taken exclusively at the dev summits, especially
BSDCAN, has gotten quite bad over the last several years. Even amongst
committers, the community has become divided between the haves who can
travel to the summit, and the have nots who can't. Note, I'm quite
sure that this statement will be met with howls of protest, from the
haves, that this isn't the case. Even if they were sincere, it's
incredibly easy for the people with the privileges to see their
privileged state as normal, and lose sight of how the world looks from
the cheap seats.

In spite of Kevin's concerns (and I don't know what working groups he's
been attending) the IETF model is really a good one to examine here. The
majority of the work gets done on the mailing lists, with working group
meetings serving as an opportunity for group discussion, presentations,
etc. The results of the meetings are then published to the mailing list
in the form of minutes, and the final decisions are made in public, on
the lists. Another incredibly important feature, the meetings are open
to remote participation in the sense that slide decks are published in
advance, the meeting audio is streamed live, and there are jabber rooms
for remote participants to interact with the people in the meeting.

I used to ask the PTB to provide *some* form of remote participation for
even a fraction of the events at the dev summit. I don't bother asking
anymore because year after year my requests were met with any of:
indifference, hostility, shrugged shoulders (that's a hard problem that
we can't solve), or embarrassment. Since if the right people around here
want something to happen, it happens; I finally came to the conclusion
that they didn't want remote participation to happen, so it won't.
That's a shame.

If the only large, open project you've ever participated in is FreeBSD,
what gets done around here feels normal to you. But don't be so quick
to dismiss the viewpoints of people who have experience in the wider world.

Doug

-- 

I am only one, but I am one.  I cannot do everything, but I can do
something.  And I will not let what I cannot do interfere with what
I can do.
-- Edward Everett Hale, (1822 - 1909)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: MPSAFE VFS -- List of upcoming actions

2012-07-18 Thread Doug Barton
On 07/17/2012 22:54, Gustau Pérez i Querol wrote:
 In fact filesystems not particulary specific and not tied our kernel
 would go to userspace; thinks like smbfs, nwfs, ntfs, ext2 o ext4 for
 example should be in userspace

A big -1 here.

The more native FS support we have the better off we are in terms of
both people migrating from other OS', and people who need to maintain
compatibility with other OS'. Personally I use both msdosfs and ext2fs
extensively for the latter purpose, and would not want to see either
removed.

Doug

-- 

Change is hard.



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: fetch(1) fails with https:// - Authentication error

2012-07-14 Thread Doug Barton
On 07/13/2012 21:21, Jan Beich wrote:
 It seems recent OpenSSL update broke fetch(1) for me.
 
   $ diff -u $SRC_BASE/crypto/openssl/apps/openssl.cnf /etc/ssl/openssl.cnf
   $ fetch https://foo/bar
   fetch: https://foo/bar: Authentication error
 
 Same error as with the patch for 1.0.0d from a year ago and
 same workaround - s/SSLv23_client_method/SSLv3_client_method/.

FWIW, I have a gcc world and I'm not seeing this problem with r238444:

fetch https://www.isc.org/
fetch: https://www.isc.org/: size of remote file is not known
fetch.out   33 kB  227 kBps




-- 

Change is hard.



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP] OpenSSL 1.0.1c merge in progress

2012-07-13 Thread Doug Barton
On 07/12/2012 05:03 PM, Jung-uk Kim wrote:
 FYI, OpenSSL 1.0.1c import is complete now.  Please let me know if you
 have any problem.

Sorry if I missed it, but did you bump OSVERSION for this change? If
not, could you? It would be helpful for dealing with ports stuff,
especially USE_OPENSSL.

Doug

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

2012-07-13 Thread Doug Barton
On 07/13/2012 05:26 AM, John Baldwin wrote:
 On Thursday, July 12, 2012 5:16:41 pm Doug Barton wrote:
 On 07/12/2012 02:11 PM, Craig Rodrigues wrote:
 You might want to view Baptiste's pkgng presentation at BSDCan:

 http://www.youtube.com/watch?v=4Hxq7AHZ27I

 Sure, the next time I have an hour to spare.

 I don't think what I'm asking for is unreasonable. One could even
 conclude that answering those 3 questions should have been a
 prerequisite for starting down this road in the first place.
 
 One could also assume that other people in the Project aren't morons and do 
 actually put thought into the things they do for starters

I certainly *want* to believe that. But considering the giant mess that
portmgr + Baptiste made of the changes to the OPTIONS framework, that
only touches a fraction of the ports, my willingness to have faith in
them to do it right is near zero.

Not to mention that I've been asking for a project plan for pkg since
long before it even hit the ports tree in beta. What I'm asking for
should have been done already considering that this change will affect
*every* port, and *every* user. So either it hasn't actually been done,
or the PTB are refusing to provide it.

Also, please keep in mind that I was criticized for *not* speaking up
about the OPTIONS changes, now I'm being criticized *for* speaking up
prior to pkg going live. In spite of the fact that I'm doing my best to
(repeatedly) be clear that I'm not against the project, I just want to
know more about it.

 Also, when other 
 people have taken time to explain an large decision because you are too lazy 
 to invest the time doesn't really help your case).

Um, I'm too lazy? I've read everything that's been written on pkg to
date. Have you? 90% of it is how to type stuff that doesn't address
what we need. The other 10% is so vague and general as to be useless as
a project plan.

You're an experienced project manager John. If someone who worked for
you came to you with a plan this vague (modern foo, decent bar), for
a critical system, how would you respond? (And yes, I realize that no
one around here works for me, that isn't my point at all.)

 In terms of the first feature (binary upgrades), the truth is that if you 
 have 
 more than 5 machines to manage, our current pkg tools completely suck.  There 
 is no automated upgrade mechanism.  If you want one you have to write your 
 own 
 set of infrastructure to do the right collection of pkg_delete/pkg_adds.  
 Certainly there is no support in the current package tools for doing batch 
 upgrades (i.e. upgrading from one completely package set to another).  pkgng 
 adds that feature, and I find it a must for supporting large installations of 
 machines that need automated management.

And as I wrote previously, I've been there and done that, so yes, I'm
interested in the feature. But I'd like to know more about the plans for
it so that those of us who *do* have experience in this topic can share
that, and we can avoid having to reinvent the wheel. Or worse, putting
out something half-assed that uses up a lot of developer cycles and
doesn't get the job done.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP] OpenSSL 1.0.1c merge in progress

2012-07-13 Thread Doug Barton
On 07/13/2012 08:52 AM, Jung-uk Kim wrote:
 On 2012-07-13 05:55:04 -0400, Doug Barton wrote:
 On 07/12/2012 05:03 PM, Jung-uk Kim wrote:
 FYI, OpenSSL 1.0.1c import is complete now.  Please let me know
 if you have any problem.
 
 Sorry if I missed it, but did you bump OSVERSION for this change?
 If not, could you? It would be helpful for dealing with ports
 stuff, especially USE_OPENSSL.
 
 Yes, it was bumped with the commit.

Thanks, and again, sorry I missed it.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

2012-07-12 Thread Doug Barton
I do not mean this e-mail to be in any way critical. I was told after
the new OPTIONS framework discussion that I should have asked questions
before the change, so I'm asking these questions now; in a genuine
attempt to get information.

On 07/12/2012 03:01 AM, Baptiste Daroussin wrote:

In the time that you have been working on this project I have asked
numerous times for you(pl.) to answer the following questions:

1. What are the goals for pkg?
2. Why can't the existing tools fulfill those goals?
3. How does pkg fulfill them?

You've put some of this in the various places where pkg is documented,
but I don't see any thorough treatment of these questions. You have some
of it below, which I'd like to see expanded on if you would be so kind. :)

 Why pkg?
 
 pkg_* tools have become hardly maintainable over the time,

I agree on this point, but the right solution (as some of us have been
saying for years) is to move the pkg_* tools into the ports tree. You
are correctly handling that by keeping pkg in the ports tree, I'm simply
pointing out that this isn't a reason we need to switch to pkg.

 it lacks lots of features most of people are expecting from a package manager:
   - binary upgrade

I'm not sure what you mean by this. We have the ability to create binary
packages now.

   - ability to search information about remote packages

This is a good feature, certainly. However there is no reason we can't
create a tool to do this, or add the functionality to an existing tool.

   - real reverse dependency tracking
   - tracking leaves

Can you expand on what these 2 mean?

What I'm looking for is compelling motivation to make this overwhelming
change to the ports infrastructure.


 Schedule
 
 
 The plan is to switch the ports tree to pkgng on CURRENT by default on July 
 25th
 No dates are planned yet for other branches.

Can you describe how this is going to be done? I assume with an
OSVERSION knob in bsd.port.mk?

 Note that there will be a NO_PKGNG knob for some time (undefined yet) for 
 people
 not will to switch on July 25th
 
 Please also note that some ports won't work with pkgng right now, because 
 pkgng
 is more strict than pkg_install on purpose.
 The major one is: nvidia drivers, because pkgng does not allow to overwrite a 
 file
 owned by another package, and we will not accept any hacks for that in pkgng.

IMO it would be a very large mistake to switch the default in any branch
until the problem with the nvidia drivers is sorted out. We have a lot
of users (myself included) who use this port, and by switching the
default there's going to be 1 of 2 outcomes for those users. Either they
will opt-out, which means you won't get the level of testing you're
looking for; or you'll break their existing ports installation. Neither
outcome is desirable.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

2012-07-12 Thread Doug Barton
On 07/12/2012 02:11 PM, Craig Rodrigues wrote:
 You might want to view Baptiste's pkgng presentation at BSDCan:
 
 http://www.youtube.com/watch?v=4Hxq7AHZ27I

Sure, the next time I have an hour to spare.

I don't think what I'm asking for is unreasonable. One could even
conclude that answering those 3 questions should have been a
prerequisite for starting down this road in the first place.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

2012-07-12 Thread Doug Barton
On 07/12/2012 03:02 PM, Baptiste Daroussin wrote:
 On Thu, Jul 12, 2012 at 11:48:41AM -0700, Doug Barton wrote:
 I do not mean this e-mail to be in any way critical. I was told after
 the new OPTIONS framework discussion that I should have asked questions
 before the change, so I'm asking these questions now; in a genuine
 attempt to get information.

 On 07/12/2012 03:01 AM, Baptiste Daroussin wrote:

 In the time that you have been working on this project I have asked
 numerous times for you(pl.) to answer the following questions:

 1. What are the goals for pkg?
 
 The why part of this mail should reply this question, no?

Well clearly not, because if it did I wouldn't keep asking the same
questions over and over again. :)

 Anyway the goal is to have a decent package manager, providing modern 
 features:
 repositories, decent dependency tracking, decent reverse dependency tracking,
 managing upgrade correctly (I'll explain this more later), provide a decent
 library for third party tools (desktop integration via PackageKit for example)

I don't know what decent means. I don't know what modern features
means (beyond the buzzwords that you've included).

 Providing easy package management for enterprise

Having set up package management systems for enterprises before, *this*
is actually a big-picture goal that I have a lot of sympathy for. But
again, what's missing is *details* about here is what large enterprises
need to make things work for them, here's why the existing tools don't
meet those needs, and here is how pkg does meet them.

 (who never got problems
 managing packages on a large set of freebsd servers, and how complicated it is
 on FreeBSD to have automated reliable puppet,salt,chef,cfengine like tools)
 One of the proof of this problem is how fast people integrated pkgng in those
 tools.

This gets to the heart of my biggest fear regarding this whole project.
It's new, it's shiny, and it looks like forward progress is being made.
Thus, it's attracted a lot of attention, input, time, etc. Heck, it may
even BE forward progress, but I don't know how to measure that because I
don't know what the goals of the project are. Thus, my fear is that
without *details* about what the project is, and what it's trying to
accomplish, we're going to put an exponentially larger volume of work
into the transition and end up no closer to the goal of having a mature
package management system.

And just to be clear, I am *not* saying that pkg sucks or that it's
bad, or wrong, or anything else. I'm saying that I don't know how to
evaluate it, because you haven't given us a criteria by which to measure
it.

So what's the problem? We *desperately* need a better system for ports
and packages. We only have so many person-hours we can devote to making
that happen. If we spend all of them on pkg, and it ends up not helping
us enough, we've burnt out our volunteers for no good reason.

 2. Why can't the existing tools fulfill those goals?
 
 The existing tools can't fulfill those goals, because they are hardly
 maintainable, the code hasn't change much since when they were written, lot of
 people have tried over the year to improve them, but all of them gave up. The
 design of the tools, (I mean the code) is really imho not adapted to be
 improved, I spent a lot of time trying to work on it before starting a 
 complete
 new project.

This paragraph really frightens me.

 For example they do not know what is a version, they do not know what are the
 reverse dependencies except through this ugly hack that is +REQUIRED_BY, the
 database is pretty fragile: who never got the package corrupted: empty @pkgdep
 line for example.

So these 2 are a lot closer to what I'd like to see ... *details* about
what isn't working now. I would tend to disagree with you that
+REQUIRED_BY is an ugly hack, it's no uglier than any of the other text
file based dependency tracking we have. But thank you for giving us more
information.

So taking your last example, how does pkg handle the situation where the
user wants to forcibly delete a package that is depended on by another
package?

 3. How does pkg fulfill them?

 You've put some of this in the various places where pkg is documented,
 but I don't see any thorough treatment of these questions. You have some
 of it below, which I'd like to see expanded on if you would be so kind. :)
 
 It is true that, I'm not very good at documenting in general, and even more in
 english, hopefully, the documentation is improving a lot recently, there is 
 the
 for usage:
 http://wiki.freebsd.org/PkgPrimer
 and for all other things:
 http://wiki.freebsd.org/pkgng
 
 Lot of native english speakers have joined the project and help with
 documentation, if you find someting missing, do not hesitate to had the 
 section
 in the apropriate wiki page, I often have a look at them, and try to fill all
 the blank section to answer user questions.

I'm not looking for how to. I've explained to you numerous times

Re: Java and NIO?

2012-07-08 Thread Doug Barton
On 07/08/2012 19:33, George Neville-Neil wrote:
 A followup.  zookeeper is now ported to Freebsd (/usr/ports/devel/zookeeper)

George, did you see the PR and the followup from me regarding the port?

-- 

This .signature sanitized for your protection


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Java and NIO?

2012-07-08 Thread Doug Barton
On 07/08/2012 20:01, George Neville-Neil wrote:
 
 On Jul 8, 2012, at 22:39 , Doug Barton wrote:
 
 On 07/08/2012 19:33, George Neville-Neil wrote:
 A followup.  zookeeper is now ported to Freebsd (/usr/ports/devel/zookeeper)

 George, did you see the PR and the followup from me regarding the port?

 
 I got a mail from jgh@ but only today figured out what the PR was.

Are you not getting your g...@freebsd.org mail?

 I'll look at the patches from him tomorrow.

I copied the text from my message below for your convenience.

 http://www.freebsd.org/cgi/query-pr.cgi?pr=169693

Furthermore the rc.d script is a mess, and should not have been
committed like it was (numerous missing bits, bad format, set_rcvar,
hard-coded /usr/local, no REQUIRE, no KEYWORD: shutdown, etc.). Please
read
http://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/rc-scripts.html
and then ask in freebsd-rc@ if you have any additional questions. Sorry
to be so blunt, but I'm really, really tired of repeating the same stuff
over and over again, and this script is really a mess.

Also, don't install the script in do-install, see the web page above
(and/or the PR) for USE_RC_SUBR.

And FYI, there is no need to have the function in that script. You could
use (for example) start_cmd=$command start just as well. Not to
mention that the function you have should be using $1 as the argument to
$command, not $rc_arg. Reasons why left as an exercise for the reader ...

Doug


-- 

This .signature sanitized for your protection


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Why NOT using FreeBSD? Re: ports/169581: editors/libreoffice:

2012-07-04 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 07/03/2012 11:34, Baptiste Daroussin wrote:
 On Tue, Jul 03, 2012 at 10:59:03AM +0200, Hartmann, O. wrote:
 On 07/02/12 08:09, Sayetsky Anton wrote:
 I will test libreoffice build on 8.3-RELEASE today or tomorrow.
 I have both gstreamer and boost installed now.



 We use FreeBSD 9.0STABLE and FreeBSD 10.0-CURRENT (both amd64).

 devel/boost-lib gets reeled in now by editors/libreoffice by default, so
 it doesn't need to be installed explicitely.

 I saw a patch flushed in yesterday, submitted by bapt@. This patch also
 installs LLVM/CLANG from the ports - with ASSERTS deactivated.

 I have on both systems, FreeBSD 9 and 10, LLVM/CLANG 3.1 as the standard
 backend compiler, I guess this version has the suspected ASSERTS activated.

 Why another LLVM port? We already have LLVM/CLANG in the base system (9
 and 19). If the ASSERTS proble is the cause for breaks reported on the
 list and elsewhere on the net, why isn't the maintainer still stuck on
 the old version?

 I just managed it to install the prior version on broken systems and was
 really lucky having LibreOffice working again. But the other day I was
 bothered by the next non-working version and now I have lots of
 notebooks remaining with NO LibreOffice on FBSD 9-STABLE.

 This is not what I expect from quality securing! It is simply a mess and
 definitely another reason and point for the thread Why NOT using FreeBSD.

 sure libreoffice is so easy to port...
 
 /me officially gives up with that libreoffice port, open for new volunteers 

If you don't have time to work on the port, then don't, that's not a
problem. But throwing a hissy fit here doesn't help at all. It's a plain
fact that a working office suite is a basic requirement for most desktop
users. If _we_ can't provide that (note, not you, personally, we, as a
project), it's a valid reason for users to avoid FreeBSD for desktop use.

If we're ever going to make progress in this area we have to be willing
to examine and absorb facts; and then act accordingly.

Doug

- -- 

This .signature sanitized for your protection


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBCAAGBQJP9LQnAAoJEFzGhvEaGryEYk4H/0TcxjVnax0xgrt4/3WMvwWU
t/GNQ7Fws3EJSrZN2vB3LSnmwRv7UbkmotXLzirS+f/SmwREUH0677DV3EFlxuxx
WvXvtYexu4XHWOeODZi5m8crbNUM94HLnwTfo2gecMah+eL6M46EoBAfCJT4NtUD
1AYIsZTpiEqEvLHJCNj+Mwt0YiH3XxAdRhhfSMolKBm7B6lqOsEA5cEnA2QTWBWI
bDeUB8hZuW1q6O5U60xdTZMjDQNGroVg6nuCtTihXj7/DWKR41Wgzezh14qFKs7m
Hki/eRGzQA3DTLKuf51PY+FO7epBeWC5YCNxe52ASqU+pKdUYpfS3vCw3BmbqPg=
=4Mp6
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Why NOT using FreeBSD? Re: ports/169581: editors/libreoffice:

2012-07-04 Thread Doug Barton
On 07/04/2012 15:02, Pedro Giffuni wrote:
 The thing, as I see it, is that people have to understand
 this is a volunteer project and if people don't do things
 by themselves they really can't demand someone else to
 do it for them.

Of course. But that's totally different from, I don't use FreeBSD
because it doesn't offer $feature, which is important to me.

Don't forget the original purpose of this thread. :)

-- 

This .signature sanitized for your protection


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-30 Thread Doug Barton
Sounds great, thanks again!

Doug

On 06/29/2012 02:20 PM, Oleg Moskalenko wrote:
 Doug,
 
 --nthreads option corresponds to --parallel option of NGNU 
 and it will be renamed. The other four proprietary options 
 will be marked as non-portable in the man page. 
 
 After nthreads==parallel renaming, NBSD will support all 
 NGNU options.
 
 Thank you for the suggestion.
 Oleg
 
 -Original Message-
 From: Doug Barton [mailto:do...@freebsd.org]
 Sent: Friday, June 29, 2012 2:02 PM
 To: Oleg Moskalenko
 Cc: FreeBSD Current
 Subject: Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

 On 06/29/2012 01:50 PM, Oleg Moskalenko wrote:
 5) NBSD adds several of its own new proprietary options:

   --mergesort
   --qsort
   --heapsort
   --radixsort
   --nthreads=... (multi-threaded build only)

 Oleg,

 First, thank you very much for providing both the performance numbers,
 and the breakdown in the differences in command line options.
 Everything
 looks great, my only concern is the above.

 Are there similar/identical options in NGNU that correspond to the
 options above? If so, I would be hesitant to add new names for them
 because it hurts portability between platforms. If these are totally
 new
 features then my assumption is that you have clearly marked them as
 non-portable in the man page?

 Once again, I really appreciate you addressing my concerns, and your
 hard work on this project.

 Doug
 


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-29 Thread Doug Barton
On 06/29/2012 01:50 PM, Oleg Moskalenko wrote:
 5) NBSD adds several of its own new proprietary options:
 
   --mergesort
   --qsort
   --heapsort
   --radixsort
   --nthreads=... (multi-threaded build only)

Oleg,

First, thank you very much for providing both the performance numbers,
and the breakdown in the differences in command line options. Everything
looks great, my only concern is the above.

Are there similar/identical options in NGNU that correspond to the
options above? If so, I would be hesitant to add new names for them
because it hurts portability between platforms. If these are totally new
features then my assumption is that you have clearly marked them as
non-portable in the man page?

Once again, I really appreciate you addressing my concerns, and your
hard work on this project.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
On 06/26/2012 11:04 PM, Gabor Kovesdan wrote:
 Hi Folks,
 
 as I announced before, the default sort in -CURRENT has been changed
 to BSD sort. 

Has this been performance tested vs. the old one? If so, where are the
results?

 Since the import, the reported minor bugs have been
 fixed and BSD sort has passed the portbuild test. If you encounter any
 problems or incompatibility with the old GNU version, please report.

Has this been thoroughly regression-tested against the old version,
option by option?

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
On 06/26/2012 11:48 PM, Oleg Moskalenko wrote:
 
 
 -Original Message- From: Doug Barton
 [mailto:do...@freebsd.org] Sent: Tuesday, June 26, 2012 11:18 PM 
 To: Gabor Kovesdan Cc: FreeBSD Current; Oleg Moskalenko Subject:
 Re: [HEADS-UP] BSD sort is the default sort in -CURRENT
 
 On 06/26/2012 11:04 PM, Gabor Kovesdan wrote:
 Hi Folks,
 
 as I announced before, the default sort in -CURRENT has been
 changed to BSD sort.
 
 Has this been performance tested vs. the old one? If so, where are
 the results?
 
 Of course it was performance-tested.

Great, can you post the results somewhere? I understand what you're
saying below that there are situations where worse performance may need
explanation, but it would be helpful if we had the data to look at.

 As this is a totally different
 program with different algorithms, it has totally different
 performance profile on different tests, comparing to the old sort. In
 the default compilation mode (single thread sort) the performance is
 on the same level as the old sort (sometimes faster, sometimes
 slower). The new sort is often significantly faster in numeric sort
 tests. In experimental multi-threading mode, the new sort is much
 faster than the old sort on multi-CPU systems.

This sounds encouraging. Is there a knob to enable the threaded build?

 The sort speed comparison is not actually fair because the old sort
 cuts some corners and has a number of bugs.

Understood, but the existing sort is what we're changing away from, so
that's what we have to test against. What we don't want is a situation
where we are switching to the new sort by default without understanding
what the tradeoffs are. (IOW, we don't want a repeat of the situation
with grep.)

 The concrete figures do not have much sense because you change the
 sort file and you get a totally different performance ratio.

I'm assuming that you'd run the performance tests on various different
input files, and report the different results.

 Has this been thoroughly regression-tested against the old
 version, option by option?
 
 Of course we have the regression tests. We have an overnight test
 that runs through probably 17 millions various sort option
 combinations. 

This sounds great, but ...

 But we actually had to compare the new sort against a
 fresh GNU sort implementation (version 8.15), because the old BSD GNU
 sort is very buggy and testing against the old GNU sort has no
 sense.

... this not so much. The existing sort is what people have now, and
what they rely on, particularly for scripts. Comparing apples to oranges
doesn't help us understand how things are going to be different with the
new version.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
On 06/27/2012 02:09 AM, Oleg Moskalenko wrote:
 Doug, I'll post some performance figures, probably tomorrow.

That's great, thanks.

 But I do not agree with you that we have to reproduce the old sort bugs.
 It makes no sense and I am not going to do that. Absolutely not.

That isn't what I said. What I asked is for you to *test* the existing
sort vs. the new one, and to report where the behavior is different.
That's a very basic part of any sort of replace a core utility project
such as this one.

 If some old scripts are relying on buggy behavior 
 (and I hope they are not) then the old scripts must be fixed. Period.

With respect, that's not your decision (or mine for that matter). We
first need the data, then as a project we decide how many old bugs we
want to be compatible with, if any.

 The system cannot grow replicating the old bugs.

And the project cannot grow if we lose users due to gratuitous
differences in core utilities.

 All system scripts that I've seen are using pretty basic sort features.

The system scripts are only a tiny fraction of how FreeBSD users use sort.

 In the basic
 area, the old sort and the new sort are 100% compatible. The 
 incompatibilities are 
 in more complex areas (numeric sorts and unusual key-based sorts).

So here's one to add to your regression test. I use the following to
sort IPv4 addresses in a list:

sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4

When used with GNU sort that will sort a list of IPv4 addresses into a
humanly-recognizable numeric order. Please ensure that this works the
same way with the new sort.

 I am actually tested the new sort against the old GNU sort. There are some 
 incompatibilities. 
 All of them are due to the bugs of the old GNU sort.

Please list all of those explicitly.

 The new BSD sort program
 is compatible with the new GNU sort, a much cleaner program than the old GNU 
 sort.

That's good, but not really relevant to the users of what we have in the
base now.

I realize that these questions may seem discouraging, but they need to
be answered. It would have been nice if Gabor had posted a we think
we're ready to make the new sort the default, any last concerns?
message, but deal with where we are at and move forward.

thanks,

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
On 06/27/2012 03:02 AM, Daniel Gerzo wrote:
 On 27.06.2012 10:43, Doug Barton wrote:
 On 06/27/2012 02:09 AM, Oleg Moskalenko wrote:
 Doug, I'll post some performance figures, probably tomorrow.

 That's great, thanks.

 But I do not agree with you that we have to reproduce the old sort bugs.
 It makes no sense and I am not going to do that. Absolutely not.

 That isn't what I said. What I asked is for you to *test* the existing
 sort vs. the new one, and to report where the behavior is different.
 That's a very basic part of any sort of replace a core utility project
 such as this one.
 
 [ snip ]
 
 Doug, are you implying that if we were about to import a new version of
 GNU sort, you would be asking for the same data?

If the compatibility with the existing version were so dramatically
different as Oleg claims, then yes, that would be a basic element of the
replacement project.

 I believe we do not
 make this kind of work with any vendor code that is being updated in the
 base;

Au contraire, we frequently avoid updating the old versions of things we
have in the base precisely because they are not bug-for-bug compatible
with existing behaviors that we count on.

 I do not really understand why should Oleg or anyone else do this
 work when the bsdsort is compatible with a recent version of GNU sort.

The first question is, where is it not compatible with the existing sort
that's already in the base. The second question is, how well does it
perform vs. the existing sort.

I think those are perfectly reasonable questions to ask, and frankly
they should have been answered before the switch was made. We already
went through this with BSD grep, I hope we can avoid a repeat.

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
On 06/27/2012 07:30 AM, Pedro Giffuni wrote:
 
 
 --- Mer 27/6/12, Doug Barton do...@freebsd.org ha scritto:
 ...

 I believe we do not
 make this kind of work with any vendor code that is
 being updated in the
 base;

 Au contraire, we frequently avoid updating the old versions
 of things we have in the base precisely because they are
 not bug-for-bug compatible with existing behaviors that we
 count on.

 
 
 Really?? I guess you are speaking for bind,

Nope.

 because the idea
 behind updating and piece of software is precisely shaking
 bugs.

Nope.

 I would think only the maintainer of the package has the
 authority to make any request in the lines of being
 bug-for-bug compatible

You have a seriously wrong idea of maintainer. The community owns the
software, it's up to the community to decide how it should work.
Historically we have looked at the maintainer as the person who
volunteers to take care of code, not the person who has the exclusive
lock on it.

 and in the case of GNU sort and
 GNU grep they are both unmaintained and replacements
 are welcome.

Actually both are maintained, it's just that we don't want to import the
new GNU versions. And yes, having BSD versions of these core tools is a
nice goal, but it's not one we should pursue for its own sake.

 Please let's stop being an obstacle towards people
 bringing real progress to FreeBSD!

In the case of grep, there were a fairly large number of people who
agreed that a BSD grep with orders of magnitude worse performance than
the previous version was not something we, as a project, were willing to
stomach. Sufficiently such that the default was switched back.

So can we please stop pretending that it's me who's the problem, and
start looking at these things rationally?

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [HEADS-UP] BSD sort is the default sort in -CURRENT

2012-06-27 Thread Doug Barton
I officially withdraw from the discussion. I hope it all works out well.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: mountd starts to early when exporting fs marked as late (patch included)

2012-06-24 Thread Doug Barton
On 06/24/2012 16:07, Steven Hartland wrote:
 We added some new mount points recently and on reboot they failed to
 come up after investigating we found that mountd runs too early in
 the boot process to be able export filesystems that are marked as
 late in /etc/fstab.
 
 Our fix was simply to mark mountd as requiring mountlate and all
 was well. I can't think of any reason why this would cause problems
 so would like the patch to be considered.
 
 The PR has all the details and the patch.
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=169373

I re-routed that PR to freebsd-rc@.


-- 

This .signature sanitized for your protection


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


PORTS_MODULES

2012-06-17 Thread Doug Barton
Howdy,

This is an FYI to let people know about a really nice feature for those
that have ports installed which include kernel modules. You can place a
list in /etc/src.conf like this:

PORTS_MODULES=  emulators/virtualbox-ose-kmod sysutils/fusefs-kmod
x11/nvidia-driver

which will cause those modules to be built and installed with all the
proper matching stuff at the same time as buildkernel and installkernel.

This feature has existed for a while, but has had issues. Thanks to a
team effort it's a lot more robust now, and ready for prime time (in
HEAD, and the -STABLE branches for now, soon to be in 9.1-RELEASE).

Enjoy,

Doug

-- 

This .signature sanitized for your protection

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: PORTS_MODULES fix

2012-06-17 Thread Doug Barton
On 06/09/2012 16:51, Doug Barton wrote:
 Ok, never mind the last one ... this patch I've actually tested. :)

Committed to HEAD and MFC'ed. Thanks everyone for the feedback and help.

Doug

-- 

This .signature sanitized for your protection


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


PORTS_MODULES fix

2012-06-09 Thread Doug Barton
I have recently tried the PORTS_MODULES knob, and found a problem. The
ports tree searches for some dependencies by finding a binary in PATH,
and that fails since by default /usr/local/ isn't there. The attached
patch fixes that problem.

It would be more robust to use PREFIX there instead of /usr/local
explicitly, but I'm not sure how to unravel the mk maze to get that
value. If anyone has a suggestion for that, I'd be happy to include it.

Any objections to making this change?

Doug

-- 

This .signature sanitized for your protection
Index: kern.post.mk
===
--- kern.post.mk(revision 236818)
+++ kern.post.mk(working copy)
@@ -38,7 +38,7 @@
 
 # Handle out of tree ports 
 .if !defined(NO_MODULES)  defined(PORTS_MODULES)
-PORTSMODULESENV=SYSDIR=${SYSDIR}
+PORTSMODULESENV=SYSDIR=${SYSDIR} PATH=${PATH}:/usr/local/bin:/usr/local/sbin
 .for __target in all install reinstall clean
 ${__target}: ports-${__target}
 ports-${__target}:
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: PORTS_MODULES fix

2012-06-09 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160

On 06/09/2012 10:40, Matthew Seaman wrote:
 On 09/06/2012 18:26, Chris Rees wrote:
 On 9 June 2012 18:15, Doug Barton do...@freebsd.org wrote:
 I have recently tried the PORTS_MODULES knob, and found a
 problem. The ports tree searches for some dependencies by
 finding a binary in PATH, and that fails since by default
 /usr/local/ isn't there. The attached patch fixes that
 problem.
 
 It would be more robust to use PREFIX there instead of
 /usr/local explicitly, but I'm not sure how to unravel the mk
 maze to get that value. If anyone has a suggestion for that,
 I'd be happy to include it.
 
 As you mention, PREFIX is only defined in ports/Mk, and it'd 
 definitely be undesirable to be including any of those files :)
 
 The most robust (but unpleasant) solution would be one of the
 following:
 
 PREFIX?=/usr/local PORTSMODULESENV=SYSDIR=${SYSDIR}
 PATH=${PATH}:${PREFIX}/bin:${PREFIX}/sbin
 
 or the equivalent (and perhaps cleaner, not leaving PREFIX
 defined)
 
 .if !defined(PREFIX) PORTSMODULESENV=SYSDIR=${SYSDIR}
 PATH=${PATH}:/usr/local/bin:/usr/local/sbin .else 
 PORTSMODULESENV=SYSDIR=${SYSDIR}
 PATH=${PATH}:${PREFIX}/bin:${PREFIX}/sbin .endif
 
 Both of these will respect make.conf's setting of PREFIX.
 
 
 Shouldn't you be looking for LOCALBASE rather than PREFIX in this
 context?

Both good points. New and improved attached.

Doug

- -- 

This .signature sanitized for your protection
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)

iEYEAREDAAYFAk/TkJkACgkQyIakK9Wy8Pv36QCeMyL31kOIIwYX/6rCoKFqhufW
unsAnjoUG31Cr5TB0GZ1YPv4+zGuz+XY
=mM5z
-END PGP SIGNATURE-
Index: kern.post.mk
===
--- kern.post.mk(revision 236818)
+++ kern.post.mk(working copy)
@@ -38,7 +38,9 @@
 
 # Handle out of tree ports 
 .if !defined(NO_MODULES)  defined(PORTS_MODULES)
-PORTSMODULESENV=SYSDIR=${SYSDIR}
+# The ports tree looks for dependencies in PATH, so we need to accommodate
+LOCALBASE?=/usr/local
+PORTSMODULESENV=SYSDIR=${SYSDIR} 
PATH=${PATH}:${LOCALBASE}/bin:${LOCALBASE}/sbin
 .for __target in all install reinstall clean
 ${__target}: ports-${__target}
 ports-${__target}:
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: PORTS_MODULES fix

2012-06-09 Thread Doug Barton
Ok, after reading your PR and discussion on IRC I have the following 
which incorporates all the suggestions so far. I haven't actually tested 
this yet, but if people agree that this is the right direction to go I 
will before I commit it of course.


Doug

--

It's always a long day; 86400 doesn't fit into a short.

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/
Index: kern.post.mk
===
--- kern.post.mk(revision 236818)
+++ kern.post.mk(working copy)
@@ -36,9 +36,30 @@
 .endif
 .endfor
 
-# Handle out of tree ports 
+# Handle ports (as defined by the user) that build kernel modules
 .if !defined(NO_MODULES)  defined(PORTS_MODULES)
-PORTSMODULESENV=SYSDIR=${SYSDIR}
+#
+# The ports tree needs some environment variables defined to match the new 
kernel
+#
+# Ports search for some dependencies in PATH, so add the location of the 
installed files
+LOCALBASE?=/usr/local
+# SRC_BASE is how the ports tree refers to the location of the base source 
files
+.if !defined(SRC_BASE)
+SRC_BASE!= realpath ${SYSDIR:H}/
+.endif
+# OSVERSION is used by some ports to determine build options
+.if !defined(OSRELDATE)
+# Definition copied from src/Makefile.inc1
+OSRELDATE!=awk '/^\#define[[:space:]]*__FreeBSD_version/ { print $$3 }' \
+   ${MAKEOBJDIRPREFIX}${SRC_BASE}/include/osreldate.h
+.endif
+# Keep the related ports builds in the obj directory so that they are only 
rebuilt once per kernel build
+WRKDIRPREFIX?= ${MAKEOBJDIRPREFIX}${SRC_BASE}/sys/${KERNCONF}/${__i}
+PORTSMODULESENV=\
+   PATH=   ${PATH}:${LOCALBASE}/bin:${LOCALBASE}/sbin \
+   SRC_BASE=   ${SRC_BASE} \
+   OSVERSION=  ${OSRELDATE} \
+   WRKDIRPREFIX=   ${WRKDIRPREFIX}
 .for __target in all install reinstall clean
 ${__target}: ports-${__target}
 ports-${__target}:
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: PORTS_MODULES fix

2012-06-09 Thread Doug Barton
Ok, never mind the last one ... this patch I've actually tested. :)

Doug

-- 

This .signature sanitized for your protection


Index: kern.post.mk
===
--- kern.post.mk(revision 236818)
+++ kern.post.mk(working copy)
@@ -36,9 +36,30 @@
 .endif
 .endfor
 
-# Handle out of tree ports 
+# Handle ports (as defined by the user) that build kernel modules
 .if !defined(NO_MODULES)  defined(PORTS_MODULES)
-PORTSMODULESENV=SYSDIR=${SYSDIR}
+#
+# The ports tree needs some environment variables defined to match the new 
kernel
+#
+# Ports search for some dependencies in PATH, so add the location of the 
installed files
+LOCALBASE?=/usr/local
+# SRC_BASE is how the ports tree refers to the location of the base source 
files
+.if !defined(SRC_BASE)
+SRC_BASE!= realpath ${SYSDIR:H}/
+.endif
+# OSVERSION is used by some ports to determine build options
+.if !defined(OSRELDATE)
+# Definition copied from src/Makefile.inc1
+OSRELDATE!=awk '/^\#define[[:space:]]*__FreeBSD_version/ { print $$3 }' \
+   ${MAKEOBJDIRPREFIX}${SRC_BASE}/include/osreldate.h
+.endif
+# Keep the related ports builds in the obj directory so that they are only 
rebuilt once per kernel build
+WRKDIRPREFIX?= ${MAKEOBJDIRPREFIX}${SRC_BASE}/sys/${KERNCONF}
+PORTSMODULESENV=\
+   PATH=${PATH}:${LOCALBASE}/bin:${LOCALBASE}/sbin \
+   SRC_BASE=${SRC_BASE} \
+   OSVERSION=${OSRELDATE} \
+   WRKDIRPREFIX=${WRKDIRPREFIX}
 .for __target in all install reinstall clean
 ${__target}: ports-${__target}
 ports-${__target}:
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: OptionalObsoleteFiles.inc completeness

2012-05-28 Thread Doug Barton
On 05/27/2012 07:05, Dmitry Marakasov wrote:
 Hi!
 
 I'm running a little pet project of improving completeness of
 tools/build/mk/OptionalObsoleteFiles.inc file and thus delete-old*
 targets with regard to all possible WITHOUT_* knobs.
 
 E.g. when WITHOUT_foo is defined in src.conf, make delete-old should
 remove related files completely, to make a system look exactly like
 it's world was installed with that knob set.
 
 First of all, an automatic script to check for leftovers after
 delete-old for all possible knobs is available from [2]. Feel free
 to run in on different architectures and FreeBSD branches; I'm
 currently running it on amd64. I also think that it should be run
 during preparation of each FreeBSD release.
 
 There are some questions I'd like to discuss.
 
 1) named config file var/named/etc/namedb/named.conf was intentionally
 left out from OptionalObsoleteFiles.inc, so I did the same for other
 configs which may be changed by users.

That's one reason to omit it. Another is that the bind ports use the
same set of config files.


-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: OptionalObsoleteFiles.inc completeness

2012-05-28 Thread Doug Barton
On 05/28/2012 12:52, Dmitry Marakasov wrote:
 * Doug Barton (do...@freebsd.org) wrote:
 
 I'm running a little pet project of improving completeness of
 tools/build/mk/OptionalObsoleteFiles.inc file and thus delete-old*
 targets with regard to all possible WITHOUT_* knobs.

 E.g. when WITHOUT_foo is defined in src.conf, make delete-old should
 remove related files completely, to make a system look exactly like
 it's world was installed with that knob set.

 First of all, an automatic script to check for leftovers after
 delete-old for all possible knobs is available from [2]. Feel free
 to run in on different architectures and FreeBSD branches; I'm
 currently running it on amd64. I also think that it should be run
 during preparation of each FreeBSD release.

 There are some questions I'd like to discuss.

 1) named config file var/named/etc/namedb/named.conf was intentionally
 left out from OptionalObsoleteFiles.inc, so I did the same for other
 configs which may be changed by users.

 That's one reason to omit it. Another is that the bind ports use the
 same set of config files.
 
 But putting in under OLD_CONFIGS+= would still be ok?

I have no idea, I don't use the Obsolete stuff, don't like it, and have
said since the beginning that it's the totally wrong approach to this
issue. The numerous problems we've had with it ever since it was
introduced seem to bear me out. :)

My point is, until such time as we remove BIND from the base (or I add
the config files as a port) I do not want the named config files removed
from a user's system.

Thanks,

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: OptionalObsoleteFiles.inc completeness

2012-05-28 Thread Doug Barton
On 05/28/2012 13:23, Alexander Leidinger wrote:
 On Mon, 28 May 2012 12:59:17 -0700 Doug Barton do...@freebsd.org
 wrote:
 
 this issue. The numerous problems we've had with it ever since it was
 introduced seem to bear me out. :)
 
 Can you list them? A missing obsolete file doesn't count.

It doesn't catch things it needs to
It catches things it shouldn't
The current incarnation is painfully slow (so I've heard)
... and the biggest problem ...
It needs to be updated manually

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: OptionalObsoleteFiles.inc completeness

2012-05-28 Thread Doug Barton
On 5/28/2012 3:05 PM, Dmitry Marakasov wrote:
 * Doug Barton (do...@freebsd.org) wrote:
 
 this issue. The numerous problems we've had with it ever since it was
 introduced seem to bear me out. :)

 Can you list them? A missing obsolete file doesn't count.

 It doesn't catch things it needs to
 It catches things it shouldn't
 The current incarnation is painfully slow (so I've heard)
  ... and the biggest problem ...
 It needs to be updated manually
 
 Pretty true. Still I'd like to fix what we have now, than not to
 have a useful feature.

A question was raised about named.conf, so I answered it. A question was
raised about why I don't like/use Obsolete, so I answered it. At no
point did I say don't work on Obsolete.

That said, my concern about this is the same as my concern about effort
being placed into other less-than-desirable solutions.

1. The effort could be better placed elsewhere
2. The fact that $SOMEONE is working on $SOMETHING gives people a warm
fuzzy feeling that has a tendency to diminish the urgency towards
putting real fixes to real problems.

So once again, I'm not saying don't do it. But since someone actually
asked for my opinion ... :)

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: WARNING: su(1) broken in head

2012-05-27 Thread Doug Barton
On 05/26/2012 16:36, Dag-Erling Smørgrav wrote:
 Dag-Erling Smørgrav d...@des.no writes:
 probably due to an issue in the latest openpam; sudo is not affected
 
 should be fixed now.

Confirmed, thanks. :)


-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Latest PAM seems to break su

2012-05-26 Thread Doug Barton
su
Segmentation fault: 11

no core is produced.

Currently broken: r236118
Previous  r235567

sudo works.

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Daily, weekly, security scripts....

2012-05-24 Thread Doug Barton
On 05/24/2012 03:49 AM, Willem Jan Withagen wrote:
 [I looked for a better list to drop this on, but other that freebsd-rc
 nothing seems close.]

freebsd-rc@ is not appropriate for discussing periodic, as the 2 are
totally unrelated.

At this time there is no dedicated maintainer for periodic, so if you
find behavior that you don't like, and you've thoroughly exhausted the
available configuration options, your only recourse is to submit a patch.

hth,

Doug
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [review request] usr.sbin/service - make showing files configurable

2012-05-17 Thread Doug Barton
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/14/2012 06:35, Bryan Drewery wrote:
 
 
 On 5/13/2012 6:15 PM, Doug Barton wrote:
 On 5/12/2012 8:23 PM, Bryan Drewery wrote:
 Hi,
 
 I found service(8) to be inconsistent that it listed files with
 `service -e`, but plain services with `service -l`
 
 That behavior is by design.
 
 
 
 Could you please elaborate on the design decision?

For services that are enabled (IOW, a tiny subset of the overall
number) I thought it was useful to indicate to the user where those
services come from. The -l option dumps everything in the directories,
even if it's not a service. Users interested in differentiating
/etc/rc.d from $local_startup can use ls.

 I did of course look in base for uses of service -e and service
 -l, before considering this patch. The only case I can find is in a
 cshrc example, which my patch does not affect.

That's not relevant, as you cannot possibly know what other uses
service(1) is being put to. Also, it's bad form to change the default
output of a tool (and/or the semantics of its command line options)
years after its introduction.

 I had expected service -e to behave like service -l, so I could
 for example, put it into a loop and check all services, using the
 service(8) script itself.
 
 for service_name in `service -e`; do service status $service_name
 || service start $service_name; done

for service in `service -e` ; do
service ${##*/service} status || service ${##*/service} start
done

(Note, your syntax for the service command is wrong above.)


hth,

Doug

- -- 

This .signature sanitized for your protection
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBCAAGBQJPtW+MAAoJEFzGhvEaGryEpokH/RbWnJZN/RCQzidxoIbAx0+5
nAEX33e0Iazfqs/km7uFP8T/4SD2b0pOmr3dNBaKHqnpz005ACzhTcWD111ik/d2
ypRKdzh+vlq+Y9bDB4PozMjnalZrhkAUIinUIDDH6xMW46fIbN2bXPqz8AIe1Umo
a8LaHW59ARJf197o7iyWNOYOcF6+S3haaSzu8UXL5MTDtKBpn5XY5Eg6ppc/ZD9J
Mzaq1k7baCrGqCSsyZusmCv7WWDcOw7tOspUKzoNMm+wBMf7MrQyPUQsaA9vfGXZ
cB39Byryvi9Rhbz/ACjgw44ZRVUcjWJaxkFVc5WwkLbCDTv4tny5q2KpIAHfhPk=
=ykfV
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [review request] usr.sbin/service - make showing files configurable

2012-05-17 Thread Doug Barton
On 05/17/2012 02:51 PM, Bryan Drewery wrote:
 Yeah it's what I get for mashing a pseudo example up and not testing it!

S'ok, I screwed up ${service##*/} in mine. :)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: svn commit: r235275 - projects user

2012-05-14 Thread Doug Barton
On 5/14/2012 12:02 AM, Ulrich Spoerlein wrote:

 Uli - amazed that a change to a document that apparently no one is
 reading can cause such a fuss.

... which is the point that several of us tried to make, and which you
seem to have ignored. The problem with committers not reading the
documentation (such as it is) is not properly solved by changing the
documentation.

What I'm objecting to is your pointless deck-chair rearranging rather
than addressing the real problem.

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [review request] usr.sbin/service - make showing files configurable

2012-05-13 Thread Doug Barton
On 5/12/2012 8:23 PM, Bryan Drewery wrote:
 Hi,
 
 I found service(8) to be inconsistent that it listed files with `service
 -e`, but plain services with `service -l`

That behavior is by design.

Thanks for your interest,

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: svn commit: r235275 - projects user

2012-05-13 Thread Doug Barton
When you proposed these changes not only did I not see a consensus for
you to move forward, I saw a non-zero number of people push back.

Why did you proceed?

Doug


On 5/11/2012 9:08 AM, Ulrich Spoerlein wrote:
 Author: uqs
 Date: Fri May 11 16:08:51 2012
 New Revision: 235275
 URL: http://svn.freebsd.org/changeset/base/235275
 
 Log:
   Update guidelines on user/ and projects/
   
   The goal is to make it clearer where future branches should be created. A
   consistent layout under projects/ would also help with conversions to other
   VCSes that do not follow the everything-is-a-subdir dogma.
   
   TL;DR
   - If it's a branch of head that you want to merge back - projects/
   - If it's something else - user/your-login/ (e.g. portmaster, stress2, 
 etc.)
   
   Discussed on:   developers
   Silence by: peter
 
 Modified:
   user/GUIDELINES.txt
 
 Changes in other areas also in this revision:
 Modified:
   projects/GUIDELINES.txt
 
 Modified: user/GUIDELINES.txt
 ==
 --- user/GUIDELINES.txt   Fri May 11 16:04:55 2012(r235274)
 +++ user/GUIDELINES.txt   Fri May 11 16:08:51 2012(r235275)
 @@ -1,16 +1,9 @@
  $FreeBSD$
  
 -Golden rules:
 -Rule #1:  TAKE IT EASY! DON'T RUSH AND MAKE A MESS!  ASK IF NEEDED!
 -Rule #2:  See rule #1, repeat as needed
 +Guidelines for what can go in /user
 +---
  
 -Peril sensitive sunglasses advisory:
 -This is in flux.  Expect refinement.  Expect typos.
 -
 -Guidelines for what can go in /user and /projects
 --
 -
 -First of all, eveyrbody needs to keep in mind that this repository is
 +First of all, everybody needs to keep in mind that this repository is
  replicated as a unit.  Anything that goes into the repository uses project
  and volunteer resources.  Once something goes in, it essentially never comes
  out.  Therefore, these are not dumping grounds to put random junk in the
 @@ -19,82 +12,39 @@ tree that we have to mirror forever.
  General guidelines:
  
  * Should be relevant to FreeBSD.
 -* Should be at least concievably of interest to somebody else.
 -* Should be in a format that is suitable to merge into the base tree.
 +* Should be at least conceivably of interest to somebody else.
  * Should be something that is worth people's time to read commit mail for.
  * Write decent commit messages!
  
 +The difference between /projects and /user wasn't very clear in the past.
 +Going forward /projects is reserved for branches of FreeBSD itself for 
 possible
 +re-integration into /head. Branches shall not be nested into e.g.
 +/projects/foo/stable8, instead /projects/foo_stable8 shall be used.
  
 -The difference between /projects and /user is mostly one of intentions.
 -
 -If some WIP is intended to be committed to the main src tree, then it
 -should go in /projects/$name/*.  We encourage people to subscribe to projects
 -commit messages.  The reason is that WIP in projects can be expected to hit
 -the base tree at some point.
 -
 -If some WIP is more of an experiment or speculative, that might not ever be
 -merged, then it goes in /user/$username/$name/*.  We don't encourage
 -people to subscribe to user commit messages.
 -
 -If it is something unrelated to the src tree, it should probably go 
 elsewhere.
 -There will be a separate repostory made available for such things, whether it
 -be a special version of mysql or xorg or gcc or whatever.
 -
 +/user can be used for tools and software tightly related to FreeBSD, but 
 which
 +is not a copy/branch of FreeBSD itself.
  
  Layout:
 -Since this is for WIP that can concievably be merged, there is an argument
 -that can be made that teaching the pre-commit scripts to sanity check WIP
 -as it goes, rather than having a mammoth fixup being needed prior to merging.
 -
 -For that to work, the layout has to be predictable.  eg: a branch of
 -head/sys/* for a project called ia65 should be  /projects/ia65/sys/*.
 -An experimental X11-aware verison of bin/ls/* in a user directory for jdoe
 -would be /user/jdoe/x11-ls/bin/ls/*.
 -
  
 -Creation and merging:
 -
 -Merging is in flux.  The procedure as understood right now:
 -
 -Assume projects/ia65/sys.   $BASE=svn+ssh://svn.freebsd.org/base
 +Since this is for auxiliary/experimental projects that might not be branched
 +from head, an argument can be made that we teach the pre-commit scripts to
 +sanity check WIP as it goes in.
  
  Initial creation:
 -  $ svn cp --parents $BASE/head/sys $BASE/projects/ia65/sys
 +  Assume user/pho/stress2.   BASE=svn+ssh://svn.freebsd.org/base
  
 -Then check it out:
 -  $ svn co $BASE/projects/ia65
 +  $ svn mkdir $BASE/user/pho/stress2
  
 -To integrate changes from head into your branch:
 -  $ cd ia65/sys ; svn update; svn status | read output!  Should preferably 
 be clean.
 -  (you may prefer to do merges in a second, clean checkout. It will be 
 easier!)
 -  $ 

Re: panic, seems related to r234386

2012-05-10 Thread Doug Barton
On 05/07/2012 23:14, Sergey Kandaurov wrote:
  Finally, should my next step be to advance to the latest current + your
  patch and see how I go from there?
 Yep, so that patches will be tested before they go to head.

For the record, I upgraded to r235151 + the removal of those 2 locks and
haven't had any problems yet. I was going to do one more update today
and report back, but you beat me to it. :)

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Binary packages for LibreOffice 3.5 or 3.4

2012-05-09 Thread Doug Barton
Has anyone answered the original question? Are there going to be
packages for libreoffice? If not, why not?

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-07 Thread Doug Barton
On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

...

 Please try this patch.
 
 Index: fs/ext2fs/ext2_vfsops.c
 ===
 --- fs/ext2fs/ext2_vfsops.c (revision 235108)
 +++ fs/ext2fs/ext2_vfsops.c (working copy)
 @@ -830,7 +830,6 @@
 /*
  * Write back each (modified) inode.
  */
 -   MNT_ILOCK(mp);
  loop:
 MNT_VNODE_FOREACH_ALL(vp, mp, mvp) {
 if (vp-v_type == VNON) {
 

Didn't help, sorry. I put 234385 through some pretty heavy load
yesterday, and everything was fine. As soon as I move up to 234386, the
panic triggered again. So I cleaned everything up, applied your patch,
built a kernel from scratch, and rebooted. It was Ok for a few seconds
after boot, then panic'ed again, I think in a different place, but I'm
not sure because subsequent attempts to fsck the file systems caused new
panics which overwrote the old ones before they could be saved.

I'd like to see this changed backed out until the proponents can
thoroughly regression test all of the file systems that it affects.

FWIW, the thing that seems to be triggering the panic is that I have the
following setup:

/dev/ad0s2a on / (ufs, local)
/dev/ad3s2 on /home (ext2fs, local)

/etc/namedb is a symlink to /home/named/etc/namedb. When I booted 234386
named failed to start because the symlink was corrupted. When I
recreated the symlink and then started named, it panic'ed.

hth,

Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic, seems related to r234386

2012-05-07 Thread Doug Barton
On 05/07/2012 13:11, Mateusz Guzik wrote:
 On Mon, May 07, 2012 at 12:28:41PM -0700, Doug Barton wrote:
 On 05/06/2012 15:19, Sergey Kandaurov wrote:
 On 7 May 2012 01:54, Doug Barton do...@freebsd.org wrote:
 I got this with today's current, previous (working) kernel is r232719.

 panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
 @ /frontier/svn/head/sys/kern/vfs_subr.c:4595

 ...

 Please try this patch.

Ok, so far so good. Again, thanks for the quick response. I'm
stress-testing my ext2fs partitions a bit atm, and everything seems Ok.
I'll let you know if I run into any problems.

So my next question is, does removing those locks present any risks?
Should they be replaced by different locks, or were they just safety
belts to start with?

Finally, should my next step be to advance to the latest current + your
patch and see how I go from there?


Doug

-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


panic, seems related to r234386

2012-05-06 Thread Doug Barton
I got this with today's current, previous (working) kernel is r232719.

panic: _mtx_lock_sleep: recursed on non-recursive mutex struct mount mtx
@ /frontier/svn/head/sys/kern/vfs_subr.c:4595

FWIW I'm using ufs2, ext2fs, and msdosfs on this system. The panic
occurred right after loading the linux kernel module, but I'm not 100%
sure that's related. Full core.txt.0 is in my home directory on freefall.

#0  doadump (textdump=1) at /frontier/svn/head/sys/kern/kern_shutdown.c:268
268 if (textdump  textdump_pending) {
(kgdb) #0  doadump (textdump=1) at
/frontier/svn/head/sys/kern/kern_shutdown.c:268
#1  0x80482999 in kern_reboot (howto=260)
at /frontier/svn/head/sys/kern/kern_shutdown.c:454
#2  0x804823ca in panic (fmt=0x0)
at /frontier/svn/head/sys/kern/kern_shutdown.c:642
#3  0x80471c25 in _mtx_lock_sleep (m=Variable m is not available.
)
at /frontier/svn/head/sys/kern/kern_mutex.c:363
#4  0x80471cb0 in _mtx_lock_flags (m=Variable m is not available.
)
at /frontier/svn/head/sys/kern/kern_mutex.c:212
#5  0x80517353 in __mnt_vnode_first_all (mvp=0xff819e9f0a58,
mp=0xfe00051dc310) at /frontier/svn/head/sys/kern/vfs_subr.c:4595
#6  0x8042bdd5 in ext2_sync (mp=0xfe00051dc310, waitfor=2)
at /frontier/svn/head/sys/fs/ext2fs/ext2_vfsops.c:835
#7  0x80521446 in sys_sync (td=Variable td is not available.
)
at /frontier/svn/head/sys/kern/vfs_syscalls.c:150
#8  0x806b1d52 in amd64_syscall (td=0xfe0005149480, traced=0)
at subr_syscall.c:135
#9  0x8069d697 in Xfast_syscall ()
at /frontier/svn/head/sys/amd64/amd64/exception.S:387
#10 0x0008008b061c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)


-- 

This .signature sanitized for your protection
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


  1   2   3   4   5   6   7   8   9   >