Re: [CFT] packaging the base system with pkg(8)

[Lyndon Nerenberg]

On 2016-04-18 8:17 PM, Alfred Perlstein wrote:

Can someone on the "too many packages" campaign here explain to me how
having too fine a granularity stops you from making macro packages
containing packages?

Because honestly I can't see how having granularity hurts at all when if
someone wanted to make it less granular all they would have to do is
make some meta-packages.


It's the *I have to put it back together* part that's annoying.  I 
didn't break something that has worked, forever.  It shouldn't be 
incumbent on me to un-break someone else's work.


Now if the system ships with each-file-in-a-package, fine.  Just give me 
gross subsets that make my life as a sysadmin liveable.   E.g., base 
POSIX functionality should be a 'group' package.  And I would hope, the 
default installation package.  I would go for the argument that, e.g., 
the dev stuff (cc, yacc, lex) could be split off, but at least include 
the headers that match what's in /lib and /usr/lib, in a compiler 
agnostic set.  Since the point of packages is to allow for selections of 
optional software.


--lyndon

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Alfred Perlstein]

Maybe what the "too many packages" folks need to do is write some code 
to hide that it's so many packages.


:)

I think the rule of two feet should be applied here.

What we have is people that have worked quite hard to bring us something 
that we can easily work with, and on the other hand some folks that want 
something they consider even better.  Personally I can't see how having 
the system less granular is better, since having it MORE granular is 
actually harder work.


Can someone on the "too many packages" campaign here explain to me how 
having too fine a granularity stops you from making macro packages 
containing packages?


Because honestly I can't see how having granularity hurts at all when if 
someone wanted to make it less granular all they would have to do is 
make some meta-packages.


-Alfred

On 4/18/16 7:23 PM, Lyndon Nerenberg wrote:

On 2016-04-18 7:01 PM, Roger Marquis wrote:

Can you explain what would be accomplished by testing all or even a
fraction of the possible permutations of base package combinations?  We
don't do that for ports.


The ports tree isn't a mandatory part of the system. And by definition 
it could not be tested that way, since it offers so many alternative 
implementations of specific functionality.



Other operating systems don't do that for
their base packages.


I'm pretty sure Solaris had some fairly hard-core regression tests to 
ensure basic system functionality wouldn't be compromised by 'oddball' 
selections of packages offered up at install time.


> Honestly, some of us are wondering what exactly is
> behind some of these concerns regarding base packages.

The concern is from all of us UNIX dinosaurs who predate the 
fine-grained packaging environment, which just worked, and who now rip 
our (little remaining) hair out due to unsolvable package dependency 
loops in the Linux machines we are forced to administer in order to 
pay rent.  For me, as a sysadmin, I derive a negative benefit from 
this optimization.


I guess what I'm really asking is: where is the peer reviewed research 
that shows this actually improves things for the not-1% of FreeBSD users?


--lyndon

P.S.  Don't turn this into a pissing match.  I really want to know how 
this is of net benefit to everyone.  But I don't want hyperbole.  I 
have looked at a lot of, e.g., USENIX and ACM, bibliographies and 
papers for justification for this, and I can't find it.  It would 
really help (me, at least) if someone could take a moment to point me 
at demonstrable evidence of the benefits of this model.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Lyndon Nerenberg]

On 2016-04-18 7:01 PM, Roger Marquis wrote:

Can you explain what would be accomplished by testing all or even a
fraction of the possible permutations of base package combinations?  We
don't do that for ports.


The ports tree isn't a mandatory part of the system. And by definition 
it could not be tested that way, since it offers so many alternative 
implementations of specific functionality.



Other operating systems don't do that for
their base packages.


I'm pretty sure Solaris had some fairly hard-core regression tests to 
ensure basic system functionality wouldn't be compromised by 'oddball' 
selections of packages offered up at install time.


> Honestly, some of us are wondering what exactly is
> behind some of these concerns regarding base packages.

The concern is from all of us UNIX dinosaurs who predate the 
fine-grained packaging environment, which just worked, and who now rip 
our (little remaining) hair out due to unsolvable package dependency 
loops in the Linux machines we are forced to administer in order to pay 
rent.  For me, as a sysadmin, I derive a negative benefit from this 
optimization.


I guess what I'm really asking is: where is the peer reviewed research 
that shows this actually improves things for the not-1% of FreeBSD users?


--lyndon

P.S.  Don't turn this into a pissing match.  I really want to know how 
this is of net benefit to everyone.  But I don't want hyperbole.  I have 
looked at a lot of, e.g., USENIX and ACM, bibliographies and papers for 
justification for this, and I can't find it.  It would really help (me, 
at least) if someone could take a moment to point me at demonstrable 
evidence of the benefits of this model.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Roger Marquis]

Lyndon Nerenberg wrote:
There aren't enough seconds in the universe to test all the viable 
combinations for one single release.


Can you explain what would be accomplished by testing all or even a
fraction of the possible permutations of base package combinations?  We
don't do that for ports.  Other operating systems don't do that for
their base packages.  Honestly, some of us are wondering what exactly is
behind some of these concerns regarding base packages.

Roger Marquis
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Lyndon Nerenberg]

On 2016-04-18 5:09 PM, Nathan Whitehorn wrote:

I'm not so sure about these statements. Maintaining groups of packages
can be easier, but it can be also be harder. The goal is to find the
right level. And I haven't seen a case where an 800-packages level of
granularity is helpful.


Not to mention regression testing. The number of combinations of 
installed packages is going to be choose(1, 800) + chose(2, 800) + ... + 
choose(800, 800).  And while some of those combinations will be 
non-nonsensical, many(!) won't. There aren't enough seconds in the 
universe to test all the viable combinations for one single release.


If fact, I would suggest a good metric for package granularity be based 
on the set of combinations that *can* be tested in a realistic timeframe 
for each release.


--lyndon

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Nathan Whitehorn]

On 04/18/16 14:29, Alfred Perlstein wrote:
Guys please stop arguing about the number of packages.  The high 
granularity is VERY useful!


Managing large groups of small packages is much easier than just 
having large packages.


I'm not so sure about these statements. Maintaining groups of packages 
can be easier, but it can be also be harder. The goal is to find the 
right level. And I haven't seen a case where an 800-packages level of 
granularity is helpful. Maybe you can suggest one? I can, however, see 
the reverse case, where it leads, in addition to system administration 
hassle, to a balkanization of the base system, the predictability of 
which is one of FreeBSD's greatest assets.


More granularity than we have is good (having the ability to strip debug 
information, profiling, sendmail, the toolchain, etc.) and it is going 
to be very nice to have that. But I can name of order 10 packages where 
that makes sense. I think we will learn to regret 800. Clearly you can 
have too few (one giant package called "FreeBSD 11" -- although this is 
a model that works for many organizations) and you can have too many 
(every file is its own package -- no one does this). I'm not sure where 
the optimum is, but this seems way too far in the direction of the latter.


All this can be done by meta-packages which depend on larger package 
groups.


Later pkg can be augmented to "remove packages not explicitly 
installed" which would remove leaf packages.


Example: you installed "base-debug" which pulls in let's say 50 small 
packages, later you want all of those removed, you can do something 
like:  "pkg delete --leafs base-debug" which should delete 
"base-debug" and any dangling packages it pulled in not required by 
other pkgs.


But what benefit do you get from the sub-packages? Let's say you have a 
single package that is the debug files for one library. Why would you 
ever care about that specific package? I can see why you might want all 
the debug files as a separate thing, but this seems way too fine-grained 
to be useful.


Put another way, the base system for FreeBSD 11 (I grabbed a powerpc64 
distfile because I had it handy) has 11765 files in it, neglecting 
symlinks. Split into 755 packages, the average package then has just 15 
files in it. You are rapidly approaching pkg info resembling ls -R / at 
that point and managing the system at the individual file level, which 
totally defeats the concept of packaging things. If you remove files in 
var, share, etc, and /usr/include from that list (which include things 
like all the timezone files, which are hopefully packaged together, or 
all the man pages), this average is 2.1 files per package.




Huge thanks to the team that implemented this!


Yes, of course. Please do not interpret my comments about the level of 
discretization of packages as in any way reflecting the project in 
general, which is broader than this issue. We need better tools than tar 
and patch to manage the system; using pkg is a huge step forward.

-Nathan



thanks.
-Alfred

On 4/18/16 1:07 PM, Lev Serebryakov wrote:

On 18.04.2016 22:40, Glen Barber wrote:


This granularity allows easy removal of things that may not be wanted
(such as *-debug*, *-profile*, etc.) on systems with little 
storage.  On

one of my testing systems, I removed the tests packages and all debug
and profiling, and the number of base system packages is 383.

  IMHO, granularity like "all base debug", "all base profile" is enough
for this. Really, I hardly could imagine why I will need only 1 debug or
profile package, say, for csh. On resource-constrained systems NanoBSD
is much better anyway (for example, my typical NanoBSD installation is
37MB base system, 12MB /boot and 10 packages), and on developer system
where you need profiled libraries it is Ok to install all of them and
don't think about 100 packages for them.

  Idea of "Roles" from old FreeBSD installers looks much better. Again,
here are some "contrib" software which have one-to-one replacements in
ports, like sendmail, ssh/sshd, ntpd, but split all other
FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static libraries.
Yes, lib32 on 64 bit system.

   It seems that it is ideological ("holy war") discussion more than
technical one...





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Tue, Apr 19, 2016 at 12:43:08AM +0300, Slawa Olhovchenkov wrote:

> On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:
> 
> > On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> > > 
> > > I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> > > packages?! WHY?! What are reasons and goals to split base in such
> > > enormous number of packages?
> > 
> > Just a guess, having done the same thing myself:  it means that updates can 
> > be
> > more targeted.
> 
> Realy?
> I am now try to upgrade test VM from CFT version (from /project) to
> current.
> Upgraded only 407 packages.
> All -lib32- packages don't upgraded.
> OK, how I do debug this?
> What system state I am have now?
> What hapened?
> What I need to do?

Oh. Not only lib32.

root@pkg-test:/usr/src # pkg info | grep kernel
FreeBSD-kernel-generic-debug-11.0.s20160418211012 FreeBSD GENERIC kernel -debug
FreeBSD-kernel-generic-release-11.0.s20160304190332 FreeBSD GENERIC kernel 
release

Realy, I am can't see in list of 756 files what is updated and what not.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 10:30:48PM +0200, Rainer Duffner wrote:

> 
> > Am 18.04.2016 um 22:07 schrieb Lev Serebryakov :
> > 
> > On 18.04.2016 22:40, Glen Barber wrote:
> > 
> >> This granularity allows easy removal of things that may not be wanted
> >> (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
> >> one of my testing systems, I removed the tests packages and all debug
> >> and profiling, and the number of base system packages is 383.
> > IMHO, granularity like "all base debug", "all base profile" is enough
> > for this. Really, I hardly could imagine why I will need only 1 debug or
> > profile package, say, for csh. On resource-constrained systems NanoBSD
> > is much better anyway (for example, my typical NanoBSD installation is
> > 37MB base system, 12MB /boot and 10 packages), and on developer system
> > where you need profiled libraries it is Ok to install all of them and
> > don't think about 100 packages for them.
> > 
> > Idea of "Roles" from old FreeBSD installers looks much better. Again,
> > here are some "contrib" software which have one-to-one replacements in
> > ports, like sendmail, ssh/sshd, ntpd, but split all other
> > FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static libraries.
> > Yes, lib32 on 64 bit system.
> > 
> >  It seems that it is ideological ("holy war") discussion more than
> > technical one...
> 
> 
> 
> From the discussion, I believe it’s primarily driven by the need/desire to 
> have small packages to make updates easier on the mirror-servers.
> 
> I’m really not sure (yet), which is worse: the current system that pulls down 
> some 14k small files for a system-upgrade - or a system where the base-system 
> is split into almost 800 packages.

Lesser of files is best. 800 packages is better then 14k. 10 packages
is better then 800.

> freebsd-update is „only" unreliable if
>  - you go through a proxy with authentication
>  - that proxy doesn’t do http-pipelining (or does it bad/is broken is this 
> respect) (certain version of Sophos UTM for example…)

freebsd-update broken on server side.
freebsd-update not relaible on client side.
freebsd-update to long even on SSD and 10Gbit connectivity.
freebsd-update to long to prepare (4x times longer of one compiling)

> AFAIK.
> 
> As for the packages: I wouldn’t mind „fatter“ packages. I’d mirror them 
> locally anyway (I hope this is possible - AFAIK, the freebsd-update files are 
> not supposed to be mirrored), and I don’t have a thousand servers to pull 
> them down all at once anyway (working on that ;-)).
> 
> I’m pretty sure the impact on the current FreeBSD delivery infrastructure 
> would be quite substantial, if updates came in 60MB chunks - esp. if there 
> was some sort of auto-update mechanism in place.
> Fast-forward to the future where a lot (millions?) more embedded devices are 
> based on FreeBSD and pull updates from the FreeBSD infrastructure.

Hundred of millions iPad and iPhones got update in near-gigabyte chunks.

> Or if the container hype-train reached FreeBSD and people started to 
> containerize everything, resulting in even more base-package update downloads.
> 
> So, I can see both sides. Neither I’m really satisfied with.
> 
> I hope a way is found to manage these number of packages without losing 
> sanity and that a normal pkg info doesn’t list them.
> And that pkg upgrade doesn’t upgrade base-packages.
> 
> 
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 12:21:28PM -0700, Nathan Whitehorn wrote:

> 
> 
> On 04/18/16 12:14, Glen Barber wrote:
> > On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:
> >> On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> >>> I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> >>> packages?! WHY?! What are reasons and goals to split base in such
> >>> enormous number of packages?
> >> Just a guess, having done the same thing myself:  it means that updates 
> >> can be
> >> more targeted.
> >>
> > This is exactly the reason, which has been answered numerous times.
> >
> > Glen
> >
> 
> That's a good reason -- and a very nice outcome of having base system 
> packages -- but I worry that it may be going too far. The most granular 
> updates would be if every file were its own package, which is obviously 

Allowing to have one file in multiple packages may be solution:

base-11.0.txz have /usr/libexec/sendmail/sendmail
base-11.0-p1.txz depends on base-11.0.txz and contains just single
/usr/libexec/sendmail/sendmail -- security update.

This packages must be installed together and presents
/usr/libexec/sendmail/sendmail in both must not be error.

> crazy, and so there is some middle ground. Needing to grab a whole new 
> base.txz is probably too much (60 MB), but splitting that into even 6 or 
> 7 pieces moves the updates to replacements with typical size (a few MB) 
> that are no larger than typical package updates for ports.
> -Nathan
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:

> On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> > 
> > I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> > packages?! WHY?! What are reasons and goals to split base in such
> > enormous number of packages?
> 
> Just a guess, having done the same thing myself:  it means that updates can be
> more targeted.

Realy?
I am now try to upgrade test VM from CFT version (from /project) to
current.
Upgraded only 407 packages.
All -lib32- packages don't upgraded.
OK, how I do debug this?
What system state I am have now?
What hapened?
What I need to do?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Alfred Perlstein]

Guys please stop arguing about the number of packages.  The high 
granularity is VERY useful!


Managing large groups of small packages is much easier than just having 
large packages.


All this can be done by meta-packages which depend on larger package groups.

Later pkg can be augmented to "remove packages not explicitly installed" 
which would remove leaf packages.


Example: you installed "base-debug" which pulls in let's say 50 small 
packages, later you want all of those removed, you can do something 
like:  "pkg delete --leafs base-debug" which should delete "base-debug" 
and any dangling packages it pulled in not required by other pkgs.


Huge thanks to the team that implemented this!

thanks.
-Alfred

On 4/18/16 1:07 PM, Lev Serebryakov wrote:

On 18.04.2016 22:40, Glen Barber wrote:


This granularity allows easy removal of things that may not be wanted
(such as *-debug*, *-profile*, etc.) on systems with little storage.  On
one of my testing systems, I removed the tests packages and all debug
and profiling, and the number of base system packages is 383.

  IMHO, granularity like "all base debug", "all base profile" is enough
for this. Really, I hardly could imagine why I will need only 1 debug or
profile package, say, for csh. On resource-constrained systems NanoBSD
is much better anyway (for example, my typical NanoBSD installation is
37MB base system, 12MB /boot and 10 packages), and on developer system
where you need profiled libraries it is Ok to install all of them and
don't think about 100 packages for them.

  Idea of "Roles" from old FreeBSD installers looks much better. Again,
here are some "contrib" software which have one-to-one replacements in
ports, like sendmail, ssh/sshd, ntpd, but split all other
FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static libraries.
Yes, lib32 on 64 bit system.

   It seems that it is ideological ("holy war") discussion more than
technical one...



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Lev Serebryakov]

On 18.04.2016 23:30, Rainer Duffner wrote:

> From the discussion, I believe it’s primarily driven by the need/desire to 
> have small packages to make updates easier on the mirror-servers.
  It is bad driver. Mirror servers are hardware. And this enormous
number of packages cause problems for people (system administrators and
operations). I've read almost whole thread and looks like people, who
raise voice against such fine-grained splitting, are mostly ops of many
servers, including legacy ones (and 11 will be legacy in 5 years, too!).
Not developers, not code contributors, but "end-users" of server OS.

> I hope a way is found to manage these number of packages without
> losing sanity and that a normal pkg info doesn’t list them.
> And that pkg upgrade doesn’t upgrade base-packages.
 And there are my (and not only my) worries, too!

-- 
// Lev Serebryakov



signature.asc
Description: OpenPGP digital signature

Re: [CFT] packaging the base system with pkg(8)

[Rainer Duffner]

> Am 18.04.2016 um 22:07 schrieb Lev Serebryakov :
> 
> On 18.04.2016 22:40, Glen Barber wrote:
> 
>> This granularity allows easy removal of things that may not be wanted
>> (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
>> one of my testing systems, I removed the tests packages and all debug
>> and profiling, and the number of base system packages is 383.
> IMHO, granularity like "all base debug", "all base profile" is enough
> for this. Really, I hardly could imagine why I will need only 1 debug or
> profile package, say, for csh. On resource-constrained systems NanoBSD
> is much better anyway (for example, my typical NanoBSD installation is
> 37MB base system, 12MB /boot and 10 packages), and on developer system
> where you need profiled libraries it is Ok to install all of them and
> don't think about 100 packages for them.
> 
> Idea of "Roles" from old FreeBSD installers looks much better. Again,
> here are some "contrib" software which have one-to-one replacements in
> ports, like sendmail, ssh/sshd, ntpd, but split all other
> FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static libraries.
> Yes, lib32 on 64 bit system.
> 
>  It seems that it is ideological ("holy war") discussion more than
> technical one...



From the discussion, I believe it’s primarily driven by the need/desire to have 
small packages to make updates easier on the mirror-servers.

I’m really not sure (yet), which is worse: the current system that pulls down 
some 14k small files for a system-upgrade - or a system where the base-system 
is split into almost 800 packages.

freebsd-update is „only" unreliable if
 - you go through a proxy with authentication
 - that proxy doesn’t do http-pipelining (or does it bad/is broken is this 
respect) (certain version of Sophos UTM for example…)

AFAIK.

As for the packages: I wouldn’t mind „fatter“ packages. I’d mirror them locally 
anyway (I hope this is possible - AFAIK, the freebsd-update files are not 
supposed to be mirrored), and I don’t have a thousand servers to pull them down 
all at once anyway (working on that ;-)).

I’m pretty sure the impact on the current FreeBSD delivery infrastructure would 
be quite substantial, if updates came in 60MB chunks - esp. if there was some 
sort of auto-update mechanism in place.
Fast-forward to the future where a lot (millions?) more embedded devices are 
based on FreeBSD and pull updates from the FreeBSD infrastructure.
Or if the container hype-train reached FreeBSD and people started to 
containerize everything, resulting in even more base-package update downloads.

So, I can see both sides. Neither I’m really satisfied with.

I hope a way is found to manage these number of packages without losing sanity 
and that a normal pkg info doesn’t list them.
And that pkg upgrade doesn’t upgrade base-packages.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Lev Serebryakov]

On 18.04.2016 22:40, Glen Barber wrote:

> This granularity allows easy removal of things that may not be wanted
> (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
> one of my testing systems, I removed the tests packages and all debug
> and profiling, and the number of base system packages is 383.
 IMHO, granularity like "all base debug", "all base profile" is enough
for this. Really, I hardly could imagine why I will need only 1 debug or
profile package, say, for csh. On resource-constrained systems NanoBSD
is much better anyway (for example, my typical NanoBSD installation is
37MB base system, 12MB /boot and 10 packages), and on developer system
where you need profiled libraries it is Ok to install all of them and
don't think about 100 packages for them.

 Idea of "Roles" from old FreeBSD installers looks much better. Again,
here are some "contrib" software which have one-to-one replacements in
ports, like sendmail, ssh/sshd, ntpd, but split all other
FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static libraries.
Yes, lib32 on 64 bit system.

  It seems that it is ideological ("holy war") discussion more than
technical one...

-- 
// Lev Serebryakov



signature.asc
Description: OpenPGP digital signature

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 08:05:05PM +, Glen Barber wrote:

> On Mon, Apr 18, 2016 at 11:02:12PM +0300, Slawa Olhovchenkov wrote:
> > > This granularity allows easy removal of things that may not be wanted
> > > (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
> > > one of my testing systems, I removed the tests packages and all debug
> > > and profiling, and the number of base system packages is 383.
> > 
> > Easy select from list of 1k items?!
> > You kidding?
> > 
> 
> See the pkg(8) '-g' option.

"You have problem. You decide to use rgexps. Your have two problems" (C)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 11.0-RELEASE pkg base & base.txz file

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 03:42:20PM -0400, Ernie Luzar wrote:

> Slawa Olhovchenkov wrote:
> > On Mon, Apr 18, 2016 at 05:27:09PM +0300, Slawa Olhovchenkov wrote:
> > 
> >> On Mon, Apr 18, 2016 at 04:16:01PM +0200, Baptiste Daroussin wrote:
> >>
> >>> On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:
>  On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:
> 
> > On 04/18/16 10:00, Ernie Luzar wrote:
> >> 11.0 will have pkg base, thats ok, but what does than mean for the
> >> base.txz file?
> >>
> >> It it going to stay as part of FBSD install?
> >>
> >> I have many scripts for creating jails which depend on the base.txz 
> >> file.
> > It's even easier now:
> >
> > # mkdir -p /usr/jails/new
> > # pkg -r /usr/jails/new install -r base -g '*'
>  And where /etc now?
> >>> What do you mean?
> >> At Jan 27 no package containing files from distributeworld.
> > 
> > At Mar 02
> > 
> >> r298107 change this?
> >>
> 
> You have NOT answered the original question, what's going to happen to 
> the base.txz file in 11.0?

Spliting to 800 packages, as I understund.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 07:40:10PM +, Glen Barber wrote:

> On Mon, Apr 18, 2016 at 12:21:28PM -0700, Nathan Whitehorn wrote:
> > 
> > 
> > On 04/18/16 12:14, Glen Barber wrote:
> > >On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:
> > >>On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> > >>>I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> > >>>packages?! WHY?! What are reasons and goals to split base in such
> > >>>enormous number of packages?
> > >>Just a guess, having done the same thing myself:  it means that updates 
> > >>can be
> > >>more targeted.
> > >>
> > >This is exactly the reason, which has been answered numerous times.
> > >
> > >Glen
> > >
> > 
> > That's a good reason -- and a very nice outcome of having base system
> > packages -- but I worry that it may be going too far. The most granular
> > updates would be if every file were its own package, which is obviously
> > crazy, and so there is some middle ground. Needing to grab a whole new
> > base.txz is probably too much (60 MB), but splitting that into even 6 or 7
> > pieces moves the updates to replacements with typical size (a few MB) that
> > are no larger than typical package updates for ports.
> 
> This granularity allows easy removal of things that may not be wanted
> (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
> one of my testing systems, I removed the tests packages and all debug
> and profiling, and the number of base system packages is 383.

Easy select from list of 1k items?!
You kidding?


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Glen Barber]

On Mon, Apr 18, 2016 at 11:02:12PM +0300, Slawa Olhovchenkov wrote:
> > This granularity allows easy removal of things that may not be wanted
> > (such as *-debug*, *-profile*, etc.) on systems with little storage.  On
> > one of my testing systems, I removed the tests packages and all debug
> > and profiling, and the number of base system packages is 383.
> 
> Easy select from list of 1k items?!
> You kidding?
> 

See the pkg(8) '-g' option.

Glen



signature.asc
Description: PGP signature

Re: 11.0-RELEASE pkg base & base.txz file

[Ernie Luzar]

Slawa Olhovchenkov wrote:

On Mon, Apr 18, 2016 at 05:27:09PM +0300, Slawa Olhovchenkov wrote:


On Mon, Apr 18, 2016 at 04:16:01PM +0200, Baptiste Daroussin wrote:


On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:

On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:


On 04/18/16 10:00, Ernie Luzar wrote:

11.0 will have pkg base, thats ok, but what does than mean for the
base.txz file?

It it going to stay as part of FBSD install?

I have many scripts for creating jails which depend on the base.txz file.

It's even easier now:

# mkdir -p /usr/jails/new
# pkg -r /usr/jails/new install -r base -g '*'

And where /etc now?

What do you mean?

At Jan 27 no package containing files from distributeworld.


At Mar 02


r298107 change this?



You have NOT answered the original question, what's going to happen to 
the base.txz file in 11.0?

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Glen Barber]

On Mon, Apr 18, 2016 at 12:21:28PM -0700, Nathan Whitehorn wrote:
> 
> 
> On 04/18/16 12:14, Glen Barber wrote:
> >On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:
> >>On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> >>>I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> >>>packages?! WHY?! What are reasons and goals to split base in such
> >>>enormous number of packages?
> >>Just a guess, having done the same thing myself:  it means that updates can 
> >>be
> >>more targeted.
> >>
> >This is exactly the reason, which has been answered numerous times.
> >
> >Glen
> >
> 
> That's a good reason -- and a very nice outcome of having base system
> packages -- but I worry that it may be going too far. The most granular
> updates would be if every file were its own package, which is obviously
> crazy, and so there is some middle ground. Needing to grab a whole new
> base.txz is probably too much (60 MB), but splitting that into even 6 or 7
> pieces moves the updates to replacements with typical size (a few MB) that
> are no larger than typical package updates for ports.

This granularity allows easy removal of things that may not be wanted
(such as *-debug*, *-profile*, etc.) on systems with little storage.  On
one of my testing systems, I removed the tests packages and all debug
and profiling, and the number of base system packages is 383.

Glen



signature.asc
Description: PGP signature

Re: [CFT] packaging the base system with pkg(8)

[Nathan Whitehorn]

On 04/18/16 12:14, Glen Barber wrote:

On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:

On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:

I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
packages?! WHY?! What are reasons and goals to split base in such
enormous number of packages?

Just a guess, having done the same thing myself:  it means that updates can be
more targeted.


This is exactly the reason, which has been answered numerous times.

Glen



That's a good reason -- and a very nice outcome of having base system 
packages -- but I worry that it may be going too far. The most granular 
updates would be if every file were its own package, which is obviously 
crazy, and so there is some middle ground. Needing to grab a whole new 
base.txz is probably too much (60 MB), but splitting that into even 6 or 
7 pieces moves the updates to replacements with typical size (a few MB) 
that are no larger than typical package updates for ports.

-Nathan
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Sean Fagan]

On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> 
> I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> packages?! WHY?! What are reasons and goals to split base in such
> enormous number of packages?

Just a guess, having done the same thing myself:  it means that updates can be
more targeted.

Sean.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Glen Barber]

On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:
> On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:
> > 
> > I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
> > packages?! WHY?! What are reasons and goals to split base in such
> > enormous number of packages?
> 
> Just a guess, having done the same thing myself:  it means that updates can be
> more targeted.
> 

This is exactly the reason, which has been answered numerous times.

Glen



signature.asc
Description: PGP signature

Re: [CFT] packaging the base system with pkg(8)

[Lev Serebryakov]

On 18.04.2016 21:52, Lev Serebryakov wrote:

> kerberos
 Ok, kerberos could not be packetized at all, as it is compilation
option for many other programs in tree. But 755 packets doesn't solve
this problem too.

-- 
// Lev Serebryakov



signature.asc
Description: OpenPGP digital signature

Re: qsort() documentation

[Warren Block]

On Mon, 18 Apr 2016, Hans Petter Selasky wrote:


Hi,

Are there any objections adding the following as part of documenting our 
kernel's qsort function?


Index: sys/libkern/qsort.c
===
--- sys/libkern/qsort.c (revision 298202)
+++ sys/libkern/qsort.c (working copy)
@@ -45,6 +45,10 @@

/*
 * Qsort routine from Bentley & McIlroy's "Engineering a Sort Function".
+ *
+ * NOTE: This implementation of qsort() was designed to not have the


"was designed to avoid the"


+ * worst case complexity of N**2, as seen with the regular quick sort
+ * functions as described by Wikipedia.
 */


Why Wikipedia, specifically?  There are a lot of places that describe 
quicksort.  How about just


  Note: This implementation of qsort() is designed to avoid the
  worst-case complexity of N**2 that is often seen with standard
  versions.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Mis-use of BUS_PASS_ORDER_MIDDLE

[John Baldwin]

On Monday, April 18, 2016 11:10:12 PM Howard Su wrote:
> I noticed several places there are code like this, especially in some arm
> low level drivers.
> EARLY_DRIVER_MODULE(aw_ccu, simplebus, aw_ccu_driver, aw_ccu_devclass,
> 0, 0, BUS_PASS_BUS + BUS_PASS_ORDER_MIDDLE);
> 
> ​I feel the usage of BUS_PASS_ORDER_MIDDLE is misused. There are another
> macro EARLY_DRIVER_MODULE_ORDERED, which take an additional parameter
> "order". I believe BUS_PASS_ORDER_xxx is used for that parameter.

No, this is actually correct.  The _ORDERED variants actually accept a
SI_ORDER_* value to determine how drivers contained in a single .ko file
are registered (in particular if you have several drivers in a .ko file
you typically want the "top-most" driver to attach last so that all the
other drivers are ready once the actual device is attached).

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: CFR: extend use of nitems() macro in the kernel.

[John Baldwin]

On Saturday, April 16, 2016 01:25:09 PM Pedro Giffuni wrote:
> Hello;
> 
> Using coccinelle, and some hand re-formatting, I generated a patch to
> make use of the nitems() macro in sys, which is too big for
> phabricator [1].
> 
> I was careful to exclude anything from the contrib directory or
> any code that is shared with userland (as to not have to include
> additional headers).
> 
> The patch is big but pretty safe, I think. The changes are small but
> still it touches many files[1].
> 
> I would like some feedback on the convenience of doing such replacement.
> If it is a good idea we could do the same for roundup2() and, in fact,
> I think DragonFly has already done this.
> 
> Regards,
> 
> Pedro.
> 
> [1] https://people.freebsd.org/~pfg/patches/sys-nitems.diff
> 
> [2] For those too lazy to check [1], here is a list of affected files.
> 
> M   sys/amd64/amd64/amd64_mem.c
> M   sys/amd64/amd64/machdep.c
> M   sys/amd64/linux/linux_sysvec.c
> M   sys/amd64/linux32/linux32_sysvec.c
> M   sys/arm/amlogic/aml8726/aml8726_clkmsr.c
> M   sys/arm/arm/db_interface.c
> M   sys/arm/at91/at91_pmc.c
> M   sys/arm/mv/armadaxp/armadaxp.c
> M   sys/arm/ti/cpsw/if_cpsw.c
> M   sys/arm/xscale/ixp425/ixp425.c
> M   sys/boot/common/part.c
> M   sys/boot/efi/loader/bootinfo.c
> M   sys/boot/mips/beri/boot2/boot2.c
> M   sys/boot/uboot/common/metadata.c
> M   sys/cam/ata/ata_da.c
> M   sys/cam/ata/ata_xpt.c

I would perhaps remove ata_quirk_table_size entirely and replace its
use with nitems() directly.  Probably best if that was a separate commit
though from the near-mechanical replacement.

> M   sys/cam/cam.c

Same here.  num_cam_status_entries is only used in one place and I think
directly using nitems() is probably clearer overall.

> M   sys/cam/scsi/scsi_all.c

Possibly the same here with sense_key_table_size.

> M   sys/cam/scsi/scsi_cd.c
> M   sys/cam/scsi/scsi_da.c
> M   sys/cam/scsi/scsi_sa.c
> M   sys/cam/scsi/scsi_xpt.c

Same as for ata_quirk_table_size (ironically this used the size in one
place and the expanded form of nitems in another while the ata variant
at least used the size in both places).

> M   sys/cam/scsi/smp_all.c
> M   sys/compat/linux/linux_socket.c
> M   sys/ddb/db_variables.c
> M   sys/dev/adb/adb_kbd.c
> M   sys/dev/advansys/adv_isa.c
> M   sys/dev/advansys/advlib.c
> M   sys/dev/advansys/adw_pci.c

Same here for num adw_num_pci_devs (only used once)

> M   sys/dev/advansys/adwlib.c

Probably the same here for adw_num_sync_rates (used twice).

> M   sys/dev/ae/if_ae.c

Same here for AE_DEVS_COUNT (only used once).

> M   sys/dev/age/if_age.c
> M   sys/dev/agp/agp.c
> M   sys/dev/agp/agp_ali.c
> M   sys/dev/agp/agp_amd64.c
> M   sys/dev/aha/aha_isa.c
> M   sys/dev/aic/aic.c
> M   sys/dev/aic/aic_cbus.c

Same here for AIC_ISA_NUMPORTS (only used once).

> M   sys/dev/aic/aic_isa.c

As above.

> M   sys/dev/ale/if_ale.c
> M   sys/dev/altera/atse/if_atse.c
> M   sys/dev/atkbdc/atkbd.c
> M   sys/dev/atkbdc/atkbdc.c
> M   sys/dev/atkbdc/psm.c
> M   sys/dev/bktr/bktr_core.c
> M   sys/dev/bwi/bwirf.c
> M   sys/dev/bwn/if_bwn.c
> M   sys/dev/cardbus/cardbus_cis.c
> M   sys/dev/digi/digi.c
> M   sys/dev/digi/digi_isa.c
> M   sys/dev/dwc/if_dwc.c
> M   sys/dev/ed/if_ed_hpp.c
> M   sys/dev/ed/if_ed_isa.c
> M   sys/dev/ed/if_ed_pci.c
> M   sys/dev/fb/creator.c

Same here for CREATOR_FB_MAP_SIZE (only used once).

> M   sys/dev/fb/fb.c
> M   sys/dev/fb/machfb.c
> M   sys/dev/fb/vesa.c
> M   sys/dev/fb/vga.c
> M   sys/dev/flash/mx25l.c
> M   sys/dev/hatm/if_hatm.c
> M   sys/dev/hifn/hifn7751.c
> M   sys/dev/hwpmc/hwpmc_amd.c

Same here for amd_event_codes_size (only used twice).

> M   sys/dev/hwpmc/hwpmc_core.c

Same here for niap_events.

> M   sys/dev/hwpmc/hwpmc_e500.c

e500_event_codes_size isn't even used.  It should just be removed.

> M   sys/dev/hwpmc/hwpmc_mips24k.c
> M   sys/dev/hwpmc/hwpmc_mips74k.c
> M   sys/dev/hwpmc/hwpmc_mpc7xxx.c

Same here for mpc7xxx_event_codes_size.

> M   sys/dev/hwpmc/hwpmc_octeon.c
> M   sys/dev/hwpmc/hwpmc_uncore.c

Same here for nucp_events.

> M   sys/dev/hwpmc/hwpmc_xscale.c

Same here for xscale_event_codes_size.

> M   sys/dev/if_ndis/if_ndis.c
> M   sys/dev/jme/if_jme.c
> M   sys/dev/kbd/kbd.c
> M   sys/dev/le/if_le_isa.c
> M   sys/dev/le/if_le_lebuffer.c

Same here for NLEMEDIA (only used once).

> M   sys/dev/le/if_le_ledma.c
> M   sys/dev/mlx/mlx.c
> M   sys/dev/mxge/if_mxge.c
> M   sys/dev/nand/nand_id.c
> M   sys/dev/ncr/ncr.c
> M   sys/dev/nctgpio/nctgpio.c
> M   sys/dev/nfe/if_nfe.c
> M   sys/dev/patm/if_patm_attach.c
> M   sys/dev/pccard/pccard_cis_quirks.c

Same here for n_pccard_cis_quirks (only used once).

> M 

Re: [CFT] packaging the base system with pkg(8)

[Lev Serebryakov]

On 03.03.2016 02:54, Glen Barber wrote:

> At present, the base system consists of 755 packages with the default
> build (empty src.conf(5) and make.conf(5)) for amd64.  The number of
> packages depends on several factors, but for most cases a runtime binary
> is split into several components.  In particular, most shared libraries
> are individually packaged, in addition to debugging symbols, profiling
> libraries, and 32-bit packaged separately.
 I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
packages?! WHY?! What are reasons and goals to split base in such
enormous number of packages?

  I understand debug symbols as separate package (one for almost whole
base, except several "contrib" parts), I could understand separate
package with all static libs (again, ONE package for all system static
libraries) and headers. I could understand separate packages for SEVERAL
"contrib" chunks: sendmail (it is often replaced by postfix / exim now),
kerberos, toolchain and, maybe, unbound.

 But extract EACH WITH_XXX feature to several separate packages? It
looks like nightmare. IMHO, it is very inconvenient for "default"
installation and it doesn't look as good replacement to NanoBSD. NanoBSD
is much more customized, typically.

 I don't have THAT number of packages even on "workstation"-like setup
with X and some desktop software now, leave sever installation alone.
And I don't see, how could this fragmentation could help me, as
administrator. But it adds load to "pkg", to many pkg-related scripts,
to "pkg version" output, at last!

 Why, or why, such fine-grained splitting (or should I say "shattering")
of base was chosen? Is here good rationale for this?

-- 
// Lev Serebryakov



signature.asc
Description: OpenPGP digital signature

Re: CFR: extend use of nitems() macro in the kernel.

[Pedro Giffuni]

On 04/18/16 01:56, Hans Petter Selasky wrote:

On 04/16/16 20:25, Pedro Giffuni wrote:

M   sys/dev/usb/input/ukbd.c
M   sys/dev/usb/serial/u3g.c
M   sys/dev/usb/serial/uchcom.c
M   sys/dev/usb/serial/umcs.c
M   sys/dev/usb/serial/uplcom.c


Approved. Maybe you can remove the superfluous pair of parenthesis after
the substitution?



Thanks: I'll look at improving the script.

FWIW, I already have a roundup2/rounddown2 semantic patch but it
is slow.

Pedro.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Ryan Stone]

On Mon, Apr 18, 2016 at 12:13 PM, Hans Petter Selasky 
wrote:

> Did anyone try to generate such a fiendish set of data, and see how
> quadratic the FreeBSD's qsort() becomes?
>

Not me, but it has been done:

http://calmerthanyouare.org/2014/06/11/algorithmic-complexity-attacks-and-libc-qsort.html
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Hans Petter Selasky]

On 04/18/16 16:49, Ed Schouten wrote:

2016-04-18 15:09 GMT+02:00 Hans Petter Selasky :

On 04/18/16 14:16, Aleksander Alekseev wrote:

I suggest also add a short description of how it was achieved
(randomization?).


I think the algorithm is switching to mergesort. I'll look up the paper and
add that correctly before commit.


As a Dutch person, I know the answer to this.

Instead of picking a fixed pivot or choosing one at random, it uses an
approach called linear time median finding to find a pivot that is
'approximately median'. There are a couple of algorithms for this, but
I think Bentley's qsort() uses this:

https://en.wikipedia.org/wiki/Dutch_national_flag_problem



Hi,

Ryan: Yes, there is quadratic behaviour still, but I believe the order 
is limited. For the matter of the topic I added a counter for the swap() 
code in the insertion fallback algorithm, and for a set of 2048 integers 
I never saw the swap() count exceed this number. For a pre-sorted array, 
values around ~2047 and reverse sorted ~2043. For random input far less.


Citing the document "bentley93engineering.pdf", a footnote says:


Of course, quadratic behavior is still possible. One can generate 
fiendish inputs by bugging Quicksort: Con-
sider key values to be unknown initially. In the code for selecting a 
partition element, assign values in increas-
ing order as unknown keys are encountered. In the partitioning code, 
make unknown keys compare high.



Did anyone try to generate such a fiendish set of data, and see how 
quadratic the FreeBSD's qsort() becomes?


Another thread, possibly related:

http://postgresql.nabble.com/Why-do-we-still-perform-a-check-for-pre-sorted-input-within-qsort-variants-td5746526.html

--HPS
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Mis-use of BUS_PASS_ORDER_MIDDLE

[Howard Su]

I noticed several places there are code like this, especially in some arm
low level drivers.
EARLY_DRIVER_MODULE(aw_ccu, simplebus, aw_ccu_driver, aw_ccu_devclass,
0, 0, BUS_PASS_BUS + BUS_PASS_ORDER_MIDDLE);

​I feel the usage of BUS_PASS_ORDER_MIDDLE is misused. There are another
macro EARLY_DRIVER_MODULE_ORDERED, which take an additional parameter
"order". I believe BUS_PASS_ORDER_xxx is used for that parameter.
​

-- 
-Howard
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: objcopy exit with signal 10 trying to update to recent -CURRENT

[Renato Botelho]

> On Apr 18, 2016, at 10:59, Renato Botelho do Couto  wrote:
> 
> I’m trying to upgrade a -CURRENT amd64 installation from r297492 to r298203 
> and got:
> 
> c++ -O2 -pipe 
> -I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/include 
> -I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/tools/clang/include
>  -I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/utils/TableGen 
> -I. 
> -I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/../../lib/clang/include
>  -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS 
> -D__STDC_CONSTANT_MACROS -fno-strict-aliasing 
> -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" 
> -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" 
> -DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -g -Qunused-arguments 
> -I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions 
> -fno-rtti -stdlib=libc++ -Wno-c++11-extensions  -static 
> -L/usr/obj/usr/src/tmp/legacy/usr/lib -o llvm-tblgen.full  
> AsmMatcherEmitter.o AsmWriterEmitter.o AsmWriterInst.o Attributes.o 
> CTagsEmitter.o CallingConvEmitter.o CodeEmitterGen.o CodeGenDAGPatterns.o 
> CodeGenInstruction.o CodeGenMapTable.o CodeGenRegisters.o CodeGenSchedule.o 
> CodeGenTarget.o DAGISelEmitter.o DAGISelMatcher.o DAGISelMatcherEmitter.o 
> DAGISelMatcherGen.o DAGISelMatcherOpt.o DFAPacketizerEmitter.o 
> DisassemblerEmitter.o FastISelEmitter.o FixedLenDecoderEmitter.o 
> InstrInfoEmitter.o IntrinsicEmitter.o OptParserEmitter.o 
> PseudoLoweringEmitter.o RegisterInfoEmitter.o SubtargetEmitter.o TableGen.o 
> X86DisassemblerTables.o X86ModRMFilters.o X86RecognizableInstr.o 
> /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/llvm-tblgen/../../../lib/clang/libllvmtablegen/libllvmtablegen.a
>  
> /usr/obj/usr/src/tmp/usr/src/usr.bin/clang/llvm-tblgen/../../../lib/clang/libllvmsupport/libllvmsupport.a
>   -lncursesw  -lpthread -legacy
> objcopy --only-keep-debug llvm-tblgen.full llvm-tblgen.debug
> objcopy --strip-debug --add-gnu-debuglink=llvm-tblgen.debug  llvm-tblgen.full 
> llvm-tblgen
> *** Signal 10
> 
> I’ve already tried a fresh build after remove /usr/obj, same problem. Any 
> ideas?

Fixed after I manually rebuilt usr.bin/elfcopy

--
Renato Botelho

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 11.0-RELEASE pkg base & base.txz file

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 05:27:09PM +0300, Slawa Olhovchenkov wrote:

> On Mon, Apr 18, 2016 at 04:16:01PM +0200, Baptiste Daroussin wrote:
> 
> > On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:
> > > On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:
> > > 
> > > > On 04/18/16 10:00, Ernie Luzar wrote:
> > > > > 11.0 will have pkg base, thats ok, but what does than mean for the
> > > > > base.txz file?
> > > > > 
> > > > > It it going to stay as part of FBSD install?
> > > > > 
> > > > > I have many scripts for creating jails which depend on the base.txz 
> > > > > file.
> > > > 
> > > > It's even easier now:
> > > > 
> > > > # mkdir -p /usr/jails/new
> > > > # pkg -r /usr/jails/new install -r base -g '*'
> > > 
> > > And where /etc now?
> > 
> > What do you mean?
> 
> At Jan 27 no package containing files from distributeworld.

At Mar 02

> r298107 change this?
> 
> ___
> freebsd-pkgb...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase
> To unsubscribe, send any mail to "freebsd-pkgbase-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Ed Schouten]

2016-04-18 15:09 GMT+02:00 Hans Petter Selasky :
> On 04/18/16 14:16, Aleksander Alekseev wrote:
>> I suggest also add a short description of how it was achieved
>> (randomization?).
>
> I think the algorithm is switching to mergesort. I'll look up the paper and
> add that correctly before commit.

As a Dutch person, I know the answer to this.

Instead of picking a fixed pivot or choosing one at random, it uses an
approach called linear time median finding to find a pivot that is
'approximately median'. There are a couple of algorithms for this, but
I think Bentley's qsort() uses this:

https://en.wikipedia.org/wiki/Dutch_national_flag_problem

-- 
Ed Schouten 
Nuxi, 's-Hertogenbosch, the Netherlands
KvK-nr.: 62051717
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

objcopy exit with signal 10 trying to update to recent -CURRENT

[Renato Botelho do Couto]

I’m trying to upgrade a -CURRENT amd64 installation from r297492 to r298203 and 
got:

c++ -O2 -pipe 
-I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/include 
-I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/tools/clang/include 
-I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/utils/TableGen -I. 
-I/usr/src/usr.bin/clang/llvm-tblgen/../../../contrib/llvm/../../lib/clang/include
 -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS 
-D__STDC_CONSTANT_MACROS -fno-strict-aliasing 
-DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" 
-DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" 
-DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -g -Qunused-arguments 
-I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions -fno-rtti 
-stdlib=libc++ -Wno-c++11-extensions  -static 
-L/usr/obj/usr/src/tmp/legacy/usr/lib -o llvm-tblgen.full  AsmMatcherEmitter.o 
AsmWriterEmitter.o AsmWriterInst.o Attributes.o CTagsEmitter.o 
CallingConvEmitter.o CodeEmitterGen.o CodeGenDAGPatterns.o CodeGenInstruction.o 
CodeGenMapTable.o CodeGenRegisters.o CodeGenSchedule.o CodeGenTarget.o 
DAGISelEmitter.o DAGISelMatcher.o DAGISelMatcherEmitter.o DAGISelMatcherGen.o 
DAGISelMatcherOpt.o DFAPacketizerEmitter.o DisassemblerEmitter.o 
FastISelEmitter.o FixedLenDecoderEmitter.o InstrInfoEmitter.o 
IntrinsicEmitter.o OptParserEmitter.o PseudoLoweringEmitter.o 
RegisterInfoEmitter.o SubtargetEmitter.o TableGen.o X86DisassemblerTables.o 
X86ModRMFilters.o X86RecognizableInstr.o 
/usr/obj/usr/src/tmp/usr/src/usr.bin/clang/llvm-tblgen/../../../lib/clang/libllvmtablegen/libllvmtablegen.a
 
/usr/obj/usr/src/tmp/usr/src/usr.bin/clang/llvm-tblgen/../../../lib/clang/libllvmsupport/libllvmsupport.a
  -lncursesw  -lpthread -legacy
objcopy --only-keep-debug llvm-tblgen.full llvm-tblgen.debug
objcopy --strip-debug --add-gnu-debuglink=llvm-tblgen.debug  llvm-tblgen.full 
llvm-tblgen
*** Signal 10

I’ve already tried a fresh build after remove /usr/obj, same problem. Any ideas?
--
Renato Botelho do Couto
Software Engineer
Netgate, Inc.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 11.0-RELEASE pkg base & base.txz file

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 04:16:01PM +0200, Baptiste Daroussin wrote:

> On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:
> > On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:
> > 
> > > On 04/18/16 10:00, Ernie Luzar wrote:
> > > > 11.0 will have pkg base, thats ok, but what does than mean for the
> > > > base.txz file?
> > > > 
> > > > It it going to stay as part of FBSD install?
> > > > 
> > > > I have many scripts for creating jails which depend on the base.txz 
> > > > file.
> > > 
> > > It's even easier now:
> > > 
> > > # mkdir -p /usr/jails/new
> > > # pkg -r /usr/jails/new install -r base -g '*'
> > 
> > And where /etc now?
> 
> What do you mean?

At Jan 27 no package containing files from distributeworld.
r298107 change this?

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 11.0-RELEASE pkg base & base.txz file

[Baptiste Daroussin]

On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:
> On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:
> 
> > On 04/18/16 10:00, Ernie Luzar wrote:
> > > 11.0 will have pkg base, thats ok, but what does than mean for the
> > > base.txz file?
> > > 
> > > It it going to stay as part of FBSD install?
> > > 
> > > I have many scripts for creating jails which depend on the base.txz file.
> > 
> > It's even easier now:
> > 
> > # mkdir -p /usr/jails/new
> > # pkg -r /usr/jails/new install -r base -g '*'
> 
> And where /etc now?

What do you mean?

Bapt


signature.asc
Description: PGP signature

Re: 11.0-RELEASE pkg base & base.txz file

[Slawa Olhovchenkov]

On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:

> On 04/18/16 10:00, Ernie Luzar wrote:
> > 11.0 will have pkg base, thats ok, but what does than mean for the
> > base.txz file?
> > 
> > It it going to stay as part of FBSD install?
> > 
> > I have many scripts for creating jails which depend on the base.txz file.
> 
> It's even easier now:
> 
> # mkdir -p /usr/jails/new
> # pkg -r /usr/jails/new install -r base -g '*'

And where /etc now?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 11.0-RELEASE pkg base & base.txz file

[Nikolai Lifanov]

On 04/18/16 10:00, Ernie Luzar wrote:
> 11.0 will have pkg base, thats ok, but what does than mean for the
> base.txz file?
> 
> It it going to stay as part of FBSD install?
> 
> I have many scripts for creating jails which depend on the base.txz file.

It's even easier now:

# mkdir -p /usr/jails/new
# pkg -r /usr/jails/new install -r base -g '*'

- Nikolai Lifanov

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

11.0-RELEASE pkg base & base.txz file

[Ernie Luzar]

11.0 will have pkg base, thats ok, but what does than mean for the 
base.txz file?


It it going to stay as part of FBSD install?

I have many scripts for creating jails which depend on the base.txz file.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Ryan Stone]

On Mon, Apr 18, 2016 at 9:09 AM, Hans Petter Selasky  wrote

> I think the algorithm is switching to mergesort. I'll look up the paper
> and add that correctly before commit.
>

No, it switches to insertion sort, assuming that it's acting on an already
sorted array.  If that assumption is wrong you still get O(n**2) complexity.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Hans Petter Selasky]

On 04/18/16 14:16, Aleksander Alekseev wrote:

I suggest also add a short description of how it was achieved
(randomization?).


I think the algorithm is switching to mergesort. I'll look up the paper 
and add that correctly before commit.


--HPS
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: qsort() documentation

[Aleksander Alekseev]

Hello.

I suggest also add a short description of how it was achieved
(randomization?).

On Mon, 18 Apr 2016 13:43:38 +0200
Hans Petter Selasky  wrote:

> Hi,
> 
> Are there any objections adding the following as part of documenting
> our kernel's qsort function?
> 
> Index: sys/libkern/qsort.c
> ===
> --- sys/libkern/qsort.c (revision 298202)
> +++ sys/libkern/qsort.c (working copy)
> @@ -45,6 +45,10 @@
> 
>   /*
>* Qsort routine from Bentley & McIlroy's "Engineering a Sort
> Function".
> + *
> + * NOTE: This implementation of qsort() was designed to not have the
> + * worst case complexity of N**2, as seen with the regular quick sort
> + * functions as described by Wikipedia.
>*/
> 
> 
> --HPS
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to
> "freebsd-current-unsubscr...@freebsd.org"
> 



-- 
Best regards,
Aleksander Alekseev
http://eax.me/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

qsort() documentation

[Hans Petter Selasky]

Hi,

Are there any objections adding the following as part of documenting our 
kernel's qsort function?


Index: sys/libkern/qsort.c
===
--- sys/libkern/qsort.c (revision 298202)
+++ sys/libkern/qsort.c (working copy)
@@ -45,6 +45,10 @@

 /*
  * Qsort routine from Bentley & McIlroy's "Engineering a Sort Function".
+ *
+ * NOTE: This implementation of qsort() was designed to not have the
+ * worst case complexity of N**2, as seen with the regular quick sort
+ * functions as described by Wikipedia.
  */


--HPS
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: CFR: extend use of nitems() macro in the kernel.

[Hans Petter Selasky]

On 04/16/16 20:25, Pedro Giffuni wrote:

M   sys/dev/usb/input/ukbd.c
M   sys/dev/usb/serial/u3g.c
M   sys/dev/usb/serial/uchcom.c
M   sys/dev/usb/serial/umcs.c
M   sys/dev/usb/serial/uplcom.c


Approved. Maybe you can remove the superfluous pair of parenthesis after 
the substitution?


--HPS
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"