subject:"\[PATCH 04\/14\] init\: deps\: order network interfaces by link order"

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-22 Thread Alexander Holler


Am 19.10.2015 um 13:31 schrieb Alexander Holler:

Am 19.10.2015 um 12:57 schrieb Alexander Holler:

Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the
patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your
boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's
easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize
stuff, I have to say that it just can't work because just starting a
thread for every probe() totally ignores possible dependencies.
Regardless if using one thread per probe() call or if feeding probe()
calls to just a few threads.

Maybe that's why previous attempts to parallelize stuff failed. But
that's just an assumption as I'm unaware of these previous attempts.


Or to describe it more verbose, if DEBUG is turned on in
init/dependencies.c (using my patches), it spits out a summary of groups
with initcalls (probe() calls) which are independent from each other and
therfore can be called in parallel. E.g. one of my systems this looks so:

[0.288229] init: vertices: 429 edges 204 count 170
[0.288295] init: group 0 length 66 (start 0)
[0.288329] init: group 1 length 33 (start 66)
[0.288364] init: group 2 length 13 (start 99)
[0.288398] init: group 3 length 7 (start 112)
[0.288432] init: group 4 length 9 (start 119)
[0.288466] init: group 5 length 8 (start 128)
[0.288500] init: group 6 length 11 (start 136)
[0.288534] init: group 7 length 6 (start 147)
[0.288569] init: group 8 length 4 (start 153)
[0.288603] init: group 9 length 8 (start 157)
[0.288637] init: group 10 length 3 (start 165)
[0.288671] init: group 11 length 2 (start 168)
[0.288705] init: using 4 threads to call annotated initcalls

That means the first group contains 66 initcalls which are called using
4 threads, and, after those have finished, the second group with 33
initcalls will be called in parallel (using the same 4 threads).


BTW. That also means that, for the above example, the worst case would 
mean an error rate of 61% if those 170 (annotated) initcalls would be 
started in parallel while ignoring dependencies.


But that's just meant as an (hopefully) interesting number when looking 
at the above numbers a bit different.


(I've understood that the patches aren't wanted.)


Regards,

Alexander Holler


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-22 Thread Alexander Holler


Am 19.10.2015 um 13:31 schrieb Alexander Holler:

Am 19.10.2015 um 12:57 schrieb Alexander Holler:

Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the
patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your
boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's
easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize
stuff, I have to say that it just can't work because just starting a
thread for every probe() totally ignores possible dependencies.
Regardless if using one thread per probe() call or if feeding probe()
calls to just a few threads.

Maybe that's why previous attempts to parallelize stuff failed. But
that's just an assumption as I'm unaware of these previous attempts.


Or to describe it more verbose, if DEBUG is turned on in
init/dependencies.c (using my patches), it spits out a summary of groups
with initcalls (probe() calls) which are independent from each other and
therfore can be called in parallel. E.g. one of my systems this looks so:

[0.288229] init: vertices: 429 edges 204 count 170
[0.288295] init: group 0 length 66 (start 0)
[0.288329] init: group 1 length 33 (start 66)
[0.288364] init: group 2 length 13 (start 99)
[0.288398] init: group 3 length 7 (start 112)
[0.288432] init: group 4 length 9 (start 119)
[0.288466] init: group 5 length 8 (start 128)
[0.288500] init: group 6 length 11 (start 136)
[0.288534] init: group 7 length 6 (start 147)
[0.288569] init: group 8 length 4 (start 153)
[0.288603] init: group 9 length 8 (start 157)
[0.288637] init: group 10 length 3 (start 165)
[0.288671] init: group 11 length 2 (start 168)
[0.288705] init: using 4 threads to call annotated initcalls

That means the first group contains 66 initcalls which are called using
4 threads, and, after those have finished, the second group with 33
initcalls will be called in parallel (using the same 4 threads).


BTW. That also means that, for the above example, the worst case would 
mean an error rate of 61% if those 170 (annotated) initcalls would be 
started in parallel while ignoring dependencies.


But that's just meant as an (hopefully) interesting number when looking 
at the above numbers a bit different.


(I've understood that the patches aren't wanted.)


Regards,

Alexander Holler


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-19 Thread Alexander Holler


Am 19.10.2015 um 12:57 schrieb Alexander Holler:

Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's
easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize
stuff, I have to say that it just can't work because just starting a
thread for every probe() totally ignores possible dependencies.
Regardless if using one thread per probe() call or if feeding probe()
calls to just a few threads.

Maybe that's why previous attempts to parallelize stuff failed. But
that's just an assumption as I'm unaware of these previous attempts.


Or to describe it more verbose, if DEBUG is turned on in 
init/dependencies.c (using my patches), it spits out a summary of groups 
with initcalls (probe() calls) which are independent from each other and 
therfore can be called in parallel. E.g. one of my systems this looks so:


[0.288229] init: vertices: 429 edges 204 count 170
[0.288295] init: group 0 length 66 (start 0)
[0.288329] init: group 1 length 33 (start 66)
[0.288364] init: group 2 length 13 (start 99)
[0.288398] init: group 3 length 7 (start 112)
[0.288432] init: group 4 length 9 (start 119)
[0.288466] init: group 5 length 8 (start 128)
[0.288500] init: group 6 length 11 (start 136)
[0.288534] init: group 7 length 6 (start 147)
[0.288569] init: group 8 length 4 (start 153)
[0.288603] init: group 9 length 8 (start 157)
[0.288637] init: group 10 length 3 (start 165)
[0.288671] init: group 11 length 2 (start 168)
[0.288705] init: using 4 threads to call annotated initcalls

That means the first group contains 66 initcalls which are called using 
4 threads, and, after those have finished, the second group with 33 
initcalls will be called in parallel (using the same 4 threads).



Regards,

Alexander Holler


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-19 Thread Alexander Holler


Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize 
stuff, I have to say that it just can't work because just starting a 
thread for every probe() totally ignores possible dependencies. 
Regardless if using one thread per probe() call or if feeding probe() 
calls to just a few threads.


Maybe that's why previous attempts to parallelize stuff failed. But 
that's just an assumption as I'm unaware of these previous attempts.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-19 Thread Alexander Holler


Am 19.10.2015 um 12:57 schrieb Alexander Holler:

Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's
easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize
stuff, I have to say that it just can't work because just starting a
thread for every probe() totally ignores possible dependencies.
Regardless if using one thread per probe() call or if feeding probe()
calls to just a few threads.

Maybe that's why previous attempts to parallelize stuff failed. But
that's just an assumption as I'm unaware of these previous attempts.


Or to describe it more verbose, if DEBUG is turned on in 
init/dependencies.c (using my patches), it spits out a summary of groups 
with initcalls (probe() calls) which are independent from each other and 
therfore can be called in parallel. E.g. one of my systems this looks so:


[0.288229] init: vertices: 429 edges 204 count 170
[0.288295] init: group 0 length 66 (start 0)
[0.288329] init: group 1 length 33 (start 66)
[0.288364] init: group 2 length 13 (start 99)
[0.288398] init: group 3 length 7 (start 112)
[0.288432] init: group 4 length 9 (start 119)
[0.288466] init: group 5 length 8 (start 128)
[0.288500] init: group 6 length 11 (start 136)
[0.288534] init: group 7 length 6 (start 147)
[0.288569] init: group 8 length 4 (start 153)
[0.288603] init: group 9 length 8 (start 157)
[0.288637] init: group 10 length 3 (start 165)
[0.288671] init: group 11 length 2 (start 168)
[0.288705] init: using 4 threads to call annotated initcalls

That means the first group contains 66 initcalls which are called using 
4 threads, and, after those have finished, the second group with 33 
initcalls will be called in parallel (using the same 4 threads).



Regards,

Alexander Holler


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-19 Thread Alexander Holler


Am 18.10.2015 um 12:11 schrieb Alexander Holler:

Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more
problems
_and_ makes things take longer.  Try it, we have done it in the
past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the
easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new
thread for every call. And it doesn't need much imagination (or
experience) that this introduces quite some overhead.

But maybe it makes sense to try out what I'm doing in my patches,
starting multiple threads once and then just giving them some work. Will


After a having second thought on your simple approach to parallelize 
stuff, I have to say that it just can't work because just starting a 
thread for every probe() totally ignores possible dependencies. 
Regardless if using one thread per probe() call or if feeding probe() 
calls to just a few threads.


Maybe that's why previous attempts to parallelize stuff failed. But 
that's just an assumption as I'm unaware of these previous attempts.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-18 Thread Alexander Holler


Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new 
thread for every call. And it doesn't need much imagination (or 
experience) that this introduces quite some overhead.


But maybe it makes sense to try out what I'm doing in my patches, 
starting multiple threads once and then just giving them some work. Will 
keep that in mind, also I don't think I will post any patch in the next 
few years. ;)


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-18 Thread Greg Kroah-Hartman

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:
> Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:
> >On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:
> >>Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:
> >>
> >>>Again, parallelizing does not solve anything, and causes more problems
> >>>_and_ makes things take longer.  Try it, we have done it in the past and
> >>>proven this, it's pretty easy to test :)
> >>
> >>Just because I'm curious, may I ask how I would test that in the easy way
> >>you have in mind? I've just posted the results of my tests (the patch
> >>series) but I wonder what you do have in mind.
> >
> >Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
> >sequence.  That should show you the drivers, or other areas, that are
> >causing your boot to be "slow".
> 
> So I've misunderstood you. I've read your paragraph as that it's easy to
> test parallelizing.

Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.

hope this helps,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-18 Thread Greg Kroah-Hartman

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:
> Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:
> >On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:
> >>Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:
> >>
> >>>Again, parallelizing does not solve anything, and causes more problems
> >>>_and_ makes things take longer.  Try it, we have done it in the past and
> >>>proven this, it's pretty easy to test :)
> >>
> >>Just because I'm curious, may I ask how I would test that in the easy way
> >>you have in mind? I've just posted the results of my tests (the patch
> >>series) but I wonder what you do have in mind.
> >
> >Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
> >sequence.  That should show you the drivers, or other areas, that are
> >causing your boot to be "slow".
> 
> So I've misunderstood you. I've read your paragraph as that it's easy to
> test parallelizing.

Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.

hope this helps,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-18 Thread Alexander Holler


Am 18.10.2015 um 07:59 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 07:20:34AM +0200, Alexander Holler wrote:

Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to
test parallelizing.


Ah, ok, if you want to parallelize everything, add some logic in the
driver core where the probe() callback is made to spin that off into a
new thread for every call, and when it's done, clean up the thread.
That's what I did many years ago to try this all out, if you dig in the
lkml archives there's probably a patch somewhere that you can base the
work off of to test it yourself.


Hmm, I don't think I will do that because that means to setup a new 
thread for every call. And it doesn't need much imagination (or 
experience) that this introduces quite some overhead.


But maybe it makes sense to try out what I'm doing in my patches, 
starting multiple threads once and then just giving them some work. Will 
keep that in mind, also I don't think I will post any patch in the next 
few years. ;)


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to 
test parallelizing.



Hope this helps,


Unfortunately no, but thanks anyway. ;)

Alexander Holler

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:
> Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:
> 
> >Again, parallelizing does not solve anything, and causes more problems
> >_and_ makes things take longer.  Try it, we have done it in the past and
> >proven this, it's pretty easy to test :)
> 
> Just because I'm curious, may I ask how I would test that in the easy way
> you have in mind? I've just posted the results of my tests (the patch
> series) but I wonder what you do have in mind.

Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".

Hope this helps,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy 
way you have in mind? I've just posted the results of my tests (the 
patch series) but I wonder what you do have in mind.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


Sorry, but I have to add another comment about that stable topological 
sort which can be easily found by searching the web.


As the author mentioned, he has done some search on that topic too and 
has found nothing. So, either I would have to trust his word (and his 
non-public proof) that it works or I would have to prove it myself.


Besides that he says it's based on bubble sort, which is known for its 
speed. So I would have had to write in C in order to see if the speed 
would be acceptable and would still be without any proof that it always 
works correctly.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:58 schrieb Alexander Holler:


Unfortunately it's quiet a lot of work to add dependencies for everything.


And (just in case of), also I'm a non-native English writer, I know the 
difference between quiet and quite. But, unfortunately, that doesn't 
prevent me to type it wrong. It's like "teh" instead of "the". 
Unfortunately I'm not very good in writing English fast without errors, 
but that doesn't mean I'm dumb (also many native English writers prefer 
to assume that).


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:

On Sat, Oct 17, 2015 at 09:14:34PM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


It's impossible to take it into account because I don't want to miss the
parallelize functionality. And without that, all the stuff doesn't offer
enough benefits to be worse the effort but just adds some time necessary to
do the sorting. It might solve the deferred probe problems, but without much
benefit.


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


I've tested it, otherwise I wouldn't have posted the patches.

Unfortunately it's quiet a lot of work to add dependencies for everything.

So maybe I'm able to offer some better numbers in a year or such, when I 
was bored often enough to add more dependencies for initcalls.


Not to mention that small changes in the order can have quiet big 
differences in the boot time, so it's quiet hard to parallelize stuff 
(add dependencies) correctly like e.g. the pci/acpi/processor stuff. 
Especially because many reasons for the current order aren't mentioned 
in the source and are hard to see without specific knowledge about the HW.


Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 12:14 PM, Alexander Holler  wrote:
>
> It's impossible to take it into account because I don't want to miss the
> parallelize functionality. And without that, all the stuff doesn't offer
> enough benefits to be worse the effort but just adds some time necessary to
> do the sorting. It might solve the deferred probe problems, but without much
> benefit.

Quite frankly, the alleged advantage of parallelization of the init
sequence is *very* questionable.

But even if you were to want to parallelize things at that level, you'd still

 (a) want to make *that* a flag (to debug) and make it opt-in (so
you'd have another "this module can be done in parallel" marker)

 (b) end up with all the other drivers *not* being parallel, because
they were all written to not have everything happen at once.

 (c) want to avoid the whole flag-day things

so even then you'd want to sort everything using that topological
stable sort, so that you can sanely mix things, and then mark
individual drivers as "I'd like to run in parallel with the rest of
the init sequence".

The main thing that being parallel would change is likely that you'd
also have to add some kind of "I'm done" completion for each module
(or dependency point), so that things that depend on other things
(that have *started*) can explicitly wait for those other things to
actually end.

We have something like that for our "sync" points right now with the
whole asynchronous machinery. It's somewhat general (although I think
almost all users end up just doing "async_synchronize_full()", even
though we do have a cookie-based one) and I suspect the cookie-based
one could be used for individual drivers and their completions.

But I really think the whole "do initcalls in parallel" is a very bad
idea _regardless_ of anything else. We almost never actually have
initcalls that really take a lot of time, and when we do, we already
handle the big ones with the async machinery, which is actually a lot
more powerful and flexible than some "run initcalls in parallel",
exactly because you can run *parts* serially (like basic discovery),
and then run other parts asynchronously (like "spin up disk" or "wait
for power stable" or whatever - *after* you have discovered the
device).

So I personally think our async thing is much more powerful and useful
than some random "let's run initcalls in parallel".

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sat, Oct 17, 2015 at 09:14:34PM +0200, Alexander Holler wrote:
> Am 17.10.2015 um 21:08 schrieb Linus Torvalds:
> >On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  
> >wrote:
> >>
> >>That isn't a flag day thing. It's a config option everyone can turn on and
> >>off whenever he wants.
> >
> >That's a flag-day thing. We've done it (drm comes to mind - several times).
> >
> >I'm disappointed, because I _know_ I pointed you in the direction of
> >stable sorting about a month ago. I really had hoped you'd have taken
> >that into account.
> 
> It's impossible to take it into account because I don't want to miss the
> parallelize functionality. And without that, all the stuff doesn't offer
> enough benefits to be worse the effort but just adds some time necessary to
> do the sorting. It might solve the deferred probe problems, but without much
> benefit.

Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


It's impossible to take it into account because I don't want to miss the 
parallelize functionality. And without that, all the stuff doesn't offer 
enough benefits to be worse the effort but just adds some time necessary 
to do the sorting. It might solve the deferred probe problems, but 
without much benefit.


Regards,

Alexander Holler

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:
>
> That isn't a flag day thing. It's a config option everyone can turn on and
> off whenever he wants.

That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:03 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:


Otherwise it's impossible to call initcalls in parallel. I've seen a stable
topological sort somewhere, but whenever you want to parallelize the
initcalls, the stable ordering would be gone anyway. So I've decided not to
look further at a stable topological sort.


So five seconds of googling gave me freely usable source code for a
stable topological sort, that also has a nice reported added
advantage:

  "An interesting property of a stable topological sort is that cyclic
dependencies are tolerated and resolved according to original order of
elements in sequence. This is a desirable feature for many
applications because it allows to sort any sequence with any
imaginable dependencies between the elements"

which seems to be *exactly* what you'd want, especially considering
that right now your patches add extra "no-dependency" markers exactly
because of the cyclical problem.


That's the stable topological sort I've mentioned the link to in the 
discussion with you.




I think it was the #2 hit on google for "stable topological sort". I
didn't look closely at the source code, but it was not big.

And no, since we don't actually want to parallelize the initcalls
anyway (I had this discussion with you just a month ago), your
objections seem even more questionable. We have separate machinery for
"do this asynchronously", and we want to _keep_ that separate.


I've understood that now.

Sorry for wasting your time.

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:
>
> Otherwise it's impossible to call initcalls in parallel. I've seen a stable
> topological sort somewhere, but whenever you want to parallelize the
> initcalls, the stable ordering would be gone anyway. So I've decided not to
> look further at a stable topological sort.

So five seconds of googling gave me freely usable source code for a
stable topological sort, that also has a nice reported added
advantage:

 "An interesting property of a stable topological sort is that cyclic
dependencies are tolerated and resolved according to original order of
elements in sequence. This is a desirable feature for many
applications because it allows to sort any sequence with any
imaginable dependencies between the elements"

which seems to be *exactly* what you'd want, especially considering
that right now your patches add extra "no-dependency" markers exactly
because of the cyclical problem.

I think it was the #2 hit on google for "stable topological sort". I
didn't look closely at the source code, but it was not big.

And no, since we don't actually want to parallelize the initcalls
anyway (I had this discussion with you just a month ago), your
objections seem even more questionable. We have separate machinery for
"do this asynchronously", and we want to _keep_ that separate.

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 20:52 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:


I'm making dependencies the only ordering for annotated initcalls.


Yeah, and quite frankly, that just means that I'm not going to merge it.

We do not do "flag-day" things. We've done them in the past, and it
has always been a major and unacceptable pain.


That isn't a flag day thing. It's a config option everyone can turn on 
and off whenever he wants.




And if the dependency ordering is "outside" of the traditional
link-time ordering, then it is by definition a flag-day event. Every
time you add a dependency to even just *one* driver, it magically
changes ordering wrt all the other drivers, because now it's in a
different ordering domain.

So please reconsider. Or stop cc'ing me. Because "flag day" changes
really are not acceptable.


I'm choosing the second option. I will answer further mails on that 
topic, but will remove you from cc. Besides that I consider these 
patches as a total failure and will not post any further version.


Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sat, Oct 17, 2015 at 08:37:35PM +0200, Alexander Holler wrote:
> I'm making dependencies the only ordering for annotated initcalls.
> 
> Otherwise it's impossible to call initcalls in parallel.

We don't ever want to call initcalls in parallel, unless they can
properly handle it.  All drivers can tell the driver core to use
parallel probing if they know they can handle it.  See 'enum probe_type'
for this.

Trying to call initcalls or driver probes in parallel blindly has been
proven to both take a longer amount of time, and cause problems for
drivers that aren't expecting it.  The research on this was done years
ago when people were starting to work on booting quicker, and the end
result is what we have now where drivers that know they can handle this,
can tell the core to let them probe in this manner.

The outcome of this work is the sub-second boot time we have today.  If
your systems are taking longer to boot, then please look into the
drivers that are causing this, that is where the real issue is, not in
the core of the kernel.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:
>
> I'm making dependencies the only ordering for annotated initcalls.

Yeah, and quite frankly, that just means that I'm not going to merge it.

We do not do "flag-day" things. We've done them in the past, and it
has always been a major and unacceptable pain.

And if the dependency ordering is "outside" of the traditional
link-time ordering, then it is by definition a flag-day event. Every
time you add a dependency to even just *one* driver, it magically
changes ordering wrt all the other drivers, because now it's in a
different ordering domain.

So please reconsider. Or stop cc'ing me. Because "flag day" changes
really are not acceptable.

   Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 20:23 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 10:14 AM, Alexander Holler  wrote:


Assuming three different ethernet-drivers, without any special code,
the dependency graph would not require any special order inbetween them
and would look like that:


This seems *fundamentally* wrong.

This is in no way specific to network drivers (or to disk drivers, or
to anything else).

So requiring extra logic for this implies that something is seriously wrong.

If two drivers aren't ordered by dependencies, they should always be
in link order, regardless of any hacks like these. If they're not,
things are wrong.

I think your problem is that you make that dependency thing a separate
ordering, so now it matters whether a driver has a dependency or not.


I'm making dependencies the only ordering for annotated initcalls.

Otherwise it's impossible to call initcalls in parallel. I've seen a 
stable topological sort somewhere, but whenever you want to parallelize 
the initcalls, the stable ordering would be gone anyway. So I've decided 
not to look further at a stable topological sort.



If something like this is to work, it has to work *with* the normal
ordering, not outside of it and then have these kinds of broken
special cases.

The normal init orderings (ie core -> postcore -> arch -> subsys -> fs
-> rootfs -> device -> late) should just be an extra dependency, I
think.

The way that you just insert the annotated dependencies in between
levels 6 and 7 ("device" and "late") can't be right. It means - for
example - that you can't have subsystems that have dependencies.


Sorry, but that's wrong.

I've choosen to place initcalls between 5 and 6 to make it easier to 
move both, subsystems as well as normal drivers to the (new) level with 
annotated initcalls. If you look at what I've already "annotated", you 
will see there are quiet a lot initcalls I've moved from below 6 to the 
new level.



So I really think that if we do dependencies, then the current levels
have to be added as dependencies, so that "subsys_initcall(xyz)"
basically means "xyz depends on the 'subsys' event, and 'subsys_end'
depends on xyz". Then within that, you might have another bus driver
that in turn depends on 'xyz'.


It would be absolutely no problem to introduce "virtual" initcalls for 
any level, e.g. just


depencies = {
everything_below
}

initcall foo()
{
return 0;
}

annotated_initcall(foo, id, dependencies),

Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 10:14 AM, Alexander Holler  wrote:
>
> Assuming three different ethernet-drivers, without any special code,
> the dependency graph would not require any special order inbetween them
> and would look like that:

This seems *fundamentally* wrong.

This is in no way specific to network drivers (or to disk drivers, or
to anything else).

So requiring extra logic for this implies that something is seriously wrong.

If two drivers aren't ordered by dependencies, they should always be
in link order, regardless of any hacks like these. If they're not,
things are wrong.

I think your problem is that you make that dependency thing a separate
ordering, so now it matters whether a driver has a dependency or not.

If something like this is to work, it has to work *with* the normal
ordering, not outside of it and then have these kinds of broken
special cases.

The normal init orderings (ie core -> postcore -> arch -> subsys -> fs
-> rootfs -> device -> late) should just be an extra dependency, I
think.

The way that you just insert the annotated dependencies in between
levels 6 and 7 ("device" and "late") can't be right. It means - for
example - that you can't have subsystems that have dependencies.

So I really think that if we do dependencies, then the current levels
have to be added as dependencies, so that "subsys_initcall(xyz)"
basically means "xyz depends on the 'subsys' event, and 'subsys_end'
depends on xyz". Then within that, you might have another bus driver
that in turn depends on 'xyz'.

Because right now afaik we do have dependencies like that, which we
sort out by link ordering (things like 'drm depends on pci' - I'm not
actually sure that one is true, but it might be).

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler

In order to provide stable interface numbers, network interface drivers
will be ordered by the link order. This is easy to accomplish by adding
dependencies.

Assuming three different ethernet-drivers, without any special code,
the dependency graph would not require any special order inbetween them
and would look like that:

eth-driver-base
   /  |   \
eth-x   eth-yeth-z

Now we just add dependencies. With the additional dependencies the graph
looks like:

 eth-driver-base
  | | |
eth-x   | |
  | | |
eth-y  -| |
  |   |
eth-z  ---|

Signed-off-by: Alexander Holler 
---
 include/linux/driver_ids.h | 11 +++
 init/dependencies.c|  9 +
 2 files changed, 20 insertions(+)

diff --git a/include/linux/driver_ids.h b/include/linux/driver_ids.h
index 60964fe..1133a68 100644
--- a/include/linux/driver_ids.h
+++ b/include/linux/driver_ids.h
@@ -15,6 +15,17 @@
 enum {
drvid_unused,
/* To be filled */
+
+   /*
+* Network drivers will be ordered according to the link order
+* (which means not necessarily according to their appearance
+* here).
+* This provides stable interface numbers.
+* Therefor their IDs have to be in the following block.
+*/
+   drvid_network_drivers_start,
+   drvid_network_drivers_end,
+
drvid_max
 };
 
diff --git a/init/dependencies.c b/init/dependencies.c
index b484f67..027fc4b 100644
--- a/init/dependencies.c
+++ b/init/dependencies.c
@@ -301,12 +301,21 @@ static int __init add_dependencies(void)
 static void __init build_inventory(void)
 {
const struct _annotated_initcall *ac;
+   unsigned id_last_network_driver = 0;
 
ac = __annotated_initcall_start;
for (; ac < __annotated_initcall_end; ++ac) {
include_node[ac->id] = true;
annotated_initcall_by_drvid[ac->id] = ac;
nvertices = max(nvertices, ac->id);
+   /* order network drivers by link order*/
+   if (ac->id > drvid_network_drivers_start &&
+   ac->id < drvid_network_drivers_end) {
+   if (id_last_network_driver)
+   add_initcall_dependency(ac->id,
+   id_last_network_driver);
+   id_last_network_driver = ac->id;
+   }
}
 }
 
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:
> Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:
> 
> >Again, parallelizing does not solve anything, and causes more problems
> >_and_ makes things take longer.  Try it, we have done it in the past and
> >proven this, it's pretty easy to test :)
> 
> Just because I'm curious, may I ask how I would test that in the easy way
> you have in mind? I've just posted the results of my tests (the patch
> series) but I wonder what you do have in mind.

Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".

Hope this helps,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 18.10.2015 um 07:14 schrieb Greg Kroah-Hartman:

On Sun, Oct 18, 2015 at 06:59:22AM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy way
you have in mind? I've just posted the results of my tests (the patch
series) but I wonder what you do have in mind.


Use the tool, scripts/bootgraph.pl to create a boot graph of your boot
sequence.  That should show you the drivers, or other areas, that are
causing your boot to be "slow".


So I've misunderstood you. I've read your paragraph as that it's easy to 
test parallelizing.



Hope this helps,


Unfortunately no, but thanks anyway. ;)

Alexander Holler

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


Just because I'm curious, may I ask how I would test that in the easy 
way you have in mind? I've just posted the results of my tests (the 
patch series) but I wonder what you do have in mind.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler

In order to provide stable interface numbers, network interface drivers
will be ordered by the link order. This is easy to accomplish by adding
dependencies.

Assuming three different ethernet-drivers, without any special code,
the dependency graph would not require any special order inbetween them
and would look like that:

eth-driver-base
   /  |   \
eth-x   eth-yeth-z

Now we just add dependencies. With the additional dependencies the graph
looks like:

 eth-driver-base
  | | |
eth-x   | |
  | | |
eth-y  -| |
  |   |
eth-z  ---|

Signed-off-by: Alexander Holler 
---
 include/linux/driver_ids.h | 11 +++
 init/dependencies.c|  9 +
 2 files changed, 20 insertions(+)

diff --git a/include/linux/driver_ids.h b/include/linux/driver_ids.h
index 60964fe..1133a68 100644
--- a/include/linux/driver_ids.h
+++ b/include/linux/driver_ids.h
@@ -15,6 +15,17 @@
 enum {
drvid_unused,
/* To be filled */
+
+   /*
+* Network drivers will be ordered according to the link order
+* (which means not necessarily according to their appearance
+* here).
+* This provides stable interface numbers.
+* Therefor their IDs have to be in the following block.
+*/
+   drvid_network_drivers_start,
+   drvid_network_drivers_end,
+
drvid_max
 };
 
diff --git a/init/dependencies.c b/init/dependencies.c
index b484f67..027fc4b 100644
--- a/init/dependencies.c
+++ b/init/dependencies.c
@@ -301,12 +301,21 @@ static int __init add_dependencies(void)
 static void __init build_inventory(void)
 {
const struct _annotated_initcall *ac;
+   unsigned id_last_network_driver = 0;
 
ac = __annotated_initcall_start;
for (; ac < __annotated_initcall_end; ++ac) {
include_node[ac->id] = true;
annotated_initcall_by_drvid[ac->id] = ac;
nvertices = max(nvertices, ac->id);
+   /* order network drivers by link order*/
+   if (ac->id > drvid_network_drivers_start &&
+   ac->id < drvid_network_drivers_end) {
+   if (id_last_network_driver)
+   add_initcall_dependency(ac->id,
+   id_last_network_driver);
+   id_last_network_driver = ac->id;
+   }
}
 }
 
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 10:14 AM, Alexander Holler  wrote:
>
> Assuming three different ethernet-drivers, without any special code,
> the dependency graph would not require any special order inbetween them
> and would look like that:

This seems *fundamentally* wrong.

This is in no way specific to network drivers (or to disk drivers, or
to anything else).

So requiring extra logic for this implies that something is seriously wrong.

If two drivers aren't ordered by dependencies, they should always be
in link order, regardless of any hacks like these. If they're not,
things are wrong.

I think your problem is that you make that dependency thing a separate
ordering, so now it matters whether a driver has a dependency or not.

If something like this is to work, it has to work *with* the normal
ordering, not outside of it and then have these kinds of broken
special cases.

The normal init orderings (ie core -> postcore -> arch -> subsys -> fs
-> rootfs -> device -> late) should just be an extra dependency, I
think.

The way that you just insert the annotated dependencies in between
levels 6 and 7 ("device" and "late") can't be right. It means - for
example - that you can't have subsystems that have dependencies.

So I really think that if we do dependencies, then the current levels
have to be added as dependencies, so that "subsys_initcall(xyz)"
basically means "xyz depends on the 'subsys' event, and 'subsys_end'
depends on xyz". Then within that, you might have another bus driver
that in turn depends on 'xyz'.

Because right now afaik we do have dependencies like that, which we
sort out by link ordering (things like 'drm depends on pci' - I'm not
actually sure that one is true, but it might be).

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:
>
> I'm making dependencies the only ordering for annotated initcalls.

Yeah, and quite frankly, that just means that I'm not going to merge it.

We do not do "flag-day" things. We've done them in the past, and it
has always been a major and unacceptable pain.

And if the dependency ordering is "outside" of the traditional
link-time ordering, then it is by definition a flag-day event. Every
time you add a dependency to even just *one* driver, it magically
changes ordering wrt all the other drivers, because now it's in a
different ordering domain.

So please reconsider. Or stop cc'ing me. Because "flag day" changes
really are not acceptable.

   Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sat, Oct 17, 2015 at 08:37:35PM +0200, Alexander Holler wrote:
> I'm making dependencies the only ordering for annotated initcalls.
> 
> Otherwise it's impossible to call initcalls in parallel.

We don't ever want to call initcalls in parallel, unless they can
properly handle it.  All drivers can tell the driver core to use
parallel probing if they know they can handle it.  See 'enum probe_type'
for this.

Trying to call initcalls or driver probes in parallel blindly has been
proven to both take a longer amount of time, and cause problems for
drivers that aren't expecting it.  The research on this was done years
ago when people were starting to work on booting quicker, and the end
result is what we have now where drivers that know they can handle this,
can tell the core to let them probe in this manner.

The outcome of this work is the sub-second boot time we have today.  If
your systems are taking longer to boot, then please look into the
drivers that are causing this, that is where the real issue is, not in
the core of the kernel.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:03 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:


Otherwise it's impossible to call initcalls in parallel. I've seen a stable
topological sort somewhere, but whenever you want to parallelize the
initcalls, the stable ordering would be gone anyway. So I've decided not to
look further at a stable topological sort.


So five seconds of googling gave me freely usable source code for a
stable topological sort, that also has a nice reported added
advantage:

  "An interesting property of a stable topological sort is that cyclic
dependencies are tolerated and resolved according to original order of
elements in sequence. This is a desirable feature for many
applications because it allows to sort any sequence with any
imaginable dependencies between the elements"

which seems to be *exactly* what you'd want, especially considering
that right now your patches add extra "no-dependency" markers exactly
because of the cyclical problem.


That's the stable topological sort I've mentioned the link to in the 
discussion with you.




I think it was the #2 hit on google for "stable topological sort". I
didn't look closely at the source code, but it was not big.

And no, since we don't actually want to parallelize the initcalls
anyway (I had this discussion with you just a month ago), your
objections seem even more questionable. We have separate machinery for
"do this asynchronously", and we want to _keep_ that separate.


I've understood that now.

Sorry for wasting your time.

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:
>
> That isn't a flag day thing. It's a config option everyone can turn on and
> off whenever he wants.

That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 20:23 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 10:14 AM, Alexander Holler  wrote:


Assuming three different ethernet-drivers, without any special code,
the dependency graph would not require any special order inbetween them
and would look like that:


This seems *fundamentally* wrong.

This is in no way specific to network drivers (or to disk drivers, or
to anything else).

So requiring extra logic for this implies that something is seriously wrong.

If two drivers aren't ordered by dependencies, they should always be
in link order, regardless of any hacks like these. If they're not,
things are wrong.

I think your problem is that you make that dependency thing a separate
ordering, so now it matters whether a driver has a dependency or not.


I'm making dependencies the only ordering for annotated initcalls.

Otherwise it's impossible to call initcalls in parallel. I've seen a 
stable topological sort somewhere, but whenever you want to parallelize 
the initcalls, the stable ordering would be gone anyway. So I've decided 
not to look further at a stable topological sort.



If something like this is to work, it has to work *with* the normal
ordering, not outside of it and then have these kinds of broken
special cases.

The normal init orderings (ie core -> postcore -> arch -> subsys -> fs
-> rootfs -> device -> late) should just be an extra dependency, I
think.

The way that you just insert the annotated dependencies in between
levels 6 and 7 ("device" and "late") can't be right. It means - for
example - that you can't have subsystems that have dependencies.


Sorry, but that's wrong.

I've choosen to place initcalls between 5 and 6 to make it easier to 
move both, subsystems as well as normal drivers to the (new) level with 
annotated initcalls. If you look at what I've already "annotated", you 
will see there are quiet a lot initcalls I've moved from below 6 to the 
new level.



So I really think that if we do dependencies, then the current levels
have to be added as dependencies, so that "subsys_initcall(xyz)"
basically means "xyz depends on the 'subsys' event, and 'subsys_end'
depends on xyz". Then within that, you might have another bus driver
that in turn depends on 'xyz'.


It would be absolutely no problem to introduce "virtual" initcalls for 
any level, e.g. just


depencies = {
everything_below
}

initcall foo()
{
return 0;
}

annotated_initcall(foo, id, dependencies),

Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 20:52 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:


I'm making dependencies the only ordering for annotated initcalls.


Yeah, and quite frankly, that just means that I'm not going to merge it.

We do not do "flag-day" things. We've done them in the past, and it
has always been a major and unacceptable pain.


That isn't a flag day thing. It's a config option everyone can turn on 
and off whenever he wants.




And if the dependency ordering is "outside" of the traditional
link-time ordering, then it is by definition a flag-day event. Every
time you add a dependency to even just *one* driver, it magically
changes ordering wrt all the other drivers, because now it's in a
different ordering domain.

So please reconsider. Or stop cc'ing me. Because "flag day" changes
really are not acceptable.


I'm choosing the second option. I will answer further mails on that 
topic, but will remove you from cc. Besides that I consider these 
patches as a total failure and will not post any further version.


Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 11:37 AM, Alexander Holler  wrote:
>
> Otherwise it's impossible to call initcalls in parallel. I've seen a stable
> topological sort somewhere, but whenever you want to parallelize the
> initcalls, the stable ordering would be gone anyway. So I've decided not to
> look further at a stable topological sort.

So five seconds of googling gave me freely usable source code for a
stable topological sort, that also has a nice reported added
advantage:

 "An interesting property of a stable topological sort is that cyclic
dependencies are tolerated and resolved according to original order of
elements in sequence. This is a desirable feature for many
applications because it allows to sort any sequence with any
imaginable dependencies between the elements"

which seems to be *exactly* what you'd want, especially considering
that right now your patches add extra "no-dependency" markers exactly
because of the cyclical problem.

I think it was the #2 hit on google for "stable topological sort". I
didn't look closely at the source code, but it was not big.

And no, since we don't actually want to parallelize the initcalls
anyway (I had this discussion with you just a month ago), your
objections seem even more questionable. We have separate machinery for
"do this asynchronously", and we want to _keep_ that separate.

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


It's impossible to take it into account because I don't want to miss the 
parallelize functionality. And without that, all the stuff doesn't offer 
enough benefits to be worse the effort but just adds some time necessary 
to do the sorting. It might solve the deferred probe problems, but 
without much benefit.


Regards,

Alexander Holler

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Greg Kroah-Hartman

On Sat, Oct 17, 2015 at 09:14:34PM +0200, Alexander Holler wrote:
> Am 17.10.2015 um 21:08 schrieb Linus Torvalds:
> >On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  
> >wrote:
> >>
> >>That isn't a flag day thing. It's a config option everyone can turn on and
> >>off whenever he wants.
> >
> >That's a flag-day thing. We've done it (drm comes to mind - several times).
> >
> >I'm disappointed, because I _know_ I pointed you in the direction of
> >stable sorting about a month ago. I really had hoped you'd have taken
> >that into account.
> 
> It's impossible to take it into account because I don't want to miss the
> parallelize functionality. And without that, all the stuff doesn't offer
> enough benefits to be worse the effort but just adds some time necessary to
> do the sorting. It might solve the deferred probe problems, but without much
> benefit.

Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Linus Torvalds

On Sat, Oct 17, 2015 at 12:14 PM, Alexander Holler  wrote:
>
> It's impossible to take it into account because I don't want to miss the
> parallelize functionality. And without that, all the stuff doesn't offer
> enough benefits to be worse the effort but just adds some time necessary to
> do the sorting. It might solve the deferred probe problems, but without much
> benefit.

Quite frankly, the alleged advantage of parallelization of the init
sequence is *very* questionable.

But even if you were to want to parallelize things at that level, you'd still

 (a) want to make *that* a flag (to debug) and make it opt-in (so
you'd have another "this module can be done in parallel" marker)

 (b) end up with all the other drivers *not* being parallel, because
they were all written to not have everything happen at once.

 (c) want to avoid the whole flag-day things

so even then you'd want to sort everything using that topological
stable sort, so that you can sanely mix things, and then mark
individual drivers as "I'd like to run in parallel with the rest of
the init sequence".

The main thing that being parallel would change is likely that you'd
also have to add some kind of "I'm done" completion for each module
(or dependency point), so that things that depend on other things
(that have *started*) can explicitly wait for those other things to
actually end.

We have something like that for our "sync" points right now with the
whole asynchronous machinery. It's somewhat general (although I think
almost all users end up just doing "async_synchronize_full()", even
though we do have a cookie-based one) and I suspect the cookie-based
one could be used for individual drivers and their completions.

But I really think the whole "do initcalls in parallel" is a very bad
idea _regardless_ of anything else. We almost never actually have
initcalls that really take a lot of time, and when we do, we already
handle the big ones with the async machinery, which is actually a lot
more powerful and flexible than some "run initcalls in parallel",
exactly because you can run *parts* serially (like basic discovery),
and then run other parts asynchronously (like "spin up disk" or "wait
for power stable" or whatever - *after* you have discovered the
device).

So I personally think our async thing is much more powerful and useful
than some random "let's run initcalls in parallel".

 Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:36 schrieb Greg Kroah-Hartman:

On Sat, Oct 17, 2015 at 09:14:34PM +0200, Alexander Holler wrote:

Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


It's impossible to take it into account because I don't want to miss the
parallelize functionality. And without that, all the stuff doesn't offer
enough benefits to be worse the effort but just adds some time necessary to
do the sorting. It might solve the deferred probe problems, but without much
benefit.


Again, parallelizing does not solve anything, and causes more problems
_and_ makes things take longer.  Try it, we have done it in the past and
proven this, it's pretty easy to test :)


I've tested it, otherwise I wouldn't have posted the patches.

Unfortunately it's quiet a lot of work to add dependencies for everything.

So maybe I'm able to offer some better numbers in a year or such, when I 
was bored often enough to add more dependencies for initcalls.


Not to mention that small changes in the order can have quiet big 
differences in the boot time, so it's quiet hard to parallelize stuff 
(add dependencies) correctly like e.g. the pci/acpi/processor stuff. 
Especially because many reasons for the current order aren't mentioned 
in the source and are hard to see without specific knowledge about the HW.


Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:58 schrieb Alexander Holler:


Unfortunately it's quiet a lot of work to add dependencies for everything.


And (just in case of), also I'm a non-native English writer, I know the 
difference between quiet and quite. But, unfortunately, that doesn't 
prevent me to type it wrong. It's like "teh" instead of "the". 
Unfortunately I'm not very good in writing English fast without errors, 
but that doesn't mean I'm dumb (also many native English writers prefer 
to assume that).


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 04/14] init: deps: order network interfaces by link order

2015-10-17 Thread Alexander Holler


Am 17.10.2015 um 21:08 schrieb Linus Torvalds:

On Sat, Oct 17, 2015 at 12:01 PM, Alexander Holler  wrote:


That isn't a flag day thing. It's a config option everyone can turn on and
off whenever he wants.


That's a flag-day thing. We've done it (drm comes to mind - several times).

I'm disappointed, because I _know_ I pointed you in the direction of
stable sorting about a month ago. I really had hoped you'd have taken
that into account.


Sorry, but I have to add another comment about that stable topological 
sort which can be easily found by searching the web.


As the author mentioned, he has done some search on that topic too and 
has found nothing. So, either I would have to trust his word (and his 
non-public proof) that it works or I would have to prove it myself.


Besides that he says it's based on bubble sort, which is known for its 
speed. So I would have had to write in C in order to see if the speed 
would be acceptable and would still be without any proof that it always 
works correctly.


Regards,

Alexander Holler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

46 matches

Mail list logo