Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread Seth Arnold
On Sat, Nov 28, 2015 at 11:08:05AM -0800, John Johansen wrote: > +static void setup_parallel_compile(void) > +{ > + /* jobs_count and paralell_max set by default, config or args */ > + long n = sysconf(_SC_NPROCESSORS_ONLN); We should handle an error return here .. > + if (jobs_count

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread Seth Arnold
On Sat, Nov 28, 2015 at 09:02:18PM -0800, John Johansen wrote: > This check is well above the range of values I would recommend (some > where between 1-2x the number of cpus. More jobs can help with smaller Two times makes more sense for most CPUs but eight may be more appropriate for e.g. POWER8

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread John Johansen
On 11/30/2015 12:41 PM, Seth Arnold wrote: > On Sat, Nov 28, 2015 at 11:08:05AM -0800, John Johansen wrote: >> +static void setup_parallel_compile(void) >> +{ >> +/* jobs_count and paralell_max set by default, config or args */ >> +long n = sysconf(_SC_NPROCESSORS_ONLN); > > We should

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread John Johansen
On 11/30/2015 12:48 PM, Seth Arnold wrote: > On Sat, Nov 28, 2015 at 09:02:18PM -0800, John Johansen wrote: >> This check is well above the range of values I would recommend (some >> where between 1-2x the number of cpus. More jobs can help with smaller > > Two times makes more sense for most

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread John Johansen
On 11/29/2015 03:11 PM, Christian Boltz wrote: > Hello, > > Am Samstag, 28. November 2015 schrieb John Johansen: >> On 11/28/2015 01:54 PM, Christian Boltz wrote: >>> Am Samstag, 28. November 2015 schrieb John Johansen: > ... >>> So the parser will error out if a too big job number is given _and_

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread apparmor
appar...@raf.org wrote: > Seth Arnold wrote: > > > On Sat, Nov 28, 2015 at 09:02:18PM -0800, John Johansen wrote: > > > This check is well above the range of values I would recommend (some > > > where between 1-2x the number of cpus. More jobs can help with smaller > > > > Two times makes more

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread apparmor
Seth Arnold wrote: > On Sat, Nov 28, 2015 at 09:02:18PM -0800, John Johansen wrote: > > This check is well above the range of values I would recommend (some > > where between 1-2x the number of cpus. More jobs can help with smaller > > Two times makes more sense for most CPUs but eight may be

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-30 Thread John Johansen
On 11/30/2015 02:42 PM, appar...@raf.org wrote: > appar...@raf.org wrote: > >> Seth Arnold wrote: >> >>> On Sat, Nov 28, 2015 at 09:02:18PM -0800, John Johansen wrote: This check is well above the range of values I would recommend (some where between 1-2x the number of cpus. More jobs

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-29 Thread Christian Boltz
Hello, Am Samstag, 28. November 2015 schrieb John Johansen: > On 11/28/2015 01:54 PM, Christian Boltz wrote: > > Am Samstag, 28. November 2015 schrieb John Johansen: ... > > So the parser will error out if a too big job number is given _and_ > > if there are enough profiles to load (otherwise the

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-28 Thread John Johansen
v2 switch to default --jobs=auto check and bail on bad debug argument update comments on work_sync_one() check for an invalid number of maximum jobs() put an upper limit on the maximum number of jobs to 8*# of cpus --- commit ce5d771201ca36530072e328ea2ed6824e7dc859 Author: John Johansen

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-28 Thread Christian Boltz
Hello, Am Samstag, 28. November 2015 schrieb John Johansen: > v2 > switch to default --jobs=auto > check and bail on bad debug argument > update comments on work_sync_one() > check for an invalid number of maximum jobs() > put an upper limit on the maximum number of jobs to 8*# of cpus >

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-28 Thread John Johansen
On 11/28/2015 01:54 PM, Christian Boltz wrote: > Hello, > > Am Samstag, 28. November 2015 schrieb John Johansen: >> v2 >> switch to default --jobs=auto >> check and bail on bad debug argument >> update comments on work_sync_one() >> check for an invalid number of maximum jobs() >> put an upper

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-20 Thread Seth Arnold
On Sat, Oct 17, 2015 at 03:49:17PM -0700, John Johansen wrote: > This adds a basic support for parallel compiles. It uses a fork()/wait > @@ -286,8 +324,13 @@ static int process_arg(int c, char *optarg) > option = OPTION_ADD; > break; > case 'd': > -

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-11-20 Thread John Johansen
On 11/20/2015 04:01 PM, Seth Arnold wrote: > On Sat, Oct 17, 2015 at 03:49:17PM -0700, John Johansen wrote: >> This adds a basic support for parallel compiles. It uses a fork()/wait >> @@ -286,8 +324,13 @@ static int process_arg(int c, char *optarg) >> option = OPTION_ADD; >>

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-26 Thread Seth Arnold
On Fri, Oct 23, 2015 at 12:47:03PM -0700, John Johansen wrote: > So one thing I wanted to ask here was whether or not we should default this > to --jobs=auto instead of 1 We may need some tweaking to the algorithm to get good results on phones, desktops, and huge servers, but --jobs=auto is

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-25 Thread John Johansen
On 10/24/2015 06:05 AM, Christian Boltz wrote: > Hello, > > Am Freitag, 23. Oktober 2015 schrieb John Johansen: >> So one thing I wanted to ask here was whether or not we should default >> this to --jobs=auto instead of 1 >> >> 1 is safe in that it is the current behavior, but I think we want

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-24 Thread Christian Boltz
Hello, Am Freitag, 23. Oktober 2015 schrieb John Johansen: > So one thing I wanted to ask here was whether or not we should default > this to --jobs=auto instead of 1 > > 1 is safe in that it is the current behavior, but I think we want this > on by default So we can choose between telling our

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-23 Thread intrigeri
John Johansen wrote (17 Oct 2015 22:49:17 GMT) : > -j or -jauto or --jobs=auto sets the max jobs to the # of cpus Thanks! I can't wait to see this merged upstream (sorry I have no time to test it here). This will be immensely helpful for replacing shell code when migrating stuff to

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-23 Thread John Johansen
On 10/23/2015 05:06 AM, intrigeri wrote: > John Johansen wrote (17 Oct 2015 22:49:17 GMT) : >> -j or -jauto or --jobs=auto sets the max jobs to the # of cpus > > Thanks! I can't wait to see this merged upstream (sorry I have no time > to test it here). This will be immensely helpful for

Re: [apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-23 Thread John Johansen
On 10/17/2015 03:49 PM, John Johansen wrote: > This adds a basic support for parallel compiles. It uses a fork()/wait > model due to the parsers current dependence on global variables and > structures. It has been setup in a similar manner to how cilk handles > multithreading to make it easy to

[apparmor] [PATCH] parser: add basic support for parallel compiles and loads

2015-10-17 Thread John Johansen
This adds a basic support for parallel compiles. It uses a fork()/wait model due to the parsers current dependence on global variables and structures. It has been setup in a similar manner to how cilk handles multithreading to make it easy to port to a managed thread model once the parser removes