Re: System init and OS eventq ensure

[marko kiiskila]

> On Dec 11, 2016, at 11:21 AM, Sterling Hughes  wrote:
> 
> Hi,
> 
>> 
>>> On Dec 11, 2016, at 10:55 AM, Christopher Collins  
>>> wrote:
>>> 
>>> On Sun, Dec 11, 2016 at 10:11:44AM -0800, will sanfilippo wrote:
 Personally, I keep wanting to try and have the OS start up right away.
>>> 
>>> I wonder if this could solve the problem that Sterling raised (no
>>> default event queue during sysinit).  The control flow in main() might
>>> look like this:
>>> 
>>> 1. Start OS
>>> 2. Create and designate default event queue.
>>> 3. sysinit()
>>> 
>>> I think it would be nice if we could avoid adding another initialization
>>> stage.
>>> 
> 
> +1

I agree. If there are too many options here, it becomes even harder to
understand. Preferably we should keep this stuff simple, making the barrier of
entry to development lower.

> 
 There are definitely “issues” with this:
 a) We do not want to waste idle task stack.
 b) When tasks are started they would start running right away. This
 might cause issues where a task does something to a piece of memory
 that another task initializes, but since that other task has not
 initialized it yet…
 
 b) can be avoided by locking the scheduler until initializations are 
 finished.
 
 a) is problematic :-) I think someone brought this up before, but I
 wonder if it is worth the effort to do something “a bit crazy” like
 the following: the idle task uses “the heap” during intialization.
 Once initializations are over (or at some point that we determine),
 the idle task stack is made smaller and the “top” of the heap is set
 to the end of the idle task stack. For example, idle task stack is at
 0x20008000 and is of size 1K bytes; the bottom of the heap is at
 0x20007000; the top of the heap is at 0x20007C00 (in my nomenclature,
 heap allocations start from the bottom). At some point, the top of the
 heap is moved to 0x20007F80.
 
 Yeah, maybe a bit crazy… :-)
>>> 
>>> I don't think that's too crazy.  It would be great if we could just
>>> malloc() a temporary stack, and then free it when initialization
>>> completes.  I guess the worry is that this will cause heap
>>> fragmentation?
>>> 
> 
> I’m not crazy about malloc()’ing this space.  Especially since system init 
> (where we’d use this memory) is where people malloc() their memory pools, and 
> so you have 1K of space that could potentially affect memory exhaustion.  
> Maybe its an awful idea… but why not let people specify the startup task 
> stack, and we can guarantee that this task gets deleted before the rest of 
> the tasks/system runs.  That way, you can just choose one of your task’s 
> stacks that is sufficiently large, and use that for startup stack.
> 

Most of the malloc()’s for packages happen when they’re initialized.
And having malloc()’d init stack present during this step will have effect
on this. And freeing the stack right after will automatically lead to heap
fragmentation.
And we’d need some new, mandatory, architecture specific code which
switches stacks for a task while the task is running. While not complex,
it is yet another thing to write/debug when adding a new architecture.

However, I like the idea of app assigning a startup task, and then executing
sysinit() in that task’s context.

Re: System init and OS eventq ensure

[Sterling Hughes]

Hi,



On Dec 11, 2016, at 10:55 AM, Christopher Collins 
 wrote:


On Sun, Dec 11, 2016 at 10:11:44AM -0800, will sanfilippo wrote:
Personally, I keep wanting to try and have the OS start up right 
away.


I wonder if this could solve the problem that Sterling raised (no
default event queue during sysinit).  The control flow in main() 
might

look like this:

1. Start OS
2. Create and designate default event queue.
3. sysinit()

I think it would be nice if we could avoid adding another 
initialization

stage.



+1


There are definitely “issues” with this:
a) We do not want to waste idle task stack.
b) When tasks are started they would start running right away. This
might cause issues where a task does something to a piece of memory
that another task initializes, but since that other task has not
initialized it yet…

b) can be avoided by locking the scheduler until initializations are 
finished.


a) is problematic :-) I think someone brought this up before, but I
wonder if it is worth the effort to do something “a bit crazy” 
like
the following: the idle task uses “the heap” during 
intialization.

Once initializations are over (or at some point that we determine),
the idle task stack is made smaller and the “top” of the heap is 
set
to the end of the idle task stack. For example, idle task stack is 
at

0x20008000 and is of size 1K bytes; the bottom of the heap is at
0x20007000; the top of the heap is at 0x20007C00 (in my 
nomenclature,
heap allocations start from the bottom). At some point, the top of 
the

heap is moved to 0x20007F80.

Yeah, maybe a bit crazy… :-)


I don't think that's too crazy.  It would be great if we could just
malloc() a temporary stack, and then free it when initialization
completes.  I guess the worry is that this will cause heap
fragmentation?



I’m not crazy about malloc()’ing this space.  Especially since 
system init (where we’d use this memory) is where people malloc() 
their memory pools, and so you have 1K of space that could potentially 
affect memory exhaustion.  Maybe its an awful idea… but why not let 
people specify the startup task stack, and we can guarantee that this 
task gets deleted before the rest of the tasks/system runs.  That way, 
you can just choose one of your task’s stacks that is sufficiently 
large, and use that for startup stack.


Sterling

Re: System init and OS eventq ensure

[will sanfilippo]

I guess, for no really great reason, I thought it would be weird to malloc, 
say, 1024 bytes, then free, say, 960 bytes. No weirder than what I was 
suggesting. :-) I guess there a number of things we could do here: malloc a 
temporary stack and free that whole thing and either do another malloc or 
change the task stack information to a bss defined idle task stack.

I did leave out the fact that we would need to modify the stack information in 
the task structure. I cannot recall exactly the information we keep in the task 
structure for the stack but this would not be hard to do; we would just need to 
do it.

Will

> On Dec 11, 2016, at 10:55 AM, Christopher Collins  wrote:
> 
> On Sun, Dec 11, 2016 at 10:11:44AM -0800, will sanfilippo wrote:
>> Personally, I keep wanting to try and have the OS start up right away.
> 
> I wonder if this could solve the problem that Sterling raised (no
> default event queue during sysinit).  The control flow in main() might
> look like this:
> 
> 1. Start OS
> 2. Create and designate default event queue.
> 3. sysinit()
> 
> I think it would be nice if we could avoid adding another initialization
> stage.
> 
>> There are definitely “issues” with this:
>> a) We do not want to waste idle task stack.
>> b) When tasks are started they would start running right away. This
>> might cause issues where a task does something to a piece of memory
>> that another task initializes, but since that other task has not
>> initialized it yet…
>> 
>> b) can be avoided by locking the scheduler until initializations are 
>> finished.
>> 
>> a) is problematic :-) I think someone brought this up before, but I
>> wonder if it is worth the effort to do something “a bit crazy” like
>> the following: the idle task uses “the heap” during intialization.
>> Once initializations are over (or at some point that we determine),
>> the idle task stack is made smaller and the “top” of the heap is set
>> to the end of the idle task stack. For example, idle task stack is at
>> 0x20008000 and is of size 1K bytes; the bottom of the heap is at
>> 0x20007000; the top of the heap is at 0x20007C00 (in my nomenclature,
>> heap allocations start from the bottom). At some point, the top of the
>> heap is moved to 0x20007F80.
>> 
>> Yeah, maybe a bit crazy… :-)
> 
> I don't think that's too crazy.  It would be great if we could just
> malloc() a temporary stack, and then free it when initialization
> completes.  I guess the worry is that this will cause heap
> fragmentation?
> 
> Chris

Re: System init and OS eventq ensure

[Christopher Collins]

On Sun, Dec 11, 2016 at 10:11:44AM -0800, will sanfilippo wrote:
> Personally, I keep wanting to try and have the OS start up right away.

I wonder if this could solve the problem that Sterling raised (no
default event queue during sysinit).  The control flow in main() might
look like this:

1. Start OS
2. Create and designate default event queue.
3. sysinit()

I think it would be nice if we could avoid adding another initialization
stage.

> There are definitely “issues” with this:
> a) We do not want to waste idle task stack.
> b) When tasks are started they would start running right away. This
> might cause issues where a task does something to a piece of memory
> that another task initializes, but since that other task has not
> initialized it yet…
> 
> b) can be avoided by locking the scheduler until initializations are finished.
> 
> a) is problematic :-) I think someone brought this up before, but I
> wonder if it is worth the effort to do something “a bit crazy” like
> the following: the idle task uses “the heap” during intialization.
> Once initializations are over (or at some point that we determine),
> the idle task stack is made smaller and the “top” of the heap is set
> to the end of the idle task stack. For example, idle task stack is at
> 0x20008000 and is of size 1K bytes; the bottom of the heap is at
> 0x20007000; the top of the heap is at 0x20007C00 (in my nomenclature,
> heap allocations start from the bottom). At some point, the top of the
> heap is moved to 0x20007F80.
> 
> Yeah, maybe a bit crazy… :-)

I don't think that's too crazy.  It would be great if we could just
malloc() a temporary stack, and then free it when initialization
completes.  I guess the worry is that this will cause heap
fragmentation?

Chris

Re: System init and OS eventq ensure

[will sanfilippo]

Personally, I keep wanting to try and have the OS start up right away. There 
are definitely “issues” with this:
a) We do not want to waste idle task stack.
b) When tasks are started they would start running right away. This might cause 
issues where a task does something to a piece of memory that another task 
initializes, but since that other task has not initialized it yet…

b) can be avoided by locking the scheduler until initializations are finished.

a) is problematic :-) I think someone brought this up before, but I wonder if 
it is worth the effort to do something “a bit crazy” like the following: the 
idle task uses “the heap” during intialization. Once initializations are over 
(or at some point that we determine), the idle task stack is made smaller and 
the “top” of the heap is set to the end of the idle task stack. For example, 
idle task stack is at 0x20008000 and is of size 1K bytes; the bottom of the 
heap is at 0x20007000; the top of the heap is at 0x20007C00 (in my 
nomenclature, heap allocations start from the bottom). At some point, the top 
of the heap is moved to 0x20007F80.

Yeah, maybe a bit crazy… :-)


> On Dec 10, 2016, at 4:04 PM, Sterling Hughes  wrote:
> 
> Hi Chris,
> 
> On 10 Dec 2016, at 13:37, Christopher Collins wrote:
> 
>> Darn, you're right. I'm writing these emails from my phone, and I didn't
>> look at the code closely enough.  For other packages, the start event
>> only gets executed the first time the event queue gets used (as you
>> said).  I guess it has worked out in practice because the application
>> uses the package shortly after the OS starts.
>> 
> 
> Yeah, that’s what I noticed too. :-)
> 
> For now, it’s OK, I can just call my init from main() after the default event 
> queue is set.
> 
>> That's not so great.  Second stage initialization sounds good to me.
>> Alternatively, the system could keep track of packages that need an
>> event queue, and enqueue their start event when a default event queue is
>> set.  Earlier, we discussed using linker sections to accomplish this
>> without requiring any RAM. I looked into this, but concluded it wasn't
>> possible without modifying the linker scripts.
>> 
> 
> I think we should probably use this opportunity to (again) review system 
> initialization, which is fast becoming our most circularly discussed topic 
> :-) I’d throw in there whether or not to initialize components in the idle 
> task (using up stack), and making sure that we map sysinit stages to the 
> driver initialization stages as well.
> 
> As far as a proposal, what do you think of having 2 initialization stages:
> 
> - Before OS
> - After OS
> 
> And we can break each stage into primary, secondary and tertiary order, so, 
> in terms of number space we have:
> 
> - 0: first init order, before OS
> - 1: second init order, before OS
> - 2: third init order, before OS
> - 3: first init order, after OS
> - 4: second init order, after OS
> - 5: third init order, after OS
> 
> I think we probably need to modify the package system configuration to 
> specify both stage & order, e.g.
> 
> pkg.init_func_startup.name: XX
> pkg.init_func_startup.order: 0
> pkg.init_func_kernel.name: YY
> pkg.init_func_kernel.order: 1
> 
> This should allow us to hook in at either of these stages.
> 
> I also think we probably need to give meaning to at least the primary and 
> secondary init orders here, e.g. designate which services are available after 
> each of these functions and come up with some documented nomenclature for it.
> 
> Sterling
> 

Re: System init and OS eventq ensure

[Christopher Collins]

Darn, you're right. I'm writing these emails from my phone, and I didn't
look at the code closely enough.  For other packages, the start event
only gets executed the first time the event queue gets used (as you
said).  I guess it has worked out in practice because the application
uses the package shortly after the OS starts.

That's not so great.  Second stage initialization sounds good to me.
Alternatively, the system could keep track of packages that need an
event queue, and enqueue their start event when a default event queue is
set.  Earlier, we discussed using linker sections to accomplish this
without requiring any RAM. I looked into this, but concluded it wasn't
possible without modifying the linker scripts.

Chris

On Sat, Dec 10, 2016 at 12:30:39PM -0800, Sterling Hughes wrote:
> How do you assign an event queue if you are relying on the default event 
> queue being there?  
> 
> Can you point me to an example of where this is done? 
> 
> Sterling 
> 
> > On Dec 10, 2016, at 12:08 PM, Christopher Collins  
> > wrote:
> > 
> > The way other packages handle this is they enqueue the startup event
> > when their event queue is assigned.  This happens automatically when you
> > call os_eventq_designate(); the last parameter is the event to enqueue
> > immediately.
> > 
> > Chris
> > 
> >> On Sat, Dec 10, 2016 at 11:30:27AM -0800, Sterling Hughes wrote:
> >> Hi,
> >> 
> >> I’m looking at using the default eventq (or allowing for it), in a 
> >> library I’m working on.  In order to do that, I have a function:
> >> 
> >> static struct os_eventq *
> >> sensor_mgr_evq_get(void)
> >> {
> >> os_eventq_ensure(_mgr.mgr_eventq, NULL);
> >> 
> >> return (sensor_mgr.mgr_eventq);
> >> }
> >> 
> >> And this function gets called within my package’s sysinit, as I want 
> >> to schedule a callout to run this function immediately on bootup:
> >> 
> >> /**
> >>  * Initialize sensor polling callout and set it to fire on boot.
> >>  */
> >> os_callout_init(_mgr.mgr_wakeup_callout, 
> >> sensor_mgr_evq_get(),
> >> sensor_mgr_wakeup_event, NULL);
> >> os_callout_reset(_mgr.mgr_wakeup_callout, 0);
> >> 
> >> The problem is that the default event queue is not setup until after 
> >> sysinit executes, as task setup is later on.
> >> 
> >> What is the right way to do this?  For now, I can move the 
> >> initialization from sysinit and to the main() function at task level, 
> >> however, I don’t think this is how we want to manage initialization 
> >> over time.  We probably need some way for a package to have a system 
> >> initialization stage that runs after the OS has started, and the default 
> >> event queue has been set.
> >> 
> >> Cheers,
> >> 
> >> Sterling

Re: System init and OS eventq ensure

[Sterling Hughes]

How do you assign an event queue if you are relying on the default event queue 
being there?  

Can you point me to an example of where this is done? 

Sterling 

> On Dec 10, 2016, at 12:08 PM, Christopher Collins  wrote:
> 
> The way other packages handle this is they enqueue the startup event
> when their event queue is assigned.  This happens automatically when you
> call os_eventq_designate(); the last parameter is the event to enqueue
> immediately.
> 
> Chris
> 
>> On Sat, Dec 10, 2016 at 11:30:27AM -0800, Sterling Hughes wrote:
>> Hi,
>> 
>> I’m looking at using the default eventq (or allowing for it), in a 
>> library I’m working on.  In order to do that, I have a function:
>> 
>> static struct os_eventq *
>> sensor_mgr_evq_get(void)
>> {
>> os_eventq_ensure(_mgr.mgr_eventq, NULL);
>> 
>> return (sensor_mgr.mgr_eventq);
>> }
>> 
>> And this function gets called within my package’s sysinit, as I want 
>> to schedule a callout to run this function immediately on bootup:
>> 
>> /**
>>  * Initialize sensor polling callout and set it to fire on boot.
>>  */
>> os_callout_init(_mgr.mgr_wakeup_callout, 
>> sensor_mgr_evq_get(),
>> sensor_mgr_wakeup_event, NULL);
>> os_callout_reset(_mgr.mgr_wakeup_callout, 0);
>> 
>> The problem is that the default event queue is not setup until after 
>> sysinit executes, as task setup is later on.
>> 
>> What is the right way to do this?  For now, I can move the 
>> initialization from sysinit and to the main() function at task level, 
>> however, I don’t think this is how we want to manage initialization 
>> over time.  We probably need some way for a package to have a system 
>> initialization stage that runs after the OS has started, and the default 
>> event queue has been set.
>> 
>> Cheers,
>> 
>> Sterling

Re: System init and OS eventq ensure

[Christopher Collins]

The way other packages handle this is they enqueue the startup event
when their event queue is assigned.  This happens automatically when you
call os_eventq_designate(); the last parameter is the event to enqueue
immediately.

Chris

On Sat, Dec 10, 2016 at 11:30:27AM -0800, Sterling Hughes wrote:
> Hi,
> 
> I’m looking at using the default eventq (or allowing for it), in a 
> library I’m working on.  In order to do that, I have a function:
> 
> static struct os_eventq *
> sensor_mgr_evq_get(void)
> {
>  os_eventq_ensure(_mgr.mgr_eventq, NULL);
> 
>  return (sensor_mgr.mgr_eventq);
> }
> 
> And this function gets called within my package’s sysinit, as I want 
> to schedule a callout to run this function immediately on bootup:
> 
>  /**
>   * Initialize sensor polling callout and set it to fire on boot.
>   */
>  os_callout_init(_mgr.mgr_wakeup_callout, 
> sensor_mgr_evq_get(),
>  sensor_mgr_wakeup_event, NULL);
>  os_callout_reset(_mgr.mgr_wakeup_callout, 0);
> 
> The problem is that the default event queue is not setup until after 
> sysinit executes, as task setup is later on.
> 
> What is the right way to do this?  For now, I can move the 
> initialization from sysinit and to the main() function at task level, 
> however, I don’t think this is how we want to manage initialization 
> over time.  We probably need some way for a package to have a system 
> initialization stage that runs after the OS has started, and the default 
> event queue has been set.
> 
> Cheers,
> 
> Sterling