Re: Newt feature: run custom commands at build time

[Christopher Collins]

On Wed, Sep 25, 2019 at 06:29:19PM +0300, marko kiiskila wrote:
> Sounds good. If we need to add more, we can do that later.

Thanks.  I have updated the PR with the discussed changes:


Chris

Re: Newt feature: run custom commands at build time

[marko kiiskila]

> On 24 Sep 2019, at 20.14, Christopher Collins  wrote:
> 
> Hi Marko,
> 
> On Tue, Sep 24, 2019 at 03:19:24PM +0300, marko kiiskila wrote:
>> Thanks, this is a very useful feature.
>> 
>>> On 24 Sep 2019, at 3.50, Christopher Collins  wrote:
>>> 
>> ...
>>> 
>>> A package specifies custom commands in its `pkg.yml` file.  There are
>>> two types of commands: 1) pre_cmds (run before the build), and 2)
>>> post_cmds (run after the build). 
>>> 
>>> ### EXAMPLE
>>> 
>>> Example (apps/blinky/pkg.yml):
>>> 
>>>   pkg.pre_cmds:
>>>   scripts/pre_build1.sh: 100
>>>   scripts/pre_build2.sh: 200
>>> 
>>>   pkg.post_cmds:
>>>   scripts/post_build.sh: 100
>>> 
>>> For each command, the string on the left specifies the command to run.
>>> The number on the right indicates the command's relative ordering.
>> 
>> I wasn’t sure about need for numbering, but I can see a use if execution
>> of custom commands depends on a syscfg variable being set.
> 
> I also wasn't sure about using numbered stages.  It would be simpler if
> each package specified a sequence of commands rather than a mapping.
> 
> I was envisioning a case where package A generates some intermediate
> file, then package B reads this file and produces the final output.  In
> this scenario, B's scripts must run after A's scripts.  Assigning stages
> to each command allows the user to enforce this ordering.  Yes you are
> right - packages would need to use syscfg settings to define the stages
> for this to be useful.
> 
> Another abstract use case: several scripts emit strings to some temp
> files, then a final script would gather these files and generate a .c /
> .h pair.  Admittedly, I can't think of a concrete example where this
> would be needed.  
> 
> My feeling is we should continue using numeric stages here because a)
> they could conceivably be useful, and b) it's already implemented this
> way :).  I'm really uncertain about this though, so if you (or anyone
> else) has a strong opinion about this, please just let me know and I'll
> go with it.

I do not have strong opinion about this one. I’m fine with the scheme
you have described.

> 
>> I think one useful addition would be to have another class of commands
>> for linking phase. Often there’s a need to convert the binary to some
>> other format, or slap in another header (like with DA1469x bootloader).
>> Maybe have such an option for BSP packages? pkg.post_cmds would
>> get executed after generating object files, and maybe pkg.post_link_cmds
>> would get executed after linking?
> 
> That is a good idea.  How about this naming scheme:
> 
>* pre_build_cmds
>* pre_link_cmds
>* post_build_cmds
> 
> ?

That seems reasonable.

> 
> I also think we would need some an additional environment variables to
> make this useful:
> 
>* MYNEWT_PKG_BIN_DIR(dir where .o / .a files get written)
>* MYNEWT_PKG_BIN_FILE   (full path of .a file)
> 
> While we are at it, might as well add one more:
> 
>* MYNEWT_USER_WORK_DIR  (temp dir shared by all scripts)
> 
> It might be useful for scripts that feed input to other scripts.
> 

Sounds good. If we need to add more, we can do that later.

—
M