Changes in log configuration

[Christopher Collins]

Hello all.  I wanted to report on some logging changes that have gone
into master of mynewt-core.  All questions and comments are welcome.

Thanks,
Chris

### SUMMARY

There have been some changes to how Mynewt logs get configured.  Now,
Log modules are defined in `syscfg.yml` files rather than in C code.
The old method of defining logs still works, but the recommended way
(and the way mynewt-core and mynewt-nimble do it) has changed.

### IMPORTANT

Setting LOG_LEVEL=0 (debug) no longer enables full verbosity for all
logs.  If you want full verbosity, you need to set LOG_LEVEL=0 *and*
all module level settings to 0.  Use the following command to get a
list of log level settings that your target uses:

newt target logcfg show 

To *decrease* system-wide log verbosity, you can simply override
`LOG_LEVEL` with the desired value (as always).

### SUGGESTED USE

* Set LOG_LEVEL to 0 (DEBUG).
* Tune individual module levels as desired.

### RATIONALE

This change was made for the following reasons:

1. Ability to set log level per module at build time. Prior to the
change, setting a global log level of DEBUG (LOG_LEVEL=0) might produce
an image that is too big, and would produce lots of uninteresting log
messages.  Now, we can enable debug logging only for the modules we are
interested in.

2. Easier to visualize system log configuration (see below).

3. Log modules can be remapped at build time in case of conflicts.

DETAILS:

*** Terminology:

* LOG: A medium where log entries get written.  Examples are: 1) the
  console, 2) a flash circular buffer (FCB), 3) a RAM circular buffer
  (cbmem).

* ENTRY: A single unit of log data.  Every entry consists of a header
  (seqeuence number, timestamp, etc.) and a body.

* MODULE: Modules are mapped to one or more logs at runtime.  When
  application code writes a log entry, it specifies the numeric module
  ID to write to.  The `sys/modlog` package then routes the entry to
  the logs that the module is mapped to.  By default, all modules are
  mapped to the console.

*** Defining a log module in a package's `syscfg.yml` file:

1. Define two new settings:
* module ID
* module level

2. Define the module itself:
* syscfg.logs

Example (from `sys/log/common/syscfg.yml`):

syscfg.defs:
DFLT_LOG_MOD:
description: 'Numeric module ID to use for default log
messages.'
value: 0
DFLT_LOG_LVL:
description: 'Minimum level for the default log.'
value: 1

syscfg.logs:
DFLT_LOG:
module: MYNEWT_VAL(DFLT_LOG_MOD)
level: MYNEWT_VAL(DFLT_LOG_LVL)

*** Visualize a target's log configuration:

There are two newt commands for this:

newt target logcfg brief 
newt target logcfg show 

Examples:

# (brief)

newt target logcfg brief slinky-nordic_pca10056

Brief log config for targets/slinky-nordic_pca10056:
   LOG | MODULE   | LEVEL
---+--+--
  DFLT_LOG | 0| 1 (INFO)
IMGMGR_LOG | 176  | 0 (DEBUG)

# (show)

newt target logcfg show slinky-nordic_pca10056

Log config for targets/slinky-nordic_pca10056:
DFLT_LOG:
Package: @apache-mynewt-core/sys/log/common
Module:  0[DFLT_LOG_MOD]
Level:   1 (INFO) [DFLT_LOG_LVL]

IMGMGR_LOG:
Package: @apache-mynewt-core/mgmt/imgmgr
Module:  176  [IMGMGR_LOG_MOD]
Level:   0 (DEBUG)[IMGMGR_LOG_LVL]

*** Macros:

At build time, newt generates the following macros for each defined
log:

_DEBUG(...)
_INFO(...)
_WARN(...)
_ERROR(...)
_CRITICAL(...)

These macros take a printf-style format string and an optional list of
arguments.

A macro writes a log entry if both of the following are true:
* Module level is <= the macro level.
* LOG_LEVEL is <= the macro level.

Otherwise, the macro evaluates to a no-op [*].

So, using the DFLT_LOG above as an example, if

DFLT_LOG_LVL=3  (ERROR)
LOG_LEVEL=0 (DEBUG)

then `DFLT_LOG_ERROR()` and `DFLT_LOG_CRITICAL()` write a log entry,
whereas the `DEBUG`, `INFO`, and `WARN` macros are no-ops.

Another example: if

DFLT_LOG_LVL=3  (ERROR)
LOG_LEVEL=4 (CRITICAL)

then only `DFLT_LOG_CRITICAL()` has any effect.  The other macros
evaluate to no-ops.

[*] These aren't strictly no-ops.  The printf-style argument list gets
evaluated and the results are discarded.  This is done to prevent "set
but unused" warnings.

*** Recap:

* Log modules are defined in `syscfg.yml` files.
* To change a module's ID or level, override the appropriate syscfg
  setting.
* LOG_LEVEL acts as a minimum log level for all modules.
* To view a target's log conguration, use the following newt commands:
* newt target logcfg brief 
* newt target logcfg show 

### OPEN QUESTIONS

1. Do we need a way to set all of a target's modules to a particular
level?  This didn't seem very useful to me.  

Re: [VOTE] Release Apache Mynewt 1.7.0-rc2 and Apache NimBLE 1.2.0-rc2

[Andrzej Kaczmarek]

+1 (binding)



On Mon, Jul 29, 2019 at 1:39 PM Szymon Janc  wrote:

> Hello all,
>
> I am pleased to be calling this vote for the source release of
> Apache Mynewt 1.7.0 and Apache NimBLE 1.2.0.
>
> Apache Mynewt is a community-driven, permissively licensed open source
> initiative for constrained, embedded applications. Mynewt provides a
> real-time operating system, flash file system, network stacks, and
> support utilities for real-world embedded systems.
>
> Apache NimBLE is Bluetooth Low Energy 5.0 stack from Apache Mynewt.
>
> This release is coordinated release of Apache Mynewt and NimBLE. Future
> NimBLE releases may be released separately depending on needs.
>
> For full release notes for both Mynewt and NimBLE, please visit the Apache
> Mynewt Wiki:
> https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes
>
> Apache Mynewt and Apache NimBLE release candidates were tested as follows:
>   1. Manual execution of the Mynewt test plan:
>  https://cwiki.apache.org/confluence/display/MYNEWT/
> Apache+Mynewt+Test+Plan
>  The test results can be found at:
>  https://cwiki.apache.org/confluence/display/MYNEWT/1.7.0+Test+Results
>   2. Manual execution of the NimBLE test plan:
>  https://cwiki.apache.org/confluence/display/MYNEWT/
> Apache+NimBLE+Test+Plan
>  The test results can be found at:
>  https://cwiki.apache.org/confluence/display/MYNEWT/
> NimBLE+1.2.0+Test+Results
>
>  Note that this testing is not yet complete and more results will show
>  while voting is ongoing.
>
>   2. The full unit test suite for both releases was executed via
>  "newt test all" with no failures. This testing was performed on the
>  following platforms:
>* OS X 10.14
>* Fedora Linux 30
>
> The release candidate to be voted on is available at:
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> https://dist.apache.org/repos/dist/dev/mynewt/apache-nimble-1.2.0/rc2/
>
> The commits under consideration are as follows:
> blinky:
>   repos: https://github.com/apache/mynewt-blinky
>   commit 1007e978f2655b3e054099ff8600d147a79b369b
> core:
>   repos: https://github.com/apache/mynewt-core
>   commit b7a5474d569d5b67152d1773627ddda010c080a3
> newt:
>   repos: https://github.com/apache/mynewt-newt
>   commit 80bcba727dfe828dcb1f8da522f0502377d18fd4
> newtmgr:
>   repos: https://github.com/apache/mynewt-newtmgr
>   commit a222bac117793db7b8444acfb744dcb822a6f448
> nimble:
>   repos: https://github.com/apache/mynewt-nimble
>   commit 5371a45890604d32564c6384eb00dfee74ee7d13
>
> In addition, the following newt and newtmgr convenience binaries are
> available:
>   linux:
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newt-bin-linux-1.7.0.tgz
> 
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newtmgr-bin-linux-1.7.0.tgz
> 
>
>   osx:
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newt-bin-osx-1.7.0.tgz
> 
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newtmgr-bin-osx-1.7.0.tgz
> 
>
>   windows:
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newt-bin-windows-1.7.0.tgz
> 
> https://dist.apache.org/repos/dist/dev/mynewt/apache-mynewt-1.7.0/rc2/
> apache-mynewt-newtmgr-bin-windows-1.7.0.tgz
> 
>
> The release candidate is signed with a GPG key available at:
> https://dist.apache.org/repos/dist/dev/mynewt/KEYS
>
> The vote is open for at least 72 hours and passes if a majority of at
> least three +1 PMC votes are cast.
>
> [ ] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because...
>
> Anyone can participate in testing and voting, not just committers,
> please feel free to try out the release candidate and provide your
> votes.
>
> A separate [DISCUSS] thread will be opened to talk about this release
> candidate.
>
> --
> pozdrawiam
> Szymon Janc
>
>
>