Re: Tutorial problem with the Docker Container

[Christopher Collins]

On Fri, Jan 20, 2017 at 11:55:59PM +0800, Li-Chun Ko wrote:
> As an update, the Docker Container works well with the "./newt test all"
> commend.

Great to hear.  Thanks for following up!

Chris

Re: bleuart example causes crash

[Christopher Collins]

On Fri, Jan 20, 2017 at 09:41:24AM +, Lm Chew wrote:
> Hi,
> 
> 
> Does any one have problem running the bleuart example?
> 
> When I run it, it crashes in the end with Unhandled interrupt error. (I am 
> using bmd300eval)
> I got the following debug output:

It has been quite a while since I have run bleuart.  I'll take a look at
it today.  In the meantime, are you sure you need the bleuart
functionality?  Just to be cler, this app implements a somewhat scaled
down version of the Nordic BLE serial port emulation service.

Chris

Re: [RFC] Reducing size of BLE Security Manager

[Christopher Collins]

On Fri, Jan 20, 2017 at 10:21:16AM +0100, Szymon Janc wrote:
> Code is available at [1]. I'm not making it a pull request yet since
> I'd like to get some feedback about this approach from others. Also I
> still need to get tests passing since SM keys related tests fail now
> (and I'm not yet sure why).   I tested this with Android phone and
> both legacy and LE SC seems to work just fine.

(you may already know all this...)

By the way, you can run the unit tests using:

newt run net/nimble/host/test

When you use the "run" command, the tests get run in gdb, and the first
test failure causes gdb to halt.  This is useful for debugging test
failures.

Some of the host unit tests are a bit... interesting to debug :).  I can
take a look at the failure if you like.

Chris

Re: [RFC] Reducing size of BLE Security Manager

[Christopher Collins]

Hi Szymon,

On Fri, Jan 20, 2017 at 10:21:16AM +0100, Szymon Janc wrote:
> Hi,
> 
> I was recently looking on how we could reduce size of SM code.
> So my proposal is to change the way PDUs are parsed and constructed.
> 
> Instead of having ble_sm_foo_parse(), ble_sm_foo_write() and ble_sm_foo_tx()
> for parsing and constructing PDU byte by byte we could use packed structures
> for describing PDU and let compiler figure out details related to
> unaligned access.
[...]

I think that's a great idea.  The ATT code does something similar,
though there is probably more work to be done there.  In my opinion,
using packed structs for parsing and encoding doesn't just reduce code
size, it also simplifies the code.

Chris

Re: Tutorial problem with the Docker Container

[Christopher Collins]

Hi Li-Chun,

I'm glad you got it working, but now I'm confused.  The docker container
should already use the correct version of gcc (multilib).  Furthermore,
as I understand it, installing a different compiler on your machine
should not have any effect on the docker container.  Are you still using
the docker container?

Thanks,
Chris

On Thu, Jan 19, 2017 at 11:55:52PM +0800, Li-Chun Ko wrote:
> Hi Fabio,
> 
> After I install gcc-multilib all problems solved!
> I guess it is worthwhile to add a note for this in the tutorial.
> Thank you very much.
> 
> BTW, I am using Ubuntu 16.04.1 64 bit version.
> 
> Br,
> Lichun
> 
> 
> 2017-01-19 22:44 GMT+08:00 Li-Chun Ko :
> 
> > Hi all,
> >
> > I tried "$newt test @repos/apache-mynewt-core/kernel/os" in Linux but it
> > still doesn't work.
> > I also attached the error message of executing "$newt test all" below.
> > It seems some files are missing.
> >
> > Br,
> > Lichun
> >
> > 1. Error message when executing $newt test @repos/apache-mynewt-core/
> > kernel/os:
> >
> > lcko@lcko-VirtualBox:~/dev/go/myproj$ $newt test
> > @repos/apache-mynewt-core/kernel/os
> > panic: runtime error: invalid memory address or nil pointer dereference
> > [signal 0xb code=0x1 addr=0x0 pc=0x6879bf]
> >
> > goroutine 1 [running]:
> > panic(0x963920, 0xc82000e170)
> > /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
> > mynewt.apache.org/newt/newt/repo.(*Repo).Name(0x0, 0x1c, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/repo/repo.go:290
> > +0x1f
> > mynewt.apache.org/newt/newt/pkg.(*Dependency).setRepoAndName(0xc82043cbe0,
> > 0x7fcd8a960200, 0x0, 0x7ffdb79b70e3, 0x1c, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:63
> > +0x147
> > mynewt.apache.org/newt/newt/pkg.(*Dependency).Init(0xc82043cbe0,
> > 0x7fcd8a960200, 0x0, 0x7ffdb79b70e3, 0x1c, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:74
> > +0x55
> > mynewt.apache.org/newt/newt/pkg.NewDependency(0x7fcd8a960200, 0x0,
> > 0x7ffdb79b70e3, 0x1c, 0xd0dc40, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:84
> > +0x7f
> > mynewt.apache.org/newt/newt/project.(*Project).ResolvePackage(0xc820054540,
> > 0x7fcd8a960200, 0xc820070510, 0x7ffdb79b70dc, 0x23, 0x4, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/project/
> > project.go:522 +0x448
> > mynewt.apache.org/newt/newt/cli.testRunCmd(0xc82042cd80, 0xc820435210,
> > 0x1, 0x1)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/cli/
> > build_cmds.go:225 +0x140e
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).execute(0xc82042cd80, 0xc8204351d0, 0x1, 0x1, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:636 +0x8e9
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).ExecuteC(0xc820075440, 0xc82042cd80, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:722 +0x55c
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).Execute(0xc820075440, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:681 +0x2d
> > main.main()
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/newt.go:146 +0x2a2
> > exit status 2
> >
> >
> > 2. Error message when executing $newt test all:
> >
> > lcko@lcko-VirtualBox:~/dev/go/myproj$ cd repos
> > lcko@lcko-VirtualBox:~/dev/go/myproj/repos$ $newt test all
> > Testing package @apache-mynewt-core/boot/boot_serial/test
> > Compiling boot_serial.c
> > In file included from /usr/include/assert.h:35:0,
> >  from boot_serial.c:19:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/boot/bootutil/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/crypto/mbedtls/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/encoding/base64/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/encoding/json/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Thu, Jan 19, 2017 at 10:57:58AM -0800, Christopher Collins wrote:
> On Thu, Jan 19, 2017 at 03:46:49AM -0800, Simon Ratner wrote:
> > A related question: how does this map to large ATT_MTU and fragmented
> > packets at the L2CAP level (assuming no data length extension)? Does each
> > fragment get its own mbuf, which are then chained together, or does the
> > entire packet get reassembled into a single mbuf if there is room?
> 
> If the host needs to send a large packet, it packs the payload into an
> mbuf chain.  By "packs," I mean each buffer holds as much data as
> possible with no regard to the maximum L2CAP fragment size.
> 
> When the host sends an L2CAP fragment, it splits the fragment payload
> off from the front of the mbuf chain, constructs an ACL data packet, and
> sends it to the controller.  If a buffer at the front of mbuf can be
> freed, now that data has been removed, the host frees it.
> 
> If you are interested, the function which handles fragmentation and
> freeing is mem_split_frag() (util/mem/src/mem.c).

I rushed this response a bit, and there are some important details I
neglected.

* For the final L2CAP fragment in a packet, the host doesn't
do an allocating or copying.  Instead, it just prepends an ACL data
header to the mbuf chain and sends it to the controller.

* For all L2CAP fragments *other than the last*, the host allocates an
additional mbuf chain to hold the ACL data packet.  The host then copies
the fragment data into this new chain, sends it, and frees buffers from
the front of the original chain if possible.  The number of buffers that
get allocated for the fragment depends on how the maximum L2CAP fragment
size compares to the msys mbuf size.  If an msys mbuf buffer has
sufficient capacity for a maximum size L2CAP fragment, then only one
buffer will get allocated.  If the mbuf capacity is less, the chain that
gets allocated will consist of multiple buffers.

* An L2CAP fragment mbuf chain contains the following:
* mbuf pkthdr   (8 bytes)
* HCI ACL data header   (4 bytes)
* Basic L2CAP header(4 bytes)
* Payload   (varies)

* For incoming data, the host does not do any packing.  Each L2CAP
fragment is simply chained together.

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Thu, Jan 19, 2017 at 03:46:49AM -0800, Simon Ratner wrote:
> A related question: how does this map to large ATT_MTU and fragmented
> packets at the L2CAP level (assuming no data length extension)? Does each
> fragment get its own mbuf, which are then chained together, or does the
> entire packet get reassembled into a single mbuf if there is room?

If the host needs to send a large packet, it packs the payload into an
mbuf chain.  By "packs," I mean each buffer holds as much data as
possible with no regard to the maximum L2CAP fragment size.

When the host sends an L2CAP fragment, it splits the fragment payload
off from the front of the mbuf chain, constructs an ACL data packet, and
sends it to the controller.  If a buffer at the front of mbuf can be
freed, now that data has been removed, the host frees it.

If you are interested, the function which handles fragmentation and
freeing is mem_split_frag() (util/mem/src/mem.c).

Chris

Using Eclipse for Mynewt development

[Christopher Collins]

Here are some of my notes for getting Mynewt to work with eclipse.  I
wrote these a while back, so there may be a few inaccuracies.  Also,
these notes assume you are building bletiny for the nRF52dk BSP, so you
may need to adjust accordingly.

Thanks,
Chris

*** Setup

1. Download Eclipse Neon (https://www.eclipse.org/downloads/).

2. Start Eclipse.

3. Eclipse -> Preferences -> C/C++ -> Environment; Add a "PATH" Variable
   with the value "/usr/local/bin" (or wherever arm-none-eabi-gdb is
   located on your machine).

4. In your newt project directory, overwrite
   repos/apache-mynewt-core/hw/bsp/nrf52dk/nrf52dk_debug.sh with the
   attached version.

*** New project

5. File -> New -> New Project -> C/C++ -> C Project ->
   Makefile project -> Empty Project

6. Select "-- Other Toolchain --" in the Toolchains panel.

7. Type your project's name in the "Project name" field.

8. Select your newt project directory in the "Location" field.

(see "new-project.png").

9. Click Finish.

*** Debug configuration

10. Right-click your project and select "Debug As..." ->
"Debug Configurations".

11. Double click "C/C++ Remote Application" to create a new debug
configuration.  The new configuration should now be selected.

12. At the bottom of the window, you should see: "Using GDB (DSF)
Automatic Remote Debugging Launcher - Select other...".  Click the
"Select other..." link.

13. Check "Use configuration specific settings"

14. Select "GDB (DSF) Manual Remote Debugging Launcher"

(see "select-launcher.png").

15. Click OK.

16. (Main tab) In the "C/C++ Application" field, select the .elf file
corresponding to your newt target.

17. (Main tab) Check "Disable auto build"

(see "debug-main.png").

18. (Debugger, Main tab) In the "GDB debugger" field, type:
arm-none-eabi-gdb

(see "debug-debugger-main.png").

19. (Debugger, Connection tab) In the "Port number" field, type: 

(see "debug-debugger-connection.png").

20. Click Apply and Close.

*** Debugging

21. In a shell, run "newt run  0".  JLinkGDBServer should
start up and listen for a connection from gdb.

21. Select your project in the Project Explorer.

22. Click the icon with a picture of a bug.  The debugger should start.

23. The board will be in a bad state and requires a reset.  Type
"monitor reset" in the Console pane

(see "monitor-reset.png")

24. Click the Resume button (play button with a yellow rectangle on its
left).  The program should now halt at the start of main().


nrf52dk_debug.sh
Description: Bourne shell script

Re: stopping scan & adv in bleprph example

[Christopher Collins]

Hi, sorry for the late response.  Somehow I missed this.

On Mon, Jan 16, 2017 at 08:22:27PM -0800, will sanfilippo wrote:
> Yes, Mynewt works the same way as FreeRTOS in this respect. Well, at least in 
> the way you are describing FreeRTOS. We have a tickless OS and when we decide 
> to go to sleep we are waiting for an interrupt to wake us up.
> 
> Regarding the radio: there are some registers that are only programmed once, 
> so if you switch to your own custom RF stack and you want to switch back to 
> bluetooth, you would either have to write some custom code or reset the link 
> layer. There is an API to do this but I am not sure if it is accessible to 
> the application developer.

It's not a part of the published host API, but the following function
should cause the host to reset the controller:

void ble_hs_sched_reset(int reason)

The reason code doesn't matter; it is just something the host uses when
reporting terminated connections.  You can just specify 1.

Since this function isn't part of the API, you'll need to declare a
prototype by hand:

void ble_hs_sched_reset(int reason);

Thanks,
Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Tue, Jan 17, 2017 at 09:06:36PM -0700, Jacob Rosenthal wrote:
[...]
> As you can see, I cant seem to get slot1 active and confirmed

It looks like one of two things is going wrong:

1. The loader (bleprph) is failing to jump into the application
   (splitty), or

2. The application (splitty) is crashing immediately.

Do you have the latest newt tool from develop?  There was a bug in the
newt tool which would cause issue 2, but it was fixed a few days ago.
If I recall the bug, splitty's sysinit was initializing the the host
package, but not the HCI transport package, causing an invalid memory
access when host started up.

Anyway, if you don't have the latest newt, try pulling from develop and
reinstalling.  Otherwise, I think the next step is to debug this in gdb.
Feel free to reach out here or on IRC.

Thanks,
Chris

Re: stopping scan & adv in bleprph example

[Christopher Collins]

Hi Chew,

On Tue, Jan 17, 2017 at 01:08:16AM +, Lm Chew wrote:
> So calling ble_gap_adv_stop and ble_gap_disc_cancel will stop all radio 
> activity is that correct?

It depends on what your application has told the stack to do.  If you
have initiated a connection, you will also need to cancel that:

ble_gap_adv_stop()
ble_gap_disc_cancel()
ble_gap_conn_cancel()

If any connections have already been established, then they will also
cause radio activity.  You can terminate connections with
ble_gap_terminate().

Chris

> Is it safe to modify the Radio setting(on the physical just like in ble_phy) 
> after just calling these functions?
> 
> Hi Will,
> 
> Not exactly a "system off" I am looking for.
> Previously I am using FreeRTOS tickless mode where the MCU will remain in 
> sleep mode most of the tire unless there is a task to perform.
> 
> I am asking this because in the bleprph example I don't see any function 
> being called to put the MCU to sleep.
> 
> Does mynewt OS work the same way as FreeRTOS?
> 
> Best Regards,
> Chew
> 
> 
> 
> 
> 
> On Tue, Jan 17, 2017 at 1:57am, will sanfilippo 
> <wi...@runtime.io<mailto:wi...@runtime.io>> wrote:
> 
> If by deep sleep you mean "system off" mode requiring some form of wakeup, it 
> is curently not implemented. You would have to hook that in yourself.
> 
> > On Jan 16, 2017, at 9:22 AM, Christopher Collins <ccoll...@apache.org> 
> > wrote:
> >
> > Hi Chew,
> >
> > On Mon, Jan 16, 2017 at 11:33:23AM +, Lm Chew wrote:
> >> Hi,
> >>
> >> How do I stop the scan &  adv in the bleprph example.
> >>
> >> I tried calling the ble_ll_scan_sm_stop(1) and  ble_ll_adv_stop in my app, 
> >> but I am still able to see the device on my phone when I perform a scan.
> >
> > To stop advertising, call: ble_gap_adv_stop()
> > (http://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_adv_stop/)
> >
> > For BLE operations, an application should only use the host interface.
> > Functions with the "ble_ll" prefix are defined by the controller, not
> > the host, so your application should not call them.
> >
> > Regarding scanning- the bleprph app doesn't perform any scanning, so
> > there is no need to stop scanning.  This application only implements the
> > peripheral role, so operations like scanning and initiating a connection
> > are not compiled in.  However, if you have a different app which does
> > support scanning, you would stop the scan procedure by calling
> > ble_gap_disc_cancel()
> > (http://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_disc_cancel/)
> >
> >> I am trying to switch between my custom rf stack  and nimble bt stack. So 
> >> I need to disable nimble  operation before running my custom RF Stack.
> >> And once I am done what I need using the custom RF Stack, I will switch 
> >> back nimble.
> >>
> >> Another question, how do you put the MCU to deep sleep while using nimble 
> >> stack? In the example the MCU does not goes to deep sleep.
> >
> > Sorry, I am not sure about this one.  I am not sure this is actually
> > supported yet, but I'll let someone more knowledgable chime in.
> >
> > Chris
> 

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Just a follow up for the list-

It turns out there were a number of issues with split images.  I believe
they have all been resolved now.  Some of the required changes were
non-trivial, so I wanted to summarize what I did.

1. bleprph doesn't jump to app (slot 2).

This one is straightforward.  The bleprph app wasn't written to behave
as a loader.  Rather, it always started the OS, even when configured to
run the second stage app.  The fix was just to copy and paste the
generic loader code from slinky.

2. Jump from loader to app required too much RAM.

When a loader transitions to the second stage app, it executes a
modified version of the boot loader.  The boot loader was using quite a
bit of RAM for caching information about the layout of sectors within
the image slots.  In normal, non-split, operation, such RAM usage isn't
a problem because there is no OS running, and therefore no other code to
share memory with.  In the modified version that runs during the split
transition, other code *is* running, so RAM for the sector cache gets
allocated via malloc().  About 4kB was getting malloced here, which was
failing for me when running on the 16 kB RAM nRF51dk BSP.  The fix is to
not cache the sector data, but to read it when needed.

3. Loader-only packages were not getting initialized in the second stage
app.

This was a bug in the newt tool.  When generating the sysinit_app()
function, the tool excluded packages which are not explicitly referenced
in the app's dependency tree.  This is wrong; all packages should get
initialized in the second stage app.

Thanks,
Chris


On Fri, Jan 13, 2017 at 04:38:25PM -0700, Jacob Rosenthal wrote:
> Ah, I expected gdb of optimized split code to be a nightmare and hadnt
> tried yet.
> newtmgr has never crashed.. it just hangs forever
> 
> I cant get the SOFT anymore now. hrmmm Just hanging out here..
> (gdb) where
> #0  os_tick_idle (ticks=1189) at
> repos/apache-mynewt-core/hw/mcu/nordic/nrf51xxx/src/hal_os_tick.c:158
> #1  0x8c1e in os_idle_task (arg=) at
> repos/apache-mynewt-core/kernel/os/src/os.c:110
> #2  0x in ?? ()
> (gdb) c
> 
> Any chance I could impose on you to debug in irc with me sometime?
> 
> 
> On Thu, Jan 12, 2017 at 9:42 PM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> > Hi Jacob,
> >
> > If that reboot log entry is recent, it looks like your device is
> > crashing.  A reason of "SOFT" indicates a firmware crash [*].  I would
> > say the quickest way to debug this is to run the code in gdb.  You can
> > do this with the "newt run" command:
> >
> > newt run  0
> >
> > When gdb comes up, type c and press enter.  Then in a different shell,
> > send a newtmgr command to the device.  If device crashes, gdb should
> > indicate such.  If Mynewt crashes, can you please type the following
> > commands in gdb:
> >
> > bt
> > p *g_current_task
> > p os_msys_init_1_mempool
> >
> > And send the output?
> >
> > Thanks,
> > Chris
> >
> > [*] Augmenting the reboot log entry with the line number and filename
> > where the crash occurred is on the to do list.
> >
> >
> > On Thu, Jan 12, 2017 at 08:41:03PM -0700, Jacob Rosenthal wrote:
> >
> >
> >
> > > turned off a ton more bluetooth shit for more ram
> > >
> > > both still hang:
> > > newtmgr -c serial1 logs log_list
> > > newtmgr -c serial1 image list
> > >
> > > but log in shell now has:
> > > newtmgr 4976:Dumping log reboot_log
> > >
> > >
> > > On Thu, Jan 12, 2017 at 8:33 PM, Jacob Rosenthal <
> > jakerosent...@gmail.com>
> > > wrote:
> > >
> > > > log on shell shows lots of these
> > > >
> > > > 9050:[0] rsn:SOFT, cnt:1, img:0.0.0.0
> > > >
> > > > On Thu, Jan 12, 2017 at 8:28 PM, Jacob Rosenthal <
> > jakerosent...@gmail.com>
> > > > wrote:
> > > >
> > > >> All newtmgr commands on serial are hanging indefinitely.
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c
> > serial1
> > > >> image list
> > > >> ^C
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr conn
> > show
> > > >> Connection profiles:
> > > >>   serial1: type=serial, connstring='/dev/tty.usbmodem1411'
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> > > >>
> > > >> Looks like it works for the splitty/slinky demo
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Fri, Jan 13, 2017 at 04:38:25PM -0700, Jacob Rosenthal wrote:
> Ah, I expected gdb of optimized split code to be a nightmare and hadnt
> tried yet.
> newtmgr has never crashed.. it just hangs forever
> 
> I cant get the SOFT anymore now. hrmmm Just hanging out here..
> (gdb) where
> #0  os_tick_idle (ticks=1189) at
> repos/apache-mynewt-core/hw/mcu/nordic/nrf51xxx/src/hal_os_tick.c:158
> #1  0x8c1e in os_idle_task (arg=) at
> repos/apache-mynewt-core/kernel/os/src/os.c:110
> #2  0x in ?? ()
> (gdb) c

It would be interesting to see how many mbufs are free:

p os_msys_init_1_mempool

> Any chance I could impose on you to debug in irc with me sometime?

Sure.  I'm on IRC, so just shout any time.

Chris

Re: Supporting multiple filesystems and "block" devices

[Christopher Collins]

On Fri, Jan 13, 2017 at 08:29:33PM -0200, Fabio Utzig wrote:
> Yes, I was expecting it! So I had 3 options in mind:
> 
> Option 1 - Use a "drive" name as was implemented. Btw, it's not limited
> to single letter drive names! Sure if someone really likes DOS, they
> could use A:, B:, etc. This choice is extremely easy to parse. And it
> also brings me fond memories of AmigaDOS!
> 
> Option 2 - Use a directory as a mount point like Unix but without
> implementing a full tree struct. Also very easy to parse, but somehow I
> find it very non-elegant because there would be no root path and such. I
> just personally dislike it because it looks incomplete, which brings me
> to:
> 
> Option 3 - Use a proper tree structure with mount points, mounts inside
> other mount's directories, etc. This is quite harder to do properly but
> would be my preferred aesthetic choice. But this needs extra things like
> ensuring the ordering of mounts, more error handling (if a root mount
> fails for any reason what to do with the remaining drives, etc). I would
> like to move to this but it could be added later because it's basically
> adding code and what is in place stays there, I think. 
> 
> If you think a tree structure is a must and "drive" names are an
> aberration, I would be OK at giving a try on implementing a proper VFS!

Re: option 2- I understand what you mean about the missing root path,
and I agree that it would be odd.  Personally, I don't think option 3 is
worth the effort, at least not now.  Just speaking for myself, I think
option 1 is not a real problem.  Like I said, it was just a knee-jerk
reaction when I saw the colon!

Thanks,
Chris

Re: [RFC] Refactor UUID handling in Nimble

[Christopher Collins]

Hi Andrzej,

That all sounds good to me.

Thanks,
Chris

On Fri, Jan 13, 2017 at 02:58:19PM +0100, Andrzej Kaczmarek wrote:
> Hi,
> 
> I've refactored Nimble code to use dedicated type for UUID handling instead
> of generic byte-array. Full diff is available here (it is split into
> several patches for now to separate changes in stack and apps):
> 
> https://github.com/andrzej-kaczmarek/incubator-mynewt-core/c
> ompare/develop...andrzej-kaczmarek:nimble/uuid
> 
> Start by looking at ble_uuid.h which defines new structures and helpers for
> UUIDs (or see below where I included essential part):
> 
> https://github.com/andrzej-kaczmarek/incubator-mynewt-core/b
> lob/1f571c7aeb22eb4e2ae96ca6d01d612afb338fd4/net/nimble/host
> /include/host/ble_uuid.h
> 
> Now, let me explain what it is all about:
> 
> Current implementation handles all UUIDs as long 128-bit values so they are
> passed like this (byte-array) in API and stored internally. This creates
> overhead for handling short 16-bit UUIDs in terms of both memory and
> processing time because:
> - short values are stored as long ones anyway,
> - they have to be converted back and forth since short UUID cannot be put
> as long one in ATT PDU,
> - there is no generic code to handle the above so similar patterns are
> repeated over and over again.
> 
> With new approach there is dedicated type to pass UUID which contains also
> its type so this information is available immediately - this is used e.g.
> BlueZ and Zephyr.
> 
> enum {
> BLE_UUID_TYPE_16 = 16,
> BLE_UUID_TYPE_32 = 32,
> BLE_UUID_TYPE_128 = 128,
> };
> 
> /* Generic UUID type, to be used only as a pointer */
> typedef struct {
> uint8_t type;
> } ble_uuid_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint16_t value;
> } ble_uuid16_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint32_t value;
> } ble_uuid32_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint8_t value[16];
> } ble_uuid128_t;
> 
> /* Universal UUID type, to be used for any-UUID static allocation */
> typedef union {
> ble_uuid_t u;
> ble_uuid16_t u16;
> ble_uuid32_t u32;
> ble_uuid128_t u128;
> } ble_uuid_any_t;
> 
> This particular approach to handle UUIDs comes from Zephyr which uses neat
> trick to have common type for any UUID yet it allows to store 16-bit values
> using less memory than for 128-bit values. There are dedicated types for
> 16- and 128-bit UUIDs so memory for an UUID can be allocated, but the API
> uses "stub" type which defines only UUID type and is used as a pointer to
> an actual UUID variable. You can use helpers to get UUID value or compare
> them quickly without need to convert between 16- and 128-bit values. See
> following examples how this works (or see sample apps):
> 
> // 16-bit UUID value with initialization
> ble_uuid16_t uuid16 = BLE_UUID16_INIT(0x2801);
> // 128-bit UUID value with initialization
> ble_uuid128_t uuid128 = BLE_UUID128_INIT(0x00, 0x11, ... 0xFF);
> 
> // "stub" type is used to pass any type of UUID via API
> ble_uuid_t *uuid1 = 
> ble_uuid_t *uuid2 = 
> 
> // comparing UUIDs is really easy now
> if (ble_uuid_cmp(uuid1, uuid2) == 0) { /* equal */ )
> 
> // since in most cases we use 16-bit UUIDs, it's convenient to easily
> retrieve 16-bit value (or 0 if UUID is not 16-bit)
> uint16_t u16 = ble_uuid_u16(uuid1);
> 
> // the above is equivalent of
> u16 = uuid1->type == BLE_UUID_TYPE_16 ? BLE_UUID16(uuid1)->value : 0;
> 
> // we can easily define UUID inline as well, e.g. when defining service
> static const struct ble_gatt_svc_def ble_svc_gap_defs[] = {
> {
> .type = BLE_GATT_SVC_TYPE_PRIMARY,
> .uuid = BLE_UUID16_DECLARE(BLE_SVC_GAP_UUID16),
> (...)
> 
> // finally, if we need to store any UUID there is an union defined to help
> ble_uuid_any_t uuid;
> uuid.u16 = uuid16;
> 
> *IMPORTANT NOTE*:
> There are two major differences between old and new API:
> - Shen defining and registering services, in old API UUID value is copied
> to attribute structure so it does not matter where UUID passed as parameter
> is defined. In new API only pointer to UUID is stored thus you cannot e.g.
> define value on stack since it will be destroyed.
> - You need to make sure that 16-bit UUIDs are defined using helpers
> dedicated for 16-bit UUIDs only, i.e. you should not create 128-bit UUID
> with Bluetooth base since it will not be detected as 16-bit UUID.
> 
> Finally, there are of course some savings in both flash and RAM:
> 
> before:
> < 136 286 *fill*
> <   307385027 apps_bletiny.a
> <   536333411 net_nimble_host.a
> < 880 290 net_nimble_host_services_ans.a
> < 496 289 net_nimble_host_services_gap.a
> < 306  86 net_nimble_host_services_gatt.a
> <  160884   2784  15828 179496  2bd28
> /home/andk/devel/mynewt/bletiny/bin/targets/bletiny/
> app/apps/bletiny/bletiny.elf
> 
> after:
> > 143 364 *fill*
> >   304705053 apps_bletiny.a
> >   534093415 net_nimble_host.a
> >   

Re: Supporting multiple filesystems and "block" devices

[Christopher Collins]

Hi Fabio,

Sounds great.  I only have one comment:

On Fri, Jan 13, 2017 at 03:09:48PM -0200, Fabio Utzig wrote:
> > All the fs_* functions where updated to support multiple disks. To
> > access a file/dir now will required adding the name of the disk as a
> > prefix like this:
> > 
> > rc = fs_opendir("mmc0:/", );
> > rc = fs_open("flash0:/testfile.txt", FS_ACCESS_READ, );

I'm sure you anticipated this comment :). Seeing a ':' in a path brings
back horrible memories for me.  I am not sure if this is just an
aesthetic judgment, or if the "UNIX way" would actually be better.  If
it's just an asthetic thing, then I'll get over it, but I can't help but
think there is code out there that would manage better with UNIX paths
rather than DOS paths.

I haven't looked at the code yet, so I don't know how much easier the
use of the colon character makes things.  What do you think about using
"/flash0/testfile.txt" rather than "flash0/textfile.txt"?

Chris

Re: Override pin settings

[Christopher Collins]

On Fri, Jan 13, 2017 at 11:35:13AM -0500, David G. Simmons wrote:
> Here's the entire syscfg.yml file:
> 
> ### Package: targets/air_q
> 
> syscfg.vals:
> OPENOCD_DEBUG: 1
> # Enable the shell task.
> SHELL_TASK: 1
> STATS_CLI: 1
> 
> CONSOLE_TICKS: 1
> CONSOLE_PROMPT: 1
> 
> UART_0_PIN_TX:
> description: 'New Pin Assignment'
> value: 23
> UART_0_PIN_RX:
> description: 'New Pin Assignment'
> value: 24
> 
> So I was trying to override them in the syscfg.vals section. And that is what 
> fails. the old value from the package's syscfg.yml is removed, but the new 
> value isn't substituted in its place.

The last two entries should look like this:

UART_0_PIN_TX: 23
UART_0_PIN_RX: 24

I think this will be a common mistake, so we should look at making newt
report an error here.

Thanks,
Chris

Re: Override pin settings

[Christopher Collins]

Hi David,

On Fri, Jan 13, 2017 at 09:52:35AM -0500, David G. Simmons wrote:
> Before I file a bug in Jira on this, I thought I'd ask if this is expected 
> behavior:
> 
> From my project's syscfg.yml:
> UART_0_PIN_TX:
> description: 'New Pin Assignment'
> value: 23
> UART_0_PIN_RX:
> description: 'New Pin Assignment'
> value: 24
> 
> Results in newt target config show air_q:
> 
> * Setting: UART_0_PIN_RX
> * Description: TBD
> * Value:
> * Overridden: targets/air_q, default=5
>   * Setting: UART_0_PIN_TX
> * Description: TBD
> * Value:
> * Overridden: targets/air_q, default=6
> 
> So it picks up that the value was overridden, but it doesn't pick up the 
> actual values. It looks like I have to go into 
> hw/bsp/arduino_primo_nrf52/syscfg.yml and change it there for any effect to 
> be seen.

It does look like there is a bug here.  Newt should report an error when
it encounters the syscfg file.  You may already know all this, but for
others' benefit- a syscfg.yml file can do two things:
1. Define new settings
2. Override existing settings

You define new settings with the "syscfg.defs" mapping, e.g.,

syscfg.defs:
UART_1:
description: 'Bitbanger UART'
value:  0

You override existing settings with "syscfg.vals", e.g.,

syscfg.vals:
UART_1: 1

That is, the description, value, and other fields are specified at the
point of definition.  When a setting is overridden, only the new value
gets specified.

>From your email, it looks like the target's syscfg.yml is redefining
existing settings, which should cause newt to raise an error.  I'll look
into this error reporting issue, but in the meantime, I would change
those UART pin overrides to use the syscfg.vals mapping rather than
syscfg.defs.

Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

If that reboot log entry is recent, it looks like your device is
crashing.  A reason of "SOFT" indicates a firmware crash [*].  I would
say the quickest way to debug this is to run the code in gdb.  You can
do this with the "newt run" command:

newt run  0

When gdb comes up, type c and press enter.  Then in a different shell,
send a newtmgr command to the device.  If device crashes, gdb should
indicate such.  If Mynewt crashes, can you please type the following
commands in gdb:

bt
p *g_current_task
p os_msys_init_1_mempool

And send the output?

Thanks,
Chris

[*] Augmenting the reboot log entry with the line number and filename
where the crash occurred is on the to do list.


On Thu, Jan 12, 2017 at 08:41:03PM -0700, Jacob Rosenthal wrote:



> turned off a ton more bluetooth shit for more ram
> 
> both still hang:
> newtmgr -c serial1 logs log_list
> newtmgr -c serial1 image list
> 
> but log in shell now has:
> newtmgr 4976:Dumping log reboot_log
> 
> 
> On Thu, Jan 12, 2017 at 8:33 PM, Jacob Rosenthal <jakerosent...@gmail.com>
> wrote:
> 
> > log on shell shows lots of these
> >
> > 9050:[0] rsn:SOFT, cnt:1, img:0.0.0.0
> >
> > On Thu, Jan 12, 2017 at 8:28 PM, Jacob Rosenthal <jakerosent...@gmail.com>
> > wrote:
> >
> >> All newtmgr commands on serial are hanging indefinitely.
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c serial1
> >> image list
> >> ^C
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr conn show
> >> Connection profiles:
> >>   serial1: type=serial, connstring='/dev/tty.usbmodem1411'
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> >>
> >> Looks like it works for the splitty/slinky demo
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c serial1
> >> image list
> >> Images:
> >>  slot=0
> >> version: 0.0.0
> >> bootable: true
> >> flags: active confirmed
> >> hash: 21439de02cbf31626856374f44cbd490fd6def3ce3062b63d55ed2c19a8b
> >> 2b83
> >>  slot=1
> >> version: 0.0.0
> >> bootable: false
> >> flags:
> >> hash: 8b64ea89bf0495c0ccb25b96b3a7f06fd5e540e221f9659f9bc6b0d0d303
> >> d6f1
> >> Split status: matching
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> >>
> >> Ram issue? Whats a good way to see a log of the failed attempt since I
> >> have the shell?
> >>
> >>
> >>
> >> On Thu, Jan 12, 2017 at 7:51 PM, Christopher Collins <ccoll...@apache.org
> >> > wrote:
> >>
> >>> On Thu, Jan 12, 2017 at 07:06:37PM -0700, Jacob Rosenthal wrote:
> >>> > Still ok...  and Im able to interact with shell over serial and I
> >>> think its
> >>> > advertising!? Not sure why my newtmgr wont connect then.. Ideas to
> >>> >  troubleshoot?
> >>>
> >>> So you are sending newtmgr commands over serial?  Do all commands fail,
> >>> or just image list?
> >>>
> >>> > Also I was digging and found someone already PRed a ble transport for
> >>> > newtmgr https://github.com/apache/incubator-mynewt-core/pull/73/files
> >>> > but the api is a bit different now.. Anyone using that?
> >>>
> >>> Yes, the newtmgr BLE characteristic is being used and should work.  The
> >>> newtmgr command line tool has rudimentary support for BLE, but only when
> >>> run on linux.
> >>>
> >>> Thanks,
> >>> Chris
> >>>
> >>
> >>
> >

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Thu, Jan 12, 2017 at 07:06:37PM -0700, Jacob Rosenthal wrote:
> Still ok...  and Im able to interact with shell over serial and I think its
> advertising!? Not sure why my newtmgr wont connect then.. Ideas to
>  troubleshoot?

So you are sending newtmgr commands over serial?  Do all commands fail,
or just image list?

> Also I was digging and found someone already PRed a ble transport for
> newtmgr https://github.com/apache/incubator-mynewt-core/pull/73/files
> but the api is a bit different now.. Anyone using that?

Yes, the newtmgr BLE characteristic is being used and should work.  The
newtmgr command line tool has rudimentary support for BLE, but only when
run on linux.

Thanks,
Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Thu, Jan 12, 2017 at 07:19:11PM -0700, Jacob Rosenthal wrote:
> OH really interesting.. In the updated documentation the biggest change
> from what I was doing was splitty is the app and bleprph is the loader. Is
> that intentional?

Yes - the loader needs to contain everything required for an image
upgrade.  The bleprph app contains the BLE stack and the newtmgr /
imgmgr server.  These components are what is required for an
over-the-air upgrade, so it makes sense to use bleprph as the loader.

Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Mon, Jan 09, 2017 at 10:48:48AM -0800, Christopher Collins wrote:
> The boot sequence and upgrade procedure for the split image setup is a
> bit complicated.  I have been working on some documentation for this
> area, but haven't quite finished it.  I hope to have something ready in
> a day or two.  In the meantime, please feel free to ask any follow up
> questions.

FYI - The split image documentation has been updated on the Mynewt site:
http://mynewt.apache.org/latest/os/modules/split/split/

Chris

Re: How to change the CPU time frequency of mynewt and nimble stack

[Christopher Collins]

Hi Jiacheng,

On Wed, Jan 11, 2017 at 10:47:24AM +0800, WangJiacheng wrote:
> More information about nimble stack with 2 MHz CPU frequency. nimble-bleprph 
> can be connected by LightBlue, but after several seconds, it is disconnected 
> with message “Disconnected Alert: The peripheral has disconnected.”  With 4 
> MHz CPU frequency, nimble-bleprph can not be scanned by LightBlue.

Could you tell me more about the device running LightBlue?  At one
point, CoreBluetooth (the Apple Bluetooth library) had a bug that was
triggered when the NimBLE controller initiated the data length extension
procedure.  When this happened, the Apple device would send a bad
response, and the NimBLE device would disconnect.  If you are using an
older version of MacOS or iOS, then this could be the problem.

You can try disabling the data length extension feature to see if that
is the problem.  Do this by adding the following text to
targets//syscfg.yml:

syscfg.vals:
  BLE_LL_CFG_FEAT_DATA_LEN_EXT: 0

Thanks,
Chris

Re: How to change the CPU time frequency of mynewt and nimble stack

[Christopher Collins]

Hi Jiacheng,

I think your version of newt is still slightly out of date.  You can
install the latest as follows:

cd $GOPATH/src/mynewt.apache.org/newt/newt &&
   git checkout develop&&
   git pull origin develop &&
   go install  ;
   cd -


Thanks,
Chris

On Wed, Jan 11, 2017 at 09:04:05AM +0800, WangJiacheng wrote:
> Sterling,
> 
> Thanks.
> 
> Yes, the newt is already updated. “newt version” has return "Apache Newt 
> (incubating) version: 1.0.0-dev”.
> 
> Best Regards,
> 
> Jiacheng
> 
> > 在 2017年1月11日，08:58，Sterling Hughes  写道：
> > 
> > Hi Jiacheng,
> > 
> > You need to update your newt tool along with the new develop.
> > 
> > Best,
> > 
> > Sterling
> > 
> > On 10 Jan 2017, at 16:46, WangJiacheng wrote:
> > 
> >> Hi, Will,
> >> 
> >> I need more help, I have an error message when compile the target.
> >> 
> >> I’m currently working on the release branch, so upgrade to dev branch by:
> >>1. change file project.yml from "vers: 0-latest” to "vers: 0-dev”
> >>2. upgrade to dev branch “newt upgrade”
> >> 
> >> Then compile the target by “newt build nrf52_boot”, an error message as:
> >> 
> >> Building target targets/nrf52_boot
> >> Compiling boot.c
> >> Archiving boot.a
> >> Compiling bootutil_misc.c
> >> Compiling image_ec.c
> >> Compiling image_ec256.c
> >> Compiling image_rsa.c
> >> Compiling image_validate.c
> >> Compiling loader.c
> >> Archiving bootutil.a
> >> Error: In file included from aes.c:29:0:
> >> /Users/jiachengwang/dev/myproj/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10:
> >>  error: #include expects "FILENAME" or 
> >> #include MBEDTLS_USER_CONFIG_FILE
> >>  ^
> >> 
> >> it seems the config file "mbedtls/config_mynewt.h” define in 
> >> “crypto/mbedtls/pkg.yml” is missed.
> >> 
> >> Thanks,
> >> 
> >> Jiacheng
> >> 
> >> 
> >> 
> >> 
> >> 
> >>> 在 2017年1月10日，11:06，WangJiacheng  写道：
> >>> 
> >>> Thanks, Will.
> >>> 
> >>> There is an Internet connection issue to GitHub.com currently, I’ll 
> >>> update the code later.
> >>> 
> >>> Best Regards,
> >>> 
> >>> Jiacheng
> >>> 
> >>> 
>  在 2017年1月10日，10:10，will sanfilippo  写道：
>  
>  Hello:
>  
>  This issue should now be fixed in the latest development branch. Note 
>  that this is not working on the nrf51 platforms but since you were using 
>  nrf52 it should work.
>  
>  Let me know if you see any issues with it.
>  
>  
> > On Jan 8, 2017, at 6:20 PM, WangJiacheng  
> > wrote:
> > 
> > Hi, Will,
> > 
> > Thanks a lot for your reply.
> > 
> > Yes，the hardwear processor clock frequency of nRF52 (Cortex M4F) is 64 
> > MHz and can not be changed.
> > 
> > The reason of changing CLOCK_FREQ is that I want re-use the internal 
> > timing of mynewt already there with more accurate timing, by calling 
> > function "os_cputime_get32()”.  I’m trying to implement a (soft) IC 
> > card reader by nRF52 with mynewt OS and nimble stack running.
> > 
> > I am also considering to use an independent timer (NRF_TIMER3 or  
> > NRF_TIMER4) at the cost of about 0.1mA current. I already use 
> > NRF_TIMER2 to provide a 4 MHz clock signal output from GPIO of nRF52. 
> > By reading the source code of apache-mynewt-core, my understanding is 
> > that NRF_TIMER0 and NRF_TIMER1 is already used by mynewt OS and nimble 
> > stack, is my understanding correct?
> > 
> > Thanks,
> > 
> > Jiacheng
> > 
> >> 在 2017年1月9日，01:10，will sanfilippo  写道：
> >> 
> >> Those should be the only two parameters you need to configure. Must be 
> >> a bug in the controller :-)
> >> 
> >> I think it is worthwhile to point out that CLOCK_FREQ only changes the 
> >> units of os cputime; it does not affect the speed at which the 
> >> processor runs. At least, I could not see any other uses of 
> >> CLOCK_FREQ. So, these settings only affect the nimble stack and the 
> >> controller specifically (internal controller timing).
> >> 
> >> I am curious why you wanted to change this variable; what were you 
> >> trying to achieve?
> >> 
> >> Thanks for pointing this out; I will take a look to see why it is not 
> >> working.
> >> 
> >>> On Jan 7, 2017, at 10:48 PM, WangJiacheng  
> >>> wrote:
> >>> 
> >>> Hi,
> >>> 
> >>> The default CPU time frequency of Mynewt OS and Nimble stack is 1 
> >>> MHz, I try to change the CPU time frequency to be 2 MHz, I modified 
> >>> the related 2 config files:
> >>> configure file 

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Sun, Jan 08, 2017 at 01:04:54PM -0700, Jacob Rosenthal wrote:
> On Mon, Jan 2, 2017 at 2:46 PM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> > Do you know how big your loader is currently?
> 
> 
> Well, could be anything based on how much Im trying to strip from the
> nimble stack at any given time. but Im generally fighting to fit it within
> one slot which on the nrf51 is 110kB.

Yeah, it is currently pretty tough to get the OS + BLE stack to fit in
110kB.  Hopefully we'll get a chance to reduce the size of NimBLE soon.

> I'm also not sure if it would buy
> > you much extra flash space.
> 
> At this point Its not even about buying space, as I now have my other 110Kb
> back from the second image slot.. What I dont have anymore is update
> capability, so its about restoring that capability.
> 
> 
> > Im somewhat unclear if the 'loader' app can/should also serve as the
> > 'bootloader', or if it was intended that a separate bootloader would also
> > be present in addition to loader and application?
> >
> > This question of mine got buried above.. in this scenario, the distinction
> between loader and bootloader gets weird... My thought is theyre merged?

Sorry, I must have missed that question.  The boot loader is present in
the split image setup.  Here is how the boot sequence would look for the
various image setups:

1. Single image
* no boot loader
* image at address 0
* hardware boots directly into image

2. Stub image
* boot loader at address 0
* hardware boots into boot loader
* boot loader jumps to image in slot 0

3. Unified image (dual bank)
* boot loader at address 0
* hardware boots into boot loader
* if required, boot loader swaps images in the two image slots
  (i.e., if an upgrade or a fallback is being performed).
* boot loader jumps to image in slot 0

4. Split image
* boot loader at address 0
* hardware boots into boot loader
* if required, boot loader swaps images in the two image slots
  (i.e., if an upgrade or a fallback *between two loaders* is being
  performed).
* boot loader jumps to image in slot 0
* loader in slot 0 jumps to "app" in slot 1.

The boot sequence and upgrade procedure for the split image setup is a
bit complicated.  I have been working on some documentation for this
area, but haven't quite finished it.  I hope to have something ready in
a day or two.  In the meantime, please feel free to ask any follow up
questions.

Thanks,
Chris

Re: Compile and build time, any way to speed up?

[Christopher Collins]

Hi Cris,

On Fri, Jan 06, 2017 at 12:54:03PM -0500, Cris Frusina wrote:
> Fixed the TCP issue by restarting Virtualbox, did an newt upgrade and a 
> newt install with the 0-dev in the project file.
> 
> Still missing something as -j is not recognized.

The -j option is only supported in the develop branch.  To get the
develop version of newt:

cd $GOPATH/src/mynewt.apache.org/newt/newt &&
   git checkout develop
   git pull origin develop
   go install

But be careful: the develop version of newt is not compatible with
Mynewt version 1.0.0-b1.  If you use newt develop, then you'll also need
to use Mynewt develop.  You can upgrade your Mynewt repositories to the
develop branch as follows:

1. Open your project.yml in an editor.
2. Change all the "vers" fields to 0-dev (they are probably
   currently set to 0-latest).
3. Run "newt upgrade"

If you ever want to revert back to 1.0.0-b1, you can reverse the above
process, i.e., set the vers fields to 0-latest and run "newt upgrade".

Chris

> 
> ~/newt version
> Apache Newt (incubating) version: 1.0.0-b1
> 
> ~/newt build -j 5 ble_usb
> Error: unknown shorthand flag: 'j' in -j
> Usage:
>newt build  [target-names...] [flags]
> 
> Global Flags:
>-l, --loglevel string   Log level (default "WARN")
>-o, --outfile stringFilename to tee output to
>-q, --quiet Be quiet; only display error output
>-s, --silentBe silent; don't output anything
>-v, --verbose   Enable verbose output when executing commands
> 
> Cris
> 
> 
> 
> On 1/6/2017 12:27 PM, David G. Simmons wrote:
> > That looks like your network isn't working from your Docker instance. If 
> > that's actually the case, it could also (possibly) explain the torturously 
> > long compile times. Maybe.
> >
> > dg
> >> On Jan 6, 2017, at 12:24 PM, Cris Frusina  wrote:
> >>
> >> David,
> >>
> >> I've tried what you suggested I'm getting this error:
> >>
> >> *apache-mynewt-core**
> >> **Error: Get 
> >> https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master:
> >>  dial tcp: lookup api.github.com on 10.0.2.3:53: read udp 
> >> 172.17.0.2:38728->10.0.2.3:53: i/o timeout*
> >>
> >>
> >> My project file looks like this now:
> >>
> >> project.name: "my_project"
> >>
> >> project.repositories:
> >> - apache-mynewt-core
> >>
> >> # Use github's distribution mechanism for core ASF libraries.
> >> # This provides mirroring automatically for us.
> >> #
> >> repository.apache-mynewt-core:
> >> type: github
> >> vers: 0-dev
> >> user: apache
> >> repo: incubator-mynewt-core
> >>
> >>
> >> Am I missing something?
> >>
> >> Thanks,
> >> Cris
> >>
> >> On 1/6/2017 12:05 PM, David G. Simmons wrote:
> >>> You could do that, but if you don't want to remake the whole Docker 
> >>> instance, you should be able to simply edit the project.yml file and 
> >>> change the vers strin for repository.apache-mynewt-core to 0-dev and then 
> >>> run newt install
> >>>
> >>> Best regards,
> >>> dg
> >>>
>  On Jan 6, 2017, at 5:23 AM, Fabio Utzig  wrote:
> 
>  I'm not using the docker version myself, but I think you could try
>  rebuilding the docker image:
> 
>  https://github.com/runtimeinc/newt-docker 
>  
> 
>  Inside the Makefile, change the branch that is checked out from "-b
>  mynewt_1_0_0_b1_tag" to "-b develop":
> 
>  https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 
>  
> 
>  Att,
>  Fabio Utzig
> >>> --
> >>> David G. Simmons
> >>> (919) 534-5099
> >>> Web  • Blog  • 
> >>> Linkedin  • Twitter 
> >>>  • GitHub 
> >>> /** Message digitally signed for security and authenticity.
> >>> * If you cannot read the PGP.sig attachment, please go to
> >>>   * http://www.gnupg.com/  Secure your email!!!
> >>>   * Public key available at keyserver.pgp.com 
> >>> **/
> >>> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
> >>>
> >>> There are only 2 hard things in computer science: Cache invalidation, 
> >>> naming things, and off-by-one errors.
> >>>
> >>>
> >>>
> > --
> > David G. Simmons
> > (919) 534-5099
> > Web  • Blog  • 
> > Linkedin  • Twitter 
> >  • GitHub 
> > /** Message digitally signed for security and authenticity.
> > * If you cannot read the PGP.sig attachment, please go to
> >   * http://www.gnupg.com/  Secure your email!!!
> >   * Public key available at 

Re: Compile and build time, any way to speed up?

[Christopher Collins]

On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
> Hi David,
> 
> Sounds like a pain!
> 
> So I got an Ubuntu VM running on the Windows machine and installed the myNewt 
> natively. I didn't really see much of a speed bump (needs a bit further 
> testing). I'll try a different computer later today. Worst case I'll find a 
> Mac to try it on. 

Just an note - I ended up adding multithreaded build support to newt
yesterday.  In the develop branch, you can enable it with the
"-j " command line option, e.g.,

newt -j 5 build my_blinky_sim

A word of warning, though: don't upgrade your newt to develop if you
are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
Since the 1.0-b1 release, some backwards-compatible changes were made to
the newt tool.

Thanks,
Chris

Re: Compile and build time, any way to speed up?

[Christopher Collins]

On Thu, Jan 05, 2017 at 08:34:04AM -0500, David G. Simmons wrote:
> Well, I spent the entire day yesterday working on this, and unfortunately, I 
> don't have any real results. 
> 
> My only Windows setup is a VM, and it turns out that running a Docker 
> instance on a Windows VM on top of Mac OS X is a fairly unrewarding 
> experience in and of itself, and getting the mynewt stuff running in that 
> setup is, to a first approximation, not easily achieved. 

Ouch, yeah... I had a similar experience a while back, with a similar
amount of wasted time.  Eventually a I concluded that it just isn't
possible to run VirtualBox in a Windows VM (the docker solution uses
VirtualBox).

Chris

[jira] [Updated] (MYNEWT-537) newt - Multithreaded compiles

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins updated MYNEWT-537:
---
Description: 
To specify the number of concurrent threads, use the "-j" option:

{noformat}
newt -j 5 build bleprph-nrf52dk
newt -j 5 create-image bleprph-nrf52dk 1.2.3.4
newt -j 5 run bleprph-nrf52dk 0
{noformat}

> newt - Multithreaded compiles
> -
>
> Key: MYNEWT-537
> URL: https://issues.apache.org/jira/browse/MYNEWT-537
> Project: Mynewt
>  Issue Type: Bug
>    Reporter: Christopher Collins
>Assignee: Christopher Collins
>
> To specify the number of concurrent threads, use the "-j" option:
> {noformat}
> newt -j 5 build bleprph-nrf52dk
> newt -j 5 create-image bleprph-nrf52dk 1.2.3.4
> newt -j 5 run bleprph-nrf52dk 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-537) newt - Multithreaded compiles

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-537.

Resolution: Fixed

> newt - Multithreaded compiles
> -
>
> Key: MYNEWT-537
> URL: https://issues.apache.org/jira/browse/MYNEWT-537
> Project: Mynewt
>  Issue Type: Bug
>    Reporter: Christopher Collins
>    Assignee: Christopher Collins
>
> To specify the number of concurrent threads, use the "-j" option:
> {noformat}
> newt -j 5 build bleprph-nrf52dk
> newt -j 5 create-image bleprph-nrf52dk 1.2.3.4
> newt -j 5 run bleprph-nrf52dk 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-537) newt - Multithreaded compiles

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-537:
--

 Summary: newt - Multithreaded compiles
 Key: MYNEWT-537
 URL: https://issues.apache.org/jira/browse/MYNEWT-537
 Project: Mynewt
  Issue Type: Bug
Reporter: Christopher Collins
Assignee: Christopher Collins






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-535) newt - Include target name in image manifest

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-535.

Resolution: Fixed

> newt - Include target name in image manifest
> 
>
> Key: MYNEWT-535
> URL: https://issues.apache.org/jira/browse/MYNEWT-535
> Project: Mynewt
>  Issue Type: Bug
>  Components: Newt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
>
> An image's manifest should be self describing when possible.  The manifest 
> doesn't currently contain the target name, except for the target package 
> descriptor hidden in the package list.
> We should add the following to each image manifest:
> {noformat}
>   "name": ""
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-535) newt - Include target name in image manifest

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-535:
--

 Summary: newt - Include target name in image manifest
 Key: MYNEWT-535
 URL: https://issues.apache.org/jira/browse/MYNEWT-535
 Project: Mynewt
  Issue Type: Bug
  Components: Newt
Reporter: Christopher Collins
Assignee: Christopher Collins


An image's manifest should be self describing when possible.  The manifest 
doesn't currently contain the target name, except for the target package 
descriptor hidden in the package list.

We should add the following to each image manifest:

{noformat}
  "name": ""
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MYNEWT-287) Host Flow Control

[Christopher Collins (JIRA)]

[ 
https://issues.apache.org/jira/browse/MYNEWT-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798679#comment-15798679
 ] 

Christopher Collins commented on MYNEWT-287:


Thanks, Johan, that is very interesting to hear.  Will and I were just 
discussing this issue yesterday: flow control for ACL data, but no means of 
stopping the controller from sending events.  After giving it some thought, I 
agree that relying on the underlying transport's flow control mechanism should 
be sufficient (better actually, because it controls the flow of both data and 
events).  I wonder why controller-to-host flow control is even specified in 
bluetooth.  Do you have any insight into the rationale?  Maybe this flow 
control was specified in case a new HCI transport which does not have its own 
flow control mechanism gets added to the spec.

> Host Flow Control
> -
>
> Key: MYNEWT-287
> URL: https://issues.apache.org/jira/browse/MYNEWT-287
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Nimble
> Environment: Using nimble from an external host stack
>Reporter: Johan Hedberg
>    Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> If the Nimble controller part is connected to an external host stack the host 
> may need to control the flow of ACL packets from the controller to the host. 
> This is particularly important for resource-constrained hosts that have a 
> limited amount of RX ACL data buffers.
> The Bluetooth core specification provides a standard feature to accomplish 
> this called Host Flow Control. (Vol 2, Part E, Section 4.2). To implement 
> this the controller needs to support the "Set Controller To Host Flow 
> Control", "Host Buffer Size" and "Host Number Of Completed Packets" HCI 
> commands.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MYNEWT-287) Host Flow Control

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins reassigned MYNEWT-287:
--

Assignee: Christopher Collins

> Host Flow Control
> -
>
> Key: MYNEWT-287
> URL: https://issues.apache.org/jira/browse/MYNEWT-287
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Nimble
> Environment: Using nimble from an external host stack
>Reporter: Johan Hedberg
>    Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> If the Nimble controller part is connected to an external host stack the host 
> may need to control the flow of ACL packets from the controller to the host. 
> This is particularly important for resource-constrained hosts that have a 
> limited amount of RX ACL data buffers.
> The Bluetooth core specification provides a standard feature to accomplish 
> this called Host Flow Control. (Vol 2, Part E, Section 4.2). To implement 
> this the controller needs to support the "Set Controller To Host Flow 
> Control", "Host Buffer Size" and "Host Number Of Completed Packets" HCI 
> commands.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-513) sysinit - Remove panic function from syscfg.yml; configure at runtime.

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-513.

Resolution: Fixed

> sysinit - Remove panic function from syscfg.yml; configure at runtime.
> --
>
> Key: MYNEWT-513
> URL: https://issues.apache.org/jira/browse/MYNEWT-513
> Project: Mynewt
>  Issue Type: Improvement
>  Components: Misc
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MYNEWT-534) A cooperative package's "start event" might not get executed when OS started

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins updated MYNEWT-534:
---
Description: 
*Vocabulary for this ticket:*

+Cooperative package:+ A package which uses an eventq to do work, but which 
doesn't have its own task.  Such a package is told which eventq / task to use 
at runtime, or it just uses the eventq designated as the default.

+Start event:+ An OS event that a package needs to execute as soon as the OS 
starts.  Such an event generally "kicks off" the package.  Example: the BLE 
host's start event initiates communication with the controller.

--

*Problem Description:*
The problem concerns cooperative packages which aren't explicitly told which 
event queue to use, i.e., a package which uses the default event queue.  This 
type of package only enqueues its start event when it tries to enqueue a second 
event.  For some packages, this is not noticible, because the application calls 
into the package's API on start up, kicking off the start event.  In other 
cases, the package is meant to do things on its own with no interaction from 
the app (example: mgmt/newtmgr).  These packages are broken unless the app 
explicitly specifies an eventq for them to use.

The reason for the odd behavior is that a package doesn't know which eventq it 
will be used when it first gets initialized.  If the app explicitly assigns an 
eventq to a package, the package is able to enqueue its start event at that 
time, evading this problem.  Packages which use the default event queue never 
get an opportunity to enqueue their start event because the default queue may 
not be designated when the package is initialized.  Furthermore, enqueueing to 
the default event queue would be premature, as the app may set the pacakge's 
event queue shortly afterwards.

--

*Possible solutions:*
1. Require the app to designate the default event queue before calling 
sysinit().  When a cooperative package is initialized, it immediately enqueues 
its start event onto the default event queue.  If a package's event queue is 
then explicitly set, the start event is removed from the default queue and 
placed on the specified queue.

Problems with this approach are: it places some odd restrictions on how the 
main() function should look.  Creating an event queue and designating it as the 
default prior to calling sysinit() probably won't look right.  

2. Cooperative packages register themselves at init-time.  When the OS is 
started, all registered packages enqueue their start event on the assigned 
queue (or default if none assigned).

The downside of this approach is that it requires extra ram to hold the 
registration list, and a small amount of code to call the registration function 
in each pacakge's init function.

3. Place all start events in a special linker section.  When the OS is started, 
each event in the linker section is enqueued.  This approach doesn't require 
any extra RAM.

Problems with this approach: linker scripts must be modified to be made aware 
of the new linker section.  Also, a package's event doesn't get included in the 
final binary if none of the package's symbols are referenced by name.  An 
example of such a package is newtmgr; this package starts itself, and is then 
accessed indirectly through function pointers.

  was:
*Vocabulary for this ticket:*

*Cooperative package:* A package which uses an eventq to do work, but which 
doesn't have its own task.  Such a package is told which eventq / task to use 
at runtime, or it just uses the eventq designated as the default.

*Start event:* An OS event that a package needs to execute as soon as the OS 
starts.  Such an event generally "kicks off" the package.  Example: the BLE 
host's start event initiates communication with the controller.

--

*Problem Description:*
The problem concerns cooperative packages which aren't explicitly told which 
event queue to use, i.e., a package which uses the default event queue.  This 
type of package only enqueues its start event when it tries to enqueue a second 
event.  For some packages, this is not noticible, because the application calls 
into the package's API on start up, kicking off the start event.  In other 
cases, the package is meant to do things on its own with no interaction from 
the app (example: mgmt/newtmgr).  These packages are broken unless the app 
explicitly specifies an eventq for them to use.

The reason for the odd behavior is that a package doesn't know which eventq it 
will be used when it first gets initialized.  If the app explicitly assigns an 
eventq to a package, the package is able to enqueue its start event at that 
time, evading this problem.  Packages which use the default event queue never 
get an opportunity to enqueue their start event because the default queue may 
not be designated when the package is initialized.  Fur

[jira] [Updated] (MYNEWT-534) A cooperative package's "start event" might not get executed when OS started

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins updated MYNEWT-534:
---
Description: 
*Vocabulary for this ticket:*

*Cooperative package:* A package which uses an eventq to do work, but which 
doesn't have its own task.  Such a package is told which eventq / task to use 
at runtime, or it just uses the eventq designated as the default.

*Start event:* An OS event that a package needs to execute as soon as the OS 
starts.  Such an event generally "kicks off" the package.  Example: the BLE 
host's start event initiates communication with the controller.

--

*Problem Description:*
The problem concerns cooperative packages which aren't explicitly told which 
event queue to use, i.e., a package which uses the default event queue.  This 
type of package only enqueues its start event when it tries to enqueue a second 
event.  For some packages, this is not noticible, because the application calls 
into the package's API on start up, kicking off the start event.  In other 
cases, the package is meant to do things on its own with no interaction from 
the app (example: mgmt/newtmgr).  These packages are broken unless the app 
explicitly specifies an eventq for them to use.

The reason for the odd behavior is that a package doesn't know which eventq it 
will be used when it first gets initialized.  If the app explicitly assigns an 
eventq to a package, the package is able to enqueue its start event at that 
time, evading this problem.  Packages which use the default event queue never 
get an opportunity to enqueue their start event because the default queue may 
not be designated when the package is initialized.  Furthermore, enqueueing to 
the default event queue would be premature, as the app may set the pacakge's 
event queue shortly afterwards.

*Possible solutions:*
1. Require the app to designate the default event queue before calling 
sysinit().  When a cooperative package is initialized, it immediately enqueues 
its start event onto the default event queue.  If a package's event queue is 
then explicitly set, the start event is removed from the default queue and 
placed on the specified queue.

Problems with this approach are: it places some odd restrictions on how the 
main() function should look.  Creating an event queue and designating it as the 
default prior to calling sysinit() probably won't look right.  

2. Cooperative packages register themselves at init-time.  When the OS is 
started, all registered packages enqueue their start event on the assigned 
queue (or default if none assigned).

The downside of this approach is that it requires extra ram to hold the 
registration list, and a small amount of code to call the registration function 
in each pacakge's init function.

3. Place all start events in a special linker section.  When the OS is started, 
each event in the linker section is enqueued.  This approach doesn't require 
any extra RAM.

Problems with this approach: linker scripts must be modified to be made aware 
of the new linker section.  Also, a package's event doesn't get included in the 
final binary if none of the package's symbols are referenced by name.  An 
example of such a package is newtmgr; this package starts itself, and is then 
accessed indirectly through function pointers.

  was:
Vocabulary for this ticket:

*Cooperative package:* A package which uses an eventq to do work, but which 
doesn't have its own task.  Such a package is told which eventq / task to use 
at runtime, or it just uses the eventq designated as the default.

*Start event:* An OS event that a package needs to execute as soon as the OS 
starts.  Such an event generally "kicks off" the package.  Example: the BLE 
host's start event initiates communication with the controller.

--

*Problem Description:*
The problem concerns cooperative packages which aren't explicitly told which 
event queue to use, i.e., a package which uses the default event queue.  This 
type of package only enqueues its start event when it tries to enqueue a second 
event.  For some packages, this is not noticible, because the application calls 
into the package's API on start up, kicking off the start event.  In other 
cases, the package is meant to do things on its own with no interaction from 
the app (example: mgmt/newtmgr).  These packages are broken unless the app 
explicitly specifies an eventq for them to use.

The reason for the odd behavior is that a package doesn't know which eventq it 
will be used when it first gets initialized.  If the app explicitly assigns an 
eventq to a package, the package is able to enqueue its start event at that 
time, evading this problem.  Packages which use the default event queue never 
get an opportunity to enqueue their start event because the default queue may 
not be designated when the package is initialized.  Furthermore, enqueu

[jira] [Created] (MYNEWT-534) A cooperative package's "start event" might not get executed when OS started

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-534:
--

 Summary: A cooperative package's "start event" might not get 
executed when OS started
 Key: MYNEWT-534
 URL: https://issues.apache.org/jira/browse/MYNEWT-534
 Project: Mynewt
  Issue Type: Bug
Reporter: Christopher Collins


Vocabulary for this ticket:

*Cooperative package:* A package which uses an eventq to do work, but which 
doesn't have its own task.  Such a package is told which eventq / task to use 
at runtime, or it just uses the eventq designated as the default.

*Start event:* An OS event that a package needs to execute as soon as the OS 
starts.  Such an event generally "kicks off" the package.  Example: the BLE 
host's start event initiates communication with the controller.

--

*Problem Description:*
The problem concerns cooperative packages which aren't explicitly told which 
event queue to use, i.e., a package which uses the default event queue.  This 
type of package only enqueues its start event when it tries to enqueue a second 
event.  For some packages, this is not noticible, because the application calls 
into the package's API on start up, kicking off the start event.  In other 
cases, the package is meant to do things on its own with no interaction from 
the app (example: mgmt/newtmgr).  These packages are broken unless the app 
explicitly specifies an eventq for them to use.

The reason for the odd behavior is that a package doesn't know which eventq it 
will be used when it first gets initialized.  If the app explicitly assigns an 
eventq to a package, the package is able to enqueue its start event at that 
time, evading this problem.  Packages which use the default event queue never 
get an opportunity to enqueue their start event because the default queue may 
not be designated when the package is initialized.  Furthermore, enqueueing to 
the default event queue would be premature, as the app may set the pacakge's 
event queue shortly afterwards.

*Possible solutions:*
1. Require the app to designate the default event queue before calling 
sysinit().  When a cooperative package is initialized, it immediately enqueues 
its start event onto the default event queue.  If a package's event queue is 
then explicitly set, the start event is removed from the default queue and 
placed on the specified queue.

Problems with this approach are: it places some odd restrictions on how the 
main() function should look.  Creating an event queue and designating it as the 
default prior to calling sysinit() probably won't look right.  

2. Cooperative packages register themselves at init-time.  When the OS is 
started, all registered packages enqueue their start event on the assigned 
queue (or default if none assigned).

The downside of this approach is that it requires extra ram to hold the 
registration list, and a small amount of code to call the registration function 
in each pacakge's init function.

3. Place all start events in a special linker section.  When the OS is started, 
each event in the linker section is enqueued.  This approach doesn't require 
any extra RAM.

Problems with this approach: linker scripts must be modified to be made aware 
of the new linker section.  Also, a package's event doesn't get included in the 
final binary if none of the package's symbols are referenced by name.  An 
example of such a package is newtmgr; this package starts itself, and is then 
accessed indirectly through function pointers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Sun, Jan 01, 2017 at 11:10:01PM -0700, Jacob Rosenthal wrote:
> On low flash devices like nrf51822 FLASH_AREA_IMAGE_1 equal to
> FLASH_AREA_IMAGE_0 is improbable, which means dual bank is likewise
> improbable and as a result we've talked previously about how to shrink
> FLASH_AREA_IMAGE_1 down to a stub.
> 
> However, my current understanding is newtmgr/imagemgr implementation is
> dual bank, wherein the application saves received bytes into
> COREDUMP_FLASH_AREA (ie FLASH_AREA_IMAGE_1) and reboots allowing bootloader
> (apps/boot) to safely make the swap.

Your understanding sounds correct to me.  If you don't have enough flash
for a second image slot, then image upgrade isn't possible.

Just a clarification- for a split image, both slots don't need to be
exactly the same size; the slot sizes just need to meet these
requirements:

* Both slots are large enough to accommodate the loader.
* Second slot is large enough to accommodate the app.

In other words, if your app is larger than your loader, you only need to
budget for the extra space in the second slot.  Unfortunately, I don't
think this allowance helps in your case.  I think the problem you're
facing is that the loader (kernel + BLE stack) is too large.

> I don't believe theres a good way to do a single bank receive from within
> the running application, which would mean this has to be moved into the
> bootloader, which means it need to share access to nimble, which means a
> bootloader fork that includes splitty, nimble, newtmgr, imagemgr.
> 
> Im somewhat unclear if the 'loader' app can/should also serve as the
> 'bootloader', or if it was intended that a separate bootloader would also
> be present in addition to loader and application?
> 
> Thoughts on this architecture, am I missing anything?

Hmm... I do see what you mean, but that sounds like a pretty fundamental
change to how image management works.  I'm also not sure if it would buy
you much extra flash space.  The boot loader is already fairly stripped
down, and it doesn't contain much code that could be shared by the image
upgrade mechanism.

I don't want to be dismissive of your idea, but my thinking is that the
best way forward is to try to reduce the size of the BLE stack.  This is
something that has been on the radar for quite a while.  We haven't
really looked at this at all, so there is probably some easy savings to
be had.  I have been meaning to do a quick survey of the BLE host to
determine what is contributing to its code size.  That said, if you want
to look at incorporating upgrade functionality into the boot loader,
don't let me stop you :).

Do you know how big your loader is currently?

Thanks,
Chris

Re: MyNewt Mqueue not working

[Christopher Collins]

Hi Then,

On Mon, Jan 02, 2017 at 01:23:33PM +0800, then yon wrote:
> Dear Support,
> 
> I been using MyNewt v1.0.0-dev, and i tried to make use of Mqueue.
> 
> What i did was following the tutorial below:
> 
> https://mynewt.apache.org/os/core_os/mqueue/mqueue/
> Since the tutorial is not updated, i tried to modify it but unfortunately i 
> can't make it work.

[...]

Your code looks good.  There is just one crucial step missing: a task
needs to poll the eventq you created (queue_task_evq).  Just to clarify
how mqueues work: when os_mqueue_put() is called, two things happen:
1. The specified mbuf is placed on the mqueue.
2. The event associated with the mqueue is put on the specified
   eventq, but only if the event is not already enqueued.

If a task is listening on the eventq, then it can your mqueue processing
function when it sees the mqueue event.

I think you have two options:
1. Remove queue_task_evq from your code; use the other eventq
   (blinky_evq) with your mqueue.  blinky_evq is already being
   processed correctly by the shell_task_handler() function, so if
   you enqueue an mqueue event to it, your mqueue data will get
   processed.

2. Create an additional task which processes the queue_task_evq
   eventq.  This task's handler would probably look identical to
   shell_task_handler(), i.e., a tight loop that repeatedly calls
   os_eventq_run().

I recommend trying the simpler option 1 first.  Generally, it is best to
minimize the number of tasks your app uses so that it requires less RAM.

Thanks,
Chris

Re: Removing or shrinking flash areas

[Christopher Collins]

Darn, I forgot to mention a critical point.  To use the slot-1 stub
solution, you will probably need to build and upload a new boot loader.
Stub slot support was added to the develop branch on Dec. 21, 2016
(https://issues.apache.org/jira/browse/MYNEWT-520).

Chris

On Fri, Dec 30, 2016 at 03:39:40PM -0800, Christopher Collins wrote:
> Hi Jacob,
> 
> On Fri, Dec 30, 2016 at 03:28:08PM -0700, Jacob Rosenthal wrote:
> > I think deps are a part of the story, but yeah the goal is to successfully
> > get a target bsp back to 1 larger flash image, or as many as the common set
> > of deps require but shrunk way down.
> 
> So you just want to eliminate the need for a second image slot in the
> flash map?
> 
> There are four flash areas that are relevant here.  Just to be clear: a
> flash area is simply a named region of flash.  Each BSP's bsp.yml file
> defines its flash map as a sequence of flash areas.
> 
> Anyway, the four relevant flash areas are:
> * FLASH_AREA_BOOTLOADER:
> * FLASH_AREA_IMAGE_0:
> * FLASH_AREA_IMAGE_1:
> * FLASH_AREA_IMAGE_SCRATCH:
> 
> The boot loader area is placed at the board's start address in flash.
> That is, the device boots directly into the boot loader.  The boot
> loader then possibly swaps the two image slots and jumps to the image in
> slot 0.  This process is described in some detail in
> boot/bootutil/design.txt.
> 
> If your device only needs a single image, you have two options:
> 1. slot-1 stub
> 2. no boot loader
> 
> Option 1 is the simplest, but option 2 gives you more flash space.  I
> will describe both options below.
> 
> *** Slot-1 stub:
> 
> This option involves greatly reducing the size of the FLASH_AREA_IMAGE_1
> area.  When I did this with the nRF51dk, I used an area size of 2kB.  I
> think you shld be able to reduce it to the size of an individual
> sector (1kB), but I have not tried this.
> 
> There are two changes you need to make:
> * (bsp.yml)
> o Increase the size of FLASH_AREA_IMAGE_0.
> o Reduce the size of FLASH_AREA_IMAGE_1.
> o Increase the offset of FLASH_AREA_IMAGE_0.
> 
> * (nrf51xxac.ld)
> o Increase the size of the FLASH region.
> 
> I have attached modified versions of these two files that you can try.
> 
> *** No boot loader:
> 
> This option involves removing the boot loader entirely.
> Instead of a boot loader, the image binary is placed at the board's
> start address in flash such that the device boots directly into it.
> Without a boot loader, you can eliminate the following two areas from
> the flash map:
> * FLASH_AREA_BOOTLOADER:
> * FLASH_AREA_IMAGE_1:
> 
> Unfortunately, the scratch area cannot be eliminated at the moment,
> since it is required by the reboot log.  I think this is something we
> should address soon, but in the meantime, you are stuck with the scratch
> area.
> 
> It has been a while since I've tried running an image without a boot
> loader, so my memory might be a bit foggy.  I think the following steps
> are all you need:
> * (bsp.yml)
> o Remove FLASH_AREA_BOOTLOADER
> o Remove FLASH_AREA_IMAGE_1
> o Increase the size of FLASH_AREA_IMAGE_0.
> o Reduce the offset of FLASH_AREA_IMAGE_0.
> 
> * (nrf51xxac.ld)
> o Increase the size of the FLASH region.
> o Set _imghdr_size to 0.
> 
> You will probably also need to move the reboot log flash area.
> Currently, most BSPs place it right between the boot loader and image
> slot 0.
> 
> Let me know if you have any questions.
> 
> Thanks,
> Chris

> #
> # Licensed to the Apache Software Foundation (ASF) under one
> # or more contributor license agreements.  See the NOTICE file
> # distributed with this work for additional information
> # regarding copyright ownership.  The ASF licenses this file
> # to you under the Apache License, Version 2.0 (the
> # "License"); you may not use this file except in compliance
> # with the License.  You may obtain a copy of the License at
> #
> #  http://www.apache.org/licenses/LICENSE-2.0
> #
> # Unless required by applicable law or agreed to in writing,
> # software distributed under the License is distributed on an
> # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> # KIND, either express or implied.  See the License for the
> # specific language governing permissions and limitations
> # under the License.
> #
> 
> bsp.arch: cortex_m0
> bsp.compiler: compiler/arm-none-eabi-m0
> bsp.linkerscript:
> - "hw/bsp/nrf51dk/nrf51xxac.ld"
> - "hw/mcu/nordic/nrf51xxx/nrf51.ld"
> bsp.linkerscript.BOOT_LOADER.OVERWRITE:
>   

Re: Removing or shrinking flash areas

[Christopher Collins]

Hi Jacob,

On Fri, Dec 30, 2016 at 03:28:08PM -0700, Jacob Rosenthal wrote:
> I think deps are a part of the story, but yeah the goal is to successfully
> get a target bsp back to 1 larger flash image, or as many as the common set
> of deps require but shrunk way down.

So you just want to eliminate the need for a second image slot in the
flash map?

There are four flash areas that are relevant here.  Just to be clear: a
flash area is simply a named region of flash.  Each BSP's bsp.yml file
defines its flash map as a sequence of flash areas.

Anyway, the four relevant flash areas are:
* FLASH_AREA_BOOTLOADER:
* FLASH_AREA_IMAGE_0:
* FLASH_AREA_IMAGE_1:
* FLASH_AREA_IMAGE_SCRATCH:

The boot loader area is placed at the board's start address in flash.
That is, the device boots directly into the boot loader.  The boot
loader then possibly swaps the two image slots and jumps to the image in
slot 0.  This process is described in some detail in
boot/bootutil/design.txt.

If your device only needs a single image, you have two options:
1. slot-1 stub
2. no boot loader

Option 1 is the simplest, but option 2 gives you more flash space.  I
will describe both options below.

*** Slot-1 stub:

This option involves greatly reducing the size of the FLASH_AREA_IMAGE_1
area.  When I did this with the nRF51dk, I used an area size of 2kB.  I
think you shld be able to reduce it to the size of an individual
sector (1kB), but I have not tried this.

There are two changes you need to make:
* (bsp.yml)
o Increase the size of FLASH_AREA_IMAGE_0.
o Reduce the size of FLASH_AREA_IMAGE_1.
o Increase the offset of FLASH_AREA_IMAGE_0.

* (nrf51xxac.ld)
o Increase the size of the FLASH region.

I have attached modified versions of these two files that you can try.

*** No boot loader:

This option involves removing the boot loader entirely.
Instead of a boot loader, the image binary is placed at the board's
start address in flash such that the device boots directly into it.
Without a boot loader, you can eliminate the following two areas from
the flash map:
* FLASH_AREA_BOOTLOADER:
* FLASH_AREA_IMAGE_1:

Unfortunately, the scratch area cannot be eliminated at the moment,
since it is required by the reboot log.  I think this is something we
should address soon, but in the meantime, you are stuck with the scratch
area.

It has been a while since I've tried running an image without a boot
loader, so my memory might be a bit foggy.  I think the following steps
are all you need:
* (bsp.yml)
o Remove FLASH_AREA_BOOTLOADER
o Remove FLASH_AREA_IMAGE_1
o Increase the size of FLASH_AREA_IMAGE_0.
o Reduce the offset of FLASH_AREA_IMAGE_0.

* (nrf51xxac.ld)
o Increase the size of the FLASH region.
o Set _imghdr_size to 0.

You will probably also need to move the reboot log flash area.
Currently, most BSPs place it right between the boot loader and image
slot 0.

Let me know if you have any questions.

Thanks,
Chris
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

bsp.arch: cortex_m0
bsp.compiler: compiler/arm-none-eabi-m0
bsp.linkerscript:
- "hw/bsp/nrf51dk/nrf51xxac.ld"
- "hw/mcu/nordic/nrf51xxx/nrf51.ld"
bsp.linkerscript.BOOT_LOADER.OVERWRITE:
- "hw/bsp/nrf51dk/boot-nrf51xxac.ld"
- "hw/mcu/nordic/nrf51xxx/nrf51.ld"
bsp.part2linkerscript: "hw/bsp/nrf51dk/split-nrf51dk.ld"
bsp.downloadscript: "hw/bsp/nrf51dk/nrf51dk_download.sh"
bsp.debugscript: "hw/bsp/nrf51dk/nrf51dk_debug.sh"

bsp.flash_map:
areas:
# System areas.
FLASH_AREA_BOOTLOADER:
device: 0
offset: 0x
size: 16kB
FLASH_AREA_IMAGE_0:
device: 0
offset: 0x8000
size: 218kB
FLASH_AREA_IMAGE_1:
device: 0
offset: 0x0003e800
size: 2kB
FLASH_AREA_IMAGE_SCRATCH:
device: 0
offset: 0x0003f000
size: 2kB

# User areas.
FLASH_AREA_REBOOT_LOG:
user_id: 0
device: 0
offset: 0x4000
size: 16kB
FLASH_AREA_NFFS:
user_id: 1
device: 0

[jira] [Commented] (MYNEWT-520) boot - Allow incompatible image slots

[Christopher Collins (JIRA)]

[ 
https://issues.apache.org/jira/browse/MYNEWT-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15788387#comment-15788387
 ] 

Christopher Collins commented on MYNEWT-520:


Hi Daniel,

Yes, exactly.  This change removes the requirement to allocate a second 
equally-sized image slot in flash.

Generally, a device that uses only a single image won't use the boot loader at 
all.  Instead, the binary executable just gets placed at start address, and the 
device boots directly into the image.  Doing this requires a few changes to the 
BSP flash map and linker scripts.  For quick testing, it is easier to just use 
the boot loader, which is why this change was made.

> boot - Allow incompatible image slots
> -
>
> Key: MYNEWT-520
> URL: https://issues.apache.org/jira/browse/MYNEWT-520
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Bootloader
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, the boot loader immediately fails if the two image slots have 
> different flash characteristics (different sector count or different sector 
> sizes).
> Instead, the boot loader should only require compatible slots if a swap is 
> required.  If the two image slots are not compatible, the boot loader should 
> just boot into slot 0 unconditionally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Removing or shrinking flash areas

[Christopher Collins]

Hi Jacob,

On Fri, Dec 30, 2016 at 01:25:11PM -0700, Jacob Rosenthal wrote:
> I worked with Sterling to shrink the nimble stack for 256k flash devices,
> but Im starting to cut into pretty basic functionality. Ive noticed that
> even not using splitty/flash reboot logging those are enable in the bsp I
> forked (nrf51dk-16kbram)
> 
> Any guidance on removing or shrinking those? Whats mandatory? Ive tried but
> Im tending to run into compile errors of nested deps looking for them.
> 
> Related, anyone done a more basic dfu style bootload without netwmgr and
> such as I wont have flash or ram for all of that.

I don't think we have a clear picture of exactly which packages are
required.  The following commands might be helpful for determining who
depends on what:

newt target dep 
newt target revdep 

The first shows a dependency graph; for each package in the build, show
the list of packages that it depends on.

The second shows the reverse dependency graph; for each pacakge in the
build, show the list of packages that depend on it.

Unfortunately, I am sure there are some dependencies that are necessary
for the wrong reasons.  We may need to do some work to remove these
dependencies, or add syscfg settings so that they can be compiled out.

If you could post the output of the two commands I mentioned above, I
can take a look to see if anything can be removed.

Chris

Re: ble_gattc_disc_svc_by_uuid problems

[Christopher Collins]

Hi Jacob,

You will need to upgrade newt as well.  You can do that as follows:

cd $GOPATH &&
git checkout develop &&
git pull &&
go install

Hopefully that fixes the compile problem.  Please report back if you
encounter any issues.

Thanks,
Chris


On Tue, Dec 27, 2016 at 04:40:34PM -0700, Jacob Rosenthal wrote:
> Thanks for the quick replies. I changed to
> repository.apache-mynewt-core:
> type: github
> vers: 0-latest
> user: apache
> repo: incubator-mynewt-core
> 
> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newt run
> nrf51822xxaa_blecenthr 0.0.0
> Compiling main.c
> Compiling misc.c
> Compiling peer.c
> Archiving blecenthr.a
> Error: In file included from aes.c:29:0:
> /Users/jacobrosenthal/Downloads/mynewt-hr-observer/repos/apache-mynewt-core/
> crypto/mbedtls/include/mbedtls/config.h:2522:10: error: #include expects
> "FILENAME" or 
>  #include MBEDTLS_USER_CONFIG_FILE
> 
> 
> Is upstream dev broken atm? or is there some quick config file fix?  Or
> whats another quick way to move to something other than latest? Id just
> apply that one patch but looks like its based on a bunch of changes I dont
> have in latest.
> 
> 
> Hrm.. also I cant find that in the github mirror.. It was closed there not
> merged, does that mean it was merged somewhere on the apache side?
> 
> @Andrzej Kaczmarek Ah, I see that error in the logs, though it looks like
> (this older) version of ble_gattc_disc_svc_by_uuid calls back (once)
> to BLE_HS_EDONE,
> even though the error exists. But who cares what this version does, out
> with the old.
> 
> 
> 
> 
> 
> On Tue, Dec 27, 2016 at 3:28 PM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> >
> > Could you show the code you are using?  I haven't noticed any issues
> > with the discover primary services by UUID operation.
> >
> > > 2357:[ts=18414036ss

Re: ble_gattc_disc_svc_by_uuid problems

[Christopher Collins]

Hi Jacob,

On Tue, Dec 27, 2016 at 02:37:01PM -0700, Jacob Rosenthal wrote:
> Hey all, been working with mynewt for a week or so now, trying to
> understand the code and slim the nimble stack down for nrf51822qfaa
> targets. Playing with central role now (modified blecent), and Im having
> some trouble searching for a single svc by uuid (to save peer struct
> memory, rather than indexing them all)
> 
> Either I dont understand ble_gattc_disc_svc_by_uuid or it has problems.
> 
> Its returning successfully but with a different uuid than I would expect
> and weird handles

Could you show the code you are using?  I haven't noticed any issues
with the discover primary services by UUID operation.

> 2357:[ts=18414036ssb, mod=4 level=1] GATT procedure initiated: discover
> service by uuid; uuid=180d--1000-80805f9b34fb
> 2360:[ts=18437472ssb, mod=4 level=0] host tx hci data; handle=1 length=27
> 2361:[ts=18445284ssb, mod=4 level=0] ble_hs_hci_acl_tx(): 0x01 0x00 0x1b
> 0x00 0x17 0x00 0x04 0x00 0x06 0x01 0x00 0xff 0xff 0x00 0x28 0xfb 0x34 0x9b
> 0x5f 0x80 0x00 0x00 0x80 0x00 0x10 0x00 0x00 0x0d 0x18 0x00 0x00
> 2364:[ts=18468720ssb, mod=4 level=0] txed att command: find type value req;
> conn=1 start_handle=0x0001 end_handle=0x attr_type=10240
> 2372:[ts=18531216ssb, mod=4 level=0] Number of Completed Packets:
> num_handles=1
> 2374:[ts=18546840ssb, mod=4 level=0] handle:1 pkts:1
> 2379:[ts=18585900ssb, mod=4 level=0] ble_hs_hci_evt_acl_process(): handle=1
> pb=2 len=9 data=0x05 0x00 0x04 0x00 0x01 0x06 0x01 0x00 0x0a
> 2382:[ts=18609336ssb, mod=4 level=0] rxed att command: error rsp; conn=1
> req_op=6 handle=0x0001 error_code=10
> 2384:[ts=18624960ssb, mod=64 level=3] Service discovery complete; status=0
> conn_handle=1

It looks like the peer is reporting that it doesn't support the service
being requested (heart rate).  The peer responds to the Find By Type
Value Request with an error response (status=10; attribute not found).

One thing to be careful about: be sure to check the status parameter in
your GATT callback.  If the status is not zero, then the service data is
not valid.  In this case, I would expect the status to be equal to 14
(BLE_HS_EDONE).  I believe this explains why you are seeing weird
handle and UUID values.

The Discover Primary Service by UUID API is not obvious.  The callback
may get called multiple times; the last time it gets called, the status
will be nonzero (BLE_HS_EDONE on successful completion).  This probably
seems weird, since you are only trying to discover a single service.
The reason for the strange semantics is that a device is allowed to have
multiple services with the same UUID; this API is hampered to allow for
this obscure scenario.

Unfortunately, I'm not sure why the device is reporting that it doesn't
support that service.  Is the peer a Mynewt device?

One thing that looks strange is that your device is trying to discover
the heart rate service's 128-bit UUID.  If a UUID can be represented in
16 bits, then the Nimble host should specify the shorter UUID.  This is
something that the host does automatically as long as the application
specifies the UUID correctly.  From the log, the 128-bit UUID looks
correct, so I am not sure why all 128 bits are being transmitted.
This is just an optimization; the 128-bit UUID should work also.  This
might be a clue, though.

[...]

> I dont know if its helpful, but I noticed ble_gattc_disc_svc_by_uuid is
> using the find api (ble_att_clt_tx_find_type_value) instead of a read api
> (ble_att_clt_tx_read_group_type in the case of ble_gattc_disc_all_svcs)

That is OK; the Discover Primary Service by Service UUID GATT procedure
uses the Find By Type Value Request ATT command.  Section 3.G.4.4.2 in
the 4.2 spec contains a sequence diagram illustrating this (I just
mention this in case you are interested).

> Im trying to understand hci packets now to see what those differences are,
> but in the mean time I thought Id post.
> 
> Thanks for the stack and for any thoughts.

If you don't mind sharing the code which calls
ble_gattc_disc_svc_by_uuid(), and also the callback, I would be happy to
take a look.

Thanks,
Chris

Re: newt build error

[Christopher Collins]

Hi Brian,

On Sat, Dec 24, 2016 at 02:44:48PM -0600, Brian Giori wrote:
> Hey all,
> I'm rebuilding bleprph to get started with mynewt again and ran into an
> issue with building the application and bootloader.
> 
> Things work smoothly up to "newt build blegpio_boot" when I get:
> Building target targets/blegpio_boot
> Error: Compiler package not specified by BSP
> 
> Here are my targets:
> targets/blegpio
> app=apps/bleprph
> bsp=@apache-mynewt-core/hw/bsp/nrf52dk
> build_profile=debug
> targets/blegpio_boot
> app=@apache-mynewt-core/apps/boot
> bsp=@apache-mynewt-core/hw/bsp/nrf52dk
> build_profile=optimized
> 
> Thoughts?

That looks like it might be a version incompatibility between newt and
apache-mynewt-core.  Specifically, that would happen if you have a
recent version of newt, but an older version and core.  Do you know
which versions you are using?

newt version
cat project.yml

You might just need to update your mynewt repos with "newt upgrade"

As an aside, it might be a good idea if newt printed out the repo state
(latest commit + dirty state) whenever the -ldebug flag is specified.  I
believe newt currently only logs this information after a successful
build.

Thanks,
Chris

Re: Fwd: Fwd: BLE quickly disconnects after connect on iPhone (LightBlue)

[Christopher Collins]

On Sat, Dec 24, 2016 at 05:37:56AM +0100, Kevin Townsend wrote:
> Hi Chris,
> > For option 2, you should modify your target's system configuration.  You
> > can increase the mbuf count to 16 as follows:
> >
> >  newt target syscfg set  syscfg=MSYS_1_BLOCK_COUNT=16
> >
> Was the 'target syscfg set' support recently added? I don't see this in 
> the help menu for newt based on the latest version in 'develop'?

No, it wasn't... I made a mistake :).  Sorry Cris, the correct command
is:

newt target set  syscfg=MSYS_1_BLOCK_COUNT=16

Chris

Re: Fwd: Fwd: BLE quickly disconnects after connect on iPhone (LightBlue)

[Christopher Collins]

Hi Chris,

On Fri, Dec 23, 2016 at 04:22:52PM -0500, Cris Frusina wrote:
> Hi All,
> 
> I'm just starting off with mynewt converting an old nrf51 project
> written with the Nordic SDK. I've compiled and uploaded the
> "bleprph_oic" example to my nrf52 custom board.
> 
> I see the device advertising with no problems, I can connect to the BLE
> device but after a few seconds it disconnects. Is there a setting I'm
> missing to keep the connection alive?

You shouldn't need to change any bluetooth settings. Part of the Mynewt
release process involves making sure LightBlue running on an iPhone
stays connected and can perform the standard GATT operations.

I generally use the bleprph and bletiny applications, so I'm not too
familiar with bleprph_oic.  In particular, I don't know if this
application reserves any mbufs up front.  One possibility is that the
bleprph_oic application is running out of mbufs while the iPhone is
performing service discovery.  I have noticed that the Apple BLE host
seems to be quite aggressive during service discovery in that it sends
lots of requests in parallel.  If the Mynewt device runs out of mbufs
during this process, it can't respond to a request, which causes the
iPhone to disconnect.  We saw this behavior when we first tested with an
iPhone, which led to increase in the default mbuf count.

Anyway, I would try one of these ideas:
1. Use the bleprph app, rather than bleprph_oic.
2. Increase the mbuf count to something greater than the default
   (12).

For option 2, you should modify your target's system configuration.  You
can increase the mbuf count to 16 as follows:

newt target syscfg set  syscfg=MSYS_1_BLOCK_COUNT=16

Thanks,
Chris

[jira] [Resolved] (MYNEWT-519) img_mgmt - Allow test+confirm as single action

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-519.

Resolution: Fixed

> img_mgmt - Allow test+confirm as single action
> --
>
> Key: MYNEWT-519
> URL: https://issues.apache.org/jira/browse/MYNEWT-519
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Image Mgmt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, to permanently run the alternate image, Mynewt requires the 
> following sequence:
> 1. image test 
> 2. reboot
> 3. image confirm
> The new feature is to remove the need for the third step.  The user should be 
> able to permanently switch images with this sequence:
> 1. image confirm 
> 2. reboot
> This ticket only represents the img_mgmt and newtmgr changes.  Required boot 
> loader changes are captured in a separate ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-520) boot - Allow incompatible image slots

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-520.

Resolution: Fixed

> boot - Allow incompatible image slots
> -
>
> Key: MYNEWT-520
> URL: https://issues.apache.org/jira/browse/MYNEWT-520
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Bootloader
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, the boot loader immediately fails if the two image slots have 
> different flash characteristics (different sector count or different sector 
> sizes).
> Instead, the boot loader should only require compatible slots if a swap is 
> required.  If the two image slots are not compatible, the boot loader should 
> just boot into slot 0 unconditionally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-518) boot - Allow test+confirm as single action

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-518.

Resolution: Fixed

> boot - Allow test+confirm as single action
> --
>
> Key: MYNEWT-518
> URL: https://issues.apache.org/jira/browse/MYNEWT-518
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Bootloader
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, to permanently run the alternate image, the boot loader requires 
> the following sequence:
> 1. image test 
> 2. reboot
> 3. image confirm
> The new feature is to remove the need for the third step.  The user should be 
> able to permanently switch images with this sequence:
> 1. image confirm 
> 2. reboot
> This ticket only represents the boot loader changes.  Required img_mgmt and 
> newtmgr changes are captured in a separate ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-520) boot - Allow incompatible image slots

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-520:
--

 Summary: boot - Allow incompatible image slots
 Key: MYNEWT-520
 URL: https://issues.apache.org/jira/browse/MYNEWT-520
 Project: Mynewt
  Issue Type: New Feature
  Components: Bootloader
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


Currently, the boot loader immediately fails if the two image slots have 
different flash characteristics (different sector count or different sector 
sizes).

Instead, the boot loader should only require compatible slots if a swap is 
required.  If the two image slots are not compatible, the boot loader should 
just boot into slot 0 unconditionally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MYNEWT-519) img_mgmt - Allow test+confirm as single action

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins updated MYNEWT-519:
---
Description: 
Currently, to permanently run the alternate image, Mynewt requires the 
following sequence:

1. image test 
2. reboot
3. image confirm

The new feature is to remove the need for the third step.  The user should be 
able to permanently switch images with this sequence:

1. image confirm 
2. reboot

This ticket only represents the img_mgmt and newtmgr changes.  Required boot 
loader changes are captured in a separate ticket.

  was:
Currently, to permanently run the alternate image, the boot loader requires the 
following sequence:

1. image test 
2. reboot
3. image confirm

The new feature is to remove the need for the third step.  The user should be 
able to permanently switch images with this sequence:

1. image confirm 
2. reboot

This ticket only represents the boot loader changes.  Required img_mgmt and 
newtmgr changes are captured in a separate ticket.

Component/s: (was: Bootloader)
 Image Mgmt
Summary: img_mgmt - Allow test+confirm as single action  (was: imgmgmt 
- Allow test+confirm as single action)

> img_mgmt - Allow test+confirm as single action
> --
>
> Key: MYNEWT-519
> URL: https://issues.apache.org/jira/browse/MYNEWT-519
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Image Mgmt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, to permanently run the alternate image, Mynewt requires the 
> following sequence:
> 1. image test 
> 2. reboot
> 3. image confirm
> The new feature is to remove the need for the third step.  The user should be 
> able to permanently switch images with this sequence:
> 1. image confirm 
> 2. reboot
> This ticket only represents the img_mgmt and newtmgr changes.  Required boot 
> loader changes are captured in a separate ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-513) sysinit - Remove panic function from syscfg.yml; configure at runtime.

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-513:
--

 Summary: sysinit - Remove panic function from syscfg.yml; 
configure at runtime.
 Key: MYNEWT-513
 URL: https://issues.apache.org/jira/browse/MYNEWT-513
 Project: Mynewt
  Issue Type: Improvement
  Components: Misc
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Replacing sysinit

[Christopher Collins]

Hello all,

Sysinit is the process Mynewt uses to initialize all the packages in a
project.  Typically, sysinit gets invoked at the very start of main().

There are a few aspects of sysinit that I really dislike, and I think we
can replace it with something better.  In this email, I will (1) summarize the
mechanism as it exists today, (2) describe the problems with the current
mechanism, and (3) propose a new solution.

(1) CURRENT SYSINIT PROCESS

* A package's pkg.yml file optionally specifies "init_function" and
  "init_stage"
* At build time, the newt tool generates sysinit C files which
  define a single function.  This function calls the packages' init
  functions in the correct order.
* Stage 0 runs first, stage 1 next, etc.
* Within a stage, initializations are ordered alphabetically by
  package name.
* For split images, two sysinit files get generated: one for the
  loader and one for the app.  The sysinit() macro (defined in
  sys/sysinit/include/sysinit.h) ensures the correct sysinit
  function gets called, depending on whether the loader or app is
  running.

The generated sysinit files are created with the following paths:
* bin/targets//generated/src/-sysinit-loader.c
* bin/targets//generated/src/-sysinit-app.c

For non-split-images, only the "app" file is created.

Here is an example of a generated sysinit function:

void
sysinit_app(void)
{
os_init();

/*** Stage 0 */
/* 0.0: kernel/os */
os_pkg_init();
/* 0.2: sys/flash_map */
flash_map_init();

/*** Stage 1 */
/* 1.0: net/nimble/transport/ram */
ble_hci_ram_pkg_init();
/* 1.1: sys/log */
log_init();

/* [...] */
}

(2) ISSUES WITH CURRENT IMPLEMENTATION

So that is how sysinit works today.  There are two aspects of the
current mechanism that I strongly dislike:
(1) Generated code.
(2) C identifiers specified in YAML files.

I think everyone hates generated code, so I probably don't have to say
much about (1).  As a general principle, I think it is good to minimize
the amount of magic newt performs during builds.

The reason I dislike (2) is that it creates an opportunity for really
confusing build errors.  Newt doesn't ensure each init_function setting
specifies a real function with the correct type.  If a setting is wrong,
the user gets a linker error which points at a generated sysinit file.

(3) PROPOSED SOLUTION

* In C code, a package optionally defines an "init entry" struct
  containing:
o Pointer to its init function.
o Stage number.
* Init entry structs are placed in a special section via
  __attribute__((section(...))).
* At startup, the sysinit() function walks the list of entries in
  the special section and calls their corresponding init functions
  in the correct order.

This solution addresses both the issues I mentioned above:
* No more init_function or init_stage settings in pkg.yml files.
* No generated sysinit code.  The sysinit() function is a generic
  function that exists in apache-mynewt-core.

I should mention that this solution introduces a new issue:
indeterminate initialization order within each stage.  In the current
mechanism, newt enforces a consistent ordering of packages with the same
stage (alphabetical).  In the proposed solution, packages within a stage
do not have a predictable ordering.  Presumably, the entries are
arranged in the order the linker encounters them.  I don't think this is
a huge problem, but it does mean package authors would need to be
careful when assigning stage numbers.

Here is the API that I came up with:

/* Package initialization function. */
typedef void sysinit_init_fn(struct sysinit_init_ctxt *ctxt);

struct sysinit_entry {
/* Initializes a package. */
sysinit_init_fn *init_fn;

/* Specifies when the init function gets called.  0=first, * 1=next, 
etc. */
uint8_t stage;
};

struct sysinit_init_ctxt {
/* Corresponds to the init function currently executing. */
const struct sysinit_entry *entry;

/* The stage that sysinit is currently processing. */
uint8_t cur_stage;
};

#define SYSINIT_REGISTER_INIT(init_cb, init_stage) /* ... */

Here is a usage example.  This is the existing mgmt/newtmgr init
function modified to use the proposed mechanism.

void
nmgr_pkg_init(struct sysinit_init_ctxt *ctxt)
{
int rc;

/* Ensure this function only gets called by sysinit. */
SYSINIT_ASSERT_ACTIVE();

rc = nmgr_task_init();
SYSINIT_PANIC_ASSERT(rc == 0);
}

SYSINIT_REGISTER_INIT(nmgr_pkg_init, 5);

I anticipate most initialization functions would not need the ctxt
parameter.  Some functions may find it useful, though.  For example, a
package might register a function several times, specifying a 

[jira] [Created] (MYNEWT-512) Replace generated sysinit code with linker section

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-512:
--

 Summary: Replace generated sysinit code with linker section
 Key: MYNEWT-512
 URL: https://issues.apache.org/jira/browse/MYNEWT-512
 Project: Mynewt
  Issue Type: Improvement
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


*Old mechanism:*
* pkg.yml file specifies "init_function" and "init_stage"
* newt generates sysinit C files which call the packages' init functions in the 
specified order.

*New mechanism:*
* Each package defines an "init entry" struct containing:
** pointer to init function
** uint8_t stage
* Init entry structs are placed in a special section.
* At startup, the sysinit() function walks the list of entries in the special 
section and calls their corresponding init functions in the correct order.

*Interface*
{code}
typedef void sysinit_init_fn(struct sysinit_init_ctxt *ctxt);
struct sysinit_entry {
sysinit_init_fn *init_fn;
uint8_t stage;
};

struct sysinit_init_ctxt {
const struct sysinit_entry *entry;
uint8_t cur_stage;
};

#define SYSINIT_REGISTER_INIT(init_cb, init_stage) // ...
{code}

User defines a package initialization function and registers it.  Registration 
creates a {{struct sysinit_entry}} instance in the special "sysinit" section.

*Example:*
{code}
static void
my_pkg_init(void)
{
/* ... */
}

SYSINIT_REGISTER(my_pkg_init, 2);
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: MYNEWT_VAL_STATS_CLI value cannot be change

[Christopher Collins]

I should also send a belated heads up to the group.  The "target config"
command has changed slightly in the latest newt in the develop branch.
Now, the command is:

target config show 

(i.e., "config" became "config show").

The reason for this change is to allow support for additional
configuration-related commands.  Unfortunately, I didn't have enough
foresight to make config a command group from the start.

Sorry for the churn, all.

Chris

On Thu, Dec 15, 2016 at 10:13:44AM -0800, Christopher Collins wrote:
> Hi Then,
> 
> I didn't see the png files; did you forgot to attach them?  Also, the
> output.txt file appears to be incomplete.  One easy way to capture newt
> output is to use the "-o " option.  For example:
> 
> newt -o output.txt target config myble
> 
> All output gets printed to the terminal, but it also gets written to the
> specified file.  Then you can just attach output.txt to your next email.
> 
> Also, just to be clear, it is recommended that you only change your
> *target*'s syscfg.  If you change other packages, like apps/bletiny, it
> might result in conflicts the next time you update your
> apache-mynewt-core repository.
> 
> So, I would expect you to have the following file:
> 
> targets/myble/syscfg.yml
> 
> This file would look something like this:
> 
> syscfg.vals:
> STATS_CLI: 1
> 
> Finally, you should not need to add the "@apache-mynewt-core/sys/stats"
> dependency to bletiny.  The net/nimble/host package depends on
> the stats package, so it should already get included in your project.
> 
> Thanks,
> Chris
> 
> On Thu, Dec 15, 2016 at 02:13:55PM +0800, then yon wrote:
> > Dear dg,
> > 
> > Thanks for your reply.
> > 
> > i had followed your step to add syscfg.yml under targets but it doesn't 
> > help.
> > 
> > Please find the attachment for:
> > 
> > 1. output.txt: output of the target config command.
> > 
> > 2. syscfg_yml_app.png: print screen of the code and directory list for the 
> > syscfg.yml under app.
> > 
> > 3. syscfg_yml_target.png: print screen of the code and directory list for 
> > the syscfg.yml under targets.
> > 
> > Please let me know if you need more information.
> > 
> > Thank you.
> > 
> > 
> > Regards,
> > Then Yoong Ze

[jira] [Created] (MYNEWT-510) sysinit - Remove from pkg.yml; put init functions in a linker section

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-510:
--

 Summary: sysinit - Remove from pkg.yml; put init functions in a 
linker section
 Key: MYNEWT-510
 URL: https://issues.apache.org/jira/browse/MYNEWT-510
 Project: Mynewt
  Issue Type: Bug
  Components: Misc
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-351) BLE Host - Delete completed GATT procedure before calling callback

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-351.

Resolution: Won't Fix

I don't think this is worth doing for the following reasons:

* Some GATT failures are only detected when a callback returns error.  In these 
cases, we can't free the procedure before the callback, because we don't know 
if we are done with it yet.  Getting this right will introduce code complexity.

* Most callbacks need to receive the procedure data.  Making a temporary copy 
of the various procedure structs will add code size and complexity.

> BLE Host - Delete completed GATT procedure before calling callback
> --
>
> Key: MYNEWT-351
> URL: https://issues.apache.org/jira/browse/MYNEWT-351
> Project: Mynewt
>  Issue Type: Bug
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> When a GATT procedure completes, its callback gets executed.  The procedure 
> doesn't get deleted until after the callback. This is problematic if the 
> application wants to initiate another procedure from within the callback and 
> the GATT proc pool is empty.  We should copy the data necessary for the 
> callback and delete the procedure, then call the callback.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Run "newt clean all" after upgrading newt

[Christopher Collins]

Hello all,

The short version is: run "newt clean all" after pulling newt from
develop.

Recently, newt changed the way it names .a files.  Originally, each .a
file was named after the "leaf" directory of its package.  Now, the full
path is included in the name.  For example, for the sys/console/full
package:

old: full.a
new: sys_console_full.a

The new naming scheme is more descriptive, and it improves the output of
the "newt size" command.

(commit=e7f31c7e0a9bff4f1e88a6df7f8ca3edd3e81b90)

The problem is that, during linking, newt indiscriminately collects all
.a files in the target's binary directory and links them together.  If
you upgrade newt and build a project, the old .a files won't get
overwritten by the new ones.  I don't know how gcc determines which .a
file to pull a symbol from when there is a conflict, and frankly, I
don't want to know :).  Anyway, definitely be sure to clean your project
after upgrading, or you risk losing your mind trying to debug your code.

I am not sure if there is a reason why newt links all the .a files it
finds.  If there isn't, then we should probably change newt so that it
only pulls in the .a files that it is supposed to.

Thanks,
Chris

[ANNOUNCE] Apache Mynewt 1.0.0-b1-incubating released

[Christopher Collins]

Hello all,

The Apache Mynewt team is pleased to announce the first release of
Apache Mynewt: 1.0.0-b1-incubating.

Apache Mynewt is a community-driven module OS for constrained, embedded
applications.  Mynewt provides a real-time operating system, flash file
system, network stacks, and support utilities for real-world embedded
systems.

The release is available here:
http://www.apache.org/dyn/closer.lua/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating

We welcome your help and feedback. For more information on the project
and how to get involved, visit the project website at
http://mynewt.incubator.apache.org/

Regards,
The Apache Mynewt team

Disclaimer:
Apache Mynewt is an effort undergoing incubation at The Apache Software
Foundation (ASF) sponsored by the Apache Incubator PMC. Incubation is
required of all newly accepted projects until a further review
indicates that the infrastructure, communications, and decision making
process have stabilized in a manner consistent with other successful
ASF projects. While incubation status is not necessarily a reflection
of the completeness or stability of the code, it does indicate that
the project has yet to be fully endorsed by the ASF.

[ANNOUNCE] Apache Mynewt 1.0.0-b1-incubating released

[Christopher Collins]

Hello all,

The Apache Mynewt team is pleased to announce the first release of
Apache Mynewt: 1.0.0-b1-incubating.

Apache Mynewt is a community-driven module OS for constrained, embedded
applications.  Mynewt provides a real-time operating system, flash file
system, network stacks, and support utilities for real-world embedded
systems.

The release is available here:
http://www.apache.org/dyn/closer.lua/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating

We welcome your help and feedback. For more information on the project
and how to get involved, visit the project website at
http://mynewt.incubator.apache.org/

Regards,
The Apache Mynewt team

Disclaimer:
Apache Mynewt is an effort undergoing incubation at The Apache Software
Foundation (ASF) sponsored by the Apache Incubator PMC. Incubation is
required of all newly accepted projects until a further review
indicates that the infrastructure, communications, and decision making
process have stabilized in a manner consistent with other successful
ASF projects. While incubation status is not necessarily a reflection
of the completeness or stability of the code, it does indicate that
the project has yet to be fully endorsed by the ASF.

Re: final release step (was: [RESULT][VOTE] Release Apache Mynewt ...)

[Christopher Collins]

Hi Greg,

On Sun, Dec 11, 2016 at 08:45:17PM -0600, Greg Stein wrote:
> Hi all,
> 
> I was looking at prior releases, and noticed that the final step to publish
> the release appears to have been done wrong. No harm done, but suboptimal.
> 
> It appears the release artifacts were *added* into
> release/incubator/mynewt/ ... but what *should* be done is to move the
> artifacts that were voted upon into the release area.
> 
> For example:
> 
> $ svn mv -m "Publish mynewt-0.8.0-b1-incubating"
> https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b1-incubating/rc2/
> https://dist.apache.org/repos/dist/release/incubator/mynewt/apache-mynewt-0.8.0-b1-incubating/
> 
> That way, you won't duplicate the content within svn, and you are
> publishing *exactly* what was voted. (sure, previous stuff is
> byte-comparable, but the svn log is also a Good Thing)

Thanks, that makes sense.  I'll update our release process document with
this information.  Now for this release, let's see if my svn skills are
any better than my git skills :).

> Also please "svn rm" all the old stuff from dev/incubator/mynewt/ ... no
> reason for that to sit around. (clearly, it will remain in version control,
> but just not in your face)

Sounds good.

Thanks,
Chris

Re: yield() and os_eventq_run()

[Christopher Collins]

On Sun, Dec 11, 2016 at 11:13:47AM -0800, Sterling Hughes wrote:
> Hi,
> 
> As a part of developing the sensor interface, I want to register a 
> “listener” on the event queue:
> 
>  listener.sl_sensor_type = type;
>  listener.sl_func = sensor_shell_read_listener;
>  listener.sl_arg = 
> 
>  rc = sensor_register_listener(sensor, );
>  if (rc != 0) {
>  goto err;
>  }
> 
> And then I want to essentially “wait” until I’ve read ’n’ 
> entries, and block in my task:
> 
> 
>  /* Block until we've read 'n' samples. */
>  while (ctx.num_entries < nsamples) {
>  os_eventq_run(sensor_mgr_evq_get());
>  os_time_delay(1);
>  }
> 
> The sensor_shell_read_listener() function updates “num_entries”, and 
> when I’ve read the desired number of samples, the loop finishes.
> 
> Just calling os_time_delay() or os_eventq_run() in and of themselves is 
> not sufficient:
> 
> - os_eventq_run() will let the current task’s events (default event 
> queue in this case) run, but won’t process other tasks events (e.g. 
> shell is run out of a separate task in this case.)

Sorry if I'm being dense here, but there is something I am missing.  Why
won't other tasks run while you block on the sensor event queue?  This
should only be an issue if the event queue is never empty, right?  Are
you expecting this eventq to be receiving events faster than you can
deqeueue them or something?

> - os_time_delay() lets the other tasks run.
> 
> q: Should we add a os_eventq_yield(evq, time) function that does both of 
> these tasks?  I think it might be clearer that this is the intention in 
> that case.

Re: System init and OS eventq ensure

[Christopher Collins]

On Sun, Dec 11, 2016 at 10:11:44AM -0800, will sanfilippo wrote:
> Personally, I keep wanting to try and have the OS start up right away.

I wonder if this could solve the problem that Sterling raised (no
default event queue during sysinit).  The control flow in main() might
look like this:

1. Start OS
2. Create and designate default event queue.
3. sysinit()

I think it would be nice if we could avoid adding another initialization
stage.

> There are definitely “issues” with this:
> a) We do not want to waste idle task stack.
> b) When tasks are started they would start running right away. This
> might cause issues where a task does something to a piece of memory
> that another task initializes, but since that other task has not
> initialized it yet…
> 
> b) can be avoided by locking the scheduler until initializations are finished.
> 
> a) is problematic :-) I think someone brought this up before, but I
> wonder if it is worth the effort to do something “a bit crazy” like
> the following: the idle task uses “the heap” during intialization.
> Once initializations are over (or at some point that we determine),
> the idle task stack is made smaller and the “top” of the heap is set
> to the end of the idle task stack. For example, idle task stack is at
> 0x20008000 and is of size 1K bytes; the bottom of the heap is at
> 0x20007000; the top of the heap is at 0x20007C00 (in my nomenclature,
> heap allocations start from the bottom). At some point, the top of the
> heap is moved to 0x20007F80.
> 
> Yeah, maybe a bit crazy… :-)

I don't think that's too crazy.  It would be great if we could just
malloc() a temporary stack, and then free it when initialization
completes.  I guess the worry is that this will cause heap
fragmentation?

Chris

Re: System init and OS eventq ensure

[Christopher Collins]

Darn, you're right. I'm writing these emails from my phone, and I didn't
look at the code closely enough.  For other packages, the start event
only gets executed the first time the event queue gets used (as you
said).  I guess it has worked out in practice because the application
uses the package shortly after the OS starts.

That's not so great.  Second stage initialization sounds good to me.
Alternatively, the system could keep track of packages that need an
event queue, and enqueue their start event when a default event queue is
set.  Earlier, we discussed using linker sections to accomplish this
without requiring any RAM. I looked into this, but concluded it wasn't
possible without modifying the linker scripts.

Chris

On Sat, Dec 10, 2016 at 12:30:39PM -0800, Sterling Hughes wrote:
> How do you assign an event queue if you are relying on the default event 
> queue being there?  
> 
> Can you point me to an example of where this is done? 
> 
> Sterling 
> 
> > On Dec 10, 2016, at 12:08 PM, Christopher Collins <ccoll...@apache.org> 
> > wrote:
> > 
> > The way other packages handle this is they enqueue the startup event
> > when their event queue is assigned.  This happens automatically when you
> > call os_eventq_designate(); the last parameter is the event to enqueue
> > immediately.
> > 
> > Chris
> > 
> >> On Sat, Dec 10, 2016 at 11:30:27AM -0800, Sterling Hughes wrote:
> >> Hi,
> >> 
> >> I’m looking at using the default eventq (or allowing for it), in a 
> >> library I’m working on.  In order to do that, I have a function:
> >> 
> >> static struct os_eventq *
> >> sensor_mgr_evq_get(void)
> >> {
> >> os_eventq_ensure(_mgr.mgr_eventq, NULL);
> >> 
> >> return (sensor_mgr.mgr_eventq);
> >> }
> >> 
> >> And this function gets called within my package’s sysinit, as I want 
> >> to schedule a callout to run this function immediately on bootup:
> >> 
> >> /**
> >>  * Initialize sensor polling callout and set it to fire on boot.
> >>  */
> >> os_callout_init(_mgr.mgr_wakeup_callout, 
> >> sensor_mgr_evq_get(),
> >> sensor_mgr_wakeup_event, NULL);
> >> os_callout_reset(_mgr.mgr_wakeup_callout, 0);
> >> 
> >> The problem is that the default event queue is not setup until after 
> >> sysinit executes, as task setup is later on.
> >> 
> >> What is the right way to do this?  For now, I can move the 
> >> initialization from sysinit and to the main() function at task level, 
> >> however, I don’t think this is how we want to manage initialization 
> >> over time.  We probably need some way for a package to have a system 
> >> initialization stage that runs after the OS has started, and the default 
> >> event queue has been set.
> >> 
> >> Cheers,
> >> 
> >> Sterling

Re: System init and OS eventq ensure

[Christopher Collins]

The way other packages handle this is they enqueue the startup event
when their event queue is assigned.  This happens automatically when you
call os_eventq_designate(); the last parameter is the event to enqueue
immediately.

Chris

On Sat, Dec 10, 2016 at 11:30:27AM -0800, Sterling Hughes wrote:
> Hi,
> 
> I’m looking at using the default eventq (or allowing for it), in a 
> library I’m working on.  In order to do that, I have a function:
> 
> static struct os_eventq *
> sensor_mgr_evq_get(void)
> {
>  os_eventq_ensure(_mgr.mgr_eventq, NULL);
> 
>  return (sensor_mgr.mgr_eventq);
> }
> 
> And this function gets called within my package’s sysinit, as I want 
> to schedule a callout to run this function immediately on bootup:
> 
>  /**
>   * Initialize sensor polling callout and set it to fire on boot.
>   */
>  os_callout_init(_mgr.mgr_wakeup_callout, 
> sensor_mgr_evq_get(),
>  sensor_mgr_wakeup_event, NULL);
>  os_callout_reset(_mgr.mgr_wakeup_callout, 0);
> 
> The problem is that the default event queue is not setup until after 
> sysinit executes, as task setup is later on.
> 
> What is the right way to do this?  For now, I can move the 
> initialization from sysinit and to the main() function at task level, 
> however, I don’t think this is how we want to manage initialization 
> over time.  We probably need some way for a package to have a system 
> initialization stage that runs after the OS has started, and the default 
> event queue has been set.
> 
> Cheers,
> 
> Sterling

[jira] [Resolved] (MYNEWT-506) newt - Compile files from base dir

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-506.

Resolution: Fixed

> newt - Compile files from base dir
> --
>
> Key: MYNEWT-506
> URL: https://issues.apache.org/jira/browse/MYNEWT-506
> Project: Mynewt
>  Issue Type: Improvement
>  Components: Newt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Currently, newt descends into a source file's directory before compiling it.  
> As a consequence, embedded paths consist only of filenames; directory 
> information is lost.
> Instead, newt should change to the project's base dir, and build everything 
> relative to there.
> Possible issue: increase in image size due to increase in length of __FILE__ 
> macros (e.g., asserts).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-507) newt - Include full path in .a filenames

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-507:
--

 Summary: newt - Include full path in .a filenames
 Key: MYNEWT-507
 URL: https://issues.apache.org/jira/browse/MYNEWT-507
 Project: Mynewt
  Issue Type: Improvement
  Components: Newt
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


Newt names .a files with the name of the package's leaf directory.  In other 
words, the net/nimble/host package is transformed into a file called host.a.  
This stripping of path information makes it difficult to match .a files with 
their source packages.

Solution: Include full path in .a filenames.  E.g., net_nimble_host.a.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-506) newt - Compile files from base dir

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-506:
--

 Summary: newt - Compile files from base dir
 Key: MYNEWT-506
 URL: https://issues.apache.org/jira/browse/MYNEWT-506
 Project: Mynewt
  Issue Type: Improvement
  Components: Newt
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


Currently, newt descends into a source file's directory before compiling it.  
As a consequence, embedded paths consist only of filenames; directory 
information is lost.

Instead, newt should change to the project's base dir, and build everything 
relative to there.

Possible issue: increase in image size due to increase in length of __FILE__ 
macros (e.g., asserts).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-349) BLE Host - API for gatt-service-changed

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-349.

Resolution: Fixed

> BLE Host - API for gatt-service-changed
> ---
>
> Key: MYNEWT-349
> URL: https://issues.apache.org/jira/browse/MYNEWT-349
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-346) BLE Host - Allow secure connections without requiring legacy pairing

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-346.

Resolution: Fixed

> BLE Host - Allow secure connections without requiring legacy pairing
> 
>
> Key: MYNEWT-346
> URL: https://issues.apache.org/jira/browse/MYNEWT-346
> Project: Mynewt
>  Issue Type: Improvement
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> There are two options in nimble_opt.h:
> * NIMBLE_OPT_SM
> * NIMBLE_OPT_SM_SC
> The first (sm) enables the security manager and legacy pairing.  The second 
> (sm_sc) enables secure connections.  Secure connections requires the security 
> manager, so when NIMBLE_OPT_SM_SC is enabled, NIMBLE_OPT_SM gets enabled 
> automatically by nimble_opt_auto.h.
> The user should be able to enable SC without also enabling legacy pairing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

NimBLE setting name change (BLE_SM --> BLE_SM_LEGACY)

[Christopher Collins]

Hello all,

I just wanted to give a quick heads up.  On develop, I renamed the
legacy pairing syscfg setting:

From: BLE_SM
To:   BLE_SM_LEGACY

The default configuration is still the same (legacy enabled, SC
disabled), so this change should only affect you if you disable legacy
pairing in your build.

The reason for the change is simply that BLE_SM was a bad name for this
setting.  It was ambiguous and inconsistent with its sibling, BLE_SM_SC.

Thanks,
Chris

[jira] [Updated] (MYNEWT-229) Newt - Commands for dependency graph and reverse-dependency graph

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins updated MYNEWT-229:
---
Summary: Newt - Commands for dependency graph and reverse-dependency graph  
(was: need a way for newt to show all dependencies for a project)

> Newt - Commands for dependency graph and reverse-dependency graph
> -
>
> Key: MYNEWT-229
> URL: https://issues.apache.org/jira/browse/MYNEWT-229
> Project: Mynewt
>  Issue Type: Improvement
>Reporter: Paul Dietrich
>    Assignee: Christopher Collins
> Fix For: v1_1_beta1
>
>
> and where they come from 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-383) BLE Host - Allow programmatic unbonding

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-383.

Resolution: Fixed

> BLE Host - Allow programmatic unbonding
> ---
>
> Key: MYNEWT-383
> URL: https://issues.apache.org/jira/browse/MYNEWT-383
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-245) newt target test should allow debug of a package test library

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-245.

Resolution: Fixed

> newt target test should allow debug of a package test library
> -
>
> Key: MYNEWT-245
> URL: https://issues.apache.org/jira/browse/MYNEWT-245
> Project: Mynewt
>  Issue Type: Improvement
>  Components: Newt
>Reporter: Paul Dietrich
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> when I hit an assert in my test, I wanted to debug to see why it was 
> asserting.  It turns out that there was no way to do that in newt.  
> It would be nice to have this capability.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-498) Newt - depgraph and syscfg calculate twice per build

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-498.

Resolution: Fixed

> Newt - depgraph and syscfg calculate twice per build
> 
>
> Key: MYNEWT-498
> URL: https://issues.apache.org/jira/browse/MYNEWT-498
> Project: Mynewt
>  Issue Type: Bug
>    Reporter: Christopher Collins
>    Assignee: Christopher Collins
>
> Building with -ldebug shows that dependencies, syscfg, and APIs get resolved 
> twice.  In the code, this happens in the following places:
> * {{resolve/resolve.go (ResolveCfg())}}
> * {{builder/build.go (PrepBuild())}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-494) Shell should assert() when a duplicate name/entry is registered.

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-494.

Resolution: Fixed

> Shell should assert() when a duplicate name/entry is registered.
> 
>
> Key: MYNEWT-494
> URL: https://issues.apache.org/jira/browse/MYNEWT-494
> Project: Mynewt
>  Issue Type: Improvement
>Affects Versions: v1_0_0_beta1
>Reporter: Sterling Hughes
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> Right now it just inserts it into the list, which can create a circular list 
> search, when, for example calling shell help.  It also just calls the first 
> handler it finds when you issue the command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-504) newt - Verify repo isn't too old for newt version

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-504.

Resolution: Fixed

> newt - Verify repo isn't too old for newt version
> -
>
> Key: MYNEWT-504
> URL: https://issues.apache.org/jira/browse/MYNEWT-504
> Project: Mynewt
>  Issue Type: Bug
>    Reporter: Christopher Collins
>    Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> We should warn the user if their version of newt is "too new" for a repo in 
> their project.
> Unfortunately, warning about the reverse (repo too new for newt) is more 
> difficult, and is not covered by this ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-503) newt - No status messages or logging during project init

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-503.

Resolution: Fixed

> newt - No status messages or logging during project init
> 
>
> Key: MYNEWT-503
> URL: https://issues.apache.org/jira/browse/MYNEWT-503
> Project: Mynewt
>  Issue Type: Bug
>  Components: Newt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> No log messages or status messages are displayed while the project is being 
> initialized.  This issue is related to tab completion: output is 
> intentionally suppressed during initialization so that the "complete" command 
> generates clean output.
> The fix is to only generate valid tab completion values when it has been 
> determined that the "complete" command is being used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-503) newt - No status messages or logging during project init

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-503:
--

 Summary: newt - No status messages or logging during project init
 Key: MYNEWT-503
 URL: https://issues.apache.org/jira/browse/MYNEWT-503
 Project: Mynewt
  Issue Type: Bug
  Components: Newt
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


No log messages or status messages are displayed while the project is being 
initialized.  This issue is related to tab completion: output is intentionally 
suppressed during initialization so that the "complete" command generates clean 
output.

The fix is to only generate valid tab completion values when it has been 
determined that the "complete" command is being used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] Release Apache Mynewt 1.0.0-b1-incubating-rc2

[Christopher Collins]

(The email I'm responding is from general@. I'm replying in dev@, as all
my comments concern Mynewt specifically.  For those who don't follow
general@, in the original email, Justin voted +1 on 1.0.0-b1-rc2, but he
raised a few issues that I believe warrant a closer look.)

Hi Justin,

Thanks for taking a look.  I want to address the three issues you
raised.  I reordered them slightly, as issues 1 and 3 are closely
related.

On Mon, Dec 05, 2016 at 11:58:58AM +1100, Justin Mclean wrote:
[...]
> A couple of minor things re compilation which other people may run into:
> 1. It would be good to have better compile instructions especially for the 
> core package. 
> 3. A newt build all in core does nothing, but a newt test all goes off and 
> compiles and runs the tests. Is that expected? How do you just build core?

Yes - I believe this has been a point of confusion in some earlier
releases as well.  I think Mynewt might be atypical this regard, and
we should address this in the core README file.  Specifically, core is
not something that can be built directly; it is a collection of
libraries and sample applications that a user's project depends on.

The "newt build all" command doesn't do anything in a bare
apache-mynewt-core repo because there are no targets to build.  A target
is the top-level Mynewt entity that actually gets built; it ties a BSP
and an application together.  To get started, a user should also acquire
the blinky repository by using the "newt new" command.  Blinky includes
a sample target for running the blinky application in a simulated
environment.

For the next release, I think we should clarify this in the core README
file.  We should make it very clear to the user what they have
downloaded, and indicate how to proceed.

> 2. It seems that blinky expects gcc-5 in /usr/local/bin/ and I only had gcc-6 
> installed and an older version of gcc in /usr/bin/gcc

Hmm, this is indeed a problem.  On OS X, /usr/bin/gcc is not actually
gcc; it's clang.  I don't recall the details, but I believe there was a
time when clang was incompatible with Mynewt.  "gcc-5" is the name of
the gcc 5.x binary that home brew installs, so that is what newt looks
for.

I just tried running "newt test all" using clang, and it mostly works.
There are a few warnings that get reported, but they should be easy to
fix.  My thought it is that we should fix the warnings and change the
compiler definition such that the "gcc" binary (i.e., clang) is used in
OS X.  It's unfortunate that this ambiguity exists, but I think it's
preferable to just use clang than to impose these gcc version headaches
on the user.

Thanks,
Chris

Re: MBEDTLS Error after 'newt install -v'

[Christopher Collins]

Hi Kevin,

This issue tends to happen if you are using a version of newt that is
incompatible with your core repo.  You mentioned you are using the
master branch of core; is it possible that you are using develop of the
newt repo?

Specifically, I think this issue is caused by this commit from last
week (newt repo): 0241d2f38e8236f03d6f20d5c7198d4a0a4cb169

I mentioned in an email last week that newt and core would need to be
upgraded together after this change.  Emails like that aren't
particularly useful, because they are only interesting to people who are
already experiencing issues!

One feature that newt needs is the ability to require minimum versions
of the core repo.  Then, compatibility-breaking changes would still be a
nuissance, but at least it would be obvious what the problem is.

Chris

On Mon, Dec 05, 2016 at 06:11:24PM +0100, Kevin Townsend wrote:
> I was updating some local dev systems, and with a clean install and then 
> running 'newt install -v' we now get the following error when we try to 
> build the bootloader:
> 
> /LOCAL_FOLDER/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10:
>  
> error: #include expects "FILENAME" or 
>   #include MBEDTLS_USER_CONFIG_FILE
>^
> 
> This is new, but unexpected since nothing has changed in the last week 
> or so in the master branch of the repo, and the last change to 'boot' 
> was 26 days ago so I'm not missing something in our local slightly 
> modified bootloader code.
> 
> Has something recently changed with the mbedtls library that I'm not 
> seeing quickly looking at the git history in master?
> 
> Just commenting the offending line out causes problems elsewhere so 
> there seems to be a missing config file:
> 
> https://github.com/apache/incubator-mynewt-core/blob/master/crypto/mbedtls/include/mbedtls/config.h#L2522
> 
> #if defined(YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE)
> #include YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE
> #elif defined(MBEDTLS_USER_CONFIG_FILE)
> //#include MBEDTLS_USER_CONFIG_FILE
> #endif
> 
> That just kicks the problems further down the field:
> 
> ctr_drbg.c:425:5: error: implicit declaration of function 'fopen'
> [-Werror=implicit-function-declaration]
>   if( ( f = fopen( path, "wb" ) ) == NULL )
>   ^
> ctr_drbg.c:425:13: error: assignment makes pointer from integer
> without a cast [-Werror]
>   if( ( f = fopen( path, "wb" ) ) == NULL )
>   ^
> ctr_drbg.c:440:5: error: implicit declaration of function 'fclose'
> [-Werror=implicit-function-declaration]
>   fclose( f );
>   ^
> ctr_drbg.c: In function 'mbedtls_ctr_drbg_update_seed_file':
> ctr_drbg.c:450:13: error: assignment makes pointer from integer
> without a cast [-Werror]
>   if( ( f = fopen( path, "rb" ) ) == NULL )
>   ^
> ctr_drbg.c:453:5: error: implicit declaration of function 'fseek'
> [-Werror=implicit-function-declaration]
>   fseek( f, 0, SEEK_END );
>   ^
> ctr_drbg.c:453:18: error: 'SEEK_END' undeclared (first use in this
> function)
>   fseek( f, 0, SEEK_END );
>^
> ctr_drbg.c:453:18: note: each undeclared identifier is reported only
> once for each function it appears in
> ctr_drbg.c:454:5: error: implicit declaration of function 'ftell'
> [-Werror=implicit-function-declaration]
>   n = (size_t) ftell( f );
>   ^
> ctr_drbg.c:455:18: error: 'SEEK_SET' undeclared (first use in this
> function)
>   fseek( f, 0, SEEK_SET );
>^
> cc1: all warnings being treated as errors
> 
> Any idea what might be causing the issue here? I'm looking at the 
> mbedtls source and don't see any recent pushes there, etc.
> 

Re: Newbie & running into an error during setup

[Christopher Collins]

Hi Mohammed,

My apologies - I intended to respond to you a few days ago.
Unfortunately, we weren't able to reproduce this issue, so I would like
to learn a bit more about your setup.

I'm afraid I don't have the version numbers available at the moment, but
the setup we used is:
* Ubuntu VM running in VirtualBox
* Host OS = OS X

Could you please provide the following information:
* OS X version
* VM software (e.g., VirtualBox, VMWare, etc.)
* Were you running as root?
* A bit more context in the console log (include the "newt new"
  command).

Thanks,
Chris

On Sun, Dec 04, 2016 at 04:22:37PM -0500, Mohammad Afaneh wrote:
> Anyone out there who can help with this? I know it's probably some stupid 
> little mistake, but so far I can't figure it out.
> 
> - Mohammad
> 
> > On Dec 2, 2016, at 8:56 AM, Mohammad Afaneh  wrote:
> > 
> > Hi everyone,
> > 
> > Newbie here, and looking forward to learning more about Mynewt!
> > 
> > I'm going through installing and setting up Mynewt on Ubuntu 16.04 (inside 
> > a VM on a macbook) using the Docker method. The issue I'm running into is:
> > 
> > newt install -v
> > apache-mynewt-core
> > Downloading repository description for apache-mynewt-core... success!
> > Downloading repository incubator-mynewt-core (branch: master; commit: 
> > mynewt_1_0_0_b1_tag) at https://github.com/apache/incubator-mynewt-core.git
> > Cloning into '/tmp/newt-repo557799235'...
> > warning: unable to access '/root/.config/git/attributes': Permission denied
> > apache-mynewt-core successfully installed version 0.9.9-none
> > 
> > After this is done, I look at the folders in my project folder and most of 
> > them are missing (only folders I have are: apps, repos and targets).
> > 
> > I made sure to add myself to the docker group, but I'm guessing this 
> > unrelated(?) I went through the steps again just to make sure I didn't miss 
> > anything, but with no luck.
> > 
> > Any help is appreciated. Thanks!
> > 
> > - Mohammad

[VOTE] Release Apache Mynewt 1.0.0-b1-incubating-rc2

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of Apache
Mynewt 1.0.0, beta 1 (rc2).  This is the second release candidate for
Mynewt 1.0.0-b1.  The vote for the first release candidate was cancelled
due to licensing issues with third-party bundled software.  Details
concerning what has changed since the previous release candidate can be
found at the end of this email.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications. Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes

[VOTE] thread:
https://lists.apache.org/thread.html/bb3e334e9b3b6b95247e46cca8b95e853f26cc9bb65a289f576dcef1@%3Cdev.mynewt.apache.org%3E

[RESULT][VOTE] thread:
https://lists.apache.org/thread.html/5a6aeb397cf5e0277aca91a34db221921d576725dcfa183abfa23da0@%3Cdev.mynewt.apache.org%3E

[DISCUSS] thread:
https://lists.apache.org/thread.html/e9f2bc01ac4257282c32116d91dc1d4064d428a99b1cc181c8a55980@%3Cdev.mynewt.apache.org%3E

This release candidate was tested as follows:
1. Manual execution of the Mynewt test plan:
   
https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Test+Plan
   The test results can be found at:
   https://cwiki.apache.org/confluence/display/MYNEWT/1.0.0-b1+Test+Results

2. The full unit test suite for this release was executed via "newt
   test all" with no failures.  This testing was performed on the
   following platforms:
 * OS X 10.10.5
 * Linux 4.4.6 (Gentoo)

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc2/

The commits under consideration are as follows:

blinky:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky
commit: b6918edc174b764f1e0b4ef4f4652aca8fedf8dd

core:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core
commit: c6a8f88f6b8dc7c8eee6dc75a34c9b6ea295b05d

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt
commit: 59ec3af372952382390cfd988e503d8cf3d5b34b

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc2/apache-mynewt-newt-bin-linux-1.0.0-b1-incubating.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc2/apache-mynewt-newt-bin-osx-1.0.0-b1-incubating.tgz


The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

As indicated, here is a list of issues with the previous release
candidate (rc1) that have been resolved:
* Pointers for several third-party libraries added to the LICENSE
  files.
* Some acknowledgements added to core's NOTICE file, as required by
  the licensing terms of some bundled third-party software.
* Removal of third-party files with licenses that are incompatible
  with an Apache project.  Specifically:
* net/ip/lwip_base/include/netif/ppp/pppoe.h
* net/ip/lwip_base/include/netif/ppp/vj.h
* net/ip/lwip_base/src/netif/ppp/pppoe.c
* net/ip/lwip_base/src/netif/ppp/vj.c
  These files specified a 4-clause BSD license (or equivalent).

Thanks,
Chris

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

[jira] [Resolved] (MYNEWT-487) BLE Host - Execute GATT procedure callback on timeout

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-487.

Resolution: Fixed

> BLE Host - Execute GATT procedure callback on timeout
> -
>
> Key: MYNEWT-487
> URL: https://issues.apache.org/jira/browse/MYNEWT-487
> Project: Mynewt
>  Issue Type: Bug
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> When a GATT procedure times out, the callback does not get executed.  The 
> rationale was probably that the app doesn't need to know about that since it 
> has gotten notified about the terminated connection instead.  However, for 
> logging and reporting reasons, every GATT procedure should be paired with a 
> callback invocation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-128) BLE Host - Update connection parameters automatically

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-128.

Resolution: Won't Fix

> BLE Host - Update connection parameters automatically
> -
>
> Key: MYNEWT-128
> URL: https://issues.apache.org/jira/browse/MYNEWT-128
> Project: Mynewt
>  Issue Type: New Feature
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MYNEWT-324) BLE Host - Ensure there is always sufficient resources to send an ATT "insufficient-resources" error response.

[Christopher Collins (JIRA)]

[ 
https://issues.apache.org/jira/browse/MYNEWT-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716023#comment-15716023
 ] 

Christopher Collins commented on MYNEWT-324:


Most request handlers reuse the request buffer for the reponse.  This is done 
to prevent out-of-memory conditions.  However, there are two handlers which do 
not reuse the request buffer:
1. Write request.
2. Indicate request.

Both of these handlers attempt to allocate a new buffer for the response prior 
to processing the request.  If allocation fails, the request is not processed, 
and the request buffer is reused for the transmission of an "insufficient 
resources" ATT error response.  These handlers don't reuse the request mbuf for 
an affirmative response because the buffer contains the attribute data that 
gets passed to the application callback.  The application may choose to retain 
the mbuf during the callback, so the stack


> BLE Host - Ensure there is always sufficient resources to send an ATT 
> "insufficient-resources" error response.
> --
>
> Key: MYNEWT-324
> URL: https://issues.apache.org/jira/browse/MYNEWT-324
> Project: Mynewt
>  Issue Type: Bug
>  Components: Nimble
>Reporter: Christopher Collins
>Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-324) BLE Host - Ensure there is always sufficient resources to send an ATT "insufficient-resources" error response.

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-324.

Resolution: Fixed

> BLE Host - Ensure there is always sufficient resources to send an ATT 
> "insufficient-resources" error response.
> --
>
> Key: MYNEWT-324
> URL: https://issues.apache.org/jira/browse/MYNEWT-324
> Project: Mynewt
>  Issue Type: Bug
>  Components: Nimble
>    Reporter: Christopher Collins
>Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: NRF52dk ADC

[Christopher Collins]

On Fri, Dec 02, 2016 at 09:30:04AM -0800, will sanfilippo wrote:
> There is a branch called “sterly_refactor” that we used when we were
> refactoring bsp and other code. There is ADC code in there. If you
> checkout that branch you can go to the apps directory and look for the
> sblinky app. In that main.c (which is where I said this stuff shouldnt
> be but that was all just test code) it should show how the ADC was
> used. You cant just bring all that code in I suspect as it is way out
> of date, but I think there is enough in there to give you an idea of
> how things are expected to work.
> 
> Be warned though; that code was just all hacked in and I am not sure
> what state it was left in.

The sterly_refactor branch was deleted recently.  However, if you have a
github fork, you probably have a copy of that branch.

Chris

Re: Subscription Request and Question.

[Christopher Collins]

Hi Rodrigo,

On Fri, Dec 02, 2016 at 12:30:12PM +0100, Rodrigo Lorenzo Leal wrote:
> OK so pre shared key is my best option I guess.
> 
> What I don't understand is, if is possible to have pre-shared key, that is
> accesible through the cloud, why I cannot generate a key on the cloud and
> access it doing a request, from both the app and the device (through GPRS).
> 
> Maybe is not a OOB pairing in the strict Bluetooth protocol definition of
> the word, but what I want to do is encrypt the BLE communication with a key
> that is not exchange via Bluetooth.

Mynewt does not currently support this.  The nimble host only issues an
LE Start Encryption HCI command to the controller after a successful
pairing or bonding restoration, and the application has no say over the
specified key.  In other words, there is no way bypass the pairing
process and inject a key of your own.  That said, I don't think it would
be too much work to implement this feature.

Sorry if I missed this earlier in the thread, but will iOS really let
you use an external key for BLE encryption?  Just based on my limited
experience with CoreBluetooth, I wouldn't expect this functionality to
be exposed.

Thanks,
Chris

Re: Subscription Request and Question.

[Christopher Collins]

On Fri, Dec 02, 2016 at 10:59:13AM +, Tim Hutt wrote:
[...]
> (there are supposed to be native Channel-Oriented Connections coming but I
> wouldn't hold my breath).

Just a small clarification: connection-oriented channels were already
added in bluetooth 4.1.  However, Nimble doesn't support them yet, and I
believe neither iOS nor Android supports them as well.

This may be what you meant.  As I said, it was just a small
clarification :).  I just wanted to mention this, because we do plan on
adding CoC support to Nimble in the future.

Chris

Re: [newtmgr] Delete/overwrite image in slot 1

[Christopher Collins]

Hi Kevin,

On Fri, Dec 02, 2016 at 09:51:25AM +0100, Kevin Townsend wrote:
> Is there a means with newtmgr today to delete an image in bank 1, for 
> example if you uploaded an image there incorrectly?

There is no way to delete the image in slot 0.  This is an instance of a
more general rule: mynewt won't let you overwrite an image that is
currently "in use" (more on this below).  Slot 0 is always in use,
because it is always the image that is currently running.

This email, while light on implementation details, may help to
explain what the various flags mean:
https://lists.apache.org/thread.html/2dd29b459958b54543b9a7b75ccf7e0879b4a11b39191ab420105b0e@%3Cdev.mynewt.apache.org%3E

> 
> Assuming the following situation:
> 
> Images:
>   slot=0
>version: 0.3.0
>bootable: true
>flags: active
>hash: 7494da56ead5fe83a0e26ad72fdf1bc8a4d1d6b01b1712e4e1bf4437218f4d8b
>   slot=1
>version: 0.2.0
>bootable: true
>flags: confirmed
>hash: bd2fb0d9b2c771b284b19d7c143754a8e26cfae837f1e50bb2467ee8dd57e9da
> 
> If you try to upload an image to bank 1 again you will get this error:
> 
> newtmgr -c serial image upload 
> /Users/antonio/Development/Adafruit/NewtManager/imgfiles/bleuart_0_3_0.img
> Error: Target error: 2
> 
> newtmgr -c serial image upload 
> /Users/antonio/Development/Adafruit/NewtManager/imgfiles/throughput_0_2_0.img 
> 
> Error: Target error: 2

In this case, both slots are in use, so you can't delete either.  Slot 0
is in use because it is running (active).  Slot 1 is also "in use"
because of the confirmed flag.  There are two ways to get out of this
state:

1. Confirm the current image state (newtmgr image confirm).  This tells
Mynewt to continue running the image in slot 0 after the next reboot,
and to stop using the slot 1 image.

2. Reboot the device.  This causes Mynewt to fall back to the confirmed
image.  After a reboot, the two slots will be swapped, slot 0 will
be confirmed and active, and slot 1 will have no flags set.

> Would it make sense to have an option to clear the second bank as an 
> option via a specific newtmgr command, or perhaps there are some 
> specific reasons to prevent the second image from being changed or 
> overwritten that I'm not aware of?

There are two reasons why the image management commands might seem
overly "fiddly".  One is to protect the user from themself.  The
situation above is one of those cases; the image in slot 0 is being
tested, and Mynewt tries to prevent the device from being bricked by
only booting into the image once unless manual confirmation is provided.

The other reason image management is tricky has to do with the boot
loader implementation.  Some of the image properties flags (e.g., "under
test") get written at the end of the image slot.  (As I'm sure you
know,) There is no way to clear this part of flash without erasing the
entire sector.  So, in some cases, you have to allow Mynewt to swap the
two images in order to clear the flags.

Today I plan on documenting the boot loader, which may also help explain
some of this behavior.

Please let me know if it is still unclear.

Thanks,
Chris

[jira] [Resolved] (MYNEWT-353) BLE Host - Timeout for ATT queued writes

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-353.

Resolution: Fixed

> BLE Host - Timeout for ATT queued writes
> 
>
> Key: MYNEWT-353
> URL: https://issues.apache.org/jira/browse/MYNEWT-353
> Project: Mynewt
>  Issue Type: Improvement
>  Components: Nimble
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> A peer performs a large attribute write by sending a series of ATT prepare 
> write requests followed by an ATT execute write request.  If the execute 
> write request never arrives, the queued prepare write requests remain 
> resident on the receiving device.  The queued write should be cleared N 
> seconds after the last received prepare write request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-341) BLE Host - Handle missing ACL data fragments

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-341.

Resolution: Fixed

> BLE Host - Handle missing ACL data fragments
> 
>
> Key: MYNEWT-341
> URL: https://issues.apache.org/jira/browse/MYNEWT-341
> Project: Mynewt
>  Issue Type: Improvement
>    Reporter: Christopher Collins
>    Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (MYNEWT-500) newt - Ambiguity when package contains two source files with the same filename

[Christopher Collins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MYNEWT-500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Collins resolved MYNEWT-500.

Resolution: Fixed

> newt - Ambiguity when package contains two source files with the same filename
> --
>
> Key: MYNEWT-500
> URL: https://issues.apache.org/jira/browse/MYNEWT-500
> Project: Mynewt
>  Issue Type: Bug
>  Components: Newt
>    Reporter: Christopher Collins
>        Assignee: Christopher Collins
> Fix For: v1_0_0_rel
>
>
> For example, the arduino repo (until very recently) contained the following 
> two files:
> * 
> {{hw/mcu/atmel/samd21xx/src/sam0/utils/cmsis/samd21/source/gcc/startup_samd21.c}}
> * {{hw/mcu/atmel/samd21xx/src/startup_samd21.c}}
> Both of these files are in the same package (hw/mcu/atmel/samd21) and they 
> both have the same filename (startup_samd21c).
> During a build, newt compiles both, but only links in one object file.
> The problem is that the bin directory structure does not mirror that of the 
> source tree.  Specifically, all of a package's object files get put in the 
> same directory; subtrees are not preserved.  Consequently, the second .o file 
> overwrites the first.
> The fix is to preserve the source directory structure in the bin directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MYNEWT-500) newt - Ambiguity when package contains two source files with the same filename

[Christopher Collins (JIRA)]

Christopher Collins created MYNEWT-500:
--

 Summary: newt - Ambiguity when package contains two source files 
with the same filename
 Key: MYNEWT-500
 URL: https://issues.apache.org/jira/browse/MYNEWT-500
 Project: Mynewt
  Issue Type: Bug
  Components: Newt
Reporter: Christopher Collins
Assignee: Christopher Collins
 Fix For: v1_0_0_rel


For example, the arduino repo (until very recently) contained the following two 
files:

* 
{{hw/mcu/atmel/samd21xx/src/sam0/utils/cmsis/samd21/source/gcc/startup_samd21.c}}
* {{hw/mcu/atmel/samd21xx/src/startup_samd21.c}}

Both of these files are in the same package (hw/mcu/atmel/samd21) and they both 
have the same filename (startup_samd21c).

During a build, newt compiles both, but only links in one object file.

The problem is that the bin directory structure does not mirror that of the 
source tree.  Specifically, all of a package's object files get put in the same 
directory; subtrees are not preserved.  Consequently, the second .o file 
overwrites the first.

The fix is to preserve the source directory structure in the bin directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

If using develop, upgrade newt and core at the same time

[Christopher Collins]

Hello all,

TL;DR: If you pull core develop, be sure to pull newt develop at the
same time.  Otherwise, your builds will fail.

I just pushed a fix to the develop branch for this issue:
https://issues.apache.org/jira/browse/MYNEWT-365

Unfortunately, this fix introduces an incompatibility with the core
repo.  Specifically, the compiler.yml definitions had to change a bit.
More details are available in the ticket and the commit messages.

Please let me know if there are any questions.

Thanks,
Chris