Mynewt Windows port

[Christopher Collins]

Hi all,

I wanted to share some details about what is planned for the Windows
port of Mynewt.  There is still some work to be done, but here is a
rough description of what is planned.

Thanks to a lot of help from Todd and Ray, we have a Windows port that
is mostly working.  I wanted to provide a summary to see if anyone
anticipates any major issues.  I am certain that I got some of the
docker concepts wrong, so please feel free to correct any of my
misconceptions :).

Rather than create Windows ports of all the necessary components, we
just run Linux versions of all the software in a virtual machine.
Windows communicates with the Linux VM via transient ssh connections.
When the user issues a command intended to run in Linux from a Windows
command prompt, the corresponding Linux command is executed in the Linux
VM.  This allows the user to utilize Linux from within Windows without
having to explicitly mess with VMs.  All commands intended to run in
Linux would need to be prefaced by a script that we provide to ensure
the necessary ssh mechanism is used.

There are a few shortcomings with this solution, as described:
* There is no protection against the VM accidentally becoming
  unusable (e.g., required files getting deleted in the VM).
* Upgrading any Linux tools, in particular the newt tool, requires
  the user to download a complete VM containing the upgrade.

These shortcomings are addressed by a tool called Docker.  When
executed, Docker creates a consistently reproducible and self-contained
environment in Linux.  By creating a custom docker image, a user can
have assurance that a particular set of commands will always execute
successfully.

In addition, docker images use a layered format.  To receive updates to
one or more of the linux tools, the user just needs to download the
newest layers (similar to applying a patch).

The Windows install of Docker ("docker toolbox") consists of:
* A Windows port of ssh.
* A copy of Oracle VirtualBox.
* A minimal Linux VM that runs in VirtualBox.
* The docker tool.

In addition, we would need to provide:
1. A docker image consisting of an environment with:
* The newt command-line tool
* Go
* A multilib-capable native gcc / glibc
* An arm-none-eabi gcc
* Native gdb

2. A Windows batch file (script) that allows the user to run Linux
   commands from a command prompt.  The batch file would also ensure
   that the VM is running.

Every time the user uses the script to run a command, the following
sequence of events occurs:
1. A new docker environment is created in the Linux VM.
2. The specified command is sent to the docker environment via ssh.
3. The Linux command runs.
4. The output from the command is sent back to Windows via ssh.
5. The output is displayed in the Windows command prompt.

All the user files (C source, yml, .o, .a, .elf, etc.) reside on the
Windows file system, not in the Linux filesystem within the VM.  So, the
user does not need to interact with the VM at all to manipulate any
files related to his or her project.

So that is the proposal for how the Windows port will work.  My main
concern with this solution is that setting up docker in Windows 10 is
problematic.  There are issues configuring the host-only interfaces
required for the ssh connections.  There is a new version of the docker
toolbox to be released shortly (within a week or two) that is supposed
to address these isses, so hopefully it is not something we need to
worry about.

There are a few other issues that I have run into, but it is difficult
to tell how serious they are because of the Windows 10 problems.  So, I
think we are stuck waiting for the next docker toolbox release before I
can determine what we can and cannot hide behind the batch file.

Finally, one annoyance is that this solution only works on an actual
Windows box.  The Linux VM fails to start if Windows itself is already
running in a VM.  I don't anticipate this being an issue for users, but
it creates quite a bit of hassle for people developing the Windows port.

Thanks for reading,
Chris 

Re: Mynewt logo - request for input

[Christopher Collins]

I think they are all really good, but my preference is for 1 or 2.

I lke 1's font the best, and I like the way the newt's tail wraps around
the y.  So actually I guess I like 1 the best :).

Chris



On Tue, Nov 17, 2015 at 09:49:24AM -0800, aditi hilbert wrote:
> Good point! Doing that now :)
> 
> > On Nov 17, 2015, at 9:47 AM, P. Taylor Goetz  wrote:
> > 
> > Cool!
> > 
> > For the purpose of discussion, it might be a good idea to annotate the PDF 
> > to give each one a number for easier reference. Otherwise we’ll have to 
> > reference them by description (e.g. “The green one with the ‘m’ inside the 
> > newt’s tail.” ;) )
> > 
> > -Taylor
> > 
> > 
> >> On Nov 17, 2015, at 12:32 PM, aditi hilbert  wrote:
> >> 
> >> Thanks, all. I just created MYNEWT-12 
> >> .
> >> 
> >> aditi
> >> 
> >> 
> >>> On Nov 17, 2015, at 9:21 AM, P. Taylor Goetz  wrote:
> >>> 
> >>> Confirming that no attachments were received on your reply.
> >>> 
> >>> Looks like another route is in order. JIRA attachments might be the 
> >>> easiest route.
> >>> 
> >>> -Taylor
> >>> 
>  On Nov 17, 2015, at 12:12 PM, Marvin Humphrey  
>  wrote:
>  
>  On Tue, Nov 17, 2015 at 7:32 AM, aditi hilbert  wrote:
> > I did attach the document with 4 drawings of “mynewt” to my mail. I am 
> > going
> > to try a second time with the original drawings of 6 “mynewts". Please
> > ignore the first page with drawings of newt on it. The following two 
> > pages
> > should show a total of 6 drawings. Hope you can see them this time!
>  
>  It seems that this emailing list is configured to strip attachments.  As 
>  extra
>  confirmation, I've attached the Incubator's egg logo to this email; I 
>  predict
>  you won't see it, though because of gmail's quirky deduping I probably 
>  will.
>  
>  The easiest way to share an image is probably to open a MYNEWT Jira 
>  issue and
>  attach it there, or possibly just to commit the files to the site svn 
>  repo.
>  For text there are lots of other options such as paste.apache.org, Github
>  gists or pull requests, and so on, but for images there's not a perfect 
>  answer
>  that I know of.  The downside is that we won't be able to view them in 
>  our
>  browsers without downloading first because of MIME type issues (unless 
>  you
>  commit to svn and fiddle with the svn properties), but since the target
>  audience is developers, it's not unreasonable to ask us to figure things 
>  out.
>  
>  Marvin Humphrey
> >>> 
> >> 
> > 
> 

Re: Mynewt Windows port

[Christopher Collins]

Oops, I meant to put the appropriate URLs in those empty parenthesis.
Here they are:

Docker hub: https://hub.docker.com/
golang docker image: https://github.com/docker-library/golang

Chris

On Wed, Dec 02, 2015 at 10:29:17AM -0800, Christopher Collins wrote:
> On Wed, Dec 02, 2015 at 08:41:45PM +1100, Justin Mclean wrote:
> > Hi,
> > 
> > Out of interest where is the mynewt docker instance hosted and what
> > does it contain?  I’m asking to make sure it couldn’t be considered an
> > Apache release.
> 
> Hi Justin,
> 
> That is a good question.  The docker image is hosted on docker hub ()
> 
> The image is based off of the golang 1.5 docker image ().  In addition,
> it pulls in the following debian packages:
> * gcc-multilib (for building 32-bit sim binaries)
> * libc6-i386 (also for sim binaries)
> * libnewlib-arm-none-eabi (for cross-compiling to arm)
> * gcc-arm-none-eabi (for cross-compiling to arm)
> 
> Below is the Dockerfile that was used to create the image.  Hopefully
> none of this runs afoul of any Apache or licensing concerns!
> 
> Thanks,
> Chris
> 
> 
> -
> 
> 
> # (Dockerfile)
> 
> FROM golang:1.5
> 
> # jessie-backports required for newlib.
> RUN echo 'deb http://httpredir.debian.org/debian jessie-backports main'
> > /etc/apt/sources.list.d/backports.list && \
> apt-get update && apt-get install -y --no-install-recommends \
> gcc-multilib \
> libc6-i386 \
> libnewlib-arm-none-eabi \
> gcc-arm-none-eabi && \
> rm -rf /var/lib/apt/lists/*
> 
> RUN go get
> git-wip-us.apache.org/repos/asf/incubator-mynewt-newt.git/newt
> 

Re: [VOTE] Release Apache Mynewt 0.9.0-incubating-rc3

[Christopher Collins]

On Mon, May 30, 2016 at 11:34:24AM -0700, Christopher Collins wrote:
> [X] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because...

Chris

Re: New Demo for stm32F3Discovery Based on Blinky

[Christopher Collins]

On Mon, Jun 06, 2016 at 12:42:38PM -0700, marko kiiskila wrote:
> I would leave blinky as is. It is beneficial to have a sample that
> shows that you can have applications which don’t need any platform
> specific changes.

I agree.  Blinky is Mynewt's "hello world" applications.hhC, and I think
it is important that it remain as simple as possible.

> As for the mynewt_pinwheel demo, you could check that into
> mynewt-stm32f3 repository as a separate app. The particular layout of
> those LEDs is specific for the STM32F3 discovery board, so to me it
> would make sense to keep the app in the same repository as the
> matching BSP.
> 
> What do you think?

I'm not David, but that sounds good to me.

Chris

Re: Tx power is always advertised as 0dBm on nrf51

[Christopher Collins]

Hi Simon,

On Thu, Jun 09, 2016 at 02:53:14PM -0700, Simon Ratner wrote:
[...]
> The above doesn't seem to have an effect on the contents of the advertising
> packet; it still advertises 0dBm. The comment in
> https://github.com/apache/incubator-mynewt-core/blob/master/net/nimble/host/src/ble_hs_adv.c#L188
> implies that this should be handled automatically, and the provided value
> is ignored. Is this implemented as described, or am I doing something wrong?
> 
> The actual tx power does change, for what it's worth - I see the expected
> difference in received signal between +4, 0, -12, -20.

As currently implemented, the nimble controller always indicates an
advertising power of 0.  I don't want to speak for Will who is more
familiar with this part of the code, but I believe this is intentional,
as the nimble link layer does not expect the transmit power to be
changed "behind its back." Apps are not expected to make direct calls to
the BLE drivers.  Rather, the intended client is the BLE link layer.
Since an interface into the transmit power has not been added to the
Nimble API yet, I don't blame you for trying!  I imagine that when
a transmit power interface is added to nimble, the adverising power
setting will get reported as configured.

Chris

Re: [VOTE] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

On Fri, May 27, 2016 at 07:56:39PM -0700, Christopher Collins wrote:
> [X] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because...

+1 (binding)

[DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

Hello all,

This thread is for any and all discussion regarding the release of
apache-mynewt-0.9.0-incubating-rc2.  All feedback is welcome.

Thanks,
Chris

[VOTE] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of Apache
Mynewt 0.9.0. Given that it is version 0.x, it is still a bit of a beta.
This is the second release candidate for Mynewt 0.9.0 (rc2); voting for
rc1 was cancelled due to a packaging mishap involving the newt source.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications. Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes.

This release candidate was tested as follows:
1. Manual execution of the Mynewt test plan
   
(https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Test+Plan).
2. The full unit test suite for this release was executed via "newt
   test all" with no failures. This testing was performed on the
   following platforms:
 * OS X 10.10.5
 * Linux 3.19.0 (Ubuntu 14.04)

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc2/

The commits under consideration are as follows:

blinky:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky
commit: 82f09fa2cc4c67fb30165ef6be0291aa76e6213e

core:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core
commit: 48df9b088c50cfc97d2518a823294ace761633d6

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt
commit: 0afe655414bbefc922d2adfddf238479bedac5c0

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc2/apache-newt-bin-linux-0.9.0-incubating.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc2/apache-newt-bin-osx-0.9.0-incubating.tgz


The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers,
please feel free to try out the release candidate and provide your
votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

Re: [DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

On Sat, May 28, 2016 at 01:38:48PM -0700, Sterling Hughes wrote:
> $ newt run  
> 
> Will do this for you.  No need to call binary directly. 

If you are building for real hardware (i.e., not sim), then you need to
add a version number to the end of the command.  So, the command would
look something like this:

newt run myblinky 0

If you leave off the 0, you will get a cryptic and intimidating error
message.

I was thinking we might want to make the version number optional.  When
people use the run command, most of the time they probably don't care
what version number the produced image has.

Chris

Re: How to determine the length of received data in a GATT callback

[Christopher Collins]

Hi James,

On Sat, May 28, 2016 at 12:53:14PM -0700, James Howarth wrote:
> Hi,
> 
> Looking at the peripheral callback gatt_svr_chr_access_gatt, is there a way
> to determine the length of data received?
> 
> Here's the whole function definition.
> 
> static int
> gatt_svr_chr_access_gatt(uint16_t conn_handle, uint16_t attr_handle,
> uint8_t op,
>  union ble_gatt_access_ctxt *ctxt, void *arg)

The length of the received data is contained in the ctxt->chr_access.len
field.  For example, bleprph handles an incoming write of the
gap-reconnect-address characteristic as follows:

case BLE_GAP_CHR_UUID16_RECONNECT_ADDR:
assert(op == BLE_GATT_ACCESS_OP_WRITE_CHR);
if (ctxt->chr_access.len != sizeof bleprph_reconnect_addr) {
return BLE_ATT_ERR_INVALID_ATTR_VALUE_LEN;
}
memcpy(bleprph_reconnect_addr, ctxt->chr_access.data,
   sizeof bleprph_reconnect_addr);
break;

The code validates the length of the incoming data immediately after the
assert() invocation.

Chris

Re: [DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

Hi David,

On Sat, May 28, 2016 at 06:05:10PM -0700, David Moshal wrote:
> how do I find the name of the target?

newt target show

e.g.,

[ccollins@iori:~/tmp/newttest]$ newt target show
targets/blink_nordic
app=apps/blinky
bsp=@apache-mynewt-core/hw/bsp/nrf52pdk
build_profile=debug
targets/my_blinky_sim
app=apps/blinky
bsp=@apache-mynewt-core/hw/bsp/native
build_profile=debug
targets/nrf52_boot
app=@apache-mynewt-core/apps/boot
bsp=@apache-mynewt-core/hw/bsp/nrf52pdk
build_profile=optimized

The "targets/" prefix is optional.  To run the my_blinky_sim target, I
would execute the following command:

newt run my_blinky_sim

Chris

Re: [VOTE] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

On Sun, May 29, 2016 at 10:56:09AM -0700, Christopher Collins wrote:
> The CMSIS-CORE files that Mynewt bundles fall into two categories:
[...]
> 2. Functional changes were made by ARM mbed before the BSD license
>was added; the oldest revisions containing the BSD license text
>are different from those in Mynewt.
> 
> The files in category 2 are the ones causing issues for us now.  Since
> the versions of these files in Mynewt are not directly derived from
> revisions containing the BSD license, I don't know what we can
> reasonably claim.

Specifically, the following files in the mynewt core repo are
problematic:

libs/cmsis-core/src/cmsis_nvic.c
hw/bsp/nrf51dk-16kbram/include/bsp/cmsis_nvic.h
hw/bsp/nrf51dk/include/bsp/cmsis_nvic.h
hw/bsp/nrf52pdk/include/bsp/cmsis_nvic.h
hw/bsp/olimex_stm32-e407_devboard/include/bsp/cmsis_nvic.h

The last four (the header files) are almost identical to one another,
and for simplicity I am considering them a single file in this email.

I'm attaching git diffs between the following two revisions:
a: commit when BSD license text was added (mbed repo)
b: first commit to the mynewt core repo.

The Mynewt files are "b" in the attached diffs.  That is, "+++" are
changes added by Mynewt; "---" is removed by mbed.

Thanks,
Chris
diff --git 
a/5b8ab176:libraries/mbed/targets/cmsis/TARGET_NORDIC/TARGET_MCU_NRF51822/cmsis_nvic.h
 
b/e08882916788cee7cb2540604f8f6ec166a1c654:hw/bsp/nrf52pdk/include/bsp/cmsis_nvic.h
old mode 100644
new mode 100755
index e1fd1a3..78d0417
--- 
a/5b8ab176:libraries/mbed/targets/cmsis/TARGET_NORDIC/TARGET_MCU_NRF51822/cmsis_nvic.h
+++ 
b/e08882916788cee7cb2540604f8f6ec166a1c654:hw/bsp/nrf52pdk/include/bsp/cmsis_nvic.h
@@ -1,48 +1,40 @@
-/* mbed Microcontroller Library
- * CMSIS-style functionality to support dynamic vectors
- 
***
- * Copyright (c) 2011 ARM Limited. All rights reserved.
- * All rights reserved.
+/**
+ * Copyright (c) 2015 Stack Inc.
  *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
  *
- * 1. Redistributions of source code must retain the above copyright notice,
- *this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright notice,
- *this list of conditions and the following disclaimer in the documentation
- *and/or other materials provided with the distribution.
- * 3. Neither the name of STMicroelectronics nor the names of its contributors
- *may be used to endorse or promote products derived from this software
- *without specific prior written permission.
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* mbed Microcontroller Library - cmsis_nvic
+ * Copyright (c) 2009-2011 ARM Limited. All rights reserved.
  *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
ARE
- * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
- * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
- * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
LIABILITY,
- * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE 
USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- 
***
- */ 
+ * CMSIS-style functionality to support dynamic vectors
+ */
 
 #ifndef MBED_CMSIS_NVIC_H
 #define MBED_CMSIS_NVIC_H
 
-#define NVIC_NUM_VECTORS  (16 + 32)   // CORE + MCU Peripherals
-#define NVIC_USER_IRQ_OFFSET  16
+#include 
 
-#include "nrf51822.h"
-#include "cmsis.h"
+#define NVIC_NUM_VECTORS  (16 + 38)   // CORE + MCU Peripherals
+#define NVIC_USER_IRQ_OFFSET  16
 
+#include "nrf52xxx/nrf52.h"
 
 #ifdef __cplusplus
 extern "C" {
 #endif
 
+void NVIC_Relocate(void);
 void NVIC_SetVector(IR

Re: BLE_GATT_CHR_F_NOTIFY and pinkey

[Christopher Collins]

On Thu, Jun 02, 2016 at 02:25:33PM +0200, Stephane D'Alu wrote:
> Adding the "| BLE_GATT_CHR_F_NOTIFY" result in an error when setting 
> advertisement, which seems to be: pinkey missing.
> 
> 105:[ts=102480ssb, mod=64 level=3] error enabling advertisement; rc=6
> 111:[ts=108336ssb, mod=4 level=0] Command Complete: cmd_pkts=1 ogf=0x0 
> ocf=0x0 status=5
> 
> I'm using bleprph (0.9-dev) as template, which is already using 
> BLE_GATT_CHR_F_NOTIFY for it's Alert service, and I haven't seen such 
> pinkey requirement. Any idea on what is wrong?

Hi Stephane,

Unfortunately, it appears you are the victim of a few red herrings :).
I will attempt to explain what is going on below.  First, the solution
is to increase the max_client_configs setting by at least two, e.g.,

cfg.max_client_configs = 8;

I believe this change will make the problem go away.

As for the cause of the errors:

> 105:[ts=102480ssb, mod=64 level=3] error enabling advertisement; rc=6

The nimble host uses its own set of error codes, separate from the HCI
error codes.  Your confusion is understandable, as we haven't documented
the host error codes anywhere!  This is something we'll need to get on
right away.  The host error codes are defined in
net/nimble/host/include/host/ble_hs.h.  An error code of 6 indicates
memory exhaustion.  In this case, the host did not pre-allocate enough
client characteistic configuration descriptors (CCCDs) for a single
connection, so it prevented the application from performing a
connectable advertisement.

You need to increase max_client_configs by 2 (rather than 1) because the
host maintains a cache of CCCDs that aren't tied to any connection.  The
formula is:

subscribable-charactericteristics * (max-connections + 1)

Where "subscribable-characteristics" is the number of characteristics
supporting notifications or indications.

The other error:

> 111:[ts=108336ssb, mod=4 level=0] Command Complete: cmd_pkts=1 ogf=0x0 
> ocf=0x0 status=5

This is caused by a bug in the hci debug output code.  The event being
logged here is the no-op that the controller sends on startup.  The
no-op is the only event which does not have a status field.  The debug
code assumes all events have statuses, and it prints a bogus value here.
This should be an easy fix, and I'll look at it shortly.

Finally, your email sparked a few thoughts.  These are just some musings
that need some more thought, but it will help me to write them down, so
I hope you'll humor me and allow me to do so :).

1. We need better diagnostics.  It would be great if a message was
printed to the debug log indicating exactly what failed when you
attempted to advertise.

2. It is too difficult to get all the configuration settings correct.
I'm wondering if we should have the application pass its service table
to ble_hs_init().  That way, the correct values for max_services,
max_attrs, and max_client_configs can all be derived from the table.
One complication here is that the application is currently allowed to
register its services in several steps (multiple calls to
ble_gatts_register_svcs(), each time passing a different table).  I
suppose the options are to either a) remove support for multistage
registration, or b) maintain the current configuration behavior, and
make the new service table method optional.

Thanks,
Chris

Re: GDB scripts

[Christopher Collins]

On Wed, Jun 01, 2016 at 10:04:05AM -0700, Vipul Rahane wrote:
> Hello,
> 
> While debugging a bunch of things, I felt the need for a few gdb scripts to 
> print out the data structures that we would use regularly. I was thinking of 
> creating a common gdb script which would contain different functions. This 
> could reside in "libs/util" since we do not have a “tools" directory.
> 
> The developer would then just source the common script.
> 
> All suggestions are welcome.

Some gdb scripts are sorely needed, and I think that is a great idea.  I
wrote some small gdb macros ("functions"?) for inspecting mbufs; if you
create one or more gdb scripts, I'll follow your lead and add mine.

Chris

Re: Adding service to bleprph

[Christopher Collins]

On Wed, Jun 01, 2016 at 07:24:01PM +0200, Stephane D'Alu wrote:
> Hello again,
> 
> I was trying to add a service/characteristic to the bleprph example, and 
> thought it would be as simple as:
> 
> * add a gatt_svr_chr_access_ function
> * add the service definition in gatt_svr_svcs
> * increase some configuration values (max_services, max_attrs)
>to be safe
> 
> Once done, I'm not able to discover services anymore.
> Did I miss something?
> 
> (Using gcc 4.9.2, and newt 0.9.0)

Hi Stephane,

That is indeed all you should need to do.  However, when I apply your
patch, I observe the same results.  I'll get back to you after I spend
some time debugging this.

Thanks,
Chris

Re: Triggering GATT Notification's

[Christopher Collins]

On Wed, Jun 01, 2016 at 07:04:02AM +0100, Wayne Keenan wrote:
> Because I'm creating a 'ble_gatt_svc_def' at runtime these are the steps
> I'm having todo to be able to keep hold of the characteristic handle in
> order to be able to call 'ble_gatts_chr_updated(handle)' to send the
> notification when needed:
> 
> 1. When initialising the 'ble_gatt_chr_def' the  'arg' field is set to a
> marker value.
> 
> 2. In my 'gatt_svr_register_cb' callback if a ble_gatt_chr_def.arg marker
> value is found then the 'ctxt->chr_reg.def_handle' is stashed in a global
> var.

That will work, but as you say, the need for the "marker arg" is a bit
unwieldy.  I think you are correct - there isn't currently a good way to
do this.  There is an alternative approach that I came up with, but I am
not sure it is an improvement over what you are currently doing, so it
has been relegated to the bottom of this email!

After reading your email and looking at the code some more, I think
there are a few changes that need to be made to the nimble host
immediately:

1. The characteristic-registration-callback should receive a pointers to
both the service definition and the characteristic definition (currently
it only receives a pointer to the characteristic definition).  Without
the service in hand, it is impossible to identify the characteristic by
UUIDs.

2. (As you mentioned) A function to retrieve the attribute handles
corresponding to the characteristic with a specified
service-UUID/characteristic-UUID pair.

In addition, this is not directly related to the problem being
discussed, but I noticed we need a third change:

3. The characteristic access callback also needs to receive a pointer to
the service definition (rather than just a pointer to the characteristic
definition).  This chnage is needed for the same reason as 1 above: so
that the characteristic can be reliably identified by UUID.

[...]
> Is it safe to cache the 'union ble_gatt_register_ctxt *ctxt' pointer
> given as a parameter 'gatt_svr_register_cb' to use later?

No, it is not safe.  The ctxt object is located on the stack, so you
need to make copies of anything you might need later.

> Being able to call Newt GATT API's like the following would help alleviate
> steps 1,2 and help with step 3:
> 
> a) uuit16_t gatt_get_handle_for_chr(serviceUUID, charUUID)

Agreed.

> b) int gatt_write_chr(uint16_t handle, void* data, uint len, bool
> sendNotificationIfEnabled);

There is the following function:

int
ble_att_svr_write_local(uint16_t attr_handle, void *data,
uint16_t data_len);

When a characteristic is written via this function call, the subsequent
characteristic access callback indicates that the write was done locally
by specifying a connection handle of BLE_HS_CONN_HANDLE_NONE (0x).

The above function will always send notifications to all clients who are
subscribed to the corresponding characteristic.  Perhaps the
characteristic access callback should have the option of disabling
notifications for a particular write.  Alternatively, If you want to
update a characteristic without sending any notifications, you can
always bypass the nimble stack entirely and just write directly to the
attribute data.

[...]

Thanks again for you feedback.  It is very much appreciated.

Chris

-

As promised, below is a technique that eliminates the need for the
marker arg.  As I said, it is a bit too ugly for me to recommend with a
straight face, but I thought I should include it for.. I don't know...
some reason :).  The technique identifies characteristics by memory
address.

#define SVC_IDX_GAP 0
#define CHR_IDX_GAP_DEV_NAME0
#define CHR_IDX_GAP_APPEARANCE  1

static const struct ble_gatt_svc_def my_svcs[] = {
[SVC_IDX_GAP] = {
/*** Service: GAP. */
.type = BLE_GATT_SVC_TYPE_PRIMARY,
.uuid128 = BLE_UUID16(BLE_GAP_SVC_UUID16),
.characteristics = (struct ble_gatt_chr_def[]) {
[CHR_IDX_GAP_DEV_NAME] = {
/*** Characteristic: Device Name. */
.uuid128 = BLE_UUID16(BLE_GAP_CHR_UUID16_DEVICE_NAME),
.access_cb = gatt_svr_chr_access_gap,
.flags = BLE_GATT_CHR_F_READ,
},

[CHR_IDX_GAP_APPEARANCE] = {
/*** Characteristic: Appearance. */
.uuid128 = BLE_UUID16(BLE_GAP_CHR_UUID16_APPEARANCE),
.access_cb = gatt_svr_chr_access_gap,
.flags = BLE_GATT_CHR_F_READ,
},

/* [...] */

0, /* No more characteristics in this service. */
}
0, /* No more services. */
},
};


static void
gatt_svr_register_cb(uint8_t op, union ble_gatt_register_ctxt *ctxt,
 void *arg)
{
switch (op) {
case BLE_GATT_REGISTER_OP_CHR:
if 

Re: Triggering GATT Notification's

[Christopher Collins]

Hi Wayne,

On Tue, May 31, 2016 at 12:59:20PM +0100, Wayne Keenan wrote:
> Hi,
> 
> I have a custom GATT service working with read/write all ok; now I am
> looking at adding notify support.
> 
> On my first look thru the newt source my first take was that the app needs
> to cache the GATT registrations in 'gatt_svr_register_cb' to obtain the
> chr_val_handle and also track BLE_GAP_EVENT_CONN events and keep hold of
> the conn_handles's
>  so at a later point it can trigger a notification (for one for more
> clients) by calling:
> ble_gattc_notify(uint16_t conn_handle, uint16_t chr_val_handle)
> 
> 
> My first reaction to that is that I must be wrong as that perhaps too much
> duplicated state in the app.
> 
> Is there something simpler like   'ble_send_notify(char_handle, data, len)'
>  that will notify all connected clients ?

There are a few ways to send notifications:

1. The stack sends them automatically to subscribed clients when a
   characteristic value changes.  The client subscribes to notifications
   (or indications) for a particular characteristic by writing to the
   corresponding client characteristic configuration descriptor (CCCD).

2. The way you don't like :).  Call ble_gattc_notify() for each
   connection-characteristic pair that needs to be sent.  These
   notifications are sent "at will": the clients don't need to subscribe
   to the characteristic via a write to the CCCD, and the characteristic
   value doesn't need to change.  As you indicated, the application will
   need to keep track of both the connection handles and characteristic
   value handles.

3. Somewhat closer to what you are asking for:

int
ble_gattc_notify_custom(uint16_t conn_handle, uint16_t att_handle,
void *attr_data, uint16_t attr_data_len)

   The difference between this and option two above is that the
   contents of the notification are specified explicitly by the
   application rather than read from a characteristic.  The attribute
   handle need not even map to characteristic (though it probably
   should).

The nimble stack hasn't gotten a lot of practical use, so there are
undoubtedly some aspects of the API that are lacking common sense.  Now
is the perfect time to make changes to the API while Mynewt is still in
its beta phase, and input like yours is extremely valuable.  If you have
the time, I was hoping you could add some more details concerning what
you as an application developer would like to see exposed by the nimble
host.

You mentioned a function which would send a custom notification to all
connected devices.  Is this exactly what you are looking for, or were
you just trying to make a compromise with nimble's current API?

An additional facility that the host needs to provide is: a means for
the application to discover who is currently connected.  As you
indicated in your email, currently it is up to the application to cache
this information, which is certainly not ideal.  This is something that
will need to get added in the next release.

Please don't hesitate to complain about anything else that seems to be
missing from the API :).

Thanks,
Chris

Re: [VOTE] Release Apache Mynewt 0.9.0-incubating-rc2

[Christopher Collins]

Thanks again, Justin.  My responses are inline below.  I rearranged your
email such that your first finding comes last, since it is the most
difficult to address :).

On Sun, May 29, 2016 at 11:23:25AM +1000, Justin Mclean wrote:
[...]

> The core release notes is out of data and talks about March release
> with the next release being in April and it being the "second source
> release”.The readme is also not up to date, and the supported board
> list seem to be missing a few boards?

The outdated release notes is definitely our screw up.  However, the
list of supported platforms in the readme file looks correct to me.  The
only discrepancy I see is that the Arduino 101 isn't mentioned in the
list, but I think that is correct since this target is not fully
supported yet.  Is there another one that I missed?

> The newt release notes are also out of date and refers to the "first
> source release”.

Darn.  We should have caught this one as well.

> I would suggest that the artefact names be renamed to include “mynewt”
> not “newt” especially the core one for the next release.

That sounds reasonable.  I think we should just add "mynewt" to all
three component names, but I am interested in hearing others' thoughts.

> For core is there now any way we can improve on this situation?  "This
> product bundles additional files from CMSIS-CORE, but these files are
> missing licensing information.”

Yes, this one is indeed troubling.  To provide some background:

Recent CMSIS-CORE releases contain the BSD license text, but the older
releases do not, so it is possible that the distributor simply forgot to
add the license text in the older releases.  Back when the CMSIS-CORE files were
added to Mynewt core, we added versions which lacked the BSD license text.

The CMSIS-CORE files that Mynewt bundles fall into two categories:
1. Revisions exist which are identical to those which were
   originally included in Mynewt; the only difference being the
   addition of the BSD license.

2. Functional changes were made by ARM mbed before the BSD license
   was added; the oldest revisions containing the BSD license text
   are different from those in Mynewt.

The licensing issue was easy to solve for the "category 1" files: we
just replaced our old copies of these files with the corresponding
BSD-licensed revisions.

The files in category 2 are the ones causing issues for us now.  Since
the versions of these files in Mynewt are not directly derived from
revisions containing the BSD license, I don't know what we can
reasonably claim.  Intuitively, I feel like the Mynewt copies of these
files are "more or less" derived from the revisions containing the BSD
text, but obviously that is not a legal argument!

Do you have suggestions on how best to proceed?

(In case I gave the opposite impression, I agree that keeping things in
their current state is not a solution).

Thanks,
Chris

Re: Adding CLI seems to fail

[Christopher Collins]

On Fri, Jun 24, 2016 at 11:59:49AM -0400, David G. Simmons wrote:
> The trouble  arises at this point. No device is ever registered for
> the USB port, so using minicom (or screen, etc.) isn’t an option as
> there is not device. I’ve tried everything I can think of, from
> removing the STLINK jumpers to plugging in both USB (STLINK and USER),
> etc. but still no joy.

Hi David,

You'll need to make sure the FTDI virtual COM port driver is installed
on your machine (http://www.ftdichip.com/Drivers/VCP.htm).  With the
driver present, plugging a USB cable into your computer should cause a
tty device to be added to /dev.

You have two options with the nRF52dk: 1) the virtual COM port, or 2)
the UART pins.  You get the virtual COM port "for free" when you power
your board via USB.  On OS X, this interface has a name beginning with
/dev/tty.usbmodem.  Alternatively, to use the UART pins, you need to
connect a three-wire serial cable to the RX, TX, and ground pins on the
board.

The virtual COM port is easier to set up, but the UART pins seem more
reliable.  In my experience, the virtual COM port sometimes gets "locked
up"; it won't send or receive data.  I should add that I don't know if
this is a problem with my computer, Mynewt, or the nRF52dk, so take it
with a grain of salt.

Chris

Re: boot_serial support

[Christopher Collins]

On Sat, Jun 18, 2016 at 11:38:02AM +0200, Kevin Townsend wrote:
> Hi Marko,
> 
> Thanks for getting this into the development branch! ... I was just 
> setting things up to test it out, and I had a look at the code directly 
> on github in the 'develop' branch, but despite my project being on 
> '0-dev' it doesn't pull the changes locally when I run 'newt upgrade'. 

Unfortunately, pulling from git manually (as you did) is currently the
only way to get the latest changes.  This is a known shortcoming, and
there is a fix planned (https://issues.apache.org/jira/browse/MYNEWT-307).

Thanks,
Chris

Re: Base a custom bsp on an existing one

[Christopher Collins]

Hi Wayne,

On Fri, Jun 17, 2016 at 02:28:26PM +0100, Wayne Keenan wrote:
> or perhaps I'll break up my app into parts that can be independently
> included as dependencies in the pkg.deps section of pkg.yml.

> > Is it possible to use 'features' to include/exclude subdirectories of
> > source and include under app/myapp/  ?
> >
> > For example, I may have an app which *can* use bluetooth if available, but
> > I may want to have a target with a hardware platform that doesn't have
> > bluetooth so I don't want to compile any app code that depends on Nimble
> >
> > although  "#ifdef'ing FEATURE_NAME" the whole contents of many .c/.h files
> > out is possible I don't see it as desirable.

As you suspected, those are your only two options at the moment.  The
design of newt favors the approach where you break your app into several
packages, since newt's "fundamental unit" is the package, rather than
the directory or file.

After removing the bluetooth-dependent parts of your app and putting
them in separate packages, there are two ways to conditionally include
them in your project:

1. #ifdefs
* A newt "feature" gets defined if your app will be using bluetooth
  (e.g., BLE).

* The bluetooth-dependent library only get included as
  a dependency if the bluetooth feature is defined, e.g.,

pkg.deps.BLE: 

* A preprocessor symbol gets defined if the BLE feature is defined,
  e.g.,

pkg.cflags.BLE: -DBLE_PRESENT

* All calls into the BLE-dependent library are enclosed in ifdefs,
  e.g.,

#ifdef BLE_PRESENT
initialize_ble();
#endif

2. Stub library
* You create two packages: my_ble_pkg and my_ble_stub.  Both of
  these packages expose the same interface via identically-named
  header files.

* The app chooses which package it depends on based on the presence
  of a feature:

pkg.deps.BLE: my_ble_pkg
pkg.deps.NO_BLE: my_ble_stub

* The my_ble_pkg contains a real implementation.  The stub package
  just contains function stubs which return not-supported error
  codes or similar.

* The BSP defines of the BLE or NO_BLE features.

The advantage of the stub library approach is that you don't need to
litter your app code with ifdefs.  However, it may not be practical to
design the app so that it is entirely ignorant of whether BLE is
supported.

I think there is some room for improvement in this area of newt, but I
don't know what the right fix would be.  I think as people build Mynewt
projects we will start to see what functionality is missing.  Any
thoughts you may have on the subject are very welcome, of course.

Thanks,
Chris

OS_EINVAL vs. OS_INVALID_PARM

[Christopher Collins]

Hello all,

I noticed libs/os/include/os/os.h defines the following two error codes:

OS_EINVAL = 2,
OS_INVALID_PARM = 3,

Do these error codes convey different results?  After a brief look
through the code, my impression is that they mean the same thing.  If
that is the case, I think one of the error codes should be removed.  My
personal preference would be to keep OS_EINVAL just because EINVAL has a
well understood meaning.

Thanks,
Chris

Re: Clarification on ble_gattc_*_by_uuid calls

[Christopher Collins]

On Mon, Jun 27, 2016 at 04:13:09PM -0700, Simon Ratner wrote:
> Hi Chris,
> 
> One quick clarification question: after receiving a callback with any
> error status, do you guarantee that the callback will never be called
> again (including with a NULL svc param)?

Hi Simon,

Yes, that is the expected behavior.  If the callback is executed with a
non-null error argument, or with a null svc / attr / etc. argument, then
the GATT procedure is complete.  If that callback gets called again, it
should only be because it is associated with an additional GATT
procedure.

Chris

Re: README.md files

[Christopher Collins]

Just an "fyi" - I am planning on tagging and restarting the release
process at about 3:00 PST today (in one hour).  If you have any
documentation or license changes that need to go into master, please let
me know before then.

Thanks,
Chris

On Tue, Feb 23, 2016 at 11:07:28AM -0800, Christopher Collins wrote:
> Hi all,
> 
> In preparation for the Apache Mynewt 0.8.0 beta 1 (incubating) release,
> I have made some minor changes to the top-level README.md files.  I
> think these files could probably use some more work, and I wanted to
> request a favor: if you have time to take a look, please do so and make
> any improvements as you see fit (or suggest changes).
> 
> Here are the minor edits I have made recently:
> 
> newt:
> * Moved newt/README.md to top-level directory (i.e., it applies to
>   newt, newtmgr, and newtvm, rather than just newt).
> * Added basic steps for building the newt tool.
> * Added link to the getting started page
>   (http://mynewt.apache.org/os/get_started/project1/)
> 
> larva:
> * added link to the getting started page
>   (http://mynewt.apache.org/os/get_started/project1/)
> 
> tadpole:
> * I didn't really change anything here; the readme looked decently
>   thorough.
> 
> Thanks,
> Chris

[RESULT][VOTE] Release Apache Mynewt 0.8.0-b1-incubating

[Christopher Collins]

Hello all,

Voting for Apache Mynewt 0.8.0-b1-incubating is now closed.  The release
has passed this step of the process.  The vote breakdown is as follows:

+1 Aditi Hilbert
+1 Justin Mclean
+1 Marko Kiiskila
+1 Sterling Hughes
+1 Will san Filippo

Total: +5

We can now call a vote on the general@incubator list.

Thank you to all who voted.

Thanks,
Chris

Re: [RESULT][VOTE] Release Apache Mynewt 0.8.0-b1-incubating

[Christopher Collins]

FYI - here is the vote thread on the general list:
http://mail-archives.apache.org/mod_mbox/incubator-general/201602.mbox/%3C20160227030138.GT14480%40iori.nightmare.heaven%3E

(Yes, I need to change my computer domain!)

Chris

On Fri, Feb 26, 2016 at 06:11:25PM -0800, Christopher Collins wrote:
> Hello all,
> 
> Voting for Apache Mynewt 0.8.0-b1-incubating is now closed.  The release
> has passed this step of the process.  The vote breakdown is as follows:
> 
> +1 Aditi Hilbert
> +1 Justin Mclean
> +1 Marko Kiiskila
> +1 Sterling Hughes
> +1 Will san Filippo
> 
> Total: +5
> 
> We can now call a vote on the general@incubator list.
> 
> Thank you to all who voted.
> 
> Thanks,
> Chris

[VOTE] Release Mynewt 0.8.0-b1-incubating

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of
mynewt-0.8.0-b1-incubating.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications.

The release candidate to be voted on is available at:

https://dist.apache.org/repos/dist/dev/incubator/mynewt/mynewt-0.8.0-b1-incubating/

The commits under consideration are as follows:

larva:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-larva
commit: bd2db03ccbd019a20267459bf46ae1e1428f1f46

tadpole:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-tadpole
commit: b00813ec355a0bc3681c232503aab92ea9157fa9

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt
commit: 27a92e159926778f24d19b0795307a34e05ec8ed

The release candidate is signed with a GPG key available at:

https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at least
three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers, please feel
free to try out the release candidate and provide your votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

Re: [3/3] incubator-mynewt-larva git commit: Add license to .gitignore file.

[Christopher Collins]

On Tue, Feb 23, 2016 at 09:39:41AM +1100, Justin Mclean wrote:
> Hi,
> 
> > Whoops, thanks Justin.  I didn't see that note in time. 
> 
> Pulling license header of files that may not require it is a non issue, I 
> very much doubt that anyone would take issue with it or in fact even notice 
> :-)
> 
> > Is there anything else you can think of that I might have missed?
> 
> AFAICS it all looks good.

At the risk of exceeding my 'thank you' quota, thanks again for your
patience and support.  I am sure the process will be a little less
painful next time :).

Chris

Re: RAT (release audit tool)

[Christopher Collins]

Thanks, Justin, that is a good idea.  I'll run both of those tools.

Chris

On Sun, Feb 21, 2016 at 04:54:17PM +1100, Justin Mclean wrote:
> Hi,
> 
> It quite likely that anyone reviewing the release on the incubator will use:
> 1) Apache rat [1]
> 2) Compliance Rocks 
> 
> We might want to put a rat exclusion file together to help people and cut 
> down of the number of false positives it currently generates.
> 
> At the very least yo shovel run the tools over the source release so are 
> prepared for any issue that they might bring up.
> 
> Thanks,
> Justin
> 
> 
> 1. http://creadur.apache.org/rat/apache-rat/
> 2. http://compliance.rocks

Re: Go include path / Git Include Path

[Christopher Collins]

The only way I could get "go get" to work with newt is by using the
".git" path.  I changed all of the newt's include paths to the current
form to allow for simpler installation instructions.

Chris

On Tue, Jan 19, 2016 at 10:07:24AM -0800, Sterling Hughes wrote:
> Howdy,
> 
> Right now there is a conflict in how newtmgr & newt include libraries using 
> go.
> 
> Newtmgr uses:
> 
> git-wip-us.apache.org/repos/asf/incubator-mynewt-newt/newtmgr
> 
> Whereas Newt uses:
> 
> git-wip-us.apache.org/repos/asf/incubator-mynewt-newt.git/newt
> 
> In order for both of these packages to compile, we're going to need to
> choose one layout and stick with it.  Personally, I'm fine with either
> -- I think newtmgr's makes more sense, but if there is a good reason
> for sticking with Newt's pathes, and I'm ok with that as well.
> 
> Thoughts?
> 
> Sterling

Re: Proposed changes to sys/stats

[Christopher Collins]

Thanks for the feedback, all.  I will commit the first three changes to
the develop branch, but leave proposal 4 unimplemented.

Chris

On Tue, Feb 16, 2016 at 01:35:16PM -0800, will sanfilippo wrote:
> +1 for 1, 2 and 3.
> 
> Not sure about 4. I am fine with using the macros and prefacing the element 
> names but not sure what other issues this might cause so I will abstain.
> 
> Will
> 
> > On Feb 16, 2016, at 1:13 PM, Sterling Hughes  wrote:
> > 
> > 
> >> 1. The STATS_SECT_START and STATS_SECT_END macros just define a struct;
> >> they don't create an instance.  Generally, these macros would be used
> >> in a header file so that source files can have access to the struct
> >> definition.
> >> 
> >> 2. The addition of a STATS_SECT_DECL macro.  This macro would be used in
> >> two places:
> >> * In source files to instantiate a stats struct.
> >> * In header files to expose an extern declaration of a stats
> >>   instance.
> >> 
> >> 3. As a consequence of the above two points: the names of struct
> >> instances are no longer auto-generated.  The user needs to specify the
> >> exact name.  All macros which derive the instance name from the struct
> >> name are changed: now they just accept the instance name directly.
> >> 
> > 
> > 
> > +1
> > 
> > 
> >> 4. Remove the "s" which prefaces the name of each stat field in a
> >> statistics struct.  By doing this, the STATS_SECT_VAR, STATS_INC, and
> >> STATS_INCN macros can be removed.
> >> 
> > 
> > -1
> > 
> > I think we want these macros used everywhere: they allow us more 
> > flexibility to refactor this, and there is a relatively well defined access 
> > mechanism for stats.
> > 
> > Sterling
> 

Re: Kicking off a release

[Christopher Collins]

Sounds good.  I will act as the release manager.

Chris

On Thu, Feb 18, 2016 at 01:48:43PM -0800, Sterling Hughes wrote:
> 
> 
> On 2/16/16 5:07 PM, Justin Mclean wrote:
> > Hi,
> >
> > Who is going to act as release manager?
> >
> 
> Good question: I nominate Chris.
> 
> >
> >> - Add release notes to the mynewt distribution
> >> - Tag the tree with a B1 release tag (mynewt_0_8_0_b1_tag)
> >> - Start a [VOTE] thread to [VOTE] on the Mynewt release (along with
> >> the given tag)
> >> - Should that [VOTE] go through, that tag can then be built into a
> >> series of release artifacts
> >
> > The release artefacts are built first. The  [VOTE] email will have links
> > to the source tar (and binary if we’re creating them) to vote on.
> >
> > After the vote here, you then need to put it up for vote on the
> > incubator general mailing list were only incubator PMC votes are
> > binding. A few of the mentors are IPMC (myself included) so that should
> > help.
> >
> 
> 
> Nice.  So Chris (or whoever) needs to create all the release artifacts 
> (source tarball, binaries, etc.) and tag the tree.  Compose an email to 
> dev@ first (with a [VOTE] thread?), and then if that passes, compose a 
> similar email to general@, with the note that it has already passed a 
> release vote within the podling, correct?
> 
> > Only after both votes pass can you do this:
> >> - That can then be posted on the Mynewt website, along with release notes.
> >
> > And send out a release announcemt!
> >
> > The LICENSE and NOTICE files also need to be updated [1] before a VOTE
> > can be called.
> >
> > Also last time I looked 2 or 3 repos still have a few header files and
> > the like to clean up. It would probably pass an incubator vote in that
> > state (given there a JIRA to fix it) but best to clean up before release
> > I think?
> >
> 
> I'll run through and make changes today.
> 
> Thanks,
> Sterling

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 01:18:19PM +1100, Justin Mclean wrote:
> Hi,
> 
> > Thanks, Justin.  If we remove this from the distribution (i.e., don't
> > bundle it), do we still need to mention it in LICENSE or NOTICE?
> 
> No only things that are bundled need to be mentioned in LICENSE and
> NOTICE, so no mention in notice.

Got it.  Thanks.

> 
> But having a dependancy on lGPL / GPL gives restrictions that are not
> compatible with the Apache license so mention at in the README and
> that it will be fixed in a future release.
> 
> I seen the send button a little to quick last email. Want me to do
> what sebb suggests and email legal / Jim.

Ah, I see the post
(http://mail-archives.apache.org/mod_mbox/incubator-general/201602.mbox/browser).
My reading of that message was that Sam (sebb?) had already sent a note
to legal, but maybe he is waiting for someone to protest :).  Either
way, it would probably be a good idea (and much appreciated!) if you
sent a note as well, or please feel free to let me know if there is
anything I can do.

Thanks again,
Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 01:43:59PM +1100, Justin Mclean wrote:
> If you’re going to be RM, you should sign up to that list and/or take
> a look previous incubating release votes so you know what to expect.

Good idea.  I have subscribed to that list.

Now... :) I am afraid I'm going to have to nag you with a few more
questions.  If you could take a look when you have a second, that would
be really helpful.  Thanks for your patience!

(1)
I was a little confused about where I should put the release artifacts
prior to sending the dev vote email.  After some reading, I have
inferred that I should put the artifacts here via svn:

https://dist.apache.org/repos/dist/dev/incubator/mynewt

Is that correct?

(2)
Below is a list of files I am planning on uploading to the server:

/repos/dist/dev/incubator/mynewt/mynewt-0.8.0-b1-incubating/
larva-0.8.0-b1.tgz  # Larva source
larva-0.8.0-b1.tgz.asc  # Larva ASCII armored detached signature
larva-0.8.0-b1.tgz.sha  # Larva SHA512 checksum

newt-0.8.0-b1.tgz   # Newt
newt-0.8.0-b1.tgz.asc
newt-0.8.0-b1.tgz.sha

tadpole-0.8.0-b1.tgz# Tadpole
tadpole-0.8.0-b1.tgz.asc
tadpole-0.8.0-b1.tgz.sha

newt-bin-linux-0.8.0-b1.tgz # Newt binary for linux
newt-bin-linux-0.8.0-b1.tgz.asc
newt-bin-linux-0.8.0-b1.tgz.sha

newt-bin-osx-0.8.0-b1.tgz   # Newt binary for Mac OS X
newt-bin-osx-0.8.0-b1.tgz.asc
newt-bin-osx-0.8.0-b1.tgz.sha

/repos/dist/dev/incubator/mynewt/
KEYS# Public keys for sig. validation

The binary files are created from the newt sources as a convenience to
the user.

Does the above directory tree look reasonable to you?

(3) I was planning on using this message as a template for the vote email
that I will send to the mynewt dev list.  Do you think it is suitable?


http://mail-archives.apache.org/mod_mbox/incubator-myriad-dev/201511.mbox/%3CCAGim3%2B2fFCE4LC7HpNPtg5r7_01sFzkfLsyGaLvXcV%2B6f-_CRg%40mail.gmail.com%3E

Thanks,
Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 04:19:57PM +1100, Justin Mclean wrote:

[convenience binaries]
> And are we 100% sure that they don’t contain the LGPL code (source or
> compiled) ?

No - the binaries certainly *do* contain LGPL code.  I take it that is a
problem :).  I think we can refrain from releasing the binaries until we
get the LGPL issue sorted out.  If others disagree with this, please
chime in.

> >
> > http://mail-archives.apache.org/mod_mbox/incubator-myriad-dev/201511.mbox/%3CCAGim3%2B2fFCE4LC7HpNPtg5r7_01sFzkfLsyGaLvXcV%2B6f-_CRg%40mail.gmail.com%3E
> >  
> > 
> 
> It’s good, but I add the git hash, tags can be deleted or changed. The
> idea is that given a successful vote someone could take the info in
> the email and produce the exact same release as you did. You may also
> offer some advice on what is needed to be checked when voting.
> 
> It also customary to send out a [DISCUSS] email as well that was the
> vote thread doesn’t get polluted by people replying with questions /
> any issues they have found..

All good points.  Noted.

Thanks,
Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 04:19:57PM +1100, Justin Mclean wrote:
> Would you like me to give the various LICENSE and NOTICE files a once
> over before you make an official RC?

Oops, I am not sure how I missed this.  Yes, that would be extremely
helpful.

Thanks,
Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 05:54:11PM +1100, Justin Mclean wrote:
> Hi,
> 
> >> Would you like me to give the various LICENSE and NOTICE files a once
> >> over before you make an official RC?
> > 
> > Oops, I am not sure how I missed this.  Yes, that would be extremely
> > helpful.
> 
> I assume they are in the top of each repo? I can’t get to this right now 
> (just about to go out to dinner on friday) but can look at it tomorrow.

Yes, that is correct.  Thanks, that would be great.

Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 06:05:20PM +1100, Justin Mclean wrote:
> What the process for generating the release artefacts?

Since we won't be releasing binaries this time around, the only
artifacts are: 1) the source tgz files, 2) the ascii signature files,
and 3) the SHA checksums.

Or maybe you are asking about the binaries which contain LGPL code?  My
apologies if I misunderstood the question.

Chris

Re: incubator-mynewt-newt git commit: LICENSE, NOTICE, and KEYS files for newt.

[Christopher Collins]

On Fri, Feb 19, 2016 at 06:18:19PM +1100, Justin Mclean wrote:
> HI,
> 
> > Since we won't be releasing binaries this time around, the only
> > artifacts are: 1) the source tgz files
> 
> Perhaps a silly question. But how are those tgz files generated?

Oh, I see.  No, not silly at all.  I used the following command to create
the tgz files (executed from the repository base directory):

git archive --format tgz --output ~/tmp/rel/bin/larva/larva-0.8.0-b1.tgz 
--prefix larva-0.8.0-b1/ mynewt_0_8_0_b1_tag

With the obvious substitutions for tadpole and newt.

Chris

Re: LICENSE and NOTICE contents / headers

[Christopher Collins]

Thanks a lot for looking at this, Justin.

On Fri, Feb 19, 2016 at 06:16:22PM +1100, Justin Mclean wrote:
> Newt repo still has:
> ./.git/hooks/pre-rebase.sample

I believe this file is not actually in the repository.  When a git repo
is cloned, the .git directory is populated with the contents of the
user's template directory, as specified by the "[init] templatedir" git
setting.  On my machine, the default templatedir is
/usr/local/share/git-core/templates.

> And a number of files (mostly .go files) still have "Copyright 2015
> Runtime Inc.”

Good catch.  I will fix this.

> Larval has a number of files with copyright that is this presumably licensed 
> under something, for example:
> Copyright (C) 2004  Kustaa Nyholm
> Copyright (C) 2010  CJlano
> Copyright (C) 2011  Petteri Aimonen
> Copyright (c) 2004,2012 Kustaa Nyholm / SpareTimeLabs
> Copyright (c) 2007, 2008, 2009, 2010 Dado Sutter and Bogdan Marinescu
> Copyright (c) 2007, 2008, 2009, 2010, 2011 Dado Sutter and Bogdan Marinescu
> Copyright (c) 2012 Petteri Aimonen 
> Copyright (c) 2015, Nordic Semiconductor ASA
> 
> See my previous email of this for all the licenses and files.
> 
> As does tadpole, for instance:
>  * Copyright (c) 1982, 1986, 1988, 1991, 1993
>  * Copyright (c) 1991, 1993
>  * Copyright (c) 1995-2001 Kungliga Tekniska Högskolan
>  * Copyright (c) 1999-2009 KEIL, 2009-2013 ARM Germany GmbH
> # Copyright (c) 2006, 2008 Junio C Hamano

I believe these are all accounted for in the larva LICENSE file. Some of
these copyrighted files are from the following sources:

tinyprintf (BSD)
eLua (MIT)
queue.h (BSD)

The rest of them are from files which contain the Apache license text at
the top of the file.  In these files, we retained the original copyright
notice below the Apache license.  Do you think this is problematic?

That said, you are entirely correct about tadpole - I forgot to add the
necessary pointers to its LICENSE file.

Thanks,
Chris

Re: LICENSE and NOTICE contents / headers

[Christopher Collins]

On Sat, Feb 20, 2016 at 09:05:58AM +1100, Justin Mclean wrote:
> > The rest of them are from files which contain the Apache license text at
> > the top of the file.  In these files, we retained the original copyright
> > notice below the Apache license.  Do you think this is problematic?
> 
> That's incorrect.while you can license the whole as Apache, each file
> needs to keep their original header and not have two headers.
> 
> The only case where this might come up is if we’re heavily modified
> the file in question.

OK, thanks for the heads up.  Is this something we should deal with
immediately, or can we fix it between now and our next release?

> Other than the above issue is it in a state that I can take another look?

Yes, I believe so.  Thanks again!

Chris

Re: LICENSE and NOTICE contents / headers

[Christopher Collins]

On Sat, Feb 20, 2016 at 09:40:40AM +1100, Justin Mclean wrote:
> I think the LICENSE is still missing a number of things, do you look
> at my email when I went through the repos and listed what was
> contained in them? I know things have changed a little in
> tadpole/larval repos but everythink that is bundled needs to be
> accounted for. e.g. Baselibc for instance has several files in it that
> are licensed under other terms. [1]

OK, I believe I have removed the Apache license from all the
"otherwise-licensed" files, and added corresponding pointers to the
larva LICENSE file.  Everything from your earlier email ("Larva content
review for license" sent on 2016-02-04) should now be accounted for.  I
did see your note about checking the FatFS license, but larva does not
contain any FatFS files, so I did not make any mention of it in the
LICENSE file.  Larva's LICENSE file has become quite a monster, I'm
afraid.

> If it helps I can go through and list out what I think they are missing.
> 
> Also I thought we were removing anything with the "MCD-ST Liberty SW
> License”? (It’s not a comparable licence) See larval
> ./hw/mcu/stm/stm32f4xx/include/mcu/system_stm32f4xx.h

I have replaced this file with a more recent version from STM. The new
version has a BSD license.

There is one nagging issue, however.  Larva includes some files from
CMSIS-CORE which lack any license information.  In subsequent releases,
ARM added the BSD license text to these files.  However, we have made
modifications to the original files, so incorporating newer versions of
these files into Larva won't be trivial.  We will fix this issue, but I
am hoping we can take care of this between the first and second release.
In the meantime, I have added a note about this to the larva LICENSE
file.

If you could take another look at where we are, it would be much
appreciated.  Once again, thanks for your unflagging patience!

Chris

Re: [DISCUSS] Release Apache Mynewt 0.8.0-incubating-b2

[Christopher Collins]

Hi Justin,

On Sat, Mar 19, 2016 at 08:49:50AM +1100, Justin Mclean wrote:
> Hi,
> 
> I took a look at the binaries and have a question. What exactly is in
> the newt binary? Is any extra 3rd party code bundled into that binary
> that is not in the source release? And if so how is that licensed?

The binary does not contain any code that isn't in the source release.
In this release, newt uses the Go "vendoring" feature, which allows
all the library dependencies to be shipped alongside the application
code.  This is a change from the b1 release, which relied on the user to
download all the dependencies using the go command line tool (b1 was
released this way due to the LGPL yaml dependency that has since been
removed).

Thanks,
Chris

[VOTE][CANCEL] Release Mynewt 0.8.0-incubating-b2

[Christopher Collins]

Hello all,

Voting for the release of 0.8.0-incubating-b2 has been cancelled due to
a bug that was discovered in the bletiny application [*].  A new vote
will be called in the near future.

I apologize for the confusion.  The next release candidate will be
clearly labelled with a candidate number (rc3).

Thanks,
Chris

[*] The bletiny application exhibits a stack overflow while logging
incoming BLE messages.

[DISCUSS] Release Apache Mynewt 0.8.0-b2-incubating-rc3

[Christopher Collins]

Hello all,

This thread is for any and all discussion regarding the release of
Apache Mynewt 0.8.0-b2-incubating-rc3.  All feedback is welcome.

Thanks,
Chris

[VOTE] Release Apache Mynewt 0.8.0-b2-incubating-rc3

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of
Apache Mynewt 0.8.0-b2-incubating-rc3.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications.  Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes

For information about bugs fixed in beta 2, please view the Issue
Tracker Summary: 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319221=12334805

This is the first source release of the "New" Newt.  More information on
the genesis of Newt, and a description can be found at:
http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C56E21C13.9050303%40apache.org%3E

The full mynewt test suite for this release was executed via "newt test
all" with no failures.  This testing was performed on the following
platforms:
* OS X 10.10.5
* Linux 3.19.0 (Ubuntu 14.10)

In addition, all other currently-supported platforms were smoke tested.

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/

The commits under consideration are as follows:

blinky (formerly tadpole):
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky.git
commit: 82f09fa2cc4c67fb30165ef6be0291aa76e6213e

core (formerly larva):
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core.git
commit: 2bec33d4dba12094f32ca560410b8671c0ce1d22

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt.git
commit: 58053579210a3541916d3e799b22cc81bc473e47

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/apache-newt-bin-linux-0.8.0-b2-incubating.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/apache-newt-bin-osx-0.8.0-b2-incubating.tgz

The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers,
please feel free to try out the release candidate and provide your
votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

Re: [VOTE] Release Apache Mynewt 0.8.0-incubating-b2

[Christopher Collins]

On Thu, Mar 17, 2016 at 10:02:01AM +1100, Justin Mclean wrote:
> > Blinky is a bit unusual in that it is not meant to be compilable nor
> > testable on its own
> 
> Any reason it has unittest as a target then? (as shown by mynewt info)

Well... it's an implementation detail that leaked out :).  The unittest
target is used internally by the "newt test" command.  In the latest
release candidate, the unittest target is hidden from the user in the
output of "newt info" and "newt target show".  The plan is to get rid of
this hack in the next release; then the unittest target won't exist at
all.

> > Are subdirectories generally not considered to be included in such a
> > reference?
> 
> All good I missed that subdirectory reference in LICENSE, so nothing needs to 
> be done there.

Great, thanks!

Chris

[VOTE] Release Apache Mynewt 0.8.0-incubating-b2

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of
Apache Mynewt 0.8.0-incubating-b2.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications.  Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes

For information about bugs fixed in beta 2, please view the Issue
Tracker Summary: 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319221=12334805

This is the first source release of the "New" Newt.  More information on
the genesis of Newt, and a description can be found at:
http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C56E21C13.9050303%40apache.org%3E

The full mynewt test suite for this release was executed via "newt test
all" with no failures.  This testing was performed on the following
platforms:
* OS X 10.10.5
* Linux 3.19.0 (Ubuntu 14.10)

In addition, all other currently-supported platforms were smoke tested.

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-incubating-b2/

The commits under consideration are as follows:

blinky (formerly tadpole):
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky.git
commit: 82f09fa2cc4c67fb30165ef6be0291aa76e6213e

core (formerly larva):
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core.git
commit: 62df74fab22302d70a12caff0082d1adb6b5b3b3

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt.git
commit: 58053579210a3541916d3e799b22cc81bc473e47

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-incubating-b2/apache-newt-bin-linux-0.8.0-incubating-b2.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-incubating-b2/apache-newt-bin-osx-0.8.0-incubating-b2.tgz

The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers,
please feel free to try out the release candidate and provide your
votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

[VOTE][CANCEL] Release Mynewt 0.8.0-incubating-b2

[Christopher Collins]

Hello all,

Voting for the release of 0.8.0-incubating-b2 has been cancelled due to
some compliance issues.  A new vote will be called in the near future.

Thanks,
Chris

Re: FYI: bletiny is over the nrf51 code size limit

[Christopher Collins]

The culprit is the security code that was recently added to the host.
This new code caused ~10kB of mbedtls code to get pulled in to the
image.

Since security support is incomplete, I have disabled the security
manager by default.  The latest code in the develop branch should fit on
the nRF51 once again.

This is obviously not a long term solution.  I think we will need to
look for ways to reduce the nimble host code size soon.

Chris

On Mon, Mar 28, 2016 at 09:52:09AM -0700, will sanfilippo wrote:
> Hello:
> 
> This is just an FYI: the nrf51 bletiny build is over the code size limit, so 
> if you pull develop and build this project it will not link. This will get 
> fixed today (hopefully).

Re: choosing between FCB and NFFS

[Christopher Collins]

That mostly sounds good to me, though I agree that the need to duplicate
app code is not ideal.

Here is an alternative idea:
* Both fs and fcb export a suitable API (e.g., "bootapi").
* By default, apps depend on nffs.
* If a particular feature is set in the target, the app depends on
  fcb instead.

Something like:

pkg.deps.base:
- libs/os
- sys/log
- libs/newtmgr
- libs/console/full
- libs/shell

pkg.deps: [pkg.deps.base, fs/nffs]
pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]

I may have gotten the yaml syntax wrong, but I think the concept works.

Chris

On Thu, Apr 07, 2016 at 09:27:37AM -0700, marko kiiskila wrote:
> Hi,
> 
> at the moment bootloader always uses NFFS. App can tell bootloader
> to pick specific image for next boot by creating a file. Bootloader also
> keeps track of image update by writing to another file.
> 
> I want to use sys/config and FCB (flash circular buffer) for
> this instead. Reason for this change is the impact on code size.
> 
> My current idea revolves around creating new versions of apps;
> one version uses NFFS and second version uses FCB. Both would
> go for the same flash map entry for location of their storage.
> 
> And you would pick your target to use either NFFS or FCB depending on
> what platform you’re building for.
> 
> Not too keen on this though, as it would require a bit of code
> duplication (main.c on boot/slinky/blinky).
> 
> Better suggestions?

Re: choosing between FCB and NFFS

[Christopher Collins]

Upon reflection, this idea has problems of its own.  The app still
needs to initialize the specific flash storage package being used.  So,
sorry for posting before thinking!  The idea could be salvaged with the
use of some #if directives in the app code, but I am not sure this is
any better than duplicating the entire app.

I think the problem you described won't be applicable to most apps.  The
example apps are affected by this because they are meant to be as
generic as possible.

Chris

On Thu, Apr 07, 2016 at 11:03:07AM -0700, Christopher Collins wrote:
> That mostly sounds good to me, though I agree that the need to duplicate
> app code is not ideal.
> 
> Here is an alternative idea:
> * Both fs and fcb export a suitable API (e.g., "bootapi").
> * By default, apps depend on nffs.
> * If a particular feature is set in the target, the app depends on
>   fcb instead.
> 
> Something like:
> 
> pkg.deps.base:
> - libs/os
> - sys/log
> - libs/newtmgr
> - libs/console/full
> - libs/shell
> 
> pkg.deps: [pkg.deps.base, fs/nffs]
> pkg.deps.FCB_BOOT.OVERWRITE: [pkg.deps.base, sys/config, sys/fcb]
> 
> I may have gotten the yaml syntax wrong, but I think the concept works.
> 
> Chris

net/nimble/host code size reduction

[Christopher Collins]

Hello all,

Yesterday I made a fairly large commit to the net/nimble/host package in the
core repository (the host-side of the BLE stack).  I wanted to provide the
community with an explanation of these changes and related changes planned for
the future.  This email will probably be long and detailed, so don't
feel bad about skimming it for anything that you find interesting.  That
said, I would love to hear any and all feedback.

First, here is the relevant commit:
Repo: http://git-wip-us.apache.org/repos/asf/incubator-mynewt-core
Branch: develop
Commit: 2fef21d6a1f9d3d83b88ecf11b825a6e5a354a47

Motivation for this change: the code size of net/nimble/host had grown
quite large over the last few months.  During development of the host,
most focus was put on correctness and maintainability, while speed and
size considerations were placed in the background.  Now that the host is
reasonably complete, it is a good time to start optimizing, with the
hope that the unit tests will protect against any regressions.

The commit consists of two major changes:

1. Use packed structs for over-the-air ATT and L2CAP commands.
2. Compile-out most assertions unless BLE_HS_DEBUG is defined.

Regarding 1:

Prior to this change, the host code was manually
marshalling fields between buffers and properly-aligned structs.  This
had the benefit of being defined behavior according to the C standard,
but at the cost of increased code size.  By using the gcc __packed__
attribute, the compiler is able to take advantage of the Cortex-M's
ability to perform unaligned accesses, resulting in a substantial code
savings.

Regarding 2:

The host code makes liberal use of the assert() macro.
Each invocation of the assert() macro was adding a surprising amount of
code to the resulting binary.  The size increase is not caused by
strings being embedded in the binary; the baselibc assert() does not include
the expression text in its expansion, and gcc collapses duplicate
filename strings.  The size increase is just caused by the comparison
and branch implicit in the assert macro.

The asserts in the host code are plentiful and conservative.  They
caused an unwelcome increase in code size, but I didn't want to remove
them entirely because they are valuable during regression testing.  The
standard assert() macro compiles to nothing if the NDEBUG symbol is
defined, but I wanted more control over which asserts actually get
disabled.  As a compromise, I added a second type of assertion that only
gets compiled in if the BLE_HS_DEBUG symbol is defined.  

I think the concept of multiple assert levels will be useful to other
packages and that it should be added to the core OS.  For this reason, I
intentionally made the new assert macro names ugly, as I hope to replace
them with something that isn't host-specific.

Here are the two macros I added:

#if BLE_HS_DEBUG
#define BLE_HS_DBG_ASSERT(x) assert(x)
#define BLE_HS_DBG_ASSERT_EVAL(x) assert(x)
#else
#define BLE_HS_DBG_ASSERT(x)
#define BLE_HS_DBG_ASSERT_EVAL(x) ((void)(x))
#endif


BLE_HS_DBG_ASSERT(x) is self-explanatory; it asserts if the necessary
debug symbol is defined.

BLE_HS_DBG_ASSERT_EVAL(x) is less obvious.  If the BLE_HS_DEBUG symbol
is defined, this macro has the same effect as the first one.  If the
symbol is not defined, this macro evaluates its argument and throws away
the result.  The reason this macro exists is to prevent gcc from raising
"variable set but not used" warnings in code like the following:

rc = operation_that_should_never_fail();
BLE_HS_DBG_ASSERT_EVAL(rc == SUCCESS);

Note that the BLE_HS_DBG_ASSERT_EVAL() macro differs from the standard
assert() macro in this respect.  The standard assert() macro does not
evalute its argument if NDEBUG is defined, which allows you to write
code like the following:

assert(expensive_sanity_check() == SUCCESS);

Future changes:

There are a number of future changes planned for further reducing the
host code size.

* Pack the HCI command and event structs.
* Rework (or remove / replace) the generic FSM implementation; the
  code has outgrown this facility.

If you have made this far and have any suggestions of your own, they
would be much appreciated, so please feel free to share them.

Thanks,
Chris

Rename newtmgr protocol

[Christopher Collins]

Hello all,

There are two things called "newtmgr":

1. A simple command-response protocol used for interfacing with mynewt
   devices (the newtmgr server code is at core:libs/newtmgr).
2. A CLI tool which communicates with mynewt devics via the newtmgr protocol
   (newt:newtmgr).

I have had a hard time discussing both of these entities due to the
overloaded name.  In the interest of preventing an endless series of
"who's on first" fiascos, I propose we rename the name of the protocol.

I suggest we rename the protocol to: nmp.  I don't really care about the
name, though, as long as it is unique :).

Alternatively, the tool could be renamed, but I thought newtmgr sounds
more like a tool than a protocol.

Thanks,
Chris

Re: Timestamp for logs

[Christopher Collins]

On Thu, Apr 07, 2016 at 10:33:26PM -0700, Vipul Rahane wrote:
> Hello,
> 
> I agree with your statement. We do not know on what kind of devices
> Mynewt would be ported to. Sleepy devices which are meant to work for
> 20 years running on a single coin cell battery will rollover the time
> stamp in 2038. We want to be able to take care of such a situation.
> While there are other solutions which can be implemented that are more
> efficient, keeping it as simple as possible is better from an end to
> end perspective as these logs would be used by applications to
> understand the state of the devices. 
> 
> I was planning on storing microseconds because the OS currently
> populates OS time in seconds and microseconds. For microseconds we do
> require 32 bits. I agree for milliseconds 16 bits are enough but
> higher resolution is always better. 

I think 12 bytes of time is more than necessary.  A few notes:

* A single 64-bit microsecond counter allows for 584942 years before
  rollover.

* A single 32-bit second counter won't actually roll over until 2106
  (the 2038 issue only applies to signed 32-bit timestamps).

If we want microsecond precision, I would just go with a single 64-bit
counter.  Otherwise, 32 bits of seconds is sufficient in my opinion.

Chris

> 
> Regards,
> Vipul Rahane
> 
> > On Apr 7, 2016, at 10:02 PM, Justin Mclean  wrote:
> > 
> > HI, 
> > 
> >> I am going to change the log structure so that it stores both(UTC 
> >> timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
> > 
> > NO objection, but just out of interest why 64 bit for seconds (when 32 bit 
> > of seconds = 60+ years and good until 2038) and 32 bits for milliseconds 
> > when 16 bits will do? See also [1]
> > 
> > Thanks,
> > Justin
> > 
> > 1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions
> > 
> 

Re: Using third-party Apache code

[Christopher Collins]

(I meant to send this several days ago, but I just noticed it in my
drafts folder.  Thanks for answering my questions!)

Hi Justin,

On Fri, Apr 08, 2016 at 08:08:56AM +1000, Justin Mclean wrote:
[...]

> IANAL etc etc
> 
> What’s the software in question your thinking of bundling?

I have pulled in some BLE security manager code from the Zephyr
project (www.zephyrproject.org).  It is released under the Apache 2.0
license, but not by the ASF.

Thanks,
Chris

Re: net/nimble/host code size reduction

[Christopher Collins]

On Tue, Apr 05, 2016 at 03:10:40PM -0700, aditi hilbert wrote:
> So how much was the size reduction?

Good question :).  The host size was reduced

from: 48526 bytes
  to: 42257 bytes

I should note that I am building with -DLOG_LEVEL=2, which also reduces
the code size somewhat.  This option sets the log level to info rather
than the default (debug).

Chris

Re: [VOTE]: Committer status for Paul Dietrich and Neel Natu

[Christopher Collins]

On Tue, Apr 05, 2016 at 10:39:01PM -0700, aditi hilbert wrote:
> This is a call to vote on conferring committer status to the following
> two contributors. They have submitted several patches and entire
> modules via pull requests on Mynewt github mirrors on a variety of
> topics. 
> 
> [+1] Paul Dietrich
> [+1] Neel Natu
> 
> They have submitted the ICLA already. Voting will be open for 72 hours. 

Thanks,
Chris

[DISCUSS] Release Apache Mynewt 0.8.0-incubating-b2

[Christopher Collins]

Hi all,

This thread is for any and all discussion regarding the release of
Apache Mynewt 0.8.0-incubating-b2.  All feedback is welcome.

Thanks,
Chris

[RESULT][VOTE] Release Apache Mynewt 0.8.0-b2-incubating-rc3

[Christopher Collins]

Hello all,

Voting for Apache Mynewt 0.8.0-b2-incubating-rc3 is now closed.  The
release has passed this step of the process.  The vote breakdown is as
follows:

+1 Christopher Collins
+1 Sterling Hughes
+1 Justin Mclean
+1 Will san Filippo

Total: +4

We can now call a vote on the general@incubator list.

Thank you to all who voted.

Thanks,
Chris

Re: Nimble HCI

[Christopher Collins]

On Wed, Mar 23, 2016 at 05:08:37PM -0700, will sanfilippo wrote:
[...]

To summarize my understanding of your proposal (please let me know if I
got anything wrong!):

1. Create several independent HCI packages in the net/nimble directory.

>   net/nimble/hci_spi
>   net/nimble/hci_combined
>   net/nimble/hci_uart

2. Each HCI package exports the "ble_hci" API in its pkg.yml.

# net/nimble/hci_spi/pkg.yml
pkg.apis: ble_hci

3. Both net/nimble/host and net/nimble/controller require the "ble"hci"
API in their pkg.yml files.

# net/nimble/host/pkg.yml
pkg.req_apis: ble_hci

4. The app specifies the hard dependency on a specific HCI package.

# apps/myapp/pkg.yml
pkg.deps: @apache-mynewt-core/net/nimble/hci_spi

I like it.  There is one annoyance, though: it is too bad that the app
has to depend on a specific HCI package.  Using bletiny as an example,
it would be nice if this app could be HCI-transport-agnostic.  However,
some piece of code has to initialize the specific HCI package being
used, and it makes sense that this would happen in the app, so I am not
sure this is an issue.  It might be worthwhile to think a bit about how
we might solve this issue cleanly.  The only solutions I can think of
add too much complexity to justify, in my opinion.

Thanks,
Chris

Apache Mynewt release process

[Christopher Collins]

Hello all,

I have created a page on the Mynewt wiki detailing the release process:

https://cwiki.apache.org/confluence/display/MYNEWT/Release+Process

If you have time, please take a look and make corrections or post
comments to the list.

Thanks,
Chris

Re: installing the Newt

[Christopher Collins]

I have committed the changes to the develop branch.  Now all go imports
use the vanity domain.

Unfortunately, this is going cause a bit of pain for the current users
because the newt sources are now in the wrong directory.  To correct
this problem, you will need to move the old newt path to the new one:

$ mkdir -p "$GOPATH"/src/mynewt.apache.org &&
  mv $GOPATH/src/git-wip-us.apache.org/repos/asf/incubator-mynewt-newt 
"$GOPATH"/src/mynewt.apache.org/newt

If you plan on building older versions of newt, you should link the path
rather than move it:

$ mkdir -p "$GOPATH"/src/mynewt.apache.org &&
  ln -s $GOPATH/src/git-wip-us.apache.org/repos/asf/incubator-mynewt-newt 
"$GOPATH"/src/mynewt.apache.org/newt

Sorry for the hassle.  Yes, we definitely need to include convenience
binaries in the next release :).

Chris

On Wed, Mar 02, 2016 at 11:21:49PM -0800, Christopher Collins wrote:
> Nice!  That seems to work.  I guess I completely glossed over those
> vanity URL emails.  I will make the necessary changes to the develop
> branch.  Unfortunately, I'm afraid it is too late to fix 0.8.0-b1.
> 
> Chris
> 
> On Wed, Mar 02, 2016 at 10:40:53PM -0800, Sterling Hughes wrote:
> > 
> > >
> > > When pointed at the apache server, on the other hand, "go get" seems to
> > > require the ".git" suffix.  An I mentioned earlier, this results in the
> > > creation of a directory that also has the ".git" suffix.
> > >
> > > The problem is: the behavior of "git clone" is in conflict with the
> > > behavior of "go get", at least with regards to the apache git server.
> > > At one point the installation documentation was accurate, but it seems
> > > we have since opted for git-friendliness rather than
> > > go-get-friendliness.
> > >
> > > We will need to find a simpler workaround.  In the meantime, we should
> > > at least update the documentation.  Also, soon newt binaries will be
> > > available for download which help to alleviate problems with go.
> > >
> > 
> > Weren't Todd & Aditi making "mynewt.apache.org" a valid import path, 
> > that would point to the proper Apache git repository?   I thought this 
> > would solve that problem?
> > 
> > Sterling

Re: Master newt does not build released software

[Christopher Collins]

I agree that that is a compatibility break, but that change was made at
the same time as a host of other backwards-incompatible changes
(0.8.0-b2).  I think this particular problem is a bit more complicated,
so strap yourself in for some mind-numbing post-mortem analysis.

The root of this issue is that the arduino repository.yml file
incorrectly pointed to the develop branch rather than an appropriate tag
on the master branch.  Under these conditions, everything worked for a
while, because the arduino bsp's pkg.yml file on the develop branch does
specify the correct package type ("bsp").  Eventually, a change was made
to the arduino develop branch which was incompatible with the core
master branch [*], which is not surprising.  This incompatible change
exposed the bug in the arduino repository.yml file.  As a consequence,
the arduino repository.yml file was modified to point to the master
branch.  The mistake here was in assuming the repository.yml change
would be sufficient for fixing the arduino build issues.

In my opinion, temporarily removing the package type restrictions is not
the right thing to do here.  We would need to re-release newt for that
fix to have any effect.  Instead, I believe we should create a new tag
on the arduino repo with the pkg.yml fix.  I can make this change and
ensure arduino apps build properly.  However, I don't have an arduino
with me, so I can't actually test image upload or debug.

Chris


[*] - C files were including HAL header files that only existed in the
develop branch of the core repo.


On Sat, Apr 02, 2016 at 12:13:45PM -0700, Sterling Hughes wrote:
> Hey,
> 
> Try:
> 
> $ newt target show
> targets/arduino_boot
>  app=@apache-mynewt-core/apps/boot
>  bsp=@mynewt_arduino_zero/hw/bsp/arduino_zero
>  build_profile=optimized
>  features=arduino_zero_pro
> targets/my_blinky_sim
>  app=apps/blinky
>  bsp=@apache-mynewt-core/hw/bsp/native
>  build_profile=debug
> $ newt build arduino_boot
> Error: bsp package (hw/bsp/arduino_zero) is not of type bsp; type is: lib
> $
> 
> I understand that we introduced package types into the new release. 
> What I'm not sure of is why we've broken compatibility here?
> 
> Yes, we're beta, and compatibility breaks are allowed, but:
> 
> A- We should call them out on dev@ _PRIOR_ to breaking compatibility, so 
> things like the docs can be changed.
> 
> B- In this case, was a compatibility break really necessary?  Couldn't 
> we have just accepted the package type if it was lib-- at least for a 
> couple of releases?
> 
> Sterling

Re: Master newt does not build released software

[Christopher Collins]

On Sat, Apr 02, 2016 at 02:00:55PM -0700, Sterling Hughes wrote:
> When making changes to a platform as popular as Arduino Zero, that will 
> likely break compat, can people please TEST to make sure the release 
> version still works.

Also, I imagine that compatibility-breaking changes like these will be
tied to changes in version numbers.  That will help to prevent this
particular issue from occurring again.

(btw, I realize I misread your comment about deprecating the old pkg.yml
rather than removing support immediately... sorry about that!).

Chris

Re: Testing rhe libs/shell

[Christopher Collins]

On Mon, Apr 04, 2016 at 11:55:54PM +0100, Nges B wrote:
> in a new created project , the newt test all fails the libs/json .
> the jlibs/json was not corrected on that repository.
> Thanks

It looks like this particular fix was already pushed to the develop
branch on 2016-03-21 (e8a365bd766ac2963ca9c757175889caf5f12461).

The reason you still see the test failure in a newly created project is
that newt's default behavior is to use the mynewt_0_8_0_b2_tag, not the
develop branch.  The json unit test fix will be visible by default in
the next release of the core repository.

If you want to try out the fix, you can configure your project to use
the develop branch of core.  Do this by changing your project.yml file
to use core 0.0.0 as follows:

FROM:

repository.apache-mynewt-core:
type: github
vers: 0-latest
user: apache
repo: incubator-mynewt-core

TO:

repository.apache-mynewt-core:
type: github
vers: 0.0.0
user: apache
repo: incubator-mynewt-core

Note that the develop branch my be broken at any time, so it is only
recommended for testing and development.  Also, it is likely to be
incompatible with non-develop-branches of other repositories (e.g., newt
tool and arduino repo).

Thanks for submitting the pull request.  As this particular fix was
already implemented, I will close the request, but please submit more if
you find other issues

Chris

Re: First draft of Coding standards in develop branch

[Christopher Collins]

I am not sure I would put it that way.  The POSIX headers are part of
the "implementation," so they are permitted to use the reserved
namespace.  Since mynewt code needs to build in sim (POSIX) and also
freestanding implementations, it is probably best to respect both
standards in our code.

Chris

On Tue, Apr 26, 2016 at 09:31:55AM -0700, Sterling Hughes wrote:
> 
> 
> On 4/26/16 9:22 AM, Christopher Collins wrote:
> > On Tue, Apr 26, 2016 at 09:06:37AM -0700, Sterling Hughes wrote:
> >> I think we do need a naming convention here - I'm fine adopting H_*.
> >> I'll point out that almost every POSIX or LIBC header I've ever seen
> >> uses underscore, and I believe this is only reserved in C++ -- that
> >> said, we need to be friendly to C++, and as you point out, we should
> >> follow a defined behavior.
> >
> > These identifiers are reserved in C as well.  From 7.1.3 of n1570:
> >
> >  All identifiers that begin with an underscore and either an
> >  uppercase letter or another underscore are always reserved for for
> >  any use.
> >
> > (http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf)
> >
> 
> Hrm, you're right.  There are a lot of POSIX headers that have undefined 
> behavior. :-)
> 
> Sterling

Re: First draft of Coding standards in develop branch

[Christopher Collins]

On Tue, Apr 26, 2016 at 09:06:37AM -0700, Sterling Hughes wrote:
> I think we do need a naming convention here - I'm fine adopting H_*. 
> I'll point out that almost every POSIX or LIBC header I've ever seen 
> uses underscore, and I believe this is only reserved in C++ -- that 
> said, we need to be friendly to C++, and as you point out, we should 
> follow a defined behavior.

These identifiers are reserved in C as well.  From 7.1.3 of n1570:

All identifiers that begin with an underscore and either an
uppercase letter or another underscore are always reserved for for
any use.

(http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf)

Chris

Re: [VOTE] Release Apache Mynewt 0.8.0-incubating-rc1

[Christopher Collins]

On Thu, Apr 28, 2016 at 01:09:45PM -0700, marko kiiskila wrote:
> The vote is open for at least 72 hours and passes if a majority of at
> least three +1 PPMC votes are cast.
> 
> [X] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because...

Chris

Re: First draft of Coding standards in develop branch

[Christopher Collins]

On Sun, Apr 24, 2016 at 10:08:09AM -0700, Sterling Hughes wrote:
> Hi,
> 
> As we start to bring on new contributors, and operate as a project, its 
> increasingly important that we document and agree upon coding standards. 
>   I think we've done a good job of maintaining this consistency 
> informally, but, now we need to vote and agree on project standards.
> 
> I've taken a first stab at this and committed it to the develop branch, 
> folks can see it here:
> 
> https://github.com/apache/incubator-mynewt-core/blob/develop/CODING_STANDARDS.md

Thanks for putting this together Sterling.  I think it looks great.  My
opinion is that a coding standards document should not be overly
prescriptive.  Everyone has his own set of coding pet peeves; I suggest
we try to keep those out of this document and keep it as short as
possible.  Otherwise, people won't adhere to the document, or they will
just hate writing code and they won't contribute as much.

For me, the important rules are:
* Maximum line length
* Brace placement
* Typedefs
* All-caps macros
* Compiler extensions (e.g., packed structs).

The first three are already captured; I think the others should be
addressed.  I think macros should always be in all-caps for reasons that
everyone is probably familiar with. Unfortunately, I don't have a good
rule for when extensions are acceptable.

I would also like to see a note about when it is OK to stray from the
conventions.  There will be times (rarely) when adhering to the
standards document just doesn't make sense.  "Zero-tolerance" rules
always seem to pave the road to hell :).

Finally, there is one point in particular that I wanted to address:
include guards in header files.  From the document:

* ```#ifdef``` aliasing, shall be in the following format, where
the package name is "os" and the file name is "callout.h":

```no-highlight
#ifndef _OS_CALLOUT_H
#define _OS_CALLOUT_H

I am not sure I like this rule for the following two reasons:
* A lot of the code base doesn't follow this particular naming
  convention.
* Identifiers starting with underscore and capital letter are
  reserved to the implementation, so technically this opens the door
  to undefined behavior.  

A leading capital E is also reserved by POSIX (e.g., EINVAL).  The
naming convention I use is:

H_CALLOUT_

I would not consider this something to worry about, and I don't think we
need to include a specific naming convention in the document.  However,
insofar as we prescribe a naming convention, it should be one which
avoids undefined behavior.

Thanks,
Chris

Re: Correct way to specify custom UUID's

[Christopher Collins]

On Thu, May 19, 2016 at 12:57:53PM -0700, James Howarth wrote:
> Hi Chris,
> 
> A)  Thanks for the style tips.  I always appreciate those.
> 
> B)  The device seems to be stuck in trying to register attributes loop,
> similar to what had happened when I didn't have enough attributes
> supported.  Because I have just changed the UUID of the attribute I had
> working before, I don't expect this to be the same problem.

Hmm, I am not sure.  I think I would need to see more code.  Could you
please send the full service definition?

Thanks,
Chris

Re: [VOTE] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

On Wed, May 18, 2016 at 11:26:50PM -0700, Christopher Collins wrote:
> [X] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because...

+1 (binding)

Thanks,
Chris

Re: [DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

All,

It is certainly not a big problem, but it is best if discussion about
the release goes in the [DISCUSS] thread rather than the [VOTE] thread.
Reserving the [VOTE] thread for votes makes it easy for lazy people such
as myself to produce a tally when the vote closes.

Please don't interpet this petty admonition as a request to stop the
conversation!

Thanks,
Chris

On Fri, May 20, 2016 at 09:05:40PM +0200, Kevin Townsend wrote:
> Hi Will,
> 
> Great, thanks.  I don't expect BLE emulation like the HAL layer, but it 
> is nice to be able to build to test shell commands and some basic logic 
> like DSP algorithms on the orientation sensors, etc.
> 
> K.
> 
> On 20/05/16 20:03, will sanfilippo wrote:
> > All:
> >
> > I committed a fix for the native ble build. Basically had to add a bunch of 
> > stubs to the phy and also include the xcvr.h header file. If there are any 
> > other issues please let me know.
> >
> > Will
> >
> >> On May 19, 2016, at 7:38 AM, Kevin Townsend  wrote:
> >>
> >> Hi Chris,
> >>
> >> Sorry this may be an old issue then so feel free to ignore. I understand
> >> that native emulation of the BLE stack doesn't currently work and there are
> >> other priorities, but copying that one file at least allows me to build a
> >> basic project to test some custom shell commands and make sure the command
> >> parsing works as expected etc.
> >>
> >> I'll wait for some feedback though to know what he plans are around nimble
> >> plus native mode. It was a 30 second issue to fix with my artificial test 
> >> case
> >> today but maybe there are other issues I'm not aware of down the road since
> >> I haven't started digging into nimble yet.
> >>
> >> K.
> 

Re: Respond to a write characteristic

[Christopher Collins]

Hi James,

On Mon, May 23, 2016 at 02:24:15PM -0700, James Howarth wrote:
> Hi,
> 
> I am looking to issue a response to a write characteristic, and seem to be
> just getting a null response.
> 
[...]
> static int
> gatt_svr_p_command_cb(uint16_t conn_handle, uint16_t attr_handle, uint8_t
> op,
>  union ble_gatt_access_ctxt *ctxt, void *arg)
> {
> static uint8_t command_int = 0x05;
> ctxt->chr_access.data = _int;
> ctxt->chr_access.len = 1;
> 
> return 0;
> }

That should do the trick. As long as the function returns 0, the write
response should get sent.

What do you mean by null response?

Chris

Re: Respond to a write characteristic

[Christopher Collins]

On Mon, May 23, 2016 at 09:22:42PM -0700, James Howarth wrote:
> Hi Chris,
> 
> Looking at the LightBlue app on my iPhone, it tells me the response is 'null'.

Hmm... I am not sure.  A write response does not contain any data of its
own; all write responses are alike.  Is it possible lightblue displays
that due to an attempt to read a characteristic (as opposed to write)?

Unfortunately I am not equipped to test this out at the moment, but I'll
take a look tomorrow.  That said, I don't believe there is anything
wrong with your code.

Chris

Re: [DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

On Thu, May 19, 2016 at 11:30:05AM +0200, Kevin Townsend wrote:
> I'm running in 'develop' which may not be the right branch, but 
> switching a bare bones BLE project to 'native' as a BSP generates this 
> error:
> 
> $ newt build bleuart
> Building target targets/bleuart
> Compiling ble_ll_adv.c
> Error: ble_ll_adv.c:24:22: fatal error: ble/xcvr.h: No such file or 
> directory
> compilation terminated.
> 
> Copying the header from here 
> (https://github.com/apache/incubator-mynewt-core/tree/develop/net/nimble/drivers/nrf51/include/ble)
>  
> solves this but the file should probably exist in 
> net/nimble/drivers/native as well, right?
> 
> If 'develop' corresponds to 0.9.0-rc1 I can submit a pull request if 
> develop is the right branch for this?

Hi Kevin,

The develop branch should be identical to the release candidate, so any
issues you are seeing also exist in 0.9.0-rc1.  Just for my own
clarification, is the issue you describe new to 0.9.0-rc1?  I was under
the impression that native support for the nimble controller
has never worked, and that it has been on the todo list for a while.

If you have a fix for the compiler error, then I am sure a pull request
would probably be welcome.  Will is more familiar with the nimble
controller, so I think I will let him and others chime in.

Thanks,
Chris

Re: [DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

On Wed, May 18, 2016 at 11:28:03PM -0700, Christopher Collins wrote:
> Hello all,
> 
> This thread is for any and all discussion regarding the release of
> apache-mynewt-0.9.0-incubating.  All feedback is welcome.

There is one thing I really should have mentioned in the vote email.  If
you want to try out 0.9.0, change your project.yml file so that the
"vers" field for all the repositories points to "0.9.0" (it probably
specifies "0-latest" at the moment).

For example:

repository.apache-mynewt-core:
type: github
vers: 0.9.0
user: apache
repo: incubator-mynewt-core

Then run "newt upgrade":

[ccollins@ccollins-mac:~/tmp/proj2]$ newt upgrade
apache-mynewt-core
Would you like to upgrade repository apache-mynewt-core from 0.8.0-none to 
0.9.0-none ? [Yn] y

All testing is greatly appreciated.

Thanks,
Chris

Re: Problem with GATT service definition

[Christopher Collins]

On Wed, May 18, 2016 at 08:28:16PM -0700, James Howarth wrote:
> Hi Chris,
> 
> You are 100% right, and I realized after I had just it send, it was a
> copy/paste error.

No worries.  I think the problem is that bleprph is trying to allocate
more attributes than it is configured for.  Bleprph's default
configuration cuts it really close.  I would try increasing the
max_attrs setting from 32 to something more comfortable (e.g., 48).
This will give some extra room for more characteristics in the future.
This setting is configured in main() with the following line:

cfg.max_attrs = 48;

Chris

[DISCUSS] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

Hello all,

This thread is for any and all discussion regarding the release of
apache-mynewt-0.9.0-incubating.  All feedback is welcome.

Thanks,
Chris

[VOTE] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

Hello all,

I am pleased to be calling this vote for the source release of Apache
Mynewt 0.9.0. Given that it is version 0.x, it is still a bit of a beta.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications. Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes.

This release candidate was tested as follows:
1. Manual execution of the Mynewt test plan
   
(https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Test+Plan).
2. The full unit test suite for this release was executed via "newt
   test all" with no failures. This testing was performed on the
   following platforms:
 * OS X 10.10.5
 * Linux 3.19.0 (Ubuntu 14.04)

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc1/

The commits under consideration are as follows:

blinky:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky
commit: 82f09fa2cc4c67fb30165ef6be0291aa76e6213e

core:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core
commit: 48df9b088c50cfc97d2518a823294ace761633d6

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt
commit: 0afe655414bbefc922d2adfddf238479bedac5c0

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc1/apache-newt-bin-linux-0.9.0-incubating.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.9.0-incubating/rc1/apache-newt-bin-osx-0.9.0-incubating.tgz


The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers,
please feel free to try out the release candidate and provide your
votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

[RESULT][VOTE] Release Apache Mynewt 0.9.0-incubating-rc1

[Christopher Collins]

Hello all,

Voting for Apache Mynewt 0.9.0-incubating-rc1 is now closed.  The
release has passed this step of the process.  The vote breakdown is as
follows:

+1 Marko Kiiskila
+1 Christopher Collins
+1 Will san Filippo
+1 Sterling Hughes

Total: +4

We can now call a vote on the general@incubator list.

Thank you to all who voted.

Thanks,
Chris

Re: BLE Stack

[Christopher Collins]

On Tue, May 24, 2016 at 03:15:22PM +0400, Vitya Gnatyuk wrote:
> I make up your stack in EmBitz 0.42 with apps\bleprph example. But an one
> error has occurred: undefined reference to `__vector_tbl_reloc__'
> What I can do with it?

Hi Vitya,

Are you using the "newt" tool to build your project, as documented here:
http://mynewt.apache.org/os/get_started/project_create/ ?  It sounds
like you are using the EmBitz IDE instead of newt.  It should certainly
possible to build Mynewt projects without using newt, but it is not
something that we have tried before, and it is not the recommended
setup.

The particular error you are seeing appears to be caused by the
appropriate bsp code not getting linked in to your project.  The newt
tool handles this for you automatically.  If you would like to forgo
using the newt tool, then you will need to manually point your build
tool at all your project's dependencies.

Chris

Re: Procedure for changing power level

[Christopher Collins]

On Thu, May 19, 2016 at 11:12:39AM -0700, James Howarth wrote:
> Hi Chris,
> 
> I think it needs to be a signed int right, as txpwer can be negative, does
> that sound right?

Yes, good catch (thanks also, Kevin!).

In that case, you should declare the tx power variable as an int8_t.

Chris

Re: Correct way to specify custom UUID's

[Christopher Collins]

On Thu, May 19, 2016 at 11:57:45AM -0700, James Howarth wrote:
> I'm not quite sure how to define a custom 128 bit UUID and pass it to
> .uuid128.
> 
> I thought it might be a global variable e.g.
> static uint8_t UUID_BASE[16] = {0x03, 0x04, 0x00, 0x00, 0x2A, 0xAE,
> 0x4D,
> 0x26, 0xAD, 0x62, 0x03, 0xE9, 0xA8, 0x63, 0x7E, 0xBD};
> 
> And then pass this to the service like:
> .uuid128 = UUID_BASE,
> 
> However this does not seem to work.  Any suggestions appreciated.

Hi James,

That is the correct way to specify a 128-bit UUID.  What goes wrong when
you try it?

I also wanted to add a few nitpicks.  None of these are issues that will
prevent your code from working, but I wanted to mention them in case you
want to come back to them later and for the benefit of other readers.

1. You might want to declare the uuid array as const, since it is a
read-only value.  This will allow the linker to place this data in flash
rather than RAM (which is generally more scarce).  Unfortunately, this
requires that you add a (void *) cast to the service definition, since
the service descriptor's uuid128 pointer is non-const [*].

2. I recommend against using all-caps for that variable name.
Generally, all-caps is reserved for macros.  Macros don't obey the usual
rules of the C language, so it is helpful to have a visual indication
that something is actually a macro rather than a regular identifier.

Thanks,
Chris

[*] Nimble and a lot of other Mynewt code has eschewed "const".  I think
there are some places in the nimble host where using const would
simplify the API by obviating the need for casts, so I think we should
change this in the next release.

Re: Mynewt Local BSP Instance Location

[Christopher Collins]

Hi Kevin,

On Wed, May 11, 2016 at 01:32:14PM +0200, Kevin Townsend wrote:
> I'm trying to document the process of creating a custom BSP based on an 
> existing model from apache-newt-core, and copied the model into the 
> local `projectroot/hw/bsp/boardname` folder, updated the `pkg.yml` file, 
> and modified the filenames/content accordingly. I then referenced this 
> BSP in the bootloader and main application target via newt.
> 
> When I reference the local `hw/bsp/boardname` instance and then try to 
> build I get the following error, though:
> 
> $ newt -v build nrf51_boot
> Building target targets/nrf51_boot
> panic: interface conversion: interfaces.PackageInterface is nil, not
> *pkg.LocalPackage
[...]

Ouch... there is clearly a bug in the newt tool.  There should be no
problem with creating a BSP package in the local repo.  Could you send
the output of 'newt target show'?  I would like to see what specifically
triggered this crash.  I'll also look see if I have the same issues with
local BSPs.

Thanks,
Chris

Re: Trouble running bleprh example

[Christopher Collins]

On Mon, May 16, 2016 at 09:29:37PM -0700, James Howarth wrote:
> Hi Chris,
> 
> Thanks.  I'm getting closer.
> 
> Using the 'LightBlue Explorer' app:
> 
>- I can see the nimble-bleprh device
>- nim-bleprh appears to be advertising no services.
>- When I try and connect to the nRF52, the app times out.

Are you using LightBlue for the iPhone or for OS X?  We have observed
an apparent bug in the CoreBluetooth implemention in OS X which results
in a disconnect shortly after the connection is established.  I am not
sure if this bug has been fixed in the latest OS X release.

The bug is related to the data-length-extension link layer procedure.
When the nimble device initiates this procedure, the mac replies with a
malformed packet.  You can disable the data length extension feature by
modifying the
repos/apache-mynewt-core/net/nimble/controller/include/controller/ble_ll.h
file as follows:

FROM:
#define  BLE_LL_CFG_FEAT_DATA_LEN_EXT

TO:
//#define  BLE_LL_CFG_FEAT_DATA_LEN_EXT

I.e., comment out the line which defines the
BLE_LL_CFG_FEAT_DATA_LEN_EXT symbol.

(disabling features such as this one has been made easier in more recent
code, but for now you need to modify the code directly).

>- After I connect once, it takes a couple of minutes, before the nimble
>device will show up in my app, I thought it was supposed to be instant.

The bleprph app should resume advertising as soon as the connection is
broken.  However, it does not advertise while it is still connected to
something.  Are you saying LightBlue does not show the device when you
tell it to scan?

Chris

Re: Trouble running bleprh example

[Christopher Collins]

Hi James,

On Mon, May 16, 2016 at 10:49:10AM -0700, James Howarth wrote:
> Hi,
> 
> I am trying to get the bleprph example up and running on a Nordic nRF52
> PDK.  But I can't seem to see the device using the LightBlue app running on
> my iPhone.
[...]

Hi James,

I'm pretty sure I know what the problem is - the application task's
stack is overflowing right out the gate.  I'm afraid I have to take the
blame for this one; the default logging settings use a lot of stack, and
I don't often test with them. 

The fix is simple: increase the application stack size from 200 to 288
words.  You can do this by changing apps/bleprph/src/main.c as follows:

#define BLEPRPH_STACK_SIZE  (OS_STACK_ALIGN(288))

This change has already been made to the development branch, but it
didn't make it into the previous release.  Using the develop branch
instead of fixing the bug yourself is also an option; you can do so by
changing your project.yml as follows:

FROM:

repository.apache-mynewt-core:
type: github
vers: 0-latest
user: apache
repo: incubator-mynewt-core

TO:

repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core

(change the "vers" field).

Then upgrade your packages with "newt upgrade", e.g.,

[ccollins@iori:~/tmp/myproj]$ newt upgrade
apache-mynewt-core
Would you like to upgrade repository apache-mynewt-core from 0.7.9-none to 
0.0.0-none ? [Yn] y

Naturally, this branch is quite unstable!

Chris

Re: Arduino zero Blinky : build bootloader error

[Christopher Collins]

Hi Pradeep,

On Sun, May 15, 2016 at 06:47:29PM +0530, Pradeep Sanjeewa wrote:
> But when I try to download the bootloader (newt load arduino_boot) , it
> gives an error.
> 
> line 61: openocd: command not found
> Error: exit status 127

Did you see the documentation for getting started in Linux?
http://mynewt.apache.org/os/get_started/cross_tools/#install-arm-cross-arm-tools-for-linux

I think the "Install OpenOCD" step should solve this problem.

Thanks,
Chris

Re: legacy pairing for nimble

[Christopher Collins]

On Mon, May 02, 2016 at 09:27:12PM +, p...@wrada.com wrote:
> 
> All,
> 
> This pull request contains legacy pairing implementation for nimble.  It also 
> contains a change from Chris which is not yet committed that I needed to 
> ensure atomic access to the l2cap state machine in multi-threaded 
> applications.
> 
> Please review if you are interested and get a chance.
> 

Thanks, Paul.  It looks great.  I have merged the pull request.

Chris

Re: callout and callout_func

[Christopher Collins]

I also prefer #2.

Chris

On Fri, May 06, 2016 at 11:31:03AM -0700, will sanfilippo wrote:
> My vote would be #2 as well.
> 
> 
> > On May 6, 2016, at 11:29 AM, marko kiiskila  wrote:
> > 
> > Hi,
> > 
> >> On May 5, 2016, at 10:47 AM, Sterling Hughes  wrote:
> >> 
> >> Salutations,
> >> 
> >> As I've been going through the callout implementation, one thing I've 
> >> noticed is that callouts and callout_funcs can't be interleaved.
> >> 
> >> The implementation of a callout, is that it has an event as the first 
> >> element of the structure.  When that event is posted to an event queue, it 
> >> is posted with the event type EVENT_T_TIMER, which is reserved for 
> >> callouts.  However, you must know a priori what type of callout it is, a 
> >> callout, or a callout_func.
> >> 
> >> I don't think this behavior is ideal, and there are a couple of options 
> >> for fixing it:
> >> 
> >> 1- Break out EVENT_T_TIMER into EVENT_T_TIMER (callout) and 
> >> EVENT_T_TIMER_FUNC (callout_func).
> >> 
> >> 2- Remove the concept of callout, and just have callout_func. callout_func 
> >> is by far the more useful of the two.
> >> 
> >> 3- Add a flags field to callout, which will tell you whether its a callout 
> >> or a callout_func.
> >> 
> >> I'm leaning towards either #2 or #3 here, because I think the first one 
> >> will end up being confusing when debugging things.  "Oh no, I put TIMER 
> >> instead of TIMER_FUNC. GRR."  My personal preference is #2, but I'm not 
> >> sure everyone wants to be forced to have a function per-timer in their 
> >> task context.
> >> 
> >> Thoughts?
> > 
> > I would prefer #2, as that would simplify the concept.
> > 
> > Also, while you have that file cracked open, cf_arg from within 
> > os_callout_func could be removed.
> > os_callout includes os_event, and that structure already has a void * which 
> > could be used as callout_func
> > argument.
> > —
> > M
> 

Re: Setting cflags correctly in bletiny

[Christopher Collins]

On Thu, May 05, 2016 at 06:28:19PM +0530, Anuj Deshpande wrote:
> Hi all,
> 
> I am on 64 bit 16.04 Ubuntu with a manually installed toolchain which has
> been added to my path.
> 
> Before building the app, while setting the cflags for the target, the
> documentation says that one should not give the - symbol.
> 
> http://mynewt.incubator.apache.org/os/tutorials/bletiny_project/
> 
> I had to give that to get it to build.
[...]

Good catch, Anuj.  The documentation is wrong.  I'll submit a pull
request to fix that typo.

Thanks,
Chris

Re: Blinky on nrf52DK needs repos/apache-mynewt-core to be on master

[Christopher Collins]

On Thu, May 05, 2016 at 05:13:25PM +0530, Anuj Deshpande wrote:
> Hi all
> 
> I followed the instructions on how to setup newt tool and then blinky on
> the nrf52 DK.
> 
> http://mynewt.apache.org/os/tutorials/nRF52/
[...]
> I had to change the branch of the repos/apache-mynewt-core repo which was
> cloned for me using the newt tool. This isn't reflected in the docs (and I
> don't think it should be). Is there something amiss with the newt tool ?

Hi Anuj,

As you suspected, you should not need to change the branch using git.
The problem here is that nrf52dk support is a new feature that is going
into the upcoming release.  If all goes well, this release will be out
in a day or two.  Until the release is out, the apache-mynewt-core
repository points to the old tag (mynewt_0_8_0_b2_tag) which lacks this
feature.

Manually pointing the repo to a different branch with git solves the
problem in the short term, but it might cause trouble for you the next
time you try to upgrade the core repo.  I don't believe we have
documented the correct way to use developmental features, so I am
certainly not criticizing you for doing what you did!

The preferred way to use developmental features is to change your
project.yml as follows:

FROM:

repository.apache-mynewt-core:
type: github
vers: 0-latest
user: apache
repo: incubator-mynewt-core

TO:

repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core

(change the "vers" field).

Then upgrade your packages with "newt upgrade", e.g.,

[ccollins@iori:~/tmp/myproj]$ newt upgrade
apache-mynewt-core
Would you like to upgrade repository apache-mynewt-core from 0.7.9-none to 
0.0.0-none ? [Yn] y

Naturally, this branch is quite unstable.

Chris

On Thu, May 05, 2016 at 05:13:25PM +0530, Anuj Deshpande wrote:
> Hi all
> 
> I followed the instructions on how to setup newt tool and then blinky on
> the nrf52 DK.
> 
> http://mynewt.apache.org/os/tutorials/nRF52/
> 
> ➜  apache-mynewt-core git:(2bec33d) ls hw/bsp
> native  nrf51dk  nrf51dk-16kbram  nrf52pdk  olimex_stm32-e407_devboard
> ➜  apache-mynewt-core git:(2bec33d) git checkout master
> Previous HEAD position was 2bec33d... Merge branch 'develop'
> Switched to branch 'master'
> Your branch is up-to-date with 'origin/master'.
> ➜  apache-mynewt-core git:(master) git pull origin master
> From https://github.com/apache/incubator-mynewt-core
>  * branchmaster -> FETCH_HEAD
> Already up-to-date.
> ➜  apache-mynewt-core git:(master) ls hw/bsp -l
> total 28
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 bmd300eval
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 native
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 nrf51dk
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 nrf51dk-16kbram
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 nrf52dk
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 nrf52pdk
> drwxrwxr-x 4 anuj anuj 4096 May  5 16:55 olimex_stm32-e407_devboard
> ➜  apach
> 
> I had to change the branch of the repos/apache-mynewt-core repo which was
> cloned for me using the newt tool. This isn't reflected in the docs (and I
> don't think it should be). Is there something amiss with the newt tool ?
> 
> I was able to build and load by following the other instructions to the T.
> 
> PS: new to the project, looks cool ! Looking forward to contribute !
> 
> Cheers,
> 
> Anuj
> ​

Re: Proposed changes to Nimble host

[Christopher Collins]

On Mon, Apr 18, 2016 at 09:18:16AM -0700, will sanfilippo wrote:
> For #2, my only “concerns” (if you could call them such) are:
> * Using OS_EVENT_TIMER as opposed to some other event. Should all
> OS_EVENT_TIMER events be caused by a timer? Probably no big deal… What
> events are going to be processed here? Do you envision many host
> events?

Yes, I agree.  I think a more appropriate event type would be
OS_EVENT_CALLBACK or similar.  I am a bit leery about adding a new OS
event type for this case, because it would require all applications to
handle an extra event type without any practical benefit.  Perhaps
mynewt could relieve this burden with an "os_handle_event()" function
which processes these generic events.  My concern there is that
applications may want to add special handling for certain event types,
so they wouldn't want to call the helper function anyway.

The OS events that the host would generate are:
* Incoming ACL data packets.
* Incoming HCI events.
* Expired timers.

> * I wonder about the complexity of this from an application developers
> standpoint. Not saying that what you propose would be more or less
> complex; just something we should consider when making these changes.

I think the taskless design reduces complexity for the application
developer.  If there is no host task, the developer can worry less about
task priorities and stack sizes.  

> On a side note (I guess it is related), we should consider how
> applications are going to initialize the host and/or the controller in
> regards to system memory requirements (i.e. mbufs). While our current
> methodology to create a BLE app is not rocket science, I think we
> could make it a bit simpler.

Yes, definitely.  As you say, the setup is not terribly complicated, but
it does involve a fair number of steps, so it will seem complicated to
someone not familiar with Mynewt.

Chris

Re: Mynewt for arduino_101 board

[Christopher Collins]

Hi Andre,

On Mon, Apr 18, 2016 at 10:55:49AM +0300, Andrei Emeltchenko wrote:
> Hi,
> 
> I am working on arduino_101 development board.
> https://www.arduino.cc/en/Main/ArduinoBoard101
> 
> It has nrf51 BLE chip, Basically the configuration is the same as for
> the nrf51dk-16kbram with the only difference:
> 
> --- a/hw/bsp/nrf51dk-16kbram/src/hal_bsp.c
> +++ b/hw/bsp/nrf51dk-16kbram/src/hal_bsp.c
> @@ -24,7 +24,7 @@
>  static const struct nrf51_uart_cfg uart_cfg = {
>  .suc_pin_tx = 9,
>  .suc_pin_rx = 11,
> -.suc_pin_rts = 8,
> +.suc_pin_rts = 12,
>  .suc_pin_cts = 10
>  };
> 
> 
> What is the best way of keeping the change? Making special BSP would be
> too expensive for this one line change.

This is indeed an annoying situation.  There is a nearly identical
dilemma with the Arduino Zero and the Arduino Zero Pro; the only
difference among these two boards is a single pin.

The solution for the Arduino Zero is to use a single BSP, but to select
the proper pin at compile time via an #ifdef:

#ifdef ARDUINO_ZERO_PRO
 ARDUINO_ZERO_D2 = (8),
 ARDUINO_ZERO_D4 = (14),
#endif

#ifdef ARDUINO_ZERO
 ARDUINO_ZERO_D2 = (14),
 ARDUINO_ZERO_D4 = (8),
#endif

The appropriate preprocessor symbol is defined by the use of a target
feature, as described in the arduino zero tutorial:
http://mynewt.apache.org/os/tutorials/arduino_zero/

That said, I am not sure this is the right approach.  I will certainly
let others weight in, but my feeling is that it is better to just bite
the bullet and create a new BSP.  Both solutions pose maintenance
headaches, but I think separate BSPs is simpler and more maintainable.
An additional benefit of separate BSPs is that it simplifies target
creation for the application developer (just specify BSP, rather than
BSP plus feature).

Thanks,
Chris

Re: MYNEWT-262 Log module needs split in logging and reporting

[Christopher Collins]

Hi Vipul,

On Mon, Apr 18, 2016 at 11:53:28PM -0700, Vipul Rahane wrote:
> - Make changes to the log_register() function to take LOG_MODULE_X as
> an argument. This module ID is present as a field in the log structure
> and can be used anytime an entry is logged in the specific log. There
> was a disconnect earlier as the log itself did not quite have a module
> ID of it’s own. If we want to be able to filter logs the module ID
> needs to be used as a filtering criteria to pick the right log node
> from the queue.

Sounds good to me.  Currently, a log has a name but not a module ID, and
there is no way to determine which log created a particular log entry.

[...]

> - When newtmgr requests to read all the logs using “logs show”
> command, the response would contain a module ID as well. If the
> response does not contain any entries, no response would be sent for
> that log. Module ID also helps in filtering the log entries while
> sending the response.
> 
> JSON Response:
> 
> {
>   "rc": 0,
>   "logs": [{
>   "name": "log",
>   "module": 0,
>   "type": 1,
>   "entries": [{
>   "msg": "GPIO toggle from 1 to 0",
>   "ts": 1007000,
>   "level": 2,
>   "index": 1
>   }, {
>   "msg": "GPIO toggle from 0 to 1",
>   "ts": 2015000,
>   "level": 2,
>   "index": 2
>   }, {
>   "msg": "GPIO toggle from 1 to 0",
>   "ts": 3019000,
>   "level": 2,
>   "index": 3
>   }, {
>   "msg": "GPIO toggle from 0 to 1",
>   "ts": 4024000,
>   "level": 2,
>   "index": 4
>   }]
>   }]
> }

I don't necessarily disagree, but I am wondering why the newtmgr user
needs to know about module IDs at all.  I was thinking module IDs were
only used internally to keep track of which log a particular entry
belongs to.  It seems like the newtmgr user would rather just deal with
log names than module IDs.

That said, I am not against providing extra information to the user if
it might be helpful.

> When the log_set_handler() function is called in the app, it makes all
> the function pointers if any nodes exist in the TAILQ to point to the
> same log handlers. One problem with this approach is that since the
> log_set_handler() function needs to be called after all the modules
> are initialized, logging to the same log during initialization is not
> possible as the logs are not created at that point.
> 
> Ways to solve this:
> 1. Passing the log handler as an argument to all the init functions
> which I don’t like personally as it is not too clean.
> 
> 2. Having a global default log handler which gets set by calling
> log_set_handler() which would take precedence over other logs as
> log_register() would also look at this default handler and if the
> default handler is present, the log infrastructure decides to use that
> instead of whats specified in the modules. As a result everything that
> gets logged would go to the log defined by the app including
> initialization logs.
> 
> Thoughts ?

As background for other readers: libraries are responsible for fully
configuring their own logs, including specifying where the log writes to
(e.g., console, a particular flash circular buffer, etc.).  This is
problematic because the library doesn't know the app's capabilities and
requirements; it is the app which should have the final say regarding
where each log gets written.

I think we need to make sure the following requirements are met (this is
off the top of my head, so feel free to call me out if I'm saying
something absurd!):

A. An app can easily arrange for all logs to be written to the same
place.

B. An app can specify where a particular library's log(s) get written
to.

Requirement A allows for easy bringup and debugging.  Requirement B is
necessary for production firmware where each log is expected to use a
particular handler.  I think your solution 2 addresses requirement A,
but it does not give an app sufficient control to implement requirement
B.

To give a hypothetical example: an app wants to log all bluetooth
activity to flash circular buffer x, and all application activity to
flash circular buffer y.  The app is not interested in logs from any
other libraries, so it has not reserved any RAM or flash to hold these
other logs.  A default handler won't meet this app's requirements
for two reasons: 1) only a single circular buffer gets used, but the app
wants to use two, and 2) logs from all libraries get written to flash,
but the app wants to save the flash space for the logs that it actually
cares about.

I can't think of any way to give an app the control it needs, except by
implementing your first solution (library 

Proposed changes to Nimble host

[Christopher Collins]

Hello all,

The Mynewt BLE stack is called Nimble.  Nimble consists of two packages:
* Controller (link-layer) [net/nimble/controller]
* Host (upper layers) [net/nimble/host]

This email concerns the Nimble host.  

As I indicated in an email a few weeks ago, the code size of the Nimble
host had increased beyond what I considered a reasonable level.  When
built for the ARM cortex-M4, with security enabled and the log level set
to INFO, the host code size was about 48 kB.  In recent days, I came up
with a few ideas for reducing the host code size.  As I explored these
ideas, I realized that they open the door for some major improvements in
the fundamental design of the host.  Making these changes would
introduce some backwards-compatibility issues, but I believe it is
absolutely the right thing to do.  If we do this, it needs to be done
now while Mynewt is still in its beta phase.  I have convinced myself
that this is the right way forward; now I would like to see what the
community thinks.  As always, all feedback is greatly appreciated.

There are two major changes that I am proposing:

1. All HCI command/acknowledgement exchanges are blocking.

Background: The host and controller communicate with one another via the
host-controller-interface (HCI) protocol.  The host sends _commands_ to
the controller; the controller sends _events_ to the host.  Whenever the
controller receives a command from the host, it immediately responds
with an acknowledgement event.  In addition, the controller also sends
unsolicited events to the host to indicate state changes or to request
information in a subsequent command.

In the current host, all HCI commands are sent asynchronously
(non-blocking).  When the host wants to send an HCI command, it
schedules a transmit operation by putting an OS event on its own event
queue.  The event points to a callback which does the actual HCI
transmission.  The callback also configures a second callback to be
executed when the expected acknowledgement is received from the
controller.  Each time the host receives an HCI event from the
controller, an OS event is put on the host's event queue.  Processing of
this OS event ultimately calls the configured callback (if it is an
acknowledgement), or a hardcoded callback (if it is an unsolicited HCI
event).

This design works, but it introduces a number of problems.  First, it
requires the host code to maintain some quite complex state machines for
what seem like simple HCI exchanges.  This FSM machinery translates into
a lot of extra code.  There is also a lot of ugliness involved in
canceling scheduled HCI transmits.

Another complication with non-blocking HCI commands is that they require
the host to jump through a lot of hoops to provide feedback to the
application.  Since all the work is done in parallel by the host task,
the host has to notify the application of failures by executing
callbacks configured by the application.  I did not want to place any
restrictions on what the application is allowed to do during these
callbacks, which means the host has to ensure that it is in a valid
state whenever a callback gets executed (no mutexes are locked, for
example).  This requires the code to use a large number of mutexes and
temporary copies of host data structures, resulting in a lot of
complicated code.

Finally, non-blocking HCI operations complicates the API presented to
the application.  A single return code from a blocking operation is
easier to manage than a return code plus the possibility of a callback
being executed sometime in the future from a different task.  A blocking
operation collapses several failure scenarios into a single function
return.

Making HCI command/acknowledgement exchanges blocking addresses all of
the above issues:
* FSM machinery goes away; controller response is indicated in the
  return code of the HCI send function.
* Nearly all HCI failures are indicated to the application
  immediately, so there is no need for lots of mutexes and temporary
  copies of data structures.
* API is simplified; operation results are indicated via a simple
  function return code.

2. The Nimble host is "taskless"

Currently the Nimble host runs in its own OS task.  This is not
necessarily a bad thing, but in the case of the host, I think the costs
outweigh the benefits.  I can think of three benefits to running a
library in its own task:
* Guarantee that timing requirements are met; just configure the
  task with an appropriate priority.
* (related to the above point) The library task can continue to work
  while the application task is blocked.
* Facilitates stack sizing. Since the library performs its
  operations in its own stack, it is easier to predict stack usage
  of both the library task and the application task.

I don't think any of these benefits are very compelling in the case of
the Nimble host for the following reasons:
* The host has nothing 

Re: Problem Loading arduino_boot

[Christopher Collins]

Hi Nges,

On Thu, Apr 14, 2016 at 03:02:49AM +0100, Nges B wrote:
> Ok thanks.
> I will be waiting for your response.

Unfortunately, I can't reproduce the problem over here.  I was able to
load the arduino boot loader in Ubuntu 14.10 without any problems.

[ccollins@ccollins-VirtualBox:~/tmp/myproj]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 14.10
Release:14.10
Codename:   utopic
[ccollins@ccollins-VirtualBox:~/tmp/myproj]$ newt target show
targets/arduino_boot
app=@apache-mynewt-core/apps/boot
bsp=@mynewt_arduino_zero/hw/bsp/arduino_zero
build_profile=optimized
features=arduino_zero 
targets/my_blinky_sim
app=apps/blinky
bsp=@apache-mynewt-core/hw/bsp/native
build_profile=debug
[ccollins@ccollins-VirtualBox:~/tmp/myproj]$ newt load arduino_boot
Loading image
[ccollins@ccollins-VirtualBox:~/tmp/myproj]$ 

Sorry I can't provide more help here.

Chris

Re: Problem Loading arduino_boot

[Christopher Collins]

Hi Nges,

On Thu, Apr 14, 2016 at 02:44:10AM +0100, Nges B wrote:
> hello Chris sorry for my late reply,
> I succeeded in installing the openocd 0.9.0 since yesterday morning.

Not a problem, and sorry for not noticing the "0.9.0" string in your
earlier emails.

You do not need to download CMSIS-DAP - this is something that is
supported by the Arduino board.  It appears there is still a USB issue
somewhere.  Unfortunately, I don't have any ideas at the moment.  I
don't have a Linux machine available at the moment, but tonight I will
try with my home Linux machine and see if I encounter the same issue.

Chris

Re: bug+patch: Read By Group Type Response

[Christopher Collins]

On Mon, Jul 25, 2016 at 02:17:40PM -0700, Christopher Collins wrote:
> On Mon, Jul 25, 2016 at 12:09:13PM -0700, Mike Ryan wrote:
> > I discovered and patched a bug in the Read Group Type Response message
> > when using small MTUs. The details are recorded in the following pull
> > request:
> > 
> > https://github.com/apache/incubator-mynewt-core/pull/79
> > 
> > This issue appears to affect both master and develop.
> 
> Hi Mike,
> 
> Thanks for the pull request!  I am not able to reproduce this issue, so
> I must be doing something wrong.  I will add some more details to the
> pull request shortly.

Actually, I think I see what is happening.  This bug was fixed shortly
after 0.9.0 was released.  The fix has since been merged to master
(43143c9027aef473c22b2b6a6accaa4cfe1237c5, Jun 1).  Is
it possible that you are working with an older version of the code?  By
the way, I do realize it is difficult to keep up to date when the code
has changed so much recently!

Please let me know if you think there is still an issue here.  And
please keep finding bugs :).

Chris