Re: [Discuss] Migrate the build system to CMake

[Matias N.]

Hi,
I'll answer a few question that came up several times about build systems
co-existing. My previous answer was about co-existing them as a way to satisfy
both set of users, which I don't think is the right approach. However, as a 
transition
approach it does makes sense to me.

This is technically feasible however it will require adapting the Make based 
system
as there were some changes needed to be made to Kconfig files, specifically 
this one:

https://github.com/apache/incubator-nuttx/pull/3704/files?file-filters%5B%5D=.dat%5B%5D=.pl%5B%5D=.sh%5B%5D=No+extension#diff-7c9693e9c5e61a1d80e09769377d47bb415ec39ca5aeaa85d13581b0105d6359

The main point is that CMake handles all steps related to build configuration 
(e.g. build flags based on
optimization level) and platform detection. On master this is done via Kconfig 
(with
settings which are stored in board defconfigs). One of these changes done to 
Kconfig
is to pass the host-related variables as environment vars to Kconfig (so that 
it still
possible to condition options based on platform).
So, co-existing the build systems would require dealing with this, which I'm 
not sure
how easy would be.

With regards to comments about others requiring some hands-on time to become
familiar with the system and try it for themselves: I agree with this point and 
was one
of the reasons I actually started the migration. As I mentioned several weeks 
ago,
the cmake branch can already be tested on the currently supported boards by 
everyone
wishing to do so, but I don't think anyone really tried it as my offers about
assistance in this were unanswered.

Technical aspects aside: I do not wish to antagonize others not liking the 
change.
I think I outlined the reasons I believe it is a positive change (and others 
did so as well)
so now it is up to the community to decide what direction they wish to follow.
In any case, if there's enough push forward, I would strongly recommend that 
this is
explicitly decided via a vote (or any other mechanism considered appropriate) 
with
a roadmap and organization, so that it becomes a goal for the NuttX project and 
not
of just a subset of its community. I think this relates to making decisions 
about the 
project roadmap, which at some point needs to be done in an "official capacity".

Best,
Matias

PS: the PR can always be reopened and the code lives on a branch on my fork.
The branch on the repo does not yet contain anything, it simply acts as a 
stable base
for the PR (and, of course, protects master from the merge, if it ever happens).

On Wed, Jun 9, 2021, at 18:32, Nathan Hartman wrote:
> On Wed, Jun 9, 2021 at 3:07 PM Matias N.  <mailto:matias%40imap.cc>> wrote:
> > I will leave up to PPMC to decide whether a call to a vote is 
> > necessary/useful, but
> > to me the community response is now quite clear. Also, voting based on 
> > majority
> > would not really consider that such a change actually requires significant
> > backing from the comunity. I will therefore will not continue working on 
> > this
> > (you can delete the branch).
> 
> Please don't delete the branch.
> 
> Although switching build systems is a scary thought, I think we should
> be open-minded about trying it. As I suggested before, and I think two
> other people said something similar, perhaps the CMake can coexist
> with the current Make build system somehow, even if it's labeled
> "Experimental." Yes I know that creates more maintenance burden, and a
> few people said it doesn't make sense to manage multiple build
> systems, but other people pointed out that all downstream users will
> need to rework their out-of-tree boards/frameworks/etc to adopt CMake.
> That might make some people upset; but it will probably make some
> people happy too, and we (or at least I) don't know which group is
> bigger. In either case, with such a big change, I really think it
> can't just happen in one atomic commit. For example, think of the
> Python 2 -> Python 3 transition. That has been going on for how many
> years now?! This proposed change won't take that long to transition,
> but it will still need time. If we could have both build systems for a
> while, it would allow people to test the waters with CMake but fall
> back on Make if they hit problems and need to get their work done; it
> will likely get a lot of feedback and fixes; and eventually depending
> on how good it gets, we can deprecate Make and take CMake out of
> "experimental" status. This is a disadvantage for maintenance, but it
> will give downstream a transition period that will be needed for this
> kind of change.
> 
> In any event, whatever you do, please DON'T delete that branch!
> 
> It would be better to find a way to merge it back to master without
> destroying the existing Make system.
> 
> That's just my two [insert 2/100 of your favorite currency here].
> 
> Nathan
> 

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

ystem anymore
> > > >>
> > > >> - since makefiles are now generated, we rely on yet another external
> > > >> tool with bugs in itself, and its idiosyncrasies and workarounds.
> > > >>
> > > >>
> > > >> Moreover:
> > > >>
> > > >> -the doc about nuttx is not hosted by the nuttx project, so 99 % of
> > > >> the nuttx documentation will become fully obsolete overnight.
> > > >>
> > > >>
> > > >> Gratuitous changes are a hell, they destroy efficiency.
> > > >>
> > > >> They tend to appear more frequently in open source projects, because
> > > >> anyone can bring it change without a single damn given to customer
> > > >> since the code has no warranty of fitness of etc etc open source
> > > >> legalese.
> > > >>
> > > >>
> > > >> If it was me, I would not do this change. If I had to take a decision
> > > >> about something similar in my company, it would be a strong no.
> > > >>
> > > >>
> > > >> Sebastien
> > > >>
> > > >>
> > > >> Le 09/06/2021 à 14:57, Matias N. a écrit :
> > > >>> Hi everyone,
> > > >>>
> > > >>> this thread has received little engagement from the community
> > > >>> in general, for a change with such impact on daily use of NuttX
> > > >>> for everyone.
> > > >>>
> > > >>> While there was positive feedback on GH and a few people have
> > > >>> expressed more interest, not much has really happened. Meanwhile,
> > > >>> the backlog of changes that would need to be backported continues
> > > >>> to increase.
> > > >>>
> > > >>> At the same time, I see many PRs addressing subtle issues with
> > > >>> current build system, which are mostly already solved with the
> > > >>> migration
> > > >>> to CMake. So there's continued effort in maintaining the current
> > system
> > > >>> which could be in part dedicated to the migration to a better system.
> > > >>>
> > > >>> I have offered technical guidance on testing and extending to other
> > > >>> platforms and also to add base support for other arch's so that the
> > > >>> focus
> > > >>> can be put mostly at the board level and on test and validation on
> > > >>> different
> > > >>> platforms and target hardware. However, after having worked on this
> > > >>> for more than a month I feel this is not really picking up the
> > > >>> interest it
> > > >>> requires for proper adoption by the community.
> > > >>>
> > > >>> Maybe the proper approach would be to call on a vote (*)
> > > >>> for this feature to have explicit support from the community and
> > > >>> ensure involvement from others than me to move forward.
> > > >>> Otherwise, or if the vote does not pass, I will not be pushing
> > > >>> forward with this.
> > > >>>
> > > >>> (*) As a commiter (non-PPMC member), I'm not sure if I can formally
> > > >>> call on
> > > >>> a vote and what the exact procedure is, but maybe other PPMC member
> > > >>> can do so.
> > > >>>
> > > >>> Best,
> > > >>> Matias
> > > >>>
> > > >>> On Tue, Jun 1, 2021, at 15:05, Nathan Hartman wrote:
> > > >>>> I am interested and I'll try to help with boards I can test. It
> > > >>>> will take a
> > > >>>> few days to get around to it because this has been a busy month,
> > > >>>> but I'm
> > > >>>> catching up.
> > > >>>>
> > > >>>> Cheers
> > > >>>> Nathan
> > > >>>>
> > > >>>> On Tue, Jun 1, 2021 at 11:25 AM Alan Carvalho de Assis
> > > >>>> mailto:acassis%40gmail.com> 
> > > >>>> <mailto:acassis%40gmail.com>>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> I think we can divide the effort to port all the boards to the new
> > > >>>>> C

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

On Wed, Jun 9, 2021, at 11:08, Nathan Hartman wrote:
> On Wed, Jun 9, 2021 at 10:01 AM  > wrote:
> >
> > I like the cmake idea but we should also consider that cmake will have 
> > implications on all projects that use the NuttX as a base for the SDK or 
> > custom projects.
> >
> > All those projects are using the old make system and such a change will 
> > have a huge impact for them.
> 
> 
> This might be a dumb question but:
> 
> Can't a CMake build be an OPTIONAL component for people who want to use it?
> 
> Nathan

Maintaining two build systems in parallel does not really make sense to me. 
There should be only one and used and maintained by the community.

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

Hi Alan,

On Wed, Jun 9, 2021, at 10:12, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> I think according to Apache rules anyone can call for vote.

Ok. I would imagine that for a vote to have any purpose there
still needs to be some discussion first. Or is it ok to have the discussion
on the vote thread?

My proposal to vote on would be to:
- confirm interest over this change by the community
- discuss potential roadmap for completing the work on the cmake branch
and integration of the branch into master
- identify community members willing to support the effort and organize
in a table who would test on which platform and which target to ensure 
sufficient
coverage

> I'm working in the ESP32(C3, S2) cmake port and probably will need your 
> review.

Sure, thanks. I know you volunteered and I appreciate it. But I think
we need larger commitment from other people as well due to the
large number of arch/boards/platforms supported.

> Moving from Makefile to CMake is not a simple task and it is prone for
> failure, OpenOCD tried it around 2008 or so and it didn't get to much
> traction, so the end up with Makefiles until now.

The current state of the migration is quite usable and I focused on critical 
aspects
such as the protected build, ELF modules, etc. This has helped me identify
the tricky bits. The remaining work should not be that hard as it is mostly
covering new platforms and most of the ground work is laid out.

> I think it is important to reinforce the benefit we will get from
> CMake, otherwise people will still with Makefile, because it is a
> "known devil".

This was already done on the original GitHub issue (which was linked
at the top of the thread):

https://github.com/apache/incubator-nuttx/issues/3455

Best,
Matias

> 
> BR,
> 
> Alan
> 
> On 6/9/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > Hi everyone,
> >
> > this thread has received little engagement from the community
> > in general, for a change with such impact on daily use of NuttX
> > for everyone.
> >
> > While there was positive feedback on GH and a few people have
> > expressed more interest, not much has really happened. Meanwhile,
> > the backlog of changes that would need to be backported continues
> > to increase.
> >
> > At the same time, I see many PRs addressing subtle issues with
> > current build system, which are mostly already solved with the migration
> > to CMake. So there's continued effort in maintaining the current system
> > which could be in part dedicated to the migration to a better system.
> >
> > I have offered technical guidance on testing and extending to other
> > platforms and also to add base support for other arch's so that the focus
> > can be put mostly at the board level and on test and validation on
> > different
> > platforms and target hardware. However, after having worked on this
> > for more than a month I feel this is not really picking up the interest it
> > requires for proper adoption by the community.
> >
> > Maybe the proper approach would be to call on a vote (*)
> > for this feature to have explicit support from the community and
> > ensure involvement from others than me to move forward.
> > Otherwise, or if the vote does not pass, I will not be pushing forward with
> > this.
> >
> > (*) As a commiter (non-PPMC member), I'm not sure if I can formally call on
> > a vote and what the exact procedure is, but maybe other PPMC member
> > can do so.
> >
> > Best,
> > Matias
> >
> > On Tue, Jun 1, 2021, at 15:05, Nathan Hartman wrote:
> >> I am interested and I'll try to help with boards I can test. It will take
> >> a
> >> few days to get around to it because this has been a busy month, but I'm
> >> catching up.
> >>
> >> Cheers
> >> Nathan
> >>
> >> On Tue, Jun 1, 2021 at 11:25 AM Alan Carvalho de Assis  >> <mailto:acassis%40gmail.com>
> >> <mailto:acassis%40gmail.com>>
> >> wrote:
> >>
> >> > I think we can divide the effort to port all the boards to the new
> >> > CMake.
> >> >
> >> > I can start take care of ESP32, ESP32-C3 and ESP32-S2.
> >> >
> >> > Let see if we get more people involved in this effort.
> >> >
> >> > BR,
> >> >
> >> > Alan
> >> >
> >> > On 6/1/21, Matias N. mailto:matias%40imap.cc> 
> >> > <mailto:matias%40imap.cc>> wrote:
> >> > > Hi,
> >> > > just wanted to add that until this is ready, the gap between master
> >> 

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

Hi everyone,

this thread has received little engagement from the community
in general, for a change with such impact on daily use of NuttX
for everyone.

While there was positive feedback on GH and a few people have
expressed more interest, not much has really happened. Meanwhile,
the backlog of changes that would need to be backported continues
to increase.

At the same time, I see many PRs addressing subtle issues with
current build system, which are mostly already solved with the migration
to CMake. So there's continued effort in maintaining the current system
which could be in part dedicated to the migration to a better system.

I have offered technical guidance on testing and extending to other
platforms and also to add base support for other arch's so that the focus
can be put mostly at the board level and on test and validation on different
platforms and target hardware. However, after having worked on this
for more than a month I feel this is not really picking up the interest it
requires for proper adoption by the community.

Maybe the proper approach would be to call on a vote (*)
for this feature to have explicit support from the community and
ensure involvement from others than me to move forward.
Otherwise, or if the vote does not pass, I will not be pushing forward with 
this.

(*) As a commiter (non-PPMC member), I'm not sure if I can formally call on
a vote and what the exact procedure is, but maybe other PPMC member
can do so.

Best,
Matias

On Tue, Jun 1, 2021, at 15:05, Nathan Hartman wrote:
> I am interested and I'll try to help with boards I can test. It will take a
> few days to get around to it because this has been a busy month, but I'm
> catching up.
> 
> Cheers
> Nathan
> 
> On Tue, Jun 1, 2021 at 11:25 AM Alan Carvalho de Assis  <mailto:acassis%40gmail.com>>
> wrote:
> 
> > I think we can divide the effort to port all the boards to the new CMake.
> >
> > I can start take care of ESP32, ESP32-C3 and ESP32-S2.
> >
> > Let see if we get more people involved in this effort.
> >
> > BR,
> >
> > Alan
> >
> > On 6/1/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > > Hi,
> > > just wanted to add that until this is ready, the gap between master and
> > the
> > > branch
> > > widens with every merged PR and this increases the backporting effort.
> > > I'm willing to do most of the remaining work but as I mentioned I cannot
> > > possibly test everything so
> > > help is needed.
> > > I'd really like your feedback on this before I continue and ensure the
> > > effort will not go to waste.
> > >
> > > Best,
> > > Matias
> > >
> > > On Sat, May 29, 2021, at 14:06, Matias N. wrote:
> > >> Hi,
> > >> for anyone not following the relevant PR, please have a look at the
> > >> current state here:
> > >>
> > >> https://github.com/apache/incubator-nuttx/pull/3704
> > >>
> > >> This is now at a point where it can be tested by others. It would be
> > very
> > >> good to get some
> > >> help testing what I got so far (sim and stm32f4discovery, both on Linux
> > >> and mac), by running
> > >> examples and test. There are some brief instructions at the end of the
> > PR
> > >> description for building.
> > >>
> > >> Other than that, I can continue porting other arch's and boards with the
> > >> help of CI but it would be
> > >> best if others with more boards could help testing (and ideally with
> > some
> > >> PRs, as the hard part
> > >> is mostly done) those as well.
> > >>
> > >> Also, note that this is a PR against a branch so we could eventually
> > merge
> > >> it before adding support
> > >> for other arch/boards. And finally, I will provide documentation to the
> > >> new build system in a separate
> > >> PR at some point, which I hope will ease the transition and help
> > >> reviewing.
> > >>
> > >> Best,
> > >> Matias
> > >>
> > >> On Sat, Apr 10, 2021, at 18:43, Xiang Xiao wrote:
> > >>> A new issue is opened recently to address this topic:
> > >>> https://github.com/apache/incubator-nuttx/issues/3455
> > >>> This proposal has the depth of the impact in our daily working, so it's
> > >>> very important to get the feedback from the community before the real
> > >>> action is taken.
> > >>> If you have any concern or suggestion, please reply to this email.
> > >>>
> > >>> Thanks
> > >>> Xiang
> > >>>
> > >>
> > >
> >
> 

Re: Nimble on U-blox Nina B112 (Nrf52832)

[Matias N.]

Hi Miguel,

if you have questions regarding using nimBLE I suggest you go to their
support channels as this is really not NuttX related.

Best,
Matias

On Tue, Jun 8, 2021, at 14:01, Miguel Wisintainer wrote:
> Hi Matias

>  

> Have you some example to make the scan ?

>  

> Enviado do Email  para 
> Windows 10

>  

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

On Tue, Jun 1, 2021, at 12:24, Alan Carvalho de Assis wrote:
> I think we can divide the effort to port all the boards to the new CMake.

That would be ideal.

> I can start take care of ESP32, ESP32-C3 and ESP32-S2.
> 
> Let see if we get more people involved in this effort.

Thanks, Alan.

> 
> BR,
> 
> Alan
> 
> On 6/1/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > Hi,
> > just wanted to add that until this is ready, the gap between master and the
> > branch
> > widens with every merged PR and this increases the backporting effort.
> > I'm willing to do most of the remaining work but as I mentioned I cannot
> > possibly test everything so
> > help is needed.
> > I'd really like your feedback on this before I continue and ensure the
> > effort will not go to waste.
> >
> > Best,
> > Matias
> >
> > On Sat, May 29, 2021, at 14:06, Matias N. wrote:
> >> Hi,
> >> for anyone not following the relevant PR, please have a look at the
> >> current state here:
> >>
> >> https://github.com/apache/incubator-nuttx/pull/3704
> >>
> >> This is now at a point where it can be tested by others. It would be very
> >> good to get some
> >> help testing what I got so far (sim and stm32f4discovery, both on Linux
> >> and mac), by running
> >> examples and test. There are some brief instructions at the end of the PR
> >> description for building.
> >>
> >> Other than that, I can continue porting other arch's and boards with the
> >> help of CI but it would be
> >> best if others with more boards could help testing (and ideally with some
> >> PRs, as the hard part
> >> is mostly done) those as well.
> >>
> >> Also, note that this is a PR against a branch so we could eventually merge
> >> it before adding support
> >> for other arch/boards. And finally, I will provide documentation to the
> >> new build system in a separate
> >> PR at some point, which I hope will ease the transition and help
> >> reviewing.
> >>
> >> Best,
> >> Matias
> >>
> >> On Sat, Apr 10, 2021, at 18:43, Xiang Xiao wrote:
> >>> A new issue is opened recently to address this topic:
> >>> https://github.com/apache/incubator-nuttx/issues/3455
> >>> This proposal has the depth of the impact in our daily working, so it's
> >>> very important to get the feedback from the community before the real
> >>> action is taken.
> >>> If you have any concern or suggestion, please reply to this email.
> >>>
> >>> Thanks
> >>> Xiang
> >>>
> >>
> >
> 

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

Hi,
just wanted to add that until this is ready, the gap between master and the 
branch
widens with every merged PR and this increases the backporting effort. 
I'm willing to do most of the remaining work but as I mentioned I cannot 
possibly test everything so
help is needed. 
I'd really like your feedback on this before I continue and ensure the effort 
will not go to waste.

Best,
Matias

On Sat, May 29, 2021, at 14:06, Matias N. wrote:
> Hi,
> for anyone not following the relevant PR, please have a look at the current 
> state here:
> 
> https://github.com/apache/incubator-nuttx/pull/3704
> 
> This is now at a point where it can be tested by others. It would be very 
> good to get some
> help testing what I got so far (sim and stm32f4discovery, both on Linux and 
> mac), by running
> examples and test. There are some brief instructions at the end of the PR 
> description for building.
> 
> Other than that, I can continue porting other arch's and boards with the help 
> of CI but it would be
> best if others with more boards could help testing (and ideally with some 
> PRs, as the hard part
> is mostly done) those as well.
> 
> Also, note that this is a PR against a branch so we could eventually merge it 
> before adding support
> for other arch/boards. And finally, I will provide documentation to the new 
> build system in a separate
> PR at some point, which I hope will ease the transition and help reviewing.
> 
> Best,
> Matias
> 
> On Sat, Apr 10, 2021, at 18:43, Xiang Xiao wrote:
>> A new issue is opened recently to address this topic:
>> https://github.com/apache/incubator-nuttx/issues/3455
>> This proposal has the depth of the impact in our daily working, so it's
>> very important to get the feedback from the community before the real
>> action is taken.
>> If you have any concern or suggestion, please reply to this email.
>> 
>> Thanks
>> Xiang
>> 
> 

Re: [Discuss] Migrate the build system to CMake

[Matias N.]

Hi,
for anyone not following the relevant PR, please have a look at the current 
state here:

https://github.com/apache/incubator-nuttx/pull/3704

This is now at a point where it can be tested by others. It would be very good 
to get some
help testing what I got so far (sim and stm32f4discovery, both on Linux and 
mac), by running
examples and test. There are some brief instructions at the end of the PR 
description for building.

Other than that, I can continue porting other arch's and boards with the help 
of CI but it would be
best if others with more boards could help testing (and ideally with some PRs, 
as the hard part
is mostly done) those as well.

Also, note that this is a PR against a branch so we could eventually merge it 
before adding support
for other arch/boards. And finally, I will provide documentation to the new 
build system in a separate
PR at some point, which I hope will ease the transition and help reviewing.

Best,
Matias

On Sat, Apr 10, 2021, at 18:43, Xiang Xiao wrote:
> A new issue is opened recently to address this topic:
> https://github.com/apache/incubator-nuttx/issues/3455
> This proposal has the depth of the impact in our daily working, so it's
> very important to get the feedback from the community before the real
> action is taken.
> If you have any concern or suggestion, please reply to this email.
> 
> Thanks
> Xiang
> 

Re: Nimble on U-blox Nina B112 (Nrf52832)

[Matias N.]

I can try tomorrow. Maybe you can try enabling another app and see if you can 
run it. The "command not found" seems unrelated to nimBLE.

On Tue, May 25, 2021, at 20:26, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> No, this is not the issue:
> 
> Symbol: BUILTIN [=y]
> Symbol: NSH_BUILTIN_APPS [=y]
> 
> As you saw I used your "nrf52832-mdk:sdc" and you enabled it there.
> 
> Is it working for you?
> 
> BR,
> 
> Alan
> 
> On 5/25/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > The problem with apps listed but not being able to run them is a common
> > error (something worth adding
> > to the FAQ) I faced many times. It is due to not having support for BUILTIN
> > apps on menuconfig (you need general
> > support as well as enabling NSH BUILTIN support). It is strange that the
> > config is not functional though.
> >
> > Best,
> > Matias
> >
> > On Tue, May 25, 2021, at 19:51, Alan Carvalho de Assis wrote:
> >> Hi Matias and Miguel,
> >>
> >> I just tried nimble on nrf52832-mdk board without success:
> >>
> >> $ ./tools/configure.sh nrf52832-mdk:sdc
> >> $ make
> >>
> >> It downloaded and compiled nimble for NuttX correctly, the nuttx.bin
> >> was about 314944 bytes.
> >>
> >> When I drop this file inside DAPLINK disk it tries to flash and create
> >> the file FAIL.TXT with this content:
> >>
> >> "The hex file cannot be decoded. Checksum calculation failure occurred."
> >>
> >> Then I ran "make menuconfig" and enabled the "Intel HEX binary format"
> >> and after copying the nuttx.hex to DAPLINK disk the error disappeared.
> >>
> >> Accessing the nsh terminal I can see the nimble binary, but it is not
> >> running:
> >>
> >> NuttShell (NSH) NuttX-10.1.0-RC1
> >> nsh> ?
> >> help usage:  help [-v] []
> >>
> >>   . cdecho  hexdump   mkdir pssource
> >> unset
> >>   [ cpexec  ifconfig  mkfatfs   pwd   test
> >> usleep
> >>   ? cmp   exit  ifdownmkrd  rmtime
> >> xd
> >>   basename  dirname   false ifup  mount rmdir true
> >>   break ddfree  kill  mvset   uname
> >>   cat   dfhelp  lsnslookup  sleep umount
> >>
> >> Builtin Apps:
> >>   nimble  sh  nsh
> >> nsh> nimble
> >> nsh: nimble: command not found
> >> nsh> ifconfig
> >> bnep0   Link encap:UNSPEC at UP
> >>
> >> nsh> nimble -h
> >> nsh: nimble: command not found
> >> nsh> nimble
> >> nsh: nimble: command not found
> >> nsh>
> >>
> >> Initially I thought it was caused by recent update of the nimble stack
> >> on NuttX, but I moved to a commit previous to that update and still
> >> facing same error.
> >>
> >> Matias, do you think it could be some issue with my crosscompiler?
> >>
> >> I'm using the default ARM gcc from Ubuntu 20.04 gcc-arm-none-eabi
> >> package:
> >>
> >> gcc version 9.2.1 20191025 (release) [ARM/arm-9-branch revision
> >> 277599] (15:9-2019-q4-0ubuntu1)
> >>
> >> Thank you very much!
> >>
> >> BR,
> >>
> >> Alan
> >>
> >> On 5/25/21, Miguel Wisintainer  >> <mailto:tcpipchip%40hotmail.com>
> >> <mailto:tcpipchip%40hotmail.com>> wrote:
> >> > Matias
> >> >
> >> > Me and Alan will investigate!
> >> >
> >> > Thank you so much!
> >> >
> >> > Enviado do Email<https://go.microsoft.com/fwlink/?LinkId=550986> para
> >> > Windows 10
> >> >
> >> >
> >>
> >
> 

Re: Nimble on U-blox Nina B112 (Nrf52832)

[Matias N.]

The problem with apps listed but not being able to run them is a common error 
(something worth adding
to the FAQ) I faced many times. It is due to not having support for BUILTIN 
apps on menuconfig (you need general
support as well as enabling NSH BUILTIN support). It is strange that the config 
is not functional though.

Best,
Matias

On Tue, May 25, 2021, at 19:51, Alan Carvalho de Assis wrote:
> Hi Matias and Miguel,
> 
> I just tried nimble on nrf52832-mdk board without success:
> 
> $ ./tools/configure.sh nrf52832-mdk:sdc
> $ make
> 
> It downloaded and compiled nimble for NuttX correctly, the nuttx.bin
> was about 314944 bytes.
> 
> When I drop this file inside DAPLINK disk it tries to flash and create
> the file FAIL.TXT with this content:
> 
> "The hex file cannot be decoded. Checksum calculation failure occurred."
> 
> Then I ran "make menuconfig" and enabled the "Intel HEX binary format"
> and after copying the nuttx.hex to DAPLINK disk the error disappeared.
> 
> Accessing the nsh terminal I can see the nimble binary, but it is not running:
> 
> NuttShell (NSH) NuttX-10.1.0-RC1
> nsh> ?
> help usage:  help [-v] []
> 
>   . cdecho  hexdump   mkdir pssource
> unset
>   [ cpexec  ifconfig  mkfatfs   pwd   test
> usleep
>   ? cmp   exit  ifdownmkrd  rmtime
> xd
>   basename  dirname   false ifup  mount rmdir true
>   break ddfree  kill  mvset   uname
>   cat   dfhelp  lsnslookup  sleep umount
> 
> Builtin Apps:
>   nimble  sh  nsh
> nsh> nimble
> nsh: nimble: command not found
> nsh> ifconfig
> bnep0   Link encap:UNSPEC at UP
> 
> nsh> nimble -h
> nsh: nimble: command not found
> nsh> nimble
> nsh: nimble: command not found
> nsh>
> 
> Initially I thought it was caused by recent update of the nimble stack
> on NuttX, but I moved to a commit previous to that update and still
> facing same error.
> 
> Matias, do you think it could be some issue with my crosscompiler?
> 
> I'm using the default ARM gcc from Ubuntu 20.04 gcc-arm-none-eabi package:
> 
> gcc version 9.2.1 20191025 (release) [ARM/arm-9-branch revision
> 277599] (15:9-2019-q4-0ubuntu1)
> 
> Thank you very much!
> 
> BR,
> 
> Alan
> 
> On 5/25/21, Miguel Wisintainer  > wrote:
> > Matias
> >
> > Me and Alan will investigate!
> >
> > Thank you so much!
> >
> > Enviado do Email para
> > Windows 10
> >
> >
> 

Re: Nimble on U-blox Nina B112 (Nrf52832)

[Matias N.]

Hi Miguel,
from the looks of it, it is a problem with NuttX's Bluetooth host-layer. 
There's an unhandled opcode which corresponds to "scan enable", which is not 
being recognized when the controller sends "command complete".

In my case I used nimBLE for host layer which works well. You can try to 
reproduce nrf52832-mdk's sdc config which AFAIK is configured to use nimBLE.
I know others use other stacks but I'm not sure which one.

Best,
Matias

On Sun, May 23, 2021, at 19:01, Miguel Wisintainer wrote:
> 
> 
> 
> Hello Brennan
> 
> Plans changed, i had spent a lot of time today on NRF52832 then i suspect 
> that should be a Softdevice Version (132)
> Then i ported the NRF52840 and repeated all the process, and finally that 
> error was removed.
> Look the boot
> 
> ABCDEG
> bt_initialize: btdev 0x47384
> bt_hci_cmd_create: opcode 0c03 param_len 0
> bt_buf_alloc: buf 0x20002c40 type 0 reserve 0
> bt_hci_cmd_create: buf 0x20002c40
> bt_buf_extend: buf 0x20002c40 len 3
> bt_hci_cmd_send_sync: opcode 0c03 len 3
> hci_tx_kthread: started
> bt_buf_addref: buf 0x20002c40 (old) ref 1 type 0
> hci_tx_kthread: Sending command 0c03 buf 0x20002c40 to driver
> bt_hci_send: passing CMD 3075 to softdevice
> bt_buf_release: buf 0x20002c40 ref 2 type 0
> bt_buf_release: Remaining references: 1
> on_hci: received CMD_COMPLETE from softdevice (opcode: 3075, status: 0x3)
> bt_buf_alloc: buf 0x20002c28 type 1 reserve 1
> bt_buf_extend: buf 0x20002c28 len 6
> bt_hci_receive: buf 0x20002c28 len 6
> priority_rx_work: list 0x20002c64
> priority_rx_work: buf 0x20002c28 type 1 len 6
> bt_buf_consume: buf 0x20002c28 len 2
> hci_cmd_complete: opcode 0c03
> bt_buf_consume: buf 0x20002c28 len 3
> hci_reset_complete: status 0
> bt_buf_addref: buf 0x20002c28 (old) ref 1 type 1
> bt_buf_release: buf 0x20002c40 ref 1 type 0
> bt_buf_release: Buffer freed: 0x20002c40
> bt_buf_release: buf 0x20002c28 ref 2 type 1
> bt_buf_release: Remaining references: 1
> bt_buf_release: buf 0x20002c40 ref 0 type 0
> bt_buf_release: Remaining references: 255
> bt_buf_release: buf 0x20002c28 ref 1 type 1
> bt_buf_release: Buffer freed: 0x20002c28
> bt_hci_cmd_create: opcode 1003 param_len 0
> bt_buf_alloc: buf 0x20002c28 type 0 reserve 0
> bt_hci_cmd_create: buf 0x20002c28
> bt_buf_extend: buf 0x20002c28 len 3
> bt_hci_cmd_send_sync: opcode 1003 len 3
> bt_buf_addref: buf 0x20002c28 (old) ref 1 type 0
> hci_tx_kthread: Sending command 1003 buf 0x20002c28 to driver
> bt_hci_send: passing CMD 4099 to softdevice
> bt_buf_release: buf 0x20002c28 ref 2 type 0
> bt_buf_release: Remaining references: 1
> on_hci: received CMD_COMPLETE from softdevice (opcode: 4099, status: 0x3)
> bt_buf_alloc: buf 0x20002c40 type 1 reserve 1
> bt_buf_extend: buf 0x20002c40 len 14
> bt_hci_receive: buf 0x20002c40 len 14
> priority_rx_work: list 0x20002c64
> priority_rx_work: buf 0x20002c40 type 1 len 14
> bt_buf_consume: buf 0x20002c40 len 2
> hci_cmd_complete: opcode 1003
> bt_buf_consume: buf 0x20002c40 len 3
> hci_cmd_complete: Unhandled opcode 1003
> bt_buf_addref: buf 0x20002c40 (old) ref 1 type 1
> bt_buf_release: buf 0x20002c28 ref 1 type 0
> bt_buf_release: Buffer freed: 0x20002c28
> bt_buf_release: buf 0x20002c40 ref 2 type 1
> bt_buf_release: Remaining references: 1
> bt_buf_release: buf 0x20002c28 ref 0 type 0
> bt_buf_release: Remaining references: 255
> read_local_features_complete: status 0
> bt_buf_release: buf 0x20002c40 ref 1 type 1
> bt_buf_release: Buffer freed: 0x20002c40
> bt_hci_cmd_create: opcode 1001 param_len 0
> bt_buf_alloc: buf 0x20002c40 type 0 reserve 0
> bt_hci_cmd_create: buf 0x20002c40
> bt_buf_extend: buf 0x20002c40 len 3
> bt_hci_cmd_send_sync: opcode 1001 len 3
> bt_buf_addref: buf 0x20002c40 (old) ref 1 type 0
> hci_tx_kthread: Sending command 1001 buf 0x20002c40 to driver
> bt_hci_send: passing CMD 4097 to softdevice
> bt_buf_release: buf 0x20002c40 ref 2 type 0
> bt_buf_release: Remaining references: 1
> on_hci: received CMD_COMPLETE from softdevice (opcode: 4097, status: 0x1)
> bt_buf_alloc: buf 0x20002c28 type 1 reserve 1
> bt_buf_extend: buf 0x20002c28 len 14
> bt_hci_receive: buf 0x20002c28 len 14
> priority_rx_work: list 0x20002c64
> priority_rx_work: buf 0x20002c28 type 1 len 14
> bt_buf_consume: buf 0x20002c28 len 2
> hci_cmd_complete: opcode 1001
> bt_buf_consume: buf 0x20002c28 len 3
> hci_cmd_complete: Unhandled opcode 1001
> bt_buf_addref: buf 0x20002c28 (old) ref 1 type 1
> bt_buf_release: buf 0x20002c40 ref 1 type 0
> bt_buf_release: Buffer freed: 0x20002c40
> bt_buf_release: buf 0x20002c28 ref 2 type 1
> bt_buf_release: Remaining references: 1
> bt_buf_release: buf 0x20002c40 ref 0 type 0
> bt_buf_release: Remaining references: 255
> read_local_ver_complete: status 0
> bt_buf_release: buf 0x20002c28 ref 1 type 1
> bt_buf_release: Buffer freed: 0x20002c28
> bt_hci_cmd_create: opcode 1009 param_len 0
> bt_buf_alloc: buf 0x20002c28 type 0 reserve 0
> bt_hci_cmd_create: buf 0x20002c28
> bt_buf_extend: buf 0x20002c28 len 3
> 

Re: CI stuck?

[Matias N.]

Thanks Brennan. So we will have to wait it out it seems.
In the meantime I looked around in the Apache builds list and found a thread 
from April
related to issues with GA. This thread included an analysis of which were the 
worst offenders
in CI time and we sadly came out on top. You can see a graph here as well:
https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status

I'm hoping that with CMake we can really improve our build time (that is the 
impression
I get locally as well).

Best,
Matias

On Tue, May 18, 2021, at 15:10, Brennan Ashton wrote:
> I saw this a few days ago when GitHub was having some issues and it looks
> like they have an active outage now as well that is recovering so I would
> suspect that.
> 
> https://www.githubstatus.com/
> 
> --Brennan
> 
> On Tue, May 18, 2021, 10:17 AM Matias N.  <mailto:matias%40imap.cc>> wrote:
> 
> > I noticed CI wasn't running jobs and found several runs from 8 hrs ago
> > which were stuck with "internal error" reports. I canceled those (event
> > though there weren't actually running) to see if the queued ones were
> > picked up (which doesn't seem to be the case). Anyone knows what could be
> > happening?
> >
> > Best,
> > Matias
> 

CI stuck?

[Matias N.]

I noticed CI wasn't running jobs and found several runs from 8 hrs ago which 
were stuck with "internal error" reports. I canceled those (event though there 
weren't actually running) to see if the queued ones were picked up (which 
doesn't seem to be the case). Anyone knows what could be happening?

Best,
Matias

Re: How to share code and Kconfig between different boards

[Matias N.]

I'm not sure if there's support for custom-board common-code (in the same sense 
as for in-tree boards). In any case, we've introduced an external/ directory 
for the purpose of adding external OS level code. Simply point nuttx/external 
symlink to your common directory.

Best,
Matias

On Wed, May 5, 2021, at 04:56, Frank-Christian Kruegel wrote:
> Hi.
> 
> I've created several board support packages, each in its own out-of-tree 
> directory. Ive configured them as custom boards with a relative path 
> like ...
> 
> +- nuttx
> +- apps
> +- company-boards
> |  +- board1
> |  +- board2
> |  +- board3
> +- company-apps
> |  +- app1
> |  +- app2
> ...
> 
> Works fine.
> 
> These boards have different microcontrollers from different 
> architectures, but they all have common hardware and configuration 
> items. It would be very helpful, it i could share code between the 
> boards like...
> 
> +- company-boards
> |  +- board1
> | +-Kconfig
> | +-src
> |  +- board2
> | +-Kconfig
> | +-src
> |  +- board3
> | +-Kconfig
> | +-src
> |  +- board-common  <-- NEW SHARED DIRECTORY
> | +-Kconfig
> | +-src
> 
> so I can reference the common codefrom each board. The shared code is 
> kernel code, custom drivers, driver initialisation etc, no application 
> logic, and since it is very specific to my hardware the nuttx tree is 
> not a good place for it.
> 
> How do I do it the correct way?
> 
> I've notices that the board Kconfig gets copied over to 
> nuttx/boards/dummy, so a mere "source ../../board-common/Kconfig" 
> doesn't work. I managed to make Kconfig work by using "source 
> $TOPDIR/../company-boards/board-common/Kconfig". Now I'm stuck with 
> makefiles.
> 
> Best regards
> 
> Frank-Christian
> 

Re: [VOTE] Apache NuttX 10.1.0 (incubating) RC0 release

[Matias N.]

Right, thanks.

I mentioned that this should be backported, not sure if the author saw it:
https://github.com/apache/incubator-nuttx/pull/3613

Best,
Matias

On Wed, Apr 28, 2021, at 10:46, Abdelatif Guettouche wrote:
> > Wasn't the stack restructure behind the 10.1 release?
> 
> Yes, but that change just exposed an old issue.
> 
> On Wed, Apr 28, 2021 at 2:45 PM Matias N.  <mailto:matias%40imap.cc>> wrote:
> >
> > Wasn't the stack restructure behind the 10.1 release?
> >
> > Best,
> > Matias
> >
> > On Wed, Apr 28, 2021, at 09:57, Gustavo Henrique Nihei wrote:
> > > Hi folks,
> > >
> > > I believe we should consider backporting the fix from this PR into the
> > > 10.1.0 release:
> > > https://github.com/apache/incubator-nuttx/pull/3614
> > >
> > > It fixes a side effect from the refactoring of the stack architecture
> > > due to a wrong stack alignment for RISC-V chips.
> > >
> > > Best regards,
> > > Gustavo.
> > >
> > > On Mon, Apr 26, 2021 at 5:58 AM  > > <mailto:Alin.Jerpelea%40sony.com> <mailto:Alin.Jerpelea%40sony.com>> 
> > > wrote:
> > > >
> > > > Hi Brenan,
> > > >
> > > > We still have the following commit that is not resolved and I think 
> > > > that it is better to hold the release until it is ready for emerge
> > > > https://github.com/apache/incubator-nuttx/pull/3601
> > > >
> > > > Regards
> > > > Alin
> > > >
> > > > -Original Message-
> > > > From: Alin Jerpelea mailto:jerpelea%40gmail.com> 
> > > > <mailto:jerpelea%40gmail.com>>
> > > > Sent: den 22 april 2021 16:14
> > > > To: dev@nuttx.apache.org <mailto:dev%40nuttx.apache.org> 
> > > > <mailto:dev%40nuttx.apache.org>
> > > > Subject: Re: [VOTE] Apache NuttX 10.1.0 (incubating) RC0 release
> > > >
> > > > It is perfect timing!
> > > >
> > > > I will cut RC1 with the fixes tonight
> > > >
> > > > Thanks
> > > > Alin
> > > >
> > > > On Thu, Apr 22, 2021, 16:12 Nathan Hartman  > > > <mailto:hartman.nathan%40gmail.com> 
> > > > <mailto:hartman.nathan%40gmail.com>> wrote:
> > > >
> > > > > On Thu, Apr 22, 2021 at 2:54 AM Alin Jerpelea  > > > > <mailto:jerpelea%40gmail.com> <mailto:jerpelea%40gmail.com>> wrote:
> > > > >
> > > > > > Hi Brennan,
> > > > > > I think that we should backport them and I will cut RC1 Thanks Alin
> > > > >
> > > > >
> > > > >
> > > > > If I'm not too late, I fixed up the ReleaseNotes a little bit in
> > > > > PR-3570
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 22, 2021, 08:30 Brennan Ashton
> > > > > > mailto:bashton%40brennanashton.com> 
> > > > > > <mailto:bashton%40brennanashton.com>>
> > > > > > wrote:
> > > > > >
> > > > > > > Alin,
> > > > > > > I think we should consider if we want to backport these into the
> > > > > > > 10.1.0 release.  I am still running more tests myself on all the
> > > > > > > hardware I have, but these seemed important to at least consider.
> > > > > > > https://urldefense.com/v3/__https://github.com/apache/incubator-nu
> > > > > > > ttx/pull/3586__;!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6ZgXhc6ZY2nV6G_l
> > > > > > > kT8Y8QJWo9Nv_bGYGjtgmeOc3mRLLOLl7Nw$
> > > > > > > https://urldefense.com/v3/__https://github.com/apache/incubator-nu
> > > > > > > ttx/pull/3580__;!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6ZgXhc6ZY2nV6G_l
> > > > > > > kT8Y8QJWo9Nv_bGYGjtgmeOc3mRKshpGeRw$
> > > > > > >
> > > > > > > --Brennan
> > > > > > >
> > > > > > > On Sun, Apr 18, 2021 at 7:06 AM Alin Jerpelea  > > > > > > <mailto:jerpelea%40gmail.com> <mailto:jerpelea%40gmail.com>>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Hello all,
> > > > > > > > Apache NuttX (Incubating) 10.1.0 RC0 has been 

Re: [VOTE] Apache NuttX 10.1.0 (incubating) RC0 release

[Matias N.]

Wasn't the stack restructure behind the 10.1 release?

Best,
Matias

On Wed, Apr 28, 2021, at 09:57, Gustavo Henrique Nihei wrote:
> Hi folks,
> 
> I believe we should consider backporting the fix from this PR into the
> 10.1.0 release:
> https://github.com/apache/incubator-nuttx/pull/3614
> 
> It fixes a side effect from the refactoring of the stack architecture
> due to a wrong stack alignment for RISC-V chips.
> 
> Best regards,
> Gustavo.
> 
> On Mon, Apr 26, 2021 at 5:58 AM  > wrote:
> >
> > Hi Brenan,
> >
> > We still have the following commit that is not resolved and I think that it 
> > is better to hold the release until it is ready for emerge
> > https://github.com/apache/incubator-nuttx/pull/3601
> >
> > Regards
> > Alin
> >
> > -Original Message-
> > From: Alin Jerpelea mailto:jerpelea%40gmail.com>>
> > Sent: den 22 april 2021 16:14
> > To: dev@nuttx.apache.org 
> > Subject: Re: [VOTE] Apache NuttX 10.1.0 (incubating) RC0 release
> >
> > It is perfect timing!
> >
> > I will cut RC1 with the fixes tonight
> >
> > Thanks
> > Alin
> >
> > On Thu, Apr 22, 2021, 16:12 Nathan Hartman  > > wrote:
> >
> > > On Thu, Apr 22, 2021 at 2:54 AM Alin Jerpelea  > > > wrote:
> > >
> > > > Hi Brennan,
> > > > I think that we should backport them and I will cut RC1 Thanks Alin
> > >
> > >
> > >
> > > If I'm not too late, I fixed up the ReleaseNotes a little bit in
> > > PR-3570
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > On Thu, Apr 22, 2021, 08:30 Brennan Ashton
> > > > mailto:bashton%40brennanashton.com>>
> > > > wrote:
> > > >
> > > > > Alin,
> > > > > I think we should consider if we want to backport these into the
> > > > > 10.1.0 release.  I am still running more tests myself on all the
> > > > > hardware I have, but these seemed important to at least consider.
> > > > > https://urldefense.com/v3/__https://github.com/apache/incubator-nu
> > > > > ttx/pull/3586__;!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6ZgXhc6ZY2nV6G_l
> > > > > kT8Y8QJWo9Nv_bGYGjtgmeOc3mRLLOLl7Nw$
> > > > > https://urldefense.com/v3/__https://github.com/apache/incubator-nu
> > > > > ttx/pull/3580__;!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6ZgXhc6ZY2nV6G_l
> > > > > kT8Y8QJWo9Nv_bGYGjtgmeOc3mRKshpGeRw$
> > > > >
> > > > > --Brennan
> > > > >
> > > > > On Sun, Apr 18, 2021 at 7:06 AM Alin Jerpelea  > > > > >
> > > > wrote:
> > > > > >
> > > > > > Hello all,
> > > > > > Apache NuttX (Incubating) 10.1.0 RC0 has been staged under [1]
> > > > > > and
> > > it's
> > > > > > time to vote on accepting it for release. If approved we will
> > > > > > seek final release approval from the IPMC. Voting will be open for 
> > > > > > 72hr.
> > > > > >
> > > > > > A minimum of 3 binding +1 votes and more binding +1 than binding
> > > > > > -1
> > > are
> > > > > > required to pass.
> > > > > >
> > > > > > The Apache requirements for approving a release can be found
> > > > > > here [3] "Before voting +1 [P]PMC members are required to
> > > > > > download the signed source code package, compile it as provided,
> > > > > > and test the resulting executable on their own platform, along
> > > > > > with also verifying that the package meets the requirements of the 
> > > > > > ASF policy on releases."
> > > > > >
> > > > > > A document to walk through some of this process has been
> > > > > > published on our project wiki and can be found here [4].
> > > > > >
> > > > > > [ ] +1 accept (indicate what you validated - e.g. performed the
> > > non-RM
> > > > > > items in [4])
> > > > > > [ ] -1 reject (explanation required)
> > > > > >
> > > > > > Thank you all,
> > > > > > Alin Jerpelea
> > > > > >
> > > > > > SCM Information:
> > > > > >   Release tag: nuttx-10.1.0-RC0
> > > > > >   Hash for the release incubating-nuttx tag:
> > > > > > f380c919f04d5ee88e0a83f5632cc83af503664f
> > > > > >   Hash for the release incubating-nuttx-apps tag:
> > > > > > 4348d91d1356335483089c3865282d80f13bedcd
> > > > > >
> > > > > > [1]
> > > https://urldefense.com/v3/__https://dist.apache.org/repos/dist/dev/inc
> > > ubator/nuttx/10.1.0-RC0/__;!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6ZgXhc6ZY
> > > 2nV6G_lkT8Y8QJWo9Nv_bGYGjtgmeOc3mRIUEGmDag$
> > > > > > [2]
> > > > >
> > > >
> > > https://urldefense.com/v3/__https://raw.githubusercontent.com/apache/i
> > > ncubator-nuttx/nuttx-10.1.0-RC0/ReleaseNotes__;!!JmoZiZGBv3RvKRSx!rb2n
> > > GxDcx1NeBVx6ZgXhc6ZY2nV6G_lkT8Y8QJWo9Nv_bGYGjtgmeOc3mRJlgBtnkg$
> > > > > > [3]
> > > > > > https://urldefense.com/v3/__https://www.apache.org/dev/release.h
> > > > > > tml*approving-a-release__;Iw!!JmoZiZGBv3RvKRSx!rb2nGxDcx1NeBVx6Z
> > > > > > gXhc6ZY2nV6G_lkT8Y8QJWo9Nv_bGYGjtgmeOc3mRJEiqu7OA$
> > > > > > [4]
> > > > >
> > > >
> > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/displa
> > > y/NUTTX/Validating*a*staged*Release__;Kysr!!JmoZiZGBv3RvKRSx!rb2nGxDcx
> > > 

Re: Mailing list usability

[Matias N.]

Most mailing list managers allow turning off sending emails (and also support 
digest mode).
If Apache uses mailman this should be possible to set via administrative 
interface once you
subscribe.

Best,
Matias

On Fri, Apr 23, 2021, at 06:34, Max Kriegleder wrote:
> Hi all,
> 
> I am an infrequent user of the mailing list, where I sometimes post 
> issues and even more rarely pull request. Hence, I would rather not 
> subscribe to the list because of spam (no pun intended!). However, I 
> find it quite difficult to use the mailing list for my purposes without 
> subscribing.
> 
> Lately, some of my messages apparently got lost. Also, I cannot answer 
> to specific messages on 
> https://www.mail-archive.com/dev@nuttx.apache.org/ as the 'Reply via 
> email to' button doesn't work in my browser (Firefox). I have tried 
> http://mail-archives.apache.org, which doesn't have a search and reply 
> functionality, and 
> https://lists.apache.org/list.html?dev@nuttx.apache.org, which seems 
> like the best alternative with search and reply functionality, but it 
> doesn't seem to be in sync with the mailing list (or takes longer to do 
> that).
> 
> Are there other alternatives for my use case or do I really need to 
> subscribe to the mailing list?
> 
> Thanks!
> Max
> 
> 

Re: Ethernet cable (network interface availability) detection

[Matias N.]

Contributing and Glossary should be left at the end. I would put it right 
before Guides.
Maybe one file with subsections is enough for now.

On Wed, Apr 14, 2021, at 11:03, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> I can submit a PR with it.
> 
> Where the FAQ should be added? Is it ok to be after Glossary?
> 
> BR,
> 
> Alan
> 
> On 4/14/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > We do not yet have a "Networking" section under "OS Components" where a lot
> > of network related stuff should
> > be. However, I have thought about a FAQ section for having brief
> > question/answers where this kind of information can be (until a proper
> > section is added). If anyone wants to start this it would be welcome. It
> > should be categorized by topic.
> >
> > Best,
> > Matias
> >
> > On Wed, Apr 14, 2021, at 10:40, Alan Carvalho de Assis wrote:
> >> I think this is kind of question that should be in some FAQ or in our
> >> documentation, it is asked often...
> >>
> >> Is there some only place in our Documentation where we could include it?
> >>
> >> BR,
> >>
> >> Alan
> >>
> >> On 4/14/21, Gregory Nutt  >> <mailto:spudaneco%40gmail.com>
> >> <mailto:spudaneco%40gmail.com>> wrote:
> >> > Normally this is done using the PHY driver at
> >> > nuttx/drivers/net/phy_notify.c that provides PHY-related events to
> >> > applications via signals.  It requires board-level support to handle
> >> > PHY
> >> > interrupts.
> >> >
> >> > The network monitor thread at apps/netutils/netinit (see
> >> > CONFIG_NETINIT_MONITOR) will handle taking the network down if the
> >> > cable
> >> > is unplugged and bringing it back up when the cable is restored.
> >> >
> >> > On 4/14/2021 7:24 AM, Flavio Castro Alves Filho wrote:
> >> >> Hello,
> >> >>
> >> >> I am implementing an application using NuttX where I need to detect if
> >> >> the network cable is plugged or not in my board.
> >> >>
> >> >> Today I implemented the function netlib_getifstatus(), which automates
> >> >> the read of SIOCGIFFLAGS, similar to what we have in Linux.
> >> >>
> >> >> But, in my tests, it seems that it is not working correctly.
> >> >>
> >> >> What is the recommended approach to have this feature implemented?
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Flavio
> >> >>
> >> >
> >> >
> >>
> >
> 

Re: Ethernet cable (network interface availability) detection

[Matias N.]

We do not yet have a "Networking" section under "OS Components" where a lot of 
network related stuff should
be. However, I have thought about a FAQ section for having brief 
question/answers where this kind of information can be (until a proper section 
is added). If anyone wants to start this it would be welcome. It should be 
categorized by topic.

Best,
Matias

On Wed, Apr 14, 2021, at 10:40, Alan Carvalho de Assis wrote:
> I think this is kind of question that should be in some FAQ or in our
> documentation, it is asked often...
> 
> Is there some only place in our Documentation where we could include it?
> 
> BR,
> 
> Alan
> 
> On 4/14/21, Gregory Nutt mailto:spudaneco%40gmail.com>> 
> wrote:
> > Normally this is done using the PHY driver at
> > nuttx/drivers/net/phy_notify.c that provides PHY-related events to
> > applications via signals.  It requires board-level support to handle PHY
> > interrupts.
> >
> > The network monitor thread at apps/netutils/netinit (see
> > CONFIG_NETINIT_MONITOR) will handle taking the network down if the cable
> > is unplugged and bringing it back up when the cable is restored.
> >
> > On 4/14/2021 7:24 AM, Flavio Castro Alves Filho wrote:
> >> Hello,
> >>
> >> I am implementing an application using NuttX where I need to detect if
> >> the network cable is plugged or not in my board.
> >>
> >> Today I implemented the function netlib_getifstatus(), which automates
> >> the read of SIOCGIFFLAGS, similar to what we have in Linux.
> >>
> >> But, in my tests, it seems that it is not working correctly.
> >>
> >> What is the recommended approach to have this feature implemented?
> >>
> >> Best regards,
> >>
> >> Flavio
> >>
> >
> >
> 

Re: ESP32S?

[Matias N.]

I looked around to get a sense of V4L complexity. This [1] (very old) article 
here gives a good broad idea
and it wouldn't seem too difficult to have something like this in NuttX, 
focusing on minimal initial support.
It could be an upper-half driver which expects the lower-half to provide the 
callbacks mentioned in
this article while at the same time registering a character driver.

On the other hand, looking at more up-to-date documentation [2] I see V4L 
nowadays is huge and
full of features. Maybe we can start with [1] in mind.

For the time being I will most likely start creating an ad-hoc driver for this 
camera (directly registering the chardev)
which could then be evolved into a v4l-like lower-half implementation.

Best,
Matias

[1] http://lwn.net/Articles/203924/
[2] http://www.linuxtv.org/downloads/video4linux/API/V4L2_API

On Sat, Apr 3, 2021, at 09:45, Marco wrote:
> Am Sa., 3. Apr. 2021 um 13:40 Uhr schrieb Matias N.  <mailto:matias%40imap.cc>>:
> 
> > Do you see a need for something more? Not sure what other aspects of video
> > handling require a specific solution.
> 
> When talking about cameras you also need to specify the camera output
> format.
> This is mostly a raw format YUV for mipi cameras, but can also be e.g.
> mjpeg or h264 when supported by the camera.
> A lot of cameras providing different output formats nowadays.
> 
> According to your map idea.
> Your implementation (the driver) must ensure that data capturing is locked
> during accessing the buffer by the user space task.
> I don't know how the driver must handle this in terms of communication with
> the camera.
> When working with v4l2 you need at least 2 registered framebuffer for that
> reason.
> One is locked for capturing and the other for processing by the application.
> 
> I think there are other things to consider, but that's what I have
> currently in my mind.
> 
> >
> 
> 
> > Best,
> > Matias
> >
> 

Re: ESP32S?

[Matias N.]

My idea was to do something similar to the framebuffer device, using map to 
access image data and ioctl's to control camera parameters and wait for a 
frame. Is this mostly how v4l works?
Do you see a need for something more? Not sure what other aspects of video 
handling require a specific solution.

Best,
Matias

Re: ESP32S?

[Matias N.]

Thanks Alan. I'll probably give it a try.

Best,
Matias

On Fri, Apr 2, 2021, at 20:43, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> No, AFAIK there is nobody working on camera support.
> 
> BR,
> 
> Alan
> 
> On 4/2/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > I've looked into this a bit further. It seems NuttX supports the OV2640
> > command interface (via I2C) with video/ov2640.c driver. However, the data
> > transfer is handled separately. From what I read, on ESP32, the I2S
> > peripheral is used to handle the data transfer using DMA. So this means that
> > an ESP32 specific driver would need to be written so that it transfers image
> > data using I2S.
> >
> > Any chance anyone already did this? Seems a bit complex.
> >
> > Best,
> > Matias
> >
> > On Thu, Apr 1, 2021, at 12:18, Matias N. wrote:
> >> I'm merely experimenting myself but as a test I will try to read the
> >> camera and offer images via a webserver. I believe the camea is supported
> >> in NuttX already.
> >>
> >> Best,
> >> Matias
> >>
> >> On Thu, Apr 1, 2021, at 11:40, Grr wrote:
> >>> I have a bunch of those modules I plan to use as security cameras and
> >>> automation control nodes, preferably under NuttX, so I have a couple of
> >>> questions:
> >>>
> >>> 1-How far away is the camera support?
> >>>
> >>> 2-How about WiFi?
> >>>
> >>> TIA
> >>> Grr
> >>>
> >>>
> >>> El jue, 1 abr 2021 a las 8:34, Matias N. ( >>> <mailto:matias%40imap.cc>>) escribió:
> >>>> __
> >>>> Yeah, I was under the impression it was single core due to the "S" as in
> >>>> the figure below. But you're right
> >>>> that ESP32-S is not a valid Espressif part number. This is the module I
> >>>> was refering to:
> >>>>
> >>>> It makes sense that it is actually the normal dual-core chip inside.
> >>>>
> >>>> Best,
> >>>> Matias
> >>>>
> >>>> On Thu, Apr 1, 2021, at 11:23, Sara da Cunha Monteiro de Souza wrote:
> >>>>> Besides that,
> >>>>> here we can see all the "official" series of Espressif:
> >>>>> https://www.espressif.com/sites/default/files/documentation/espressif_products_ordering_information_en.pdf
> >>>>> Or a more simple and interactive way, using the product selector:
> >>>>> https://products.espressif.com/#/product-selector
> >>>>> So, ESP32S or ESP32-S modules are from third party.
> >>>>>
> >>>>>
> >>>>> Em qui., 1 de abr. de 2021 às 11:14, Sara da Cunha Monteiro de Souza
> >>>>>  >>>>> <mailto:saramonteirosouza44%40gmail.com>> escreveu:
> >>>>>> Hi Matias,
> >>>>>> These devkits with cameras and sd card slot with ESP32S modules are
> >>>>>> third party modules and the S doesn't necessarily stand for "single
> >>>>>> core".
> >>>>>> I recommend take a look at this post to see the differences between
> >>>>>> different devkits:
> >>>>>> https://diyprojects.io/esp32-cam-which-model-choose-esp-eye-aithinker-ttgo-tcamera-m5stack/#.YFJoh-1KhH4.
> >>>>>> FY, this is how you can interpret Part numbers of ESP32 modules :
> >>>>>> image.png
> >>>>>> And if you don't have this information and you have the devkit on
> >>>>>> hands, you can use esptool to discover this information from your
> >>>>>> ESP32
> >>>>>> by using the following command:
> >>>>>>
> >>>>>> esptool.py --port /dev/ttyUSB0 chip_id
> >>>>>>
> >>>>>> You should have something like this:
> >>>>>>
> >>>>>> Serial port /dev/ttyUSB0 Connecting. Detecting chip type... ESP32
> >>>>>> Chip is ESP32-D0WDQ6 (revision 1) Features: WiFi, BT, Dual Core,
> >>>>>> 240MHz, VRef calibration in efuse, Coding Scheme None Crystal is 40MHz
> >>>>>> MAC: 9c:9c:1f:ea:6e:5c
> >>>>>>
> >>>>>> It can be very useful to know other information as you can see, for
> >>>>>> example, the chip revision.
> >>>>>>
> >>>>>> But regarding the support for this DevKit, the board implementation is
> >>>>>> relatively simple, but we still
> >>>>>> don't have the I2S driver for the camera.
> >>>>>>
> >>>>>>
> >>>>>> Em qui., 1 de abr. de 2021 às 10:49, Matias N.  >>>>>> <mailto:matias%40imap.cc>>
> >>>>>> escreveu:
> >>>>>>> Hi,
> >>>>>>> I'm considering buying one of those ESP32 modules which includes the
> >>>>>>> camera and SD card and I noticed it seems it comes with a single-core
> >>>>>>> variant of the typical ESP32 module.
> >>>>>>> I looked around NuttX and I think there's no explicit support yet but
> >>>>>>> I imagine it shouldn't be hard to add, right?
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Matias
> >>>>>
> >>>>> *Attachments:*
> >>>>>  * image.png
> >>>>
> >>>
> >>> *Attachments:*
> >>>  * image.png
> >>>  * image.png
> >>
> >>
> >> *Attachments:*
> >>  * image.png
> >>  * image.png
> >
> 

ESP32S?

[Matias N.]

Hi,
I'm considering buying one of those ESP32 modules which includes the camera and 
SD card and I noticed it seems it comes with a single-core variant of the 
typical ESP32 module.
I looked around NuttX and I think there's no explicit support yet but I imagine 
it shouldn't be hard to add, right?

Best,
Matias

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

We're already using ccache, but it is used intra run (ie, for arm-01, sim, 
etc). We tried persisting it across runs (from different PRs) but it seems 
there was issues with that.

What I suggest means we have one cache pero board/config. The scripting 
shouldn't be too difficult but it requires much more storage. Not sure if we 
would be able to store it all.

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

On Wed, Mar 31, 2021, at 11:26, Nathan Hartman wrote:
> I wonder if we need the tests to be a bit smarter in restricting what they
> test.
> 
> For example if the change only affects files in arch/arm/stm32h7 then only
> stm32h7 configs will be tested.

That can be difficult to do. A header might define something that ultimately 
reaches
hardware-specific code and make it fail.

My reasoning with the per board+config ccache is that it should more or less 
detect
something like this without hardcoding any rules: only files that are impacted 
by any
changes in the current PR would be rebuilt, any other file would be 
instantanously 
obtained from cache. This is mostly what I observe when I use ccache locally: 
going
to make menuconfig indeed rebuilds everything, but it is quite faster than an 
initial
build with 100% miss rate.

We should also eventually check if we're not adding some cache-breaking 
definition
(encode the date of the build in nuttx/config.h for example).

> 
> Also if the PR only affects Documentation then there's no need to build any
> board configs.

That's already being done. Any chances contained under Documentation/ only
rebuild docs.

> 
> Changes in areas like sched will affect all configs so the tests cannot be
> restricted. But it seems to me that most PRs are in arch, so even if some
> PRs still have to run the full set of tests, it will still be a big
> optimization.
> 
> Nathan
> 

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

On Tue, Mar 30, 2021, at 21:44, Brennan Ashton wrote:
> We were sharing cache but ran into some strange issues with collisions and
> disabled it unfortunately.

Do you remember the exact issue?

What if we had one ccache cache per : run shared across runs? 
(not sure if that would be too much storage) So it would be like if you are 
locally running under the same conditions and just rebuild. If the specific 
board/config combination is not affected by the current PR (no relevant config 
is changed nor the changed code is part of that build) it should be fairly fast.

Anyway, I see the "other" includes a very long list of very different 
platforms. Maybe splitting it into avr, risc-v, xtensa could help?

Best,
Matias

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

On Wed, Mar 31, 2021, at 09:33, Abdelatif Guettouche wrote:
> > Also, I agree about simplifying macOS build if it will help while we get to 
> > q better situation.
> 
> We can have a special test list for macOS that includes just a few
> configs from the simulator and other chips.

I think just dropping one of the three runs we can have a large gain. The 
"other" list of macOS jobs takes about 143m, while the other two ("sim" and 
"arm-12") take about 84m. For reference, all linux jobs take about 30-50m. 
Linux's "other" is also longer (60m). Maybe we can try to fine tune that one as 
well (find faster toolchains, remove some configs, etc).

Best,
Matias

All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

Then situation is a bit better now, since we did not continue to submit as many 
Para. However, the builds are still lagging (there are ones from 20hr ago 
running). I see many of the new "cancelling duplicates" jobs queued but they 
have not yet run.
I have cancelled a few myself. In case I cancelled a running one by mistake, 
please apologize. I would suggest we give the system a while to catch up.

In the meantime, if you are force pushing a PR, please rebase first.

Also, I agree about simplifying macOS build if it will help while we get to q 
better situation.

Best,
Matias

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

Most likely a single very powerful machine could be actually quite faster than 
GH
since we could parallelize much harder and have all resources to ourselves.
The problem is that rolling our own can be quite a pain to maintain IMHO
(unless someone has access to some powerful high-availabilty spare machine).
We would also have to redo all CI handling since it wouldn't be GitHub's.

I also looked at alternative CI systems but I think we will consume free 
credits easily.

Indeed the macOS build is really slow. But if we start to cut on tests we loose
the assurance given by the automated testing. 

Maybe we could try to share the ccache across runs. I personally have never 
found
a ccache collision nor any issue (I have ccache enabled for years). I remember 
we
tried this but I'm not sure if the result was inconclusive. 

Anyway, cancelling previous flows should get us back to a better place where we 
"only"
wait for ~1.5hr for the build to complete.

Best,
Matias

On Tue, Mar 30, 2021, at 17:17, Alan Carvalho de Assis wrote:
> We definitely need better server to support the CI, it doesn't have
> processing power enough to run the CI when there are more than 5 PRs.
> It doesn't scale well.
> 
> Also I think we could keep only one test for MacOS because it is too
> slow! Normally MacOS delays more than 2h to complete.
> 
> Maybe we could create some distributed CI farm and we could include
> low power hardware (i.e. Raspberry Pi boards) running from our homes
> to help it, hehehe.
> 
> Suggestions are welcome!
> 
> BR,
> 
> Alan
> 
> On 3/30/21, Nathan Hartman  <mailto:hartman.nathan%40gmail.com>> wrote:
> > On Tue, Mar 30, 2021 at 3:30 PM Matias N.  > <mailto:matias%40imap.cc>> wrote:
> >>
> >> It appears we overwhelmed CI. There are a couple of running jobs (notably
> >> one is a macOS run which is taking about 2hrs as of now) but they are for
> >> PRs from 12hs ago at least. There are a multitude of queued runs for many
> >> recent PRs. The problem is that new runs (from force pushes) do not cancel
> >> previous runs so they remain queued apparently.
> >
> > Ouch!
> >
> >> I will see what can be done to have new pushes cancel new pending runs. In
> >> the meantime we may have to manually cancel all queued workflows. Not sure
> >> if there's a mass cancel to be done.
> >
> > Thanks for looking into it. Yes, it would be a good thing if new force
> > pushes could cancel in-progress runs.
> >
> > Thanks,
> > Nathan
> >
> 

Re: All PRs stuck in "Queued -- Waiting to run this check..."

[Matias N.]

It appears we overwhelmed CI. There are a couple of running jobs (notably one 
is a macOS run which is taking about 2hrs as of now) but they are for PRs from 
12hs ago at least. There are a multitude of queued runs for many recent PRs. 
The problem is that new runs (from force pushes) do not cancel previous runs so 
they remain queued apparently.

I will see what can be done to have new pushes cancel new pending runs. In the 
meantime we may have to manually cancel all queued workflows. Not sure if 
there's a mass cancel to be done.

Best,
Matias

On Tue, Mar 30, 2021, at 15:33, Nathan Hartman wrote:
> Hi,
> 
> Does anyone know why the GitHub PR prechecks don't seem to be running?
> 
> In particular, it seems that these four:
> 
> * Build / Fetch-Source (pull_request)
> * Build Documentation / build-html (pull_request)
> * Check / check (pull_request)
> * Lint / YAML (pull_request)
> 
> are all stuck in "Queued -- Waiting to run this check..."
> 
> for all PRs.
> 
> Thanks,
> Nathan
> 

Re: mqtt library on NuttX?

[Matias N.]

Maybe we should create an issue to track that? Sounds like an important problem.

On Sun, Mar 28, 2021, at 17:36, Abdelatif Guettouche wrote:
> > Abdelatif, let me know if you want me to prepare the PR. If licensing is 
> > taken care of, I can simply
> place Apache header and submit it myself, right?
> 
> I did prepare the PR, however testing with latest master I found out
> that fcntl isn't working on socket descriptors.. I'm suspecting
> https://github.com/apache/incubator-nuttx/pull/2899 & the follow up
> https://github.com/apache/incubator-nuttx/pull/3045
> The little debugging I did showed that fs_getfilep is returning a bad
> file structure in case of a socket descriptor.
> 
> I still didn't debug it any further.
> 
> On Sun, Mar 28, 2021 at 11:33 PM Matias N.  <mailto:matias%40imap.cc>> wrote:
> >
> > Abdelatif, let me know if you want me to prepare the PR. If licensing is 
> > taken care of, I can simply
> > place Apache header and submit it myself, right?
> >
> > Best,
> > Matias
> >
> > On Thu, Mar 25, 2021, at 19:05, Abdelatif Guettouche wrote:
> > > Espressif has already signed an SGA, licensing shouldn't be an issue.
> > > That said, I can submit it later if you guys want.
> > > One question: where do we put the library?
> > > A new "mqtt" folder?
> > >
> > > On Thu, Mar 25, 2021 at 10:59 PM Alan Carvalho de Assis
> > > mailto:acassis%40gmail.com> 
> > > <mailto:acassis%40gmail.com>> wrote:
> > > >
> > > > On 3/25/21, Matias N. mailto:matias%40imap.cc> 
> > > > <mailto:matias%40imap.cc>> wrote:
> > > > >
> > > > >
> > > > > On Thu, Mar 25, 2021, at 15:21, Alan Carvalho de Assis wrote:
> > > > >> Hi Matias,
> > > > >>
> > > > >> On 3/25/21, Matias N. mailto:matias%40imap.cc> 
> > > > >> <mailto:matias%40imap.cc> <mailto:matias%40imap.cc>> wrote:
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Mar 25, 2021, at 15:02, Abdelatif Guettouche wrote:
> > > > >> >> Someone was asking the other day on Linkedin, so I pushed this:
> > > > >> >> https://github.com/Ouss4/MQTTC_NuttX
> > > > >> >
> > > > >> > Great, thanks! I see the wrapper app is quite simple. Could we 
> > > > >> > submit
> > > > >> > it
> > > > >> > to nuttx apps repo?
> > > > >> >
> > > > >>
> > > > >> Yes, please!
> > > > >
> > > > > The copyright header is from Espressif so I guess it would be best if
> > > > > someone at Espressif can submit it
> > > > > (not sure if it would require a SGA).
> > > > >
> > > >
> > > > It is not a code from IDF, it was Abdelatif whom added it :-)
> > > >
> > > > But you are right, maybe it is better if we submit it because there is
> > > > an Espressif copyright on it.
> > > >
> > > > BR,
> > > >
> > > > Alan
> > >
> 

Re: mqtt library on NuttX?

[Matias N.]

Abdelatif, let me know if you want me to prepare the PR. If licensing is taken 
care of, I can simply
place Apache header and submit it myself, right?

Best,
Matias

On Thu, Mar 25, 2021, at 19:05, Abdelatif Guettouche wrote:
> Espressif has already signed an SGA, licensing shouldn't be an issue.
> That said, I can submit it later if you guys want.
> One question: where do we put the library?
> A new "mqtt" folder?
> 
> On Thu, Mar 25, 2021 at 10:59 PM Alan Carvalho de Assis
> mailto:acassis%40gmail.com>> wrote:
> >
> > On 3/25/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > >
> > >
> > > On Thu, Mar 25, 2021, at 15:21, Alan Carvalho de Assis wrote:
> > >> Hi Matias,
> > >>
> > >> On 3/25/21, Matias N. mailto:matias%40imap.cc> 
> > >> <mailto:matias%40imap.cc>> wrote:
> > >> >
> > >> >
> > >> > On Thu, Mar 25, 2021, at 15:02, Abdelatif Guettouche wrote:
> > >> >> Someone was asking the other day on Linkedin, so I pushed this:
> > >> >> https://github.com/Ouss4/MQTTC_NuttX
> > >> >
> > >> > Great, thanks! I see the wrapper app is quite simple. Could we submit
> > >> > it
> > >> > to nuttx apps repo?
> > >> >
> > >>
> > >> Yes, please!
> > >
> > > The copyright header is from Espressif so I guess it would be best if
> > > someone at Espressif can submit it
> > > (not sure if it would require a SGA).
> > >
> >
> > It is not a code from IDF, it was Abdelatif whom added it :-)
> >
> > But you are right, maybe it is better if we submit it because there is
> > an Espressif copyright on it.
> >
> > BR,
> >
> > Alan
> 

Re: userspace assert?

[Matias N.]

Yes, it seems that is the issue.
I will make a PR to remove those lines (I tried removing them and works as 
expected).

Best,
Matias

On Sat, Mar 27, 2021, at 16:44, Gregory Nutt wrote:
> 
> > Or maybe is sim's up_assert() wrong to exit simulation? Thinking about it, 
> > doing up_assert() (which would just print the error) and exit() would 
> > indeed exit the app only.
> 
> What do you mean by exit the simulation.  It should exit the task that 
> caused the assertion but the simulation should continue to run.
> 
> Hmm.. so yes, you are right.  the up_assert() implementation is wrong.  
> Compare to arch/arm/src/armv-7m/arm_assert.c.  For crash occurs only if:
> 
> 342 static void _up_assert(void)
> 343 {
> 344   /* Flush any buffered SYSLOG data */
> 345
> 346   syslog_flush();
> 347
> 348   /* Are we in an interrupt handler or the idle task? */
> 349
> 350   if (CURRENT_REGS || (running_task())->flink == NULL)
> 351 {
> 
> Otherwise, it just returns (eventually) to __assert which calls exit().  
> That is the correct behavior.
> 
> 

Re: userspace assert?

[Matias N.]

Or maybe is sim's up_assert() wrong to exit simulation? Thinking about it, 
doing up_assert() (which would just print the error) and exit() would indeed 
exit the app only.

On Sat, Mar 27, 2021, at 16:30, Matias N. wrote:
> I was using assert in an app (testing on sim) and realized the sim exited 
> upon hitting the assert. From the code I see it calls into up_assert() (which 
> would also be a violation of OS/Userspace separation AFAIK).
> What about writing a similar simple function that only sends the message to 
> syslog and does exit() so the app exists?
> This would only be for lowercase assert() of course.
> 
> Best,
> Matias

userspace assert?

[Matias N.]

I was using assert in an app (testing on sim) and realized the sim exited upon 
hitting the assert. From the code I see it calls into up_assert() (which would 
also be a violation of OS/Userspace separation AFAIK).
What about writing a similar simple function that only sends the message to 
syslog and does exit() so the app exists?
This would only be for lowercase assert() of course.

Best,
Matias

Re: mqtt library on NuttX?

[Matias N.]

On Thu, Mar 25, 2021, at 15:21, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> On 3/25/21, Matias N. mailto:matias%40imap.cc>> wrote:
> >
> >
> > On Thu, Mar 25, 2021, at 15:02, Abdelatif Guettouche wrote:
> >> Someone was asking the other day on Linkedin, so I pushed this:
> >> https://github.com/Ouss4/MQTTC_NuttX
> >
> > Great, thanks! I see the wrapper app is quite simple. Could we submit it
> > to nuttx apps repo?
> >
> 
> Yes, please!

The copyright header is from Espressif so I guess it would be best if someone 
at Espressif can submit it
(not sure if it would require a SGA).

> >>
> >> > Regarding SSL, is it required to build? Maybe it can be initially
> >> > integrated without
> >> SSL support?
> >>
> >> You can use it with no SSL support.  But MQTT-C supports multiple
> >> crypto libraries, mbedTLS being one of them, I guess it was
> >> contributed by Yamamoto.  I used mbedTLS + MQTT-C before as well.
> >
> > Sure, if the mbedTLS PR gets finished it would be a good combination.
> >
> 
> Yes, I don't see Brennan around during this and last week, I hope he is fine!
> 
> BR,
> 
> Alan
> 

Re: mqtt library on NuttX?

[Matias N.]

On Thu, Mar 25, 2021, at 15:02, Abdelatif Guettouche wrote:
> Someone was asking the other day on Linkedin, so I pushed this:
> https://github.com/Ouss4/MQTTC_NuttX

Great, thanks! I see the wrapper app is quite simple. Could we submit it
to nuttx apps repo?

> 
> > Regarding SSL, is it required to build? Maybe it can be initially 
> > integrated without
> SSL support?
> 
> You can use it with no SSL support.  But MQTT-C supports multiple
> crypto libraries, mbedTLS being one of them, I guess it was
> contributed by Yamamoto.  I used mbedTLS + MQTT-C before as well.

Sure, if the mbedTLS PR gets finished it would be a good combination.

Best,
Matias

> 
> On Thu, Mar 25, 2021 at 6:23 PM Matias N.  <mailto:matias%40imap.cc>> wrote:
> >
> > Great, maybe some draft PR could be submitted?
> >
> > Regarding SSL, is it required to build? Maybe it can be initially 
> > integrated without
> > SSL support?
> >
> > Best,
> > Matias
> >
> > On Thu, Mar 25, 2021, at 14:13, Alan Carvalho de Assis wrote:
> > > Hi Matias,
> > >
> > > Yes, some people here are using mqtt-c on NuttX and it is working fine.
> > >
> > > It needs some clean-up to be submitted to apps/
> > >
> > > BR,
> > >
> > > Alan
> > >
> > > On 3/25/21, Matias N. mailto:matias%40imap.cc> 
> > > <mailto:matias%40imap.cc>> wrote:
> > > > Hi,
> > > > I was wondering if anyone is already using some MQTT library on NuttX. 
> > > > I see
> > > > many options some of which look like potential candidates for easy 
> > > > porting,
> > > > but if anyone already went through this process I would appreciate some
> > > > feedback.
> > > >
> > > > Thanks,
> > > > Matias
> > >
> 

Re: mqtt library on NuttX?

[Matias N.]

Great, maybe some draft PR could be submitted?

Regarding SSL, is it required to build? Maybe it can be initially integrated 
without
SSL support?

Best,
Matias

On Thu, Mar 25, 2021, at 14:13, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> Yes, some people here are using mqtt-c on NuttX and it is working fine.
> 
> It needs some clean-up to be submitted to apps/
> 
> BR,
> 
> Alan
> 
> On 3/25/21, Matias N. mailto:matias%40imap.cc>> wrote:
> > Hi,
> > I was wondering if anyone is already using some MQTT library on NuttX. I see
> > many options some of which look like potential candidates for easy porting,
> > but if anyone already went through this process I would appreciate some
> > feedback.
> >
> > Thanks,
> > Matias
> 

mqtt library on NuttX?

[Matias N.]

Hi,
I was wondering if anyone is already using some MQTT library on NuttX. I see 
many options some of which look like potential candidates for easy porting, but 
if anyone already went through this process I would appreciate some feedback.

Thanks,
Matias

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

On Wed, Mar 24, 2021, at 19:27, Byron Ellacott wrote:
> Here's what I found in libc that would need task (thread) specific data:
> 
>   - libs/libc/misc/lib_umask.c has g_mask
>   - libs/libc/libgen/lib_dirname.c and libs/libc/libgen/lib_basename each
> have a g_retchar
>   - libs/libc/syslog/lib_setlogmask.c has g_syslog_mask (and a comment
> describing this issue)
>   - libs/libc/pwd/* uses either g_passwd and g_passwd_buffer or g_pwd and
> g_buf
>   - libs/libc/grp/* uses a similar pair for group data
>   - libs/libc/unistd/lib_getopt.c we know of, it has four words of global
> data
>   - libs/libc/time/lib_localtime.c uses g_tm and may need per-task timezone
> settings
>   - libs/libc/netdb/lib_netdb.c specifies h_errno as a global
>   - libs/libc/netdb/lib_gethostbyname2.c  and lib_gethostbyaddr.c use
> g_hostent and g_hostbuffer
>   - libs/libc/stdlib/lib_srand.c uses a variety of globals depending on
> build options
>   - libs/libc/string/lib_strtok.c uses g_saveptr

Thanks for this list, I will update the issue and make it into a task list.

> 
> Statically allocating a TLS key for each module would consume around 11
> keys in each task. Dynamically allocated TLS keys cannot ever be released,
> because these are globals handed over to user code with no indication when
> they're no longer needed. It may be better to have an additional static
> element in tls_info_s pointing to a heap-allocated structure containing the
> libc globals. Functionally this is the same as a statically reserved TLS
> key, but it's clearer what it's for.

I think that is more or less the idea discussed in the PR. It will be done 
later on.

Best,
Matias

> -- 
> Byron
> 
> On Thu, Mar 25, 2021 at 12:51 AM Gregory Nutt  <mailto:spudaneco%40gmail.com>> wrote:
> 
> > On 3/24/2021 8:38 AM, Matias N. wrote:
> > > So, if I follow correctly, we could maybe have one TLS pointer pointing
> > to a struct of pointers, one per each group of globals (one of this groups,
> > would be the set of variables used by getopt()), like:
> > >
> > > struct task_globals_s
> > > {
> > >struct getopt_globals_s *getopt_globals;
> > >/* ...others */
> > > };
> > >
> > > Then getopt globals would only be allocated once for each task, and only
> > when getopt() is called.
> > >
> > > Something like that?
> >
> > Yes, that is a possibility.  But that is already implemented just as you
> > describe as POSIX thread-specific data.
> >
> > The TLS data structure is defined in include/nuttx/tls.h as following.
> > it is just an array of pointer size things and the errno variable.
> >
> > struct tls_info_s
> > {
> > #if CONFIG_TLS_NELEM > 0
> >uintptr_t tl_elem[CONFIG_TLS_NELEM]; /* TLS elements */
> > #endif
> >int tl_errno;/* Per-thread error number */
> > };
> >
> > This structure lies at the "bottom" of stack of every thread in user space.
> >
> > The standard pthread_getspecific() is then implemented as:
> >
> > FAR void *pthread_getspecific(pthread_key_t key)
> > {
> >return (FAR void *)tls_get_value((int)key);
> > }
> >
> > Where
> >
> > uintptr_t tls_get_value(int tlsindex)
> > {
> >FAR struct tls_info_s *info;
> >uintptr_t ret = 0;
> >
> >DEBUGASSERT(tlsindex >= 0 && tlsindex < CONFIG_TLS_NELEM);
> >if (tlsindex >= 0 && tlsindex < CONFIG_TLS_NELEM)
> >  {
> >/* Get the TLS info structure from the current threads stack */
> >
> >info = up_tls_info();
> >DEBUGASSERT(info != NULL);
> >
> >/* Get the element value from the TLS info. */
> >
> >ret = info->tl_elem[tlsindex];
> >  }
> >
> >return ret;
> > }
> >
> > The POSIX interface supports a pthread_key_create() to manage the indexing.
> >
> >
> >
> >
> 

Re: USB Host and Bluetooth "dongle".

[Matias N.]

Not entirely sure about your scenario here, but if the dongle simply expects 
HCI communication and you manage
to get a /dev/ttyUSB0 or whatever on NuttX side, then you should be able to use 
the bt_uart bridge to expose the controller
to NuttX. From then on, you can choose to use NuttX host layer or an external 
one such as nimBLE.

Best,
Matias

On Wed, Mar 24, 2021, at 14:34, Tim wrote:
> I hope this is not too much if a newbie/dumb question as I have tried to
> research this first.
> 
>  
> 
> I've been playing with Nuttx on a SAMA5D2-XULT board ahead of porting to my
> custom board (SAMA5D27C-5M, actually), especially in regard to USB Host
> functionality as that's what's drawn me to NuttX in the first place.
> 
>  
> 
> USB memory sticks are recognised and I can mount them (not got automount to
> work, yet - I'm still learning!; and date/time stamping of files written
> doesn't seem to work either) - nice to have things like this work "out of
> the box"! 
> 
>  
> 
> I can also build in Bluetooth support and the demo app, but as far as I can
> tell there are no actual Bluetooth host drivers in NuttX to properly support
> a detected Bluetooth device. My custom board actually has a SiLabs Bluetooth
> module on it (connected to the SAM via UART and proven to work) but I quite
> like the idea of just using plug-in Bluetooth devices instead :)
> 
>  
> 
> I can see the device is detected (it gets powered, and dmesg reports stuff)
> but no drivers are loaded as best as I can tell. Is that correct or have I
> missed something?
> 
>  
> 
> Drivers are not my strong point but, if I'm right and a driver will be
> needed, is a generic Linux driver a good place to start were I to look to
> add/create one to NuttX? Or maybe someone has already done this?
> 
>  
> 
>  
> 
> Thanks,
> 
>  
> 
> Tim.
> 
>  
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Great, thanks!
I was just writing an issue to have this noted somewhere.

Best,
Matias

On Wed, Mar 24, 2021, at 13:23, Gregory Nutt wrote:
> I think it is not very much work to implement.  Perhaps I will submit a 
> draft PR for your review.
> 
> 
> On 3/24/2021 9:34 AM, Matias N. wrote:
> > Yes, you're right, TLS is the way to go.
> > I only wonder how to minimize the impact. Could this array inside the TLS 
> > struct be grown as needed during runtime? That way if no application calls 
> > to getopt() (or any other function requiring similar solution), no extra 
> > space on TLS is used.
> >
> > On Wed, Mar 24, 2021, at 12:32, Gregory Nutt wrote:
> >>>> Se we can either add something special just as for errno or use
> >>>> entries in that array (which would require establishing a minimum
> >>>> number of entries to satisfy the case of getopt en potentially
> >>>> others). I think it is better to somehow "reserve" space for the
> >>>> known required cases.
> >>>>
> >>>> What i'm worried about is: how many other cases like this there could
> >>>> be? Maybe there will be a considerable number of this entries added
> >>>> to TLS structure (yes, four bytes, but they can add up quickly). I
> >>>> would personally prefer to use reentrant versions when they are
> >>>> available, instead of increasing memory use of every thread. Not sure
> >>>> what is really best here...
> >>> Standardization is certainly the highest value of the OS and the thing
> >>> that makes NuttX what it is.  Sacrificing standardization sacrifices
> >>> the core value of the OS.
> >> Standardization supports portability.  If we bring in code from Linux,
> >> it will not use getopt_r(), it will use getopt() or getopt_long() and
> >> may not work as expected without the TLS-based change.  Similarly, if we
> >> write applications that depend on the non-standard getop_r(), that code
> >> will not compile or build under Linux.  We will have lost portability.
> >>
> >> Many people support code components that operate under either Linux or
> >> NuttX and they depend on having this compatibility.  Why break it?  It
> >> is not consistent with the principles set out in INVIOLABLES.md
> >>
> >>
> >>
> >>
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Yes, you're right, TLS is the way to go.
I only wonder how to minimize the impact. Could this array inside the TLS 
struct be grown as needed during runtime? That way if no application calls to 
getopt() (or any other function requiring similar solution), no extra space on 
TLS is used.

On Wed, Mar 24, 2021, at 12:32, Gregory Nutt wrote:
> 
> >
> >> Se we can either add something special just as for errno or use 
> >> entries in that array (which would require establishing a minimum 
> >> number of entries to satisfy the case of getopt en potentially 
> >> others). I think it is better to somehow "reserve" space for the 
> >> known required cases.
> >>
> >> What i'm worried about is: how many other cases like this there could 
> >> be? Maybe there will be a considerable number of this entries added 
> >> to TLS structure (yes, four bytes, but they can add up quickly). I 
> >> would personally prefer to use reentrant versions when they are 
> >> available, instead of increasing memory use of every thread. Not sure 
> >> what is really best here...
> > Standardization is certainly the highest value of the OS and the thing 
> > that makes NuttX what it is.  Sacrificing standardization sacrifices 
> > the core value of the OS.
> 
> Standardization supports portability.  If we bring in code from Linux, 
> it will not use getopt_r(), it will use getopt() or getopt_long() and 
> may not work as expected without the TLS-based change.  Similarly, if we 
> write applications that depend on the non-standard getop_r(), that code 
> will not compile or build under Linux.  We will have lost portability.
> 
> Many people support code components that operate under either Linux or 
> NuttX and they depend on having this compatibility.  Why break it?  It 
> is not consistent with the principles set out in INVIOLABLES.md
> 
> 
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Se we can either add something special just as for errno or use entries in that 
array (which would require establishing a minimum number of entries to satisfy 
the case of getopt en potentially others). I think it is better to somehow 
"reserve" space for the known required cases.

What i'm worried about is: how many other cases like this there could be? Maybe 
there will be a considerable number of this entries added to TLS structure 
(yes, four bytes, but they can add up quickly). I would personally prefer to 
use reentrant versions when they are available, instead of increasing memory 
use of every thread. Not sure what is really best here...

On Wed, Mar 24, 2021, at 11:51, Gregory Nutt wrote:
> On 3/24/2021 8:38 AM, Matias N. wrote:
> > So, if I follow correctly, we could maybe have one TLS pointer pointing to 
> > a struct of pointers, one per each group of globals (one of this groups, 
> > would be the set of variables used by getopt()), like:
> >
> > struct task_globals_s
> > {
> >struct getopt_globals_s *getopt_globals;
> >/* ...others */
> > };
> >
> > Then getopt globals would only be allocated once for each task, and only 
> > when getopt() is called.
> >
> > Something like that?
> 
> Yes, that is a possibility.  But that is already implemented just as you 
> describe as POSIX thread-specific data.
> 
> The TLS data structure is defined in include/nuttx/tls.h as following.  
> it is just an array of pointer size things and the errno variable.
> 
> struct tls_info_s
> {
> #if CONFIG_TLS_NELEM > 0
>uintptr_t tl_elem[CONFIG_TLS_NELEM]; /* TLS elements */
> #endif
>int tl_errno;/* Per-thread error number */
> };
> 
> This structure lies at the "bottom" of stack of every thread in user space.
> 
> The standard pthread_getspecific() is then implemented as:
> 
> FAR void *pthread_getspecific(pthread_key_t key)
> {
>return (FAR void *)tls_get_value((int)key);
> }
> 
> Where
> 
> uintptr_t tls_get_value(int tlsindex)
> {
>FAR struct tls_info_s *info;
>uintptr_t ret = 0;
> 
>DEBUGASSERT(tlsindex >= 0 && tlsindex < CONFIG_TLS_NELEM);
>if (tlsindex >= 0 && tlsindex < CONFIG_TLS_NELEM)
>  {
>/* Get the TLS info structure from the current threads stack */
> 
>info = up_tls_info();
>DEBUGASSERT(info != NULL);
> 
>/* Get the element value from the TLS info. */
> 
>ret = info->tl_elem[tlsindex];
>  }
> 
>return ret;
> }
> 
> The POSIX interface supports a pthread_key_create() to manage the indexing.
> 
> 
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

So, if I follow correctly, we could maybe have one TLS pointer pointing to a 
struct of pointers, one per each group of globals (one of this groups, would be 
the set of variables used by getopt()), like:

struct task_globals_s
{
  struct getopt_globals_s *getopt_globals;
  /* ...others */
};

Then getopt globals would only be allocated once for each task, and only when 
getopt() is called.

Something like that?

On Wed, Mar 24, 2021, at 11:24, Gregory Nutt wrote:
> 
> > Of course, I would only call getopt() once. My question was if we use TLS, 
> > would the memory use scale with the number of threads? Or would this memory 
> > for getopt() only be allocated on getopt() calls?
> 
> Yes and yes, but the memory use might be as small as a single pointer.  
> Per task data would be better and exists now in the OS, but we would 
> have to implement some internal OS api's that libc could use to access it.
> 
> Another thing to consider is that the current per-task-data is protected 
> and can only be accessed in supervisor mode.  That is not a problem for 
> the FLAT build but might add complication to the PROTECTED build (or at 
> least more system call overhead).
> 
> KERNEL build mode does not need per-task data at all.  It would be 
> better if the data could just be kept in globals for KERNEL build, just 
> as with Linux.
> 
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Of course, I would only call getopt() once. My question was if we use TLS, 
would the memory use scale with the number of threads? Or would this memory for 
getopt() only be allocated on getopt() calls?

On Wed, Mar 24, 2021, at 10:56, Gregory Nutt wrote:
> 
> >> You would expect getopt() to be used only on the many thread since that
> >> is the only thread that receives argc and argv.
> > So if it is only used in one thread there would only be a copy of the data? 
> > What if I spawn multiple threads and call getopt only on one?
> 
> It is hard to imagine how you could could call getopt() on any pthread 
> created with pthread_create():  The thread has no argc and argv inputs 
> so how could a pthread use getopt() unless you contrive something very 
> artificial situation.  pthreads do not receive argument lists and, 
> hence, don't need to parse argument lists.  The single thread that 
> starts with main() is the only thread that has argc and argv and the 
> only thread that can normally call getopt().
> 
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

On Wed, Mar 24, 2021, at 10:37, Gregory Nutt wrote:
> 
> > Thanks for all answers. I don't entirely understand most of them though as 
> > I'm not really familiar with the implications of TLS or how to use it 
> > correctly. Also, do we need per-thread or per-task data here?
> 
> You would expect getopt() to be used only on the many thread since that 
> is the only thread that receives argc and argv.

So if it is only used in one thread there would only be a copy of the data? 
What if I spawn multiple threads and call getopt only on one?

> 
> A faithful, bug-for-bug, implementation would require a per task, but 
> AFAIK there would be no real problem with per thread either.

Yes, my thinking is that getopt() does not provide thread safety guarantees but 
it is not wrong to provide them.
I think that some obscure case of changing getopt() globals from different 
threads is worst to support than just not
doing anything in our case.

> 
> >
> > What I'm thinking is that, besides the TLS based solution, adding a 
> > non-standard getopt() seems to be a good option anyway, since it is a 
> > lightweight solution to this particular function.
> 
> Except that NuttX is a standards based OS and we avoid non-standard 
> interfaces like the plague.  Using TLS is 100% transparent and 100% 
> compatible.  Why would you adopt a non-standard solution when a better, 
> fully compliant implementation is readily available?

My thinking is that this could be a case of something that in FLAT mode would 
give the wrong results if just used as is and the non-standard function allows 
to overcome this. But of course if the TLS solution is the right approach and 
does not incur in extra resource usage, the extra function would of course not 
be needed. My concern was that "the right approach" would incur in too much 
resource usage. But again, I don't understand how TLS works yet.

Best,
Matias

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Thanks for all answers. I don't entirely understand most of them though as I'm 
not really familiar with the implications of TLS or how to use it correctly. 
Also, do we need per-thread or per-task data here? 

What I'm thinking is that, besides the TLS based solution, adding a 
non-standard getopt() seems to be a good option anyway, since it is a 
lightweight solution to this particular function.

So, how should we proceed to address this somehow? 

Best,
Matias

On Wed, Mar 24, 2021, at 10:22, Gregory Nutt wrote:
> 
> >> The custom handler isn't enough here, because the real problem is we need
> >> the global variables per task/process.
> >> As Greg suggests, we need something like TLS but per task/process not per
> >> thread(e.g. task_getspecific/task_setspecific).
> >> Once the mechanism is done, getopt can be converted to confirm the standard
> >> trivally.
> >>
> > I was looking at this exact issue last week (see comment in
> > https://github.com/apache/incubator-nuttx/pull/3054).
> >
> > The basis for this mechanism exists in the way errno is handled. Perhaps a
> > structure defined for all libc globals added to TLS and a call from the
> > task creation code to initialise it?
> 
> That would, of course, make the stack usage much larger.  Perhaps an 
> allocate-on-demand approach would make that doable.
> 
> POSIX thread specific data has a good, interface for such 
> allocate-on-demand usage.  POSIX thread specific data is built on top of 
> TLS.
> 
> 
> 

Re: avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

On Tue, Mar 23, 2021, at 22:09, Nathan Hartman wrote:
> On Tue, Mar 23, 2021 at 8:39 PM Matias N.  <mailto:matias%40imap.cc>> wrote:
> 
> > Hi,
> > while using getopt() from a task started from NSH I realized subsequent
> > calls reused the global optind and similar variables resulting in different
> > results each time. I'm aware this is expected in FLAT mode and is related
> > to the issue of static C++ constructors (they would only be called once,
> > not every time the task is started).
> >
> > What I wonder is what could we do to avoid this common pitfall:
> > - document it somewhere (a common issues/troubleshooting section in the
> > docs would be good to have anyways) and just accept the issue
> > - religiously initialize globals myself before being used (a pain, error
> > prone, and a bit adhoc, working only for FLAT mode)
> 
> 
> When using globals, best practice is to make it really clear that the
> variables are global. Many programmers do this by prefixing global variable
> names with g_*.
> 
> I take a different approach: A long time ago, I started grouping all
> globals in a struct, which has one global instance called Global. It makes
> it easy to find all globals, and furthermore at the start of the program as
> a matter of policy the first thing I do is memset() the Global struct to 0.
> Yes, I know that is often redundant to the startup code, but in some
> situations the startup code doesn't initialize globals. The FLAT model is
> one example of this (from the 2nd invocation onwards). I've seen other
> examples of this over the years. By memset()ing your globals at the start
> of main() you can rest assured that the globals are in fact zeroed,
> regardless of whatever else happened before main(). It has another side
> benefit: with globals grouped this way, it becomes trivial to take a
> standalone program and turn it into a component of a larger program.
> tl;dr, this
> approach has worked great for me for a long time.

That sounds like a good approach.

> 
> Caveat: It won't help if your program (or any API called by it) uses
> globals that are outside your control, and therefore, not initialized by
> you. :-/

Yes, my concern is about functions such as getopt(). If you just follow the
description of the API and use it as normal you reach this pitfall. I was 
looking
for some approach to avoid this as much as possible. For getopt() I see there's
even no standard getopt_r(), so we would have to provide our own, which may not
be a bad idea.
Still, this issue will probably present in many other places.

> 
> Nathan
> 

Best,
Matias

avoiding pitfal of reuse of globals in FLAT mode?

[Matias N.]

Hi,
while using getopt() from a task started from NSH I realized subsequent calls 
reused the global optind and similar variables resulting in different results 
each time. I'm aware this is expected in FLAT mode and is related to the issue 
of static C++ constructors (they would only be called once, not every time the 
task is started).

What I wonder is what could we do to avoid this common pitfall:
- document it somewhere (a common issues/troubleshooting section in the docs 
would be good to have anyways) and just accept the issue
- religiously initialize globals myself before being used (a pain, error prone, 
and a bit adhoc, working only for FLAT mode)
- support reentrant versions of all possible APIs and always use these in FLAT 
mode
- devise a mechanism to mimic what would be done by OS in KERNEL mode (add some 
custom handler to APIs internally using globals, such as getopt, that can be 
called either manually by user or by the OS itself when the task is started?)
- other?

Just trying to avoid stepping n-times on the same stone. I think this can also 
happen to unsuspecting users since
a lot of these POSIX interfaces kind of promote these interfaces which use 
globals internally.

Best,
Matias

Re: [EXT] Re: Project specific power management configuration

[Matias N.]

There's somewhat of a (reduced) PM interface to userspace, via boardctl(), 
which passes on the PM_STAY, PM_RELAX and others to PM subsystem. This could be 
extended or indeed you could create a specific driver. The downside of this is 
that it requires runtime registration on board initialization for all boards.
procfs interface could be useful but I think it becomes awkward to be handled 
from code (it is most useful from CLI by using echo and cat).

Anyway, I think the main point is to really think what is the expected 
interface to offer to userspace so it is really useful for the general case and 
not tied to a particular scenario. Maybe we need to think this through 
carefully and maybe even look at the Linux PM subsystem for some inspiration 
for a relatively standard interface.

Anyway, as I mentioned before, the IDLE loop is usually a good place to 
interact with hardware. When I need to interact with board code from userspace, 
I simply extend boardctl to provide any required functionality. Have you looked 
into this options before moving onto introducing a common driver?

Best,
Matias

On Fri, Mar 19, 2021, at 07:08, Cis Van Mierlo wrote:
> Hi Matias,
> 
> Because we have an application that manages the power state machine by 
> itself, therefore we don't need a governor and would like to view and change 
> the power state. 
> For testing we've been calling the OS interface directly from the user space 
> application. 
> This is indeed not ideal, therefore we want to create an API to do this 
> nicely from user space.
> 
> Currently we are considering the following two options:
> 1. Create a driver in nuttx/drivers/power/pm_driver.c that exposes the 
> blockdriver /dev/pm, the user could use this to invoke sys calls that nicely 
> comply with the OS.
> 2. Add a procfs option to the PM driver that exposes /procfs/power/state 
> which can be modified from the user space as well.
> 
> Which option would you prefer or do you have other suggestions?
> 
> Kind regards,
> 
> Cis van Mierlo
> Embedded Software Engineer
> NXP Semiconductors, CTO Systems Innovations
> 
> High Tech Campus 46, room 0.64, 5656 AE Eindhoven, The Netherlands
> E-mail: cis.van.mie...@nxp.com <mailto:cis.van.mierlo%40nxp.com> 
> Internet: http://www.nxp.com 
> 
> -Original Message-
> From: Matias N. mailto:matias%40imap.cc>> 
> Sent: Thursday, March 11, 2021 4:55 PM
> To: dev@nuttx.apache.org <mailto:dev%40nuttx.apache.org>
> Subject: RE: [EXT] Re: Project specific power management configuration
> 
> Caution: EXT Email
> 
> Calling those functions from applications would be a violation of 
> OS/application isolation, as these are OS level calls.
> The general idea is that a driver only knows what to do on each sleep level 
> and the IDLE loop uses those functions to drive the change of states of PM 
> system.
> From application side you can forbid/allow the PM system go into each state 
> which let's you control how devices behave.
> 
> I think the typical approach for low-power applications is typically to have 
> a very short active period and a long sleep period. So, the application logic 
> can forbid entering sleep, interact with all required devices, allow sleep, 
> and go to sleep itself (so the IDLE loop kicks in). This way there's no need 
> to control device state directly. Note that you can still wakeup due to 
> external events, which can be made to go back to normal state or not, which 
> will turn on required devices until you service the request and go back to 
> sleep.
> 
> If for some reason your project requires a different architecture I would say 
> that you need to have a custom board-level driver and offer some kind of 
> interface the user to tune this. An easy path is to use boardctl for this 
> interface.
> 
> Best,
> Matias
> 

Re: GSoC 2021 - How Mentors and Students will apply?

[Matias N.]

Just created the issue here: 
https://github.com/apache/incubator-nuttx/issues/3031
Please let me know if something needs more clarification.

Best,
Matias

On Thu, Mar 11, 2021, at 13:02, Matias N. wrote:
> I think I will open a GH issue for this, so as not to derail this thread.
> I don't know the mentor's GitHub username so please CC him if you do.
> 
> Best,
> Matias
> 
> On Thu, Mar 11, 2021, at 12:22, Alan Carvalho de Assis wrote:
>> Hi Matias,
>> 
>> For sure! I recall you even start to creating some prototype of this idea.
>> 
>> We should to discuss it in more details here. I think Mr. Bhargav is
>> already subscribed to this mailing list, he could align with you too.
>> 
>> BR,
>> 
>> Alan
>> 
>> On 3/11/21, Matias N.  wrote:
>> > Hi Alan,
>> > Just saw the proposal about device tree integration. While I cannot commit
>> > to being a mentor it would be great to be part of the discussion on how to
>> > bring it forward. I thought about it for some time now and I have some sort
>> > of roadmap that I was intending to propose eventually. Maybe I can write
>> > that down somewhere?
>> >
>> > Best,
>> > Matias
>> >
>> 
> 

Re: GSoC 2021 - How Mentors and Students will apply?

[Matias N.]

I think I will open a GH issue for this, so as not to derail this thread.
I don't know the mentor's GitHub username so please CC him if you do.

Best,
Matias

On Thu, Mar 11, 2021, at 12:22, Alan Carvalho de Assis wrote:
> Hi Matias,
> 
> For sure! I recall you even start to creating some prototype of this idea.
> 
> We should to discuss it in more details here. I think Mr. Bhargav is
> already subscribed to this mailing list, he could align with you too.
> 
> BR,
> 
> Alan
> 
> On 3/11/21, Matias N.  wrote:
> > Hi Alan,
> > Just saw the proposal about device tree integration. While I cannot commit
> > to being a mentor it would be great to be part of the discussion on how to
> > bring it forward. I thought about it for some time now and I have some sort
> > of roadmap that I was intending to propose eventually. Maybe I can write
> > that down somewhere?
> >
> > Best,
> > Matias
> >
> 

RE: [EXT] Re: Project specific power management configuration

[Matias N.]

Calling those functions from applications would be a violation of 
OS/application isolation, as these are OS level calls.
The general idea is that a driver only knows what to do on each sleep level and 
the IDLE loop uses those functions to drive the change of states of PM system.
>From application side you can forbid/allow the PM system go into each state 
>which let's you control how devices behave.

I think the typical approach for low-power applications is typically to have a 
very short active period and a long sleep period. So, the application logic can 
forbid entering sleep, interact with all required devices, allow sleep, and go 
to sleep itself (so the IDLE loop kicks in). This way there's no need to 
control device state directly. Note that you can still wakeup due to external 
events, which can be made to go back to normal state or not, which will turn on 
required devices until you service the request and go back to sleep.

If for some reason your project requires a different architecture I would say 
that you need to have a custom board-level driver and offer some kind of 
interface the user to tune this. An easy path is to use boardctl for this 
interface.

Best,
Matias

Re: [EXT] Re: Project specific power management configuration

[Matias N.]

The interaction between hardware on/off and sleep states usually occurs on IDLE 
loop. I'm not sure if you are aware of this option but you can have a board 
specific idle loop implementation. This is what I usually do to implement 
project specific handling of sleep states. 

Best,
Matias

On Wed, Mar 10, 2021, at 11:39, Cis Van Mierlo wrote:
> Hi Matias,

>  

> I indeed saw that there are several governors.

>  

> I implemented PM switches in the application code (you could see this as a 
> governor), there is for example a sleep state of the application, there the 
> PM_SLEEP state is used to set the MCU in a very low power state.


> Another one is the PM_STANDBY, where the MCU will be put in very low power 
> run mode, with only one SPI bus and the console UART enabled. All on a slower 
> clock source. To reduce as much power as possible. 
> 
> But the problem that I have is that these peripherals and MCU modes are chip 
> specific and you want to do the configuration of which MCU mode, which 
> peripherals, etc. to enable in a power mode per board/project. Because then 
> you know which peripherals you can turn off in a certain mode to reduce as 
> much power, but still have the application states functional.  
> 
> This is what I wanted to bring to a discussion on how to solve this. 
>  

> Kind regards,

>  


> Cis van Mierlo
> Embedded Software Engineer

> NXP Semiconductors, CTO Systems Innovations


> 
> High Tech Campus 46, room 0.64, 5656 AE Eindhoven, The Netherlands
> E-mail: cis.van.mie...@nxp.com 
> Internet: http://www.nxp.com

>  


> *From:* Matias N.  
> *Sent:* Friday, March 5, 2021 4:32 PM
> *To:* dev@nuttx.apache.org
> *Subject:* [EXT] Re: Project specific power management configuration

>  

> *Caution: *EXT Email

> Hi,

>  

> I'm not entirely sure what is the specific control over PM you need but some 
> time ago I introduced the notion of "governor" which lets you select either 
> the activity based one (which uses a specific algorithm to control states), a 
> greedy governor (which lets an application directly control states via 
> relax/stay calls). This can easily be extended to have another governor or 
> even an application defined one (in that case, there need to be some extra 
> logic to receive the lower-half of the governor from board logic).

>  

> Best,

> Matias

>  

> On Fri, Mar 5, 2021, at 07:15, Cis Van Mierlo wrote:

>> Hi all,

>>  

>> Me and my colleagues (NXP mobile robotics) encountered a problem and we 
>> would like to start a discussion on how to solve that. We're making a 
>> development board using NuttX and we would like to add the power mode 
>> configuration to it. But this needs to be specific for this "project".

>>  

>> The problem:

>> The pm.h specific logic is in the chip dependent code (for example the 
>> up_pm_prepare() and up_pm_notify() callback functions are located in 
>> nuttx/arch/arm/src/s32k1xx/s32k1xx_clockconfig.c for the power mode switch 
>> of the MCU). And for peripherals in the chip specific files as well, for 
>> example to enable/disable peripherals, like in 
>> nuttx/arch/arm/src/s32k1xx/s32k1xx_serial.c to enable the serial on the 
>> other clock which is active in the MCU low power run mode.

>>  

>> But you should not decide on chip level what your current board is doing, 
>> each board/project would want to have its own power optimizations and 
>> ability to enable/disable peripherals depending on the use case.

>>  

>> So in that case you would want to give the user/developer the option to 
>> refine which power reductions he will use for his product/project.

>>  

>> We have several ideas which we would like to use to start a discussion on 
>> this.

>>  

>> Idea 1:

>> Add the configuration with the kconfig files to for example "System Type" in 
>> menuconfig. Make a PM configure part in which you can enable/disable the 
>> various PM states (PM_IDLE, PM_STANDBY, PM_SLEEP) and going in this PM state 
>> will allow you to configure in what state the MCU should be and which 
>> peripherals should be enabled.

>>  

>>  Machine generated alternative text: menuconfig Build Setup System Type 
>> Allows user to specify which peripherals are enabled In e specflc power mode 
>> and in which mode the MCI-I Is PM configuration IDLE SLEEP Copy 
>> configuratjon from menuconfig (pherjpherals)? chip specific MCIJ mode RUN 
>> chip specific MCIJ mode UART CARTO LIARTI only show PM suppoäed peripherals 
>> HSRUN RUN VLPR STOP 

>> Idea 2:

>> Another idea is that you add the PM configuration of 

Re: Project specific power management configuration

[Matias N.]

Hi,

I'm not entirely sure what is the specific control over PM you need but some 
time ago I introduced the notion of "governor" which lets you select either the 
activity based one (which uses a specific algorithm to control states), a 
greedy governor (which lets an application directly control states via 
relax/stay calls). This can easily be extended to have another governor or even 
an application defined one (in that case, there need to be some extra logic to 
receive the lower-half of the governor from board logic).

Best,
Matias

On Fri, Mar 5, 2021, at 07:15, Cis Van Mierlo wrote:
> Hi all,

>  

> Me and my colleagues (NXP mobile robotics) encountered a problem and we would 
> like to start a discussion on how to solve that. We're making a development 
> board using NuttX and we would like to add the power mode configuration to 
> it. But this needs to be specific for this "project".

>  

> The problem:

> The pm.h specific logic is in the chip dependent code (for example the 
> up_pm_prepare() and up_pm_notify() callback functions are located in 
> nuttx/arch/arm/src/s32k1xx/s32k1xx_clockconfig.c for the power mode switch of 
> the MCU). And for peripherals in the chip specific files as well, for example 
> to enable/disable peripherals, like in 
> nuttx/arch/arm/src/s32k1xx/s32k1xx_serial.c to enable the serial on the other 
> clock which is active in the MCU low power run mode.

>  

> But you should not decide on chip level what your current board is doing, 
> each board/project would want to have its own power optimizations and ability 
> to enable/disable peripherals depending on the use case.

>  

> So in that case you would want to give the user/developer the option to 
> refine which power reductions he will use for his product/project.

>  

> We have several ideas which we would like to use to start a discussion on 
> this.

>  

> Idea 1:

> Add the configuration with the kconfig files to for example "System Type" in 
> menuconfig. Make a PM configure part in which you can enable/disable the 
> various PM states (PM_IDLE, PM_STANDBY, PM_SLEEP) and going in this PM state 
> will allow you to configure in what state the MCU should be and which 
> peripherals should be enabled.

>  

>  Machine generated alternative text: menuconfig Build Setup System Type 
> Allows user to specify which peripherals are enabled In e specflc power mode 
> and in which mode the MCI-I Is PM configuration IDLE SLEEP Copy configuratjon 
> from menuconfig (pherjpherals)? chip specific MCIJ mode RUN chip specific 
> MCIJ mode UART CARTO LIARTI only show PM suppoäed peripherals HSRUN RUN VLPR 
> STOP 

> Idea 2:

> Another idea is that you add the PM configuration of the peripherals, to the 
> peripheral configuration.

> 
> Machine generated alternative text: menuconfig Build Setup System Type 
> Peripheral Selection UART CARTO LIARTI BAUD rate OMA support Power Management 
> NORMAL PM IDLE PM STANDBY PM SLEEP 
> Of course we are open to other ideas. We just wanted to get a discussion 
> started on how to do this the best way.

>  

> Kind regards,

>  


> Cis van Mierlo
> Embedded Software Engineer

> NXP Semiconductors, CTO Systems Innovations


> 
> High Tech Campus 46, room 0.64, 5656 AE Eindhoven, The Netherlands
> E-mail: cis.van.mie...@nxp.com 
> Internet: http://www.nxp.com

>  

> 
> *Attachments:*
>  * image001.png
>  * image002.png
>  * PM_mode_configuration_option.png
>  * PM_mode_configuration_option2.png

Re: Proper handling of arch header files

[Matias N.]

> DTS mean huge changes and maybe even ditching Kconfig

DTS does not replace Kconfig. Kconfig enables OS options and support
for features. DTS describes presence of resources and how they are
mapped to on-board/on-chip resources (CS line to GPIO is one such example).
Moreover, Kconfig does not (easily/nicely) allow for multiple instances of a
same driver.

DTS support does not have to mean a big change, if we follow the approach
I suggested in that issue, which is what Zephyr follows. But again, I think this
is not actually necessary right now to improve what has been discussed. 

Moving to a standardized device driver interface based on 
initialize/deinitialize/etc.
callbacks and adding ability to interact with GPIOs using the abstract interface
I mentioned already would be a good step.

Best,
Matias

Re: Proper handling of arch header files

[Matias N.]

That is a separate issue from what we're discussing (arch independent GPIO 
handling) and I agree
with you. I would like to eventually move to a different initialization system 
based on callbacks
and a hierarchical resource description strategy which can describe on-board 
devices but can be
extended (without changing code) to describe off-board devices.

I started to work on the resource description by attempting to use devicetree 
(see length discussion
here: 
https://github.com/apache/incubator-nuttx/issues?q=is%3Aissue+is%3Aopen+devicetree)
 which
I think is a good option for that part.

But from my thinking about this, the best way would be to work on the other 
part of the problem first
(callback based initialization + standardization of device interface 
operations). I've been meaning to work
on this for some time, but since it is something that requires careful planning 
and discussion, I've been
putting it off until I'm sure I can pursue it.

Anyway, abstracting GPIO handling from an arch-indepedent device driver is a 
first good step towards
that goal. I think that it would be good to improve the ioexpander interface 
and start changing how this
is handled in SPI device drivers. The goal in the end is the same: minimize 
board logic when not really
required.

Best,
Matias

On Sat, Feb 6, 2021, at 12:38, Grr wrote:
> >
> >
> > The board would initialize this device as:
> >
> 
> What if device is external to board? You need to hack code to include
> something that's not in any board
> 
> That's the proof GPIO subsystem must be configured outside of board logic
> 

Re: Proper handling of arch header files

[Matias N.]

On Sat, Feb 6, 2021, at 12:06, Grr wrote:
> > No, you wouldn't need that. The device driver will receive a generic GPIO
> > handling
> > interface and it would simply assert/deassert CS line without knowing the
> > specifics
> > of the GPIO impementation on the given architecture.
> 
> 
> That's not what you said. You said:
> 
> "In other words, make a SPI device driver _provide the select operation_
> (instead of the typical _spiNselect() at board logic level."
> 
> "without knowing the specifics" and "provide the select operation" are
> mutually exclusive
> 

They are not. Providing the operation means that the SPI device driver would do 
something like:

static void spi_select(bool enable)
{
  ...
  priv->cs_gpiodev->assert(enable);
}

The board would initialize this device as:

struct gpiodev* d = _gpiodev_init(cs_pincfg);
myspisensor_initialize(spi, d);

Where gpiodev is something like gpioexpander system. The _gpiodev_init 
provides the lower-half
of this system, which is implemented only once per arch.

Matias

Re: Proper handling of arch header files

[Matias N.]

> That's moving the same problem from one place to another instead of
> eliminating it: you need to rewrite some part of the driver for each
> chip/subchip.
 
No, you wouldn't need that. The device driver will receive a generic GPIO 
handling
interface and it would simply assert/deassert CS line without knowing the 
specifics
of the GPIO impementation on the given architecture. The GPIO interface would be
implemented at arch level (and I believe some arch's are already implemented).

During initialization of this device driver lower-half, the GPIO interface 
instance
would be initialized to the correct CS line for this driver instance.

Further improvements (minimizing board bringup code to support a driver) could 
be achieved
but with more involved changes. I have some ideas on that (related to my 
initial attempt at
supporting device tree) but I would like to discuss that only when I think I 
would be
able to pursue it.

Best,
Matias

Re: Proper handling of arch header files

[Matias N.]

I understand you are looking for a generic API that can be used from drivers to 
access GPIOs, right?
I understand this may come up in situations like handling CS lines from SPI 
driver. Currently we handle this by delegating the select line handling to 
board logic completely.

I thought about this a few times and I think this can be made better without 
breaking modularity. I think the closest approach is to use something in the 
lines of what Xiang suggested: give the lower-half part of the SPI device 
driver the ability to control the GPIO via some interface (ioexpander in this 
case). One improvement I can think of is to avoid the devid -> CS line routing 
to be made at board-logic level and make a SPI device driver (eg: a sensor) 
handle it itself. In other words, make a SPI device driver provide the select 
operation (instead of the typical _spiNselect() at board logic level.
This could also be used for the spiNstatus and D/C lines, if present.

That could go a long way to reduce boilerplate code to enable use of a SPI 
device from a given board.

Best,
Matias

On Sat, Feb 6, 2021, at 09:16, Grr wrote:
> El vie, 5 feb 2021 a las 23:15, Brennan Ashton ()
> escribió:
> 
> > On Fri, Feb 5, 2021, 8:43 PM Grr  wrote:
> >
> >
> > Arch specific ones are not allowed in arch/ or your application
> 
> 
> There is no reason to provide your application
> >
> 
> GPIO system is _not_ my application
> 
> We are talking in circles here at this point. You would have to show what
> > you are planning on doing with the information you are trying to put in
> > board.h because I don't see how it helps
> 
> 
> I already did that
> 
> Where would this base address be
> > used, what exactly would call it?
> >
> 
> GPIO system  gpio_setpin() function
> 

Re: Proper handling of arch header files

[Matias N.]

I'm not entirely following the problem, but it sounds like you could decouple 
your arch-independent driver logic in an upper-half and then have a lower-half 
per-arch, using board-common drivers (ie. boards/stm32/drivers). The lower-half 
would then be able to use stm32 internal headers. In this scenario, you don't 
need to replicate things per-board, only per-arch (which I guess is the minimum 
you could do if you're using stm32 specific pin handling).

Best,
Matias

On Fri, Feb 5, 2021, at 23:13, Grr wrote:
> > Is this helpful?
> >
> > https://cwiki.apache.org/confluence/display/NUTTX/Including+Files+in+board.h
> 
> 
> Maybe but I think it would obscure things. I explain:
> 
> Per my idea, exporting available GPIOs is done from board.h like this:
> 
> #ifdef CONFIG_GPIO_1
> # if defined(CONFIG_GPIO_1_OUT)
> #  define GPIO_1_TYPE GPIO_OUTPUT_PIN
> # elif defined(CONFIG_GPIO_1_IN)
> #  define GPIO_1_TYPE GPIO_INPUT_PIN
> # else /* CONFIG_GPIO_1_IRQ */
> #  define GPIO_1_TYPE GPIO_INTERRUPT_PIN
> # endif
> # define GPIO_1_PIN   2
> # define GPIO_1_PORT  5
> # define GPIO_1_ADDR  STM32_GPIOE_BASE
> #endif  /* CONFIG_GPIO_1  */
> 
> That means I should be able to access
> arch/arm/src/stm32/hardware/stm32f40xxx_memorymap.h from board.h, which is
> no dangerous at all, except that it's not easy
> 
> Why not follow that page advice and do it from the C file? Because next C
> file is a system-wide file in drivers/ that fills system-wide GPIO struct
> with board information. Otherwise, I'd have to make a board function that
> fills system-wide struct and replicate that for every board thus recreating
> the problem I want to fix.
> 
> If STM32 header files were in include/, there would be no problem, so I'm
> almost convinced that will be the best solution
> 
> 
> >
> > On 2/5/2021 7:34 PM, Brennan Ashton wrote:
> > > On Fri, Feb 5, 2021, 4:40 PM Grr  wrote:
> > >
> > >>> You cannot  include arch specific headers in board.h like that! That
> > will
> > >>> break a lot of things.
> > >>>
> > >> board.h is arch specific
> > >
> > >> It absolutely is not including arch headers will break things.
> > > board.h:
> > >
> > https://www.github.com/apache/incubator-nuttx/tree/master/boards%2Farm%2Fstm32%2Fstm32f4discovery%2Finclude%2Fboard.h
> > >
> > > And the arch specific board configuration:
> > >
> > https://www.github.com/apache/incubator-nuttx/tree/master/boards%2Farm%2Fstm32%2Fstm32f4discovery%2Fsrc%2Fstm32f4discovery.h
> > >
> > >
> > >
> > >>
> > >>> The expectation is that you pass the interfaces that you need into the
> > >>> drivers.  If you have a particular "module" which contains multiple
> > >> devices
> > >>> that you want to support across different boards then just create a
> > thin
> > >>> driver that wraps everything else inside of it. Bypassing things like
> > the
> > >>> existing SPI CS or interrupt infrastructure is asking for trouble.
> > >>>
> > >> Combining my example (I forgot to mention ESP32) with that technique
> > means
> > >> creating at least 10 thin
> > >>
> > > You should not need to do that.  Here is an example of how GPIO is
> > normally
> > > attached. Note this be done in per arch common code but it is arch
> > specific.
> > >
> > >
> > https://www.github.com/apache/incubator-nuttx/tree/master/boards%2Farm%2Fstm32%2Fstm32f4discovery%2Fsrc%2Fstm32_gs2200m.c
> > >
> > > You could also create a shim for the driver to talk to the generic GPIO
> > > interface as we suggested earlier, but I'm not convinced that we would
> > gain
> > > much with that.
> > >
> > >
> > >>> For example I have a breakout module with a touchscreen, keyboard, and
> > SD
> > >>> card. This requires the I2C bus, SPI bus, and GPIO for card detection
> > and
> > >>> chip select.  I just define a common name for the IO on the device and
> > >> then
> > >>> map that in board.h and then add the callbacks for the GPIO interrupt
> > and
> > >>> the chip select.  This in total is about 50 lines of actual board code
> > >> and
> > >>> that is if you are not already using things like the SPI chip select.
> > >>>
> > >> My idea goes like this:
> > >>
> > >> 1-Board exports available GPIOs (among other things)
> > >> 2-Menuconfig offer GPIOs to user for selection. Selected ones become
> > >> available to drivers
> > >> 3-User configures needed drivers with selected GPIOs, also in Menuconfig
> > >> 4-Compiles
> > >> 5-If developing, user can test with incorporated char driver that
> > allows to
> > >> turn on and off selected GPIOs from a system testing utility
> > >
> > > To do this you would use the ioexpander/GPIO interface that was
> > mentioned.
> > > It can provide that functionality but you would still have to write lower
> > > half shims for the drivers that would consume GPIO this way.
> > >
> > >
> > >>> --Brennan
> > >>>
> > >>> On Fri, Feb 5, 2021, 3:08 PM Grr  wrote:
> > >>>
> > > Yes, we need an additional struct for port number:
> > >
> > >
> > >>
> > 

Re: testing wapi on esp32 devkitc

[Matias N.]

Yes, but it was really low level logging and didn't understand much. I'll try 
again later and paste it here.

Best,
Matias

RE: I saw a new option for merging in GH: Auto merge when CI completes

[Matias N.]

Yes this would be a useful feature.
Assuming that the trigger disarms if there's a new push in between, which may 
require new review.

Re: Anyone else getting repeated emails from the GitBox?

[Matias N.]

I actually delete gitbox messages on arrival. I find GitHub notifications much 
more usable.

Best,
Matias

Re: testing wapi on esp32 devkitc

[Matias N.]

Thanks, I was looking at the DevKitC readme which does not have these 
instructions.
I cannot seem to connect though, not sure if it is because my router is set to 
WPA2 TKIP+CCMP.

Best,
Matias

On Thu, Feb 4, 2021, at 20:06, Abdelatif Guettouche wrote:
> Basically you can just check the commands in the wlan_ioctl function
> in esp32_wlan.c to see what's implemented.  But not much going on
> there for the moment.
> You can connect to access points.  Commands are given in the README.txt file
> wapi psk wlan0 mypassword 1
> wapi essid wlan0 myssid 1
> You can also set these from menuconfig.
> 
> > Also, I'm trying to scan and I get:
> 
> That's because SIOCSIWSCAN isn't handled by the driver.
> We have a branch internally to improve the driver and add more
> features (SIOCSIWSCAN being one of them), I don't know when this one
> will be submitted though.
> 
> On Thu, Feb 4, 2021 at 11:55 PM Matias N.  wrote:
> >
> > Thanks, now the interface appears.
> > What functionality is currently supported? Can I connect to WPA2 AP? Can I 
> > create an AP?
> >
> > Also, I'm trying to scan and I get:
> >
> > nsh> wapi scan wlan0
> > ioctl(SIOCSIWSCAN): 25
> > ERROR: Process command (scan) failed.
> >
> > Thanks,
> > Matias
> >
> > On Thu, Feb 4, 2021, at 19:36, Abdelatif Guettouche wrote:
> > > It could be because there are some issues with getting the paramters from
> > > the flash.  Please try disabling Save Paramters from the Wifi 
> > > Configuration
> > > menu.
> > >
> > > On Thu, Feb 4, 2021, 11:32 PM Matias N.  wrote:
> > >
> > > > Hi,
> > > > I'm trying to test WiFi support on ESP32 DevKitC (I used wapi config) 
> > > > and
> > > > it does not seem to work. I get this:
> > > >
> > > > rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
> > > > configsip: 0, SPIWP:0xee
> > > > clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
> > > > mode:DIO, clock div:2
> > > > load:0x3fff0030,len:4
> > > > load:0x3fff0034,len:6924
> > > > load:0x40078000,len:13272
> > > > load:0x40080400,len:3604
> > > > entry 0x40080670
> > > > I (29) boot: ESP-IDF v4.3-dev-1901-g178b122c1 2nd stage bootloader
> > > > I (29) boot: compile time 17:16:52
> > > > I (29) boot: chip revision: 1
> > > > I (33) boot_comm: chip revision: 1, min. bootloader chip revision: 0
> > > > I (40) boot.esp32: SPI Speed  : 40MHz
> > > > I (45) boot.esp32: SPI Mode   : DIO
> > > > I (49) boot.esp32: SPI Flash Size : 2MB
> > > > I (54) boot: Enabling RNG early entropy source...
> > > > I (59) boot: Partition Table:
> > > > I (63) boot: ## LabelUsage  Type ST Offset   Length
> > > > I (70) boot:  0 nvs  WiFi data01 02 9000 
> > > > 6000
> > > > I (78) boot:  1 phy_init RF data  01 01 f000 
> > > > 1000
> > > > I (85) boot:  2 factory  factory app  00 00 0001 
> > > > 0010
> > > > I (93) boot: End of partition table
> > > > I (97) boot_comm: chip revision: 1, min. application chip revision: 0
> > > > I (104) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020
> > > > size=0x0c22c ( 49708) map
> > > > I (132) esp_image: segment 1: paddr=0x0001c254 vaddr=0x3ffb81e0
> > > > size=0x0181c (  6172) load
> > > > I (135) esp_image: segment 2: paddr=0x0001da78 vaddr=0x4008
> > > > size=0x00400 (  1024) load
> > > > I (139) esp_image: segment 3: paddr=0x0001de80 vaddr=0x40080400
> > > > size=0x02198 (  8600) load
> > > > I (151) esp_image: segment 4: paddr=0x00020020 vaddr=0x400d0020
> > > > size=0x59678 (366200) map
> > > > I (296) esp_image: segment 5: paddr=0x000796a0 vaddr=0x40082598
> > > > size=0x08718 ( 34584) load
> > > > I (317) boot: Loaded app from partition at offset 0x1
> > > > I (317) boot: Disabling RNG early entropy source...
> > > > esp32_rng_initialize: Initializing RNG
> > > > esp_nvs_get_blob: INFO: No file /mnt/esp/wifi/wifi.misc.log
> > > > esp_nvs_set_blob: ERROR: Failed to set open /mnt/esp/wifi/wifi.misc.log
> > > > W (85) wifi:nvs_log_init, osi_nvs_set_blob fail, key=log ret=
> > > >
> > > > esp_wifi_init: ERROR: Failed to initialize WiFi error=-1
> > > > esp_wifi_adapter_init: ERROR: Failed to initialize WiFi error=-1
> > > > ERROR: Failed to initialize WiFi
> > > >
> > > > NuttShell (NSH) NuttX-10.0.1
> > > > nsh> ifconfig
> > > > nsh> ifup wlan0
> > > > ifup wlan0...OK
> > > > nsh> ifconfig
> > > > nsh>
> > > >
> > > > I have used bootloader and partitions bin from
> > > > https://github.com/saramonteiro/esp32_binaries_nuttx
> > > >
> > > > Thanks,
> > > > Matias
> > > >
> > >
> 

Re: testing wapi on esp32 devkitc

[Matias N.]

Thanks, now the interface appears.
What functionality is currently supported? Can I connect to WPA2 AP? Can I 
create an AP?

Also, I'm trying to scan and I get:

nsh> wapi scan wlan0
ioctl(SIOCSIWSCAN): 25
ERROR: Process command (scan) failed.

Thanks,
Matias

On Thu, Feb 4, 2021, at 19:36, Abdelatif Guettouche wrote:
> It could be because there are some issues with getting the paramters from
> the flash.  Please try disabling Save Paramters from the Wifi Configuration
> menu.
> 
> On Thu, Feb 4, 2021, 11:32 PM Matias N.  wrote:
> 
> > Hi,
> > I'm trying to test WiFi support on ESP32 DevKitC (I used wapi config) and
> > it does not seem to work. I get this:
> >
> > rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
> > configsip: 0, SPIWP:0xee
> > clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
> > mode:DIO, clock div:2
> > load:0x3fff0030,len:4
> > load:0x3fff0034,len:6924
> > load:0x40078000,len:13272
> > load:0x40080400,len:3604
> > entry 0x40080670
> > I (29) boot: ESP-IDF v4.3-dev-1901-g178b122c1 2nd stage bootloader
> > I (29) boot: compile time 17:16:52
> > I (29) boot: chip revision: 1
> > I (33) boot_comm: chip revision: 1, min. bootloader chip revision: 0
> > I (40) boot.esp32: SPI Speed  : 40MHz
> > I (45) boot.esp32: SPI Mode   : DIO
> > I (49) boot.esp32: SPI Flash Size : 2MB
> > I (54) boot: Enabling RNG early entropy source...
> > I (59) boot: Partition Table:
> > I (63) boot: ## LabelUsage  Type ST Offset   Length
> > I (70) boot:  0 nvs  WiFi data01 02 9000 6000
> > I (78) boot:  1 phy_init RF data  01 01 f000 1000
> > I (85) boot:  2 factory  factory app  00 00 0001 0010
> > I (93) boot: End of partition table
> > I (97) boot_comm: chip revision: 1, min. application chip revision: 0
> > I (104) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020
> > size=0x0c22c ( 49708) map
> > I (132) esp_image: segment 1: paddr=0x0001c254 vaddr=0x3ffb81e0
> > size=0x0181c (  6172) load
> > I (135) esp_image: segment 2: paddr=0x0001da78 vaddr=0x4008
> > size=0x00400 (  1024) load
> > I (139) esp_image: segment 3: paddr=0x0001de80 vaddr=0x40080400
> > size=0x02198 (  8600) load
> > I (151) esp_image: segment 4: paddr=0x00020020 vaddr=0x400d0020
> > size=0x59678 (366200) map
> > I (296) esp_image: segment 5: paddr=0x000796a0 vaddr=0x40082598
> > size=0x08718 ( 34584) load
> > I (317) boot: Loaded app from partition at offset 0x1
> > I (317) boot: Disabling RNG early entropy source...
> > esp32_rng_initialize: Initializing RNG
> > esp_nvs_get_blob: INFO: No file /mnt/esp/wifi/wifi.misc.log
> > esp_nvs_set_blob: ERROR: Failed to set open /mnt/esp/wifi/wifi.misc.log
> > W (85) wifi:nvs_log_init, osi_nvs_set_blob fail, key=log ret=
> >
> > esp_wifi_init: ERROR: Failed to initialize WiFi error=-1
> > esp_wifi_adapter_init: ERROR: Failed to initialize WiFi error=-1
> > ERROR: Failed to initialize WiFi
> >
> > NuttShell (NSH) NuttX-10.0.1
> > nsh> ifconfig
> > nsh> ifup wlan0
> > ifup wlan0...OK
> > nsh> ifconfig
> > nsh>
> >
> > I have used bootloader and partitions bin from
> > https://github.com/saramonteiro/esp32_binaries_nuttx
> >
> > Thanks,
> > Matias
> >
> 

testing wapi on esp32 devkitc

[Matias N.]

Hi,
I'm trying to test WiFi support on ESP32 DevKitC (I used wapi config) and it 
does not seem to work. I get this:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4
load:0x3fff0034,len:6924
load:0x40078000,len:13272
load:0x40080400,len:3604
entry 0x40080670
I (29) boot: ESP-IDF v4.3-dev-1901-g178b122c1 2nd stage bootloader
I (29) boot: compile time 17:16:52
I (29) boot: chip revision: 1
I (33) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (40) boot.esp32: SPI Speed  : 40MHz
I (45) boot.esp32: SPI Mode   : DIO
I (49) boot.esp32: SPI Flash Size : 2MB
I (54) boot: Enabling RNG early entropy source...
I (59) boot: Partition Table:
I (63) boot: ## LabelUsage  Type ST Offset   Length
I (70) boot:  0 nvs  WiFi data01 02 9000 6000
I (78) boot:  1 phy_init RF data  01 01 f000 1000
I (85) boot:  2 factory  factory app  00 00 0001 0010
I (93) boot: End of partition table
I (97) boot_comm: chip revision: 1, min. application chip revision: 0
I (104) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x0c22c ( 
49708) map
I (132) esp_image: segment 1: paddr=0x0001c254 vaddr=0x3ffb81e0 size=0x0181c (  
6172) load
I (135) esp_image: segment 2: paddr=0x0001da78 vaddr=0x4008 size=0x00400 (  
1024) load
I (139) esp_image: segment 3: paddr=0x0001de80 vaddr=0x40080400 size=0x02198 (  
8600) load
I (151) esp_image: segment 4: paddr=0x00020020 vaddr=0x400d0020 size=0x59678 
(366200) map
I (296) esp_image: segment 5: paddr=0x000796a0 vaddr=0x40082598 size=0x08718 ( 
34584) load
I (317) boot: Loaded app from partition at offset 0x1
I (317) boot: Disabling RNG early entropy source...
esp32_rng_initialize: Initializing RNG
esp_nvs_get_blob: INFO: No file /mnt/esp/wifi/wifi.misc.log
esp_nvs_set_blob: ERROR: Failed to set open /mnt/esp/wifi/wifi.misc.log
W (85) wifi:nvs_log_init, osi_nvs_set_blob fail, key=log ret=

esp_wifi_init: ERROR: Failed to initialize WiFi error=-1
esp_wifi_adapter_init: ERROR: Failed to initialize WiFi error=-1
ERROR: Failed to initialize WiFi

NuttShell (NSH) NuttX-10.0.1
nsh> ifconfig
nsh> ifup wlan0
ifup wlan0...OK
nsh> ifconfig
nsh>

I have used bootloader and partitions bin from 
https://github.com/saramonteiro/esp32_binaries_nuttx

Thanks,
Matias

Re: limitation in SIGEV_THREAD?

[Matias N.]

I was refering to the wait for the signal on the worker thread, which would 
simply be waiting on a semaphore/condition variable which indicates work 
(pending notification) is available. 

See here for how I changed how nimBLE uses SIGEV_THREAD due to the 
aforementioned issue:

https://github.com/apache/mynewt-nimble/blob/241911bd02b9bd3445f0194dab4539281fa3f849/porting/npl/nuttx/src/os_callout.c

In this case I'm using a condition variable. 

Any signals which are not desired from the thread can be masked, so no reason 
for it to be interrupted.

Best,
Matias

On Sun, Jan 31, 2021, at 00:49, Brennan Ashton wrote:
> On Sat, Jan 30, 2021 at 6:11 PM Matias N.  wrote:
> >
> >
> > > This is actually fairly in line with how the Musl libc implements this (at
> > > least from a quick look).  There are a few important details in there, but
> > > it looks quite clean.
> > >
> > > http://git.musl-libc.org/cgit/musl/tree/src/time/timer_create.c
> > >
> > > While Musl is usually not as fast as glibc, I find the implementations
> > > usually more straight forward and smaller code size.
> > >
> > > >
> > >
> >
> > From what I see glibc does a similar thing: has one thread that expects a 
> > signal (and waits for it with sigwait) and
> > spawns a new thread to run the user-supplied callback. I think that using a 
> > single thread to run the callback should be enough (although it could 
> > easily be made an option to spawn a new thread each time). In this scenario 
> > SIGEV_THREAD_ID ammounts to simply skipping the thread creation and letting 
> > the user specify which thread so send the signal to.
> >
> > What I'm unsure about is if using a signal is really necessary. It seems 
> > that using a semaphore/condition variable would be simpler and maybe faster.
> >
> 
> I think I must not be understanding what you are asking here.  Are you
> talking about somehow replacing the sigwaiitinfo with
> sem_wait/sem_timedwait? I'm not sure how that would actually be faster
> or simpler, with sem_wait you will be interrupted by any signal and
> then will have to figure out why and possibly wait again with a race
> condition (timer could have expired).  But it is very possible I am
> just not understanding how you would be using the semaphore.
> 
> --Brennan
> 

Re: limitation in SIGEV_THREAD?

[Matias N.]

> This is actually fairly in line with how the Musl libc implements this (at
> least from a quick look).  There are a few important details in there, but
> it looks quite clean.
> 
> http://git.musl-libc.org/cgit/musl/tree/src/time/timer_create.c
> 
> While Musl is usually not as fast as glibc, I find the implementations
> usually more straight forward and smaller code size.
> 
> >
> 

>From what I see glibc does a similar thing: has one thread that expects a 
>signal (and waits for it with sigwait) and
spawns a new thread to run the user-supplied callback. I think that using a 
single thread to run the callback should be enough (although it could easily be 
made an option to spawn a new thread each time). In this scenario 
SIGEV_THREAD_ID ammounts to simply skipping the thread creation and letting the 
user specify which thread so send the signal to.

What I'm unsure about is if using a signal is really necessary. It seems that 
using a semaphore/condition variable would be simpler and maybe faster.

Do you think we could do something like this? It would be great to make 
SIGEV_THREAD more usable.

Best,
Matias

Re: limitation in SIGEV_THREAD?

[Matias N.]

I'm thinking again about this. Why wouldn't it be possible to make functions
using SIGEV_THREAD (such as timer_settime) create a pthread behind
the scenes (only the first time a SIGEV_THREAD is setup)? The underlying
watchdog would go to a handler that posts a semaphore/condition variable
that the helper thread is waiting on. When this thread unblocks, it calls the
user handler directly.

The thread would be created per-process in KERNEL mode, so that shouldn't
be a problem (inside same address space as user handler). I suspect the
unblocking of the thread should also be possible in multi-process somehow
(named semaphore?).

This is essentially what I'm doing myself around a call to timer_settime.

Best,
Matias

On Wed, Jan 27, 2021, at 15:26, Gregory Nutt wrote:
> 
> > Perhaps you could use a pool of application threads as is done with 
> > the kernel threads for the low-priority work queue.  So you could have 
> > a small number of threads that service all tasks.  When a user-space 
> > thread is needed, it could be removed from the pool and be assigned to 
> > the task to run the event.  When the event processing completes, the 
> > thread returned to the pool until it is again needed.  Tasks could 
> > wait for availability if there are no available threads in the pool. 
> Nevermind!  This would not work in KERNEL mode.  In that case, each task 
> (now better called processes) have there own separate protected address 
> enviroments and threads could never be shared across processes.  It 
> would work fine in FLAT and PROTECTED modes where all tasks share the 
> same address space.
> 

Re: timerfd

[Matias N.]

On Sat, Jan 30, 2021, at 12:26, Xiang Xiao wrote:
> On Sat, Jan 30, 2021 at 7:00 AM Matias N.  wrote:
> 
> > But that isn't a userspace API, is it? It runs the handler in interrupt
> > context.
> >
> >
> The core logic for timerfd is part of the kernel(actually, timerfd/eventfd
> implements file_operation like a normal driver). libc timerfd api just a
> simple wrapper of ioctl.

I was referring to the watchdog you suggested.

Best,
Matias

Re: timerfd

[Matias N.]

But that isn't a userspace API, is it? It runs the handler in interrupt context.

I also realize now that timerfd has a limitation: you will not know about the 
timer expiration
while not yet doing poll, so it looses a bit on the real-time aspect.

Best,
Matias

On Sat, Jan 30, 2021, at 09:41, Xiang Xiao wrote:
> timerfd is a very nice feature, but is it better to call wd_* api than timer_ 
> api?
> 
> > -Original Message-
> > From: Matias N. 
> > Sent: Saturday, January 30, 2021 10:01 AM
> > To: dev@nuttx.apache.org
> > Subject: timerfd
> > 
> > Hi,
> > 
> > I would like to implement timerfd interface to overcome some of the issues 
> > around handling signals and threads and the limitation
> of
> > SIGEV_THREAD we discussed. I see eventfd is supported and looking at the 
> > implementation I think it can be done relatively simple
> using
> > most of the existing timer_* functionality. I was wondering if anyone 
> > already did this work outside of mainline or had any
> thoughts about
> > what to consider when doing so.
> > 
> > Best,
> > Matias
> 
> 

timerfd

[Matias N.]

Hi,

I would like to implement timerfd interface to overcome some of the issues 
around handling signals and threads and the limitation of SIGEV_THREAD we 
discussed. I see eventfd is supported and looking at the implementation I think 
it can be done relatively simple using most of the existing timer_* 
functionality. I was wondering if anyone already did this work outside of 
mainline or had any thoughts about what to consider when doing so.

Best,
Matias

Re: Raspberry Pi Pico a nice board for NuttX

[Matias N.]

There's an SVD file, so no need to use any other headers:

https://github.com/raspberrypi/pico-sdk/blob/master/src/rp2040/hardware_regs/rp2040.svd

We discussed in the past about the license that would apply to a header 
auto-generated from the SVD but I'm not sure if we arrived at any conclusion 
(the rationale was that in the end was data that would be obtained from the 
datasheet or from other public sources). Still, the SVD is BSD3 so we would be 
OK if we had to use that license to the output.

Best,
Matias

On Wed, Jan 27, 2021, at 21:37, Justin Mclean wrote:
> Hi,
> 
> > Copyright © 2020 Raspberry Pi (Trading) Ltd. The documentation of the 
> > RP2040 microcontroller is licensed under a Creative Commons 
> > Attribution-NoDerivatives 4.0 International (CC BY-ND). Portions Copyright 
> > © 2019 Synopsys, Inc. All rights reserved. Used with permission. Synopsys & 
> > DesignWare are registered trademarks of Synopsys, Inc. Portions Copyright © 
> > 2000-2001, 2005, 2007, 2009, 2011-2012, 2016 ARM Limited. All rights 
> > reserved. Used with permission.
> 
> Being BSD it abdul be OK. The other bits of code “used with permission” might 
> be an issue as they are unlikely to be under an open source license and NuttX 
> would need to get permission to use them. May need to clarify on the legal 
> discuss list.
> 
> Thanks,
> Justin

Re: limitation in SIGEV_THREAD?

[Matias N.]

I think the independent ELF solution would again not be very good for 
constrained systems.
The again, reading about SIGEV_THREAD makes me think that the specs are very 
loose in terms of
what is guaranteed by the thread running the callback.

These kind of issues (and the fact that mixing signals with threads is not 
good) is what I think makes
timerfd/signalfd/eventfd attractive. Having those the need for these kind of 
notifications is greatly diminished,
as you simply poll/select on your own thread, and it can be done for other kind 
of fds simultaneously.

Best,
Matias

On Wed, Jan 27, 2021, at 15:06, Gregory Nutt wrote:
> 
> > My thinking is that maybe upon setting up the first SIGEV_THREAD 
> > notification for a task, a child thread would be created, which would act 
> > as a worker, waiting on some semaphore for example. The bad thing is that 
> > it would linger until task is finished (not sure even how easy would be to 
> > ensure it is deleted on task exit).
> 
> This would be difficult because in PROTECTED and KERNEL modes, the OS 
> has no knowledge of any user-space symbols.  Creating a pthread is 
> different from starting a task because you have to the user-space 
> address of the pthread entry point.  That is not knowable in the cases 
> where the OS is separately linked.  So this could not be done by the 
> core OS as a general solution.  It could be done in some library 
> function if we had a user-space crt0.o to start the task.
> 
> There is already a crt0.o for ELF modules and it could start such a thread.
> 
> > Or is there a way for an lpwork thread to temporarily get into a task's 
> > environment?
> No, because an LP thread is a kernel thread.  It is privileged and if it 
> were to run user code in supervisor mode, that would be a major security 
> hole.
> > BTW, for C++ global constructors I think there's also the issue that they 
> > are only called at boot and not each time a task starts. At least I had 
> > that problem in the past.
> 
> This would require a different build model with tasks (or at least 
> task-related constructors/destructors) that are separately built and a 
> crt0.o to start any threads.
> 
> So one solution to both problems is to make all tasks into separately 
> linked ELF modules.
> 
> 
> 

Re: limitation in SIGEV_THREAD?

[Matias N.]

My thinking is that maybe upon setting up the first SIGEV_THREAD notification 
for a task, a child thread would be created, which would act as a worker, 
waiting on some semaphore for example. The bad thing is that it would linger 
until task is finished (not sure even how easy would be to ensure it is deleted 
on task exit).

Linux also supports (possibly due to this reason) SIGEV_THREAD_ID, which lets 
you specify a thread ID to send a signal to, although this is stated to be used 
internally.

Or is there a way for an lpwork thread to temporarily get into a task's 
environment?

BTW, for C++ global constructors I think there's also the issue that they are 
only called at boot and not each time a task starts. At least I had that 
problem in the past.

Best,
Matias

On Wed, Jan 27, 2021, at 14:45, Xiang Xiao wrote:
> All global C++ objects are constructed in init task, so if these objects
> call open, fopen or fstream related API, the similar issue happens too. How
> can we fix this issue?
> BTW, is it acceptable to create a pthread in each task to
> process SIGEV_THREAD dedicatedly in the small embedded device? Can we find
> a lightweight solution to fix it?
> We have encountered this issue many times, and workaround by calling
> file_open directly from the userspace, but it is obviously not the right
> way to go.
> 
> Thanks
> Xiang
> 
> On Tue, Jan 26, 2021 at 3:12 PM Gregory Nutt  wrote:
> 
> > Yes, you are right.  Perhaps this could be added to
> > https://github.com/apache/incubator-nuttx/issues/1352
> >
> > That is a different issue description but might still be a good location
> > because both issues have the same solution.  The solution to both issues
> > is to enhance the OS by adding a new OS facility to start a pthread from
> > within OS just as if pthread_create() were called from that application
> > task.  That is not really that difficult.  The SIGEV_THREAD logic would
> > then run on that pthread.
> >
> > This same facility is also needed to correct the implementation of the
> > AIO APIs.  They also should run on pthreads as well. However, I am not
> > aware of any functional shortcoming of the current AIO implementation
> > that would justify the change.
> >
> > The only unattractive thing about this solution is that is does require
> > more resources and is less efficient in general.
> >
> > Greg
> >
> > On 1/26/2021 4:26 PM, Matias N. wrote:
> > > Hi,
> > > working with nimBLE I found that I had an issue when scheduling a
> > callback to be made from within the signal handler for a timer, which was
> > set with SIGEV_THREAD. The issue was that I was pushing to a POSIX queue
> > from within the handler, and it was failing with BADFD. From debugging I
> > realized that when trying to obtain the inode for the queue it was looking
> > up at the open file descriptors from the lpwork thread, which of course
> > would not share open file descriptors with main task.
> > >
> > > Since this is working for Linux (I had copied this part of the porting
> > layer from Linux) my reasoning is that SIGEV_THREAD is there implemented as
> > a thread that is a child of the process which registered the signal handler
> > and thus shares the open file descriptors. In NuttX this is implemented via
> > a work queue so this is not the case.
> > >
> > > Is my reading correct? If so, it may be worth adding this limitation to
> > https://github.com/apache/incubator-nuttx/issues/1352
> > >
> > > Best,
> > > Matias
> >
> 

Re: limitation in SIGEV_THREAD?

[Matias N.]

Thanks, I'll add a comment on that issue for reference.

On Tue, Jan 26, 2021, at 20:12, Gregory Nutt wrote:
> Yes, you are right.  Perhaps this could be added to 
> https://github.com/apache/incubator-nuttx/issues/1352
> 
> That is a different issue description but might still be a good location 
> because both issues have the same solution.  The solution to both issues 
> is to enhance the OS by adding a new OS facility to start a pthread from 
> within OS just as if pthread_create() were called from that application 
> task.  That is not really that difficult.  The SIGEV_THREAD logic would 
> then run on that pthread.
> 
> This same facility is also needed to correct the implementation of the 
> AIO APIs.  They also should run on pthreads as well. However, I am not 
> aware of any functional shortcoming of the current AIO implementation 
> that would justify the change.
> 
> The only unattractive thing about this solution is that is does require 
> more resources and is less efficient in general.
> 
> Greg
> 
> On 1/26/2021 4:26 PM, Matias N. wrote:
> > Hi,
> > working with nimBLE I found that I had an issue when scheduling a callback 
> > to be made from within the signal handler for a timer, which was set with 
> > SIGEV_THREAD. The issue was that I was pushing to a POSIX queue from within 
> > the handler, and it was failing with BADFD. From debugging I realized that 
> > when trying to obtain the inode for the queue it was looking up at the open 
> > file descriptors from the lpwork thread, which of course would not share 
> > open file descriptors with main task.
> >
> > Since this is working for Linux (I had copied this part of the porting 
> > layer from Linux) my reasoning is that SIGEV_THREAD is there implemented as 
> > a thread that is a child of the process which registered the signal handler 
> > and thus shares the open file descriptors. In NuttX this is implemented via 
> > a work queue so this is not the case.
> >
> > Is my reading correct? If so, it may be worth adding this limitation to 
> > https://github.com/apache/incubator-nuttx/issues/1352
> >
> > Best,
> > Matias
> 

limitation in SIGEV_THREAD?

[Matias N.]

Hi,
working with nimBLE I found that I had an issue when scheduling a callback to 
be made from within the signal handler for a timer, which was set with 
SIGEV_THREAD. The issue was that I was pushing to a POSIX queue from within the 
handler, and it was failing with BADFD. From debugging I realized that when 
trying to obtain the inode for the queue it was looking up at the open file 
descriptors from the lpwork thread, which of course would not share open file 
descriptors with main task.

Since this is working for Linux (I had copied this part of the porting layer 
from Linux) my reasoning is that SIGEV_THREAD is there implemented as a thread 
that is a child of the process which registered the signal handler and thus 
shares the open file descriptors. In NuttX this is implemented via a work queue 
so this is not the case.

Is my reading correct? If so, it may be worth adding this limitation to 
https://github.com/apache/incubator-nuttx/issues/1352

Best,
Matias

Re: Raspberry Pi Pico a nice board for NuttX

[Matias N.]

> Is there any easy way to get all of the register definition header files 
> in place (with Apache 2 headers)?

If there's an SVD file somewhere, you can use this:

https://gitlab.com/nuttx_projects/svdgen

I've used it many times and requires minimal tweaking afterwards.

Best,
Matias

Re: software i2c?

[Matias N.]

Thanks a lot!
I will convert it to a proper NuttX driver, test it and open a PR.

Best,
Matias

On Mon, Jan 18, 2021, at 17:05, Fotis Panagiotopoulos wrote:
> Also the Maple lib is C++, while my code is pure C.
> 
> The code is not published anywhere, till now.
> 
> I am attaching it here.
> If you have any problems, I can create a new github repo for you.
> 
> 
> 
> *Attachments:*
>  * i2c.c
>  * i2c.h

Re: software i2c?

[Matias N.]

Ok, so if you feel that your code is now no longer related to that since it is 
actually completely rewritten
I think it is no problem. Otherwise, if you think there may still be code from 
there, the one you linked is MIT
so at least I think it could be a compatible license with Apache 2.0.

Do you have your code available somewhere? That way I can take a look and think 
how it can be integrated
to NuttX. Also, if you can place the apropriate license header already it would 
be better.

Best,
Matias

On Mon, Jan 18, 2021, at 15:42, Fotis Panagiotopoulos wrote:
> It is very possible that this was the original library:
> 
> https://github.com/leaflabs/libmaple/tree/master/libraries/Wire
> 
> By a quick look of it, it "seems" unrelated to my current code.
> 
> 
> 
> Στις Δευ, 18 Ιαν 2021 στις 8:40 μ.μ., ο/η Fotis Panagiotopoulos <
> f.j.pa...@gmail.com> έγραψε:
> 
> > >(maybe by googling a piece of code or comment)
> > Nah... Impossible. It certainly looks nothing like the original code. They
> > may not even share a common line of code any more.
> > This was not a "port" of an external library, rather I used the lib as a
> > basis to write my own think.
> >
> > At least to the part I remember, it was many years ago.
> >
> > I am looking for this maple library, but I think that it is not publicly
> > available any more.
> > The current libraries are a different thing I believe.
> >
> >
> > Στις Δευ, 18 Ιαν 2021 στις 8:22 μ.μ., ο/η Matias N. 
> > έγραψε:
> >
> >> Hi,
> >> that's great.
> >>
> >> I don't mind adapting it for NuttX. I think it could be made to have a
> >> hardware dependant lower-half to abstract
> >> the calls to GPIO handling.
> >>
> >> The part I'm unsure about is the licensing. If it is based on some code
> >> you cannot identify I think it would not
> >> be possible to include it unless we know the original license is
> >> compatible. Do you think you can identify
> >> where it came from? (maybe by googling a piece of code or comment)
> >>
> >> Best,
> >> Matias
> >>
> >> On Mon, Jan 18, 2021, at 15:12, Fotis Panagiotopoulos wrote:
> >> > Hi,
> >> >
> >> > I do have one.
> >> > It was designed to run on STM32F103s where the I2C peripheral has some
> >> > errata limitations.
> >> > I have used it extensively on many projects without any issues.
> >> > Although I used it only for "simple" transfers of just a few bytes, I
> >> don't
> >> > see any reason it wouldn't work in any other scenario. Apart from the
> >> low
> >> > performance, of course.
> >> >
> >> > It is properly documented, and generally uses my own HAL. So hardware
> >> calls
> >> > are abstracted and self-explanatory.
> >> >
> >> > Two issues:
> >> > 1. It needs some work to get into NuttX. It has a very different
> >> structure,
> >> > it's not just styling that needs to be changed.
> >> > I believe however that it is a good basis. I am also willing to help
> >> with
> >> > testing this on my targets.
> >> >
> >> > 2. License. I am only using it on some of my small personal projects,
> >> so I
> >> > just didn't pay much attention to licencing.
> >> > It loosely originates from an open source library. I think it was the
> >> > arduino-compatible library of Maple (https://www.leaflabs.com/maple),
> >> but I
> >> > am not sure.
> >> > Surely I have re-written most of it, to the point that I don't know
> >> whether
> >> > there is actually any original code in there, but again it may need to
> >> be
> >> > checked.
> >> > In any case, it may serve as a basis for freshly written NuttX code.
> >> >
> >> > If this is of any interest to you, please tell me and I can provide the
> >> > code.
> >> >
> >> >
> >> > Στις Δευ, 18 Ιαν 2021 στις 7:29 μ.μ., ο/η Matias N. 
> >> έγραψε:
> >> >
> >> > > Hi,
> >> > >
> >> > > by any chance does anyone have a software I2C implementation laying
> >> around
> >> > > (but tested) that could be submitted to NuttX? If the basis is there
> >> I can
> >> > > do any required adaptation.
> >> > >
> >> > > Thanks,
> >> > > Matias
> >> >
> >>
> >
> 

Re: software i2c?

[Matias N.]

Hi, 
that's great.

I don't mind adapting it for NuttX. I think it could be made to have a hardware 
dependant lower-half to abstract
the calls to GPIO handling. 

The part I'm unsure about is the licensing. If it is based on some code you 
cannot identify I think it would not
be possible to include it unless we know the original license is compatible. Do 
you think you can identify 
where it came from? (maybe by googling a piece of code or comment)

Best,
Matias

On Mon, Jan 18, 2021, at 15:12, Fotis Panagiotopoulos wrote:
> Hi,
> 
> I do have one.
> It was designed to run on STM32F103s where the I2C peripheral has some
> errata limitations.
> I have used it extensively on many projects without any issues.
> Although I used it only for "simple" transfers of just a few bytes, I don't
> see any reason it wouldn't work in any other scenario. Apart from the low
> performance, of course.
> 
> It is properly documented, and generally uses my own HAL. So hardware calls
> are abstracted and self-explanatory.
> 
> Two issues:
> 1. It needs some work to get into NuttX. It has a very different structure,
> it's not just styling that needs to be changed.
> I believe however that it is a good basis. I am also willing to help with
> testing this on my targets.
> 
> 2. License. I am only using it on some of my small personal projects, so I
> just didn't pay much attention to licencing.
> It loosely originates from an open source library. I think it was the
> arduino-compatible library of Maple (https://www.leaflabs.com/maple), but I
> am not sure.
> Surely I have re-written most of it, to the point that I don't know whether
> there is actually any original code in there, but again it may need to be
> checked.
> In any case, it may serve as a basis for freshly written NuttX code.
> 
> If this is of any interest to you, please tell me and I can provide the
> code.
> 
> 
> Στις Δευ, 18 Ιαν 2021 στις 7:29 μ.μ., ο/η Matias N.  έγραψε:
> 
> > Hi,
> >
> > by any chance does anyone have a software I2C implementation laying around
> > (but tested) that could be submitted to NuttX? If the basis is there I can
> > do any required adaptation.
> >
> > Thanks,
> > Matias
> 

software i2c?

[Matias N.]

Hi,

by any chance does anyone have a software I2C implementation laying around (but 
tested) that could be submitted to NuttX? If the basis is there I can do any 
required adaptation.

Thanks,
Matias

Re: adding support for stack coloration on nRF52

[Matias N.]

Thanks, I tested simply copying the code and it works. I will probably make a 
PR for that and we can then refactor the code to have a single place for each 
arch.

Best,
Matias

On Sun, Jan 17, 2021, at 05:06, Xiang Xiao wrote:
> On Sat, Jan 16, 2021 at 6:08 PM Matias N.  wrote:
> 
> > Hi,
> > I would like to add this feature so I can measure used stack on nRF52. I'm
> > trying to figure out what
> > is needed and it would seem I need to replicate the go_nx_start() function
> > found on other _start.c
> > files. Is that all there is to it?
> >
> >
> it's only for idle thread coloration, other thread stack coloration is
> handled by common code.
> 
> 
> > Also, any reason why this is chip dependant and not simply arch dependant?
> > It would seem this piece of code
> > is the same for all chip families inside a given arch.
> >
> >
> Yes,  go_nx_start could share the same code in one arch.
> 
> While we're at it, stack check code also involves replicating the few lines
> > found on _start.c right? Is this
> > also really chip dependant?
> >
> > Best,
> > Matias
> 

adding support for stack coloration on nRF52

[Matias N.]

Hi, 
I would like to add this feature so I can measure used stack on nRF52. I'm 
trying to figure out what
is needed and it would seem I need to replicate the go_nx_start() function 
found on other _start.c
files. Is that all there is to it?

Also, any reason why this is chip dependant and not simply arch dependant? It 
would seem this piece of code
is the same for all chip families inside a given arch.

While we're at it, stack check code also involves replicating the few lines 
found on _start.c right? Is this
also really chip dependant?

Best,
Matias

monitoring task stack use from gdb?

[Matias N.]

Hi,

I'm wondering if the gdbinit script could be extended to read task memory usage 
(as would be displayed by ps command) for each task, when using the 
info_nxthreads command. It would be really useful to easily detect improper 
stack size configuration.

Best,
Matias

Re: condition variables and signals

[Matias N.]

Thanks Greg for the insight into the problem. The fact that the signal handler
runs in the same thread was not something I thought. Also I think it confirms 
using
SIGEV_THREAD for this is the safe approach in this scenario.

Best,
Matias

On Wed, Jan 13, 2021, at 19:06, Gregory Nutt wrote:
> 
> > I am thinking that there must be some race condition:  When the 
> > condition variable is created the cond->sem value will be set to 
> > zero.  While waiting for cond->sem, it will be set to -1. If you see 
> > cond->sem equal to zero, my guess would be that the signal was 
> > received and cond->sem was incremented asychronously by the signal 
> > delivery logic.  But that is only a guess.
> >
> I know what is going on.  There is no race condition.  It is normal 
> behavior for a signal handler:
> 
>   * The signal handler runs on the same thread that receives the signal
>   * In order for the signal handler to run, the thread must be running. 
> If the thread is blocked, then it must be unblocked so that the
> signal handler can run (that is why EINTR is returned under POSIX...
> To indicate only that the thread had to be unblocked and awakened to
> process a signal).
>   * If the thread is blocked waiting on a semaphore, then the semaphore
> count must be incremented in order to unblock the thread.
>   * This must happen BEFORE the signal handler runs.
> 
> So there is no race condition, instead there is a simple ordering issue  
> The sequence above should be modified like:
> 
>   * When the condition variable is created the cond->sem value will be
> set to zero.
>   * While waiting for cond->sem, the value will be decremented to -1
>   * When a signal is received, the thread will be awakened be
> incremented back to zero.
>   * THEN the thread will run and signal handler will be invoked with the
> cond->sem value ALWAYS equal to zero.
> 
> So it is perfectly normal behavior that the cond->sem count is zero in 
> this case.  It just indicates that the thread is running. So, no, I was 
> wrong.  It is not possible to use pthread_cond_signal() form a thread to 
> wake itself up BECAUSE it already had to be awakened in order for the 
> signal handler to run.
> 
> When the signal handler returns, pthread_cond_wait() will run inside of 
> nxsem_wait_interruptible() and it will correctly ignore the EINTR error 
> and call nxsem_wait() again.
> 
> 
> 
> 

Re: condition variables and signals

[Matias N.]

Thanks. I'm using FLAT mode so no problem.

Best,
Matias

On Wed, Jan 13, 2021, at 11:40, Gregory Nutt wrote:
> Just beware of https://github.com/apache/incubator-nuttx/issues/1352
> 
> On 1/13/2021 8:09 AM, Matias N. wrote:
>> Hi,
>> thanks for your responses. Yes, upon more reading I realized mixing signals 
>> with pthread mutexes was not safe. I guess
>> I was getting a race condition inside the mutex locking.
>> 
>> As a workaround, I resorted to using SIGEV_THREAD notification for POSIX 
>> timers. This appears to work and I guess it
>> should be safe. Eventually I will look into implementing timerfd, if anyone 
>> else hasn't done so yet.
>> 
>> Best,
>> Matias
>> 
>> On Wed, Jan 13, 2021, at 05:52, Juha Niskanen (Haltian) wrote:
>>>> BTW, looking at the spec for pthread_cond_wait, there's actually no 
>>>> mention about a limitation regarding using pthread_cond_signal invoked 
>>>> from within a signal handler to unblock a pthread_cond_wait.
>>> The restriction is elsewhere. pthread_cond_signal() is not in the list of 
>>> async-signal-safe functions at
>>> 
>>> https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>>> 
>>> Therefore, portable programs may not call it from a signal handler and 
>>> except it to work. Asynchronous signal handler can get called between any 
>>> two instructions so safe synchronization with condition variables might be 
>>> hard to implement. Use semaphores and sem_post() instead.
>>> 
>>> -Juha
>>> 
>>> 
>>> 
>>> *From:* Gregory Nutt 
>>> *Sent:* Wednesday, January 13, 2021 4:24 AM
>>> *To:* dev@nuttx.apache.org 
>>> *Subject:* Re: condition variables and signals
>>>  
>>> 
>>>> BTW, looking at the spec for pthread_cond_wait, there's actually no 
>>>> mention about a limitation regarding using pthread_cond_signal invoked 
>>>> from within a signal handler to unblock a pthread_cond_wait. However, this 
>>>> does not work
>>>> either for me. If you follow the signal operation, I can see that when it 
>>>> reaches the nxsem_get_value() on the cond->sem, the semaphore value is 
>>>> zero and the sem_give is not done.
>>>> 
>>>> 
>>>> Should this work?
>>> Yes, calling pthread_cond_signal() should work, however, no signal should 
>>> be delivered.  The word "signal" is overloaded:  Here the condition is 
>>> signaled but no signal should be sent so I am not quite sure what I am 
>>> looking at.  Apparently some other logic is signaling the task group 
>>> asynchronously?

>>> I am thinking that there must be some race condition:  When the condition 
>>> variable is created the cond->sem value will be set to zero.  While waiting 
>>> for cond->sem, it will be set to -1.  If you see cond->sem equal to zero, 
>>> my guess would be that the signal was received and cond->sem was 
>>> incremented asychronously by the signal delivery logic.  But that is only a 
>>> guess.

>>> It would be good to check the behavior without the signal interfering.

>>> 

>>> 

>>> 

>>> 
>>> *Attachments:*
>>>  * image.png
>> 
> 

> 
> *Attachments:*
>  * image.png

Re: condition variables and signals

[Matias N.]

Hi,
thanks for your responses. Yes, upon more reading I realized mixing signals 
with pthread mutexes was not safe. I guess
I was getting a race condition inside the mutex locking.

As a workaround, I resorted to using SIGEV_THREAD notification for POSIX 
timers. This appears to work and I guess it
should be safe. Eventually I will look into implementing timerfd, if anyone 
else hasn't done so yet.

Best,
Matias

On Wed, Jan 13, 2021, at 05:52, Juha Niskanen (Haltian) wrote:
>> BTW, looking at the spec for pthread_cond_wait, there's actually no mention 
>> about a limitation regarding using pthread_cond_signal invoked from within a 
>> signal handler to unblock a pthread_cond_wait.
> The restriction is elsewhere. pthread_cond_signal() is not in the list of 
> async-signal-safe functions at
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
> 
> Therefore, portable programs may not call it from a signal handler and except 
> it to work. Asynchronous signal handler can get called between any two 
> instructions so safe synchronization with condition variables might be hard 
> to implement. Use semaphores and sem_post() instead.
> 
> -Juha
> 
> 
> 
> *From:* Gregory Nutt 
> *Sent:* Wednesday, January 13, 2021 4:24 AM
> *To:* dev@nuttx.apache.org 
> *Subject:* Re: condition variables and signals 
>  
> 
>> BTW, looking at the spec for pthread_cond_wait, there's actually no mention 
>> about a limitation regarding using pthread_cond_signal invoked from within a 
>> signal handler to unblock a pthread_cond_wait. However, this does not work
>> either for me. If you follow the signal operation, I can see that when it 
>> reaches the nxsem_get_value() on the cond->sem, the semaphore value is zero 
>> and the sem_give is not done.
>> 
>> 
>> Should this work?
> Yes, calling pthread_cond_signal() should work, however, no signal should be 
> delivered.  The word "signal" is overloaded:  Here the condition is signaled 
> but no signal should be sent so I am not quite sure what I am looking at.  
> Apparently some other logic is signaling the task group asynchronously?

> I am thinking that there must be some race condition:  When the condition 
> variable is created the cond->sem value will be set to zero.  While waiting 
> for cond->sem, it will be set to -1.  If you see cond->sem equal to zero, my 
> guess would be that the signal was received and cond->sem was incremented 
> asychronously by the signal delivery logic.  But that is only a guess.

> It would be good to check the behavior without the signal interfering.

> 

> 

> 

> 
> *Attachments:*
>  * image.png

Re: condition variables and signals

[Matias N.]

> Note that I was initially signaling the condition variable from within the 
> signal handler but this did not work either. However, looking at pthread 
> documentation this is supposedly not supported: 
> https://linux.die.net/man/3/pthread_cond_signal
> (I'm not sure if that really applies to NuttX though).
> 

BTW, looking at the spec for pthread_cond_wait, there's actually no mention 
about a limitation regarding using pthread_cond_signal invoked from within a 
signal handler to unblock a pthread_cond_wait. However, this does not work
either for me. If you follow the signal operation, I can see that when it 
reaches the nxsem_get_value() on the cond->sem, the semaphore value is zero and 
the sem_give is not done.

Should this work?

Best,
Matia

Re: condition variables and signals

[Matias N.]

Ahh ok. Thanks for that, should've looked further into the spec. I assumed from 
other documentation I read that it could work.

So, now I'm a bit stuck on mixing the signal-based POSIX timers with mutexes. I 
will see about other synchronization mechanism.

Best,
Matias

On Tue, Jan 12, 2021, at 21:19, Gregory Nutt wrote:
> 
> > pthread_cond_wait() is not supposed to return if a signal is received: 
> > https://pubs.opengroup.org/onlinepubs/009696699/functions/pthread_cond_wait.html
> >
> > EINTR is not one of the valid return values from pthread_cond_wait().
> 
> This is the specific POSIX requirement that must be followed: "If a 
> signal is delivered to a thread waiting for a condition variable, upon 
> return from the signal handler the thread resumes waiting for the 
> condition variable as if it was not interrupted, or it shall return zero 
> due to spurious wakeup."
> 
> "Spurious wakeups" are not handled.
> 
> 

Re: condition variables and signals

[Matias N.]

I'm not expecting the task to be killed, but the pthread_cond_wait to return 
when the process receives a signal (which I'm handling in a signal handler).

What I need is to exit the pthread_cond_wait upon reception of the signal. As I 
mentioned, doing pthread_cond_signal from the handler did not work (and seems 
not to be safe) and now I was trying to enable cancellation points so that the 
signal itself cancels pthread_cond_wait.

Best,
Matias

On Tue, Jan 12, 2021, at 21:00, Gregory Nutt wrote:
> 
> 
> > I'm having an issue when waiting on a pthread condition variable from 
> > the main thread and then a signal handler runs, which should cancel 
> > the wait since I have cancellation points enabled, however this did 
> > not happen.
> 
> I don't understand this either.  A signal will not, in general, cause a 
> task to be killed.  The DEFAULT behavior of some signals (if enabled) 
> will kill the task.
> 
> However, if you have connected a signal handler via sigaction(), then 
> you have also disabled the default behavior and the task will not be killed.
> 
> 

Re: condition variables and signals

[Matias N.]

The wait I'm referring to is the one on the semaphore underlying to the 
condition variable (pthread_sem_take on cond->sem). This ends up as a call to 
nxsem_wait_uninterruptible which loops on the wait to ignore EINTR errors. So I 
understand that a signal would not interrupt the wait (and thus, the call to 
pthread_cond_wait).

On Tue, Jan 12, 2021, at 20:34, Gregory Nutt wrote:
> 
> > I'm having an issue when waiting on a pthread condition variable from 
> > the main thread and then a signal handler runs, which should cancel 
> > the wait since I have cancellation points enabled, however this did 
> > not happen. While debugging I see the main thread blocked when 
> > standing inside the signal handler. Here's the call graph at that point:
> 
> There are two semaphores associated with a condition variable wait:  a) 
> One for the conditional variable and b) one for a mutex that protects 
> the condition variable.  When you wait for a condition, the mutex (b) is 
> unlocked and the logic waits on (a). After (a) wakes up, then mutex (b) 
> is re-relocked.  So there can be two waits.
> 
> What is not clear from the call graph is if the sem_wait() is the first 
> for the condition variable or the second for the mutex. 
> nxsem_wait_interruptible() is fine for for the first, but not for the 
> second if ECANCELED was returned.
> 
> 
> 

Re: condition variables and signals

[Matias N.]

As a quick test, I made pthread_cond_wait call to pthread_sem_take with the 
last argument to true (which then use the interruptible version) and it now 
cancels the wait. So it would seem that this is a bug right?

Best,
Matias

On Tue, Jan 12, 2021, at 20:13, Matias N. wrote:
> Hi,
> I'm having an issue when waiting on a pthread condition variable from the 
> main thread and then a signal handler runs, which should cancel the wait 
> since I have cancellation points enabled, however this did not happen. While 
> debugging I see the main thread blocked when standing inside the signal 
> handler. Here's the call graph at that point:
> 
> If I continue execution, the main thread never runs. I'm wondering if that 
> call to nxsem_wait_uninterruptible is correct since it would seem that is 
> exactly why this is not being canceled.
> 
> Note that I was initially signaling the condition variable from within the 
> signal handler but this did not work either. However, looking at pthread 
> documentation this is supposedly not supported: 
> https://linux.die.net/man/3/pthread_cond_signal
> (I'm not sure if that really applies to NuttX though).
> 
> Also, I'm mixing signals with threads because I'm using POSIX timers which 
> require signals to be handled, so I'm stuck with this. I think supporting 
> timerfd would be really useful (as I could then have a thread that waits on 
> the fd of the timer).
> 
> Best,
> Matias
> 
> *Attachments:*
>  * image.png

condition variables and signals

[Matias N.]

Hi,
I'm having an issue when waiting on a pthread condition variable from the main 
thread and then a signal handler runs, which should cancel the wait since I 
have cancellation points enabled, however this did not happen. While debugging 
I see the main thread blocked when standing inside the signal handler. Here's 
the call graph at that point:

If I continue execution, the main thread never runs. I'm wondering if that call 
to nxsem_wait_uninterruptible is correct since it would seem that is exactly 
why this is not being canceled.

Note that I was initially signaling the condition variable from within the 
signal handler but this did not work either. However, looking at pthread 
documentation this is supposedly not supported: 
https://linux.die.net/man/3/pthread_cond_signal
(I'm not sure if that really applies to NuttX though).

Also, I'm mixing signals with threads because I'm using POSIX timers which 
require signals to be handled, so I'm stuck with this. I think supporting 
timerfd would be really useful (as I could then have a thread that waits on the 
fd of the timer).

Best,
Matias

Re: I2C on nRF52840

[Matias N.]

I actually found it was my mistake (I was using the wrong pinout).

Thanks!
Matias

On Mon, Jan 11, 2021, at 04:46, raiden00pl wrote:
> Does your I2C device use I2C_M_NOSTART or I2C_M_NOSTOP flags for
> communication ? They are not supported by the current NRF52 I2C
> implementation.
> 
> niedz., 10 sty 2021 o 05:42 Matias N.  napisał(a):
> 
> > I was about to enable error support but I noticed it is disabled in-code.
> > From the look of it there's not that
> > much else to do but to issue a STOP.
> >
> > Regarding the error I'm receiving, I'm seeing a NACK for every possible
> > address, so I either have the wrong set of
> > pins for the device or there something else going on.
> >
> > Best,
> > Matias
> >
> > On Sun, Jan 10, 2021, at 00:36, Brennan Ashton wrote:
> > > If I recall the error handling logic is largely missing for the I2C in
> > > interrupt mode. I added some more to improve the situation for polled
> > mode.
> > > Have you tried that?
> > >
> > > --Brennan
> > >
> > > On Sat, Jan 9, 2021, 7:34 PM Matias N.  wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm having a hard time getting I2C working on nRF52840, the I2C read
> > > > operation seems to hang (never reaches ISR). Depending on which set of
> > pins
> > > > (connected to different I2C device) it either hangs on TXSTARTED or I
> > see
> > > > the ERROR event set to 0x1. Since this is on a closed device I cannot
> > > > verify signals so I'm not sure if I'm making a mistake or the I2C code
> > is
> > > > faulty. Anyone currently using I2C on this chip with code from master?
> > > >
> > > > Best,
> > > > Matias
> > >
> >
> 

Re: nrf52 input buffer

[Matias N.]

I will make a PR with a fix soon.
It seems that the input buffer is internally connected under certain 
conditions. But I found some case where the pin value was not being read 
correctly so I suppose this could be the issue.

Best,
Matias

On Mon, Jan 11, 2021, at 06:33, raiden00pl wrote:
> It looks like a bug. But for some reason with the input buffer turned off,
> reading the input pin still works (nrf52840).
> 
> niedz., 10 sty 2021 o 19:17 Matias N.  napisał(a):
> 
> > Hi,
> > while debugging the issue I mentioned in the other thread, I found
> > something strange: when configuring a pin as input on nRF52, the input
> > buffer is not enabled (by clearing the relevant INPUT bit of the PIN_CFG
> > register):
> > https://github.com/apache/incubator-nuttx/blob/master/arch/arm/src/nrf52/nrf52_gpio.c#L94
> >
> > Is this a bug? Or am I missing something?
> >
> > Best,
> > Matias
> 

nrf52 input buffer

[Matias N.]

Hi,
while debugging the issue I mentioned in the other thread, I found something 
strange: when configuring a pin as input on nRF52, the input buffer is not 
enabled (by clearing the relevant INPUT bit of the PIN_CFG register): 
https://github.com/apache/incubator-nuttx/blob/master/arch/arm/src/nrf52/nrf52_gpio.c#L94

Is this a bug? Or am I missing something?

Best,
Matias