Hi Fabio, What are the capabilities?
It this 1 RPi/BBB per board nuttx board? David -----Original Message----- From: Fabio Balzano [mailto:fa...@elfarolab.com] Sent: Friday, December 20, 2019 5:06 AM To: dev@nuttx.apache.org Subject: Re: [DISCUSS - NuttX Workflow] Hello, yes the buildbot server is down, later today I will bring it up. Yes you can do remote builds using a RPI/BBB or similars or local builds performed by the server itself. I can setup and maintain the server for the Nuttx project in case you think it is useful. Thank you so much Fabio Balzano > On 20 Dec 2019, at 13:00, Alan Carvalho de Assis <acas...@gmail.com> > wrote: > > Hi David, > > Sorry for scolding you in public as well, but I think we don't need to > find guilt. > > So, I got the impression you were doing it to promote PX4 test > workflow as the best solution for all the NuttX issues. > > And although 300K drones are a lot, there are many commercial products > using NuttX. Many Sony audio recorders, Moto Z Snaps, Thermal > printers, etc. Probably we have products that overcome that number. > > I think recently Fabio changed the buildbot link. BTW I just remember > other alternative that Sebastien and I did about 3 years ago: > > https://bitbucket.org/acassis/raspi-nuttx-farm/src/master/ > > The idea was to use low cost Raspberry PIs as a distributed build test > for NuttX. It worked fine! You just define a board file with the > configuration you want to test and it is done. > > BR, > > Alan > >> On 12/20/19, David Sidrane <davi...@apache.org> wrote: >> Hi Alan, >> >> Sorry if my intent was misunderstood. I am merely stating facts on were >> we >> are and how got there.I am not assigning blame. I am not forcing anything >> I >> am giving some examples of how we can make it the project complete and >> better. We can use all of it, some of it none of it. The is a group >> decision. >> >> Also Pleases do fill us in on where we can see the SW CI & HW CI you >> mentioned. Do you have links maybe be we can use it now? >> >> Again Sorry! >> >> David >> >>> On 2019/12/20 11:44:23, Alan Carvalho de Assis <acas...@gmail.com> >>> wrote: >>> Hi David, >>> >>>> On 12/20/19, David Sidrane <davi...@apache.org> wrote: >>>> Hi Nathan, >>>> >>>> On 2019/12/20 02:51:56, Nathan Hartman <hartman.nat...@gmail.com> >>>> wrote: >>>>> On Thu, Dec 19, 2019 at 6:24 PM Gregory Nutt <spudan...@gmail.com> >>>>> wrote: >>>>>>>> ] A bad build system change can cause serious problems for a lot >>>>>>>> of >>>>> people around the world. A bad change in the core OS can destroy the >>>>> good >>>>> reputation of the OS. >>>>>>> Why is this the case? Users should not be using unreleased code or >>>>>>> be >>>>> encouraged to use it.. If they are one solution is to make more >>>>> frequent >>>>> releases. >>>>>> I don't think that the number of releases is the factor. It is time >>>>>> in >>>>>> people's hand. Subtle corruption of OS real time behavior is not >>>>>> easily >>>>>> testing. You normally have to specially instrument the software >>>>>> and >>>>>> setup a special test environment perhaps with a logic analyzer to >>>>>> detect >>>>>> these errors. Errors in the core OS can persists for months and in >>>>>> at >>>>>> least one case I am aware of, years, until some sets up the correct >>>>>> instrumented test. >>>>> >>>>> And: >>>>> >>>>> On Thu, Dec 19, 2019 at 4:20 PM Justin Mclean >>>>> <jus...@classsoftware.com> >>>>> wrote: >>>>>>> ] A bad build system change can cause serious problems for a lot >>>>>>> of >>>>> people around the world. A bad change in the core OS can destroy the >>>>> good >>>>> reputation of the OS. >>>>>> >>>>>> Why is this the case? Users should not be using unreleased code or >>>>>> be >>>>> encouraged to use it.. If they are one solution is to make more >>>>> frequent >>>>> releases. >>>>> >>>>> Many users are only using released code. However, whatever is in >>>>> "master" >>>>> eventually gets released. So if problems creep in unnoticed, >>>>> downstream >>>>> users will be affected. It is only delayed. >>>>> >>>>> I can personally attest that those kinds of errors are extremely >>>>> difficult >>>>> to detect and trace. It does require a special setup with logic >>>>> analyzer >>>>> or >>>>> oscilloscope, and sometimes other tools, not to mention a whole setup >>>>> to >>>>> produce the right stimuli, several pieces of software that may have to >>>>> be >>>>> written specifically for the test.... >>>>> >>>>> I have been wracking my brain on and off thinking about how we could >>>>> set >>>>> up >>>>> an automated test system to find errors related to timing etc. >>>>> Unfortunately unlike ordinary software for which you can write an >>>>> automated >>>>> test suite, this sort of embedded RTOS will need specialized hardware >>>>> to >>>>> conduct the tests. That's a subject for another thread and i don't >>>>> know >>>>> if >>>>> now is the time, but I will post my thoughts eventually. >>>>> >>>>> Nathan >>>> >>>> From the proposal >>>> >>>> "Community >>>> >>>> NuttX has a large, active community. Communication is via a Google >>>> group at >>>> https://groups.google.com/forum/#!forum/nuttx where there are 395 >>>> members as >>>> of this writing. Code is currently maintained at Bitbucket.org at >>>> https://bitbucket.org/nuttx/. Other communications are through >>>> Bitbucket >>>> issues and also via Slack for focused, interactive discussions." >>>> >>>> >>>>> Many users are only using released code. >>>> >>>> Can we ask the 395 members? >>>> >>>> I can only share my experience with NuttX since I began working on the >>>> project in 2012 for multiple companies. >>>> >>>> Historically (based on my time on the project) releases - were build >>>> tested >>>> - by this I mean that the configurations were updated and the thus >>>> created a >>>> set of "Build Test vectors" BTV. Given the number of permutations >>>> solely >>>> based on the load time of >>>> (http://nuttx.org/doku.php?id=documentation:configvars) with 95,338 >>>> CONFIG_* >>>> hits. Yes there are duplicates on the page and dependencies. This is >>>> just >>>> meant to give a number of bits.... >>>> >>>> The total space is very large >>>> >>>> The BTV space was very sparse coverage. >>>> >>>> IIRC Greg gave the build testing task a day of time. It was repeated >>>> after >>>> errors were found. I am not aware of any other testing. Are you? >>>> >>>> There were no Release Candidate (rc) nor alpha nor beta test that ran >>>> this >>>> code one real systems and very little, if any Run Test Vectors (RTV) - >>>> I >>>> have never seen a test report - has anyone? >>>> >>>> One way to look at this is Sporadic Integration. (SI) with limited BTV >>>> and >>>> minimal RTV. Total Test Vector Coverage TTVC = BTV + RTV; The ROI of >>>> way >>>> of working, from a reliability perspective was and is very small. >>>> >>>> A herculean effort Greg's part with little return: We released code >>>> with >>>> many significant and critical errors in it. See the ReleaseNotes and >>>> the >>>> commit log. >>>> >>>> Over the years Greg referred to TRUNK (yes it was on SVN) and master as >>>> his >>>> "own sandbox" stating is should not be considered stable or build-able. >>>> This >>>> is evident in the commit log. >>> >>> Please stop focusing on the people (Greg) and let talk about how the >>> workflow. >>> We are here to discuss how we can improve the process, we are not >>> talking about throw away NuttX Build System and move to PX4. >>> >>> You are picturing something that is not true. >>> >>> We have issues, as FreeRTOS, MBEB and Zephyr also have. But it is not >>> Greg or the Build System guilt. >>> >>> Please, stop! It is disgusting! >>> >>>> I have personally never used a release from a tarball. Given the above >>>> why >>>> would I? It is less stable then master at TC = N >>>> (https://www.electronics-tutorials.ws/rc/rc_1.html) where N Is some >>>> number >>>> of days after a release. - unfortunately based on the current practices >>>> (a >>>> very unprofessional workflow) N is also dictated by when apps and >>>> nuttx >>>> actually building for a given target's set of BTV. >>> >>> It is not "unprofessional" it was what we could do based or our >>> hardware limitations. >>> >>>> With the tools and resources that exist in our work today, Quite >>>> frankly: >>>> This unacceptable and is an embarrassment. >>> >>> Oh my Gosh! Please don't do it. >>> >>> >>>> I suspect this is why there is a Tizen. The modern era - gets it. >>>> (Disclaimer I am an old dog - I am learning to get it) >>> >>> Tizen exists because companies want to have control. >>> This is the same logic why Redhat and others maintain their own Linux >>> kernel by themselves. >>> >>>> --- Disclaimer --- >>>> >>>> In the following, I'm am not bragging about PX4 or selling tools, I am >>>> merely trying to share our experiences for the betterment of NuttX. >>>> >>>> From what I understand PX4 has the most instances of NuttX running on >>>> real >>>> HW in the world. Over 300K. (I welcome other users to share their >>>> numbers) >>>> >>>> PX4's Total TTVC is still limited, but much, much greater than NuttX. >>>> >>>> We use Continuous integration (CI) on Nuttx on PX4 on every commit on >>>> PRs. >>>> >>>> C/C++ CI / build (push) Successful in 3m >>>> Compile MacOS Pending — This commit is being built >>>> Compile All Boards — This commit looks good >>>> Hardware Test — This commit looks good >>>> SITL Tests — This commit looks good >>>> SITL Tests (code coverage) — This commit looks good >>>> ci/circleci — Your tests passed on CircleCI! >>>> continuous-integration/appveyor/pr — AppVeyor build succeeded >>>> continuous-integration/jenkins/pr-head — This commit looks good >>>> >>>> >>>> We run tests on HW. >>>> >>>> http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-hardware/detail/pr-mag-str-preflt/1/pipeline >>>> >>>> I say limited because of the set of arch we use and the way we >>>> configure >>>> the >>>> OS. >>>> >>>> I believe this to be true of all users. >>>> >>>> The benefit of a community is that the sum of all TTVC that finds the >>>> problems and fix them. >>>> >>>> Why not maximize TTVC - if it will have a huge ROI and it is free: >>>> >>>> PX4 will contribute all that we have. We just need to build temporally >>>> consistent build. Yeah he is on the submodule thing AGAIN :) >>> >>> Just to make the history short: we already have solutions for SW and HW >>> CI. >>> >>> Besides the buildbot (https://buildbot.net) that was implemented and >>> tested by Fabio Balzano, Xiaomi also has a build test for NuttX. >>> >>> At end of the day, it is not only Greg testing the system, we all are >>> testing it as well. >>> >>> Don't try to push PX4 down your throat, it will not work this way. >>> Let's keep the Apache way, it is a democracy! >>> >>> BR, >>> >>> Alan >>
