Re: [OT] Learning Makefiles
On Fri, May 19, 2023, 3:23 PM Tomek CEDRO wrote: > I am thinking about this. "If it works don't fix it" comes to my mind. > > Current build system is amazingly simple coherent and fast. Building > firmware takes 17 seconds. Why change it? > > Such change will flip everything upside down. Adds lots of work and > even more possible problems. > > What would be the real benefit? > > How would it improve that 17 seconds? > > I think the practical presentation and comparison of results (i.e. ide > integration, ci automation) along with numbers (i.e. build time) needs > to take place before making any decisions. > > -- > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info I very much disagree. The current build system is much slower with a ton of boilerplate that is difficult to maintain and make cross platform. Are we building an RTOS or reinventing the modern build tools? Infact I have pull requests unmerged because the hacks we have in place break and I have very little interest in tacking on even more hacks when we could just not worry about it with cmake. I would encourage those that thing this complicates things to actually look at the pull requests for cmake and see how much cleaner it is. --Brennan > >
Re: [OT] Learning Makefiles
I am thinking about this. "If it works don't fix it" comes to my mind. Current build system is amazingly simple coherent and fast. Building firmware takes 17 seconds. Why change it? Such change will flip everything upside down. Adds lots of work and even more possible problems. What would be the real benefit? How would it improve that 17 seconds? I think the practical presentation and comparison of results (i.e. ide integration, ci automation) along with numbers (i.e. build time) needs to take place before making any decisions. -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: [OT] Learning Makefiles
On Fri, May 19, 2023 at 12:53 PM Gregory Nutt wrote: > On 5/19/2023 10:25 AM, Lwazi Dube wrote: > > Alan, > > > > Can you summarize? I have not been following this PR. Is make going away? > > > > Thanks, > > > > -Lwazi > > > > On Fri, 19 May 2023 at 11:47, Alan C. Assis wrote: > >> Hi Everyone, > >> > >> While PR #6718 is waiting to get merged, please take a look: > >> > >> https://makefiletutorial.com > >> > >> BR, > >> > >> Alan > > The decision to switch from GNU Make to CMake is highly controversial. > Many people are wildly in favor and a large number are strong opposed. > This is a change that must be brought before the PMC for a vote before > any action is taken. > > Special rules apply to code change votes. We don't often do votes on > code changes. But such votes are critical in order to serve the > community with fairness. I'm neutral regarding CMake but *if* the project wants to think about a switch, I do think that we should consider very carefully the pros and cons, and think about any disruption that might affect users. If we want to consider supporting both GNU Make and CMake, then we should consider the benefits against the added maintenance burden and possible different behaviors that could occur because of inconsistencies between the two systems. I am usually opposed to spreading our limited resources thin over redundant systems. Nevertheless I am neutral and happy for the project to choose whatever is best, as long as we can reach a useful consensus as a community. Note that work began on CMake support in PRs such as 3704 and 6718. If the community is interested in CMake support, we shouldn't throw away that work. Cheers, Nathan
Re: [OT] Learning Makefiles
On Fri, May 19, 2023 at 8:10 PM Gregory Nutt wrote: > > Such a big change needs good description.. risks.. clear list of > > advantages and disadvantages :-) > > And if it comes down to switching from one to the other as you suggest, > then it needs a vote to understand the will of the whole community, not > the preference of a few. The whole community would have to support the > switch in every product build environment they maintain. > > So I would add the we would need a good understanding of the impact to > product builds as well. I do not suggest switching.. when it comes to IT I am really conservative.. and I like NuttX for NOT following fashions blindly :-) I remember there was a discussion some years back about cmake and it was no go.. I just want everyone to understand the status, process and, implications. There was in idea to use Python in the build and it was no go as well back then, but today I can see Python is involved in the build process. So it looks more like evolution rather than revolution..? I like CMake, lots of projects use it already, even on BSD everything "just works".. although it has different syntax and is more restrictive, it seems also well established solution, allows using different build mechanisms (even Makefiles), and that enables good integration with other automation / development tools.. I must admit that. Another advantage is build being done in a dedicated directory without touching the source tree. That may enable building variants, even different applications for different boards on the same source tree. This may enable more efficient automation. I am wondering about the initial configuration process, this part will have to change too, but it will not change dramatically (i.e. configure myboard:myconfig -> cmake -B myboard_myconfig -S . BOARD=myboard CONFIG=myconfig, and then the rest will remain the same)? If the change gets vote in favor I think that eventually Make will be replaced with CMake. I do not really like division of the community into "Make" and "CMake" groups.. this will lead to incoherent solution, bugs, mess, and interpersonal fights. We should really avoid that. Tremendous amount of work seems to be done. I am not really pro or against. If things work well I do not like changing them. But if the new feature proves itself with clear advantages then why not. -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: [OT] Learning Makefiles
On 5/19/2023 12:11 PM, Lwazi Dube wrote: On Fri, 19 May 2023 at 13:51, Alan C. Assis wrote: Lwazi, I think Greg summarized it well. Yes, and Maciej too. Thanks But we need to get away from statements of fears and marketing statements to understand the clear, real world impacts.
Re: [OT] Learning Makefiles
On Fri, 19 May 2023 at 13:51, Alan C. Assis wrote: > > Lwazi, I think Greg summarized it well. > Yes, and Maciej too. Thanks
Re: [OT] Learning Makefiles
Such a big change needs good description.. risks.. clear list of advantages and disadvantages :-) And if it comes down to switching from one to the other as you suggest, then it needs a vote to understand the will of the whole community, not the preference of a few. The whole community would have to support the switch in every product build environment they maintain. So I would add the we would need a good understanding of the impact to product builds as well.
Re: [OT] Learning Makefiles
On 5/19/23, Gregory Nutt wrote: > On 5/19/2023 10:25 AM, Lwazi Dube wrote: >> Alan, >> >> Can you summarize? I have not been following this PR. Is make going away? >> >> Thanks, >> >> -Lwazi >> >> On Fri, 19 May 2023 at 11:47, Alan C. Assis wrote: >>> Hi Everyone, >>> >>> While PR #6718 is waiting to get merged, please take a look: >>> >>> https://makefiletutorial.com >>> >>> BR, >>> >>> Alan > > The decision to switch from GNU Make to CMake is highly controversial. > Many people are wildly in favor and a large number are strong opposed. > This is a change that must be brought before the PMC for a vote before > any action is taken. > > Special rules apply to code change votes. We don't often do votes on > code changes. But such votes are critical in order to serve the > community with fairness. > > Lwazi, I think Greg summarized it well. This CMake is a long date discussion and it already created wounds in our community (we "lost" a great contributor/developer that proposed and submitted the initial patches to support CMake). I don't strong opinion about Makefile or CMake, the current Makefile works fine, the make advantage of CMake is it will integrated better with "modern" software that are build using CMake. Maybe others with better experience with it could give more details about its advantages and/or disadvantages. BR, Alan
Re: [OT] Learning Makefiles
I see the following advantages of having CMake: - Everything is way more readable then the current Make files. - Easier on-boarding. - Less build-related bugs. - Less boiler-plate code. - Faster builds. - Great IDE integration. Adding CMake as an optional feature seems to be a great solution. Maintaining two systems for a while requires more work, but it would be a backward-compatible way to polish the solution. For trivial problems it is still just adding files to lists, right? For more complex issues, I am pretty sure that parties attached to Make will maintain it just fine and people who want CMake will fix it for themselves. The discussion so far was that: - Replacing Make breaks existing workflows. - CMake is less flexible. - Duplicated maintenance effort. By just adding CMake no workflows are broken If the new solution is able to recreate the previous build system, then it is flexible enough. I guess that overlapping maintenance effort is the only way to offer backward compatibility. I would like to point out that offering CMake is what people expect to have from a modern project and NuttX is currently not offering it. Am Fr., 19. Mai 2023 um 19:26 Uhr schrieb Tomek CEDRO : > On Fri, May 19, 2023 at 7:17 PM Xiang Xiao wrote: > > The change doesn't replace Makefile with CMake, both can work. So I would > > suggest the vote is "Enable CMake support". > > Isn't the goal of CMake to generate Makefiles? > > What is the chance of keeping both makefiles and cmakefiles out of sync? > > Wouldn't that double the work? > > Wouldn't that imply switch to CMake anyways? > > From what I understand CMake can offer faster builds using different > build methods / systems, it could eliminate GNU Make vs BSD Make > problem, but also it will change whole build process, changes > frequently, etc..? > > Such a big change needs good description.. risks.. clear list of > advantages and disadvantages :-) > > -- > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >
Re: [OT] Learning Makefiles
On Fri, May 19, 2023 at 7:17 PM Xiang Xiao wrote: > The change doesn't replace Makefile with CMake, both can work. So I would > suggest the vote is "Enable CMake support". Isn't the goal of CMake to generate Makefiles? What is the chance of keeping both makefiles and cmakefiles out of sync? Wouldn't that double the work? Wouldn't that imply switch to CMake anyways? >From what I understand CMake can offer faster builds using different build methods / systems, it could eliminate GNU Make vs BSD Make problem, but also it will change whole build process, changes frequently, etc..? Such a big change needs good description.. risks.. clear list of advantages and disadvantages :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: [OT] Learning Makefiles
The change doesn't replace Makefile with CMake, both can work. So I would suggest the vote is "Enable CMake support". On Sat, May 20, 2023 at 12:53 AM Gregory Nutt wrote: > On 5/19/2023 10:25 AM, Lwazi Dube wrote: > > Alan, > > > > Can you summarize? I have not been following this PR. Is make going away? > > > > Thanks, > > > > -Lwazi > > > > On Fri, 19 May 2023 at 11:47, Alan C. Assis wrote: > >> Hi Everyone, > >> > >> While PR #6718 is waiting to get merged, please take a look: > >> > >> https://makefiletutorial.com > >> > >> BR, > >> > >> Alan > > The decision to switch from GNU Make to CMake is highly controversial. > Many people are wildly in favor and a large number are strong opposed. > This is a change that must be brought before the PMC for a vote before > any action is taken. > > Special rules apply to code change votes. We don't often do votes on > code changes. But such votes are critical in order to serve the > community with fairness. > > > >
Re: [OT] Learning Makefiles
On 5/19/2023 10:25 AM, Lwazi Dube wrote: Alan, Can you summarize? I have not been following this PR. Is make going away? Thanks, -Lwazi On Fri, 19 May 2023 at 11:47, Alan C. Assis wrote: Hi Everyone, While PR #6718 is waiting to get merged, please take a look: https://makefiletutorial.com BR, Alan The decision to switch from GNU Make to CMake is highly controversial. Many people are wildly in favor and a large number are strong opposed. This is a change that must be brought before the PMC for a vote before any action is taken. Special rules apply to code change votes. We don't often do votes on code changes. But such votes are critical in order to serve the community with fairness.
Re: [OT] Learning Makefiles
Alan, Can you summarize? I have not been following this PR. Is make going away? Thanks, -Lwazi On Fri, 19 May 2023 at 11:47, Alan C. Assis wrote: > > Hi Everyone, > > While PR #6718 is waiting to get merged, please take a look: > > https://makefiletutorial.com > > BR, > > Alan
[OT] Learning Makefiles
Hi Everyone, While PR #6718 is waiting to get merged, please take a look: https://makefiletutorial.com BR, Alan
Re: Recommended configuration practices question
On 5/19/23, Nathan Hartman wrote: > On Fri, May 19, 2023 at 8:24 AM Milan Obuch wrote: > >> Hi, >> >> I started testing NuttX on Raspberry Pi Pico board, using FreeBSD as >> host for building. I was able to build NuttX using all configuration >> files for this board (but one - nshsram, it does not finish, but I >> don't care about it for now). All those configurations use default UART >> placement (pins 0, 1). >> >> These pins are not available on 01space board RP2040-0.42LCD, so I >> changed UART placement to pins 28, 29 running 'make menuconfig'. This >> way, .config in NuttX working/build directory is modified, and >> resulting binary (or uf2 for this board) file works as expected. >> >> This file, however, gets deleted when running 'make distclean' for >> different build. Thus I run 'make savedefconfig', altering defconfig >> file in working directory. This gets deleted on 'make distclean' as >> well, so I manually created new directory, where I copied defconfig >> file - boards/arm/rp2040/raspberrypi-pico/configs/nsh_28_29, and now my >> config is listed in 'tools/configure.sh -L' output, so I can use it as >> any other config file. >> >> So my questions now - is this intended way for saving and reusing >> config file? While it works for me, I'd like to have my configs >> separated from standard source files, maybe as third parallel directory >> to standard nuttx and apps, with ability to see them in >> 'tools/configure.sh -L' output, and using my configuration with >> 'tools/configure.sh '. >> >> Any hints for this one? >> >> Regards, >> Milan > > > > There are various possibilities: > > After running 'make savedefconfig' you would normally want to copy the > resulting defconfig file somewhere. > I think we need a better Documentation about it. Reading Milan's question makes it clean that the information is missing. In fact searching for savedefconfig in our documentation returns nothing: https://nuttx.apache.org/docs/latest/search.html?q=savedefconfig_keywords=yes=default# Do you have suggestion there in the documentation we could put it? I think we could have a "Creating my custom configuration" in the Getting Started, or maybe we could create a main entry "Developing with NuttX" and include it. Also it is important to explain that this defconfig is used to create a board profile, then you can use with configure.sh boardname:boardprofile > You could copy it to a new config under the board in question, and if you > feel this configuration could be helpful to the wider community, you could > opt to open a PR to upstream it. > Been more clear and more didactic: After running "make savedefconfig" you can create a directory inside your board config that will retain your config customization: $ make savedefconfig $ mkdir boardsconfig/mynewcustomconfig $ mv defconfig boardsconfig/mynewcustomconfig/ > Or you could create your own variant of the board by copying the whole > board directory and then customizing it. The copy could be in-tree (in > boards/arm64/) or out-of-tree (leaving the nuttx sources alone and > putting your board in another directory, which you can specify). This > option (copying the whole board directory) makes sense if you want to > customize more than just the config and don't plan to upstream your > customizations. > This is a good option! Mainly when you have a board that is not in the mainline, your a personal board that will not be accepted on mainline. Normally all boards in the mainline need to be available to the public, if it is a personal board it should be kept in your local repository. > As a side note, if you are customizing any in-tree nuttx sources, it's > probably a good idea to create your own long-lived branch, keep your master > branch up-to-date with upstream, add your customizations to your private > branch, and occasionally merge updates from master into your branch. Or you > could base your branch off of a release tag if you prefer to use released > versions. > Thank is also a great suggestion! We need to have it documented to avoid spending time saying it again and again! :-) BR, Alan
Re: Recommended configuration practices question
On Fri, May 19, 2023 at 8:24 AM Milan Obuch wrote: > Hi, > > I started testing NuttX on Raspberry Pi Pico board, using FreeBSD as > host for building. I was able to build NuttX using all configuration > files for this board (but one - nshsram, it does not finish, but I > don't care about it for now). All those configurations use default UART > placement (pins 0, 1). > > These pins are not available on 01space board RP2040-0.42LCD, so I > changed UART placement to pins 28, 29 running 'make menuconfig'. This > way, .config in NuttX working/build directory is modified, and > resulting binary (or uf2 for this board) file works as expected. > > This file, however, gets deleted when running 'make distclean' for > different build. Thus I run 'make savedefconfig', altering defconfig > file in working directory. This gets deleted on 'make distclean' as > well, so I manually created new directory, where I copied defconfig > file - boards/arm/rp2040/raspberrypi-pico/configs/nsh_28_29, and now my > config is listed in 'tools/configure.sh -L' output, so I can use it as > any other config file. > > So my questions now - is this intended way for saving and reusing > config file? While it works for me, I'd like to have my configs > separated from standard source files, maybe as third parallel directory > to standard nuttx and apps, with ability to see them in > 'tools/configure.sh -L' output, and using my configuration with > 'tools/configure.sh '. > > Any hints for this one? > > Regards, > Milan There are various possibilities: After running 'make savedefconfig' you would normally want to copy the resulting defconfig file somewhere. You could copy it to a new config under the board in question, and if you feel this configuration could be helpful to the wider community, you could opt to open a PR to upstream it. Or you could create your own variant of the board by copying the whole board directory and then customizing it. The copy could be in-tree (in boards/arm64/) or out-of-tree (leaving the nuttx sources alone and putting your board in another directory, which you can specify). This option (copying the whole board directory) makes sense if you want to customize more than just the config and don't plan to upstream your customizations. As a side note, if you are customizing any in-tree nuttx sources, it's probably a good idea to create your own long-lived branch, keep your master branch up-to-date with upstream, add your customizations to your private branch, and occasionally merge updates from master into your branch. Or you could base your branch off of a release tag if you prefer to use released versions. Hope this helps, Nathan
Re: Recommended configuration practices question
On Fri, May 19, 2023 at 2:24 PM Milan Obuch wrote: > > Hi, > > I started testing NuttX on Raspberry Pi Pico board, using FreeBSD as > host for building. I was able to build NuttX using all configuration > files for this board (but one - nshsram, it does not finish, but I > don't care about it for now). All those configurations use default UART > placement (pins 0, 1). > > These pins are not available on 01space board RP2040-0.42LCD, so I > changed UART placement to pins 28, 29 running 'make menuconfig'. This > way, .config in NuttX working/build directory is modified, and > resulting binary (or uf2 for this board) file works as expected. > > This file, however, gets deleted when running 'make distclean' for > different build. Thus I run 'make savedefconfig', altering defconfig > file in working directory. This gets deleted on 'make distclean' as > well, so I manually created new directory, where I copied defconfig > file - boards/arm/rp2040/raspberrypi-pico/configs/nsh_28_29, and now my > config is listed in 'tools/configure.sh -L' output, so I can use it as > any other config file. > > So my questions now - is this intended way for saving and reusing > config file? While it works for me, I'd like to have my configs > separated from standard source files, maybe as third parallel directory > to standard nuttx and apps, with ability to see them in > 'tools/configure.sh -L' output, and using my configuration with > 'tools/configure.sh '. > > Any hints for this one? Hey there Milan my fellow BSD daemon :-) :-) I am also working that way.. because nuttx and nuttx-apps are git submodules of my project repo and I want to work on a release and this is the cleanest way to maintain. Also I have a custom board. I keep my custom application and board definition outside those two release directories according to these hints: https://nuttx.apache.org/docs/latest/guides/customboards.html https://nuttx.apache.org/docs/latest/guides/customapps.html You can create your own board definition based on existing board definition, just copy whole directory, update names, and update required pin definitions. If the board differs a lot and it is not yet in the upstream you can send a PR so everyone also can use it :-) Note that board is just kind of description and setup of the "hardware product", while all common code resides in the arch/ (MCU with internal peripherals) and drivers/ (external peripherals onboard). https://github.com/apache/nuttx/tree/master/arch https://github.com/apache/nuttx/tree/master/drivers Here is the full description of the directory structure: https://nuttx.apache.org/docs/latest/quickstart/organization.html I noticed that some boards definitions are more kind of template rather than ready for use fully featured setups (i.e. devkits with not much peripherals onboard). User needs to adapt these boards configuration for particular needs. I guess this was done so far directly in the source code. But with custom board approach you can just copy selected board config and update it as you wish. Hope that helps :-) Tomek -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: nxsem_tickwait_uninterruptible randomly timeouts one tick too soon?
Yes, it worked before, but a long time ago. I tested this on both on arm (stm32f7) and risc-v (mpfs) platforms. I tracked the problem down to this patch: commit 19758788356f8623bac5f439419e231ff81cac14 Author: Huang Qi Date: Mon Apr 11 18:42:24 2022 +0800 arch/risc-v: Apply common mtime driver to mtime based chps Signed-off-by: Huang Qi The problem seems to be specific to RISC-V platforms If I revert the changes in my platform (mpfs) in file arch/risc-v/src/mpfs/mpfs_timerisr.c, and handle the timer interrupt there, everything seems to be working again. The "common mtimer driver" seems a bit complex (using the alarm interface), and I don't have time to debug that right now, need to come back to the issue later. Maybe there is some race condition somewhere. Warning to others: this might be broken for other risc-v platforms as well. - Jukka On 17.5.2023 18.51, Nathan Hartman wrote: Was it working before? If so, are you able to use a git bisect to find the commit where the bug was introduced? This might minimize the amount of testing and debugging that needs to be done. On Wed, May 17, 2023 at 11:12 AM Jukka Laitinen wrote: Petro Karashchenko kirjoitti keskiviikko 17. toukokuuta 2023: How do you measure the wait period? Are you togging a pin or used MCU free running HW timer? I used RISC-V MTIMER, so it is a free running HW counter at 1us resolution Best regards, Petro On Wed, May 17, 2023, 5:43 PM Jukka Laitinen wrote: On 17.5.2023 16.38, Gregory Nutt wrote: On 5/17/2023 7:21 AM, Gregory Nutt wrote: On 5/17/2023 4:21 AM, Jukka Laitinen wrote: Hi, I just observed the behaviour mentioned in the subject; I tried just calling in a loop: " sem_t sem =SEM_INITIALIZER(0); int ret; ret = nxsem_tickwait_uninterruptible(, 1); " , and never posting the sem from anywhere. The function return -ETIMEDOUT properly on every call. But when measuring the time spent in the wait, I see randomly that sometimes the sleep time was less than one systick. If I set systick to 10ms, I see typical (correct) sleep time between 1 - 2us. But sometimes (very randomly) between 0 - 1us. Also in these error cases the return value is correct (-110, -ETIMEDOUT). When sleeping for 2 ticks, I see randomly sleep times between 1-2us, for 3 ticks 2-3us. So, randomly it is exactly one systick too small. I looked through the implementation of the "nxsem_tickwait_uninterruptible" itself, and didn't saw problem there. (Actually, I think there is a bug if -EINTR occurs; in that case it should always sleep at least one tick more - now it doesn't. But it is not related to this, in my test there was no -EINTR). I believe the problem might be somewhere in sched/wdog/ , but so far couldn't track down what causes it. Has anyone else seen the same issue? Br, Jukka If I understand what you are seeing properly, then it is normal and correct behavior for a arbitrary (asynchonous) timer. See https://cwiki.apache.org/confluence/display/NUTTX/Short+Time+Delays for an explanation. NuttX timers have always worked that way and has confused people that use the timers near the limits of their resolution. A solution is to use a very high resolution timer in tickless mode. Oops. You are seeing a timer that is 1 tick too short. That is an error and should never happen. Sorry for reading incorrectly. It was still early in the morning here. The timer logic adds +1 tick to the requested to assure that that error never occurs. If +1 were not added, the bad result would be exactly as you describe (and as explained in the confluence reference). Hi, yes, exactly. Seeing timeout 1 tick too short. Sorry for not explaining it clearly enough :) I fear that there is now some bug. It was rather easy to re-produce, just a loop with few thousand iterations, and it occurs (infinite loop, 10 ms tick, less than a minute to catch). Most of the time it works ok; the sleep time is longer than the requested ticks. But when it triggers, the sleep is exactly one tick too short (and shorter than the requested timeout in ticks). I was just asking, if others have seen this as well; I'd like to know if it is really a bug in current nuttx main. It is always possible that there is something funny in our local build - although I can't see what it could be. -Jukka
Recommended configuration practices question
Hi, I started testing NuttX on Raspberry Pi Pico board, using FreeBSD as host for building. I was able to build NuttX using all configuration files for this board (but one - nshsram, it does not finish, but I don't care about it for now). All those configurations use default UART placement (pins 0, 1). These pins are not available on 01space board RP2040-0.42LCD, so I changed UART placement to pins 28, 29 running 'make menuconfig'. This way, .config in NuttX working/build directory is modified, and resulting binary (or uf2 for this board) file works as expected. This file, however, gets deleted when running 'make distclean' for different build. Thus I run 'make savedefconfig', altering defconfig file in working directory. This gets deleted on 'make distclean' as well, so I manually created new directory, where I copied defconfig file - boards/arm/rp2040/raspberrypi-pico/configs/nsh_28_29, and now my config is listed in 'tools/configure.sh -L' output, so I can use it as any other config file. So my questions now - is this intended way for saving and reusing config file? While it works for me, I'd like to have my configs separated from standard source files, maybe as third parallel directory to standard nuttx and apps, with ability to see them in 'tools/configure.sh -L' output, and using my configuration with 'tools/configure.sh '. Any hints for this one? Regards, Milan
