On 6/2/2023 7:07 pm, Christian MAUDERER wrote: > Hello Chris, > > thanks for your feedback on the patch.
No problem :) > On 2023-02-06 05:16, Chris Johns wrote: >> On 3/2/2023 6:31 pm, Christian Mauderer wrote: >>> This patch tries to make the most important goals of LibBSD development >>> more clear based on the results of the discussion on the mailing list: >>> >>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html >>> --- >>> CONTRIBUTING.rst | 39 ++++++++++++++++++++++++++++++--------- >>> 1 file changed, 30 insertions(+), 9 deletions(-) >>> >>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst >>> index 0b6fc7a0..31561f6a 100644 >>> --- a/CONTRIBUTING.rst >>> +++ b/CONTRIBUTING.rst >>> @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what >>> modifications to the >>> FreeBSD source are permitted, and some other topics. For building the >>> library, >>> see the `README <README.rst>`_. >>> -Goals of the LibBSD activity are >>> - >>> -* provide functionality from FreeBSD to RTEMS, >>> -* ease updating to future FreeBSD versions, >>> -* ease tracking changes in FreeBSD code, >>> -* minimize manual changes in FreeBSD code. >>> - >>> -We will work to push our changes upstream to the FreeBSD Project and >>> minimize >>> -changes required at each update point. >>> +Every change to LibBSD has to consider the following points in groups of >>> +descending priority: >>> + >>> +#. Real-time Impacts + Maintainability Loss >>> + * LibBSD itself doesn't make any real time claims because it is >>> basically >>> + FreeBSD and they don't make any real time claims either. But all >>> + development in LibBSD must make sure that the real time ability of the >>> + RTEMS core system or the application is not affected. >>> + * It's utterly important that LibBSD is simple to maintain. One of the >>> most >>> + important points for that is that upgrades to newer FreeBSD versions >>> have >>> + to be easy. >> >> Correct and it is important we adopt and use what FreeBSD provides rather >> than >> adding extra complexity. I wrote about this in the FreeBSD journal years ago. >> The bespoke handing of fd and file objects is something I consider an >> unneeded >> complexity for specific system related reasons. We need NFSv4 and that uses >> VFS >> and that in turn uses the FreeBSD fd and file objects. >> > > I'm trying to find out what rules are agreed by most of us. I feel we need simplicity. I was surprised by the number of undefined rules my changes tripped over and to be honest I still have no idea what they are, why they exist and who made them? > It's of course OK to > bring examples. But I think we should try to evaluate the currently discussed > patch set based on the results together with as many of the other maintainers > as > possible as soon as we agree on the rules that we want to use to evaluate > them. The rules for the changes I made are simple. It is transparency of the FreeBSD code. > The file descriptors are an example where I know that Sebastian and you will > argument that their solution is the right one. He may but his view is only one view and fixed and I am sorry but the forceful and sometimes short nature of his responses has not helped his cause. I found the port of NFSv4 easy until I hit the fd/file parts. The complexity then grew and as I attempted to sort one piece out I hit another location and in the end I had little confidence about the changes I had made. Each change started to appear like a slight variation of the same thing for each descriptor type. In the end I decided the size of changes had grown to big so I decided to switch direction and bring across the FreeBSD code as close as possible and to move libbsd in the that direction because the closer we had VFS to FreeBSD the simpler the testing of complex pieces like the vnode cache and lookup would be. It is important to note we need code the project can maintain and as a result of my deep dive I now question who can maintain the fd/file support as it was? > I think that can only be decided with the help of some more people. I agree. This is about the long term and beyond any one of us. >> Source transparency is what matters and as a test of what is acceptable it >> must >> be preferred between competing implementations. >> >>> +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase >>> + * LibBSD code should be easy to read and easy to debug. Changes should >>> be >>> + integrated in a way that are easy to understand. Of course that's a >>> + subjective goal. As a general rule: If it adds lot's of extra code or >>> even >>> + more layers than already exist in FreeBSD, it's harder to understand. >>> + * There are a number of methods used in LibBSD to make it modular. If >>> you >>> + add new functionality, use one of the existing methods to allow >>> enabling or >>> + disabling your module. For example make sure that the linker can >>> remove >>> + unused modules. If that doesn't work for your feature, try to use the >>> + buildsets to allow to switch a module on or off. >>> + * A lot of different hardware uses LibBSD as network or USB stack or >>> maybe in >>> + the future even only for other subsystems. Some of the targets have >>> + hundreds of megabytes memory. Others can only have a few megabytes >>> (like >>> + the ATSAMV71). Make sure that changes don't increase the RAM / Flash >>> size >>> + of the default build so that it can't be used on the small targets. >> >> This is not realistic or achievable and I find confusing the reasons it is >> being >> pushed over and over. I would have not have agreed to this being added before >> now and nothing has changed. The central reason for rejecting this statement >> is >> a change in FreeBSD may add a few meg of memory to the footprint and this >> type >> of statement would conflict with that addition and that in turn would >> conflict >> with the need for transparency of source. And as stated before transparency >> must >> be preferred. > > Maintainability is the point that has the highest priority in my suggestion. I > grouped transparency, modularity and size like Gedare suggested because there > might be cases where we want to prefer one over the other. > > Transparency is also a bit difficult and often subjective. I defined it here > as > 'easy to read' and 'easy to debug'. These two alone can conflict each other. > For > example more layers can sometimes make something simpler to read because it is > nicely abstracted, but it can make it horrible hard to debug a problem because > you have to step through all the layers. I use the term transparency to reflect how much of the code we bring in from FreeBSD remains unchanged. It is a crude measure because a small change we make could be complicated but as a simple measure it tends to work. It does not reflect the complexity of the FreeBSD code and that is a key reason we want the code to be used as transparently as it can be. I documented all this in .. https://freebsdfoundation.org/wp-content/uploads/2016/08/FreeBSD-and-RTEMS-Unix-in-a-Real-Time-Operating-System.pdf What I wrote in 2016 is still valid. FYI Joel posted this link last year in relation to this topic. > Can you suggest a better choice or a better goal instead of the main point > 'transparency, modularity and size' that I wanted to add here? I am sensitive to statements like the one you have suggested. I made a large change to bring in important changes the project needs and I have been hit first with a patch with revert and then further rules I never knew existed and I would have never agreed to for this project. >> System requirements are for the developers of those systems and not RTEMS. >> Derating designs is an important part of system design and not the domain of >> this project. Memory constrained systems can consider another networking >> stack >> option or bespoke changes internally maintained. That is a cost trade off no >> one >> here can help make. > > As said in the other discussion: LibBSD is more than a network stack. It > started > as a pure USB stack, and it can still be that. My understanding of the history of LibBSD has always been an effort to move on from the legacy stack taking the lessons we learned. I still remember the GSoC meeting when it happened. USB was one of the first working milestone but it was networking and IPv6 that was the original discussion point. I pushed for libbsd as the name so it was not just networking. > We don't have another USB stack > to choose from. It now can also be used as a SDMMC stack. We don't have > another > one of those too. I think LibBSD is now more a driver framework than a pure > network stack. I shipped a FreeBSD USB stack on a resource limited NIOS-II product years before libbsd existed so I am across what is needed. As I said in the article it helped shape how we approach libbsd. > Even if we take a look at the pure network functionality: lwip was added as > another network stack for RTEMS not too long ago. I think it's great, and it > will be very useful for small targets that need limited network functionality. > But does it provide stuff like VLAN, packet filter, IPSec, WiFi? It does not and if you are going to use those features then I suggest memory and performance is considered up front when selected the platform to use. > I know of at least one user who uses VLAN, packet filter and IPSec on an ATSAM > BSP and another one who uses USB and WiFi on another variant of the ATSAM. > SDMMC > is used on the STM32H7. I think pure network is used on some of the older MPC* > or ColdFire BSPs where it replaced the legacy stack before lwip was added to > RTEMS. These are constraints I never knew existed so are you saying only EB makes major changes to libbsd and you are the gate keeper because you are the only people who understand these bounds? Knowing you all I doubt this is the case but this is the optics of what you are saying. > The users used it because it currently works well. Let's assume we just tell > the > existing users, that it won't work in the next RTEMS version any more (which > is > an unusual step for RTEMS): Where would you see the minimum requirements for > LibBSD? We should agree on some minimum requirements and add it to the > documentation as 'If your system has less than that: Don't use LibBSD! It > might > not work with that in the future.' I am sorry you have lost me. >>> +#. Performance Loss >>> + * There are applications, that require (for example) a high network >>> + throughput or a fast storage access. Make sure to take that into >>> account >>> + if adding changes. >> >> I prefer we do not add statements that have no definition or boundaries >> someone >> could use to test against. Selection of RTEMS in a systems is the choice of >> the >> system designer. There are systems RTEMS is not a good fit for. > > Do we give any good guidance to a user where RTEMS is a good fit? If not: What > should be added to the documentation? At the moment a new user would take a > look > at the supported BSPs and would see that we support low memory targets like > STM32F4 (not with LibBSD but with RTEMS) as well as high performance targets > with 24 cores. So he will assume that RTEMS works for all of these. If we say > that RTEMS is not a good fit for some of the systems, we should think about > removing all BSPs that are lower than the minimum or higher than the maximum > requirements. Note that this is of course a provocative statement to find the > limits that we want to support. So please don't take it as a serious > suggestion > to remove STM32F4. The suitability is for the system designers to answer. It is an open source project and not a product with a product specification. > An example for the upper end of the requirements: I know of a system with a > 10GBit Ethernet with a high load and low delays in the processing that is > currently supported by RTEMS. It's a real time application so RTEMS should be > a > good fit for it. Network performance is an important point for it. Real-time and networking? This sounds like something specialised I often refer to as "engineered networking". Engineered networks are specialised and engineered at all layers unlike an office or home network. Should we all be subject to these limitation now and into the future in case there is still a need? > If you want, > I can find some of the requirements of that system (network packet processed > in > this many clocks, minimum throughput on platform X, ...) and add it here so > that > we have a clear boundary. > > Let's assume a change reduces the network throughput of libbsd to 50%. But it > adds a new feature for some use case: Should we really not take a look at > performance and just add it? Where would you put the limit for performance > impacts for LibBSD? FreeBSD. Chris _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel