Hi Eshan, Ok, great! Thank you for letting me know. By the way, where will it be implemented when it's patched in? Thanks again and have a great rest of your Sunday.
Sincerely, Matt On Sun, Mar 28, 2021 at 6:54 AM Eshan Dhawan <eshandhawa...@gmail.com> wrote: > > > > On 27-Mar-2021, at 1:49 AM, Matthew Joyce <mfjoyce2...@gmail.com> wrote: > > > > Hi Dr. Joel, > > > > I finally built rtems-libbsd and see that pselect and sockatmark are > > both defined there. I went ahead and added a "In rtems-libbsd" column > > in the spreadsheet to reflect that. > > > > With those two defined, it looks like the only methods from the FACE > > 3.0 General Purpose Profile that aren't currently supported are > > confstr() and the <spawn.h> set. Could I ask, what is the thinking on > > those? The man page suggests that spawn was created with embedded > > systems in mind, but I'd guess a conscious decision was made to leave > > them out? How about confstr? > > > > Thank you! > > > > Matt > > > Hi Matt > Confstr code is ready just under styling issues. > So maybe you could count it as Present. > > Thanks > - Eshan > > > >> On Thu, Mar 25, 2021 at 2:18 PM Joel Sherrill <j...@rtems.org> wrote: > >> > >> > >> > >>> On Thu, Mar 25, 2021 at 7:14 AM Matthew Joyce <mfjoyce2...@gmail.com> > >>> wrote: > >>> > >>> Hi Dr. Joel, > >>> > >>> Thanks very much, that's a big help! Correct, I've been updating the > >>> spreadsheet as I go along. Ok, now I see that strlcat/strlcpy are used > >>> in rtems/cpukit and implemented in Newlib. > >>> > >>> One additional question, please: I haven't yet looked into the source > >>> of NetBSD or FreeBSD, but I do see that Newlib already implements > >>> ppoll (poll.cc), dladdr (dlfcn.cc), pselect (select.cc), and > >>> sockatmark (net.cc). None of them are defined in the rtems environment > >>> yet. Is there any reason why the NetBSD/FreeBSD version would be > >>> preferable to Newlib for these? Or is it just a matter of testing > >>> what's out there to find what works well in the rtems environment? > >> > >> > >> Without looking at the newlib git repo, I can tell you that the files > >> you cite are the implementation of those methods for Cygwin. Just > >> because they are in C++. :) > >> > >> The parts of the newlib repo RTEMS uses are under the newlib/ > >> subdirectory not the cygwin one. Within that, there is a libc/sys and > >> only libc/sys/rtems is used for RTEMS. The others are for different > >> operating systems. There are a few places with "machine" directory > >> structures. Only the ones for the architecture you are building for > >> is used. > >> > >> As to why NetBSD for libdl, that is because portions of the code > >> originated there. > >> > >> And rtems-libbsd is based on FreeBSD. It is as close to the FreeBSD > >> source as we can keep it. > >> > >>> > >>> > >>> In my proposal I'll take your advice and work on some of the easier > >>> ones first in order to get the experience and process down. > >> > >> > >> There are tickets for a lot of methods. The rtems-docs repo has the > >> csv file (e.g. spreadsheet) which tracks RTEMS support against > >> various standards. The RTEMS POSIX Compliance Guide is generated > >> from that csv file. Between those, you can find other methods to ask > >> about. In general, if it is required by the Software Communications > >> Architecture (SCA) or FACE Technical Standard, then it is a method > >> someone expected to possibly be used in an embedded system. > >> SCA is a set of POSIX profiles focused on software defined radios and > >> the FACE Technical Standard was developed with avionics in mind. > >> > >> But any are fair game if they are actually implementable. I don;t think > >> the Compliance Guide says it yet, but we decided last year that > >> wordexp() is likely not supportable on RTEMS. The newlib > >> implementation assumes the presence of a shell with wildcard expansion > >> and ability to fork a process. > >> > >> --joel > >> > >>> > >>> > >>> Thank you again for your time! > >>> > >>> Matt > >>> > >>> On Wed, Mar 24, 2021 at 5:03 PM Joel Sherrill <j...@rtems.org> wrote: > >>>> > >>>> Wow! Good work. There is a lot to digest here. Comments interspersed. > >>>> > >>>> I assume the spreadsheet is updated. > >>>> > >>>> On Wed, Mar 24, 2021 at 7:38 AM Matthew Joyce <mfjoyce2...@gmail.com> > >>>> wrote: > >>>>> > >>>>> Hi Dr. Joel, > >>>>> > >>>>> I've gone over the list a few times now and see a few categories > >>>>> shaping up: > >>>>> > >>>>> 1) Already done (In Newlib source, defined in libc.a): > >>>>> a) reallocarray > >>>>> b) qsort_r > >>>>> c) memmem > >>>>> d) strlcat / strlcpy > >>>>> d) wcslcat / wcslcpy > >>>>> *Out of this group, strlcat and strlcpy also show up in > >>>>> src/rtems/cpukit. Why is that? > >>>> > >>>> > >>>> The good news is that we support these. :) > >>>> > >>>> It looks to me that strlcat and strlcpy are used in cpukit but not > >>>> implemented > >>>> there. Where do you think they are implemented. > >>>> > >>>> This is a good example where a source code browser is helpful. grep can > >>>> often answer the question but a source code browser can be easier. > >>>> Personally, > >>>> I use cscope but that is exceedingly old school. Any modern IDE should be > >>>> helpful. > >>>> > >>>>> > >>>>> 2) Not done yet (Do not show up in Newlib source or RTEMS): > >>>>> a) getlocalename_l > >>>>> b) posix_getdents > >>>>> c) sem_clockwait > >>>>> d) sig2str / str2sig > >>>>> > >>>>> 3) Not in Newlib; Referenced in RTEMS but hidden behind #ifdef: > >>>>> a) pthread_cond_clockwait > >>>>> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/condition_variable) > >>>>> b) pthread_mutex_clocklock > >>>>> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/mutex) > >>>>> c) pthread_rwlock_clockrdlock > >>>>> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex) > >>>>> c) pthread_rwlock_clockwrlock > >>>>> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex) > >>>>> *It looks like some groundwork was done, but the methods are not yet > >>>>> supported. > >>>> > >>>> > >>>> The paths you point to are C++ files that would implement C++ features > >>>> using the available POSIX services. So they are users, not providers. > >>>> > >>>> All of the pthread services related to these are implemented in > >>>> cpukit/posix/src. I think you can configure a clock for all these now > >>>> to be used by detailed on wait and timedwait calls. My understanding > >>>> is that these would let you use a specific clock on a per blocking call > >>>> basis. > >>>> > >>>> First question is which clocks are intended to be supported. > >>>> > >>>> Second is the pattern of picking which timeout queue to go on when > >>>> now it is coded to let you pick one which is used for the life of the > >>>> object. > >>>> > >>>>> > >>>>> 4) Misc (In Newlib source, not defined in libc.a, appear in RTEMS in > >>>>> various ways) > >>>>> a) getentropy (an alternate version is defined in RTEMS librtemsbsd.a, > >>>>> in src/rtems/bsps/shared/dev/getentropy/getentropy-cpucounter.c. The > >>>>> comments note that it is not cryptographically secure, so it may not > >>>>> fit the bill for the getentropy() mentioned in the Open Group > >>>>> document) > >>>> > >>>> > >>>> I am far from a cryptography expert but this looks like a case where > >>>> this method would be considered supported with the disclaimer that > >>>> the quality of the entropy value depends on the BSP. If the user has > >>>> specific requirements, they will need to verify the implementation > >>>> used by the BSP by default is appropriate. > >>>> > >>>>> > >>>>> b) ppoll (appears in rtems/6/share/gdb/syscalls) > >>>> > >>>> > >>>> You need to be more careful with the grep. These again are in the > >>>> installed tools and in this case, they appear in an XML file. Referenced > >>>> but not implemented. > >>>> > >>>> ppoll() will need to come from rtems-libbsd. The required system call > >>>> is included but disabled currently. AFAIK this means it is possible to > >>>> provide this but that would require a more detailed discussion in case > >>>> some underlying capability is missing. Chris Johns and Sebastian > >>>> Huber would be the ones to guide here. > >>>> > >>>> Ruling: Likely possible. > >>>> > >>>>> > >>>>> c) dladdr (appears in rtems/cpukit but not defined) > >>>> > >>>> > >>>> I think this can be implemented in libdl but I am not sure if the > >>>> code from NetBSD from this would directly work or just be a guide. > >>>> > >>>>> > >>>>> > >>>>> 5) Others? > >>>>> It looks like there was work done on methods like sockatmark and > >>>>> pselect, but I don't see them supported as yet. Should those be added > >>>>> to the list or are they still being worked on? > >>>> > >>>> > >>>> These would come from rtems-libbsd. > >>>> > >>>> I think sockatmark.c is implemented in freebsd/lib/libc/net/sockatmark.c > >>>> but I don't know if the ioctl() is implemented. I expect it is but this > >>>> would > >>>> at least require a test. It may just work. > >>>> > >>>> pselect() looks to be missing and would have to be ported from FreeBSD. > >>>> > >>>>> > >>>>> As you suggested, I'll look into NetBSD for dladdr and do some digging > >>>>> on the implementation of the other outstanding methods. You mentioned > >>>>> that the "clock" ones have to be strictly added to rtems/cpukit, but > >>>>> the references I found above are all in lib/gcc/sparc-rtems6/10.2.1. > >>>>> Why is that the case and what is 10.2.1? Also, I'm not sure what to > >>>>> make of getentropy and ppoll based on what I found above...at your > >>>>> convenience could you please advise? > >>>> > >>>> > >>>> Hopefully the above helped. > >>>> > >>>> You don't have to restrict your possible set to these new additions. > >>>> There are others. I think Eshan has done the research for where to > >>>> get implementations of the missing long double methods for newlib. > >>>> And there are tickets for other missing methods or specific capabilities > >>>> in methods that are supported. Those are quite possible to have > >>>> some alternatives that are easier to approach. > >>>> > >>>> --joel > >>>> > >>>> > >>>>> > >>>>> > >>>>> Thank you very much! > >>>>> > >>>>> Matt > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Sun, Mar 21, 2021 at 6:38 PM Joel Sherrill <j...@rtems.org> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Sun, Mar 21, 2021 at 2:28 AM Matthew Joyce <mfjoyce2...@gmail.com> > >>>>>> wrote: > >>>>>>> > >>>>>>> Gentlemen, > >>>>>>> > >>>>>>> Awesome, thanks! I see how that works now...I'll give it a thorough > >>>>>>> look tomorrow and will update the spreadsheet accordingly. I'll pipe > >>>>>>> back up when I have a more accurate look of what's currently there. > >>>>>> > >>>>>> > >>>>>> Knowing what doesn't have to be done is the first step. (rtems, > >>>>>> newlib, and libbsd) > >>>>>> > >>>>>> I'd be prone to look for things that are easy to add first. > >>>>>> > >>>>>> Some may not be implementable on RTEMS due to only supporting a > >>>>>> single process and no virtual memory. If you have doubts on whether it > >>>>>> is possible to support a specific method, speak up and let's try to > >>>>>> decide. > >>>>>> > >>>>>> Then find upstream places for an implementation where possible. I > >>>>>> suspect > >>>>>> all the new "clock" methods will require discussion on an > >>>>>> implementation > >>>>>> pattern but those must strictly be added to rtems/cpukit with tests and > >>>>>> documentation. At least I can throw you that much. :) > >>>>>> > >>>>>>> > >>>>>>> Thanks again and have a great Sunday! > >>>>>>> > >>>>>>> Matt > >>>>>>> > >>>>>>> On Fri, Mar 19, 2021 at 8:27 PM Joel Sherrill <j...@rtems.org> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Fri, Mar 19, 2021 at 1:08 PM Gedare Bloom <ged...@rtems.org> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On Fri, Mar 19, 2021 at 11:16 AM Matthew Joyce > >>>>>>>>> <mfjoyce2...@gmail.com> wrote: > >>>>>>>>>> > >>>>>>>>>> Dr. Joel, > >>>>>>>>>> > >>>>>>>>>> Thanks very much...I'll keep working to get a sense of what goes > >>>>>>>>>> where! In the meantime, where can I look to get the ground truth of > >>>>>>>>>> which methods are "in RTEMS" as opposed to those in newlib? > >>>>>>>>>> > >>>>>>>>> There is only one ground truth: > >>>>>>>>> git://git.rtems.org/rtems.git > >>>>>>>>> > >>>>>>>>> And for newlib > >>>>>>>>> > >>>>>>>>> git://sourceware.org/git/newlib-cygwin.git > >>>>>>>>> > >>>>>>>>> That said, searching for the function name symbols in compiled > >>>>>>>>> libraries is a good first step to rule out newlib. Then, you can > >>>>>>>>> 'grep' the RTEMS source code for the function names to see if they > >>>>>>>>> exist there. > >>>>>>>> > >>>>>>>> > >>>>>>>> rtems/cpukit to be specitic. It won't be implemented anywhere else. > >>>>>>>> > >>>>>>>> And clearly we both have forgotten that networking APIs are in the > >>>>>>>> rtems-libbsd repository. > >>>>>>>> > >>>>>>>> https://git.rtems.org/rtems-libbsd/ > >>>>>>>> > >>>>>>>> I suspect ppoll() might already be in there. Or at least supported by > >>>>>>>> FreeBSD. > >>>>>>>> > >>>>>>>> You should clone everything and grep the sources. newlib already has > >>>>>>>> qsort_r. This is the nm I used: > >>>>>>>> > >>>>>>>> $ ~/rtems-work/tools/6/bin/sparc-rtems6-nm > >>>>>>>> ~/rtems-work/tools/6/sparc-rtems6/lib/libc.a | grep qsort_r > >>>>>>>> lib_a-bsd_qsort_r.o: > >>>>>>>> 00000000 T __bsd_qsort_r > >>>>>>>> lib_a-qsort_r.o: > >>>>>>>> 00000000 T qsort_r > >>>>>>>> > >>>>>>>> Notice the last line has "T qsort_r" which says it is defined. > >>>>>>>> > >>>>>>>> grep -r in the newlib source shows it is in ./libc/search/qsort_r.c > >>>>>>>> > >>>>>>>> dladdr() looks to be prototyped in RTEMS but hidden behind an ifdef > >>>>>>>> like it > >>>>>>>> wasn't ported from NetBSD so that looks possible. It is in rtems. > >>>>>>>> > >>>>>>>> Those two examples should help you figure out why you missed > >>>>>>>> finding some things that were implemented. > >>>>>>>> > >>>>>>>> I need to figure out what this next POSIX version is to be called > >>>>>>>> so I can update the tracking spreadsheet that generates the RTEMS > >>>>>>>> POSIX Compliance Guide, :) > >>>>>>>> > >>>>>>>> --joel > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Thanks again! > >>>>>>>>>> > >>>>>>>>>> Matt > >>>>>>>>>> > >>>>>>>>>> On Fri, Mar 19, 2021 at 1:58 PM Joel Sherrill <j...@rtems.org> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Keep devel@ on the list. :) > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Mar 19, 2021 at 7:51 AM Matthew Joyce > >>>>>>>>>>> <mfjoyce2...@gmail.com> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Sir, > >>>>>>>>>>>> > >>>>>>>>>>>> Thank you for the link! I see that you're right, those last four > >>>>>>>>>>>> are > >>>>>>>>>>>> in newlib, plus memmem(). I updated those in the Google Sheet. > >>>>>>>>>>>> > >>>>>>>>>>>> Now I see the newlib part, but where are you referring to > >>>>>>>>>>>> specifically > >>>>>>>>>>>> when you say RTEMS, as in "POSIX support comes from a mix of > >>>>>>>>>>>> RTEMS and > >>>>>>>>>>>> newlib"? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> POSIX is a HUGE HUGE standard and references other standards. One > >>>>>>>>>>> it references and pulls in is the C99 Standard C Library which is > >>>>>>>>>>> libc and > >>>>>>>>>>> libm. RTEMS mostly does not implement this functionality and > >>>>>>>>>>> relies on > >>>>>>>>>>> another open source project for those APIs. Newlib is an open > >>>>>>>>>>> source > >>>>>>>>>>> C Library used by RTEMS, Cygwin, and most embedded systems GNU > >>>>>>>>>>> tools > >>>>>>>>>>> chains. > >>>>>>>>>>> > >>>>>>>>>>> Most of the POSIX header files with RTEMS are actually in Newlib > >>>>>>>>>>> even > >>>>>>>>>>> if they originated with RTEMS. Many are shared with Cygwin. > >>>>>>>>>>> > >>>>>>>>>>> So methods like the string, memory, and *printf come from Newlib > >>>>>>>>>>> since they > >>>>>>>>>>> are in C99. We provide POSIX like threading, signals, core file > >>>>>>>>>>> access, and > >>>>>>>>>>> much more. > >>>>>>>>>>> > >>>>>>>>>>> It's a complementary relationship but it takes a bit to figure > >>>>>>>>>>> out when > >>>>>>>>>>> something should be in one or the other. The line gets blurred at > >>>>>>>>>>> times. > >>>>>>>>>>> > >>>>>>>>>>> Say you added a new CPU architecture implementation of a math > >>>>>>>>>>> method (like Eshan did last year), then it goes in newlib. But he > >>>>>>>>>>> also > >>>>>>>>>>> added some POSIX methods which go in RTEMS. In either case, > >>>>>>>>>>> we like tests for them in RTEMS to show they work in our > >>>>>>>>>>> environment. > >>>>>>>>>>> > >>>>>>>>>>> --joel > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks again! > >>>>>>>>>>>> > >>>>>>>>>>>> Matt > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Mar 19, 2021 at 1:13 PM Joel Sherrill <j...@rtems.org> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Mar 19, 2021, 6:40 AM Joel Sherrill <j...@rtems.org> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Fri, Mar 19, 2021, 5:48 AM Matthew Joyce > >>>>>>>>>>>>>> <mfjoyce2...@gmail.com> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> https://docs.google.com/spreadsheets/d/1reCNOIZC5JTwQENgl-hvG8THfQqNtlUDVy_07PYodic/edit?usp=sharing > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hello, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> As suggested by Dr. Sherril, I've taken an initial look > >>>>>>>>>>>>>>> through this > >>>>>>>>>>>>>>> document > >>>>>>>>>>>>>>> https://www.opengroup.org/austin/docs/austin_1110.pdf and > >>>>>>>>>>>>>>> added the new methods to a Googe Sheet, linked above. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> None of them appear to be in the RTEMS POSIX API Users Guide, > >>>>>>>>>>>>>>> but > >>>>>>>>>>>>>>> maybe that's not the right place to look. I'll stand by for > >>>>>>>>>>>>>>> your > >>>>>>>>>>>>>>> feedback regarding what's possible / desirable to add to > >>>>>>>>>>>>>>> RTEMS. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> It is possible they are in our C Library or Math Library. Or > >>>>>>>>>>>>>> just not in the manual. The POSIX manual tends to be sparse > >>>>>>>>>>>>>> since you can always use man pages or the POSIX standard. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Since you have RTEMS and tools built. Find one of the libc.a > >>>>>>>>>>>>>> and libm.a files in the tools install and librtemscpu.a in the > >>>>>>>>>>>>>> RTEMS build or install. Then try a command something like this: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> CPU-rtems6-nm LIBRARY | grep SYMBOL > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> If you see it list with T then it is in the text section and > >>>>>>>>>>>>>> there. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Following up, I initially answered from my phone and didn't > >>>>>>>>>>>>> look at source. I am still on my phone but looked through the > >>>>>>>>>>>>> list and think the last four methods are probably the only ones > >>>>>>>>>>>>> currently supported. > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://sourceware.org/git/?p=newlib-cygwin.git;a=tree;f=newlib/libc/string;h=ceeec602cdd0e6b5c6b002b741bda9b41da4e441;hb=HEAD > >>>>>>>>>>>>> > >>>>>>>>>>>>> POSIX support comes from a mix of RTEMS and newlib. That's key > >>>>>>>>>>>>> to this type of project. > >>>>>>>>>>>>> > >>>>>>>>>>>>> --joel > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks very much for your time! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Sincerely, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Matt > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> devel mailing list > >>>>>>>>>> devel@rtems.org > >>>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel > > _______________________________________________ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel