Oh wow. That makes so much more sense now! Thanks very much for the clarification!
Sincerely, Matt On Thu, Mar 25, 2021 at 2:18 PM Joel Sherrill <j...@rtems.org> wrote: > > > > On Thu, Mar 25, 2021 at 7:14 AM Matthew Joyce <mfjoyce2...@gmail.com> wrote: >> >> Hi Dr. Joel, >> >> Thanks very much, that's a big help! Correct, I've been updating the >> spreadsheet as I go along. Ok, now I see that strlcat/strlcpy are used >> in rtems/cpukit and implemented in Newlib. >> >> One additional question, please: I haven't yet looked into the source >> of NetBSD or FreeBSD, but I do see that Newlib already implements >> ppoll (poll.cc), dladdr (dlfcn.cc), pselect (select.cc), and >> sockatmark (net.cc). None of them are defined in the rtems environment >> yet. Is there any reason why the NetBSD/FreeBSD version would be >> preferable to Newlib for these? Or is it just a matter of testing >> what's out there to find what works well in the rtems environment? > > > Without looking at the newlib git repo, I can tell you that the files > you cite are the implementation of those methods for Cygwin. Just > because they are in C++. :) > > The parts of the newlib repo RTEMS uses are under the newlib/ > subdirectory not the cygwin one. Within that, there is a libc/sys and > only libc/sys/rtems is used for RTEMS. The others are for different > operating systems. There are a few places with "machine" directory > structures. Only the ones for the architecture you are building for > is used. > > As to why NetBSD for libdl, that is because portions of the code > originated there. > > And rtems-libbsd is based on FreeBSD. It is as close to the FreeBSD > source as we can keep it. > >> >> >> In my proposal I'll take your advice and work on some of the easier >> ones first in order to get the experience and process down. > > > There are tickets for a lot of methods. The rtems-docs repo has the > csv file (e.g. spreadsheet) which tracks RTEMS support against > various standards. The RTEMS POSIX Compliance Guide is generated > from that csv file. Between those, you can find other methods to ask > about. In general, if it is required by the Software Communications > Architecture (SCA) or FACE Technical Standard, then it is a method > someone expected to possibly be used in an embedded system. > SCA is a set of POSIX profiles focused on software defined radios and > the FACE Technical Standard was developed with avionics in mind. > > But any are fair game if they are actually implementable. I don;t think > the Compliance Guide says it yet, but we decided last year that > wordexp() is likely not supportable on RTEMS. The newlib > implementation assumes the presence of a shell with wildcard expansion > and ability to fork a process. > > --joel > >> >> >> Thank you again for your time! >> >> Matt >> >> On Wed, Mar 24, 2021 at 5:03 PM Joel Sherrill <j...@rtems.org> wrote: >> > >> > Wow! Good work. There is a lot to digest here. Comments interspersed. >> > >> > I assume the spreadsheet is updated. >> > >> > On Wed, Mar 24, 2021 at 7:38 AM Matthew Joyce <mfjoyce2...@gmail.com> >> > wrote: >> >> >> >> Hi Dr. Joel, >> >> >> >> I've gone over the list a few times now and see a few categories shaping >> >> up: >> >> >> >> 1) Already done (In Newlib source, defined in libc.a): >> >> a) reallocarray >> >> b) qsort_r >> >> c) memmem >> >> d) strlcat / strlcpy >> >> d) wcslcat / wcslcpy >> >> *Out of this group, strlcat and strlcpy also show up in >> >> src/rtems/cpukit. Why is that? >> > >> > >> > The good news is that we support these. :) >> > >> > It looks to me that strlcat and strlcpy are used in cpukit but not >> > implemented >> > there. Where do you think they are implemented. >> > >> > This is a good example where a source code browser is helpful. grep can >> > often answer the question but a source code browser can be easier. >> > Personally, >> > I use cscope but that is exceedingly old school. Any modern IDE should be >> > helpful. >> > >> >> >> >> 2) Not done yet (Do not show up in Newlib source or RTEMS): >> >> a) getlocalename_l >> >> b) posix_getdents >> >> c) sem_clockwait >> >> d) sig2str / str2sig >> >> >> >> 3) Not in Newlib; Referenced in RTEMS but hidden behind #ifdef: >> >> a) pthread_cond_clockwait >> >> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/condition_variable) >> >> b) pthread_mutex_clocklock >> >> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/mutex) >> >> c) pthread_rwlock_clockrdlock >> >> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex) >> >> c) pthread_rwlock_clockwrlock >> >> (rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex) >> >> *It looks like some groundwork was done, but the methods are not yet >> >> supported. >> > >> > >> > The paths you point to are C++ files that would implement C++ features >> > using the available POSIX services. So they are users, not providers. >> > >> > All of the pthread services related to these are implemented in >> > cpukit/posix/src. I think you can configure a clock for all these now >> > to be used by detailed on wait and timedwait calls. My understanding >> > is that these would let you use a specific clock on a per blocking call >> > basis. >> > >> > First question is which clocks are intended to be supported. >> > >> > Second is the pattern of picking which timeout queue to go on when >> > now it is coded to let you pick one which is used for the life of the >> > object. >> > >> >> >> >> 4) Misc (In Newlib source, not defined in libc.a, appear in RTEMS in >> >> various ways) >> >> a) getentropy (an alternate version is defined in RTEMS librtemsbsd.a, >> >> in src/rtems/bsps/shared/dev/getentropy/getentropy-cpucounter.c. The >> >> comments note that it is not cryptographically secure, so it may not >> >> fit the bill for the getentropy() mentioned in the Open Group >> >> document) >> > >> > >> > I am far from a cryptography expert but this looks like a case where >> > this method would be considered supported with the disclaimer that >> > the quality of the entropy value depends on the BSP. If the user has >> > specific requirements, they will need to verify the implementation >> > used by the BSP by default is appropriate. >> > >> >> >> >> b) ppoll (appears in rtems/6/share/gdb/syscalls) >> > >> > >> > You need to be more careful with the grep. These again are in the >> > installed tools and in this case, they appear in an XML file. Referenced >> > but not implemented. >> > >> > ppoll() will need to come from rtems-libbsd. The required system call >> > is included but disabled currently. AFAIK this means it is possible to >> > provide this but that would require a more detailed discussion in case >> > some underlying capability is missing. Chris Johns and Sebastian >> > Huber would be the ones to guide here. >> > >> > Ruling: Likely possible. >> > >> >> >> >> c) dladdr (appears in rtems/cpukit but not defined) >> > >> > >> > I think this can be implemented in libdl but I am not sure if the >> > code from NetBSD from this would directly work or just be a guide. >> > >> >> >> >> >> >> 5) Others? >> >> It looks like there was work done on methods like sockatmark and >> >> pselect, but I don't see them supported as yet. Should those be added >> >> to the list or are they still being worked on? >> > >> > >> > These would come from rtems-libbsd. >> > >> > I think sockatmark.c is implemented in freebsd/lib/libc/net/sockatmark.c >> > but I don't know if the ioctl() is implemented. I expect it is but this >> > would >> > at least require a test. It may just work. >> > >> > pselect() looks to be missing and would have to be ported from FreeBSD. >> > >> >> >> >> As you suggested, I'll look into NetBSD for dladdr and do some digging >> >> on the implementation of the other outstanding methods. You mentioned >> >> that the "clock" ones have to be strictly added to rtems/cpukit, but >> >> the references I found above are all in lib/gcc/sparc-rtems6/10.2.1. >> >> Why is that the case and what is 10.2.1? Also, I'm not sure what to >> >> make of getentropy and ppoll based on what I found above...at your >> >> convenience could you please advise? >> > >> > >> > Hopefully the above helped. >> > >> > You don't have to restrict your possible set to these new additions. >> > There are others. I think Eshan has done the research for where to >> > get implementations of the missing long double methods for newlib. >> > And there are tickets for other missing methods or specific capabilities >> > in methods that are supported. Those are quite possible to have >> > some alternatives that are easier to approach. >> > >> > --joel >> > >> > >> >> >> >> >> >> Thank you very much! >> >> >> >> Matt >> >> >> >> >> >> >> >> >> >> On Sun, Mar 21, 2021 at 6:38 PM Joel Sherrill <j...@rtems.org> wrote: >> >> > >> >> > >> >> > >> >> > On Sun, Mar 21, 2021 at 2:28 AM Matthew Joyce <mfjoyce2...@gmail.com> >> >> > wrote: >> >> >> >> >> >> Gentlemen, >> >> >> >> >> >> Awesome, thanks! I see how that works now...I'll give it a thorough >> >> >> look tomorrow and will update the spreadsheet accordingly. I'll pipe >> >> >> back up when I have a more accurate look of what's currently there. >> >> > >> >> > >> >> > Knowing what doesn't have to be done is the first step. (rtems, newlib, >> >> > and libbsd) >> >> > >> >> > I'd be prone to look for things that are easy to add first. >> >> > >> >> > Some may not be implementable on RTEMS due to only supporting a >> >> > single process and no virtual memory. If you have doubts on whether it >> >> > is possible to support a specific method, speak up and let's try to >> >> > decide. >> >> > >> >> > Then find upstream places for an implementation where possible. I >> >> > suspect >> >> > all the new "clock" methods will require discussion on an implementation >> >> > pattern but those must strictly be added to rtems/cpukit with tests and >> >> > documentation. At least I can throw you that much. :) >> >> > >> >> >> >> >> >> Thanks again and have a great Sunday! >> >> >> >> >> >> Matt >> >> >> >> >> >> On Fri, Mar 19, 2021 at 8:27 PM Joel Sherrill <j...@rtems.org> wrote: >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Fri, Mar 19, 2021 at 1:08 PM Gedare Bloom <ged...@rtems.org> >> >> >> > wrote: >> >> >> >> >> >> >> >> On Fri, Mar 19, 2021 at 11:16 AM Matthew Joyce >> >> >> >> <mfjoyce2...@gmail.com> wrote: >> >> >> >> > >> >> >> >> > Dr. Joel, >> >> >> >> > >> >> >> >> > Thanks very much...I'll keep working to get a sense of what goes >> >> >> >> > where! In the meantime, where can I look to get the ground truth >> >> >> >> > of >> >> >> >> > which methods are "in RTEMS" as opposed to those in newlib? >> >> >> >> > >> >> >> >> There is only one ground truth: >> >> >> >> git://git.rtems.org/rtems.git >> >> >> >> >> >> >> >> And for newlib >> >> >> >> >> >> >> >> git://sourceware.org/git/newlib-cygwin.git >> >> >> >> >> >> >> >> That said, searching for the function name symbols in compiled >> >> >> >> libraries is a good first step to rule out newlib. Then, you can >> >> >> >> 'grep' the RTEMS source code for the function names to see if they >> >> >> >> exist there. >> >> >> > >> >> >> > >> >> >> > rtems/cpukit to be specitic. It won't be implemented anywhere else. >> >> >> > >> >> >> > And clearly we both have forgotten that networking APIs are in the >> >> >> > rtems-libbsd repository. >> >> >> > >> >> >> > https://git.rtems.org/rtems-libbsd/ >> >> >> > >> >> >> > I suspect ppoll() might already be in there. Or at least supported by >> >> >> > FreeBSD. >> >> >> > >> >> >> > You should clone everything and grep the sources. newlib already has >> >> >> > qsort_r. This is the nm I used: >> >> >> > >> >> >> > $ ~/rtems-work/tools/6/bin/sparc-rtems6-nm >> >> >> > ~/rtems-work/tools/6/sparc-rtems6/lib/libc.a | grep qsort_r >> >> >> > lib_a-bsd_qsort_r.o: >> >> >> > 00000000 T __bsd_qsort_r >> >> >> > lib_a-qsort_r.o: >> >> >> > 00000000 T qsort_r >> >> >> > >> >> >> > Notice the last line has "T qsort_r" which says it is defined. >> >> >> > >> >> >> > grep -r in the newlib source shows it is in ./libc/search/qsort_r.c >> >> >> > >> >> >> > dladdr() looks to be prototyped in RTEMS but hidden behind an ifdef >> >> >> > like it >> >> >> > wasn't ported from NetBSD so that looks possible. It is in rtems. >> >> >> > >> >> >> > Those two examples should help you figure out why you missed >> >> >> > finding some things that were implemented. >> >> >> > >> >> >> > I need to figure out what this next POSIX version is to be called >> >> >> > so I can update the tracking spreadsheet that generates the RTEMS >> >> >> > POSIX Compliance Guide, :) >> >> >> > >> >> >> > --joel >> >> >> >> >> >> >> >> >> >> >> >> > Thanks again! >> >> >> >> > >> >> >> >> > Matt >> >> >> >> > >> >> >> >> > On Fri, Mar 19, 2021 at 1:58 PM Joel Sherrill <j...@rtems.org> >> >> >> >> > wrote: >> >> >> >> > > >> >> >> >> > > Keep devel@ on the list. :) >> >> >> >> > > >> >> >> >> > > On Fri, Mar 19, 2021 at 7:51 AM Matthew Joyce >> >> >> >> > > <mfjoyce2...@gmail.com> wrote: >> >> >> >> > >> >> >> >> >> > >> Sir, >> >> >> >> > >> >> >> >> >> > >> Thank you for the link! I see that you're right, those last >> >> >> >> > >> four are >> >> >> >> > >> in newlib, plus memmem(). I updated those in the Google Sheet. >> >> >> >> > >> >> >> >> >> > >> Now I see the newlib part, but where are you referring to >> >> >> >> > >> specifically >> >> >> >> > >> when you say RTEMS, as in "POSIX support comes from a mix of >> >> >> >> > >> RTEMS and >> >> >> >> > >> newlib"? >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > POSIX is a HUGE HUGE standard and references other standards. >> >> >> >> > > One >> >> >> >> > > it references and pulls in is the C99 Standard C Library which >> >> >> >> > > is libc and >> >> >> >> > > libm. RTEMS mostly does not implement this functionality and >> >> >> >> > > relies on >> >> >> >> > > another open source project for those APIs. Newlib is an open >> >> >> >> > > source >> >> >> >> > > C Library used by RTEMS, Cygwin, and most embedded systems GNU >> >> >> >> > > tools >> >> >> >> > > chains. >> >> >> >> > > >> >> >> >> > > Most of the POSIX header files with RTEMS are actually in >> >> >> >> > > Newlib even >> >> >> >> > > if they originated with RTEMS. Many are shared with Cygwin. >> >> >> >> > > >> >> >> >> > > So methods like the string, memory, and *printf come from >> >> >> >> > > Newlib since they >> >> >> >> > > are in C99. We provide POSIX like threading, signals, core file >> >> >> >> > > access, and >> >> >> >> > > much more. >> >> >> >> > > >> >> >> >> > > It's a complementary relationship but it takes a bit to figure >> >> >> >> > > out when >> >> >> >> > > something should be in one or the other. The line gets blurred >> >> >> >> > > at times. >> >> >> >> > > >> >> >> >> > > Say you added a new CPU architecture implementation of a math >> >> >> >> > > method (like Eshan did last year), then it goes in newlib. But >> >> >> >> > > he also >> >> >> >> > > added some POSIX methods which go in RTEMS. In either case, >> >> >> >> > > we like tests for them in RTEMS to show they work in our >> >> >> >> > > environment. >> >> >> >> > > >> >> >> >> > > --joel >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > >> >> >> >> >> > >> Thanks again! >> >> >> >> > >> >> >> >> >> > >> Matt >> >> >> >> > >> >> >> >> >> > >> On Fri, Mar 19, 2021 at 1:13 PM Joel Sherrill <j...@rtems.org> >> >> >> >> > >> wrote: >> >> >> >> > >> > >> >> >> >> > >> > >> >> >> >> > >> > >> >> >> >> > >> > On Fri, Mar 19, 2021, 6:40 AM Joel Sherrill <j...@rtems.org> >> >> >> >> > >> > wrote: >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> On Fri, Mar 19, 2021, 5:48 AM Matthew Joyce >> >> >> >> > >> >> <mfjoyce2...@gmail.com> wrote: >> >> >> >> > >> >>> >> >> >> >> > >> >>> https://docs.google.com/spreadsheets/d/1reCNOIZC5JTwQENgl-hvG8THfQqNtlUDVy_07PYodic/edit?usp=sharing >> >> >> >> > >> >>> >> >> >> >> > >> >>> Hello, >> >> >> >> > >> >>> >> >> >> >> > >> >>> As suggested by Dr. Sherril, I've taken an initial look >> >> >> >> > >> >>> through this >> >> >> >> > >> >>> document >> >> >> >> > >> >>> https://www.opengroup.org/austin/docs/austin_1110.pdf and >> >> >> >> > >> >>> added the new methods to a Googe Sheet, linked above. >> >> >> >> > >> >>> >> >> >> >> > >> >>> None of them appear to be in the RTEMS POSIX API Users >> >> >> >> > >> >>> Guide, but >> >> >> >> > >> >>> maybe that's not the right place to look. I'll stand by >> >> >> >> > >> >>> for your >> >> >> >> > >> >>> feedback regarding what's possible / desirable to add to >> >> >> >> > >> >>> RTEMS. >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> It is possible they are in our C Library or Math Library. >> >> >> >> > >> >> Or just not in the manual. The POSIX manual tends to be >> >> >> >> > >> >> sparse since you can always use man pages or the POSIX >> >> >> >> > >> >> standard. >> >> >> >> > >> >> >> >> >> >> > >> >> Since you have RTEMS and tools built. Find one of the >> >> >> >> > >> >> libc.a and libm.a files in the tools install and >> >> >> >> > >> >> librtemscpu.a in the RTEMS build or install. Then try a >> >> >> >> > >> >> command something like this: >> >> >> >> > >> >> >> >> >> >> > >> >> CPU-rtems6-nm LIBRARY | grep SYMBOL >> >> >> >> > >> >> >> >> >> >> > >> >> If you see it list with T then it is in the text section >> >> >> >> > >> >> and there. >> >> >> >> > >> > >> >> >> >> > >> > >> >> >> >> > >> > Following up, I initially answered from my phone and didn't >> >> >> >> > >> > look at source. I am still on my phone but looked through >> >> >> >> > >> > the list and think the last four methods are probably the >> >> >> >> > >> > only ones currently supported. >> >> >> >> > >> > >> >> >> >> > >> > https://sourceware.org/git/?p=newlib-cygwin.git;a=tree;f=newlib/libc/string;h=ceeec602cdd0e6b5c6b002b741bda9b41da4e441;hb=HEAD >> >> >> >> > >> > >> >> >> >> > >> > POSIX support comes from a mix of RTEMS and newlib. That's >> >> >> >> > >> > key to this type of project. >> >> >> >> > >> > >> >> >> >> > >> > --joel >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >>> >> >> >> >> > >> >>> Thanks very much for your time! >> >> >> >> > >> >>> >> >> >> >> > >> >>> Sincerely, >> >> >> >> > >> >>> >> >> >> >> > >> >>> Matt >> >> >> >> > _______________________________________________ >> >> >> >> > devel mailing list >> >> >> >> > devel@rtems.org >> >> >> >> > http://lists.rtems.org/mailman/listinfo/devel _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel