Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
Hi There, Running latest head with libc-only *context function. mtasker test program works, but powerdns-recursor still doesnt compile (even with gcc 3.4.6) /usr/include/machine/endian.h:70:1: warning: this is the location of the previous definition pdns_recursor.cc: In function `void startDoResolve(void*)': pdns_recursor.cc:563: error: invalid conversion from `void*' to `char*' pdns_recursor.cc:564: error: invalid conversion from `void*' to `char*' pdns_recursor.cc:1139: confused by earlier errors, bailing out [wrapper.sh] note: The real command line, after the pkgsrc wrapper, was: /usr/obj/pkgsrc/wip/powerdns-recursor/work/.gcc/bin/g++ -O -pipe -I/usr/obj/pkgsrc/wip/powerdns-recursor/work/.buildlink/include -Wall -O3 -c -o pdns_recursor.o pdns_recursor.cc -L/usr/obj/pkgsrc/wip/powerdns-recursor/work/.buildlink/lib gmake: *** [pdns_recursor.o] Error 1 *** Error code 2 Stop. bmake: stopped in /usr/pkgsrc/wip/powerdns-recursor *** Error code 1 Stop. bmake: stopped in /usr/pkgsrc/wip/powerdns-recursor Petr [EMAIL PROTECTED] wrote: :I revisited these two kernel files and agree partially with your assessment: :- machdep.c fixed a missing linebreak; the other change (mc_len) was cruft. :- ucontext.h contains the protos for the *context routines; they still need :to be somewhere so autoconf can find them, and I felt that this was the best :place for them. The other stuff (uc_flags, UCF_SWAPPED) was cruft. : :-- :Matt Emmerton Ok, I restored those bits. I have implemented the assembly and done some other cleanups and a bit of testing. I am committing the completed work now. I have not yet implemented the floating point save/restore. I will check save-used for FP registers and see if I can optimize what gets saved. When switching occurs synchronously I don't think the FP regs need to be saved/restored unless the prorgam is compiled to use FP regs and general regs so this may be good enough for the moment. Switching from a signal dispatch is more problematic. That almost certainly requires FP save/restore. Now we need some comprehensive testing with packages that actually use this beast! -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
:I revisited these two kernel files and agree partially with your assessment: :- machdep.c fixed a missing linebreak; the other change (mc_len) was cruft. :- ucontext.h contains the protos for the *context routines; they still need :to be somewhere so autoconf can find them, and I felt that this was the best :place for them. The other stuff (uc_flags, UCF_SWAPPED) was cruft. : :-- :Matt Emmerton Ok, I restored those bits. I have implemented the assembly and done some other cleanups and a bit of testing. I am committing the completed work now. I have not yet implemented the floating point save/restore. I will check save-used for FP registers and see if I can optimize what gets saved. When switching occurs synchronously I don't think the FP regs need to be saved/restored unless the prorgam is compiled to use FP regs and general regs so this may be good enough for the moment. Switching from a signal dispatch is more problematic. That almost certainly requires FP save/restore. Now we need some comprehensive testing with packages that actually use this beast! -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
> > :Code changes uploaded to website, rev4. > : > :-- > :Matt Emmerton > > Ok, looks pretty good. I'm going to run with this so don't post any > more updates. rev4 had some kernel cruft left over from rev3. I > removed it. > > -Matt I revisited these two kernel files and agree partially with your assessment: - machdep.c fixed a missing linebreak; the other change (mc_len) was cruft. - ucontext.h contains the protos for the *context routines; they still need to be somewhere so autoconf can find them, and I felt that this was the best place for them. The other stuff (uc_flags, UCF_SWAPPED) was cruft. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
:Code changes uploaded to website, rev4. : :-- :Matt Emmerton Ok, looks pretty good. I'm going to run with this so don't post any more updates. rev4 had some kernel cruft left over from rev3. I removed it. -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
Matt, > :A first cut of the libc-based implementation, using a (new) syscall to > :get/set the signal its is available, and can be found here: > :http://www.gsicomp.on.ca/~matt/dfly/ (rev3, bottom of page) > > Ok, I see a few things, all easy to fix: > > * sigpend is not swapped in and out. This is the bitmask of pending > signals. It stays as it is. I misunderstood your earlier comment about the pending mask. > * sigmask must be adjusted by the kernel to remove unmaskable signals > using SIG_CANTMASK(sigmask); > > * There is a naming conflict between the sigmask() system call and > thethe 'sigmask' identifier, but (see next item)... > > * You shouldn't need a new system call anyway, the sigprocmask() system > call can be used to retrieve and set the signal mask. I recommend > using that. Absolutely. The only reason I invented a new syscall was because of my misunderstanding above :) > There is one more issue, and that is an atomicy issue. We want the > new signal mask to be set simultaniously with the restoration of the > context (or most of it). This means that all signals must be masked > temporarily while restoring the context. But don't worry about it, > I can handle that part as well. It does mean an extra system call > but that isn't our concern at the moment. > > I can fill in the missing code assembly, no problem. If you do one > more rev using sigprocmask() instead of creating a new system call > I can then take it from there. Code changes uploaded to website, rev4. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
[EMAIL PROTECTED] wrote: matt, your email seems mangled There is one more issue, and that is an atomicy issue. We want the new signal mask to be set simultaniously with the restoration of the context (or most of it). This means that all signals must be masked temporarily while restoring the context. But don't worry about it, I can handle that part as well. It does mean an extra system call but that isn't our concern at the moment. so the sequence is something like sigprocmask(SIG_BLOCK, ALL, &ocntx->uc_sigmask); swap registers+stacks sigprocmask(SIG_SETMASK, &ncntx->us_sigmask, NULL); for swapcontext. am i wrong or is setcontext essentially swapcontext(&tmpcontext, newcntx); ? cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
:A first cut of the libc-based implementation, using a (new) syscall to :get/set the signal its is available, and can be found here: :http://www.gsicomp.on.ca/~matt/dfly/ (rev3, bottom of page) : :Note that this implementation is not complete, as it is missing the userland :get/set_mcontext code. I'm sure I've also missed some nuance wrt signal :handling (locking, ordering, etc) in the design and use of the new sigmask :syscall. : :-- :Matt Emmerton Ok, I see a few things, all easy to fix: * sigpend is not swapped in and out. This is the bitmask of pending signals. It stays as it is. * sigmask must be adjusted by the kernel to remove unmaskable signals using SIG_CANTMASK(sigmask); * There is a naming conflict between the sigmask() system call and thethe 'sigmask' identifier, but (see next item)... * You shouldn't need a new system call anyway, the sigprocmask() system call can be used to retrieve and set the signal mask. I recommend using that. There is one more issue, and that is an atomicy issue. We want the new signal mask to be set simultaniously with the restoration of the context (or most of it). This means that all signals must be masked temporarily while restoring the context. But don't worry about it, I can handle that part as well. It does mean an extra system call but that isn't our concern at the moment. I can fill in the missing code assembly, no problem. If you do one more rev using sigprocmask() instead of creating a new system call I can then take it from there. -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc [ revision 3 ]
> :I agree with the assessment. Last night I moved all the requisite bits from > :kernel-space into libc, with the exception of the low-level > :[get|set]_mcontext routines. (My Intel assembly knowledge is very limited, > :especially for anything after the 8086, so I could use some help to write > :this code.) I'm almost done the syscall work to get/set the signal mask and > :pending signal mask, so should have something up on the web before the end > :of the afternoon (EST) for review. > :-- > :Matt Emmerton > > I can do that bit. If you post the bits you have I can supply and > test the low level switch code. > > -Matt > Matthew Dillon A first cut of the libc-based implementation, using a (new) syscall to get/set the signal its is available, and can be found here: http://www.gsicomp.on.ca/~matt/dfly/ (rev3, bottom of page) Note that this implementation is not complete, as it is missing the userland get/set_mcontext code. I'm sure I've also missed some nuance wrt signal handling (locking, ordering, etc) in the design and use of the new sigmask syscall. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
I found one other issue, and that is we can't really determine whether the FP unit is being used from userland or generate an exception-on-first-use visible to userland, which means we can't optimize-out the FP state save/restore. I don't think this is a showstopper, though. For now we can just unconditionally save/restore the FP state... it only takes 69ns or so on my test box for a save+restore sequence. Later on the kernel can easily supply the required information via shared memory or something similar (along with the signal mask and pending bits). Either way, just doing it uncoditionally is still far faster then making a system call. -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc
:I agree with the assessment. Last night I moved all the requisite bits from :kernel-space into libc, with the exception of the low-level :[get|set]_mcontext routines. (My Intel assembly knowledge is very limited, :especially for anything after the 8086, so I could use some help to write :this code.) I'm almost done the syscall work to get/set the signal mask and :pending signal mask, so should have something up on the web before the end :of the afternoon (EST) for review. : :Regards, :-- :Matt Emmerton I can do that bit. If you post the bits you have I can supply and test the low level switch code. -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc
> :So is the current context code going into cvs HEAD or whats happening? > : > :Petr > > I'm going to let the discussion continue for another day or two but > I will note that Matt is the person doing the work so it depends on > how much time he has available to do the libc version prior to the > release (assuming he agrees with the assessment that it is better to > do the libc version for the release then the syscall version). > > -Matt I agree with the assessment. Last night I moved all the requisite bits from kernel-space into libc, with the exception of the low-level [get|set]_mcontext routines. (My Intel assembly knowledge is very limited, especially for anything after the 8086, so I could use some help to write this code.) I'm almost done the syscall work to get/set the signal mask and pending signal mask, so should have something up on the web before the end of the afternoon (EST) for review. Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Petr Janda wrote: Devs: have you got any comment on this and what are the chances of updating gcc 3.4 before the release? the commit just went in. cheers simon PS: could you (and others) please try and avoid top-posting? this full quote thing is making mails really unreadable. -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
On Sun, Jan 14, 2007 at 03:17:33AM +1100, Petr Janda wrote: > Ah sh*t. > > Devs: have you got any comment on this and what are the chances of > updating gcc 3.4 before the release? > > Petr I'd say the chances are very high. --Peter pgpniW7BHjGIr.pgp Description: PGP signature
Re: Request for swapcontext and getcontext to be ported to our libc
Ah sh*t. Devs: have you got any comment on this and what are the chances of updating gcc 3.4 before the release? Petr Matt Emmerton wrote: Petr, Based on information scattered about the Internet, as well as what the FreeBSD folks have done, it appears that powerdns-recursor requires gcc-3.4.6 to compile properly. This is a problem as DFly's native gcc is 3.4.5 and the pkgsrc version of 3.4.6 does not support DFly. To do this right, we will need DFly to support the 3.4.6 compiler before we can support this package. Regards, -- Matt Emmerton I see. What about getting someone to get powerdns-recusor compiling and running? Once that is done we shall arrange for your payment. Petr Matt Emmerton wrote: Hey, Any progress on the fully libc *context functions? I've toyed with it a bit, but there are some issues I'm not sure how to solve (such as accessing curthread -- a kernel construct -- from userland). I would welcome discussion with the person who suggested it (corecode?). Also, I felt it pertinent to focus on the quick-start FreeBSD port to get it into the 1.8-release. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Petr, Based on information scattered about the Internet, as well as what the FreeBSD folks have done, it appears that powerdns-recursor requires gcc-3.4.6 to compile properly. This is a problem as DFly's native gcc is 3.4.5 and the pkgsrc version of 3.4.6 does not support DFly. To do this right, we will need DFly to support the 3.4.6 compiler before we can support this package. Regards, -- Matt Emmerton > > I see. > > What about getting someone to get powerdns-recusor compiling and running? > > > > Once that is done we shall arrange for your payment. > > > > Petr > > > > > > Matt Emmerton wrote: > > >> Hey, > > >> Any progress on the fully libc *context functions? > > >> > > > > > > I've toyed with it a bit, but there are some issues I'm not sure how to > > > solve (such as accessing curthread -- a kernel construct -- from > userland). > > > I would welcome discussion with the person who suggested it (corecode?). > > > > > > Also, I felt it pertinent to focus on the quick-start FreeBSD port to > get it > > > into the 1.8-release. > > > > > > -- > > > Matt Emmerton > > > > > > > > >
Re: Request for swapcontext and getcontext to be ported to our libc
I think we're debating whether to "do it right" or "do it now". Tuesday is a long way off yet :) -- Matt Emmerton > So is the current context code going into cvs HEAD or whats happening? > > Petr > > Simon 'corecode' Schubert wrote: > > Matthew Dillon wrote: > >> The answer is: yes, it is possible. There is no reason why the > >> signal mask (and signal pending bitmap) couldn't be stored in user > >> memory, allowing the userland threading system to swap out the mask > >> without entering the kernel. > > > > I absolutely agree. I am thinking of a special page mapped in kernel > > and userspace memory, where things like signal mask and pending (per > > thread) are located, and maybe also a timecounter, pid, etc. > > > >> So I think there is a very good argument for making the current > >> release > >> implementation do these calls in libc and not as system calls, and > >> initially have libc just get the signal mask with a system call, > >> then > >> at some later date add the feature sthat allow the userland context > >> routines to run fully without having to make any system calls at > >> all. > > > > If Matt Emmerton finds the time to implement it, I am all for it. > > Having these functions as syscalls seems wrong in my eyes. > > > > cheers > > simon > > >
Re: Request for swapcontext and getcontext to be ported to our libc
: :So is the current context code going into cvs HEAD or whats happening? : :Petr I'm going to let the discussion continue for another day or two but I will note that Matt is the person doing the work so it depends on how much time he has available to do the libc version prior to the release (assuming he agrees with the assessment that it is better to do the libc version for the release then the syscall version). -Matt
Re: Request for swapcontext and getcontext to be ported to our libc
So is the current context code going into cvs HEAD or whats happening? Petr Simon 'corecode' Schubert wrote: Matthew Dillon wrote: The answer is: yes, it is possible. There is no reason why the signal mask (and signal pending bitmap) couldn't be stored in user memory, allowing the userland threading system to swap out the mask without entering the kernel. I absolutely agree. I am thinking of a special page mapped in kernel and userspace memory, where things like signal mask and pending (per thread) are located, and maybe also a timecounter, pid, etc. So I think there is a very good argument for making the current release implementation do these calls in libc and not as system calls, and initially have libc just get the signal mask with a system call, then at some later date add the feature sthat allow the userland context routines to run fully without having to make any system calls at all. If Matt Emmerton finds the time to implement it, I am all for it. Having these functions as syscalls seems wrong in my eyes. cheers simon
Re: Request for swapcontext and getcontext to be ported to our libc
Matthew Dillon wrote: The answer is: yes, it is possible. There is no reason why the signal mask (and signal pending bitmap) couldn't be stored in user memory, allowing the userland threading system to swap out the mask without entering the kernel. I absolutely agree. I am thinking of a special page mapped in kernel and userspace memory, where things like signal mask and pending (per thread) are located, and maybe also a timecounter, pid, etc. So I think there is a very good argument for making the current release implementation do these calls in libc and not as system calls, and initially have libc just get the signal mask with a system call, then at some later date add the feature sthat allow the userland context routines to run fully without having to make any system calls at all. If Matt Emmerton finds the time to implement it, I am all for it. Having these functions as syscalls seems wrong in my eyes. cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
Ok, well, I'll bring up a counterpoint - one that argues for it to remain in userland. When I examine the FreeBSD code, the ONLY thing the kernel does that one needs to be in kernel mode for is setting the signal mask. That's it. Everything else can be done in userland. So the question becomes, is it possible to have a threaded system where you can set the signal mask during a context switch and NOT have to switch into the kernel? The answer is: yes, it is possible. There is no reason why the signal mask (and signal pending bitmap) couldn't be stored in user memory, allowing the userland threading system to swap out the mask without entering the kernel. It would require having a system call that allows the userland threading system to specify the user address where the signal mask resides, but it could be done and it would not be difficult. So I think there is a very good argument for making the current release implementation do these calls in libc and not as system calls, and initially have libc just get the signal mask with a system call, then at some later date add the feature sthat allow the userland context routines to run fully without having to make any system calls at all. -Matt
Re: Request for swapcontext and getcontext to be ported to our libc
:However, I recall it being mentioned that the proliferation of syscalls is :unneccessary, so a libc implementation would keep the kernel lean. The :reason we need curthread is to get access to the frame construct, which is :neccessary to save the context. : :Regards, :-- :Matt Emmerton Nah. The frame is just made up of entirely user-accessible registers. The only thing that is kernel-only is the signal mask (and one can make a system call to get it). The FP state can be controlled in userland too. The really interesting thing about it is that if you do it in userland you don't have to save or restore the call-used registers. I think the FreeBSD folks did these as system calls because they believe the context switching routines may have to do other kernel-only things in the future. That isn't a bad argument. On the otherhand, if one is talking about operating in virtualized environments (any sort of virtualization), having to do a system call at all is very, very expensive. -Matt Matthew Dillon <[EMAIL PROTECTED]>
Re: Request for swapcontext and getcontext to be ported to our libc
Matt Emmerton wrote: Simon wrote: Matt Emmerton wrote: I've toyed with it a bit, but there are some issues I'm not sure how to solve (such as accessing curthread -- a kernel construct -- from userland). I would welcome discussion with the person who suggested it (corecode?). In which way do you have to access curthread? That's a kernel-only entity, I don't think you have to access that in userland. Besides, if we take in the system call version, there is no need for a pure libc based version anymore. Point taken, and that's definitely the path we're going down at the moment. However, I recall it being mentioned that the proliferation of syscalls is unneccessary, so a libc implementation would keep the kernel lean. Once we have introduced this syscall, we can't remove it anymore for binary compatibility. The reason we need curthread is to get access to the frame construct, which is neccessary to save the context. when you are in userland, you have to build the frame yourself. cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
Simon wrote: > Matt Emmerton wrote: > > I've toyed with it a bit, but there are some issues I'm not sure how to > > solve (such as accessing curthread -- a kernel construct -- from userland). > > I would welcome discussion with the person who suggested it (corecode?). > > In which way do you have to access curthread? That's a kernel-only > entity, I don't think you have to access that in userland. Besides, if we > take in the system call version, there is no need for a pure libc based > version anymore. Point taken, and that's definitely the path we're going down at the moment. However, I recall it being mentioned that the proliferation of syscalls is unneccessary, so a libc implementation would keep the kernel lean. The reason we need curthread is to get access to the frame construct, which is neccessary to save the context. Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Petr, I'm starting on that now :) -- Matt Emmerton > I see. > What about getting someone to get powerdns-recusor compiling and running? > > Once that is done we shall arrange for your payment. > > Petr > > > Matt Emmerton wrote: > >> Hey, > >> Any progress on the fully libc *context functions? > >> > > > > I've toyed with it a bit, but there are some issues I'm not sure how to > > solve (such as accessing curthread -- a kernel construct -- from userland). > > I would welcome discussion with the person who suggested it (corecode?). > > > > Also, I felt it pertinent to focus on the quick-start FreeBSD port to get it > > into the 1.8-release. > > > > -- > > Matt Emmerton > > > > >
Re: Request for swapcontext and getcontext to be ported to our libc
I see. What about getting someone to get powerdns-recusor compiling and running? Once that is done we shall arrange for your payment. Petr Matt Emmerton wrote: Hey, Any progress on the fully libc *context functions? I've toyed with it a bit, but there are some issues I'm not sure how to solve (such as accessing curthread -- a kernel construct -- from userland). I would welcome discussion with the person who suggested it (corecode?). Also, I felt it pertinent to focus on the quick-start FreeBSD port to get it into the 1.8-release. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Matt Emmerton wrote: I've toyed with it a bit, but there are some issues I'm not sure how to solve (such as accessing curthread -- a kernel construct -- from userland). I would welcome discussion with the person who suggested it (corecode?). In which way do you have to access curthread? That's a kernel-only entity, I don't think you have to access that in userland. Besides, if we take in the system call version, there is no need for a pure libc based version anymore. cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
> Hey, > Any progress on the fully libc *context functions? I've toyed with it a bit, but there are some issues I'm not sure how to solve (such as accessing curthread -- a kernel construct -- from userland). I would welcome discussion with the person who suggested it (corecode?). Also, I felt it pertinent to focus on the quick-start FreeBSD port to get it into the 1.8-release. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Hey, Any progress on the fully libc *context functions? Petr
Re: Request for swapcontext and getcontext to be ported to our libc
Well for my purposes I need NSS+PAM working together. I dont care how its done as long as its working. Petr Simon 'corecode' Schubert wrote: Petr Janda wrote: You're the man! If you can do all that for me and youre still interested, in a couple of months id like to port NSS into DragonFly (of course thats a longer job, which means more money towards your way too) I am very happy to hear that somebody (you) is actually funding dragonfly work now. Absolutely cool. For NSS, we planned (about 3 years ago) to have a nssd + libc interfacing with this nssd, instead of using shared libraries. I'd love to have this working for PAM as well, but unfortunately it doesn't quite work :/ Well, maybe now is the time to start fleshing out the details again. cheers simon
Re: Request for swapcontext and getcontext to be ported to our libc
Petr Janda wrote: You're the man! If you can do all that for me and youre still interested, in a couple of months id like to port NSS into DragonFly (of course thats a longer job, which means more money towards your way too) I am very happy to hear that somebody (you) is actually funding dragonfly work now. Absolutely cool. For NSS, we planned (about 3 years ago) to have a nssd + libc interfacing with this nssd, instead of using shared libraries. I'd love to have this working for PAM as well, but unfortunately it doesn't quite work :/ Well, maybe now is the time to start fleshing out the details again. cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
Matt Emmerton wrote: Just letting you know that mtasker now compiles after buildworld/installworld. I assume the "mt" test program runs correctly? I should output an alternating stream of "a" or "b" characters. You can terminate with ^C. Yes, thats what is on the screen. Though, wip/powerdns-recusor doesnt compile yet (a new error), not mtasker related. If someone could please take a look at it, I would be very happy. I'm looking into it. After the error you cite is fixed, there are many more, and I will have to work with the pkgsrc and PowerDNS folks to get them fixed. -- Matt Emmerton You're the man! If you can do all that for me and youre still interested, in a couple of months id like to port NSS into DragonFly (of course thats a longer job, which means more money towards your way too) Cheers, Petr
Re: Request for swapcontext and getcontext to be ported to our libc
> Just letting you know that mtasker now compiles after > buildworld/installworld. I assume the "mt" test program runs correctly? I should output an alternating stream of "a" or "b" characters. You can terminate with ^C. > Though, wip/powerdns-recusor doesnt compile yet > (a new error), not mtasker related. > > If someone could please take a look at it, I would be very happy. I'm looking into it. After the error you cite is fixed, there are many more, and I will have to work with the pkgsrc and PowerDNS folks to get them fixed. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Just letting you know that mtasker now compiles after buildworld/installworld. Though, wip/powerdns-recusor doesnt compile yet (a new error), not mtasker related. pdns_recursor.cc: In function `void startDoResolve(void*)': pdns_recursor.cc:563: error: invalid conversion from `void*' to `char*' pdns_recursor.cc:564: error: invalid conversion from `void*' to `char*' pdns_recursor.cc:1139: confused by earlier errors, bailing out [wrapper.sh] note: The real command line, after the pkgsrc wrapper, was: /usr/obj/pkgsrc/wip/powerdns-recursor/work/.gcc/bin/g++ -O -pipe -I/usr/obj/pkgsrc/wip/powerdns-recursor/work/.buildlink/include -Wall -O3 -c -o pdns_recursor.o pdns_recursor.cc -L/usr/obj/pkgsrc/wip/powerdns-recursor/work/.buildlink/lib gmake: *** [pdns_recursor.o] Error 1 *** Error code 2 If someone could please take a look at it, I would be very happy. Petr Matt Emmerton wrote: [ top-posting corrected ] Petr Janda wrote: Matt Emmerton wrote: The first cut of this work has been completed. I've ported over the FreeBSD i386 work, which is a hybrid libc + kernel syscall implementation. You can find the source changes and a tiny test program here: http://www.gsicomp.on.ca/~matt/dfly/ I hope to have a libc-only implementation completed before the end of the week. Libc and kernel compiled, but wip/powerdns-recursor still gives me the mtasker errors. I'm assuming that you did installworld/installkernel/reboot before attempting to build wip/powerdns-recursor. After a bit of searching, I added the wip/ bits to my pkgsrc installation and have found that powerdns-recursor has compile warnings/errors/ICE on DFly that are totally unrelated to the *context functions. I'm curious to know what patches/tweaks you are using in your environment to get to the point where you see *context-related failures. # uname -a DragonFly jekell.gsicomp.on.ca 1.7.0-DEVELOPMENT DragonFly 1.7.0-DEVELOPMENT #0: Wed Jan 3 18:21:26 EST 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 # gcc -v Using built-in specs. Configured with: DragonFly/i386 system compiler Thread model: posix gcc version 3.4.5 20050809 (prerelease) [DragonFly] (propolice, visibility) # grep DISTNAME /usr/pkgsrc/wip/powerdns-recursor/Makefile DISTNAME= pdns-recursor-3.1.4 Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
[ top-posting corrected ] > Petr Janda wrote: > > Matt Emmerton wrote: > > The first cut of this work has been completed. I've ported over the FreeBSD > > i386 work, which is a hybrid libc + kernel syscall implementation. > > > > You can find the source changes and a tiny test program here: > > http://www.gsicomp.on.ca/~matt/dfly/ > > > > I hope to have a libc-only implementation completed before the end of the > > week. > > > Libc and kernel compiled, but wip/powerdns-recursor still gives me the > mtasker errors. I'm assuming that you did installworld/installkernel/reboot before attempting to build wip/powerdns-recursor. After a bit of searching, I added the wip/ bits to my pkgsrc installation and have found that powerdns-recursor has compile warnings/errors/ICE on DFly that are totally unrelated to the *context functions. I'm curious to know what patches/tweaks you are using in your environment to get to the point where you see *context-related failures. # uname -a DragonFly jekell.gsicomp.on.ca 1.7.0-DEVELOPMENT DragonFly 1.7.0-DEVELOPMENT #0: Wed Jan 3 18:21:26 EST 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 # gcc -v Using built-in specs. Configured with: DragonFly/i386 system compiler Thread model: posix gcc version 3.4.5 20050809 (prerelease) [DragonFly] (propolice, visibility) # grep DISTNAME /usr/pkgsrc/wip/powerdns-recursor/Makefile DISTNAME= pdns-recursor-3.1.4 Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
your test doesnt work either: [elevator] ~/projects/mtasker% make c++ -I/usr/pkg/include -c mt.cc In file included from mtasker.hh:114, from mt.cc:2: mtasker.cc: In member function `void MTasker::yield()': mtasker.cc:51: error: there are no arguments to `swapcontext' that depend on a template parameter, so a declaration of `swapcontext' must be available mtasker.cc:51: error: (if you use `-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated) mtasker.cc: In member function `int MTaskerEventVal>::sendEvent(const EventKey&, const EventVal*)': mtasker.cc:78: error: there are no arguments to `swapcontext' that depend on a template parameter, so a declaration of `swapcontext' must be available mtasker.cc: In member function `void MTaskerEventVal>::makeThread(void (*)(void*), void*, const std::string&)': mtasker.cc:94: error: there are no arguments to `getcontext' that depend on a template parameter, so a declaration of `getcontext' must be available mtasker.cc: In member function `bool MTaskerEventVal>::schedule()': mtasker.cc:125: error: there are no arguments to `swapcontext' that depend on a template parameter, so a declaration of `swapcontext' must be available mt.cc: In function `void printer(void*)': mt.cc:10: warning: cast from pointer to integer of different size mtasker.cc: In member function `void MTaskerEventVal>::yield() [with EventKey = int, EventVal = int]': mt.cc:13: instantiated from here mtasker.cc:51: error: `swapcontext' undeclared (first use this function) mtasker.cc:51: error: (Each undeclared identifier is reported only once for each function it appears in.) mtasker.cc: In member function `void MTaskerEventVal>::makeThread(void (*)(void*), void*, const std::string&) [with EventKey = int, EventVal = int]': mt.cc:20: instantiated from here mtasker.cc:94: error: `getcontext' undeclared (first use this function) mt.cc:20: instantiated from here mtasker.cc:104: error: `makecontext' undeclared (first use this function) mtasker.cc: In member function `bool MTaskerEventVal>::schedule() [with EventKey = int, EventVal = int]': mt.cc:24: instantiated from here mtasker.cc:125: error: `swapcontext' undeclared (first use this function) *** Error code 1 Stop in /home/petr/projects/mtasker. I took the tarball of all kernel and libc changes/new files and unpacked it in /usr/src. However Ive only done quickworld. Was i meant to do buildworld Petr Matt EMatt Emmerton <[EMAIL PROTECTED]>mmerton wrote: The first cut of this work has been completed. I've ported over the FreeBSD i386 work, which is a hybrid libc + kernel syscall implementation. You can find the source changes and a tiny test program here: http://www.gsicomp.on.ca/~matt/dfly/ I hope to have a libc-only implementation completed before the end of the week. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Libc and kernel compiled, but wip/powerdns-recursor still gives me the mtasker errors. Petr Matt Emmerton wrote: The first cut of this work has been completed. I've ported over the FreeBSD i386 work, which is a hybrid libc + kernel syscall implementation. You can find the source changes and a tiny test program here: http://www.gsicomp.on.ca/~matt/dfly/ I hope to have a libc-only implementation completed before the end of the week. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
The first cut of this work has been completed. I've ported over the FreeBSD i386 work, which is a hybrid libc + kernel syscall implementation. You can find the source changes and a tiny test program here: http://www.gsicomp.on.ca/~matt/dfly/ I hope to have a libc-only implementation completed before the end of the week. -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Ok Matt, hope you can make it into the release. petr Matt Emmerton wrote: Petr, Given that I'm doing a fair amount of learning-as-I-go with this particular task, it would be unfair to charge my usual hourly rate. I am aware of your budget and I am willing to work within it, regardless of how long this task ultimately takes me. Regards, -- Matt Emmerton Hey Matt, Sweet as. How much do you charge per hour? Petr Matt Emmerton wrote: Noone? Im willing to pay up to $150 AUD ($110 USD, approx 80-90 euro) for those couple of functions + ensuring that PowerDNS-Recursor works after that. Please consider an estimate of how long its going to take and let me know(i suppose it might take only like 30 mins for some of you). Let me know your hourly charge. However keep in mind that i will always double check with other devs that im not being ripped off(not that im cheap, but i know you understand i dont do C thus i cant say how long its going to take...parents taught me to always be careful with money and all) I've started working on this, using the FreeBSD implementation as a first cut for DFly. It was asked earlier on in the thread why FreeBSD used syscalls for this -- it's best to look at the commit comments for rev 1.9 of ucontext.h for their reasons -- http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/include/ucontext.h I'll be following the FreeBSD lead for the first cut; I'll see what we can do to improve on this in our world as a follow-on. Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
Petr, Given that I'm doing a fair amount of learning-as-I-go with this particular task, it would be unfair to charge my usual hourly rate. I am aware of your budget and I am willing to work within it, regardless of how long this task ultimately takes me. Regards, -- Matt Emmerton > Hey Matt, > Sweet as. How much do you charge per hour? > > Petr > > Matt Emmerton wrote: > >> Noone? Im willing to pay up to $150 AUD ($110 USD, approx 80-90 euro) > >> for those couple of functions + ensuring that PowerDNS-Recursor works > >> after that. Please consider an estimate of how long its going to take > >> and let me know(i suppose it might take only like 30 mins for some of > >> you). Let me know your hourly charge. However keep in mind that i will > >> always double check with other devs that im not being ripped off(not > >> that im cheap, but i know you understand i dont do C thus i cant say how > >> long its going to take...parents taught me to always be careful with > >> money and all) > >> > > > > I've started working on this, using the FreeBSD implementation as a first > > cut for DFly. > > > > It was asked earlier on in the thread why FreeBSD used syscalls for this -- > > it's best to look at the commit comments for rev 1.9 of ucontext.h for their > > reasons -- > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/include/ucontext.h > > > > I'll be following the FreeBSD lead for the first cut; I'll see what we can > > do to improve on this in our world as a follow-on. > > > > Regards, > > -- > > Matt Emmerton > > > > >
Re: Request for swapcontext and getcontext to be ported to our libc
Hey Matt, Sweet as. How much do you charge per hour? Petr Matt Emmerton wrote: Noone? Im willing to pay up to $150 AUD ($110 USD, approx 80-90 euro) for those couple of functions + ensuring that PowerDNS-Recursor works after that. Please consider an estimate of how long its going to take and let me know(i suppose it might take only like 30 mins for some of you). Let me know your hourly charge. However keep in mind that i will always double check with other devs that im not being ripped off(not that im cheap, but i know you understand i dont do C thus i cant say how long its going to take...parents taught me to always be careful with money and all) I've started working on this, using the FreeBSD implementation as a first cut for DFly. It was asked earlier on in the thread why FreeBSD used syscalls for this -- it's best to look at the commit comments for rev 1.9 of ucontext.h for their reasons -- http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/include/ucontext.h I'll be following the FreeBSD lead for the first cut; I'll see what we can do to improve on this in our world as a follow-on. Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
> Noone? Im willing to pay up to $150 AUD ($110 USD, approx 80-90 euro) > for those couple of functions + ensuring that PowerDNS-Recursor works > after that. Please consider an estimate of how long its going to take > and let me know(i suppose it might take only like 30 mins for some of > you). Let me know your hourly charge. However keep in mind that i will > always double check with other devs that im not being ripped off(not > that im cheap, but i know you understand i dont do C thus i cant say how > long its going to take...parents taught me to always be careful with > money and all) I've started working on this, using the FreeBSD implementation as a first cut for DFly. It was asked earlier on in the thread why FreeBSD used syscalls for this -- it's best to look at the commit comments for rev 1.9 of ucontext.h for their reasons -- http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/include/ucontext.h I'll be following the FreeBSD lead for the first cut; I'll see what we can do to improve on this in our world as a follow-on. Regards, -- Matt Emmerton
Re: Request for swapcontext and getcontext to be ported to our libc
On Tue, Dec 26, 2006 at 02:38:48PM +0100, Simon 'corecode' Schubert wrote: > Petr Janda wrote: > >I'm being serious. > > Nobody doubts that you are. Just give the people some time to look into > the matter and have some christmas holidays in between. Don't worry, I'm > sure somebody will come back to you. Right. I looked into FBSD code a little. (I am not a kernel GURU, just looking) Puzzles me how this could be ported into some "middle layer", not being part of the "no more monolithic" kernel. The middle layer code would have to mask signals coming from the kernel (sounds not too hard) and switch contexts in an MP_SAFE manner. (locking?) Is there some sort of preemption needed? Can this fit into "middle layer"? Just asking. Trying to understand DFly concepts. Armin -- PUBBOX Postmaster + spam-killer. Free email addresses at http://pubbox.net/
Re: Request for swapcontext and getcontext to be ported to our libc
Petr Janda wrote: I'm being serious. Nobody doubts that you are. Just give the people some time to look into the matter and have some christmas holidays in between. Don't worry, I'm sure somebody will come back to you. cheers simon -- Serve - BSD +++ RENT this banner advert +++ASCII Ribbon /"\ Work - Mac +++ space for low €€€ NOW!1 +++ Campaign \ / Party Enjoy Relax | http://dragonflybsd.org Against HTML \ Dude 2c 2 the max ! http://golden-apple.biz Mail + News / \ signature.asc Description: OpenPGP digital signature
Re: Request for swapcontext and getcontext to be ported to our libc
Noone? Im willing to pay up to $150 AUD ($110 USD, approx 80-90 euro) for those couple of functions + ensuring that PowerDNS-Recursor works after that. Please consider an estimate of how long its going to take and let me know(i suppose it might take only like 30 mins for some of you). Let me know your hourly charge. However keep in mind that i will always double check with other devs that im not being ripped off(not that im cheap, but i know you understand i dont do C thus i cant say how long its going to take...parents taught me to always be careful with money and all) I'm being serious. Cheers, Petr Petr Janda wrote: This is an offer to anyone by the way... Petr Janda wrote: How much would you like me to pay you? Honest, give me estimate on how long this is going to take. If price is good, id also hire someone to add NSS support. Cheers, Petr Simon 'corecode' Schubert wrote: Petr Janda wrote: Hi, Could someone please port these from FreeBSD or NetBSD? Without them it seem is impossible to install powerdns-recusor. Can anyone do it in their spare time please? As I keep saying, open source usually doesn't work this way. This is how I see open source working: (a) you need something, you do it (b) you need something, you nag about it (c) you need something, you offer to pay for it (code bounty), somebody takes it and does it for you so, what we can see, (b) doesn't help if nobody matches (a) nor (c). of course you could have success by doing (b) so long until somebody either shows enough mercy and does it or gets interested himself and goes to (a). so if you *really* want something, (a) and (c) are the fastest ways. if your abilities do not match (a), you have to match (c). it's simple as that. people are volunteering and (as you observed correctly) doing stuff in their spare time. of course this leads to the result that people only actually do what they like to do and this does not necessarily match what *other* people would like them to do. offering money can overcome this problem. i think this is a fair model where everybody can contribute, either by writing code ((a)) or by supporting somebody who writes code ((c)). cheers simon ps: i think *context does not have to be a kernel function like in freebsd, but i'd like to hear more opinions on that
Re: Request for swapcontext and getcontext to be ported to our libc
This is an offer to anyone by the way... Petr Janda wrote: How much would you like me to pay you? Honest, give me estimate on how long this is going to take. If price is good, id also hire someone to add NSS support. Cheers, Petr Simon 'corecode' Schubert wrote: Petr Janda wrote: Hi, Could someone please port these from FreeBSD or NetBSD? Without them it seem is impossible to install powerdns-recusor. Can anyone do it in their spare time please? As I keep saying, open source usually doesn't work this way. This is how I see open source working: (a) you need something, you do it (b) you need something, you nag about it (c) you need something, you offer to pay for it (code bounty), somebody takes it and does it for you so, what we can see, (b) doesn't help if nobody matches (a) nor (c). of course you could have success by doing (b) so long until somebody either shows enough mercy and does it or gets interested himself and goes to (a). so if you *really* want something, (a) and (c) are the fastest ways. if your abilities do not match (a), you have to match (c). it's simple as that. people are volunteering and (as you observed correctly) doing stuff in their spare time. of course this leads to the result that people only actually do what they like to do and this does not necessarily match what *other* people would like them to do. offering money can overcome this problem. i think this is a fair model where everybody can contribute, either by writing code ((a)) or by supporting somebody who writes code ((c)). cheers simon ps: i think *context does not have to be a kernel function like in freebsd, but i'd like to hear more opinions on that
Re: Request for swapcontext and getcontext to be ported to our libc
How much would you like me to pay you? Honest, give me estimate on how long this is going to take. If price is good, id also hire someone to add NSS support. Cheers, Petr Simon 'corecode' Schubert wrote: Petr Janda wrote: Hi, Could someone please port these from FreeBSD or NetBSD? Without them it seem is impossible to install powerdns-recusor. Can anyone do it in their spare time please? As I keep saying, open source usually doesn't work this way. This is how I see open source working: (a) you need something, you do it (b) you need something, you nag about it (c) you need something, you offer to pay for it (code bounty), somebody takes it and does it for you so, what we can see, (b) doesn't help if nobody matches (a) nor (c). of course you could have success by doing (b) so long until somebody either shows enough mercy and does it or gets interested himself and goes to (a). so if you *really* want something, (a) and (c) are the fastest ways. if your abilities do not match (a), you have to match (c). it's simple as that. people are volunteering and (as you observed correctly) doing stuff in their spare time. of course this leads to the result that people only actually do what they like to do and this does not necessarily match what *other* people would like them to do. offering money can overcome this problem. i think this is a fair model where everybody can contribute, either by writing code ((a)) or by supporting somebody who writes code ((c)). cheers simon ps: i think *context does not have to be a kernel function like in freebsd, but i'd like to hear more opinions on that
Re: Request for swapcontext and getcontext to be ported to our libc
Petr Janda wrote: Hi, Could someone please port these from FreeBSD or NetBSD? Without them it seem is impossible to install powerdns-recusor. Can anyone do it in their spare time please? As I keep saying, open source usually doesn't work this way. This is how I see open source working: (a) you need something, you do it (b) you need something, you nag about it (c) you need something, you offer to pay for it (code bounty), somebody takes it and does it for you so, what we can see, (b) doesn't help if nobody matches (a) nor (c). of course you could have success by doing (b) so long until somebody either shows enough mercy and does it or gets interested himself and goes to (a). so if you *really* want something, (a) and (c) are the fastest ways. if your abilities do not match (a), you have to match (c). it's simple as that. people are volunteering and (as you observed correctly) doing stuff in their spare time. of course this leads to the result that people only actually do what they like to do and this does not necessarily match what *other* people would like them to do. offering money can overcome this problem. i think this is a fair model where everybody can contribute, either by writing code ((a)) or by supporting somebody who writes code ((c)). cheers simon ps: i think *context does not have to be a kernel function like in freebsd, but i'd like to hear more opinions on that
Request for swapcontext and getcontext to be ported to our libc
Hi, Could someone please port these from FreeBSD or NetBSD? Without them it seem is impossible to install powerdns-recusor. Can anyone do it in their spare time please? Thank you, Petr
