Re: [9fans] syscall silently kill processes
В Вт, 21/06/2022 в 11:26 +, adr пишет:
> On Tue, 21 Jun 2022, andrey100100...@gmail.com wrote:
> > ? ??, 20/06/2022 ? 15:29 -0700, Skip Tavakkolian ?:
> > > It's cleaner to use channels with separate io and timer threads
> > > that
> > > do their syscalls via ioproc; this one doesn't require any
> > > changes to
> > > libthread:
> > >
> > > https://gist.github.com/9nut/aaa9b9b6a22d69996b75ccdc6e615c61
> >
> > Thanks for the work you've done!
> > Yes, I have considered this possibility.
> > But it was precisely this kind of code bloat that I wanted to
> > avoid.
>
> It looks like code bloat, but it really isn't. It is doing the job
> with the tools of the api according to the paradigm designed in
> libthread. That's why the word "cleaner" is completely correct.
And one more problem:
cpu% 6.out -a 'udp!52.43.121.77!10011' -n 250 -t 7 | grep time | wc -l
6.out 50974: warning: process exceeds 100 file descriptors
6.out 51053: warning: process exceeds 200 file descriptors
6.out 51238: warning: process exceeds 300 file descriptors
6.out 51314: warning: process exceeds 400 file descriptors
6.out 51342: warning: process exceeds 500 file descriptors
6.out 51414: warning: process exceeds 600 file descriptors
6.out 51158: warning: process exceeds 700 file descriptors
250
cpu% ps | grep 6.out | wc -l
751
Сonsumes three times more resources than it should.
>
> I think note.c was added to resolve some particual case, and for
> the state of note.c, I don't think it has been used too much.
>
> For example, let's remove note.c. You could obtain the same result
> in your example (all processes using the same handler) using atnotify
> because the notes are registered to the children when proccreate
> uses rfork:
>
> void
> threadmain(int argc, char *argv[])
> {
> atnotify(handler_alarm, 1);
>
> ./5.out | grep end | wc -l
> 80
>
> If you have to use a different handler for each processes you can't
> use atnotify because of RFMEM, but you can use the syscalls notify
> and noted:
>
> #include
> #include
> #include
>
> static void
> handler_alarm(void *, char *msg)
> {
> if(strstr(msg, "alarm")){
> print("yes");
> noted(NCONT);
> return; /* just in case */
> }
> noted(NDFLT);
> }
>
> static void
> proc_udp(void *)
> {
> char resp[512];
> char req[] = "request";
> int fd;
> notify(handler_alarm);
> if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
> 0){
> if(write(fd, req, strlen(req)) == strlen(req)){
> fprint(1, "start\n");
> alarm(2000);
> read(fd, resp, sizeof(resp));
> alarm(0);
> fprint(1, "end\n");
> }
> close(fd);
> }
> threadexits(nil);
> }
>
> int mainstacksize = 5242880;
>
> void
> threadmain(int argc, char *argv[])
> {
> for(int i = 0; i < 80; i++)
> proccreate(proc_udp, nil, 10240);
> sleep(5000);
> threadexitsall(nil);
> }
>
> ./5.out | grep end | wc -l
> 80
>
> Threadnotify is trying to do an atnotify that works with RFMEM,
> but to do that onnote should be allocated to grow or shrink (or
> have a size thinking in the maximum number of processes the program
> could spawn, not the number of handlers a process could register
> as in atnotify), instead of pointers to handlers, it should be an
> array of pointers to arrays of handlers allocated by each process.
>
> Now again, does the notes mechanism actually fit in libthread? If
> it does it should be fixed, if not removed.
>
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M813e896da7e1be8ab8338198
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Tue, 21 Jun 2022, andrey100100...@gmail.com wrote:
For example, let's remove note.c. You could obtain the same result
Just for clarity, you actually don't need to remove note.c to do
what I said below.
in your example (all processes using the same handler) using atnotify
because the notes are registered to the children when proccreate
uses rfork:
void
threadmain(int argc, char *argv[])
{
atnotify(handler_alarm, 1);
./5.out | grep end | wc -l
80
If you have to use a different handler for each processes you can't
use atnotify because of RFMEM, but you can use the syscalls notify
and noted:
#include
#include
#include
static void
handler_alarm(void *, char *msg)
{
if(strstr(msg, "alarm")){
print("yes");
noted(NCONT);
return; /* just in case */
}
noted(NDFLT);
}
static void
proc_udp(void *)
{
char resp[512];
char req[] = "request";
int fd;
notify(handler_alarm);
if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
0){
if(write(fd, req, strlen(req)) == strlen(req)){
fprint(1, "start\n");
alarm(2000);
read(fd, resp, sizeof(resp));
alarm(0);
fprint(1, "end\n");
}
close(fd);
}
threadexits(nil);
}
int mainstacksize = 5242880;
void
threadmain(int argc, char *argv[])
{
for(int i = 0; i < 80; i++)
proccreate(proc_udp, nil, 10240);
sleep(5000);
threadexitsall(nil);
}
./5.out | grep end | wc -l
80
Threadnotify is trying to do an atnotify that works with RFMEM,
but to do that onnote should be allocated to grow or shrink (or
have a size thinking in the maximum number of processes the program
could spawn, not the number of handlers a process could register
as in atnotify), instead of pointers to handlers, it should be an
array of pointers to arrays of handlers allocated by each process.
Now again, does the notes mechanism actually fit in libthread? If
it does it should be fixed, if not removed.
I vote for the fix.
Perhaps the notification is being used somewhere or by someone.
adr.
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M198ebde18601eb82a1a5b8d8
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вт, 21/06/2022 в 11:26 +, adr пишет:
> On Tue, 21 Jun 2022, andrey100100...@gmail.com wrote:
> > ? ??, 20/06/2022 ? 15:29 -0700, Skip Tavakkolian ?:
> > > It's cleaner to use channels with separate io and timer threads
> > > that
> > > do their syscalls via ioproc; this one doesn't require any
> > > changes to
> > > libthread:
> > >
> > > https://gist.github.com/9nut/aaa9b9b6a22d69996b75ccdc6e615c61
> >
> > Thanks for the work you've done!
> > Yes, I have considered this possibility.
> > But it was precisely this kind of code bloat that I wanted to
> > avoid.
>
> It looks like code bloat, but it really isn't. It is doing the job
> with the tools of the api according to the paradigm designed in
> libthread. That's why the word "cleaner" is completely correct.
Yes, ready to agree, this solution is more independent of low-level
features and more obvious.
But, i would like more compact code.
I wonder how threadnotify() in plan9port will behave...
>
> I think note.c was added to resolve some particual case, and for
> the state of note.c, I don't think it has been used too much.
>
> For example, let's remove note.c. You could obtain the same result
> in your example (all processes using the same handler) using atnotify
> because the notes are registered to the children when proccreate
> uses rfork:
>
> void
> threadmain(int argc, char *argv[])
> {
> atnotify(handler_alarm, 1);
>
> ./5.out | grep end | wc -l
> 80
>
> If you have to use a different handler for each processes you can't
> use atnotify because of RFMEM, but you can use the syscalls notify
> and noted:
>
> #include
> #include
> #include
>
> static void
> handler_alarm(void *, char *msg)
> {
> if(strstr(msg, "alarm")){
> print("yes");
> noted(NCONT);
> return; /* just in case */
> }
> noted(NDFLT);
> }
>
> static void
> proc_udp(void *)
> {
> char resp[512];
> char req[] = "request";
> int fd;
> notify(handler_alarm);
> if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
> 0){
> if(write(fd, req, strlen(req)) == strlen(req)){
> fprint(1, "start\n");
> alarm(2000);
> read(fd, resp, sizeof(resp));
> alarm(0);
> fprint(1, "end\n");
> }
> close(fd);
> }
> threadexits(nil);
> }
>
> int mainstacksize = 5242880;
>
> void
> threadmain(int argc, char *argv[])
> {
> for(int i = 0; i < 80; i++)
> proccreate(proc_udp, nil, 10240);
> sleep(5000);
> threadexitsall(nil);
> }
>
> ./5.out | grep end | wc -l
> 80
>
> Threadnotify is trying to do an atnotify that works with RFMEM,
> but to do that onnote should be allocated to grow or shrink (or
> have a size thinking in the maximum number of processes the program
> could spawn, not the number of handlers a process could register
> as in atnotify), instead of pointers to handlers, it should be an
> array of pointers to arrays of handlers allocated by each process.
>
> Now again, does the notes mechanism actually fit in libthread? If
> it does it should be fixed, if not removed.
I vote for the fix.
Perhaps the notification is being used somewhere or by someone.
>
> adr.
>
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mf8c7cfd50b4091a520d792e5
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Tue, 21 Jun 2022, andrey100100...@gmail.com wrote:
? ??, 20/06/2022 ? 15:29 -0700, Skip Tavakkolian ?:
It's cleaner to use channels with separate io and timer threads that
do their syscalls via ioproc; this one doesn't require any changes to
libthread:
https://gist.github.com/9nut/aaa9b9b6a22d69996b75ccdc6e615c61
Thanks for the work you've done!
Yes, I have considered this possibility.
But it was precisely this kind of code bloat that I wanted to avoid.
It looks like code bloat, but it really isn't. It is doing the job
with the tools of the api according to the paradigm designed in
libthread. That's why the word "cleaner" is completely correct.
I think note.c was added to resolve some particual case, and for
the state of note.c, I don't think it has been used too much.
For example, let's remove note.c. You could obtain the same result
in your example (all processes using the same handler) using atnotify
because the notes are registered to the children when proccreate
uses rfork:
void
threadmain(int argc, char *argv[])
{
atnotify(handler_alarm, 1);
./5.out | grep end | wc -l
80
If you have to use a different handler for each processes you can't
use atnotify because of RFMEM, but you can use the syscalls notify
and noted:
#include
#include
#include
static void
handler_alarm(void *, char *msg)
{
if(strstr(msg, "alarm")){
print("yes");
noted(NCONT);
return; /* just in case */
}
noted(NDFLT);
}
static void
proc_udp(void *)
{
char resp[512];
char req[] = "request";
int fd;
notify(handler_alarm);
if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >= 0){
if(write(fd, req, strlen(req)) == strlen(req)){
fprint(1, "start\n");
alarm(2000);
read(fd, resp, sizeof(resp));
alarm(0);
fprint(1, "end\n");
}
close(fd);
}
threadexits(nil);
}
int mainstacksize = 5242880;
void
threadmain(int argc, char *argv[])
{
for(int i = 0; i < 80; i++)
proccreate(proc_udp, nil, 10240);
sleep(5000);
threadexitsall(nil);
}
./5.out | grep end | wc -l
80
Threadnotify is trying to do an atnotify that works with RFMEM,
but to do that onnote should be allocated to grow or shrink (or
have a size thinking in the maximum number of processes the program
could spawn, not the number of handlers a process could register
as in atnotify), instead of pointers to handlers, it should be an
array of pointers to arrays of handlers allocated by each process.
Now again, does the notes mechanism actually fit in libthread? If
it does it should be fixed, if not removed.
adr.
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mf12e91abda54e653de320337
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Mon, 20 Jun 2022, Skip Tavakkolian wrote: It's cleaner to use channels with separate io and timer threads that do their syscalls via ioproc; this one doesn't require any changes to libthread: https://gist.github.com/9nut/aaa9b9b6a22d69996b75ccdc6e615c61 Nice example, but I strongly recomend changing libthread or removing notes support. Right now: If a process try to remove a handler which has been register for another process before, it will remove the handler for that process instead of its own. When a process exits its handlers are not cleared, so if the process doesn't call threadnotify to clean its handlers, this space is wasted for the rest of the program. Also it makes sense to clean them all at exit instead of doing it explicity for each handler. threadnotify will insert duplicated handlers, wasting the limited space in onnote[], onnotepid[]. Note that those duplicated handlers will be completly ignored, they will just occupy space. If you are using notes and all your processes are going to share the same array of handlers, doesn't make sense to be able to register handlers which could be used by all of them? Another discussion is if the notes mechanism really fits in libthread, as I said, maybe is better to remove notes support completly and force the programmer to exploit the api. Yours is a good example. Regards, adr. -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mc9df1d1014565db0ee016754 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Пн, 20/06/2022 в 15:29 -0700, Skip Tavakkolian пишет:
> It's cleaner to use channels with separate io and timer threads that
> do their syscalls via ioproc; this one doesn't require any changes to
> libthread:
>
> https://gist.github.com/9nut/aaa9b9b6a22d69996b75ccdc6e615c61
Thanks for the work you've done!
Yes, I have considered this possibility.
But it was precisely this kind of code bloat that I wanted to avoid.
And yet, the libthread needs fixing. What adr suggested, at first
glance, is acceptable.
And it turns out that we are trying to get around the problems of the
libthread using the libthread...
By not fixing the libthread, we reduce the number of use cases for it,
which is bad.
PS. Philosophical question: should someone monitor the process or
should the process serve itself?
>
>
> On Mon, Jun 20, 2022 at 8:57 AM wrote:
> >
> > В Пн, 20/06/2022 в 05:59 +, adr пишет:
> > > On Mon, 20 Jun 2022, adr wrote:
> > > > But I have something in mind for a case like
> > > > this, when all the processes are going to use the same handler
> > > > (that's why I was asking). Let me play with it a litle before I
> > > > share it.
> > >
> > > Ok, the idea is this: If in is bigger than zero in
> > > threadnotify(int (*f)(void*, char*), int in), the handler is
> > > register
> > > for the calling process. If in is 0, then the handler is cleared
> > > for the calling process. If in is -1, the handler is register for
> > > all processes and if in is less than -1, it is cleared for all
> > > processes (expect for those who have already registered it for
> > > themselves).
> > >
> > > Now back to your example, as all the processes are going to use
> > > the
> > > same handler,
> > > you just have to register it once in threadmain:
> > >
> > > #include
> > > #include
> > > #include
> > >
> > > static int
> > > handler_alarm(void *, char *msg)
> > > {
> > > if(strstr(msg, "alarm"))
> > > return 1;
> > > return 0;
> > > }
> > >
> > > static void
> > > proc_udp(void *)
> > > {
> > > char resp[512];
> > > char req[] = "request";
> > > int fd;
> > > if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil))
> > > 0){
> > > if(write(fd, req, strlen(req)) == strlen(req)){
> > > fprint(1, "start\n");
> > > alarm(2000);
> > > read(fd, resp, sizeof(resp));
> > > alarm(0);
> > > fprint(1, "end\n");
> > > }
> > > close(fd);
> > > }
> > > threadexits(nil);
> > > }
> > >
> > > int mainstacksize = 5242880;
> > >
> > > void
> > > threadmain(int argc, char *argv[])
> > > {
> > > threadnotify(handler_alarm, -1);
> > > for(int i = 0; i < 80; i++)
> > > proccreate(proc_udp, nil, 10240);
> > > sleep(5000);
> > > threadexitsall(nil);
> > > }
> > > Now,
> > > ; ./5.out | grep end | wc -l
> > > 80
> > >
> > > Are you happy Andrej?
> >
> >
> > Yes. Thank you very much! It's working!
> >
> > How convenient it is to use - more experiments are needed.
> >
> >
> > >
> > > adr.
> > >
> > > /sys/src/libthread/sched.c:
> > > [...]
> > > if(t == nil){
> > > _threaddebug(DBGSCHED, "all threads gone;
> > > exiting");
> > > cancelnotes(p->pid);
> > > _schedexit(p);
> > > }
> > > [...]
> > > /sys/src/libthread/note.c
> > > [...]
> > > int
> > > threadnotify(int (*f)(void*, char*), int in)
> > > {
> > > int i, frompid, topid;
> > > int (*from)(void*, char*), (*to)(void*, char*);
> > >
> > > if(in && in>-2){
> > > from = nil;
> > > frompid = 0;
> > > to = f;
> > > topid = (in == -1)? -1 : _threadgetproc()->pid;
> > > lock();
> > > for(i=0; i > > if(onnote[i]==to && onnotepid[i]==topid){
> > > unlock();
> > > return i > > }
> > > unlock();
> > > }else{
> > > from = f;
> > > frompid = (in < -1)? -1 : _threadgetproc()->pid;
> > > to = nil;
> > > topid = 0;
> > > }
> > > lock();
> > > for(i=0; i > > if(onnote[i]==from && onnotepid[i]==frompid){
> > > onnote[i] = to;
> > > onnotepid[i] = topid;
> > > break;
> > > }
> > > unlock();
> > > return i > > }
> > >
> > > void
> > > cancelnotes(int pid)
> > > {
> > > int i;
> > >
> > > lock();
> > > for(i=0; i > > if(onnotepid[i] == pid){
> > > onnote[i] = nil;
> > >
