Re: [9fans] syscall silently kill processes (fwd)
The last mail got screwed, I'm resending this for clarity.
/sys/src/libthread/sched.c:
[...]
if(t == nil){
_threaddebug(DBGSCHED, "all threads gone; exiting");
cancelnotes(p->pid);
_schedexit(p);
}
[...]
/sys/src/libthread/note.c
[...]
int
threadnotify(int (*f)(void*, char*), int in)
{
int i, frompid, topid;
int (*from)(void*, char*), (*to)(void*, char*);
if(in && in>-2){
from = nil;
frompid = 0;
to = f;
topid = (in == -1)? -1 : _threadgetproc()->pid;
lock(&onnotelock);
for(i=0; ipid;
to = nil;
topid = 0;
}
lock(&onnotelock);
for(i=0; ipending)
return;
p->pending = 0;
for(n=notes; nproc == p){
for(i=0; ipid && onnotepid[i]!=-1) ||
(fn = onnote[i])==nil)
continue;
if((*fn)(v, n->s))
break;
[...]
/sys/include/thread.h
[...]
void cancelnotes(int pid);
[...]
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/T5a58cfad5bc60d5c-Mcbab164c50af064e722c9315
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Mon, 20 Jun 2022, adr wrote:
But I have something in mind for a case like
this, when all the processes are going to use the same handler
(that's why I was asking). Let me play with it a litle before I
share it.
Ok, the idea is this: If in is bigger than zero in
threadnotify(int (*f)(void*, char*), int in), the handler is register
for the calling process. If in is 0, then the handler is cleared
for the calling process. If in is -1, the handler is register for
all processes and if in is less than -1, it is cleared for all
processes (expect for those who have already registered it for themselves).
Now back to your example, as all the processes are going to use the same
handler,
you just have to register it once in threadmain:
#include
#include
#include
static int
handler_alarm(void *, char *msg)
{
if(strstr(msg, "alarm"))
return 1;
return 0;
}
static void
proc_udp(void *)
{
char resp[512];
char req[] = "request";
int fd;
if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >= 0){
if(write(fd, req, strlen(req)) == strlen(req)){
fprint(1, "start\n");
alarm(2000);
read(fd, resp, sizeof(resp));
alarm(0);
fprint(1, "end\n");
}
close(fd);
}
threadexits(nil);
}
int mainstacksize = 5242880;
void
threadmain(int argc, char *argv[])
{
threadnotify(handler_alarm, -1);
for(int i = 0; i < 80; i++)
proccreate(proc_udp, nil, 10240);
sleep(5000);
threadexitsall(nil);
}
Now,
; ./5.out | grep end | wc -l
80
Are you happy Andrej?
adr.
/sys/src/libthread/sched.c:
[...]
if(t == nil){
_threaddebug(DBGSCHED, "all threads gone; exiting");
cancelnotes(p->pid);
_schedexit(p);
}
[...]
/sys/src/libthread/note.c
[...]
int
threadnotify(int (*f)(void*, char*), int in)
{
int i, frompid, topid;
int (*from)(void*, char*), (*to)(void*, char*);
if(in && in>-2){
from = nil;
frompid = 0;
to = f;
topid = (in == -1)? -1 : _threadgetproc()->pid;
lock(&onnotelock);
for(i=0; ipid;
to = nil;
topid = 0;
}
lock(&onnotelock);
for(i=0; ipending)
return;
p->pending = 0;
for(n=notes; nproc == p){
for(i=0; ipid && onnotepid[i]!=-1) ||
(fn = onnote[i])==nil)
continue;
if((*fn)(v, n->s))
break;
[...]
/sys/include/thread.h
[...]
void cancelnotes(int pid);
[...]
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Meda7cacd5cce9cc74c1ddaf8
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Пн, 20/06/2022 в 04:41 +, adr пишет: > On Mon, 20 Jun 2022, andrey100100...@gmail.com wrote: > > The note 'alarm' is needed to interrupt the system call on timeout > > since system calls to lan 9 can be of a network nature, a notes is > > indispensable. > > A great example of this is the read() system call on a udp- > > connection. > > How else can this system call be interrupted? > > Start two processes, one which just make the call, another as a > timer to kill it. Yes, I had such an idea, but in the thread library, process IDs increase monotonically, which is bad (due to IDs overflow) for long- lived and actively spawning programs. On the other hand, with process IDs, as in the kernel (reuse of IDs), it is also required to receive group notes in case of an unexpected termination of the child. I.e. the problem is in identifying the processes to kill. > But I have something in mind for a case like > this, when all the processes are going to use the same handler > (that's why I was asking). It would be great if could undo handler (delete from onnote or something similar). > Let me play with it a litle before I > share it. > > adr. > Regards, Andrej -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M6bc161e383ef6a388d7c5b0a Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Mon, 20 Jun 2022, andrey100100...@gmail.com wrote: The note 'alarm' is needed to interrupt the system call on timeout since system calls to lan 9 can be of a network nature, a notes is indispensable. A great example of this is the read() system call on a udp-connection. How else can this system call be interrupted? Start two processes, one which just make the call, another as a timer to kill it. But I have something in mind for a case like this, when all the processes are going to use the same handler (that's why I was asking). Let me play with it a litle before I share it. adr. -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mc017bb6a54415db715338ff2 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Пн, 20/06/2022 в 00:22 +0300, andrey100100...@gmail.com пишет: > В Вс, 19/06/2022 в 16:41 +, adr пишет: > > On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote: > > > Yes, you were absolutely right, the thread library needs some > > > work. > > > > > > It is impossible to use multiple processes with notes, due to the > > > exhaustion of the NFN limit. > > > > Andrej, what are you going to do with alarm in the real thing? > > The note 'alarm' is needed to interrupt the system call on timeout > since system calls to lan 9 can be of a network nature, a notes is > indispensable. > A great example of this is the read() system call on a udp- > connection. > How else can this system call be interrupted? > > > > > You could use threads (cooperative round-ribbon multitasking) using > > some data log to register their state and use a "master" thread to > > control them (kill them, change some data structure, etc). > > > > You can use rfork, locks, pipes, etc and forget about libthread. Channels are a good abstraction for exchanging data between processes. Even if not to use the libthread, I would have to (probably) write it. But yeah, maybe it's easier to do without it. > > > > You could experiment with the topology of the nodes, for example > > instead of a big star you can simplify things a lot using chains > > of nodes where the last node sends the chain data to the analyzer > > (you were talking about polling sensors): > > > > aNode1 --> aNode2 --> aNode3 --> ... --> aNoden --> > > > > > > > --- > Anz > > bNode1 --> bNode2 --> bNode3 --> ... --> bNoden --> > > > > Imagine that n=100. Each node only has to make a connection with > > 2 nodes (the first of the chain just one), adding data to the > > received one and send it to the next. Anz only has to make a > > connection with the last nodes of the chains, two in this case, > > and process the data received. Of course you have to play with your > > numbers, the acceptable delay, etc. > > > > I'm pretty sure people here could point you to some examples using > > plan9, and note that you could use raspberry pi zeros as nodes. > > > > Have fun. > > Regards, Andrej -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M06156a275c41ac100d18311b Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вс, 19/06/2022 в 16:41 +, adr пишет: > On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote: > > Yes, you were absolutely right, the thread library needs some work. > > > > It is impossible to use multiple processes with notes, due to the > > exhaustion of the NFN limit. > > Andrej, what are you going to do with alarm in the real thing? The note 'alarm' is needed to interrupt the system call on timeout since system calls to lan 9 can be of a network nature, a notes is indispensable. A great example of this is the read() system call on a udp-connection. How else can this system call be interrupted? > You could use threads (cooperative round-ribbon multitasking) using > some data log to register their state and use a "master" thread to > control them (kill them, change some data structure, etc). > > You can use rfork, locks, pipes, etc and forget about libthread. > > You could experiment with the topology of the nodes, for example > instead of a big star you can simplify things a lot using chains > of nodes where the last node sends the chain data to the analyzer > (you were talking about polling sensors): > > aNode1 --> aNode2 --> aNode3 --> ... --> aNoden --> > > > --- > Anz > bNode1 --> bNode2 --> bNode3 --> ... --> bNoden --> > > Imagine that n=100. Each node only has to make a connection with > 2 nodes (the first of the chain just one), adding data to the > received one and send it to the next. Anz only has to make a > connection with the last nodes of the chains, two in this case, > and process the data received. Of course you have to play with your > numbers, the acceptable delay, etc. > > I'm pretty sure people here could point you to some examples using > plan9, and note that you could use raspberry pi zeros as nodes. > > Have fun. > > -- > 9fans: 9fans > Permalink: > https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Md1b869bf61deccadf9733908 > Delivery options: > https://9fans.topicbox.com/groups/9fans/subscription -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M24ba5edacde926075a431016 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote: Yes, you were absolutely right, the thread library needs some work. It is impossible to use multiple processes with notes, due to the exhaustion of the NFN limit. Andrej, what are you going to do with alarm in the real thing? You could use threads (cooperative round-ribbon multitasking) using some data log to register their state and use a "master" thread to control them (kill them, change some data structure, etc). You can use rfork, locks, pipes, etc and forget about libthread. You could experiment with the topology of the nodes, for example instead of a big star you can simplify things a lot using chains of nodes where the last node sends the chain data to the analyzer (you were talking about polling sensors): aNode1 --> aNode2 --> aNode3 --> ... --> aNoden --> > --- > Anz bNode1 --> bNode2 --> bNode3 --> ... --> bNoden --> Imagine that n=100. Each node only has to make a connection with 2 nodes (the first of the chain just one), adding data to the received one and send it to the next. Anz only has to make a connection with the last nodes of the chains, two in this case, and process the data received. Of course you have to play with your numbers, the acceptable delay, etc. I'm pretty sure people here could point you to some examples using plan9, and note that you could use raspberry pi zeros as nodes. Have fun. -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Md1b869bf61deccadf9733908 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вс, 19/06/2022 в 10:32 +, adr пишет:
> On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote:
> > No way. All processes must run simultaneously.
> > NFN limit cannot be bypassed.
>
> Yeah, that's why I said it was silly:
> > > > The solution is obvious, cancel the process' handlers before it
> > > > exits so we don't run out of space.
> > >
> > > This was really silly...
>
> The changes I'm testing are not for evading the limit, but for
> making the handler managment more efficient and specially to avoid
> the case when a process could remove another process' handler from
> onnote[].
>
Yes, you were absolutely right, the thread library needs some work.
It is impossible to use multiple processes with notes, due to the
exhaustion of the NFN limit.
as example:
-
#include
#include
#include
static int
handler_alarm(void *, char *msg)
{
if(strstr(msg, "alarm")){
return 1;
}
return 0;
}
static void
proc_func(void *c)
{
Channel *chan = c;
int fd;
char req[] = "request";
char resp[512], *r = nil;
if(threadnotify(handler_alarm, 1) == 0){
fprint(1, "handler not registred\n");
}
alarm(2000);
if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
0){
alarm(0);
alarm(2000);
if(write(fd, req, strlen(req)) == strlen(req)){
alarm(0);
alarm(2000);
if(read(fd, resp, sizeof(resp)) > 0){
alarm(0);
if((r = malloc(sizeof(resp))) == nil){
sysfatal("malloc error: %r");
}
memmove(r, resp, sizeof(resp));
}
}
close(fd);
}
alarm(0);
send(chan, r);
threadexits(nil);
}
int mainstacksize = 5242880;
void
threadmain(int argc, char *argv[])
{
Channel *chan = nil;
char *data = nil;
int nproc = 0;
ARGBEGIN{
case 'n':
nproc = atoi(EARGF(threadexitsall(nil)));
break;
default:
threadexitsall(nil);
}ARGEND;
if((chan = chancreate(sizeof(char *), 0)) == nil){
sysfatal("channel error: %r");
}
for(int j = 0; j < 10 ; j++){
for(int i = 0; i < nproc; i++){
proccreate(proc_func, chan, 10240);
}
for(int i = 0; i < nproc; i++){
if(data = recvp(chan)){
free(data);
}
}
fprint(1, "j: %d\n", j);
}
if(nproc)
fprint(1, "EXIT with nproc: %d\n", nproc);
threadexitsall(nil);
}
-
cpu% 6.out -n 10
j: 0
j: 1
j: 2
handler not registred
handler not registred
handler not registred
handler not registred
handler not registred
handler not registred
handler not registred
and stalled forever...
And yes, threadnotify(handler_alarm, 0) in proc_func
does not help.
I don't know how best to get out of this situation.
Probably will have to rewrite the program, so that it does not matter
how many processes have fallen. But it's an increase in complexity.
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M2fc7967213269fe01e89ac5c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вс, 19/06/2022 в 14:40 +0300, andrey100100...@gmail.com пишет:
> В Вс, 19/06/2022 в 10:32 +, adr пишет:
> > On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote:
> > > No way. All processes must run simultaneously.
> > > NFN limit cannot be bypassed.
> >
> > Yeah, that's why I said it was silly:
> > > > > The solution is obvious, cancel the process' handlers before
> > > > > it
> > > > > exits so we don't run out of space.
> > > >
> > > > This was really silly...
> >
> > The changes I'm testing are not for evading the limit, but for
> > making the handler managment more efficient and specially to avoid
> > the case when a process could remove another process' handler from
> > onnote[].
>
> Ok.
>
>
> More complete example with thread library:
>
> -
> #include
> #include
> #include
>
> static int
> handler_alarm(void *, char *msg)
> {
> if(strstr(msg, "alarm")){
> return 1;
> }
>
> return 0;
> }
>
> static void
> proc_func(void *c)
> {
> Channel *chan = c;
>
> int fd, resp_len;
> char req[] = "request";
> char resp[512], *r = nil;
>
> if(threadnotify(handler_alarm, 1) == 0){
> fprint(1, "handler not registred\n");
> }
>
> alarm(2000);
> if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
> 0){
> alarm(0);
> alarm(2000);
> if(write(fd, req, strlen(req)) == strlen(req)){
> alarm(0);
> alarm(2000);
> if((resp_len = read(fd, resp, sizeof(resp)))
> 0){
> alarm(0);
> if((r = malloc(sizeof(resp))) ==
> nil){
> sysfatal("malloc error: %r");
> }
> memmove(r, resp, sizeof(resp));
> }
> }
> close(fd);
> }
>
+ alarm(0);
> send(chan, r);
> threadexits(nil);
> }
>
> int mainstacksize = 5242880;
>
> void
> threadmain(int argc, char *argv[])
> {
> Channel *chan = nil;
> char *data = nil;
> int nproc = 0;
>
> ARGBEGIN{
> case 'n':
> nproc = atoi(EARGF(threadexitsall(nil)));
> break;
> default:
> threadexitsall(nil);
> }ARGEND;
>
> if((chan = chancreate(sizeof(char *), 0)) == nil){
> sysfatal("channel error: %r");
> }
>
> for(int i = 0; i < nproc; i++){
> proccreate(proc_func, chan, 10240);
> }
>
> for(int i = 0; i < nproc; i++){
> if(data = recvp(chan)){
> free(data);
> }
> }
>
> if(nproc)
> fprint(1, "EXIT with nproc: %d\n", nproc);
>
> threadexitsall(nil);
> }
> -
>
> cpu% 6.out -n 33
> EXIT with nproc: 33
>
> with 34:
>
> cpu% 6.out -n 34
> handler not registred
>
> and stalled.
>
>
> So it is important for me that all processes respond.
> Such use, it seems to me, simplifies the program.
>
>
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M1bf9fc766759c5c66a48278e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вс, 19/06/2022 в 10:32 +, adr пишет:
> On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote:
> > No way. All processes must run simultaneously.
> > NFN limit cannot be bypassed.
>
> Yeah, that's why I said it was silly:
> > > > The solution is obvious, cancel the process' handlers before it
> > > > exits so we don't run out of space.
> > >
> > > This was really silly...
>
> The changes I'm testing are not for evading the limit, but for
> making the handler managment more efficient and specially to avoid
> the case when a process could remove another process' handler from
> onnote[].
Ok.
More complete example with thread library:
-
#include
#include
#include
static int
handler_alarm(void *, char *msg)
{
if(strstr(msg, "alarm")){
return 1;
}
return 0;
}
static void
proc_func(void *c)
{
Channel *chan = c;
int fd, resp_len;
char req[] = "request";
char resp[512], *r = nil;
if(threadnotify(handler_alarm, 1) == 0){
fprint(1, "handler not registred\n");
}
alarm(2000);
if((fd = dial("udp!185.157.221.201!5678", nil, nil, nil)) >=
0){
alarm(0);
alarm(2000);
if(write(fd, req, strlen(req)) == strlen(req)){
alarm(0);
alarm(2000);
if((resp_len = read(fd, resp, sizeof(resp))) >
0){
alarm(0);
if((r = malloc(sizeof(resp))) == nil){
sysfatal("malloc error: %r");
}
memmove(r, resp, sizeof(resp));
}
}
close(fd);
}
send(chan, r);
threadexits(nil);
}
int mainstacksize = 5242880;
void
threadmain(int argc, char *argv[])
{
Channel *chan = nil;
char *data = nil;
int nproc = 0;
ARGBEGIN{
case 'n':
nproc = atoi(EARGF(threadexitsall(nil)));
break;
default:
threadexitsall(nil);
}ARGEND;
if((chan = chancreate(sizeof(char *), 0)) == nil){
sysfatal("channel error: %r");
}
for(int i = 0; i < nproc; i++){
proccreate(proc_func, chan, 10240);
}
for(int i = 0; i < nproc; i++){
if(data = recvp(chan)){
free(data);
}
}
if(nproc)
fprint(1, "EXIT with nproc: %d\n", nproc);
threadexitsall(nil);
}
-
cpu% 6.out -n 33
EXIT with nproc: 33
with 34:
cpu% 6.out -n 34
handler not registred
and stalled.
So it is important for me that all processes respond.
Such use, it seems to me, simplifies the program.
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mefb1eb17df6e6f347f6c5bf9
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
On Sun, 19 Jun 2022, andrey100100...@gmail.com wrote: No way. All processes must run simultaneously. NFN limit cannot be bypassed. Yeah, that's why I said it was silly: The solution is obvious, cancel the process' handlers before it exits so we don't run out of space. This was really silly... The changes I'm testing are not for evading the limit, but for making the handler managment more efficient and specially to avoid the case when a process could remove another process' handler from onnote[]. adr. -- 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-M4d8837da259920e0095176e2 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
Re: [9fans] syscall silently kill processes
В Вс, 19/06/2022 в 05:01 +, adr пишет:
> On Sun, 19 Jun 2022, adr wrote:
> > The solution is obvious, cancel the process' handlers before it
> > exits so we don't run out of space.
>
> This was really silly...
>
> > Now, is there any reason to not do that in threadexits() when it
> > terminates the process?
> >
> > Shouldn't threadnotify() cancel only the process' handlers? We are
> > sharing onnote[NFN] and the code as it is right now removes the
> > first handler that match the pointer, it can belong to another
> > process.
>
> I ended up playing with this (do not register duplicated handlers,
> cancel only the notes of the thread's process and cancel all notes
> when the process exits):
>
> /sys/src/libthread/sched.c:
> [...]
> if(t == nil){
> _threaddebug(DBGSCHED, "all threads gone;
> exiting");
> cancelnotes(p->pid);
> _schedexit(p);
> }
> [...]
> /sys/src/libthread/note.c
> [...]
> int
> threadnotify(int (*f)(void*, char*), int in)
> {
> int i, frompid, topid;
> int (*from)(void*, char*), (*to)(void*, char*);
>
> if(in){
> from = nil;
> frompid = 0;
> to = f;
> topid = _threadgetproc()->pid;
> lock(&onnotelock);
> for(i=0; i if(onnote[i]==to && onnotepid[i]==topid){
> unlock(&onnotelock);
> return i }
> unlock(&onnotelock);
> }else{
> from = f;
> frompid = _threadgetproc()->pid;
> to = nil;
> topid = 0;
> }
> lock(&onnotelock);
> for(i=0; i if(onnote[i]==from && onnotepid[i]==frompid){
> onnote[i] = to;
> onnotepid[i] = topid;
> break;
> }
> unlock(&onnotelock);
> return i }
>
> void
> cancelnotes(int pid)
> {
> int i;
>
> lock(&onnotelock);
> for(i=0; i if(onnotepid[i] == pid){
> onnote[i] = nil;
> onnotepid[i] = 0;
> }
> unlock(&onnotelock);
> return;
> }
> /sys/include/thread.h
> [...]
> void cancelnotes(int pid);
> [...]
No way. All processes must run simultaneously.
NFN limit cannot be bypassed.
>
> Anyway, I would like to know a real example when it is useful to
> span a hundred processes using libthread without really exploiting
> threads at all. I mean, we have been streching things a little
> here!
>
In general, problem not in processes, threads or notes. Problem in
network nature. In the unreliable nature of network communication,
requiring timeouts, packet loss handling, retransmission, etc.
I'm trying to solve it using Plan 9.
In this particular case, I am trying to reduce the total polling time
of, for example, a sensor network by increasing the number of sensors
polled at the same time.
Ready to hear the best solution.
Regards,
Andrej
--
9fans: 9fans
Permalink:
https://9fans.topicbox.com/groups/9fans/Tfa6823048ad90a21-Mf178309eb46992e6940a5ea4
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
