Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew wrote: > 5-10% slowdown on fork is expected, but > why was exec slower? Thanks for the summary, Andrew. Guillaume (or anyone else tempted to do this) - it's a good idea, when posting 100 lines of data, to summarize with a line or two of words, as Andrew did here. It is far more efficient for one writer to do this, than each of a thousand readers. Hmmm ... so why was exec slower? -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <[EMAIL PROTECTED]> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-03-02 at 01:06 -0800, Andrew Morton wrote:
> Guillaume Thouvenin <[EMAIL PROTECTED]> wrote:
> >
> > So I ran the lmbench with three different kernels with the fork
> > connector patch I just sent. Results are attached at the end of the mail
> > and there are three different lines which are:
> >
> > o First line is a linux-2.6.11-rc4-mm1-cnfork
> > o Second line is a linux-2.6.11-rc4-mm1
> > o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space
> >application. The user space application listened during 15h
> >and received 6496 messages.
> >
> > Each test has been ran only once.
> >
> > ...
> >
> > --
> > Host OS Mhz null null open slct sig sig fork exec
> > sh
> > call I/O stat clos TCP inst hndl proc proc
> > proc
> > - -
> >
> > account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629.
> > 2415
> > account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628.
> > 2417
> > account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664.
> > 2456
>
> This is the interesting bit, yes? 5-10% slowdown on fork is expected, but
> why was exec slower?
I can't explain it for the moment. I will run test more than once to see
if this difference is still here.
> What does "The user space application listened during 15h" mean?
It means that I ran the user space application before the test and
stop it 15 hours later (this morning for me). The test ran during
5h30mn.
The user space application increments a counter to show how many
processes have been created during a period of time. I have not use the
user space daemon that manages group of processes because the it still
uses the old mechanism (a signal sends from the do_fork()) and as I
wanted to provide quick results, I used another user space application.
I attache the test program (get_fork_info.c) that I'm using at the end
of the mail to clearly show what it does.
I will run new tests with the real user space daemon but it will be
ready next week, sorry for the delay.
Best regards,
Guillaume
---
/*
* get_fork_info.c
*
* This program listens netlink interface to retreive information
* sends by the kernel when forking. It increments a counter for
* each forks and when the user hit CRL-C, it displays how many
* fork occured during the period.
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#define CN_FORK_OFF 0
#define CN_FORK_ON 1
#define MESSAGE_SIZE(sizeof(struct nlmsghdr) + \
sizeof(struct cn_msg) + \
sizeof(int))
int sock;
unsigned long total_p;
struct timeval test_time;
static inline void switch_cn_fork(int sock, int action)
{
char buff[128]; /* must be > MESSAGE_SIZE */
struct nlmsghdr *hdr;
struct cn_msg *msg;
/* Clear the buffer */
memset(buff, '\0', sizeof(buff));
/* fill the message header */
hdr = (struct nlmsghdr *) buff;
hdr->nlmsg_len = MESSAGE_SIZE;
hdr->nlmsg_type = NLMSG_DONE;
hdr->nlmsg_flags = 0;
hdr->nlmsg_seq = 0;
hdr->nlmsg_pid = getpid();
/* the message */
msg = (struct cn_msg *) NLMSG_DATA(hdr);
msg->id.idx = CN_IDX_FORK;
msg->id.val = CN_VAL_FORK;
msg->seq = 0;
msg->ack = 0;
msg->len = sizeof(int);
msg->data[0] = action;
send(sock, hdr, hdr->nlmsg_len, 0);
}
static void cleanup()
{
struct timeval tmp_time;
switch_cn_fork(sock, CN_FORK_OFF);
tmp_time = test_time;
gettimeofday(_time, NULL);
printf("%lu processes were created in %li seconds.\n",
total_p, test_time.tv_sec - tmp_time.tv_sec);
close(sock);
exit(EXIT_SUCCESS);
}
int main()
{
int err;
struct sockaddr_nl sa; /* information for NETLINK interface */
/*
* To be able to quit the application properly we install a
* signal handler that catch the CTRL-C
*/
signal(SIGTERM, cleanup);
signal(SIGINT, cleanup);
/*
* Create an endpoint for communication. Use the kernel user
* interface device (PF_NETLINK) which is a datagram oriented
* service (SOCK_DGRAM). The protocol used is the netfilter/iptables
* ULOG protocol (NETLINK_NFLOG)
*/
sock = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_NFLOG);
if (sock == -1) {
perror("socket");
return -1;
}
sa.nl_family = AF_NETLINK;
sa.nl_groups = CN_IDX_FORK;
sa.nl_pid = getpid();
err =
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > So I ran the lmbench with three different kernels with the fork > connector patch I just sent. Results are attached at the end of the mail > and there are three different lines which are: > > o First line is a linux-2.6.11-rc4-mm1-cnfork > o Second line is a linux-2.6.11-rc4-mm1 > o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space >application. The user space application listened during 15h >and received 6496 messages. > > Each test has been ran only once. > > ... > > -- > Host OS Mhz null null open slct sig sig fork exec sh > > call I/O stat clos TCP inst hndl proc proc > proc > - - > > account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629. > 2415 > account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628. > 2417 > account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664. > 2456 This is the interesting bit, yes? 5-10% slowdown on fork is expected, but why was exec slower? What does "The user space application listened during 15h" mean? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote: > > I tested without user space listeners and the cost is negligible. I will > > test with a user space listeners and see the results. I'm going to run > > the test this week after improving the mechanism that switch on/off the > > sending of the message. > > I'm also trying to mesure the process-creation/destruction performance on > following three environment. > Archtechture: i686 / Distribution: Fedora Core 3 > * Kernel Preemption is DISABLE > * SMP kernel but UP-machine / Not Hyper Threading > [1] 2.6.11-rc4-mm1 normal > [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module > [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled) > > When 367th-fork() was called after fork-connector notification, kernel was > locked up. > (User-Space-Listener has been also run until 366th-fork() notification was > received) So I ran the lmbench with three different kernels with the fork connector patch I just sent. Results are attached at the end of the mail and there are three different lines which are: o First line is a linux-2.6.11-rc4-mm1-cnfork o Second line is a linux-2.6.11-rc4-mm1 o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space application. The user space application listened during 15h and received 6496 messages. Each test has been ran only once. Best regards, Guillaume --- cd results && make summary percent 2>/dev/null | more make[1]: Entering directory `/home/guill/benchmark/lmbench/lmbench-3.0-a4/results' L M B E N C H 3 . 0 S U M M A R Y (Alpha software, do not distribute) Basic system parameters -- Host OS Description Mhz tlb cache mem scal pages line par load bytes - - --- - - -- account Linux 2.6.11- i686-pc-linux-gnu 276563 128 2.49001 account Linux 2.6.11- i686-pc-linux-gnu 276567 128 2.42001 account Linux 2.6.11- i686-pc-linux-gnu 276569 128 2.44001 Processor, Processes - times in microseconds - smaller is better -- Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc - - account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629. 2415 account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628. 2417 account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664. 2456 Basic integer operations - times in nanoseconds - smaller is better --- Host OS intgr intgr intgr intgr intgr bit addmuldivmod - - -- -- -- -- -- account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 Basic float operations - times in nanoseconds - smaller is better - Host OS float float float float addmuldivbogo - - -- -- -- -- account Linux 2.6.11- 1.7300 2.4800 15.5 15.4 account Linux 2.6.11- 1.7300 2.4800 15.5 15.6 account Linux 2.6.11- 1.7400 2.5000 15.7 15.6 Basic double operations - times in nanoseconds - smaller is better -- Host OS double double double double addmuldivbogo - - -- -- -- -- account Linux 2.6.11- 1.7300 2.4800 15.5 15.4 account Linux 2.6.11- 1.7300 2.4800 15.5 15.6 account Linux 2.6.11- 1.7400 2.5000 15.7 15.6 Context switching - times in microseconds - smaller is better - Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw - - -- -- -- -- -- --- --- account Linux 2.6.11- 5.1300 5.2900 4.9700 3.1700 10.9 6.332.6 account Linux 2.6.11- 4.9000 5.2100 5.1600 4.4700 20.3 6.4800027.7 account Linux 2.6.11- 4.8600 5.3000 4.9200 3.5600 20.5 6.8700031.5 *Local* Communication
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote: I tested without user space listeners and the cost is negligible. I will test with a user space listeners and see the results. I'm going to run the test this week after improving the mechanism that switch on/off the sending of the message. I'm also trying to mesure the process-creation/destruction performance on following three environment. Archtechture: i686 / Distribution: Fedora Core 3 * Kernel Preemption is DISABLE * SMP kernel but UP-machine / Not Hyper Threading [1] 2.6.11-rc4-mm1 normal [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled) When 367th-fork() was called after fork-connector notification, kernel was locked up. (User-Space-Listener has been also run until 366th-fork() notification was received) So I ran the lmbench with three different kernels with the fork connector patch I just sent. Results are attached at the end of the mail and there are three different lines which are: o First line is a linux-2.6.11-rc4-mm1-cnfork o Second line is a linux-2.6.11-rc4-mm1 o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space application. The user space application listened during 15h and received 6496 messages. Each test has been ran only once. Best regards, Guillaume --- cd results make summary percent 2/dev/null | more make[1]: Entering directory `/home/guill/benchmark/lmbench/lmbench-3.0-a4/results' L M B E N C H 3 . 0 S U M M A R Y (Alpha software, do not distribute) Basic system parameters -- Host OS Description Mhz tlb cache mem scal pages line par load bytes - - --- - - -- account Linux 2.6.11- i686-pc-linux-gnu 276563 128 2.49001 account Linux 2.6.11- i686-pc-linux-gnu 276567 128 2.42001 account Linux 2.6.11- i686-pc-linux-gnu 276569 128 2.44001 Processor, Processes - times in microseconds - smaller is better -- Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc - - account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629. 2415 account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628. 2417 account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664. 2456 Basic integer operations - times in nanoseconds - smaller is better --- Host OS intgr intgr intgr intgr intgr bit addmuldivmod - - -- -- -- -- -- account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 account Linux 2.6.11- 0.1800 0.1700 4.9900 20.8 23.1 Basic float operations - times in nanoseconds - smaller is better - Host OS float float float float addmuldivbogo - - -- -- -- -- account Linux 2.6.11- 1.7300 2.4800 15.5 15.4 account Linux 2.6.11- 1.7300 2.4800 15.5 15.6 account Linux 2.6.11- 1.7400 2.5000 15.7 15.6 Basic double operations - times in nanoseconds - smaller is better -- Host OS double double double double addmuldivbogo - - -- -- -- -- account Linux 2.6.11- 1.7300 2.4800 15.5 15.4 account Linux 2.6.11- 1.7300 2.4800 15.5 15.6 account Linux 2.6.11- 1.7400 2.5000 15.7 15.6 Context switching - times in microseconds - smaller is better - Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw - - -- -- -- -- -- --- --- account Linux 2.6.11- 5.1300 5.2900 4.9700 3.1700 10.9 6.332.6 account Linux 2.6.11- 4.9000 5.2100 5.1600 4.4700 20.3 6.4800027.7 account Linux 2.6.11- 4.8600 5.3000 4.9200 3.5600 20.5 6.8700031.5 *Local* Communication latencies in microseconds -
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin [EMAIL PROTECTED] wrote: So I ran the lmbench with three different kernels with the fork connector patch I just sent. Results are attached at the end of the mail and there are three different lines which are: o First line is a linux-2.6.11-rc4-mm1-cnfork o Second line is a linux-2.6.11-rc4-mm1 o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space application. The user space application listened during 15h and received 6496 messages. Each test has been ran only once. ... -- Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc - - account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629. 2415 account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628. 2417 account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664. 2456 This is the interesting bit, yes? 5-10% slowdown on fork is expected, but why was exec slower? What does The user space application listened during 15h mean? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-03-02 at 01:06 -0800, Andrew Morton wrote:
Guillaume Thouvenin [EMAIL PROTECTED] wrote:
So I ran the lmbench with three different kernels with the fork
connector patch I just sent. Results are attached at the end of the mail
and there are three different lines which are:
o First line is a linux-2.6.11-rc4-mm1-cnfork
o Second line is a linux-2.6.11-rc4-mm1
o Third line is a linux-2.6.11-rc4-mm1-cnfork with a user space
application. The user space application listened during 15h
and received 6496 messages.
Each test has been ran only once.
...
--
Host OS Mhz null null open slct sig sig fork exec
sh
call I/O stat clos TCP inst hndl proc proc
proc
- -
account Linux 2.6.11- 2765 0.17 0.26 3.57 4.19 16.9 0.51 2.31 162. 629.
2415
account Linux 2.6.11- 2765 0.16 0.26 3.56 4.17 17.6 0.50 2.30 163. 628.
2417
account Linux 2.6.11- 2765 0.16 0.27 3.67 4.25 17.6 0.51 2.28 176. 664.
2456
This is the interesting bit, yes? 5-10% slowdown on fork is expected, but
why was exec slower?
I can't explain it for the moment. I will run test more than once to see
if this difference is still here.
What does The user space application listened during 15h mean?
It means that I ran the user space application before the test and
stop it 15 hours later (this morning for me). The test ran during
5h30mn.
The user space application increments a counter to show how many
processes have been created during a period of time. I have not use the
user space daemon that manages group of processes because the it still
uses the old mechanism (a signal sends from the do_fork()) and as I
wanted to provide quick results, I used another user space application.
I attache the test program (get_fork_info.c) that I'm using at the end
of the mail to clearly show what it does.
I will run new tests with the real user space daemon but it will be
ready next week, sorry for the delay.
Best regards,
Guillaume
---
/*
* get_fork_info.c
*
* This program listens netlink interface to retreive information
* sends by the kernel when forking. It increments a counter for
* each forks and when the user hit CRL-C, it displays how many
* fork occured during the period.
*/
#include stdio.h
#include stdlib.h
#include unistd.h
#include string.h
#include errno.h
#include signal.h
#include asm/types.h
#include sys/types.h
#include sys/socket.h
#include sys/time.h
#include linux/netlink.h
#include linux/connector.h
#define CN_FORK_OFF 0
#define CN_FORK_ON 1
#define MESSAGE_SIZE(sizeof(struct nlmsghdr) + \
sizeof(struct cn_msg) + \
sizeof(int))
int sock;
unsigned long total_p;
struct timeval test_time;
static inline void switch_cn_fork(int sock, int action)
{
char buff[128]; /* must be MESSAGE_SIZE */
struct nlmsghdr *hdr;
struct cn_msg *msg;
/* Clear the buffer */
memset(buff, '\0', sizeof(buff));
/* fill the message header */
hdr = (struct nlmsghdr *) buff;
hdr-nlmsg_len = MESSAGE_SIZE;
hdr-nlmsg_type = NLMSG_DONE;
hdr-nlmsg_flags = 0;
hdr-nlmsg_seq = 0;
hdr-nlmsg_pid = getpid();
/* the message */
msg = (struct cn_msg *) NLMSG_DATA(hdr);
msg-id.idx = CN_IDX_FORK;
msg-id.val = CN_VAL_FORK;
msg-seq = 0;
msg-ack = 0;
msg-len = sizeof(int);
msg-data[0] = action;
send(sock, hdr, hdr-nlmsg_len, 0);
}
static void cleanup()
{
struct timeval tmp_time;
switch_cn_fork(sock, CN_FORK_OFF);
tmp_time = test_time;
gettimeofday(test_time, NULL);
printf(%lu processes were created in %li seconds.\n,
total_p, test_time.tv_sec - tmp_time.tv_sec);
close(sock);
exit(EXIT_SUCCESS);
}
int main()
{
int err;
struct sockaddr_nl sa; /* information for NETLINK interface */
/*
* To be able to quit the application properly we install a
* signal handler that catch the CTRL-C
*/
signal(SIGTERM, cleanup);
signal(SIGINT, cleanup);
/*
* Create an endpoint for communication. Use the kernel user
* interface device (PF_NETLINK) which is a datagram oriented
* service (SOCK_DGRAM). The protocol used is the netfilter/iptables
* ULOG protocol (NETLINK_NFLOG)
*/
sock = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_NFLOG);
if (sock == -1) {
perror(socket);
return -1;
}
sa.nl_family = AF_NETLINK;
sa.nl_groups = CN_IDX_FORK;
Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew wrote: 5-10% slowdown on fork is expected, but why was exec slower? Thanks for the summary, Andrew. Guillaume (or anyone else tempted to do this) - it's a good idea, when posting 100 lines of data, to summarize with a line or two of words, as Andrew did here. It is far more efficient for one writer to do this, than each of a thousand readers. Hmmm ... so why was exec slower? -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson [EMAIL PROTECTED] 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Just a thought - perhaps you could see if Jay can test the performance scaling of these changes on larger systems (8 to 64 CPUs, give or take, small for SGI, but big for some vendors.) Things like a global lock, for example, might be harmless on smaller systems, but hurt big time on bigger systems. I don't know if you have any such constructs ... perhaps this doesn't matter. At the very least, we need to know that performance and scaling are not significantly impacted, on systems not using accounting, either because it is obvious from the code, or because someone has tested it. And if performance or scaling was impacted when accounting was enabled, then at least we would want to know how much performance was impacted, so that users would know what to expect when they use accounting. > the process-creation/destruction performance on following three environment. I think this is a good choice of what to measure, and where. Thank-you. > kernel was also locked up after 366th-fork() I have no idea what this is -- good luck finding it. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <[EMAIL PROTECTED]> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Jamal wrote: > What was wrong with just going ahead and just always > invoking your netlink_send()? I think the hope was to reduce the cost of the accounting hook in fork to "next-to-zero" if accounting is not being used on that system. See Andrew's query earlier: > b) they are next-to-zero cost if something is listening on the netlink >socket but no accounting daemon is running. Presumably sending an ignored packet costs something, quite possibly more than "next-to-zero". -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <[EMAIL PROTECTED]> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 14:53 +0100, Guillaume Thouvenin wrote:
> On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote:
> > > I tested without user space listeners and the cost is negligible. I will
> > > test with a user space listeners and see the results. I'm going to run
> > > the test this week after improving the mechanism that switch on/off the
> > > sending of the message.
> >
> > I'm also trying to mesure the process-creation/destruction performance on
> > following three environment.
> > Archtechture: i686 / Distribution: Fedora Core 3
> > * Kernel Preemption is DISABLE
> > * SMP kernel but UP-machine / Not Hyper Threading
> > [1] 2.6.11-rc4-mm1 normal
> > [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module
> > [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled)
> >
> > When 367th-fork() was called after fork-connector notification, kernel was
> > locked up.
> > (User-Space-Listener has been also run until 366th-fork() notification was
> > received)
>
> I don't see this limit on my computer. I'm currently running the lmbench
> with a new fork connector patch (one that enable/disable fork connector)
> on an SMP computer. I will send results and the new patch tomorrow
> because the test takes a while...
>
> I'm using a small patch provided by Evgeniy and not included in the
> 2.6.11-rc4-mm1 tree.
2.6.11-rc4-mm1 tree does not have the latest connector.
Various fixes were added, not only that.
I run the latest patch Guillaume sent to me(with small updates),
fork bomb with more than 100k forks passed already without any freeze.
I do not have numbers thought.
> Best regards,
> Guillaume
>
> --- orig/connector.c
> +++ mod/connector.c
> @@ -168,12 +168,11 @@
> group = NETLINK_CB((skb)).groups;
> msg = (struct cn_msg *)NLMSG_DATA(nlh);
>
> - if (msg->len != nlh->nlmsg_len - sizeof(*msg) - sizeof(*nlh)) {
> + if (NLMSG_SPACE(msg->len + sizeof(*msg)) != nlh->nlmsg_len) {
> printk(KERN_ERR "skb does not have enough length: "
> - "requested msg->len=%u[%u],
> nlh->nlmsg_len=%u[%u], skb->len=%u[must be %u].\n",
> - msg->len, NLMSG_SPACE(msg->len),
> - nlh->nlmsg_len, nlh->nlmsg_len - sizeof(*nlh),
> - skb->len, msg->len + sizeof(*msg));
> + "requested msg->len=%u[%u],
> nlh->nlmsg_len=%u, skb->len=%u.\n",
> + msg->len, NLMSG_SPACE(msg->len +
> sizeof(*msg)),
> + nlh->nlmsg_len, skb->len);
> kfree_skb(skb);
> return -EINVAL;
> }
>
--
Evgeniy Polyakov
Crash is better than data corruption -- Arthur Grabowski
signature.asc
Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote:
> > I tested without user space listeners and the cost is negligible. I will
> > test with a user space listeners and see the results. I'm going to run
> > the test this week after improving the mechanism that switch on/off the
> > sending of the message.
>
> I'm also trying to mesure the process-creation/destruction performance on
> following three environment.
> Archtechture: i686 / Distribution: Fedora Core 3
> * Kernel Preemption is DISABLE
> * SMP kernel but UP-machine / Not Hyper Threading
> [1] 2.6.11-rc4-mm1 normal
> [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module
> [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled)
>
> When 367th-fork() was called after fork-connector notification, kernel was
> locked up.
> (User-Space-Listener has been also run until 366th-fork() notification was
> received)
I don't see this limit on my computer. I'm currently running the lmbench
with a new fork connector patch (one that enable/disable fork connector)
on an SMP computer. I will send results and the new patch tomorrow
because the test takes a while...
I'm using a small patch provided by Evgeniy and not included in the
2.6.11-rc4-mm1 tree.
Best regards,
Guillaume
--- orig/connector.c
+++ mod/connector.c
@@ -168,12 +168,11 @@
group = NETLINK_CB((skb)).groups;
msg = (struct cn_msg *)NLMSG_DATA(nlh);
- if (msg->len != nlh->nlmsg_len - sizeof(*msg) - sizeof(*nlh)) {
+ if (NLMSG_SPACE(msg->len + sizeof(*msg)) != nlh->nlmsg_len) {
printk(KERN_ERR "skb does not have enough length: "
- "requested msg->len=%u[%u],
nlh->nlmsg_len=%u[%u], skb->len=%u[must be %u].\n",
- msg->len, NLMSG_SPACE(msg->len),
- nlh->nlmsg_len, nlh->nlmsg_len - sizeof(*nlh),
- skb->len, msg->len + sizeof(*msg));
+ "requested msg->len=%u[%u], nlh->nlmsg_len=%u,
skb->len=%u.\n",
+ msg->len, NLMSG_SPACE(msg->len + sizeof(*msg)),
+ nlh->nlmsg_len, skb->len);
kfree_skb(skb);
return -EINVAL;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hello, > I tested without user space listeners and the cost is negligible. I will > test with a user space listeners and see the results. I'm going to run > the test this week after improving the mechanism that switch on/off the > sending of the message. I'm also trying to mesure the process-creation/destruction performance on following three environment. Archtechture: i686 / Distribution: Fedora Core 3 * Kernel Preemption is DISABLE * SMP kernel but UP-machine / Not Hyper Threading [1] 2.6.11-rc4-mm1 normal [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled) When 367th-fork() was called after fork-connector notification, kernel was locked up. (User-Space-Listener has been also run until 366th-fork() notification was received) Does this number have any sort of means ? In my second trial, kernel was also locked up after 366th-fork() notification. Currently, I don't know its causition. Is there a person encounted it? # I wanted to say "[2] is faster than [3]" when process-grouping is enable, but the plan came off. :( Thanks. -- Linux Promotion Center, NEC KaiGai Kohei <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 19:17 +0300, Evgeniy Polyakov wrote: > On 28 Feb 2005 10:31:33 -0500 > jamal <[EMAIL PROTECTED]> wrote: > > I would bet the benefit you are seeing has to do with batching rather > > than such an optimization flag. Different ballgame. > > I relooked at their code snippet, they dont even have skbs built nor > > even figured out what sock or PID. That work still needs to be done it > > seems in cn_netlink_send(). So probably all they need to do is move the > > check in cn_netlink_send() instead. This is assuming they are not > > scratching their heads with some realted complexities. > [...] > As connector author, I still doubt it worth copying several lines > from netlink_broadcast() before skb allocation in cn_netlink_send(). > Of course it is easy and can be done, but I do not see any profit here. > Atomic allocation is fast, if it succeds, but there are no groups/socket to > send, > skb will be freed, if allocation fails, then group check is useless. > > I would prefer Guillaume Thouvenin as fork connector author to test > his current implementation and show that connector's cost is negligible > both with and without userspace listeners. > As far as I remember it is first entry in fork connector's TODO list. I tested without user space listeners and the cost is negligible. I will test with a user space listeners and see the results. I'm going to run the test this week after improving the mechanism that switch on/off the sending of the message. Best regards, Guillaume - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 19:17 +0300, Evgeniy Polyakov wrote: On 28 Feb 2005 10:31:33 -0500 jamal [EMAIL PROTECTED] wrote: I would bet the benefit you are seeing has to do with batching rather than such an optimization flag. Different ballgame. I relooked at their code snippet, they dont even have skbs built nor even figured out what sock or PID. That work still needs to be done it seems in cn_netlink_send(). So probably all they need to do is move the check in cn_netlink_send() instead. This is assuming they are not scratching their heads with some realted complexities. [...] As connector author, I still doubt it worth copying several lines from netlink_broadcast() before skb allocation in cn_netlink_send(). Of course it is easy and can be done, but I do not see any profit here. Atomic allocation is fast, if it succeds, but there are no groups/socket to send, skb will be freed, if allocation fails, then group check is useless. I would prefer Guillaume Thouvenin as fork connector author to test his current implementation and show that connector's cost is negligible both with and without userspace listeners. As far as I remember it is first entry in fork connector's TODO list. I tested without user space listeners and the cost is negligible. I will test with a user space listeners and see the results. I'm going to run the test this week after improving the mechanism that switch on/off the sending of the message. Best regards, Guillaume - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hello, I tested without user space listeners and the cost is negligible. I will test with a user space listeners and see the results. I'm going to run the test this week after improving the mechanism that switch on/off the sending of the message. I'm also trying to mesure the process-creation/destruction performance on following three environment. Archtechture: i686 / Distribution: Fedora Core 3 * Kernel Preemption is DISABLE * SMP kernel but UP-machine / Not Hyper Threading [1] 2.6.11-rc4-mm1 normal [2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module [3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled) When 367th-fork() was called after fork-connector notification, kernel was locked up. (User-Space-Listener has been also run until 366th-fork() notification was received) Does this number have any sort of means ? In my second trial, kernel was also locked up after 366th-fork() notification. Currently, I don't know its causition. Is there a person encounted it? # I wanted to say "[2] is faster than [3]" when process-grouping is enable, but the plan came off. :( Thanks. -- Linux Promotion Center, NEC KaiGai Kohei [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote:
I tested without user space listeners and the cost is negligible. I will
test with a user space listeners and see the results. I'm going to run
the test this week after improving the mechanism that switch on/off the
sending of the message.
I'm also trying to mesure the process-creation/destruction performance on
following three environment.
Archtechture: i686 / Distribution: Fedora Core 3
* Kernel Preemption is DISABLE
* SMP kernel but UP-machine / Not Hyper Threading
[1] 2.6.11-rc4-mm1 normal
[2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module
[3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled)
When 367th-fork() was called after fork-connector notification, kernel was
locked up.
(User-Space-Listener has been also run until 366th-fork() notification was
received)
I don't see this limit on my computer. I'm currently running the lmbench
with a new fork connector patch (one that enable/disable fork connector)
on an SMP computer. I will send results and the new patch tomorrow
because the test takes a while...
I'm using a small patch provided by Evgeniy and not included in the
2.6.11-rc4-mm1 tree.
Best regards,
Guillaume
--- orig/connector.c
+++ mod/connector.c
@@ -168,12 +168,11 @@
group = NETLINK_CB((skb)).groups;
msg = (struct cn_msg *)NLMSG_DATA(nlh);
- if (msg-len != nlh-nlmsg_len - sizeof(*msg) - sizeof(*nlh)) {
+ if (NLMSG_SPACE(msg-len + sizeof(*msg)) != nlh-nlmsg_len) {
printk(KERN_ERR skb does not have enough length:
- requested msg-len=%u[%u],
nlh-nlmsg_len=%u[%u], skb-len=%u[must be %u].\n,
- msg-len, NLMSG_SPACE(msg-len),
- nlh-nlmsg_len, nlh-nlmsg_len - sizeof(*nlh),
- skb-len, msg-len + sizeof(*msg));
+ requested msg-len=%u[%u], nlh-nlmsg_len=%u,
skb-len=%u.\n,
+ msg-len, NLMSG_SPACE(msg-len + sizeof(*msg)),
+ nlh-nlmsg_len, skb-len);
kfree_skb(skb);
return -EINVAL;
}
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-03-01 at 14:53 +0100, Guillaume Thouvenin wrote:
On Tue, 2005-03-01 at 22:38 +0900, Kaigai Kohei wrote:
I tested without user space listeners and the cost is negligible. I will
test with a user space listeners and see the results. I'm going to run
the test this week after improving the mechanism that switch on/off the
sending of the message.
I'm also trying to mesure the process-creation/destruction performance on
following three environment.
Archtechture: i686 / Distribution: Fedora Core 3
* Kernel Preemption is DISABLE
* SMP kernel but UP-machine / Not Hyper Threading
[1] 2.6.11-rc4-mm1 normal
[2] 2.6.11-rc4-mm1 with PAGG based Process Accounting Module
[3] 2.6.11-rc4-mm1 with fork-connector notification (it's enabled)
When 367th-fork() was called after fork-connector notification, kernel was
locked up.
(User-Space-Listener has been also run until 366th-fork() notification was
received)
I don't see this limit on my computer. I'm currently running the lmbench
with a new fork connector patch (one that enable/disable fork connector)
on an SMP computer. I will send results and the new patch tomorrow
because the test takes a while...
I'm using a small patch provided by Evgeniy and not included in the
2.6.11-rc4-mm1 tree.
2.6.11-rc4-mm1 tree does not have the latest connector.
Various fixes were added, not only that.
I run the latest patch Guillaume sent to me(with small updates),
fork bomb with more than 100k forks passed already without any freeze.
I do not have numbers thought.
Best regards,
Guillaume
--- orig/connector.c
+++ mod/connector.c
@@ -168,12 +168,11 @@
group = NETLINK_CB((skb)).groups;
msg = (struct cn_msg *)NLMSG_DATA(nlh);
- if (msg-len != nlh-nlmsg_len - sizeof(*msg) - sizeof(*nlh)) {
+ if (NLMSG_SPACE(msg-len + sizeof(*msg)) != nlh-nlmsg_len) {
printk(KERN_ERR skb does not have enough length:
- requested msg-len=%u[%u],
nlh-nlmsg_len=%u[%u], skb-len=%u[must be %u].\n,
- msg-len, NLMSG_SPACE(msg-len),
- nlh-nlmsg_len, nlh-nlmsg_len - sizeof(*nlh),
- skb-len, msg-len + sizeof(*msg));
+ requested msg-len=%u[%u],
nlh-nlmsg_len=%u, skb-len=%u.\n,
+ msg-len, NLMSG_SPACE(msg-len +
sizeof(*msg)),
+ nlh-nlmsg_len, skb-len);
kfree_skb(skb);
return -EINVAL;
}
--
Evgeniy Polyakov
Crash is better than data corruption -- Arthur Grabowski
signature.asc
Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
Jamal wrote: What was wrong with just going ahead and just always invoking your netlink_send()? I think the hope was to reduce the cost of the accounting hook in fork to next-to-zero if accounting is not being used on that system. See Andrew's query earlier: b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Presumably sending an ignored packet costs something, quite possibly more than next-to-zero. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson [EMAIL PROTECTED] 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Just a thought - perhaps you could see if Jay can test the performance scaling of these changes on larger systems (8 to 64 CPUs, give or take, small for SGI, but big for some vendors.) Things like a global lock, for example, might be harmless on smaller systems, but hurt big time on bigger systems. I don't know if you have any such constructs ... perhaps this doesn't matter. At the very least, we need to know that performance and scaling are not significantly impacted, on systems not using accounting, either because it is obvious from the code, or because someone has tested it. And if performance or scaling was impacted when accounting was enabled, then at least we would want to know how much performance was impacted, so that users would know what to expect when they use accounting. the process-creation/destruction performance on following three environment. I think this is a good choice of what to measure, and where. Thank-you. kernel was also locked up after 366th-fork() I have no idea what this is -- good luck finding it. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson [EMAIL PROTECTED] 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On 28 Feb 2005 10:31:33 -0500 jamal <[EMAIL PROTECTED]> wrote: > On Mon, 2005-02-28 at 09:25, Thomas Graf wrote: > > * jamal <[EMAIL PROTECTED]> 2005-02-28 09:10 > [..] > > > To justify writting the new code, I am assuming someone has actually sat > > > down and in the minimal stuck their finger in the air > > > and said "yes, there is definetely wind there". > > > > I did, not for this problem though. The code this idea comes from sends > > batched events > > I would bet the benefit you are seeing has to do with batching rather > than such an optimization flag. Different ballgame. > I relooked at their code snippet, they dont even have skbs built nor > even figured out what sock or PID. That work still needs to be done it > seems in cn_netlink_send(). So probably all they need to do is move the > check in cn_netlink_send() instead. This is assuming they are not > scratching their heads with some realted complexities. > > I am gonna disapear for a while; hopefully the original posters have > gathered some ideas from what we discussed. As connector author, I still doubt it worth copying several lines from netlink_broadcast() before skb allocation in cn_netlink_send(). Of course it is easy and can be done, but I do not see any profit here. Atomic allocation is fast, if it succeds, but there are no groups/socket to send, skb will be freed, if allocation fails, then group check is useless. I would prefer Guillaume Thouvenin as fork connector author to test his current implementation and show that connector's cost is negligible both with and without userspace listeners. As far as I remember it is first entry in fork connector's TODO list. > cheers, > jamal > Evgeniy Polyakov Only failure makes us experts. -- Theo de Raadt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 09:25, Thomas Graf wrote: > * jamal <[EMAIL PROTECTED]> 2005-02-28 09:10 [..] > > To justify writting the new code, I am assuming someone has actually sat > > down and in the minimal stuck their finger in the air > > and said "yes, there is definetely wind there". > > I did, not for this problem though. The code this idea comes from sends > batched events I would bet the benefit you are seeing has to do with batching rather than such an optimization flag. Different ballgame. I relooked at their code snippet, they dont even have skbs built nor even figured out what sock or PID. That work still needs to be done it seems in cn_netlink_send(). So probably all they need to do is move the check in cn_netlink_send() instead. This is assuming they are not scratching their heads with some realted complexities. I am gonna disapear for a while; hopefully the original posters have gathered some ideas from what we discussed. cheers, jamal - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* jamal <[EMAIL PROTECTED]> 2005-02-28 09:10 > On Mon, 2005-02-28 at 08:53, Thomas Graf wrote: > > * jamal <[EMAIL PROTECTED]> 2005-02-28 08:40 > > > > > > netlink broadcast or a wrapper around it. > > > Why even bother doing the check with netlink_has_listeners()? > > > > To implement the master enable/disable switch they want. The messages > > don't get send out anyway but why bother doing all the work if nothing > > will get send out in the end? It implements a well defined flag > > controlled by open/close on fds (thus handles dying applications) > > stating whether the whole code should be enabled or disabled. It is of > > course not needed to avoid sending unnecessary messages. > > To justify writting the new code, I am assuming someone has actually sat > down and in the minimal stuck their finger in the air > and said "yes, there is definetely wind there". I did, not for this problem though. The code this idea comes from sends batched events of skb passing points to userspace. Not every call invokes has_listeneres() but rather the kernel thread processing the ring buffer sending the events to userspaces does. The result is globally cached in a atomic_t making it possible to check for it at zero-cost and really saving time and effort. I have no clue wether it does make sense in this case I just pointed out how to do it properly at my point of view. > Which leadsto Marcello's question in other email: > Theres some overhead. > - Message needs to be built with skbs allocated (not the cn_xxx thing > that seems to be invoked - I suspect that thing will build the skbs); > - the netlink table needs to be locked > -and searched and only then do you find theres nothing to send to. > > The point i was making is if you actually had to post this question, > then you must be running into some problems of complexity ;-> > which implies to me that the delta overhead maybe worth it compared to > introducing the complexity or any new code. > I wasnt involved in the discussion - I just woke up and saw the posting > and was bored. So the justification for the optimization has probably > been explained and it may be worth doing the check (but probably such > check should go into whatever that cn_xxx is). I wasn't involved in the discussion either. Using rtmsg_ifinfo as example, the check should probably go in straight at the beginning _IFF_ rtmsg_ifinfo was subject to performance overhead which obviously isn't the case but just served as an example. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, Feb 28, 2005 at 02:53:07PM +0100, Thomas Graf wrote: > * jamal <[EMAIL PROTECTED]> 2005-02-28 08:40 > > > > netlink broadcast or a wrapper around it. > > Why even bother doing the check with netlink_has_listeners()? > > To implement the master enable/disable switch they want. The messages > don't get send out anyway but why bother doing all the work if nothing > will get send out in the end? It implements a well defined flag > controlled by open/close on fds (thus handles dying applications) > stating whether the whole code should be enabled or disabled. Yep - this far from "reinventing the wheel". ;) > It is of course not needed to avoid sending unnecessary messages. Thats the goal, thanks. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 08:53, Thomas Graf wrote: > * jamal <[EMAIL PROTECTED]> 2005-02-28 08:40 > > > > netlink broadcast or a wrapper around it. > > Why even bother doing the check with netlink_has_listeners()? > > To implement the master enable/disable switch they want. The messages > don't get send out anyway but why bother doing all the work if nothing > will get send out in the end? It implements a well defined flag > controlled by open/close on fds (thus handles dying applications) > stating whether the whole code should be enabled or disabled. It is of > course not needed to avoid sending unnecessary messages. To justify writting the new code, I am assuming someone has actually sat down and in the minimal stuck their finger in the air and said "yes, there is definetely wind there". Which leadsto Marcello's question in other email: Theres some overhead. - Message needs to be built with skbs allocated (not the cn_xxx thing that seems to be invoked - I suspect that thing will build the skbs); - the netlink table needs to be locked -and searched and only then do you find theres nothing to send to. The point i was making is if you actually had to post this question, then you must be running into some problems of complexity ;-> which implies to me that the delta overhead maybe worth it compared to introducing the complexity or any new code. I wasnt involved in the discussion - I just woke up and saw the posting and was bored. So the justification for the optimization has probably been explained and it may be worth doing the check (but probably such check should go into whatever that cn_xxx is). cheers, jamal - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* jamal <[EMAIL PROTECTED]> 2005-02-28 08:40 > > netlink broadcast or a wrapper around it. > Why even bother doing the check with netlink_has_listeners()? To implement the master enable/disable switch they want. The messages don't get send out anyway but why bother doing all the work if nothing will get send out in the end? It implements a well defined flag controlled by open/close on fds (thus handles dying applications) stating whether the whole code should be enabled or disabled. It is of course not needed to avoid sending unnecessary messages. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
I'm net ignorant, so just hit me with a cluebat if thats appropriate. On Mon, Feb 28, 2005 at 07:10:58AM -0500, jamal wrote: > > Havent seen the beginnings of this thread. But whatever you are trying > to do seems to suggest some complexity that you are trying to > workaround. What was wrong with just going ahead and just always > invoking your netlink_send()? What overhead does the netlink_send() impose if there are no listeners? Sure, it wont go anywhere, but the message will have to be assembled and sent anyway. Correct? The way things are now, its necessary to make the decision to invoke or not netlink_send() due to the supposed overhead. Thats what Guillaume is doing, and thats what will have to be done whenever one wants to send information through netlink from performance critical paths. Can't the assembly/etc overhead associated with netlink_send() be avoided earlier, approaching zero-cost ? Being able to get rid of the decision to invoke or not the sendmsg would be nice. TIA > If there are nobody in user space (or > kernel) listening, it wont go anywhere. > > cheers, > jamal > > On Mon, 2005-02-28 at 02:39, Andrew Morton wrote: > > Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > > > > >Ok the protocol is maybe too "basic" but with this mechanism the user > > > space application that uses the fork connector can start and stop the > > > send of messages. This implementation needs somme improvements because > > > currently, if two application are using the fork connector one can > > > enable it and the other don't know if it is enable or not, but the idea > > > is here I think. > > > > Yes. But this problem can be solved in userspace, with a little library > > function and a bit of locking. > > > > IOW: use the library to enable/disable the fork connector rather than > > directly doing syscalls. > > > > It has the problem that if a client of that library crashes, the counter > > gets out of whack, but really, it's not all _that_ important, and to handle > > this properly in-kernel each client would need an open fd against some > > object so we can do the close-on-exit thing properly. You'd need to create > > a separate netlink socket for the purpose. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
netlink broadcast or a wrapper around it.
Why even bother doing the check with netlink_has_listeners()?
cheers,
jamal
On Mon, 2005-02-28 at 08:20, Thomas Graf wrote:
> > Havent seen the beginnings of this thread. But whatever you are trying
> > to do seems to suggest some complexity that you are trying to
> > workaround. What was wrong with just going ahead and just always
> > invoking your netlink_send()?
>
> I guess parts of the wheel are broken and need to be reinvented ;->
>
> > If there are nobody in user space (or kernel) listening, it wont go
> > anywhere.
>
> Additional you may want to extend netlink a bit to check whether
> there is a listener before creating the messages. The method to do so
> depends on whether you use netlink_send() or netlink_brodacast(). The
> latter is more flexiable because you can add more groups later on
> and the userspace applications can decicde which ones they want to
> listen to. Both methods handle dying clients perfectly fine, the
> association to the netlink socket gets destroyed as soon as the socket
> is closed. Therefore you can simply check mc_list of the netlink
> protocol you use to see if there are any listeners registered:
>
> static inline int netlink_has_listeners(struct sock *sk)
> {
> int ret;
>
> read_lock(_table_lock);
> ret = list_empty(_table[sk->sk_protocol].mc_list)
> read_unlock(_table_lock);
>
> return !ret;
> }
>
> This is simplified and ignores the actual group assignments, i.e. you
> might want to extend it to have it check if there are listeners for
> a certain group.
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
> Havent seen the beginnings of this thread. But whatever you are trying
> to do seems to suggest some complexity that you are trying to
> workaround. What was wrong with just going ahead and just always
> invoking your netlink_send()?
I guess parts of the wheel are broken and need to be reinvented ;->
> If there are nobody in user space (or kernel) listening, it wont go anywhere.
Additional you may want to extend netlink a bit to check whether
there is a listener before creating the messages. The method to do so
depends on whether you use netlink_send() or netlink_brodacast(). The
latter is more flexiable because you can add more groups later on
and the userspace applications can decicde which ones they want to
listen to. Both methods handle dying clients perfectly fine, the
association to the netlink socket gets destroyed as soon as the socket
is closed. Therefore you can simply check mc_list of the netlink
protocol you use to see if there are any listeners registered:
static inline int netlink_has_listeners(struct sock *sk)
{
int ret;
read_lock(_table_lock);
ret = list_empty(_table[sk->sk_protocol].mc_list)
read_unlock(_table_lock);
return !ret;
}
This is simplified and ignores the actual group assignments, i.e. you
might want to extend it to have it check if there are listeners for
a certain group.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Havent seen the beginnings of this thread. But whatever you are trying to do seems to suggest some complexity that you are trying to workaround. What was wrong with just going ahead and just always invoking your netlink_send()? If there are nobody in user space (or kernel) listening, it wont go anywhere. cheers, jamal On Mon, 2005-02-28 at 02:39, Andrew Morton wrote: > Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > > >Ok the protocol is maybe too "basic" but with this mechanism the user > > space application that uses the fork connector can start and stop the > > send of messages. This implementation needs somme improvements because > > currently, if two application are using the fork connector one can > > enable it and the other don't know if it is enable or not, but the idea > > is here I think. > > Yes. But this problem can be solved in userspace, with a little library > function and a bit of locking. > > IOW: use the library to enable/disable the fork connector rather than > directly doing syscalls. > > It has the problem that if a client of that library crashes, the counter > gets out of whack, but really, it's not all _that_ important, and to handle > this properly in-kernel each client would need an open fd against some > object so we can do the close-on-exit thing properly. You'd need to create > a separate netlink socket for the purpose. > > > --- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://ads.osdn.com/?ad_id=6595_id=14396=click > ___ > Lse-tech mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/lse-tech > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Sun, 2005-02-27 at 23:39 -0800, Andrew Morton wrote: > Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > > >Ok the protocol is maybe too "basic" but with this mechanism the user > > space application that uses the fork connector can start and stop the > > send of messages. This implementation needs somme improvements because > > currently, if two application are using the fork connector one can > > enable it and the other don't know if it is enable or not, but the idea > > is here I think. > > Yes. But this problem can be solved in userspace, with a little library > function and a bit of locking. > > IOW: use the library to enable/disable the fork connector rather than > directly doing syscalls. > > It has the problem that if a client of that library crashes, the counter > gets out of whack, but really, it's not all _that_ important, and to handle > this properly in-kernel each client would need an open fd against some > object so we can do the close-on-exit thing properly. You'd need to create > a separate netlink socket for the purpose. Why dont just extend protocol a bit? Add header after cn_msg, which will have get/set field and that is all. Properly using seq/ack fields userspace can avoid locks. -- Evgeniy Polyakov Crash is better than data corruption -- Arthur Grabowski signature.asc Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
On Sun, 2005-02-27 at 23:39 -0800, Andrew Morton wrote: Guillaume Thouvenin [EMAIL PROTECTED] wrote: Ok the protocol is maybe too basic but with this mechanism the user space application that uses the fork connector can start and stop the send of messages. This implementation needs somme improvements because currently, if two application are using the fork connector one can enable it and the other don't know if it is enable or not, but the idea is here I think. Yes. But this problem can be solved in userspace, with a little library function and a bit of locking. IOW: use the library to enable/disable the fork connector rather than directly doing syscalls. It has the problem that if a client of that library crashes, the counter gets out of whack, but really, it's not all _that_ important, and to handle this properly in-kernel each client would need an open fd against some object so we can do the close-on-exit thing properly. You'd need to create a separate netlink socket for the purpose. Why dont just extend protocol a bit? Add header after cn_msg, which will have get/set field and that is all. Properly using seq/ack fields userspace can avoid locks. -- Evgeniy Polyakov Crash is better than data corruption -- Arthur Grabowski signature.asc Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
Havent seen the beginnings of this thread. But whatever you are trying to do seems to suggest some complexity that you are trying to workaround. What was wrong with just going ahead and just always invoking your netlink_send()? If there are nobody in user space (or kernel) listening, it wont go anywhere. cheers, jamal On Mon, 2005-02-28 at 02:39, Andrew Morton wrote: Guillaume Thouvenin [EMAIL PROTECTED] wrote: Ok the protocol is maybe too basic but with this mechanism the user space application that uses the fork connector can start and stop the send of messages. This implementation needs somme improvements because currently, if two application are using the fork connector one can enable it and the other don't know if it is enable or not, but the idea is here I think. Yes. But this problem can be solved in userspace, with a little library function and a bit of locking. IOW: use the library to enable/disable the fork connector rather than directly doing syscalls. It has the problem that if a client of that library crashes, the counter gets out of whack, but really, it's not all _that_ important, and to handle this properly in-kernel each client would need an open fd against some object so we can do the close-on-exit thing properly. You'd need to create a separate netlink socket for the purpose. --- SF email is sponsored by - The IT Product Guide Read honest candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595alloc_id=14396op=click ___ Lse-tech mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/lse-tech - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Havent seen the beginnings of this thread. But whatever you are trying
to do seems to suggest some complexity that you are trying to
workaround. What was wrong with just going ahead and just always
invoking your netlink_send()?
I guess parts of the wheel are broken and need to be reinvented ;-
If there are nobody in user space (or kernel) listening, it wont go anywhere.
Additional you may want to extend netlink a bit to check whether
there is a listener before creating the messages. The method to do so
depends on whether you use netlink_send() or netlink_brodacast(). The
latter is more flexiable because you can add more groups later on
and the userspace applications can decicde which ones they want to
listen to. Both methods handle dying clients perfectly fine, the
association to the netlink socket gets destroyed as soon as the socket
is closed. Therefore you can simply check mc_list of the netlink
protocol you use to see if there are any listeners registered:
static inline int netlink_has_listeners(struct sock *sk)
{
int ret;
read_lock(nl_table_lock);
ret = list_empty(nl_table[sk-sk_protocol].mc_list)
read_unlock(nl_table_lock);
return !ret;
}
This is simplified and ignores the actual group assignments, i.e. you
might want to extend it to have it check if there are listeners for
a certain group.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
netlink broadcast or a wrapper around it.
Why even bother doing the check with netlink_has_listeners()?
cheers,
jamal
On Mon, 2005-02-28 at 08:20, Thomas Graf wrote:
Havent seen the beginnings of this thread. But whatever you are trying
to do seems to suggest some complexity that you are trying to
workaround. What was wrong with just going ahead and just always
invoking your netlink_send()?
I guess parts of the wheel are broken and need to be reinvented ;-
If there are nobody in user space (or kernel) listening, it wont go
anywhere.
Additional you may want to extend netlink a bit to check whether
there is a listener before creating the messages. The method to do so
depends on whether you use netlink_send() or netlink_brodacast(). The
latter is more flexiable because you can add more groups later on
and the userspace applications can decicde which ones they want to
listen to. Both methods handle dying clients perfectly fine, the
association to the netlink socket gets destroyed as soon as the socket
is closed. Therefore you can simply check mc_list of the netlink
protocol you use to see if there are any listeners registered:
static inline int netlink_has_listeners(struct sock *sk)
{
int ret;
read_lock(nl_table_lock);
ret = list_empty(nl_table[sk-sk_protocol].mc_list)
read_unlock(nl_table_lock);
return !ret;
}
This is simplified and ignores the actual group assignments, i.e. you
might want to extend it to have it check if there are listeners for
a certain group.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
I'm net ignorant, so just hit me with a cluebat if thats appropriate. On Mon, Feb 28, 2005 at 07:10:58AM -0500, jamal wrote: Havent seen the beginnings of this thread. But whatever you are trying to do seems to suggest some complexity that you are trying to workaround. What was wrong with just going ahead and just always invoking your netlink_send()? What overhead does the netlink_send() impose if there are no listeners? Sure, it wont go anywhere, but the message will have to be assembled and sent anyway. Correct? The way things are now, its necessary to make the decision to invoke or not netlink_send() due to the supposed overhead. Thats what Guillaume is doing, and thats what will have to be done whenever one wants to send information through netlink from performance critical paths. Can't the assembly/etc overhead associated with netlink_send() be avoided earlier, approaching zero-cost ? Being able to get rid of the decision to invoke or not the sendmsg would be nice. TIA If there are nobody in user space (or kernel) listening, it wont go anywhere. cheers, jamal On Mon, 2005-02-28 at 02:39, Andrew Morton wrote: Guillaume Thouvenin [EMAIL PROTECTED] wrote: Ok the protocol is maybe too basic but with this mechanism the user space application that uses the fork connector can start and stop the send of messages. This implementation needs somme improvements because currently, if two application are using the fork connector one can enable it and the other don't know if it is enable or not, but the idea is here I think. Yes. But this problem can be solved in userspace, with a little library function and a bit of locking. IOW: use the library to enable/disable the fork connector rather than directly doing syscalls. It has the problem that if a client of that library crashes, the counter gets out of whack, but really, it's not all _that_ important, and to handle this properly in-kernel each client would need an open fd against some object so we can do the close-on-exit thing properly. You'd need to create a separate netlink socket for the purpose. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* jamal [EMAIL PROTECTED] 2005-02-28 08:40 netlink broadcast or a wrapper around it. Why even bother doing the check with netlink_has_listeners()? To implement the master enable/disable switch they want. The messages don't get send out anyway but why bother doing all the work if nothing will get send out in the end? It implements a well defined flag controlled by open/close on fds (thus handles dying applications) stating whether the whole code should be enabled or disabled. It is of course not needed to avoid sending unnecessary messages. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 08:53, Thomas Graf wrote: * jamal [EMAIL PROTECTED] 2005-02-28 08:40 netlink broadcast or a wrapper around it. Why even bother doing the check with netlink_has_listeners()? To implement the master enable/disable switch they want. The messages don't get send out anyway but why bother doing all the work if nothing will get send out in the end? It implements a well defined flag controlled by open/close on fds (thus handles dying applications) stating whether the whole code should be enabled or disabled. It is of course not needed to avoid sending unnecessary messages. To justify writting the new code, I am assuming someone has actually sat down and in the minimal stuck their finger in the air and said yes, there is definetely wind there. Which leadsto Marcello's question in other email: Theres some overhead. - Message needs to be built with skbs allocated (not the cn_xxx thing that seems to be invoked - I suspect that thing will build the skbs); - the netlink table needs to be locked -and searched and only then do you find theres nothing to send to. The point i was making is if you actually had to post this question, then you must be running into some problems of complexity ;- which implies to me that the delta overhead maybe worth it compared to introducing the complexity or any new code. I wasnt involved in the discussion - I just woke up and saw the posting and was bored. So the justification for the optimization has probably been explained and it may be worth doing the check (but probably such check should go into whatever that cn_xxx is). cheers, jamal - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, Feb 28, 2005 at 02:53:07PM +0100, Thomas Graf wrote: * jamal [EMAIL PROTECTED] 2005-02-28 08:40 netlink broadcast or a wrapper around it. Why even bother doing the check with netlink_has_listeners()? To implement the master enable/disable switch they want. The messages don't get send out anyway but why bother doing all the work if nothing will get send out in the end? It implements a well defined flag controlled by open/close on fds (thus handles dying applications) stating whether the whole code should be enabled or disabled. Yep - this far from reinventing the wheel. ;) It is of course not needed to avoid sending unnecessary messages. Thats the goal, thanks. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* jamal [EMAIL PROTECTED] 2005-02-28 09:10 On Mon, 2005-02-28 at 08:53, Thomas Graf wrote: * jamal [EMAIL PROTECTED] 2005-02-28 08:40 netlink broadcast or a wrapper around it. Why even bother doing the check with netlink_has_listeners()? To implement the master enable/disable switch they want. The messages don't get send out anyway but why bother doing all the work if nothing will get send out in the end? It implements a well defined flag controlled by open/close on fds (thus handles dying applications) stating whether the whole code should be enabled or disabled. It is of course not needed to avoid sending unnecessary messages. To justify writting the new code, I am assuming someone has actually sat down and in the minimal stuck their finger in the air and said yes, there is definetely wind there. I did, not for this problem though. The code this idea comes from sends batched events of skb passing points to userspace. Not every call invokes has_listeneres() but rather the kernel thread processing the ring buffer sending the events to userspaces does. The result is globally cached in a atomic_t making it possible to check for it at zero-cost and really saving time and effort. I have no clue wether it does make sense in this case I just pointed out how to do it properly at my point of view. Which leadsto Marcello's question in other email: Theres some overhead. - Message needs to be built with skbs allocated (not the cn_xxx thing that seems to be invoked - I suspect that thing will build the skbs); - the netlink table needs to be locked -and searched and only then do you find theres nothing to send to. The point i was making is if you actually had to post this question, then you must be running into some problems of complexity ;- which implies to me that the delta overhead maybe worth it compared to introducing the complexity or any new code. I wasnt involved in the discussion - I just woke up and saw the posting and was bored. So the justification for the optimization has probably been explained and it may be worth doing the check (but probably such check should go into whatever that cn_xxx is). I wasn't involved in the discussion either. Using rtmsg_ifinfo as example, the check should probably go in straight at the beginning _IFF_ rtmsg_ifinfo was subject to performance overhead which obviously isn't the case but just served as an example. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 09:25, Thomas Graf wrote: * jamal [EMAIL PROTECTED] 2005-02-28 09:10 [..] To justify writting the new code, I am assuming someone has actually sat down and in the minimal stuck their finger in the air and said yes, there is definetely wind there. I did, not for this problem though. The code this idea comes from sends batched events I would bet the benefit you are seeing has to do with batching rather than such an optimization flag. Different ballgame. I relooked at their code snippet, they dont even have skbs built nor even figured out what sock or PID. That work still needs to be done it seems in cn_netlink_send(). So probably all they need to do is move the check in cn_netlink_send() instead. This is assuming they are not scratching their heads with some realted complexities. I am gonna disapear for a while; hopefully the original posters have gathered some ideas from what we discussed. cheers, jamal - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On 28 Feb 2005 10:31:33 -0500 jamal [EMAIL PROTECTED] wrote: On Mon, 2005-02-28 at 09:25, Thomas Graf wrote: * jamal [EMAIL PROTECTED] 2005-02-28 09:10 [..] To justify writting the new code, I am assuming someone has actually sat down and in the minimal stuck their finger in the air and said yes, there is definetely wind there. I did, not for this problem though. The code this idea comes from sends batched events I would bet the benefit you are seeing has to do with batching rather than such an optimization flag. Different ballgame. I relooked at their code snippet, they dont even have skbs built nor even figured out what sock or PID. That work still needs to be done it seems in cn_netlink_send(). So probably all they need to do is move the check in cn_netlink_send() instead. This is assuming they are not scratching their heads with some realted complexities. I am gonna disapear for a while; hopefully the original posters have gathered some ideas from what we discussed. As connector author, I still doubt it worth copying several lines from netlink_broadcast() before skb allocation in cn_netlink_send(). Of course it is easy and can be done, but I do not see any profit here. Atomic allocation is fast, if it succeds, but there are no groups/socket to send, skb will be freed, if allocation fails, then group check is useless. I would prefer Guillaume Thouvenin as fork connector author to test his current implementation and show that connector's cost is negligible both with and without userspace listeners. As far as I remember it is first entry in fork connector's TODO list. cheers, jamal Evgeniy Polyakov Only failure makes us experts. -- Theo de Raadt - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > >Ok the protocol is maybe too "basic" but with this mechanism the user > space application that uses the fork connector can start and stop the > send of messages. This implementation needs somme improvements because > currently, if two application are using the fork connector one can > enable it and the other don't know if it is enable or not, but the idea > is here I think. Yes. But this problem can be solved in userspace, with a little library function and a bit of locking. IOW: use the library to enable/disable the fork connector rather than directly doing syscalls. It has the problem that if a client of that library crashes, the counter gets out of whack, but really, it's not all _that_ important, and to handle this properly in-kernel each client would need an open fd against some object so we can do the close-on-exit thing properly. You'd need to create a separate netlink socket for the purpose. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 10:59 +0900, Kaigai Kohei wrote:
> Marcelo Tosatti wrote:
> > Yep, the netlink people should be able to help - they known what would be
> > required for not sending messages in case there is no listener registered.
> >
> > Maybe its already possible? I have never used netlink myself.
>
> If we notify the fork/exec/exit-events to user-space directly as you said,
> I don't think some hackings on netlink is necessary.
> For example, such packets is sent only when /proc/sys/.../process_grouping is
> set,
> and user-side daemon set this value, and unset when daemon will exit.
> It's not necessary to take too seriously.
I wrote a new fork connector patch with a callback to enable/disable
messages in case there is or isn't listener. I will post it this week.
Basically there is a global variable that is manipulated with a
connector callback so a user space daemon can manipulate the variable.
In the fork_connector() function you have:
static inline void fork_connector(pid_t parent, pid_t child)
{
static DEFINE_SPINLOCK(cn_fork_lock);
static __u32 seq; /* used to test if message is lost */
if (cn_fork_enable) {
[...]
cn_netlink_send(msg, CN_IDX_FORK);
}
}
and in the cn_fork module (drivers/connector/cn_fork.c) the callback is
defined as:
static void cn_fork_callback(void *data)
{
if (cn_already_initialized)
cn_fork_enable = cn_fork_enable ? 0 : 1;
}
Ok the protocol is maybe too "basic" but with this mechanism the user
space application that uses the fork connector can start and stop the
send of messages. This implementation needs somme improvements because
currently, if two application are using the fork connector one can
enable it and the other don't know if it is enable or not, but the idea
is here I think.
Regards,
Guillaume
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 10:59 +0900, Kaigai Kohei wrote: > Hello, > > Marcelo Tosatti wrote: > > Yep, the netlink people should be able to help - they known what would be > > required for not sending messages in case there is no listener registered. > > > > Maybe its already possible? I have never used netlink myself. > > If we notify the fork/exec/exit-events to user-space directly as you said, > I don't think some hackings on netlink is necessary. > For example, such packets is sent only when /proc/sys/.../process_grouping is > set, > and user-side daemon set this value, and unset when daemon will exit. > It's not necessary to take too seriously. Kernel accounting already was discussed in lkml week ago - I'm quite sure Guillaume Thouvenin created exactly that. His module creates do_fork() hook and broadcasts various process' states over netlink. Discussion at http://lkml.org/lkml/2005/2/17/87 -- Evgeniy Polyakov Crash is better than data corruption -- Arthur Grabowski signature.asc Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
First of all, I'm not aware of the whole discussion, ignore this if it has been brought to attention already. > > Yep, the netlink people should be able to help - they known what would be > > required for not sending messages in case there is no listener registered. > > > > Maybe its already possible? I have never used netlink myself. The easiest way is to use netlink_broadcast() and have userspace register to a netlink multicast group (set .nl_groups before connecting the socket). The netlink message will be sent to only those netlink sockets assigned to the group, no message will be send out if no userspace listeners has registered. Did you have a look at the syscall enter/exit audit netlink hooks before trying to invent your own thing? I can also give you some code if you want, I use it to track the path of skbs in the net stack. It puts events into a preallocated ring buffer and a separate kernel thread broadcasts them over netlink. The events can be enqueued in any context at the cost of a possible ring buffer overrun resulting in loss of events. It's just a debugging hack though. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hello,
Marcelo Tosatti wrote:
> Yep, the netlink people should be able to help - they known what would be
> required for not sending messages in case there is no listener registered.
>
> Maybe its already possible? I have never used netlink myself.
If we notify the fork/exec/exit-events to user-space directly as you said,
I don't think some hackings on netlink is necessary.
For example, such packets is sent only when /proc/sys/.../process_grouping is
set,
and user-side daemon set this value, and unset when daemon will exit.
It's not necessary to take too seriously.
>>And, why can't netlink packets send always?
>>If there are fork/exec/exit hooks, and they call CSA or other
>>process-grouping modules,
>>then those modules will decide whether packets for interaction with the
>>daemon should be
>>sent or not.
>
>
> The netlink data will be sent to userspace at fork/exec/exit hooks - one wants
> to avoid that if there are no listeners, so setups which dont want to run the
> accounting daemon dont pay the cost of building and sending the information
> through netlink.
>
> Thats what Andrew asked for if I understand correctly.
Does it mean "netlink packets shouled be sent to userspace unconditionally." ?
I have advocated steadfastly that fork/exec/exit hooks is necessary to support
process-grouping and to account per process-grouping.
It intend to be decided whether packets should be sent or not by hooked
functions,
in my understanding.
Is it also one of the implementations whether using netlink-socket or not ?
>>In most considerable case, CSA's kernel-loadable-module using such hooks
>>will not be loaded
>>when no accounting daemon is running. Adversely, this module must be loaded
>>when accounting
>>daemon needs CSA's netlink packets.
>>Thus, it is only necessary to refer flag valiable and to execute
>>conditional-jump
>>when no-accounting daemon is running.
>
>
> That would be one hack, although it is uglier than the pure netlink
> selection.
No, I can't agree this opinion.
It means netlink-packets will be sent unconditionally when fork/exec/exit occur.
Nobady can decide which packet is sent user-space, I think.
In addition, the definition of process grouping is lightweight in many cases.
For example, CpuSet can define own process-group by one increment-operation.
I think it's not impossible to implement it in userspace, but it's not
reasonable.
An implementation as a kernel loadable module is reasonable and enough tiny.
>>In my estimation, we must pay additional cost for an increment-operation,
>>an decrement-op,
>>an comparison-op and an conditional jump-op. It's enough lightweight, I
>>think.
>>
>>For example:
>>If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
>>
>>inline int on_fork_hook(task_struct *parent, task_struct *newtask){
>> rcu_read_lock();
>> if( !list_empty(>privates_for_grouping) ){
>>;
>> }
>> rcu_read_unlock();
>>}
>
>
> Andrew has been talking about sending data over netlink to implement the
> accounting at userspace, so this piece of code is out of the game, no?
Indeed, I'm not opposed to implement the accounting in userspace and
using netlink-socket for kernel-daemon communication. But definition
of process-grouping based on any grouping policy should be done
in kernel space at reasonability viewpoint.
Thanks.
--
Linux Promotion Center, NEC
KaiGai Kohei <[EMAIL PROTECTED]>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Sun, 27 Feb 2005 11:03:55 -0300 Marcelo Tosatti <[EMAIL PROTECTED]> wrote: > Yep, the netlink people should be able to help - they known what would be > required for not sending messages in case there is no listener registered. Please CC: netdev@oss.sgi.com to get some netlink discussions going if wanted. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, Feb 28, 2005 at 12:20:40AM +0900, KaiGai Kohei wrote:
> Hi,
>
> >>Kaigai Kohei <[EMAIL PROTECTED]> wrote:
> >>
> >>>In my understanding, what Andrew Morton said is "If target functionality
> >>>can
> >>>implement in user space only, then we should not modify the kernel-tree".
> >>
> >>fork, exec and exit upcalls sound pretty good to me. As long as
> >>
> >>a) they use the same common machinery and
> >>
> >>b) they are next-to-zero cost if something is listening on the netlink
> >> socket but no accounting daemon is running.
> >
> >
> >b) would involved being able to avoid sending netlink messages in case
> >there are no listeners. AFAIK that isnt possible currently, netlink sends
> >packets unconditionally.
> >
> >Am I wrong?
>
> In current implementaion, you might be right.
> But we should make an effort to achieve the requirement-(b) from now.
Yep, the netlink people should be able to help - they known what would be
required for not sending messages in case there is no listener registered.
Maybe its already possible? I have never used netlink myself.
> And, why can't netlink packets send always?
> If there are fork/exec/exit hooks, and they call CSA or other
> process-grouping modules,
> then those modules will decide whether packets for interaction with the
> daemon should be
> sent or not.
The netlink data will be sent to userspace at fork/exec/exit hooks - one wants
to avoid that if there are no listeners, so setups which dont want to run the
accounting daemon dont pay the cost of building and sending the information
through netlink.
Thats what Andrew asked for if I understand correctly.
> In most considerable case, CSA's kernel-loadable-module using such hooks
> will not be loaded
> when no accounting daemon is running. Adversely, this module must be loaded
> when accounting
> daemon needs CSA's netlink packets.
> Thus, it is only necessary to refer flag valiable and to execute
> conditional-jump
> when no-accounting daemon is running.
That would be one hack, although it is uglier than the pure netlink
selection.
> In my estimation, we must pay additional cost for an increment-operation,
> an decrement-op,
> an comparison-op and an conditional jump-op. It's enough lightweight, I
> think.
>
> For example:
> If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
>
> inline int on_fork_hook(task_struct *parent, task_struct *newtask){
> rcu_read_lock();
> if( !list_empty(>privates_for_grouping) ){
> ;
> }
> rcu_read_unlock();
> }
Andrew has been talking about sending data over netlink to implement the
accounting at userspace, so this piece of code is out of the game, no?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi,
Kaigai Kohei <[EMAIL PROTECTED]> wrote:
In my understanding, what Andrew Morton said is "If target functionality can
implement in user space only, then we should not modify the kernel-tree".
fork, exec and exit upcalls sound pretty good to me. As long as
a) they use the same common machinery and
b) they are next-to-zero cost if something is listening on the netlink
socket but no accounting daemon is running.
b) would involved being able to avoid sending netlink messages in case there are
no listeners. AFAIK that isnt possible currently, netlink sends
packets unconditionally.
Am I wrong?
In current implementaion, you might be right.
But we should make an effort to achieve the requirement-(b) from now.
And, why can't netlink packets send always?
If there are fork/exec/exit hooks, and they call CSA or other process-grouping
modules,
then those modules will decide whether packets for interaction with the daemon
should be
sent or not.
In most considerable case, CSA's kernel-loadable-module using such hooks will
not be loaded
when no accounting daemon is running. Adversely, this module must be loaded
when accounting
daemon needs CSA's netlink packets.
Thus, it is only necessary to refer flag valiable and to execute
conditional-jump
when no-accounting daemon is running.
In my estimation, we must pay additional cost for an increment-operation, an
decrement-op,
an comparison-op and an conditional jump-op. It's enough lightweight, I think.
For example:
If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
inline int on_fork_hook(task_struct *parent, task_struct *newtask){
rcu_read_lock();
if( !list_empty(>privates_for_grouping) ){
;
}
rcu_read_unlock();
}
Thanks,
--
Linux Promotion Center, NEC
KaiGai Kohei <[EMAIL PROTECTED]>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Thu, Feb 24, 2005 at 09:28:39PM -0800, Andrew Morton wrote: > Kaigai Kohei <[EMAIL PROTECTED]> wrote: > > > > In my understanding, what Andrew Morton said is "If target functionality can > > implement in user space only, then we should not modify the kernel-tree". > > fork, exec and exit upcalls sound pretty good to me. As long as > > a) they use the same common machinery and > > b) they are next-to-zero cost if something is listening on the netlink >socket but no accounting daemon is running. b) would involved being able to avoid sending netlink messages in case there are no listeners. AFAIK that isnt possible currently, netlink sends packets unconditionally. Am I wrong? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Thu, Feb 24, 2005 at 09:28:39PM -0800, Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. b) would involved being able to avoid sending netlink messages in case there are no listeners. AFAIK that isnt possible currently, netlink sends packets unconditionally. Am I wrong? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi,
Kaigai Kohei [EMAIL PROTECTED] wrote:
In my understanding, what Andrew Morton said is If target functionality can
implement in user space only, then we should not modify the kernel-tree.
fork, exec and exit upcalls sound pretty good to me. As long as
a) they use the same common machinery and
b) they are next-to-zero cost if something is listening on the netlink
socket but no accounting daemon is running.
b) would involved being able to avoid sending netlink messages in case there are
no listeners. AFAIK that isnt possible currently, netlink sends
packets unconditionally.
Am I wrong?
In current implementaion, you might be right.
But we should make an effort to achieve the requirement-(b) from now.
And, why can't netlink packets send always?
If there are fork/exec/exit hooks, and they call CSA or other process-grouping
modules,
then those modules will decide whether packets for interaction with the daemon
should be
sent or not.
In most considerable case, CSA's kernel-loadable-module using such hooks will
not be loaded
when no accounting daemon is running. Adversely, this module must be loaded
when accounting
daemon needs CSA's netlink packets.
Thus, it is only necessary to refer flag valiable and to execute
conditional-jump
when no-accounting daemon is running.
In my estimation, we must pay additional cost for an increment-operation, an
decrement-op,
an comparison-op and an conditional jump-op. It's enough lightweight, I think.
For example:
If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
inline int on_fork_hook(task_struct *parent, task_struct *newtask){
rcu_read_lock();
if( !list_empty(parent-privates_for_grouping) ){
..Calling to any process grouping module..;
}
rcu_read_unlock();
}
Thanks,
--
Linux Promotion Center, NEC
KaiGai Kohei [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, Feb 28, 2005 at 12:20:40AM +0900, KaiGai Kohei wrote:
Hi,
Kaigai Kohei [EMAIL PROTECTED] wrote:
In my understanding, what Andrew Morton said is If target functionality
can
implement in user space only, then we should not modify the kernel-tree.
fork, exec and exit upcalls sound pretty good to me. As long as
a) they use the same common machinery and
b) they are next-to-zero cost if something is listening on the netlink
socket but no accounting daemon is running.
b) would involved being able to avoid sending netlink messages in case
there are no listeners. AFAIK that isnt possible currently, netlink sends
packets unconditionally.
Am I wrong?
In current implementaion, you might be right.
But we should make an effort to achieve the requirement-(b) from now.
Yep, the netlink people should be able to help - they known what would be
required for not sending messages in case there is no listener registered.
Maybe its already possible? I have never used netlink myself.
And, why can't netlink packets send always?
If there are fork/exec/exit hooks, and they call CSA or other
process-grouping modules,
then those modules will decide whether packets for interaction with the
daemon should be
sent or not.
The netlink data will be sent to userspace at fork/exec/exit hooks - one wants
to avoid that if there are no listeners, so setups which dont want to run the
accounting daemon dont pay the cost of building and sending the information
through netlink.
Thats what Andrew asked for if I understand correctly.
In most considerable case, CSA's kernel-loadable-module using such hooks
will not be loaded
when no accounting daemon is running. Adversely, this module must be loaded
when accounting
daemon needs CSA's netlink packets.
Thus, it is only necessary to refer flag valiable and to execute
conditional-jump
when no-accounting daemon is running.
That would be one hack, although it is uglier than the pure netlink
selection.
In my estimation, we must pay additional cost for an increment-operation,
an decrement-op,
an comparison-op and an conditional jump-op. It's enough lightweight, I
think.
For example:
If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
inline int on_fork_hook(task_struct *parent, task_struct *newtask){
rcu_read_lock();
if( !list_empty(parent-privates_for_grouping) ){
..Calling to any process grouping module..;
}
rcu_read_unlock();
}
Andrew has been talking about sending data over netlink to implement the
accounting at userspace, so this piece of code is out of the game, no?
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Sun, 27 Feb 2005 11:03:55 -0300 Marcelo Tosatti [EMAIL PROTECTED] wrote: Yep, the netlink people should be able to help - they known what would be required for not sending messages in case there is no listener registered. Please CC: netdev@oss.sgi.com to get some netlink discussions going if wanted. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hello,
Marcelo Tosatti wrote:
Yep, the netlink people should be able to help - they known what would be
required for not sending messages in case there is no listener registered.
Maybe its already possible? I have never used netlink myself.
If we notify the fork/exec/exit-events to user-space directly as you said,
I don't think some hackings on netlink is necessary.
For example, such packets is sent only when /proc/sys/.../process_grouping is
set,
and user-side daemon set this value, and unset when daemon will exit.
It's not necessary to take too seriously.
And, why can't netlink packets send always?
If there are fork/exec/exit hooks, and they call CSA or other
process-grouping modules,
then those modules will decide whether packets for interaction with the
daemon should be
sent or not.
The netlink data will be sent to userspace at fork/exec/exit hooks - one wants
to avoid that if there are no listeners, so setups which dont want to run the
accounting daemon dont pay the cost of building and sending the information
through netlink.
Thats what Andrew asked for if I understand correctly.
Does it mean netlink packets shouled be sent to userspace unconditionally. ?
I have advocated steadfastly that fork/exec/exit hooks is necessary to support
process-grouping and to account per process-grouping.
It intend to be decided whether packets should be sent or not by hooked
functions,
in my understanding.
Is it also one of the implementations whether using netlink-socket or not ?
In most considerable case, CSA's kernel-loadable-module using such hooks
will not be loaded
when no accounting daemon is running. Adversely, this module must be loaded
when accounting
daemon needs CSA's netlink packets.
Thus, it is only necessary to refer flag valiable and to execute
conditional-jump
when no-accounting daemon is running.
That would be one hack, although it is uglier than the pure netlink
selection.
No, I can't agree this opinion.
It means netlink-packets will be sent unconditionally when fork/exec/exit occur.
Nobady can decide which packet is sent user-space, I think.
In addition, the definition of process grouping is lightweight in many cases.
For example, CpuSet can define own process-group by one increment-operation.
I think it's not impossible to implement it in userspace, but it's not
reasonable.
An implementation as a kernel loadable module is reasonable and enough tiny.
In my estimation, we must pay additional cost for an increment-operation,
an decrement-op,
an comparison-op and an conditional jump-op. It's enough lightweight, I
think.
For example:
If CSA's module isn't loaded, 'privates_for_grouping' will be empty.
inline int on_fork_hook(task_struct *parent, task_struct *newtask){
rcu_read_lock();
if( !list_empty(parent-privates_for_grouping) ){
..Calling to any process grouping module..;
}
rcu_read_unlock();
}
Andrew has been talking about sending data over netlink to implement the
accounting at userspace, so this piece of code is out of the game, no?
Indeed, I'm not opposed to implement the accounting in userspace and
using netlink-socket for kernel-daemon communication. But definition
of process-grouping based on any grouping policy should be done
in kernel space at reasonability viewpoint.
Thanks.
--
Linux Promotion Center, NEC
KaiGai Kohei [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
First of all, I'm not aware of the whole discussion, ignore this if it has been brought to attention already. Yep, the netlink people should be able to help - they known what would be required for not sending messages in case there is no listener registered. Maybe its already possible? I have never used netlink myself. The easiest way is to use netlink_broadcast() and have userspace register to a netlink multicast group (set .nl_groups before connecting the socket). The netlink message will be sent to only those netlink sockets assigned to the group, no message will be send out if no userspace listeners has registered. Did you have a look at the syscall enter/exit audit netlink hooks before trying to invent your own thing? I can also give you some code if you want, I use it to track the path of skbs in the net stack. It puts events into a preallocated ring buffer and a separate kernel thread broadcasts them over netlink. The events can be enqueued in any context at the cost of a possible ring buffer overrun resulting in loss of events. It's just a debugging hack though. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 10:59 +0900, Kaigai Kohei wrote: Hello, Marcelo Tosatti wrote: Yep, the netlink people should be able to help - they known what would be required for not sending messages in case there is no listener registered. Maybe its already possible? I have never used netlink myself. If we notify the fork/exec/exit-events to user-space directly as you said, I don't think some hackings on netlink is necessary. For example, such packets is sent only when /proc/sys/.../process_grouping is set, and user-side daemon set this value, and unset when daemon will exit. It's not necessary to take too seriously. Kernel accounting already was discussed in lkml week ago - I'm quite sure Guillaume Thouvenin created exactly that. His module creates do_fork() hook and broadcasts various process' states over netlink. Discussion at http://lkml.org/lkml/2005/2/17/87 -- Evgeniy Polyakov Crash is better than data corruption -- Arthur Grabowski signature.asc Description: This is a digitally signed message part
Re: [Lse-tech] Re: A common layer for Accounting packages
On Mon, 2005-02-28 at 10:59 +0900, Kaigai Kohei wrote:
Marcelo Tosatti wrote:
Yep, the netlink people should be able to help - they known what would be
required for not sending messages in case there is no listener registered.
Maybe its already possible? I have never used netlink myself.
If we notify the fork/exec/exit-events to user-space directly as you said,
I don't think some hackings on netlink is necessary.
For example, such packets is sent only when /proc/sys/.../process_grouping is
set,
and user-side daemon set this value, and unset when daemon will exit.
It's not necessary to take too seriously.
I wrote a new fork connector patch with a callback to enable/disable
messages in case there is or isn't listener. I will post it this week.
Basically there is a global variable that is manipulated with a
connector callback so a user space daemon can manipulate the variable.
In the fork_connector() function you have:
static inline void fork_connector(pid_t parent, pid_t child)
{
static DEFINE_SPINLOCK(cn_fork_lock);
static __u32 seq; /* used to test if message is lost */
if (cn_fork_enable) {
[...]
cn_netlink_send(msg, CN_IDX_FORK);
}
}
and in the cn_fork module (drivers/connector/cn_fork.c) the callback is
defined as:
static void cn_fork_callback(void *data)
{
if (cn_already_initialized)
cn_fork_enable = cn_fork_enable ? 0 : 1;
}
Ok the protocol is maybe too basic but with this mechanism the user
space application that uses the fork connector can start and stop the
send of messages. This implementation needs somme improvements because
currently, if two application are using the fork connector one can
enable it and the other don't know if it is enable or not, but the idea
is here I think.
Regards,
Guillaume
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin [EMAIL PROTECTED] wrote: Ok the protocol is maybe too basic but with this mechanism the user space application that uses the fork connector can start and stop the send of messages. This implementation needs somme improvements because currently, if two application are using the fork connector one can enable it and the other don't know if it is enable or not, but the idea is here I think. Yes. But this problem can be solved in userspace, with a little library function and a bit of locking. IOW: use the library to enable/disable the fork connector rather than directly doing syscalls. It has the problem that if a client of that library crashes, the counter gets out of whack, but really, it's not all _that_ important, and to handle this properly in-kernel each client would need an open fd against some object so we can do the close-on-exit thing properly. You'd need to create a separate netlink socket for the purpose. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew Morton wrote:
Jay Lan <[EMAIL PROTECTED]> wrote:
Andrew Morton wrote:
> Kaigai Kohei <[EMAIL PROTECTED]> wrote:
>
>>In my understanding, what Andrew Morton said is "If target functionality can
>> implement in user space only, then we should not modify the kernel-tree".
>
>
> fork, exec and exit upcalls sound pretty good to me. As long as
>
> a) they use the same common machinery and
>
> b) they are next-to-zero cost if something is listening on the netlink
>socket but no accounting daemon is running.
>
> Question is: is this sufficient for CSA?
Yes, fork, exec, and exit upcalls are sufficient for CSA.
The framework i proposed earlier should satisfy your requirement a
and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i
misunderstood your concern of the 'very light weight' framework
i proposed besides being "overkill"?
"upcall" is poorly defined.
What I meant was that ELSA can perform its function when the kernel merely
sends asynchronous notifications of forks out to userspace via netlink.
Further, I'm wondering if CSA can perform its function with the same level
of kernel support, perhaps with the addition of netlink-based notification
of exec and exit as well.
The framework patch which you sent was designed to permit the addition of
more kernel accounting code, which is heading in the opposite direction.
In other words: given that ELSA can do its thing via existing accounting
interfaces and a fork notifier, why does CSA need to add lots more kernel
code?
Here are some codes from do_exit() starting line 813 (based on
2.6.11-rc4-mm1):
813acct_update_integrals(tsk);
814update_mem_hiwater(tsk);
815group_dead = atomic_dec_and_test(>signal->live);
816if (group_dead) {
817del_timer_sync(>signal->real_timer);
818acct_process(code);
819}
820exit_mm(tsk);
The acct_process() is called to save off BSD accounting data at
line 818. The next statement at 820, tsk->mm is disposed and all
data saved at tsk->mm is gone, including memory hiwater marks
information saved at line 814. The complete tsk is disposed
before exit of do_exit() routine.
In separate emails discussion thread among interested parties,
i asked Guillaume to clarify this question. I suspect ELSA counts
on BSD's acct_process() at line 818 to save most accounting data.
If that is the case and since ELSA wants extended accounting data
collection, a way to save the extended acct data would be essential
to ELSA as well.
I can better asnwer your "why ELSA can do but CSA can't" question
after i learn more from Guilluame.
Later,
- jay
---
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595_id=14396=click
___
Lse-tech mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/lse-tech
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Jay Lan <[EMAIL PROTECTED]> wrote: > > Andrew Morton wrote: > > Kaigai Kohei <[EMAIL PROTECTED]> wrote: > > > >>In my understanding, what Andrew Morton said is "If target functionality > can > >> implement in user space only, then we should not modify the kernel-tree". > > > > > > fork, exec and exit upcalls sound pretty good to me. As long as > > > > a) they use the same common machinery and > > > > b) they are next-to-zero cost if something is listening on the netlink > >socket but no accounting daemon is running. > > > > Question is: is this sufficient for CSA? > > Yes, fork, exec, and exit upcalls are sufficient for CSA. > > The framework i proposed earlier should satisfy your requirement a > and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i > misunderstood your concern of the 'very light weight' framework > i proposed besides being "overkill"? "upcall" is poorly defined. What I meant was that ELSA can perform its function when the kernel merely sends asynchronous notifications of forks out to userspace via netlink. Further, I'm wondering if CSA can perform its function with the same level of kernel support, perhaps with the addition of netlink-based notification of exec and exit as well. The framework patch which you sent was designed to permit the addition of more kernel accounting code, which is heading in the opposite direction. In other words: given that ELSA can do its thing via existing accounting interfaces and a fork notifier, why does CSA need to add lots more kernel code? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Chris Wright wrote: * Jay Lan ([EMAIL PROTECTED]) wrote: Andrew Morton wrote: Kaigai Kohei <[EMAIL PROTECTED]> wrote: In my understanding, what Andrew Morton said is "If target functionality can implement in user space only, then we should not modify the kernel-tree". fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. As soon as you want to throttle tasks at the Job level, this would be insufficient. But, IIRC, that's not one of PAGG/Job/CSA's requirements right? PAGG serves more than JOB+CSA. I am looking into possiblity/feasibility of implementing JOB at userspace. However, even with JOB as a kernel module, the fork, exec and exit upcalls would be sufficient to support JOB+CSA. Thanks, - jay thanks, -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* Jay Lan ([EMAIL PROTECTED]) wrote: > Andrew Morton wrote: > >Kaigai Kohei <[EMAIL PROTECTED]> wrote: > > > >>In my understanding, what Andrew Morton said is "If target functionality > >>can > >>implement in user space only, then we should not modify the kernel-tree". > > > > > >fork, exec and exit upcalls sound pretty good to me. As long as > > > >a) they use the same common machinery and > > > >b) they are next-to-zero cost if something is listening on the netlink > > socket but no accounting daemon is running. > > > >Question is: is this sufficient for CSA? > > Yes, fork, exec, and exit upcalls are sufficient for CSA. As soon as you want to throttle tasks at the Job level, this would be insufficient. But, IIRC, that's not one of PAGG/Job/CSA's requirements right? thanks, -chris -- Linux Security Modules http://lsm.immunix.org http://lsm.bkbits.net - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew Morton wrote: Kaigai Kohei <[EMAIL PROTECTED]> wrote: In my understanding, what Andrew Morton said is "If target functionality can implement in user space only, then we should not modify the kernel-tree". fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. The framework i proposed earlier should satisfy your requirement a and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i misunderstood your concern of the 'very light weight' framework i proposed besides being "overkill"? - jay - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. The framework i proposed earlier should satisfy your requirement a and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i misunderstood your concern of the 'very light weight' framework i proposed besides being overkill? - jay - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
* Jay Lan ([EMAIL PROTECTED]) wrote: Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. As soon as you want to throttle tasks at the Job level, this would be insufficient. But, IIRC, that's not one of PAGG/Job/CSA's requirements right? thanks, -chris -- Linux Security Modules http://lsm.immunix.org http://lsm.bkbits.net - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Chris Wright wrote: * Jay Lan ([EMAIL PROTECTED]) wrote: Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. As soon as you want to throttle tasks at the Job level, this would be insufficient. But, IIRC, that's not one of PAGG/Job/CSA's requirements right? PAGG serves more than JOB+CSA. I am looking into possiblity/feasibility of implementing JOB at userspace. However, even with JOB as a kernel module, the fork, exec and exit upcalls would be sufficient to support JOB+CSA. Thanks, - jay thanks, -chris - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Jay Lan [EMAIL PROTECTED] wrote: Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? Yes, fork, exec, and exit upcalls are sufficient for CSA. The framework i proposed earlier should satisfy your requirement a and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i misunderstood your concern of the 'very light weight' framework i proposed besides being overkill? upcall is poorly defined. What I meant was that ELSA can perform its function when the kernel merely sends asynchronous notifications of forks out to userspace via netlink. Further, I'm wondering if CSA can perform its function with the same level of kernel support, perhaps with the addition of netlink-based notification of exec and exit as well. The framework patch which you sent was designed to permit the addition of more kernel accounting code, which is heading in the opposite direction. In other words: given that ELSA can do its thing via existing accounting interfaces and a fork notifier, why does CSA need to add lots more kernel code? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Andrew Morton wrote:
Jay Lan [EMAIL PROTECTED] wrote:
Andrew Morton wrote:
Kaigai Kohei [EMAIL PROTECTED] wrote:
In my understanding, what Andrew Morton said is If target functionality can
implement in user space only, then we should not modify the kernel-tree.
fork, exec and exit upcalls sound pretty good to me. As long as
a) they use the same common machinery and
b) they are next-to-zero cost if something is listening on the netlink
socket but no accounting daemon is running.
Question is: is this sufficient for CSA?
Yes, fork, exec, and exit upcalls are sufficient for CSA.
The framework i proposed earlier should satisfy your requirement a
and b, and provides upcalls needed by BSD, ELSA and CSA. Maybe i
misunderstood your concern of the 'very light weight' framework
i proposed besides being overkill?
upcall is poorly defined.
What I meant was that ELSA can perform its function when the kernel merely
sends asynchronous notifications of forks out to userspace via netlink.
Further, I'm wondering if CSA can perform its function with the same level
of kernel support, perhaps with the addition of netlink-based notification
of exec and exit as well.
The framework patch which you sent was designed to permit the addition of
more kernel accounting code, which is heading in the opposite direction.
In other words: given that ELSA can do its thing via existing accounting
interfaces and a fork notifier, why does CSA need to add lots more kernel
code?
Here are some codes from do_exit() starting line 813 (based on
2.6.11-rc4-mm1):
813acct_update_integrals(tsk);
814update_mem_hiwater(tsk);
815group_dead = atomic_dec_and_test(tsk-signal-live);
816if (group_dead) {
817del_timer_sync(tsk-signal-real_timer);
818acct_process(code);
819}
820exit_mm(tsk);
The acct_process() is called to save off BSD accounting data at
line 818. The next statement at 820, tsk-mm is disposed and all
data saved at tsk-mm is gone, including memory hiwater marks
information saved at line 814. The complete tsk is disposed
before exit of do_exit() routine.
In separate emails discussion thread among interested parties,
i asked Guillaume to clarify this question. I suspect ELSA counts
on BSD's acct_process() at line 818 to save most accounting data.
If that is the case and since ELSA wants extended accounting data
collection, a way to save the extended acct data would be essential
to ELSA as well.
I can better asnwer your why ELSA can do but CSA can't question
after i learn more from Guilluame.
Later,
- jay
---
SF email is sponsored by - The IT Product Guide
Read honest candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595alloc_id=14396op=click
___
Lse-tech mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/lse-tech
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Kaigai Kohei <[EMAIL PROTECTED]> wrote: > > In my understanding, what Andrew Morton said is "If target functionality can > implement in user space only, then we should not modify the kernel-tree". fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Sorry for this late reply. >> [1] Is it necessary 'fork/exec/exit' event handling framework ? .. >> Some process-aggregation model have own philosophy and implemantation, >> so it's hard to integrate. Thus, I think that common 'fork/exec/exit' >> event handling >> framework to implement any kinds of process-aggregation. > > > BSD needs an exit hook and ELSA needs a fork hook. I am still > evaluating whether CSA can use the ELSA module. If CSA can use the > ELSA module, CSA maybe would be fine with the fork hook. If CSA can use an ELSA module, then we must modify the kernel-tree for ELSA's fork-connecter. This means it's hard to implement the fork/exec/exit event notification to userspace (,or any kernel module) without kernel-support. How CSA shoule be implemented is interesting and important, but should it be main subject in this discussion that such a kinds of kernel hook is necessary to implement process-accounting per process-aggregation reasonable ? In my understanding, what Andrew Morton said is "If target functionality can implement in user space only, then we should not modify the kernel-tree". But, any kind of kernel support was required to handle process lifecycle events for the accounting per process-aggregation and so on, from our discussion. I'm also opposed to an adhoc approach, like CSA depending on ELSA. We should walk hight road. Thanks, -- Linux Promotion Center, NEC KaiGai Kohei <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Tim Schmielau wrote: On Tue, 22 Feb 2005, Andrew Morton wrote: We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. (In fact, why do we even need the notifications? /bin/ps can work this stuff out). I had started a proof of concept implementation that could reconstruct the whole process tree from userspace just from the BSD accounting currently in the kernel (+ the conceptual bug-fix that I misnamed "[RFC] "biological parent" pid"). This could do the whole job ID thing from userspace. Unfortunately, I haven't had time to work on it recently. Also, doing per-job accounting might actually be more lightweight than per-process accounting, so I'm not at all opposed to unifying CSA and BSD accounting into one mechanism that just writes different file formats. Thanks, Tim! After spending some time studying how ELSA works, it appeared to me that CSA still needs a hook for do_exit. Since people agreed that a complete framework was an overkill, i would be glad to submit another patch later just to provide a CSA exit-handling inside the acct_process(). Thanks, - jay A complete framework seems like overkill to me, too. Tim --- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595_id=14396=click ___ Lse-tech mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/lse-tech - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Tim Schmielau wrote: On Tue, 22 Feb 2005, Andrew Morton wrote: We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. (In fact, why do we even need the notifications? /bin/ps can work this stuff out). I had started a proof of concept implementation that could reconstruct the whole process tree from userspace just from the BSD accounting currently in the kernel (+ the conceptual bug-fix that I misnamed [RFC] biological parent pid). This could do the whole job ID thing from userspace. Unfortunately, I haven't had time to work on it recently. Also, doing per-job accounting might actually be more lightweight than per-process accounting, so I'm not at all opposed to unifying CSA and BSD accounting into one mechanism that just writes different file formats. Thanks, Tim! After spending some time studying how ELSA works, it appeared to me that CSA still needs a hook for do_exit. Since people agreed that a complete framework was an overkill, i would be glad to submit another patch later just to provide a CSA exit-handling inside the acct_process(). Thanks, - jay A complete framework seems like overkill to me, too. Tim --- SF email is sponsored by - The IT Product Guide Read honest candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595alloc_id=14396op=click ___ Lse-tech mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/lse-tech - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Sorry for this late reply. [1] Is it necessary 'fork/exec/exit' event handling framework ? ...ommited... Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. BSD needs an exit hook and ELSA needs a fork hook. I am still evaluating whether CSA can use the ELSA module. If CSA can use the ELSA module, CSA maybe would be fine with the fork hook. If CSA can use an ELSA module, then we must modify the kernel-tree for ELSA's fork-connecter. This means it's hard to implement the fork/exec/exit event notification to userspace (,or any kernel module) without kernel-support. How CSA shoule be implemented is interesting and important, but should it be main subject in this discussion that such a kinds of kernel hook is necessary to implement process-accounting per process-aggregation reasonable ? In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. But, any kind of kernel support was required to handle process lifecycle events for the accounting per process-aggregation and so on, from our discussion. I'm also opposed to an adhoc approach, like CSA depending on ELSA. We should walk hight road. Thanks, -- Linux Promotion Center, NEC KaiGai Kohei [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Kaigai Kohei [EMAIL PROTECTED] wrote: In my understanding, what Andrew Morton said is If target functionality can implement in user space only, then we should not modify the kernel-tree. fork, exec and exit upcalls sound pretty good to me. As long as a) they use the same common machinery and b) they are next-to-zero cost if something is listening on the netlink socket but no accounting daemon is running. Question is: is this sufficient for CSA? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-02-23 at 11:11 -0800, Jay Lan wrote: > Guillaume Thouvenin wrote: > > It's what I'm proposing. The problem is to be alerted when a new process > > is created in order to add it in the correct group of processes if the > > parent belongs to one (or several) groups. The notification can be done > > with the fork connector patch. > > I am not quite comfortable of ELSA requesting a fork hook this way. > How many hooks in the stock kernel that are related to accounting? Can > anyone answer this question? I know of 'acct_process()' in exit.c used > by the BSD accounting and ELSA is requesting a hook in fork. If people > raise the same question again a few years later, how many people will > still remember this ELSA hook? The fork connector is not related to accounting. It's a connector that allows to send information to a user space application when a fork occurs in the kernel. This information is used by ELSA by I think that this hook will be used by some others user space applications and IMHO, it's not incompatible with a specific hook for accounting tool if needed. Guillaume - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Jay wrote: > I think the microbenchmarking your link provides is irrelevant. In the cases such as you describe where it's just some sort of empty function call, then yes, I am willing to accept a wave of the hands and a simple explanation of how it's not significant. I've done the same myself ;). What about the case where accounting is enabled, and thus actually has to do work? How does that compare with just doing the traditional BSD accounting? I presume in that case that the benchmarking is no longer irrelevant. Though if you can make a decent case that it is, I'm willing to listen. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <[EMAIL PROTECTED]> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi Paul,
I think the microbenchmarking your link provides is irrelevant.
Your link provides benchmarking of doing a fork.
However, we are talking about inserting a callback routine
in a fork and/or an exit. The overhead is a function
call and time spent in the routine. The callback routine
can be configured to "do {} while (0)" if a certain CONFIG
flag is not set.
Thanks,
- jay
Paul Jackson wrote:
So, I think such a fork/execve/exit hooks is harmless now.
I don't recall seeing any microbenchmarking of the impact on fork/exit
of such hooks. You might find such a benchmark in lmbench, or at
http://bulk.fefe.de/scalability/.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
> So, I think such a fork/execve/exit hooks is harmless now. I don't recall seeing any microbenchmarking of the impact on fork/exit of such hooks. You might find such a benchmark in lmbench, or at http://bulk.fefe.de/scalability/. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <[EMAIL PROTECTED]> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Kaigai Kohei wrote:
Hi, Thanks for your comments.
>> I think there are two issues about system accounting framework.
>>
>> Issue: 1) How to define the appropriate unit for accounting ?
>> Current BSD-accountiong make a collection per process accounting
>> information.
>> CSA make additionally a collection per process-aggregation accounting.
>
>
> The 'enhanced acct data collection' patches that were added to
> 2-6-11-rc* tree still do collection of per process data.
Hmm, I have not noticed this extension. But I made sure about it.
The following your two patches implements enhanced data collection,
didn't it?
Yes!
- ChangeLog for 2.6.11-rc1
[PATCH] enhanced I/O accounting data patch
[PATCH] enhanced Memory accounting data collection
Since making a collection per process accounting is unified to the stock
kernel,
I want to have a discussion about remaining half, "How to define the
appropriate
unit for accounting ?"
We can agree that only per process-accounting is so rigid, I think.
Then, process-aggregation should be provided in one way or another.
[1] Is it necessary 'fork/exec/exit' event handling framework ?
The common agreement for the method of dealing with process aggregation
has not been constructed yet, I understood. And, we will not able to
integrate each process aggregation model because of its diverseness.
For example, a process which belong to JOB-A must not belong any other
'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can
concurrently
belong to BANK-B1 which is a child of BANK-B.
And, there are other defferences:
Whether a process not to belong to any process-aggregation is permitted
or not ?
Whether a process-aggregation should be inherited to child process or not ?
(There is possibility not to be inherited in a rule-based process
aggregation like CKRM)
Guillaume answered this question, and i think a policy would work.
Some process-aggregation model have own philosophy and implemantation,
so it's hard to integrate. Thus, I think that common 'fork/exec/exit'
event handling
framework to implement any kinds of process-aggregation.
BSD needs an exit hook and ELSA needs a fork hook. I am still
evaluating whether CSA can use the ELSA module. If CSA can use the
ELSA module, CSA maybe would be fine with the fork hook.
[2] What implementation should be adopted ?
I think registerable hooks on fork/execve/exit is necessary, not only
exit() hook.
Because a rule or policy based process-aggregation model requirees to catch
the transition of a process status.
It might be enough to hook the exit() event only in process-accounting,
but it's not kind for another customer.
Thus, I recommend SGI's PAGG.
In my understanding, the reason for not to include such a framework is that
increase of unidentifiable (proprietary) modules is worried.
If we code the hooks explicitly in the kernel, such as in acct.c,
then the concern of unidentifiable modules should be taken care of.
A registerable framework was my preference. But if that causes
concern, it would be fine for me to do it explicit way. An
example is for acct_process() to invoke do_acct_process().
That means whoever intends to use even an existing hook needs to
present their cases, i guess.
Thanks,
- jay
But, SI can divert LSM to implemente process-aggregation if they ignore
the LSM's original purpose, for example.
# I'm strongly opposed to such a movement as a SELinux's user :-)
So, I think such a fork/execve/exit hooks is harmless now.
Is this the time to unify it?
Thanks.
> CSA added those per-process data to per-aggregation ("job") data
> structure at do_exit() time when a process termintes.
>
>>
>> It is appropriate to make the fork-exit event handling framework for
>> definition
>> of the process-aggregation, such as PAGG.
>>
>> This system-accounting per process-aggregation is quite useful,
>> thought I tried the SGI's implementation named 'job' in past days.
>>
>>
>> Issue: 2) What items should be collected for accounting information ?
>> BSD-accounting collects PID/UID/GID, User/Sys/Elapsed-Time, and # of
>> minor/major page faults. SGI's CSA collects VM/RSS size on exit time,
>> Integral-VM/RSS, and amount of block-I/O additionally.
>
>
> These data are now collected in 2.6.11-rc* code. Note that these data
> are still per-process.
>
>>
>> I think it's hard to implement the accounting-engine as a kernel
loadable
>> module using any kinds of framework. Because, we must put callback
>> functions
>> into all around the kernel for this purpose.
>>
>> Thus, I make a proposion as follows:
>> We should separate the process-aggregation functionality and collecting
>> accounting informations.
>
>
> I totally agree with this! Actually that was what we have done. The data
> collection part of code has been unified.
>
>> Something of framework to implement process-aggregation is necessary.
>> And, making a collection of accounting information should be merged
>> into BSD-accounting and
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin wrote: On Tue, 2005-02-22 at 23:20 -0800, Andrew Morton wrote: Kaigai Kohei <[EMAIL PROTECTED]> wrote: The common agreement for the method of dealing with process aggregation has not been constructed yet, I understood. And, we will not able to integrate each process aggregation model because of its diverseness. For example, a process which belong to JOB-A must not belong any other 'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can concurrently belong to BANK-B1 which is a child of BANK-B. And, there are other defferences: Whether a process not to belong to any process-aggregation is permitted or not ? Whether a process-aggregation should be inherited to child process or not ? (There is possibility not to be inherited in a rule-based process aggregation like CKRM) Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. I can add "policies". With ELSA, a process belongs to one or several groups and if a process is removed from one group, its children still belong to the group. Thus a good idea could be to associate a "philosophy" to a group. For exemple, when a group of processes is created it can be tagged as UNIQUE or SHARED. UNIQUE means that a process that belongs to it could not be added in another group by opposition to SHARED. It's not needed inside the kernel. This makes sense to me. CSA can use the UNIQUE policy to enforce its "can't escape from job container" philisophy. We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. I am not quite comfortable of ELSA requesting a fork hook this way. How many hooks in the stock kernel that are related to accounting? Can anyone answer this question? I know of 'acct_process()' in exit.c used by the BSD accounting and ELSA is requesting a hook in fork. If people raise the same question again a few years later, how many people will still remember this ELSA hook? That was the reason i thought a central piece was a good idea. I would rather see the fork hook is coded in acct.c and then invokes a routine that handles what ELSA needs. If CSA would adopt the ELSA's daemon's approach, CSA may also need to use the fork hook. Actually the acct_process() was modified not long ago to become a wrapper, which then invokes do_acct_process() which is completely BSD specific. The fork hook can be the same. - jay (In fact, why do we even need the notifications? /bin/ps can work this stuff out). Yes it can but the risk is to lose some forks no? I think that /bin/ps is using the /proc interface. If we're polling the /proc to catch process creation we may lost some of them. With the fork connector we catch all forks and we can check that by using the sequence number (incremented by each fork) of the message. Guillaume - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi, Thanks for your comments. Andrew Morton wrote: >> Some process-aggregation model have own philosophy and implemantation, >> so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling >> framework to implement any kinds of process-aggregation. > > > We really want to avoid doing such stuff in-kernel if at all possible, of > course. > > Is it not possible to implement the fork/exec/exit notifications to > userspace so that a daemon can track the process relationships and perform > aggregation based upon individual tasks' accounting? That's what one of > the accounting systems is proposing doing, I believe. > > (In fact, why do we even need the notifications? /bin/ps can work this > stuff out). It's hard to prove that we can't implement the process-aggregation only in user-space, but there are some difficulties on imaplementation, I think. For example, each process must have a tag or another identifier to explain what process-aggregation does it belong, but kernel does not support thoes kind of information, currently. Thus, we can't guarantee associating one process-aggregation with one process. # /proc//loginuid might be candidate, but it's out of original purpose. We might be able to make alike system, but is it hard to implement strict process-aggregation without any kernel supports? I think that well thought out kernel-modification is better than ad-hoc implementation on user-space. Thanks. -- Linux Promotion Center, NEC KaiGai Kohei <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 22 Feb 2005, Andrew Morton wrote: > We really want to avoid doing such stuff in-kernel if at all possible, of > course. > > Is it not possible to implement the fork/exec/exit notifications to > userspace so that a daemon can track the process relationships and perform > aggregation based upon individual tasks' accounting? That's what one of > the accounting systems is proposing doing, I believe. > > (In fact, why do we even need the notifications? /bin/ps can work this > stuff out). I had started a proof of concept implementation that could reconstruct the whole process tree from userspace just from the BSD accounting currently in the kernel (+ the conceptual bug-fix that I misnamed "[RFC] "biological parent" pid"). This could do the whole job ID thing from userspace. Unfortunately, I haven't had time to work on it recently. Also, doing per-job accounting might actually be more lightweight than per-process accounting, so I'm not at all opposed to unifying CSA and BSD accounting into one mechanism that just writes different file formats. A complete framework seems like overkill to me, too. Tim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > I will run benchmarks found at http://bulk.fefe.de/scalability/ to see > how the fork connector impacts on the kernel. The lmbench fork microbenchmark would suffice. >All stuff that was previously done in kernel space and provided by the > 2.6.8.1 ELSA patch has been moved in the ELSA user space daemon called > "jobd". Excellent. Will it work? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-02-23 at 00:51 -0800, Andrew Morton wrote: > > It's what I'm proposing. The problem is to be alerted when a new process > > is created in order to add it in the correct group of processes if the > > parent belongs to one (or several) groups. The notification can be done > > with the fork connector patch. > > Yes, it sounds sane. > > The 2.6.8.1 ELSA patch adds quite a bit of kernel code, but from what > you're saying it seems like most of that has become redundant, and all > you now need is the fork notifier. Is that correct? Yes, that's correct. All I need is the fork connector patch. It needs more work like, as you said, sending an on/off message down the netlink socket. I'm working on this (thank you very much Andrew for your comments). I will run benchmarks found at http://bulk.fefe.de/scalability/ to see how the fork connector impacts on the kernel. All stuff that was previously done in kernel space and provided by the 2.6.8.1 ELSA patch has been moved in the ELSA user space daemon called "jobd". Best, Guillaume - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin <[EMAIL PROTECTED]> wrote: > > ... > > > We really want to avoid doing such stuff in-kernel if at all possible, of > > course. > > > > Is it not possible to implement the fork/exec/exit notifications to > > userspace so that a daemon can track the process relationships and perform > > aggregation based upon individual tasks' accounting? That's what one of > > the accounting systems is proposing doing, I believe. > > It's what I'm proposing. The problem is to be alerted when a new process > is created in order to add it in the correct group of processes if the > parent belongs to one (or several) groups. The notification can be done > with the fork connector patch. Yes, it sounds sane. The 2.6.8.1 ELSA patch adds quite a bit of kernel code, but from what you're saying it seems like most of that has become redundant, and all you now need is the fork notifier. Is that correct? That 2.6.8.1 ELSA patch looks reasonable to me - it only adds two lines to generic code and the rest looks pretty straightforward. Are we sure that this level of functionality is not sufficient for everyone else? > > (In fact, why do we even need the notifications? /bin/ps can work this > > stuff out). > > Yes it can but the risk is to lose some forks no? > I think that /bin/ps is using the /proc interface. If we're polling > the /proc to catch process creation we may lost some of them. With the > fork connector we catch all forks and we can check that by using the > sequence number (incremented by each fork) of the message. Oh, I wasn't proposing that it all be done via existing /proc interfaces - I was just getting my head straight ;) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-02-22 at 23:20 -0800, Andrew Morton wrote: > Kaigai Kohei <[EMAIL PROTECTED]> wrote: > > > > The common agreement for the method of dealing with process aggregation > > has not been constructed yet, I understood. And, we will not able to > > integrate each process aggregation model because of its diverseness. > > > > For example, a process which belong to JOB-A must not belong any other > > 'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can > > concurrently > > belong to BANK-B1 which is a child of BANK-B. > > > > And, there are other defferences: > > Whether a process not to belong to any process-aggregation is permitted or > > not ? > > Whether a process-aggregation should be inherited to child process or not ? > > (There is possibility not to be inherited in a rule-based process > > aggregation like CKRM) > > > > Some process-aggregation model have own philosophy and implemantation, > > so it's hard to integrate. Thus, I think that common 'fork/exec/exit' > > event handling > > framework to implement any kinds of process-aggregation. I can add "policies". With ELSA, a process belongs to one or several groups and if a process is removed from one group, its children still belong to the group. Thus a good idea could be to associate a "philosophy" to a group. For exemple, when a group of processes is created it can be tagged as UNIQUE or SHARED. UNIQUE means that a process that belongs to it could not be added in another group by opposition to SHARED. It's not needed inside the kernel. > We really want to avoid doing such stuff in-kernel if at all possible, of > course. > > Is it not possible to implement the fork/exec/exit notifications to > userspace so that a daemon can track the process relationships and perform > aggregation based upon individual tasks' accounting? That's what one of > the accounting systems is proposing doing, I believe. It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. > (In fact, why do we even need the notifications? /bin/ps can work this > stuff out). Yes it can but the risk is to lose some forks no? I think that /bin/ps is using the /proc interface. If we're polling the /proc to catch process creation we may lost some of them. With the fork connector we catch all forks and we can check that by using the sequence number (incremented by each fork) of the message. Guillaume - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 2005-02-22 at 23:20 -0800, Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: The common agreement for the method of dealing with process aggregation has not been constructed yet, I understood. And, we will not able to integrate each process aggregation model because of its diverseness. For example, a process which belong to JOB-A must not belong any other 'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can concurrently belong to BANK-B1 which is a child of BANK-B. And, there are other defferences: Whether a process not to belong to any process-aggregation is permitted or not ? Whether a process-aggregation should be inherited to child process or not ? (There is possibility not to be inherited in a rule-based process aggregation like CKRM) Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. I can add policies. With ELSA, a process belongs to one or several groups and if a process is removed from one group, its children still belong to the group. Thus a good idea could be to associate a philosophy to a group. For exemple, when a group of processes is created it can be tagged as UNIQUE or SHARED. UNIQUE means that a process that belongs to it could not be added in another group by opposition to SHARED. It's not needed inside the kernel. We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. (In fact, why do we even need the notifications? /bin/ps can work this stuff out). Yes it can but the risk is to lose some forks no? I think that /bin/ps is using the /proc interface. If we're polling the /proc to catch process creation we may lost some of them. With the fork connector we catch all forks and we can check that by using the sequence number (incremented by each fork) of the message. Guillaume - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin [EMAIL PROTECTED] wrote: ... We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. Yes, it sounds sane. The 2.6.8.1 ELSA patch adds quite a bit of kernel code, but from what you're saying it seems like most of that has become redundant, and all you now need is the fork notifier. Is that correct? That 2.6.8.1 ELSA patch looks reasonable to me - it only adds two lines to generic code and the rest looks pretty straightforward. Are we sure that this level of functionality is not sufficient for everyone else? (In fact, why do we even need the notifications? /bin/ps can work this stuff out). Yes it can but the risk is to lose some forks no? I think that /bin/ps is using the /proc interface. If we're polling the /proc to catch process creation we may lost some of them. With the fork connector we catch all forks and we can check that by using the sequence number (incremented by each fork) of the message. Oh, I wasn't proposing that it all be done via existing /proc interfaces - I was just getting my head straight ;) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-02-23 at 00:51 -0800, Andrew Morton wrote: It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. Yes, it sounds sane. The 2.6.8.1 ELSA patch adds quite a bit of kernel code, but from what you're saying it seems like most of that has become redundant, and all you now need is the fork notifier. Is that correct? Yes, that's correct. All I need is the fork connector patch. It needs more work like, as you said, sending an on/off message down the netlink socket. I'm working on this (thank you very much Andrew for your comments). I will run benchmarks found at http://bulk.fefe.de/scalability/ to see how the fork connector impacts on the kernel. All stuff that was previously done in kernel space and provided by the 2.6.8.1 ELSA patch has been moved in the ELSA user space daemon called jobd. Best, Guillaume - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin [EMAIL PROTECTED] wrote: I will run benchmarks found at http://bulk.fefe.de/scalability/ to see how the fork connector impacts on the kernel. The lmbench fork microbenchmark would suffice. All stuff that was previously done in kernel space and provided by the 2.6.8.1 ELSA patch has been moved in the ELSA user space daemon called jobd. Excellent. Will it work? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Tue, 22 Feb 2005, Andrew Morton wrote: We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. (In fact, why do we even need the notifications? /bin/ps can work this stuff out). I had started a proof of concept implementation that could reconstruct the whole process tree from userspace just from the BSD accounting currently in the kernel (+ the conceptual bug-fix that I misnamed [RFC] biological parent pid). This could do the whole job ID thing from userspace. Unfortunately, I haven't had time to work on it recently. Also, doing per-job accounting might actually be more lightweight than per-process accounting, so I'm not at all opposed to unifying CSA and BSD accounting into one mechanism that just writes different file formats. A complete framework seems like overkill to me, too. Tim - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi, Thanks for your comments. Andrew Morton wrote: Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. (In fact, why do we even need the notifications? /bin/ps can work this stuff out). It's hard to prove that we can't implement the process-aggregation only in user-space, but there are some difficulties on imaplementation, I think. For example, each process must have a tag or another identifier to explain what process-aggregation does it belong, but kernel does not support thoes kind of information, currently. Thus, we can't guarantee associating one process-aggregation with one process. # /proc/uid/loginuid might be candidate, but it's out of original purpose. We might be able to make alike system, but is it hard to implement strict process-aggregation without any kernel supports? I think that well thought out kernel-modification is better than ad-hoc implementation on user-space. Thanks. -- Linux Promotion Center, NEC KaiGai Kohei [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Guillaume Thouvenin wrote: On Tue, 2005-02-22 at 23:20 -0800, Andrew Morton wrote: Kaigai Kohei [EMAIL PROTECTED] wrote: The common agreement for the method of dealing with process aggregation has not been constructed yet, I understood. And, we will not able to integrate each process aggregation model because of its diverseness. For example, a process which belong to JOB-A must not belong any other 'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can concurrently belong to BANK-B1 which is a child of BANK-B. And, there are other defferences: Whether a process not to belong to any process-aggregation is permitted or not ? Whether a process-aggregation should be inherited to child process or not ? (There is possibility not to be inherited in a rule-based process aggregation like CKRM) Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. I can add policies. With ELSA, a process belongs to one or several groups and if a process is removed from one group, its children still belong to the group. Thus a good idea could be to associate a philosophy to a group. For exemple, when a group of processes is created it can be tagged as UNIQUE or SHARED. UNIQUE means that a process that belongs to it could not be added in another group by opposition to SHARED. It's not needed inside the kernel. This makes sense to me. CSA can use the UNIQUE policy to enforce its can't escape from job container philisophy. We really want to avoid doing such stuff in-kernel if at all possible, of course. Is it not possible to implement the fork/exec/exit notifications to userspace so that a daemon can track the process relationships and perform aggregation based upon individual tasks' accounting? That's what one of the accounting systems is proposing doing, I believe. It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. I am not quite comfortable of ELSA requesting a fork hook this way. How many hooks in the stock kernel that are related to accounting? Can anyone answer this question? I know of 'acct_process()' in exit.c used by the BSD accounting and ELSA is requesting a hook in fork. If people raise the same question again a few years later, how many people will still remember this ELSA hook? That was the reason i thought a central piece was a good idea. I would rather see the fork hook is coded in acct.c and then invokes a routine that handles what ELSA needs. If CSA would adopt the ELSA's daemon's approach, CSA may also need to use the fork hook. Actually the acct_process() was modified not long ago to become a wrapper, which then invokes do_acct_process() which is completely BSD specific. The fork hook can be the same. - jay (In fact, why do we even need the notifications? /bin/ps can work this stuff out). Yes it can but the risk is to lose some forks no? I think that /bin/ps is using the /proc interface. If we're polling the /proc to catch process creation we may lost some of them. With the fork connector we catch all forks and we can check that by using the sequence number (incremented by each fork) of the message. Guillaume - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Kaigai Kohei wrote: Hi, Thanks for your comments. I think there are two issues about system accounting framework. Issue: 1) How to define the appropriate unit for accounting ? Current BSD-accountiong make a collection per process accounting information. CSA make additionally a collection per process-aggregation accounting. The 'enhanced acct data collection' patches that were added to 2-6-11-rc* tree still do collection of per process data. Hmm, I have not noticed this extension. But I made sure about it. The following your two patches implements enhanced data collection, didn't it? Yes! - ChangeLog for 2.6.11-rc1 [PATCH] enhanced I/O accounting data patch [PATCH] enhanced Memory accounting data collection Since making a collection per process accounting is unified to the stock kernel, I want to have a discussion about remaining half, How to define the appropriate unit for accounting ? We can agree that only per process-accounting is so rigid, I think. Then, process-aggregation should be provided in one way or another. [1] Is it necessary 'fork/exec/exit' event handling framework ? The common agreement for the method of dealing with process aggregation has not been constructed yet, I understood. And, we will not able to integrate each process aggregation model because of its diverseness. For example, a process which belong to JOB-A must not belong any other 'JOB-X' in CSA-model. But, In ELSA-model, a process in BANK-B can concurrently belong to BANK-B1 which is a child of BANK-B. And, there are other defferences: Whether a process not to belong to any process-aggregation is permitted or not ? Whether a process-aggregation should be inherited to child process or not ? (There is possibility not to be inherited in a rule-based process aggregation like CKRM) Guillaume answered this question, and i think a policy would work. Some process-aggregation model have own philosophy and implemantation, so it's hard to integrate. Thus, I think that common 'fork/exec/exit' event handling framework to implement any kinds of process-aggregation. BSD needs an exit hook and ELSA needs a fork hook. I am still evaluating whether CSA can use the ELSA module. If CSA can use the ELSA module, CSA maybe would be fine with the fork hook. [2] What implementation should be adopted ? I think registerable hooks on fork/execve/exit is necessary, not only exit() hook. Because a rule or policy based process-aggregation model requirees to catch the transition of a process status. It might be enough to hook the exit() event only in process-accounting, but it's not kind for another customer. Thus, I recommend SGI's PAGG. In my understanding, the reason for not to include such a framework is that increase of unidentifiable (proprietary) modules is worried. If we code the hooks explicitly in the kernel, such as in acct.c, then the concern of unidentifiable modules should be taken care of. A registerable framework was my preference. But if that causes concern, it would be fine for me to do it explicit way. An example is for acct_process() to invoke do_acct_process(). That means whoever intends to use even an existing hook needs to present their cases, i guess. Thanks, - jay But, SI can divert LSM to implemente process-aggregation if they ignore the LSM's original purpose, for example. # I'm strongly opposed to such a movement as a SELinux's user :-) So, I think such a fork/execve/exit hooks is harmless now. Is this the time to unify it? Thanks. CSA added those per-process data to per-aggregation (job) data structure at do_exit() time when a process termintes. It is appropriate to make the fork-exit event handling framework for definition of the process-aggregation, such as PAGG. This system-accounting per process-aggregation is quite useful, thought I tried the SGI's implementation named 'job' in past days. Issue: 2) What items should be collected for accounting information ? BSD-accounting collects PID/UID/GID, User/Sys/Elapsed-Time, and # of minor/major page faults. SGI's CSA collects VM/RSS size on exit time, Integral-VM/RSS, and amount of block-I/O additionally. These data are now collected in 2.6.11-rc* code. Note that these data are still per-process. I think it's hard to implement the accounting-engine as a kernel loadable module using any kinds of framework. Because, we must put callback functions into all around the kernel for this purpose. Thus, I make a proposion as follows: We should separate the process-aggregation functionality and collecting accounting informations. I totally agree with this! Actually that was what we have done. The data collection part of code has been unified. Something of framework to implement process-aggregation is necessary. And, making a collection of accounting information should be merged into BSD-accounting and implemented as a part of monolithic kernel as Guillaume said. This sounds
Re: [Lse-tech] Re: A common layer for Accounting packages
So, I think such a fork/execve/exit hooks is harmless now. I don't recall seeing any microbenchmarking of the impact on fork/exit of such hooks. You might find such a benchmark in lmbench, or at http://bulk.fefe.de/scalability/. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson [EMAIL PROTECTED] 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Hi Paul,
I think the microbenchmarking your link provides is irrelevant.
Your link provides benchmarking of doing a fork.
However, we are talking about inserting a callback routine
in a fork and/or an exit. The overhead is a function
call and time spent in the routine. The callback routine
can be configured to do {} while (0) if a certain CONFIG
flag is not set.
Thanks,
- jay
Paul Jackson wrote:
So, I think such a fork/execve/exit hooks is harmless now.
I don't recall seeing any microbenchmarking of the impact on fork/exit
of such hooks. You might find such a benchmark in lmbench, or at
http://bulk.fefe.de/scalability/.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
Jay wrote: I think the microbenchmarking your link provides is irrelevant. In the cases such as you describe where it's just some sort of empty function call, then yes, I am willing to accept a wave of the hands and a simple explanation of how it's not significant. I've done the same myself ;). What about the case where accounting is enabled, and thus actually has to do work? How does that compare with just doing the traditional BSD accounting? I presume in that case that the benchmarking is no longer irrelevant. Though if you can make a decent case that it is, I'm willing to listen. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson [EMAIL PROTECTED] 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Lse-tech] Re: A common layer for Accounting packages
On Wed, 2005-02-23 at 11:11 -0800, Jay Lan wrote: Guillaume Thouvenin wrote: It's what I'm proposing. The problem is to be alerted when a new process is created in order to add it in the correct group of processes if the parent belongs to one (or several) groups. The notification can be done with the fork connector patch. I am not quite comfortable of ELSA requesting a fork hook this way. How many hooks in the stock kernel that are related to accounting? Can anyone answer this question? I know of 'acct_process()' in exit.c used by the BSD accounting and ELSA is requesting a hook in fork. If people raise the same question again a few years later, how many people will still remember this ELSA hook? The fork connector is not related to accounting. It's a connector that allows to send information to a user space application when a fork occurs in the kernel. This information is used by ELSA by I think that this hook will be used by some others user space applications and IMHO, it's not incompatible with a specific hook for accounting tool if needed. Guillaume - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
