Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
> Let me get on the record. It's basically ridiculous to allow GCC 4.8 > to redefine the set of permitted C expressions such that it breaks > BSD. gcc 4.8 isn't. C99 did; what's distinctive about gcc 4.8 is that before that gcc didn't take advantage of the leeway C99 said it had. These macros have been out-of-spec since the day NetBSD decided it was going to use C99 rather than some older version of C; it's just that only now is that out-of-spec-ness actually biting anyone. (That decision may have been implicit in a compiler version change.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
Let me get on the record. It's basically ridiculous to allow GCC 4.8 to
redefine the
set of permitted C expressions such that it breaks BSD.
Matt
- "Christos Zoulas" wrote:
> On Nov 26, 8:20pm, m...@linuxbox.com ("Matt W. Benjamin") wrote:
> -- Subject: Re: in which we present an ugly hack to make sys/queue.h
> CIRCLEQ
>
> | What's your issue with TAILQ?
>
> #define TAILQ_LAST(head, headname) \
> (*(((struct headname *)((head)->tqh_last))->tqh_last))
> #define TAILQ_PREV(elm, headname, field) \
> (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
>
>
> etc.
>
> christos
--
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104
http://linuxbox.com
tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309
Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
On Nov 26, 8:20pm, m...@linuxbox.com ("Matt W. Benjamin") wrote:
-- Subject: Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ
| What's your issue with TAILQ?
#define TAILQ_LAST(head, headname) \
(*(((struct headname *)((head)->tqh_last))->tqh_last))
#define TAILQ_PREV(elm, headname, field) \
(*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
etc.
christos
Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
What's your issue with TAILQ? Matt - "Christos Zoulas" wrote: > In article <96497888-f8c7-49f2-958b-532a2093b...@gmail.com>, > Dennis Ferguson wrote: > > >I think getting rid of uses of the CIRCLEQ macros was the right > thing > >to do in any case, since code which works like that doesn't need to > >exist. I'm not sure that that TAILQ macros are the best answer > >to the problem, though. > > Unfortunately I agree... But in the absence of a better > alternative... > > christos -- Matt Benjamin The Linux Box 206 South Fifth Ave. Suite 150 Ann Arbor, MI 48104 http://linuxbox.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309
Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
In article <96497888-f8c7-49f2-958b-532a2093b...@gmail.com>, Dennis Ferguson wrote: >I think getting rid of uses of the CIRCLEQ macros was the right thing >to do in any case, since code which works like that doesn't need to >exist. I'm not sure that that TAILQ macros are the best answer >to the problem, though. Unfortunately I agree... But in the absence of a better alternative... christos
Re: in which we present an ugly hack to make sys/queue.h CIRCLEQ work
On 24 Nov, 2013, at 03:37 , Mouse wrote: >> [The compiler] also couldn't know if pointers whose types it did know >> were referring to different members of the same union, perhaps with >> the union declared in another compilation unit > > The text I have says > > [#5] One special guarantee is made in order to simplify the > use of unions: if a union contains several structures that > share a common initial sequence (see below), and if the > union object currently contains one of these structures, it > is permitted to inspect the common initial part of any of > them anywhere that a declaration of the completed type of > the union is visible. Two structures share a common initial > sequence if corresponding members have compatible types > (and, for bit-fields, the same widths) for a sequence of one > or more initial members. > > Note that the completed union declaration must be visible for this > exemption to apply, so as I read it the part after the last comma of > the quote I opened this mail with is wrong Ahh, I misremembered that. Since this misunderstanding was the basis for thinking it might be okay to access objects through different structure pointers as long as the member types matched what was stored, and since I know of no other part of the standard which would allow this, I can see why the compiler could choose to treat this as illegal when no union is visible. I guess that means the TAILQ macros really are an accident waiting to happen. That also makes it clear that, in the CIRCLEQ case, when the pointers compare equal then any use of both the pointers to access members would be out-of-spec aliasing. The comparison being deleted is in fact surrounded by code which uses the both pointers, so I can see why the compiler might be suspicious enough to warn about it, but the only actual aliased access is in a branch of the if()s the comparison would be correctly avoiding if the code were generated. It seems like the optimizer has moved from requiring that code not do anything which isn't explicitly allowed by the standard (the TAILQ case) to requiring code not do things which the standard explicitly allows but which may, in some circumstance, not be perfectly portable. I think getting rid of uses of the CIRCLEQ macros was the right thing to do in any case, since code which works like that doesn't need to exist. I'm not sure that that TAILQ macros are the best answer to the problem, though. Dennis Ferguson
Re: posix message queues and multiple receivers
>> Why do you think it is meant to connect only two processes? It is >> [...] just a FIFO queue of messages. [...] > So what is the purpose of those interface? When I inject a message, > I don't know which of the possibly many receivers is getting it? Right. To rephrase that, "when I make a request, I don't know which worker process will service it". But you also don't _care_ which. Consider nfsd: it typically has multiple worker processes (four by default); when a client sends a request, it doesn't know which worker will handle it, nor does it care. (It typically uses sockets, but they are the same in this respect: each message goes to exactly one reader, no matter how mnay readers are reading.) > I somewhat fail to understand the utility of more than one receivers. When serving a request takes nontrivial time, and multiple requests can usefully be in progress at once, it is useful - it typically improves performance - to have multiple workers serving requests. NFS, as mentioned above, is a fairly good example (in these respects). /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: [patch] put Lua standard libraries into the kernel
On Tue, Nov 26, 2013 at 9:50 AM, Marc Balmer wrote: > Am 26.11.13 12:13, schrieb Lourival Vieira Neto: >> Hi Marc, >> >> On Tue, Nov 26, 2013 at 6:18 AM, Marc Balmer wrote: >>> Am 26.11.13 02:50, schrieb Lourival Vieira Neto: Hi Folks, Here is a patch that puts some Lua standard libraries into the kernel: - Auxiliary library (C API); - Base library; - String library; - Table library. >>> >>> In the kernel, Lua states are created empty _on purpose_. So the Lua >>> standard library should be a module (or modules) in kernel space. >> >> Why? >> >> Note, the Lua states still empty, but, with this patch, you can call >> luaL_openlibs(). >> >>> Whether all standard libraries should be one module, or rather multiple >>> kernel modules, is to be discussed. >> >> Let's discuss then =). > > My suggestion is this: > > Build with lauxlib and also build the standard libraries > Create a new sysctl kern.lua.stdlib, set to one by default > If a new state is created in lua(4), run lua_openlib(...) on that state > if kern.lua.stdlib is set to a vaue != 0 > > So by default a kernel Lua state would then have the stdlibs available, > much you have them available when running lua(1). It is just fine for me =). Regards, -- Lourival Vieira Neto
Re: posix message queues and multiple receivers
Marc Balmer wrote: > Am 26.11.13 15:13, schrieb Mindaugas Rasiukevicius: > > Hi, > > > > The question is not really kernel related. Possibly tech-userlevel@, > > but neither it is related with NetBSD per se. > > I asked here because it is implemented in the kernel and because what I > see might as well be a buglet (given that aio does not work either as > expected). No, this is exactly how a message queue works (whether it is POSIX, SysV or some other implementation). > > Why do you think it is meant to connect only two processes? It is an > > asynchronous inter-process communication mechanism, it is just a FIFO > > queue of messages. To expand what Martin said, you can have multiple > > producers and multiple consumers (M:N, not only 1:N or M:1) since it > > really depends on what are you build on top of this interface. > > So what is the purpose of those interface? When I inject a message, I > don't know which of the possibly many receivers is getting it? > > I somewhat fail to understand the utility of more than one receivers. > I think Martin already explained.. Imagine you just want to get a message, process it (parse, decrypt, transform, perhaps consume multiple messages and aggregate them, or whatever you want to do) and either pass to a next component in the system or perhaps store it (write to disk, memory, or some database). Instead of processing the messages in a single-threaded manner, you can spawn multiple workers (receivers) and process them in parallel. -- Mindaugas
Re: posix message queues and multiple receivers
Am 26.11.13 15:13, schrieb Mindaugas Rasiukevicius: > Hi, > > The question is not really kernel related. Possibly tech-userlevel@, > but neither it is related with NetBSD per se. I asked here because it is implemented in the kernel and because what I see might as well be a buglet (given that aio does not work either as expected). > Marc Balmer wrote: >> What is the prupose or reasoning behind the fact that multiple processes >> can open a message queue for reading using mq_open()? >> >> I wrote simple mq sender and mq receiver programs; when I start multiple >> receivers on the same mq, and send a message to it, only one of the >> receivers gets the message, in a round robin fashion. That is probably >> by design, but if a mq is meant to connect only two processes, why can >> more than two processes open it? > > Why do you think it is meant to connect only two processes? It is an > asynchronous inter-process communication mechanism, it is just a FIFO > queue of messages. To expand what Martin said, you can have multiple > producers and multiple consumers (M:N, not only 1:N or M:1) since it > really depends on what are you build on top of this interface. So what is the purpose of those interface? When I inject a message, I don't know which of the possibly many receivers is getting it? I somewhat fail to understand the utility of more than one receivers. > > These are basic IPC concepts. I would suggest to Google for "POSIX > message queue" or just check the Wikipedia page first. We also have > a pretty good mqueue(3) manual page. And you really think I did not read the man pages and other docs before? I am almost spending the whole trying to find out more about posix queues...
Re: posix message queues and multiple receivers
Hi, The question is not really kernel related. Possibly tech-userlevel@, but neither it is related with NetBSD per se. Marc Balmer wrote: > What is the prupose or reasoning behind the fact that multiple processes > can open a message queue for reading using mq_open()? > > I wrote simple mq sender and mq receiver programs; when I start multiple > receivers on the same mq, and send a message to it, only one of the > receivers gets the message, in a round robin fashion. That is probably > by design, but if a mq is meant to connect only two processes, why can > more than two processes open it? Why do you think it is meant to connect only two processes? It is an asynchronous inter-process communication mechanism, it is just a FIFO queue of messages. To expand what Martin said, you can have multiple producers and multiple consumers (M:N, not only 1:N or M:1) since it really depends on what are you build on top of this interface. These are basic IPC concepts. I would suggest to Google for "POSIX message queue" or just check the Wikipedia page first. We also have a pretty good mqueue(3) manual page. -- Mindaugas
Re: [patch] put Lua standard libraries into the kernel
Am 26.11.13 12:13, schrieb Lourival Vieira Neto: > Hi Marc, > > On Tue, Nov 26, 2013 at 6:18 AM, Marc Balmer wrote: >> Am 26.11.13 02:50, schrieb Lourival Vieira Neto: >>> Hi Folks, >>> >>> Here is a patch that puts some Lua standard libraries into the kernel: >>> >>> - Auxiliary library (C API); >>> - Base library; >>> - String library; >>> - Table library. >> >> In the kernel, Lua states are created empty _on purpose_. So the Lua >> standard library should be a module (or modules) in kernel space. > > Why? > > Note, the Lua states still empty, but, with this patch, you can call > luaL_openlibs(). > >> Whether all standard libraries should be one module, or rather multiple >> kernel modules, is to be discussed. > > Let's discuss then =). My suggestion is this: Build with lauxlib and also build the standard libraries Create a new sysctl kern.lua.stdlib, set to one by default If a new state is created in lua(4), run lua_openlib(...) on that state if kern.lua.stdlib is set to a vaue != 0 So by default a kernel Lua state would then have the stdlibs available, much you have them available when running lua(1). > >> But please don't load the automatically. > > Do you mean 'don't link Lua stdlib with Lua'? What is the reason? > > Regards, >
Re: [patch] put Lua standard libraries into the kernel
Hi Marc, On Tue, Nov 26, 2013 at 6:18 AM, Marc Balmer wrote: > Am 26.11.13 02:50, schrieb Lourival Vieira Neto: >> Hi Folks, >> >> Here is a patch that puts some Lua standard libraries into the kernel: >> >> - Auxiliary library (C API); >> - Base library; >> - String library; >> - Table library. > > In the kernel, Lua states are created empty _on purpose_. So the Lua > standard library should be a module (or modules) in kernel space. Why? Note, the Lua states still empty, but, with this patch, you can call luaL_openlibs(). > Whether all standard libraries should be one module, or rather multiple > kernel modules, is to be discussed. Let's discuss then =). > But please don't load the automatically. Do you mean 'don't link Lua stdlib with Lua'? What is the reason? Regards, -- Lourival Vieira Neto
Re: [patch] using luaL_register() on luacore kernel module
On Tue, Nov 26, 2013 at 6:22 AM, Marc Balmer wrote: > Am 26.11.13 03:26, schrieb Lourival Vieira Neto: >> Hi again.. >> >> Just a tiny patch to use luaL_register() on luacore kernel module. >> >> Regards > > You are now using the auxiliary library, which I did not include in > kernel Lua on prupose (to keep stuff smaller). It doesn't seemed reasonable. Lua standard libraries don't add significant footprint to Lua kernel module. In fact, on am64, both versions (with or without Lua stdlib) have 240K. > So what is the gain of including the Lua auxiliary library in kernel > space Lua? I think it should be discussed for five minutes. Using luaL_register() instead of reinvent it, just pays the price for me (which is 0KB =). Also, I think that Lua is really less useful without standard libraries. Regards, -- Lourival Vieira Neto
Re: posix message queues and multiple receivers
On Tue, Nov 26, 2013 at 09:39:44AM +0100, Marc Balmer wrote: > What is the prupose or reasoning behind the fact that multiple processes > can open a message queue for reading using mq_open()? You can dispatch messages from one producer to several workers (one writer, multiple readers), or inject work for a single worker from various clients (one reader, multiple writers). It is just a mesage queue, not a multiplexor or whatever you are thinking of. A single message stays a single message and gets delivered once. Martin
posix message queues and multiple receivers
What is the prupose or reasoning behind the fact that multiple processes can open a message queue for reading using mq_open()? I wrote simple mq sender and mq receiver programs; when I start multiple receivers on the same mq, and send a message to it, only one of the receivers gets the message, in a round robin fashion. That is probably by design, but if a mq is meant to connect only two processes, why can more than two processes open it?
Re: [patch] using luaL_register() on luacore kernel module
Am 26.11.13 03:26, schrieb Lourival Vieira Neto: > Hi again.. > > Just a tiny patch to use luaL_register() on luacore kernel module. > > Regards You are now using the auxiliary library, which I did not include in kernel Lua on prupose (to keep stuff smaller). So what is the gain of including the Lua auxiliary library in kernel space Lua? I think it should be discussed for five minutes. (My approach was to use only the bare minimum of Lua, that is also the reason why I did not include the standard libraries)
Re: [patch] put Lua standard libraries into the kernel
Am 26.11.13 02:50, schrieb Lourival Vieira Neto: > Hi Folks, > > Here is a patch that puts some Lua standard libraries into the kernel: > > - Auxiliary library (C API); > - Base library; > - String library; > - Table library. In the kernel, Lua states are created empty _on purpose_. So the Lua standard library should be a module (or modules) in kernel space. Whether all standard libraries should be one module, or rather multiple kernel modules, is to be discussed. But please don't load the automatically.
