Re: Lua in-kernel (lbuf library)
First, I find the usage of the buf terminology confusing. In kernel
context I associate buf with the file system buffe cache buf structure.
Packet buffers a called mbufs. I would appreciate it if the terminology
was consistent with the kernel or at least not confusing.
Also, having to switch mentally between zero-based arrays in the kernel C
code and 1-based arrays in the Lua code make my head ache.
On Thu, Oct 10, 2013 at 03:15:54PM -0300, Lourival Vieira Neto wrote:
C API:
lbuf_new(lua_State L, void * buffer, size_t length, lua_Alloc free, bool net);
* creates a new lbuf userdatum and pushes it on the Lua stack. The net
flag indicates if it is necessary to perform endianness conversion.
I what is buffer and how does it relate to mbufs? How do I create a new
lbuf from an mbuf? Or from an array of bytes?
In order to indicate that endianness conversion is necessary I need to
know the future uses of the buffer. Clairvoyance excepted, that is kinda
hard.
If you are going to make the buffers endianness aware, why not record the
endianness that the packet is encoded in. And byteswapping can be
performed automatically depending on the consumers endianness. I think
this way a lot of redundant code can be avoided.
And you don't describe under what circumstances endianness convresion is
performed.
Lua API:
- array access (1)
lbuf:mask(alignment [, offset, length])
buf[ix] ~ accesses 'alignment' bits from 'alignment*(ix -1)+offset' position
e.g.:
buf:mask(3)
buf[3] ~ accesses 3 bits from bit-6 position
What does that mean? Does it return the top-most 2 bits from the first
byte plus the least significant bit fom the second byte of the buffer?
What is 'length' for?
How does endianness conversion fit in?
- array access (2)
buf:mask{ length_pos1, length_pos2, ... }
buf[ix] ~ accesses 'length_pos(ix)' bits from 'length_pos1 + ...
length_pos(ix-1)' position
e.g.:
buf:mask{ 2, 2, 32, 9 }
buf[2] ~ accesses 2 bits from bit-2 position
What exactly would buf[3] return. Please be explicit in whether you are
counting byte offsets or bit offsets. I can't figure that out from your
description.
Personally, the idea of making array access to the buffer depend on
state stored in the buffer does not look appealing to me. It prevents
buffers to be passed around because consumers don't know what they will
get back on array access.
buf:mask{ field = { offset, length }, ... }
buf.field ~ 'field.length' bits from 'offset' position
This actually makes some sense to me.
buf:segment(offset [, length])
returns a new lbuf corresponding a 'buf' segment.
What is a a 'segment' actually?
- mask reusing
lbuf.mask{ ... }
This makes sense again...
function filter(packet)
packet:mask(ethernet_mask)
if packet.type == 0x88CC then
lldp_pdu = packet.segment(payload_offset):mask(lldp_mask)
if packet.version 1 return DROP end
end
return PASS
end
... except the code seems to be not runnable. Where does 'payload_offset'
come from? And don't you mean lldp_pdu.version?
I find it not helpful when the examples do not actually work.
--chris
Re: Moving Lua source codes
Taylor, I found out that several programs already have Lua bindings, but for unknown reasons that code is not enabled by default, which is kind of weird given that the Lua library is in base... So we have Lua bindings in our code, whe have Lua as well, but yet we don't enable it and continue to whine about nothing using Lua, that's weirder than weird ;) Bindings do not an application make. For years, many people have been asking you for real, working Lua code that (a) serves some nontrivial purpose beyond mapping between the C world and the Lua world, and (b) has a compelling reason to rely on Lua in base rather than Lua in pkgsrc. But so far, all I've seen is a few bindings to C libraries. Lua is a scripting language in the original sense of the word, a language to be embedded in a hosting program, to make this program extensible. As such, Lua bindings in hosting programs are key. But not only there, it also important to provide access to key functionality in the form of Lua bindings. That is why Lua modules like sqlite, pgsql, json, gpio, or whatever exist. Lua provides an easy to use, easy to learn, yet powerful interface to the system. It is a tool not only for developers, but also for users of NetBSD. It's like a shell: We provide it to our users so that they can do magic with it. It's a tool. httpd(8) uses Lua by default. You can use Lua to create dynamic content, like you do with a CGI script, but the startup time is faster. You can access SQLite3 database files from it, creating e.g. a fast blog or whatever web application. Lua is a tool, it's your imagination that sets the limits. It is entirely plausible to me that we could benefit from using Lua in base, or sysinst, or maybe even in the kernel. But that argument must be made by showing evidence of real, working code that has compelling benefits, together with confidence in its robustness -- not by saying that if we let users do it then it will happen. There is real word, real working code. In userland and in kernel space. There are developers waiting for the kernel parts to be committet, so they can continue their work as well. I myself use Lua since years in the kernel, sysinst, and now also in httpd(8). I did not commit all uses of Lua, because they are very specific to _our_ (i.e. my company) uses. And that is what Lua is about: I can use it to taylor (no pun intended) a certain piece of software to my needs. I agree that the sysinst Lua integration should be commited. We use Lua in sysinst since ever to install pkgsrc packages during install time, btw. I use Lua in the kernel to simulate certain types of devices. Why don't you experiment with some more radical changes for Lua in a branch, mbalmer-lua? You needn't worry about anyone else's approval, you can break anything you want, and you can let others review and play with your work. There is nothing radical about Lua. In userland we already have it, no need for a branch. Use it, if you like. Adding Lua to your software is easy and straightforward and will not bloat it. In the kernel space, all of Lua we did so far are loadable modules, no existing kernel subsystem needed to be changed source code wise. So it's perfectly self contained and does not interfere with the existing kernel code. Again no branch needed. Kernel mode Lua is a few kernel modules plus a utility luactl(8) to control matters from userland. A new tty line discipline attachment program will probably make use of kernel mode Lua. I see no need for a branch. A branch would probably only mean that no one will ever look at this code anyways. -current is where development should take place. And I am careful enough that this development does not interfere with the work of others. - Marc
Re: Moving Lua source codes
Am 11.10.13 09:19, schrieb David Holland: lua(4) comes with examples. Such as? - Several examples om how to use kernel mode Lua, like how to provide and use modules, call module functions etc. - tty(4) line disciplines written in Lua, decoding serial data - Obscure GPIO stuff that I did not plan to commit, because it only helps me and my oscilloscope... - How to write extension modules (C bindings) and making them available to kernel mode Lua
Re: Moving Lua source codes
-current is where development should take place. I disagree. That way - doing development in the master tree - lies the madness that has given Linux some of its worst problems. Development should take place on branches, or, preferably, outside the master tree entirely. When something is developed and is past initial testing, _then_ is the time to bring it into the tree. For example: AF_TIMER sockets. I developed them on my systems, never going near NetBSD's main tree with them. I'd've suggested bringing them in long since, except I think they're not ready. Not, as always, that anyone has to agree with me, or care what I think /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Moving Lua source codes
On Mon, 14 Oct 2013, Mouse wrote: -current is where development should take place. I disagree. That way - doing development in the master tree - lies the madness that has given Linux some of its worst problems. Development should take place on branches, or, preferably, outside the master tree entirely. When something is developed and is past initial testing, _then_ is the time to bring it into the tree. For example: AF_TIMER sockets. I developed them on my systems, never going near NetBSD's main tree with them. I'd've suggested bringing them in long since, except I think they're not ready. Not, as always, that anyone has to agree with me, or care what I think I agree with Mouse. It seems to me that this is why branches exist. - | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com| | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net | | Kernel Developer | | pgoyette at netbsd.org | -
