Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Date: Sat, 19 Oct 2013 09:47:14 +0200 From: Marc Balmer m...@msys.ch And now to give you a practical example what I personally do with lua(4) right now: In the past I wrote several tty line disciplines to decode various serial formats. Now I have a need for that again. Doing this in C is of course possible, but I want something more dynamic. So I wrote a tty line discipline that uses Lua to do all the decoding. That works like a charm: Load the script, test, change the script and reload. Really practical. I will release this code once I sorted out a few remaining details. And in the course of this work, I also found deficencies in slattach(8). Please show us the code. It doesn't have to be perfect, but it has to be there in order to make a case.
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Sat, 19 Oct 2013, Marc Balmer wrote: The inclusion and use of Lua in base, for use in userland and the kernel, [...] has, last but not least, core's blessing. Would you please either present some evidence for that claim, or stop making the claim. To the best of my knowledge, userland Lua was approved by core in 2010, but kernel Lua has never been approved by core. Can we now please stop this useless discussion? People will continue to ask questions until they receive some satisfactory answers. --apb (Alan Barrett)
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Fri, 18 Oct 2013, Lourival Vieira Neto wrote: I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. Yes. But I'm talking about interesting *user* work. I'm not claiming that they should be in the kernel. I'm just saying that, IMHO, we should incorporate a small device driver that facilitates this kind of development (outside the tree). You seem to want the lua device driver to be inside the tree, to facilitate experimental work outside the tree. Other people have asked why the lua(4) device driver itself can't be developed outside the tree (with a view to importing it later, if it ever proves to be more than an experiment), and I have seen no good answer to that. --apb (Alan Barrett)
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Am 19.10.13 09:03, schrieb Alan Barrett: On Sat, 19 Oct 2013, Marc Balmer wrote: The inclusion and use of Lua in base, for use in userland and the kernel, [...] has, last but not least, core's blessing. Would you please either present some evidence for that claim, or stop making the claim. I am not making a claim. And what is this, a trial, that you ask me to present evidence? You were not a core team member at the time, so I really can't blame you that you don't remember it. But I blame you for making this up as if it was sweeping kernel change or so. It's a tiny device driver that uses source code that is already in the tree since about three years. I will eventually dig out the email exchange, but that will have to wait, I am at a trade show right now. To the best of my knowledge, userland Lua was approved by core in 2010, but kernel Lua has never been approved by core. Can we now please stop this useless discussion? People will continue to ask questions until they receive some satisfactory answers. Me and others are giving answers. It just seem so that it is not the answers some people want to hear. What Lua is good for as been discussed many times, there are presentation slides around etc.
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Am 19.10.13 09:12, schrieb Alan Barrett: On Fri, 18 Oct 2013, Lourival Vieira Neto wrote: I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. Yes. But I'm talking about interesting *user* work. I'm not claiming that they should be in the kernel. I'm just saying that, IMHO, we should incorporate a small device driver that facilitates this kind of development (outside the tree). You seem to want the lua device driver to be inside the tree, to facilitate experimental work outside the tree. Other people have asked why the lua(4) device driver itself can't be developed outside the tree (with a view to importing it later, if it ever proves to be more than an experiment), and I have seen no good answer to that. I can answer that. Lua, the library in userland, the bindings to libararies like sqlite or gpio, and lua(4) plus luactl(8) serve a specific purpose, namely to allow NetBSD _users_ to work with the system in an easy to learn and more dynamic approach. They are tools for users. This has been the goal that I stated in almost any talk about the subject matter. It's like gpio: That is also something for users. Hasn't Unix alwas provided the tools, but it was up to the user to do with them what he wanted? Lua is a scripting language in the original sense of the word: It allows a user to script or change the behaviour of software. It is not even seen as a freestanding language by it's inventors, but rather a hosted language, a small library to be incorporated in other software to make it extensible and more dynamic. Yes, there is a lua command, but that is merely an example use of the Lua library (as stated by the Lua developers). More and more people that were aware of the benefits of Lua, both in userspace, but also in the form of a driver, have pushed me to finally commit this. Also from a marketing perspective it's a good thing (and I speaking as a member of marketing@ now): It makes NetBSD more attractive for newcomers, for hardware tinkerers and such. The can now use their Soekris or Alix board and use GPIO lines to do play with hardware. I showed NetBSD with Lua at a local ham (radio amateur) meeting and they were thrilled. A sofar Linux only community all of a sudden created new NetBSD users. And that is good. There will always be folks who don't like Lua for various reasons. Unfortunately they are very vocal and sometimes I do think that they not always understand what Lua is and what it's general goals are. They should just not use it, then. And now to give you a practical example what I personally do with lua(4) right now: In the past I wrote several tty line disciplines to decode various serial formats. Now I have a need for that again. Doing this in C is of course possible, but I want something more dynamic. So I wrote a tty line discipline that uses Lua to do all the decoding. That works like a charm: Load the script, test, change the script and reload. Really practical. I will release this code once I sorted out a few remaining details. And in the course of this work, I also found deficencies in slattach(8). In previous work I used Lua to create a software gpio device, a modified version of gpiosim(4) that uses a Lua script to mimick a real device. Also handy. In one email thread I once specifically said that Lua should not necesseraly be used to write system software, e.g. startup scripts or so. Even when you see no use for Lua for your personal work, I hope that you understand that there are users that have a valid and sound use for Lua. Or do you really thing we do a GSoC project, go to conferences, talk about, bring new developers on board just to annoy anyone? No, it's because we (the Lua users) want it, not for the sake of it, but to work with it. - Marc
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Oct 19, 2013, at 12:26 AM, Marc Balmer m...@msys.ch wrote: Am 19.10.13 09:03, schrieb Alan Barrett: On Sat, 19 Oct 2013, Marc Balmer wrote: The inclusion and use of Lua in base, for use in userland and the kernel, [...] has, last but not least, core's blessing. Would you please either present some evidence for that claim, or stop making the claim. I am not making a claim. And what is this, a trial, that you ask me to present evidence? You were not a core team member at the time, so I really can't blame you that you don't remember it. But I blame you for making this up as if it was sweeping kernel change or so. It's a tiny device driver that uses source code that is already in the tree since about three years. I will eventually dig out the email exchange, but that will have to wait, I am at a trade show right now. Well, I've been on core a lot longer (over a decade now) and I don't remember approving in-kernel Lua either. I checked my mail archives. The relevant mail is from around October 24th, 2010. The only kernel references are for things like exec_script support and to make sure userland Lua does not conflict with the kernel Lua from his [Lourival Neto] GSoC project. That strongly implies that you were only asking for userland lua support and that's what core granted permission for. Looking through past mail, it saddens me to note that the bozohttpd changes took nearly 4 years to get into the tree.
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Sat, 19 Oct 2013, Marc Balmer wrote: And now to give you a practical example what I personally do with lua(4) right now: In the past I wrote several tty line disciplines to decode various serial formats. Now I have a need for that again. Doing this in C is of course possible, but I want something more dynamic. So I wrote a tty line discipline that uses Lua to do all the decoding. That works like a charm: Load the script, test, change the script and reload. Really practical. I will release this code once I sorted out a few remaining details. And in the course of this work, I also found deficencies in slattach(8). In previous work I used Lua to create a software gpio device, a modified version of gpiosim(4) that uses a Lua script to mimick a real device. Also handy. Thank you. Those seem like useful example. --apb (Alan Barrett)
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Le 19/10/13 00:47, Marc Balmer a écrit : Am 19.10.13 00:14, schrieb Aleksej Saushev: [...] I'm of opinion that this device driver can and should stay outside the tree until its utility can be demonstrated without this much strain. At last this is one of the reasons why we support kernel modules. The inclusion and use of Lua in base, for use in userland and the kernel, has been the subject to public discussion, it has been the topic of a GSoC project, it has been presented at many conferences, it is well received by the community at large, it has users, it has attracted new users to NetBSD, I would like to underline the attracted new users to NetBSD with an e-mail thread I've just read on DragonFly BSD kernel mailing list: http://lists.dragonflybsd.org/pipermail/kernel/2013-October/063013.htm It seems true that Lua support is attractive to users and developers :) Best regards, -- Yann Sionneau
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Date: Thu, 17 Oct 2013 19:16:16 -0300 From: Lourival Vieira Neto lourival.n...@gmail.com Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. This is not a chicken-and-egg problem. You can make an experimental kernel with Lua support and make an experimental application in Lua, all before anything has to be committed to HEAD[*]. Then you can show that the application serves a useful function, has compelling benefits over writing it in C, and can offer confidence in robustness. [*] You could do this in a branch, you could do this in a private Git repository, or you could even just do this in a local CVS checkout (since kernel Lua requires no invasive changes, right?). That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). Prototyping and experimentation is great! Show examples! What hurts is getting bitrotten code that nobody actually maintains or uses (when was the last Lua update in src?) and provides a new Turing machine with device access in the kernel for attack vectors. [1] https://github.com/dergraf/PacketScript [2] http://www.pdsw.org/pdsw12/papers/grawinkle-pdsw12.pdf In the two links you gave, I found precisely five lines of Lua code, buried in the paper, and those five lines seemed to exist only for the purpose of measuring how much overhead Lua adds to the existing pNFS code or something.
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hi, The linked research was performed on Linux, which has NFsv4.1 and pNFS client implementations. Evidently, you can do this kind of thing with an out-of-tree Lua kernel extension. Matt - Taylor R Campbell riastr...@netbsd.org wrote: [1] https://github.com/dergraf/PacketScript [2] http://www.pdsw.org/pdsw12/papers/grawinkle-pdsw12.pdf In the two links you gave, I found precisely five lines of Lua code, buried in the paper, and those five lines seemed to exist only for the purpose of measuring how much overhead Lua adds to the existing pNFS code or something. -- Matt Benjamin The Linux Box 206 South Fifth Ave. Suite 150 Ann Arbor, MI 48104 http://linuxbox.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hello, Lourival Vieira Neto lourival.n...@gmail.com writes: On Thu, Oct 17, 2013 at 1:26 PM, Jeff Rizzo r...@tastylime.net wrote: On 10/14/13 1:46 PM, Marc Balmer wrote: There is real word, real working code. In userland and in kernel space. There are developers waiting for the kernel parts to be committet, so they can continue their work as well. *Where* is this code? The pattern I see happening over and over again is: NetBSD Community: Please show us the real working code that needs this mbalmer: the code is there! (pointer to actual code not in evidence) I do not doubt that something exists, but the onus is on the person proposing the import to convince the skeptics, or at least to make an actual effort. I see lots of handwaving, and little actual code. YEARS after the import of lua into the main tree, I see very little in-tree evidence of its use. In fact, what I see is limited to : 1) evidence of lua bindings for netpgp. 2) evidence of some tests in external/bsd/lutok 3) the actual lua arc in external/mit/lua 4) gpio and sqlite stuff in liblua 5) some lua bindings in libexec/httpd (bozohttpd) 6) two example files in share/examples/lua 7) the luactl/lua module/lua(4) stuff you imported yesterday ...and counting. There is also ongoing working happening =). As Jeff points what is counting is support code. Am I missing something major here? The only actual usage I see is netpgp and httpd; the rest is all in support of lua itself. I do not see evidence that anyone is actually using lua in such a way that requires it in-tree. When you originally proposed importing lua back in 2010, you talked a lot about how uses would materialize. It's now been 3 years, and I just don't see them. If I am wrong about this, I would love some solid pointers to evidence of my wrongness. Now you're using very similar arguments for bringing lua into the kernel; I would very much like to see some real, practical, *useful* code demonstrating just why this is a good thing. Beyond the 'gee, whiz' factor, I just don't see it. Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. The problem with your approach is that such chicken-and-egg problems are to be solved _at_once_ rather than laying eggs everywhere around and have everyone else wait till at least one chicken appears. Sure, we do not *need* a script language interpreter embedded in the kernel, as we do not need a specific file system. But I do not get why we should not. There is current development of applications being done right now. Also, there is a few interesting works that used Lunatik in Linux [1, 2] that could be done more easily now in NetBSD just because we have the right environment for that. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). I think that is why we *should* (not need) have this on the tree. IMHO. I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. You tell that there's interesting work using Lua in Linux. Was it accepted in any experimental Linux distribution like Fedora? What was the outcome of discussion among linux kernel developers? Currently there's no indication that it was accepted anywhere. I doubt very much that we want such unreliable development practices like agile ones in the kernel, and experimentation work can be done easier and better in a branch or a personal repository. And last. The appeal to why not is defective. NetBSD is not your personal playground, there exist other people who have to deal with the inadvertent mess you can leave after you. That's why you ought to present solid arguments that justify why other people should tolerate your experimentations. -- BCE HA MOPE!
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Fri, 18 Oct 2013 20:31:05 +0400 Aleksej Saushev a...@inbox.ru wrote: I doubt very much that we want such unreliable development practices like agile ones in the kernel, and experimentation work can be done easier and better in a branch or a personal repository. I think I agree with your sentiment but it seems like you are misusing the word agile here. The term refers to a very specific programming methodology and would be very beneficial if applied to kernel programming. Throwing random code into a system does not constitute agile development. http://en.wikipedia.org/wiki/Agile_software_development -- D'Arcy J.M. Cain da...@netbsd.org http://www.NetBSD.org/ IM:da...@vex.net
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hi, On Fri, Oct 18, 2013 at 11:09 AM, Matt W. Benjamin m...@linuxbox.com wrote: Hi, The linked research was performed on Linux, which has NFsv4.1 and pNFS client implementations. Evidently, you can do this kind of thing with an out-of-tree Lua kernel extension. Matt Evidently. I'm not arguing that we need that. I'm just arguing that I see benefits and none harm. Regards, -- Lourival Vieira Neto
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On Fri, Oct 18, 2013 at 9:08 AM, Taylor R Campbell riastr...@netbsd.org wrote: Date: Thu, 17 Oct 2013 19:16:16 -0300 From: Lourival Vieira Neto lourival.n...@gmail.com Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. This is not a chicken-and-egg problem. You can make an experimental kernel with Lua support and make an experimental application in Lua, all before anything has to be committed to HEAD[*]. Then you can show that the application serves a useful function, has compelling benefits over writing it in C, and can offer confidence in robustness. [*] You could do this in a branch, you could do this in a private Git repository, or you could even just do this in a local CVS checkout (since kernel Lua requires no invasive changes, right?). Yes, but how do we do device driver development? We are branching the tree for each non-intrusive and disabled-by-default device driver? If we have developed a device driver for an uncommon device, we have to put it in a branch? (Please, note I'm friendly asking that). That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). Prototyping and experimentation is great! Show examples! What hurts is getting bitrotten code that nobody actually maintains or uses (when was the last Lua update in src?) and provides a new Turing machine with device access in the kernel for attack vectors. I don't see how an optional module could be used for attacks. If users enable that, they should know what they are doing (such as loading a kernel module). [1] https://github.com/dergraf/PacketScript [2] http://www.pdsw.org/pdsw12/papers/grawinkle-pdsw12.pdf In the two links you gave, I found precisely five lines of Lua code, buried in the paper, and those five lines seemed to exist only for the purpose of measuring how much overhead Lua adds to the existing pNFS code or something. I'm just showing examples of how it could be useful for user applications. I understand that you do not agree with that. But I'm not arguing that we have to add these applications into the tree. I'm arguing that we could benefit users with such a tool. Regards, -- Lourival Vieira Neto
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hi, On Fri, Oct 18, 2013 at 1:31 PM, Aleksej Saushev a...@inbox.ru wrote: (...) Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. The problem with your approach is that such chicken-and-egg problems are to be solved _at_once_ rather than laying eggs everywhere around and have everyone else wait till at least one chicken appears. No. I'm talking about put just one egg, just a device driver. Sure, we do not *need* a script language interpreter embedded in the kernel, as we do not need a specific file system. But I do not get why we should not. There is current development of applications being done right now. Also, there is a few interesting works that used Lunatik in Linux [1, 2] that could be done more easily now in NetBSD just because we have the right environment for that. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). I think that is why we *should* (not need) have this on the tree. IMHO. I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. Yes. But I'm talking about interesting *user* work. I'm not claiming that they should be in the kernel. I'm just saying that, IMHO, we should incorporate a small device driver that facilitates this kind of development (outside the tree). You tell that there's interesting work using Lua in Linux. Was it accepted in any experimental Linux distribution like Fedora? What was the outcome of discussion among linux kernel developers? Currently there's no indication that it was accepted anywhere. Really don't know. I'm not a member of these communities neither I'm claiming to incorporate such works here. However, I think that there was a discussion about PacketScript on OpenWRT, but I don't know how it evolved. I doubt very much that we want such unreliable development practices like agile ones in the kernel, and experimentation work can be done easier and better in a branch or a personal repository. I agree with you in this point: experimental work should be done aside from the tree. And last. The appeal to why not is defective. NetBSD is not your personal playground, there exist other people who have to deal with the inadvertent mess you can leave after you. That's why you ought to present solid arguments that justify why other people should tolerate your experimentations. I guess you misunderstood that. I'm not arguing that we should do it just because there is no contrary argument. I sincerely asked 'why not?' trying to understand the contrary argumentation. Also, I'm not saying that you should tolerate my experimentation. Far away from that. I haven't committed anything nor tried to impose nothing. I'm just trying to make a point of view and understand yours. When I talked about experimentation, I was trying to say that providing support for that kind of experimentation for users sounds a good idea for me and I don't see how it is prejudicial. Which doesn't mean that I'm proposing that my personal experimentation should be in tree. Regards, -- Lourival Vieira Neto
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hello, Lourival Vieira Neto lourival.n...@gmail.com writes: On Fri, Oct 18, 2013 at 1:31 PM, Aleksej Saushev a...@inbox.ru wrote: (...) Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. The problem with your approach is that such chicken-and-egg problems are to be solved _at_once_ rather than laying eggs everywhere around and have everyone else wait till at least one chicken appears. No. I'm talking about put just one egg, just a device driver. Sorry, but this is not just one egg. And counting was your reaction to complaints that almost all the code related to Lua is the code to support Lua itself rather than anything else. Sure, we do not *need* a script language interpreter embedded in the kernel, as we do not need a specific file system. But I do not get why we should not. There is current development of applications being done right now. Also, there is a few interesting works that used Lunatik in Linux [1, 2] that could be done more easily now in NetBSD just because we have the right environment for that. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). I think that is why we *should* (not need) have this on the tree. IMHO. I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. Yes. But I'm talking about interesting *user* work. I'm not claiming that they should be in the kernel. I'm just saying that, IMHO, we should incorporate a small device driver that facilitates this kind of development (outside the tree). I'm of opinion that this device driver can and should stay outside the tree until its utility can be demonstrated without this much strain. At last this is one of the reasons why we support kernel modules. You tell that there's interesting work using Lua in Linux. Was it accepted in any experimental Linux distribution like Fedora? What was the outcome of discussion among linux kernel developers? Currently there's no indication that it was accepted anywhere. Really don't know. I'm not a member of these communities neither I'm claiming to incorporate such works here. However, I think that there was a discussion about PacketScript on OpenWRT, but I don't know how it evolved. This demonstrates that Lua isn't actually useful in the kernel. I doubt very much that we want such unreliable development practices like agile ones in the kernel, and experimentation work can be done easier and better in a branch or a personal repository. I agree with you in this point: experimental work should be done aside from the tree. And last. The appeal to why not is defective. NetBSD is not your personal playground, there exist other people who have to deal with the inadvertent mess you can leave after you. That's why you ought to present solid arguments that justify why other people should tolerate your experimentations. I guess you misunderstood that. I'm not arguing that we should do it just because there is no contrary argument. I sincerely asked 'why not?' trying to understand the contrary argumentation. Also, I'm not saying that you should tolerate my experimentation. Far away from that. I haven't committed anything nor tried to impose nothing. On my side it sounded like that, sorry, if I'm wrong. I'm just trying to make a point of view and understand yours. When I talked about experimentation, I was trying to say that providing support for that kind of experimentation for users sounds a good idea for me and I don't see how it is prejudicial. Which doesn't mean that I'm proposing that my personal experimentation should be in tree. The problem as I see it is that we have one developer (two at most) pushing hard for Lua in base and in kernel and providing no satisfactory arguments why this is to be done at all. Lack of any real code for years reinforces such doubts. Why not sounds as an argument for highly experimental work in this context. And I wouldn't have anything against this why not if all the work were dressed accordingly. For now I'd say that Lua support hasn't demonstrated any benefit. I'd say that it should be removed and the work continued in a branch until benefits become more clear. -- BCE HA MOPE!
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hello, Lourival Vieira Neto lourival.n...@gmail.com writes: On Fri, Oct 18, 2013 at 9:08 AM, Taylor R Campbell riastr...@netbsd.org wrote: Date: Thu, 17 Oct 2013 19:16:16 -0300 From: Lourival Vieira Neto lourival.n...@gmail.com Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. This is not a chicken-and-egg problem. You can make an experimental kernel with Lua support and make an experimental application in Lua, all before anything has to be committed to HEAD[*]. Then you can show that the application serves a useful function, has compelling benefits over writing it in C, and can offer confidence in robustness. [*] You could do this in a branch, you could do this in a private Git repository, or you could even just do this in a local CVS checkout (since kernel Lua requires no invasive changes, right?). Yes, but how do we do device driver development? We are branching the tree for each non-intrusive and disabled-by-default device driver? If we have developed a device driver for an uncommon device, we have to put it in a branch? (Please, note I'm friendly asking that). We didn't import yet another programming language interpreter for driver development previously. Besides, what are drivers developed in Lua so far? If I understand it correctly, the only driver is the Lua interpreter itself. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). Prototyping and experimentation is great! Show examples! What hurts is getting bitrotten code that nobody actually maintains or uses (when was the last Lua update in src?) and provides a new Turing machine with device access in the kernel for attack vectors. I don't see how an optional module could be used for attacks. If users enable that, they should know what they are doing (such as loading a kernel module). Was anything done to warn users? [1] https://github.com/dergraf/PacketScript [2] http://www.pdsw.org/pdsw12/papers/grawinkle-pdsw12.pdf In the two links you gave, I found precisely five lines of Lua code, buried in the paper, and those five lines seemed to exist only for the purpose of measuring how much overhead Lua adds to the existing pNFS code or something. I'm just showing examples of how it could be useful for user applications. I understand that you do not agree with that. But I'm not arguing that we have to add these applications into the tree. I'm arguing that we could benefit users with such a tool. The problem is that the number of such users is negligible and all of them are developers that are able to build their kernel module outside the tree. -- BCE HA MOPE!
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Am 19.10.13 00:14, schrieb Aleksej Saushev: [...] I'm of opinion that this device driver can and should stay outside the tree until its utility can be demonstrated without this much strain. At last this is one of the reasons why we support kernel modules. The inclusion and use of Lua in base, for use in userland and the kernel, has been the subject to public discussion, it has been the topic of a GSoC project, it has been presented at many conferences, it is well received by the community at large, it has users, it has attracted new users to NetBSD, it has attracted users that are now developers, and it has, last but not least, core's blessing. Can we now please stop this useless discussion? Lua is part of NetBSD. [...]
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. The problem with your approach is that such chicken-and-egg problems are to be solved _at_once_ rather than laying eggs everywhere around and have everyone else wait till at least one chicken appears. No. I'm talking about put just one egg, just a device driver. Sorry, but this is not just one egg. And counting was your reaction to complaints that almost all the code related to Lua is the code to support Lua itself rather than anything else. And counting == there is ongoing work happening outside the tree. Sure, we do not *need* a script language interpreter embedded in the kernel, as we do not need a specific file system. But I do not get why we should not. There is current development of applications being done right now. Also, there is a few interesting works that used Lunatik in Linux [1, 2] that could be done more easily now in NetBSD just because we have the right environment for that. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). I think that is why we *should* (not need) have this on the tree. IMHO. I have to point out that interesting work is commonly used as a sort of euphemism to refer to highly experimental work with unclear future. Yes. But I'm talking about interesting *user* work. I'm not claiming that they should be in the kernel. I'm just saying that, IMHO, we should incorporate a small device driver that facilitates this kind of development (outside the tree). I'm of opinion that this device driver can and should stay outside the tree until its utility can be demonstrated without this much strain. At last this is one of the reasons why we support kernel modules. Understand. You tell that there's interesting work using Lua in Linux. Was it accepted in any experimental Linux distribution like Fedora? What was the outcome of discussion among linux kernel developers? Currently there's no indication that it was accepted anywhere. Really don't know. I'm not a member of these communities neither I'm claiming to incorporate such works here. However, I think that there was a discussion about PacketScript on OpenWRT, but I don't know how it evolved. This demonstrates that Lua isn't actually useful in the kernel. I don't think so. It could even evince that, but not demonstrate. And last. The appeal to why not is defective. NetBSD is not your personal playground, there exist other people who have to deal with the inadvertent mess you can leave after you. That's why you ought to present solid arguments that justify why other people should tolerate your experimentations. I guess you misunderstood that. I'm not arguing that we should do it just because there is no contrary argument. I sincerely asked 'why not?' trying to understand the contrary argumentation. Also, I'm not saying that you should tolerate my experimentation. Far away from that. I haven't committed anything nor tried to impose nothing. On my side it sounded like that, sorry, if I'm wrong. It could sound as you want, but it wasn't what I meant. I'm just trying to make a point of view and understand yours. When I talked about experimentation, I was trying to say that providing support for that kind of experimentation for users sounds a good idea for me and I don't see how it is prejudicial. Which doesn't mean that I'm proposing that my personal experimentation should be in tree. The problem as I see it is that we have one developer (two at most) pushing hard for Lua in base and in kernel and providing no satisfactory arguments why this is to be done at all. Lack of any real code for years reinforces such doubts. Why not sounds as an argument for highly experimental work in this context. And I wouldn't have anything against this why not if all the work were dressed accordingly. For now I'd say that Lua support hasn't demonstrated any benefit. I'd say that it should be removed and the work continued in a branch until benefits become more clear. Understand. Regards, -- Lourival Vieira Neto
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. This is not a chicken-and-egg problem. You can make an experimental kernel with Lua support and make an experimental application in Lua, all before anything has to be committed to HEAD[*]. Then you can show that the application serves a useful function, has compelling benefits over writing it in C, and can offer confidence in robustness. [*] You could do this in a branch, you could do this in a private Git repository, or you could even just do this in a local CVS checkout (since kernel Lua requires no invasive changes, right?). Yes, but how do we do device driver development? We are branching the tree for each non-intrusive and disabled-by-default device driver? If we have developed a device driver for an uncommon device, we have to put it in a branch? (Please, note I'm friendly asking that). We didn't import yet another programming language interpreter for driver development previously. Besides, what are drivers developed in Lua so far? If I understand it correctly, the only driver is the Lua interpreter itself. I meant traditional device driver, but never mind. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). Prototyping and experimentation is great! Show examples! What hurts is getting bitrotten code that nobody actually maintains or uses (when was the last Lua update in src?) and provides a new Turing machine with device access in the kernel for attack vectors. I don't see how an optional module could be used for attacks. If users enable that, they should know what they are doing (such as loading a kernel module). Was anything done to warn users? The code is not linked yet. Regards, -- Lourival Vieira Neto
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Date: Fri, 18 Oct 2013 18:12:29 -0300 From: Lourival Vieira Neto lourival.n...@gmail.com On Fri, Oct 18, 2013 at 9:08 AM, Taylor R Campbell riastr...@netbsd.org wrote: [*] You could do this in a branch, you could do this in a private Git repository, or you could even just do this in a local CVS checkout (since kernel Lua requires no invasive changes, right?). Yes, but how do we do device driver development? We are branching the tree for each non-intrusive and disabled-by-default device driver? If we have developed a device driver for an uncommon device, we have to put it in a branch? (Please, note I'm friendly asking that). Device drivers usually have trivially demonstrable useful functions related to physical devices that one encounters on the market. Example: I wrote uatp(4) because the trackpad in my MacBook didn't work very well. I also developed uatp(4) in a local Git branch because at first it was an experiment which I expected to throw away. In the two links you gave, I found precisely five lines of Lua code, buried in the paper, and those five lines seemed to exist only for the purpose of measuring how much overhead Lua adds to the existing pNFS code or something. I'm just showing examples of how it could be useful for user applications. I understand that you do not agree with that. But I'm not arguing that we have to add these applications into the tree. I'm arguing that we could benefit users with such a tool. I don't disagree that Lua could be useful for user applications, and I'm not asking you to propose applications to add to the tree. All I'm asking for is examples of applications at all, which I couldn't find in either of the links you gave. Where is the Lua code?
Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
On 10/14/13 1:46 PM, Marc Balmer wrote: It is entirely plausible to me that we could benefit from using Lua in base, or sysinst, or maybe even in the kernel. But that argument must be made by showing evidence of real, working code that has compelling benefits, together with confidence in its robustness -- not by saying that if we let users do it then it will happen. There is real word, real working code. In userland and in kernel space. There are developers waiting for the kernel parts to be committet, so they can continue their work as well. *Where* is this code? The pattern I see happening over and over again is: NetBSD Community: Please show us the real working code that needs this mbalmer: the code is there! (pointer to actual code not in evidence) I do not doubt that something exists, but the onus is on the person proposing the import to convince the skeptics, or at least to make an actual effort. I see lots of handwaving, and little actual code. YEARS after the import of lua into the main tree, I see very little in-tree evidence of its use. In fact, what I see is limited to : 1) evidence of lua bindings for netpgp. 2) evidence of some tests in external/bsd/lutok 3) the actual lua arc in external/mit/lua 4) gpio and sqlite stuff in liblua 5) some lua bindings in libexec/httpd (bozohttpd) 6) two example files in share/examples/lua 7) the luactl/lua module/lua(4) stuff you imported yesterday Am I missing something major here? The only actual usage I see is netpgp and httpd; the rest is all in support of lua itself. I do not see evidence that anyone is actually using lua in such a way that requires it in-tree. When you originally proposed importing lua back in 2010, you talked a lot about how uses would materialize. It's now been 3 years, and I just don't see them. If I am wrong about this, I would love some solid pointers to evidence of my wrongness. Now you're using very similar arguments for bringing lua into the kernel; I would very much like to see some real, practical, *useful* code demonstrating just why this is a good thing. Beyond the 'gee, whiz' factor, I just don't see it. +j
Re: Why do we need lua in-tree again? Yet another call for actual evidence, please. (was Re: Moving Lua source codes)
Hi Jeff, On Thu, Oct 17, 2013 at 1:26 PM, Jeff Rizzo r...@tastylime.net wrote: On 10/14/13 1:46 PM, Marc Balmer wrote: It is entirely plausible to me that we could benefit from using Lua in base, or sysinst, or maybe even in the kernel. But that argument must be made by showing evidence of real, working code that has compelling benefits, together with confidence in its robustness -- not by saying that if we let users do it then it will happen. There is real word, real working code. In userland and in kernel space. There are developers waiting for the kernel parts to be committet, so they can continue their work as well. *Where* is this code? The pattern I see happening over and over again is: NetBSD Community: Please show us the real working code that needs this mbalmer: the code is there! (pointer to actual code not in evidence) I do not doubt that something exists, but the onus is on the person proposing the import to convince the skeptics, or at least to make an actual effort. I see lots of handwaving, and little actual code. YEARS after the import of lua into the main tree, I see very little in-tree evidence of its use. In fact, what I see is limited to : 1) evidence of lua bindings for netpgp. 2) evidence of some tests in external/bsd/lutok 3) the actual lua arc in external/mit/lua 4) gpio and sqlite stuff in liblua 5) some lua bindings in libexec/httpd (bozohttpd) 6) two example files in share/examples/lua 7) the luactl/lua module/lua(4) stuff you imported yesterday ...and counting. There is also ongoing working happening =). Am I missing something major here? The only actual usage I see is netpgp and httpd; the rest is all in support of lua itself. I do not see evidence that anyone is actually using lua in such a way that requires it in-tree. When you originally proposed importing lua back in 2010, you talked a lot about how uses would materialize. It's now been 3 years, and I just don't see them. If I am wrong about this, I would love some solid pointers to evidence of my wrongness. Now you're using very similar arguments for bringing lua into the kernel; I would very much like to see some real, practical, *useful* code demonstrating just why this is a good thing. Beyond the 'gee, whiz' factor, I just don't see it. Lua is a tool, not an end in itself. I think that you are formulating a chicken-and-egg problem: we need the basic support for then having applications, and we need applications for then having basic support. Sure, we do not *need* a script language interpreter embedded in the kernel, as we do not need a specific file system. But I do not get why we should not. There is current development of applications being done right now. Also, there is a few interesting works that used Lunatik in Linux [1, 2] that could be done more easily now in NetBSD just because we have the right environment for that. That is not about needing, but it is about supporting a certain kind of agile development, prototyping, customization and experimentation in the NetBSD kernel (how could it be hurtful?). I think that is why we *should* (not need) have this on the tree. IMHO. [1] https://github.com/dergraf/PacketScript [2] http://www.pdsw.org/pdsw12/papers/grawinkle-pdsw12.pdf +j Regards, -- Lourival Vieira Neto
