Re: Another tool for updating /etc -- lua||other script language bikeshed
Ivan Voras writes: > Dag-Erling Smørgrav writes: > > Bourne shell is a perfectly fine programming language if you know > > how to use it. > I'll agree that it's fine but only in the abstract - e.g. that it is > Turing complete :) Emphasis on "if you know how to use it". It's like Perl - it's easy to learn how to write bad Perl or Bourne shell code, but much harder to learn how to use them properly. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On Fri, 26 Mar 2010, per...@pluto.rain.com wrote: Robert Watson wrote: ... web browsers [are] basically operating systems at this point ... Isn't this a bit of an exaggeration? Not too many browsers have to deal with process/thread scheduling, or device drivers, or booting, or file system issues -- they rely on the OS for that (as does any other application). I think it's more of an anaology than an exageration. The FreeBSD kernel, including device drivers and architectures, is around 3.9 million lines of code. Google's Chromium, including WebKit, is around 4.1 million lines of code. Both provide an extensive runtime environment for applications that run on top of them, security domains, storage services, and management models. I'm not arguing that web browsers are a substitute for our current operating system layer: they clearly build on it. However, in terms of their goals in providing an execution environment, user interface, etc, they fill a very similar niche by being a general-purpose platform for many specific things. And, to get back to the point I was making: if you toast your Chromium update or get configuration management wrong, then your applications (Google Docs, GMail, ...) on ChromeOS won't work any more than if you toasted your /lib or /etc in FreeBSD. For example, if the Chromium configuration files change and it forgets about web proxies, Chromium won't be able to call home to pick up a fix any more than if etcmerge toasts resolv.conf. Making updates easy is, to a large extent, about avoiding the creation of foot-shooting opportunities. Some of it is about tools (binary updates, mergers, rollbac, etc), but most of it is about avoiding scenarios in which a previously valid configuration becomes invalid. And if we look at problems FreeBSD has had with updates in a past, a lot come down to precisely that: for example, renaming serial port device names (several times in as many years). Robert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
Robert Watson wrote: > ... web browsers [are] basically operating systems at this point ... Isn't this a bit of an exaggeration? Not too many browsers have to deal with process/thread scheduling, or device drivers, or booting, or file system issues -- they rely on the OS for that (as does any other application). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On 03/25/10 09:51, Robert Watson wrote: On Wed, 24 Mar 2010, Ivan Voras wrote: Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script language interpreter with a syntax that has evolved since the 1970-ies? (with a side-glance to C that *has* evolved since the K&R style). ... As a possible alternative, or at least to learn about others' opinion on the subject, I'd like to suggest Lua (http://www.lua.org/). I think there are lots of good arguments for Lua in the base, but that etcmerge is definitely not one of them :-). Of course! Maybe I should have worded it better - my post has nothing specifically to do with etcmerge. At most it could serve some future developer and/or project. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On Wed, 24 Mar 2010, Ivan Voras wrote: Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script language interpreter with a syntax that has evolved since the 1970-ies? (with a side-glance to C that *has* evolved since the K&R style). ... As a possible alternative, or at least to learn about others' opinion on the subject, I'd like to suggest Lua (http://www.lua.org/). I think there are lots of good arguments for Lua in the base, but that etcmerge is definitely not one of them :-). An important goals for a tool like etcmerge is a minimal dependency footprint, so that you can use it with all the existing versions of FreeBSD floating around and upgrade to new versions. None of those existing versions have lua. Good arguments for lua in the base might include: - Moving to Lua as the scripting language for the boot loader - Improving scripting capabilities in the installer etcmerge sounds very exciting, especially for shops that want a more automated upgrade path. It's easy to upgrade web browsers, and they're basically operating systems at this point, so it would be nice if we could offer FreeBSD upgrades with similar ease. Quite a bit of our automated configuration update problem comes down to configuration file formats and the way diff/patch perform merges. Consider files like inetd.conf, master.passwd, group, etc: they essentially ensure that there will be a conflict if you have any local changes and the vendor (us) makes an upstream change. We used to have this problem with /etc/rc and /etc/rc.local, but rc.d has basically eliminated the problem by allowing boot-time custtomization through file insertion rather than file changes. Choices made in the configuration design for launchd, xinetd, and others avoid this mistake. Perhaps we shold be considering similar sorts of redesigns, focusing on how configuration files could be reworked to maximize automated update support. Where there's a true semantic conflict, an update conflict requiring resolution is fine, but where there's no semantic conflict (i.e., we add _anotheruser to the base master.passwd), no upgrade conflict should arise. (And definitely keeping this mind as we add new configuration files) Robert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
2010/3/24 Dag-Erling Smørgrav : > Ivan Voras writes: >> C is good enough. I'm after /bin/sh here. > > Bourne shell is a perfectly fine programming language if you know how to > use it. I'll agree that it's fine but only in the abstract - e.g. that it is Turing complete :) (But then again, I've heard sendmail.cf is also Turing complete so there...) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On 24 March 2010 20:03, Peter Jeremy wrote: > On 2010-Mar-24 14:11:21 +0100, Ivan Voras wrote: > There's awk (though it's somewhat restricted in its abilities to do > anything more than text manipulation) but in principle, I agree. The > requirements as I see them are (in no particular order): > - BSD-compatible license > - must be compatible with buildworld (primarily, it must be possible > to cross-build) > - contains a critical mass of users in the FreeBSD developer (and > ideally committer) community > - language must be reasonably stable - will a script written today still > work correctly in (say) 5 years. > - must be acceptable to the vast majority of the user base (no religious > wars allowed) > >>There was once Perl in base and even though I personally dislike Perl at >>least it was a standard of sorts and guaranteed to be there if needed. > > It was removed because it didn't support cross-building (buildworld is > always done as a cross-build) and was evolving at a rate incompatible > with the base system. > >>As a possible alternative, or at least to learn about others' opinion on >>the subject, I'd like to suggest Lua (http://www.lua.org/). > > As someone who has never used Lua, how well does it meet the > requirements above? I would also like to avoid holy wars so I really want this to be discussed. I'm more pushing in the direction of "give us something nicer than sh" than I'm specifically pushing Lua. To the best of my knowledge, the answers to your questions for Lua are: > - BSD-compatible license it's MIT-licensed. > - must be compatible with buildworld (primarily, it must be possible > to cross-build) I'm not sure what you mean by "cross-build". If you mean it needs to support some specific compiler magic, then I don't know. I do know this: its src directory is flat (32 .c files, 23 .h), its Makefile basically says "compile all these .c files into an executable or a library with CC options varying by platform", and it's ported to at lest these operating systems: aix ansi bsd freebsd generic linux macosx mingw posix solaris - the presence of solaris indicates 64-bit and big endian - aka the big portability test. I've just verified that a working executable can be compiled with "gcc -o lua -lm *.c" (if you remove the "luac.c" with duplicates a main() in addition to the interpreter "lua.c"). > - contains a critical mass of users in the FreeBSD developer (and > ideally committer) community > - must be acceptable to the vast majority of the user base (no religious > wars allowed) Have no idea. Possibly the biggest single obstacles unless they're modified to "has the biggest number of people not specifically objecting to it" > - language must be reasonably stable - will a script written today still > work correctly in (say) 5 years. The Wikipedia entry for it (http://en.wikipedia.org/wiki/Lua_%28programming_language%29) says it's been started in 1993. AFAIK it has always been backward compatible and the recent version is 5.3. One thing I remember it for is that many Windows games (including some classics like Baldur's Gate II... I'm sure some people remember it) were scripted by it. I'd like to summarize why I think Lua is a good idea here: * Small footprint (150-200k executable). Can be compiled with gcc -o lua -lm *.c. * Can be embedded into C apps for scripting, can use C for extending it (designed to make both directions trivially easy) * Is kind of like Python (supports OOP, partly functional, etc. styles) without the extensive libraries. (but has no "significant whitespace" issue). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On Wed, Mar 24, 2010 at 6:11 AM, Ivan Voras wrote: > On 03/23/10 16:08, John Baldwin wrote: > > [snip - looks like a good utility, will probably use it instead of > mergemaster if it gets committed, like the idea about automated updates] > >> To that end, I wrote a new tool that I think does a decent job of solving >> these goals. > > Since the issue comes around very rarely, I assume there are not many people > who also get the shivers when they see a shell script (and then a "posixy" > /bin/sh shell script) more than a 100 lines long? :) > > Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script > language interpreter with a syntax that has evolved since the 1970-ies? > (with a side-glance to C that *has* evolved since the K&R style). > > There was once Perl in base and even though I personally dislike Perl at > least it was a standard of sorts and guaranteed to be there if needed. Now > there are some fairly large chunks of code written in plain shell script, > like mergemaster, freebsd-update, portsup and adduser. I'm not specifically > against shell scripts but (which might just be my personal opinion) I think > they are even less maintainable in the long term than Perl scripts. I also > think the bus factor on good shell script programmers must be pretty low. > > As a possible alternative, or at least to learn about others' opinion on the > subject, I'd like to suggest Lua (http://www.lua.org/). > > The reasons: > > 1) Very light-weight in terms of system integration. Basically, there are > one or two executables and libraries and the libraries can be discarded if > only the interpreter executable is needed and not the ability to integrate > it into C apps. No "libs directories" needed. Written in C, designed to be > easy to invoked from C (from which the interpreter executable is built on). > > This ability to integrate is important because it allows for some nifty > things like implementing "system" commands through C, e.g. a "sysctl()" > function as a wrapper for sysctl(3), or a "GEOM Class" class that wraps > control of GEOM objects. > > The basic interpreter executable and the library are ~~ 150 kB each. The > /bin/sh executable is 130 kB. > > 2) Easy syntax, which even kind of resembles shell scripts in its flow. > Examples: http://lua-users.org/wiki/SampleCode . Unfortunately, its error > handling is not much better than plain C (no "exceptions"). It has nice > C-like formatting (e.g. "%4.2f, %d":format(3.14, 42)) and goodies like > lexical scoping, foreach and coroutines. It's default OOP implementation is > a bit specific (the "tables" and "metatables" system) but usable. > > 3) MIT license. Friendly. > > What would be gained? I guess what I'm trying to suggest is that a 3000 line > shell script (like portsnap, as a random example) could be more readable, > easier to write and maintain were it a 3000 line Lua script. Code in general becomes a pain to maintain unless it's well written when it evolves beyond 500 lines (unless it's C / C++, which is 1000 lines in my book). So, I don't think that changing sh to Lua would improve anything (in fact it may make things worse because it's a less mainstream language). There are a host of other languages out there that could also serve the needs of the infrastructure, but the fact remains that the more divergence the project goes in terms of defacto standard languages, the more of a pain in the ass it's going to be to maintain longterm because someone decided to write something in language X.Y.Z. And there are actually some things which could be done to simplify the mergemaster and portsnap scripts as well (minor bloat?), but I digress. Thanks, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
Ivan Voras writes: > C is good enough. I'm after /bin/sh here. Bourne shell is a perfectly fine programming language if you know how to use it. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On Wednesday 24 March 2010 10:49:12 am Ivan Voras wrote: > On 03/24/10 15:02, John Baldwin wrote: > > On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote: > >> On 03/23/10 16:08, John Baldwin wrote: > >> > >> [snip - looks like a good utility, will probably use it instead of > >> mergemaster if it gets committed, like the idea about automated updates] > >> > >>> To that end, I wrote a new tool that I think does a decent job of solving > >>> these goals. > >> > >> Since the issue comes around very rarely, I assume there are not many > >> people who also get the shivers when they see a shell script (and then a > >> "posixy" /bin/sh shell script) more than a 100 lines long? :) > >> > >> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script > >> language interpreter with a syntax that has evolved since the 1970-ies? > >> (with a side-glance to C that *has* evolved since the K&R style). > > > > "You can write Fortran in any language." > > I feel I should quote some saying from a holy book saying "Ah, but some > languages make it harder than the others!" but I don't know of any such :) > > > If there are specific things in specific scripts that are poorly commented > > or > > implemented then I would work on fixing those. The same is true of the > > mountain of C code in the tree. Rewriting them in a different language will > > not automatically make them any better. > > C is good enough. I'm after /bin/sh here. I think if you were to read the source to etcupdate.sh you would find that it is actually rather close to how I would write it if I were to do it in C (except it is far less complicated now since I can use tools like diff directly). -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On 2010-Mar-24 14:11:21 +0100, Ivan Voras wrote: >Since the issue comes around very rarely, I assume there are not many >people who also get the shivers when they see a shell script (and then a >"posixy" /bin/sh shell script) more than a 100 lines long? :) With the specific exception of GNU configure and related horrors, I personally don't have anything against shell scripts. You can write good or bad code in any language. >Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script >language interpreter with a syntax that has evolved since the 1970-ies? There's awk (though it's somewhat restricted in its abilities to do anything more than text manipulation) but in principle, I agree. The requirements as I see them are (in no particular order): - BSD-compatible license - must be compatible with buildworld (primarily, it must be possible to cross-build) - contains a critical mass of users in the FreeBSD developer (and ideally committer) community - language must be reasonably stable - will a script written today still work correctly in (say) 5 years. - must be acceptable to the vast majority of the user base (no religious wars allowed) >There was once Perl in base and even though I personally dislike Perl at >least it was a standard of sorts and guaranteed to be there if needed. It was removed because it didn't support cross-building (buildworld is always done as a cross-build) and was evolving at a rate incompatible with the base system. >As a possible alternative, or at least to learn about others' opinion on >the subject, I'd like to suggest Lua (http://www.lua.org/). As someone who has never used Lua, how well does it meet the requirements above? -- Peter Jeremy pgpLSK4hUsCTp.pgp Description: PGP signature
Re: Another tool for updating /etc -- lua||other script language bikeshed
On 03/24/10 15:02, John Baldwin wrote: On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote: On 03/23/10 16:08, John Baldwin wrote: [snip - looks like a good utility, will probably use it instead of mergemaster if it gets committed, like the idea about automated updates] To that end, I wrote a new tool that I think does a decent job of solving these goals. Since the issue comes around very rarely, I assume there are not many people who also get the shivers when they see a shell script (and then a "posixy" /bin/sh shell script) more than a 100 lines long? :) Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script language interpreter with a syntax that has evolved since the 1970-ies? (with a side-glance to C that *has* evolved since the K&R style). "You can write Fortran in any language." I feel I should quote some saying from a holy book saying "Ah, but some languages make it harder than the others!" but I don't know of any such :) If there are specific things in specific scripts that are poorly commented or implemented then I would work on fixing those. The same is true of the mountain of C code in the tree. Rewriting them in a different language will not automatically make them any better. C is good enough. I'm after /bin/sh here. "Whatever language you write in, your task as a programmer is to do the best you can with the tools at hand. A good programmer can overcome a poor language or a clumsy operating system, but even a great programming environment will not rescue a bad programmer." (Kernighan and Pike) I'll accept it. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote: > On 03/23/10 16:08, John Baldwin wrote: > > [snip - looks like a good utility, will probably use it instead of > mergemaster if it gets committed, like the idea about automated updates] > > > To that end, I wrote a new tool that I think does a decent job of solving > > these goals. > > Since the issue comes around very rarely, I assume there are not many > people who also get the shivers when they see a shell script (and then a > "posixy" /bin/sh shell script) more than a 100 lines long? :) > > Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script > language interpreter with a syntax that has evolved since the 1970-ies? > (with a side-glance to C that *has* evolved since the K&R style). "You can write Fortran in any language." If there are specific things in specific scripts that are poorly commented or implemented then I would work on fixing those. The same is true of the mountain of C code in the tree. Rewriting them in a different language will not automatically make them any better. "Whatever language you write in, your task as a programmer is to do the best you can with the tools at hand. A good programmer can overcome a poor language or a clumsy operating system, but even a great programming environment will not rescue a bad programmer." (Kernighan and Pike) -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Another tool for updating /etc -- lua||other script language bikeshed
On 03/23/10 16:08, John Baldwin wrote: [snip - looks like a good utility, will probably use it instead of mergemaster if it gets committed, like the idea about automated updates] To that end, I wrote a new tool that I think does a decent job of solving these goals. Since the issue comes around very rarely, I assume there are not many people who also get the shivers when they see a shell script (and then a "posixy" /bin/sh shell script) more than a 100 lines long? :) Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script language interpreter with a syntax that has evolved since the 1970-ies? (with a side-glance to C that *has* evolved since the K&R style). There was once Perl in base and even though I personally dislike Perl at least it was a standard of sorts and guaranteed to be there if needed. Now there are some fairly large chunks of code written in plain shell script, like mergemaster, freebsd-update, portsup and adduser. I'm not specifically against shell scripts but (which might just be my personal opinion) I think they are even less maintainable in the long term than Perl scripts. I also think the bus factor on good shell script programmers must be pretty low. As a possible alternative, or at least to learn about others' opinion on the subject, I'd like to suggest Lua (http://www.lua.org/). The reasons: 1) Very light-weight in terms of system integration. Basically, there are one or two executables and libraries and the libraries can be discarded if only the interpreter executable is needed and not the ability to integrate it into C apps. No "libs directories" needed. Written in C, designed to be easy to invoked from C (from which the interpreter executable is built on). This ability to integrate is important because it allows for some nifty things like implementing "system" commands through C, e.g. a "sysctl()" function as a wrapper for sysctl(3), or a "GEOM Class" class that wraps control of GEOM objects. The basic interpreter executable and the library are ~~ 150 kB each. The /bin/sh executable is 130 kB. 2) Easy syntax, which even kind of resembles shell scripts in its flow. Examples: http://lua-users.org/wiki/SampleCode . Unfortunately, its error handling is not much better than plain C (no "exceptions"). It has nice C-like formatting (e.g. "%4.2f, %d":format(3.14, 42)) and goodies like lexical scoping, foreach and coroutines. It's default OOP implementation is a bit specific (the "tables" and "metatables" system) but usable. 3) MIT license. Friendly. What would be gained? I guess what I'm trying to suggest is that a 3000 line shell script (like portsnap, as a random example) could be more readable, easier to write and maintain were it a 3000 line Lua script. Thoughts? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
