Re: config(5) break down
On Fri, Mar 26, 2010 at 10:24:02AM +0900, Masao Uebayashi wrote: (Honestly, I see benefit to not convincing you; objection only from dholland@ sounds more convincing to me than no objections.) Um. I'm sorry you think that. I guess there is no point continuing this discussion, then. Or much of any other. -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Fri, Mar 26, 2010 at 01:45:51PM +, Andrew Doran wrote: I'm speaking of low level kernel code and driver drivers, areas that to date you have had relatively little involvement in. That's not entirely true, but fair enough. I will however consider discussing the points you raise if/when I launch a jihad against emulations and file system code. I thought you already had! :-p :-) -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Fri Mar 26 2010 at 13:25:43 +0900, Masao Uebayashi wrote:
syntax. I spent a whole weekend to read sys/conf/files, ioconf.c, and
module stubs in sys/dev/usb/uaudio.c. I wasted a whole weekend. I've
This patch should work and make it easier. No, it doesn't solve
dependencies, but gets developers at least halfway there without having
to waste weekends (with code). Unfortunately I can't test, since I
forgot to buy a usb audio device from Akihabara ;)
Index: dev/usb/uaudio.c
===
RCS file: /cvsroot/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.117
diff -p -u -r1.117 uaudio.c
--- dev/usb/uaudio.c12 Nov 2009 19:50:01 - 1.117
+++ dev/usb/uaudio.c26 Mar 2010 06:11:39 -
@@ -3065,67 +3065,21 @@ uaudio_set_speed(struct uaudio_softc *sc
MODULE(MODULE_CLASS_DRIVER, uaudio, NULL);
-static const struct cfiattrdata audiobuscf_iattrdata = {
- audiobus, 0, { { NULL, NULL, 0 }, }
-};
-static const struct cfiattrdata * const uaudio_attrs[] = {
- audiobuscf_iattrdata, NULL
-};
-CFDRIVER_DECL(uaudio, DV_DULL, uaudio_attrs);
-extern struct cfattach uaudio_ca;
-static int uaudioloc[6/*USBIFIFCF_NLOCS*/] = {
- -1/*USBIFIFCF_PORT_DEFAULT*/,
- -1/*USBIFIFCF_CONFIGURATION_DEFAULT*/,
- -1/*USBIFIFCF_INTERFACE_DEFAULT*/,
- -1/*USBIFIFCF_VENDOR_DEFAULT*/,
- -1/*USBIFIFCF_PRODUCT_DEFAULT*/,
- -1/*USBIFIFCF_RELEASE_DEFAULT*/};
-static struct cfparent uhubparent = {
- usbifif, NULL, DVUNIT_ANY
-};
-static struct cfdata uaudio_cfdata[] = {
- {
- .cf_name = uaudio,
- .cf_atname = uaudio,
- .cf_unit = 0,
- .cf_fstate = FSTATE_STAR,
- .cf_loc = uaudioloc,
- .cf_flags = 0,
- .cf_pspec = uhubparent,
- },
- { NULL }
-};
+#include ioconf.c
static int
uaudio_modcmd(modcmd_t cmd, void *arg)
{
- int err;
switch (cmd) {
case MODULE_CMD_INIT:
- err = config_cfdriver_attach(uaudio_cd);
- if (err) {
- return err;
- }
- err = config_cfattach_attach(uaudio, uaudio_ca);
- if (err) {
- config_cfdriver_detach(uaudio_cd);
- return err;
- }
- err = config_cfdata_attach(uaudio_cfdata, 1);
- if (err) {
- config_cfattach_detach(uaudio, uaudio_ca);
- config_cfdriver_detach(uaudio_cd);
- return err;
- }
- return 0;
+ return config_init_component(cfdriver_comp_uaudio,
+ cfattach_comp_uaudio, cfdata_uaudio);
+
case MODULE_CMD_FINI:
- err = config_cfdata_detach(uaudio_cfdata);
- if (err)
- return err;
- config_cfattach_detach(uaudio, uaudio_ca);
- config_cfdriver_detach(uaudio_cd);
- return 0;
+ return config_fini_component(cfdriver_comp_uaudio,
+ cfattach_comp_uaudio, cfdata_uaudio);
+
default:
return ENOTTY;
}
Index: modules/uaudio/Makefile
===
RCS file: /cvsroot/src/sys/modules/uaudio/Makefile,v
retrieving revision 1.1
diff -p -u -r1.1 Makefile
--- modules/uaudio/Makefile 28 Jun 2008 09:14:56 - 1.1
+++ modules/uaudio/Makefile 26 Mar 2010 06:11:39 -
@@ -5,6 +5,7 @@
.PATH: ${S}/dev/usb
KMOD= uaudio
+IOCONF=UAUDIO.ioconf
SRCS= uaudio.c
.include bsd.kmodule.mk
Index: modules/uaudio/UAUDIO.ioconf
===
RCS file: modules/uaudio/UAUDIO.ioconf
diff -N modules/uaudio/UAUDIO.ioconf
--- /dev/null 1 Jan 1970 00:00:00 -
+++ modules/uaudio/UAUDIO.ioconf26 Mar 2010 06:11:39 -
@@ -0,0 +1,12 @@
+# $NetBSD$
+#
+
+ioconf uaudio
+
+include conf/files
+include dev/usb/files.usb
+
+pseudo-root uhub*
+
+# USB audio
+uaudio* at uhub? port ? configuration ?
Re: config(5) break down
I do know you've wasted longer time than me. ;) Actually what I need
are host controller drivers. Just picked up uaudio(4) to learn things
because it looked odder than others. But thanks for the patch.
On Fri, Mar 26, 2010 at 3:14 PM, Antti Kantee po...@cs.hut.fi wrote:
(snip)
Index: modules/uaudio/Makefile
===
RCS file: /cvsroot/src/sys/modules/uaudio/Makefile,v
retrieving revision 1.1
diff -p -u -r1.1 Makefile
--- modules/uaudio/Makefile 28 Jun 2008 09:14:56 - 1.1
+++ modules/uaudio/Makefile 26 Mar 2010 06:11:39 -
@@ -5,6 +5,7 @@
.PATH: ${S}/dev/usb
KMOD= uaudio
+IOCONF= UAUDIO.ioconf
SRCS= uaudio.c
.include bsd.kmodule.mk
Surely this intermediate makefile and directory can go away, eventually.
Index: modules/uaudio/UAUDIO.ioconf
===
RCS file: modules/uaudio/UAUDIO.ioconf
diff -N modules/uaudio/UAUDIO.ioconf
--- /dev/null 1 Jan 1970 00:00:00 -
+++ modules/uaudio/UAUDIO.ioconf 26 Mar 2010 06:11:39 -
@@ -0,0 +1,12 @@
+# $NetBSD$
+#
+
+ioconf uaudio
+
+include conf/files
+include dev/usb/files.usb
+
+pseudo-root uhub*
+
+# USB audio
+uaudio* at uhub? port ? configuration ?
I see this is toward the direction I'm going too. We can teach
config(1) to analyze dependency generate the necessary stub for
modules, including the default cfdata part, *in theory*.
Masao
Re: config(5) break down
Date:Fri, 26 Mar 2010 14:52:09 +0900 From:Masao Uebayashi uebay...@gmail.com Message-ID: 70f62c5e1003252252h6e5ba506xfafceb76f854b...@mail.gmail.com | You need to include dependency. You don't need to care the order of | include lines. This is pretty much same as C headers including its | dependency vs. not include in *.h but managing the order in *.c. But C include files are a disaster - a huge mess inherited from history. No-one sane would deliberately design that as a solution for anything. As a user of the things I'd much rather all that just happened automatically, when I use fopen() if the compiler needs stdio.h, it should just go and get stdio.h - it's absurd to make me explicitly say I need it. For C, because of the nature of the world 30+ years ago, and development since, there's nothing we can do about it. But copying that as if it was a great system would be absurd. | Same as pkgsrc buildlink3.mk including its dependency too. No, those are totally different - those are different files because they're managed separately, as a part of each package - the maintainer of graphics/jpeg looks after jpeg/builink3.mk and makes it does whatever is necessary to correctly link whatever -ljpeg required. Those are separate files, not because separate files is a good thing, but because the management of anything different would be a nightmare. If your reason for lots of little conf files was because there are lots of little independent projects, each producing and looking after their own little piece of the puzzle, that would be something that might reach agreement (if it were correct). But the reason seems to just be this dependency ordering thing - which it looks like you're planning on solving it the .h file way (with each file including anything it needs, then added protection against including anything more than once, and ... which is all just a huge mess). That's not the way to fix this problem. If it has become serious enough that doing it manually is no longer sensible, then just fix config to remove the ordering rule - I don't think there's any good reason for it (and you certainly are not proposing making it stronger). Making config do a two pass parse of the config file(s) would be an afternoon's work - it was never done that way because manual ordering started out being so simple to do, that the afternoon's work was never justified. Then, as each new entry was added, manually ordering that new entry was always less than that afternoon's work. Almost certainly the sum of all of that effort is (now) greater than the effort it would have taken to fix config, but no-one has ever really fallen against it hard enough that they could justify spending the afternoon to fix it. Maybe that's changed now - certainly I suspect that there's been more time spent on this discussion than it would take to have just fixed config in the first place. | Having a | single *.conf per module is natural. As natural as userland commands | and libraries have a single Makefile for each. This one is a different argument (and as I remember this is more where you started, before you got onto that ordering nonsense), this relates more to management, and is much more reasonable. However, since I'm a believer in the single module kernel, that also makes me a believer in the single files.conf file - and for people like me, following this line of reasoning, you'd agree right? (Regardless of whether or not you agree with the single module approach). | I'm not saying changing syntax is bad. My complaint is that David | Holland is suggesting things without understanding the existing | syntax. I spent a whole weekend to read sys/conf/files, ioconf.c, and | module stubs in sys/dev/usb/uaudio.c. I wasted a whole weekend. I've | been wasting more time to convince David Holland who has never | bothered to understand the existing syntax. I'm pretty sure he understands the existing syntax. I am pretty sure that I understand the existing syntax as well. The syntax of what is in the file(s) is pretty much an orthagonal issue - I certainly agree that the syntax could be improved, but I don't see that having almost any bearing upon the multiple files issue - it doesn't need to change to allow a 2 pass parser to remove any ordering relationship on the config file lines, and it can change, whether we have one config file (as it once was), a dozen or whatever it is now, or a hundred or more (your proposal). When mrg said he wanted to cry, or laugh, at your message, I suspect it was because if there had been a zillion little files, the weekend you spent would probably have been at least a week And lastly, it isn't just David you need to convince - that only one person is arguing with you doesn't mean only one person believes what he is saying (and that only you are arguing for your proposal doesn't mean that only you believe that is the
Re: config(5) break down
On Thu, Mar 25, 2010 at 09:09:32AM +, David Holland wrote: On Thu, Mar 25, 2010 at 12:46:10PM +0900, Masao Uebayashi wrote: % grep ':.*,' sys/conf/files | wc -l 86 And? I don't understand your point. There are a lot more than 86 entities in sys/conf/files. There are many instances where modules have multiple dependencies. Masao -- Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635
Re: config(5) break down
On Thu, Mar 25, 2010 at 06:22:17PM +0900, Masao Uebayashi wrote: % grep ':.*,' sys/conf/files | wc -l 86 And? I don't understand your point. There are a lot more than 86 entities in sys/conf/files. There are many instances where modules have multiple dependencies. And? I still don't understand your point. -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Fri, Mar 26, 2010 at 12:41 PM, matthew green m...@eterna.com.au wrote: On Fri, Mar 26, 2010 at 6:49 AM, David Holland dholland-t...@netbsd.org wrote: On Thu, Mar 25, 2010 at 06:22:17PM +0900, Masao Uebayashi wrote: % grep ':.*,' sys/conf/files | wc -l 86 And? I don't understand your point. There are a lot more than 86 entities in sys/conf/files. There are many instances where modules have multiple dependencies. And? I still don't understand your point. OK, then I may be misunderstanding what you meant by diamonds. But anyway, never mind. I give up convincing you because this seems to be never going to be constructive. I see no benefit in convincing you. please keep going. i am yet to see any benefit to your ideas in this thread. you've not given any good reason for breaking up files*. To manage dependency nicely. When a module A dpend on B, you write define A: B, where B has to be already define'ed in the current syntax. We're managing such ordering by hand in sys/conf/files. By splitting files use include, we don't need to manage such a mess by hand. David Holland already understood this: http://mail-index.netbsd.org/tech-kern/2010/03/11/msg007633.html So we have already agreed that split has some benefit. The point is, my split+include way doesn't need any syntax change. I'm not saying changing syntax is bad. My complaint is that David Holland is suggesting things without understanding the existing syntax. I spent a whole weekend to read sys/conf/files, ioconf.c, and module stubs in sys/dev/usb/uaudio.c. I wasted a whole weekend. I've been wasting more time to convince David Holland who has never bothered to understand the existing syntax. Masao
re: config(5) break down
To manage dependency nicely. When a module A dpend on B, you write define A: B, where B has to be already define'ed in the current syntax. We're managing such ordering by hand in sys/conf/files. By splitting files use include, we don't need to manage such a mess by hand. David Holland already understood this: http://mail-index.netbsd.org/tech-kern/2010/03/11/msg007633.html So we have already agreed that split has some benefit. The point is, my split+include way doesn't need any syntax change. the above mail doesn't agree with splitting files* for every device. it agrees that some better ideas might be useful. i like your ideas about using define better, and being more clear about dependancies, but why do you need so many files? (given the above url, i like the (2) option for the split.) i have no idea how splitting them makes the ordering no longer by hand. the ordering is still explicitly done by hand, it is just done in a bunch of files now, using more lines than would previously be required. eg, instead of all the drivers in files.pci being how they are today, they each get to have to include all the PCI definitions, and other ones. each driver / module file will have a large list of includes, all managed by hand. seems ugly and inefficient to me. I'm not saying changing syntax is bad. My complaint is that David Holland is suggesting things without understanding the existing syntax. I spent a whole weekend to read sys/conf/files, ioconf.c, and module stubs in sys/dev/usb/uaudio.c. I wasted a whole weekend. I've been wasting more time to convince David Holland who has never bothered to understand the existing syntax. i read this and i look at the devfs thread and i am not sure if i want to laugh or cry. .mrg.
Re: config(5) break down
On Fri, Mar 12, 2010 at 4:31 AM, David Holland dholland-t...@netbsd.org wrote:
On Mon, Mar 08, 2010 at 06:33:04PM +0900, Masao Uebayashi wrote:
Well, first of all nothing says you can't read the whole file before
resolving dependencies; there's nothing inherently wrong with
define foo: bar
:
define bar: baz
:
I have no idea if config currently allows this but it's not exactly
difficult to arrange.
I don't think it's worth.
Well, maybe. It seems like one of the things you're concerned about is
the maintenance overhead of making sure everything in files appears
in the right order. If you make it order-independent that goes away,
and files can be sorted by some more meaningful criteria.
That seems like generally a good idea.
Split *.conf files can be also distributed with *.[ch]. Its notation is
self-descriptive. The only problem I'm aware of is it consumes more disk
space.
That and it becomes harder to find what you're looking for, which is I
guess what I'm mostly concerned about. It's already somewhat of a pain
to find any particular declaration in the various files.* files.
I think there are two reasonable ways to arrange it and we ought to
pick one and then hack on config itself until it works acceptably that
way.
These are:
(1) one big file, or as close to it as we can reasonably manage given
that we have all these different ports with different stuff;
(2) one file per source directory, like makefiles in a traditional
project.
The Linux folks after much pain and suffering finally settled on the
second way; however, their kernel build system uses recursive make
invocations so the structure all matches up.
The way the world works in NetBSD I tend to think one big file is
better. I'd even maybe argue that we should drop files.$(ARCH) in
favor of syntax, maybe something like
if (ARCH == i386) {
file arch/i386/i386/autoconf.c
:
}
but perhaps not.
Still, I don't think one file per module (which given modular drivers
will tend to end up being one files.*/*.conf file per *.c file in many
parts of the tree) is a good idea.
No, it seems like you're intending to use whole files to define groups
instead of braces or some other punctuation. There is no need to split
things into separate files just to show grouping.
True.
config already seems to use braces for declaring bus locators (or
something like that, probably not quite the right terminology) -- what
grouping syntax should we use? (Of course, the braces could be changed
if necessary; I count 579 uses but that's not really a major
massediting task.)
(Besides, it's not necessarily as flat as all that, either.)
It's necessary to be flat to be modular.
Mm... not strictly. That's only true when there are diamonds in the
dependency graph; otherwise, declaring B inside A just indicates that
B depends on A. Consider the following hackup of files.ufs:
There're diamonds (for example, ppp-deflate depends on ppp and zlib).
file ufs/mfs/mfs_miniroot.c
module UFS {
option UFS_DIRHASH {
flag opt_ffs.h
file ufs/ufs/ufs_dirhash.c
}
option QUOTA {
flag commandline
file ufs/ufs/ufs_quota.c
}
file ufs/ufs/ufs_ihash.c
file ufs/ufs/ufs_inode.c
file ufs/ufs/ufs_lookup.c
file ufs/ufs/ufs_bmap.c
file ufs/ufs/ufs_vfsops.c
file ufs/ufs/ufs_vnops.c
file ufs/ffs/ffs_alloc.c
file ufs/ffs/ffs_balloc.c
file ufs/ffs/ffs_inode.c
file ufs/ffs/ffs_snapshot.c
file ufs/ffs/ffs_subr.c
file ufs/ffs/ffs_tables.c
file ufs/ffs/ffs_vfsops.c
file ufs/ffs/ffs_vnops.c
module FFS : filesystem {
option FFS_NO_SNAPSHOT {
flag opt_ffs.h
}
option FFS_EI {
flag opt_ffs.h
file ufs/ffs/ffs_bswap.c
}
option APPLE_UFS {
flag opt_ffs.h
file ufs/ffs/ffs_appleufs.c
}
option UFS_EXTATTR {
flag opt_ffs.h
option UFS_EXTATTR_AUTOSTART {
flag opt_ffs.h
}
file ufs/ufs/ufs_extattr.c
}
ifoption WAPBL {
file ufs/ufs/ufs_wapbl.c
file ufs/ffs/ffs_wapbl.c
}
module MFS : filesystem {
file ufs/mfs/mfs_vfsops.c
file ufs/mfs/mfs_vnops.c
}
}
module EXT2FS : filesystem {
file ufs/ext2fs/ext2fs_alloc.c
file ufs/ext2fs/ext2fs_balloc.c
file ufs/ext2fs/ext2fs_bmap.c
file ufs/ext2fs/ext2fs_bswap.c
file ufs/ext2fs/ext2fs_inode.c
file ufs/ext2fs/ext2fs_lookup.c
file ufs/ext2fs/ext2fs_readwrite.c
file ufs/ext2fs/ext2fs_subr.c
file ufs/ext2fs/ext2fs_vfsops.c
file ufs/ext2fs/ext2fs_vnops.c
}
module LFS : filesystem {
option
Re: config(5) break down
2010/3/16 Wojciech A. Koszek wkos...@freebsd.org: On Tue, Mar 16, 2010 at 06:55:36AM +0900, Masao Uebayashi wrote: On Tue, Mar 16, 2010 at 1:57 AM, Eduardo Horvath e...@netbsd.org wrote: On Sun, 14 Mar 2010, Wojciech A. Koszek wrote: I was wondering how does Linux/Solaris kernel build system work in terms of opt_*.h files? Do they have some alternative solutions for #ifdef's based on what has been included into the kernel at configuration time? It's been a while since I poked around with Linux, but I think they have a single file that contains all that info. My understanding is splitting opt_*.h into small files is just only to save rebuild time. Is this right? It's also same as GNU Autoconf + configure + #include config.h do. I think it is also for narrowing the impact of particular options; it tends to act as a sort of layering and encapsulation. And saves a bit of confusion when a commiter enabled his debugging facility with DEBUG, which may not be unique. And since everything is compiled separately you can often just replace one module with another one that is compiled with DEBUG turned on. Without rebooting the machine. (Certain inter-module interfaces are affected by DEBUG while others are not. YMMV.) Thanks, and this is also pretty much expected too. IIRC Windows did similar thing; providing foo.dll foodbg.dll. I think this strategy (== providing normal+debug binaries as official module binaries) would work for us too. You mean that the delivery of two versions of each kernel module could eventually solve the problem for you? The basic rule is to distribute single binaries of kernel / modules. I meant that providing debug modules exceptionally might make sense, because Windows camp does that, and it seems working. (This is just an idea, not important. So never mind anyway.) Masao
Re: config(5) break down
2010/3/16 Wojciech A. Koszek wkos...@freebsd.org:
On Mon, Mar 15, 2010 at 11:50:09PM +0900, Masao Uebayashi wrote:
2010/3/15 Wojciech A. Koszek wkos...@freebsd.org:
device X
builds device X staticly into the kernel (and maybe does something
device-specific), while:
module X
Only builds a KLD. I picked module, since it seems to be more-or-less an
oposite of:
static X
which could build feature X, which is not a device staticly. I think
your
config(8) has this problem solved somehow, since you seem to have
filesystem
keyword as well. Nowadays, given that as you mentioned for NetBSD, in
FreeBSD
we also have no scoping for config(8), we must build all KLDs just in case
someone needs them, since we don't know which file belongs to which module.
Who writes these in what file?
Every module has separate Makefile in src/sys/modules/... Isn't it the same in
NetBSD?
I was wondering how does Linux/Solaris kernel build system work in terms of
opt_*.h files? Do they have some alternative solutions for #ifdef's based
on
what has been included into the kernel at configuration time?
Without looking them, I don't think any infrastructural (== config(1)
itself) change helps. Why not fix source code rather than improving
config(1)?
You mean that you have a solution for:
struct mystruct {
#ifdef DEBUG_MYSTRUCT
int line;
char *file;
char *func;
void *another_pointer;
#endif
...
};
within a kernel code? That's the simpliest example, of course. There are
areas where you simply can't prevent this kind of #ifdef's.
As others said, just don't do that. You can't fix wrong code by
improving config...
Masao
Re: config(5) break down
On Thu, Mar 25, 2010 at 01:16:02AM +0900, Masao Uebayashi wrote: On Mon, Mar 22, 2010 at 6:14 AM, Sverre Froyen sve...@viewmark.com wrote: Could this be resolved by adding a get label ioctl that could be used on any device? ?It might return a descriptive string from the device driver / config file. That's part of the plan of my devfs. yuck. One of the nice things about having symlinks, instead of requiring a new ioctl, is that you don't need any special tool to see how things are connected together: you can just use ls. eric
Re: config(5) break down
Could this be resolved by adding a get label ioctl that could be used on any device? That's part of the plan of my devfs. yuck. Agreed. For one thing, you can't ioctl anything until you open it, which is a nontrivial operation for many devices - and closing it when you're done is a nontrivial operation for even more devices. One of the nice things about having symlinks, instead of requiring a new ioctl, is that you don't need any special tool to see how things are connected together: you can just use ls. I just had a thought. When symlink mode bits went in, I thought about possible meanings for the set-id bits; I think they are a solution that has just found its problem. The principal (only, as far as I can see) reason why the symlink into a devfs tree paradigm isn't good is that it breaks permissions: the devfs nodes' permissions will either allow access they shouldn't or deny access they shouldn't, with no way for the symlinks to fix it. But if the set-ID bit on a symlink means that the symlink's owner is used for the access to the linked-to thing, then all devfs nodes can be owner root mode 600 and user-visible device nodes can be setuid-root symlinks. Then the permissions on the symlink control access. I'm not sure I like it - I think it makes more sense to have the set-ID bit control the rest of the path walk rather than access to the thing at the end of that walk - but that can be dealt with either of two ways: either devfs mounts are mode 700 directories containing mode 666 devices, or the set-ID bit *does* control the path walk - but the sticky bit, if set, says it also controls the access at the end of it. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: config(5) break down
On Thu, Mar 25, 2010 at 01:14:51AM +0900, Masao Uebayashi wrote: ? (Besides, it's not necessarily as flat as all that, either.) ? ? It's necessary to be flat to be modular. Mm... not strictly. That's only true when there are diamonds in the dependency graph; otherwise, declaring B inside A just indicates that B depends on A. Consider the following hackup of files.ufs: There're diamonds (for example, ppp-deflate depends on ppp and zlib). Sure. But mostly there aren't. [...] module UFS [...] module FFS [...] module MFS [...] module EXT2FS [...] module LFS [...] In this plan, what *.kmod will be generated? The ones declared? Or one big one, or one per source file, or whatever the blazes you want, actually... I'm perfectly happy to rework the parser to support syntax like the above if we can all agree on what it should be. So you're proposing a syntax change without understanding the existing syntax? (You don't know what braces are for, you didn't know define behavior, ...) I have to say that your proposal is not convincing to me... Um. I know perfectly well that config currently uses braces for something else. That's irrelevant. There's no need to use braces for grouping; it just happens to be readily comprehensible to passersby. There's an infinite number of possible other grouping symbols that can be used, ranging from to (! !) or even things like *( )*. Furthermore, the existing use of braces can just as easily be changed to something else if that seems desirable. There's a reason I said syntax like the above and if we can all agree on what it should be. That wasn't a concrete proposal, it wasn't meant to be a concrete proposal, no concrete proposal is complete without an analysis of whether the grammar remains unambiguous, and nitpicking it on those grounds is futile. You seem to be completely missing the point. -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Fri, Mar 19, 2010 at 02:49:37PM +, Andrew Doran wrote: I *do* think it's a useful datapoint to note that sun2, pmax, algor, etc. are never, ever downloaded any more. Right, and these dead ports must be euthanized. The mountain of unused device drivers and core kernel code is a signficant hinderance to people working in the kernel. Speaking from the point of view of repeatedly touching every namei call anywhere in the kernel... I'd have to disagree. Sure, it'd go faster if there weren't a pile of legacy binary compat implementations or if we removed all the mostly-unused fses, but if I wanted a toy kernel I already have a pile of those in the office. Most of the issues that the dead ports or fses trigger are real design or structural problems that would be only masked, not resolved, by removing that code. Supporting all the random bells and whistles that e.g. compat_svr4 wants from namei is part of doing it correctly, and having the correct infrastructure in place that can support these things is important because the need/desire/demand will come along again; it always does. For example, the $ORIGIN thing in ld.elf_so is actually the same as one of the annoying cases in (IIRC) compat_svr4... I know we don't exactly see eye to eye on these issues but perhaps we can reach some kind of middle ground? -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Thu, Mar 25, 2010 at 12:18 PM, David Holland dholland-t...@netbsd.org wrote: On Thu, Mar 25, 2010 at 01:14:51AM +0900, Masao Uebayashi wrote: ? (Besides, it's not necessarily as flat as all that, either.) ? ? It's necessary to be flat to be modular. Mm... not strictly. That's only true when there are diamonds in the dependency graph; otherwise, declaring B inside A just indicates that B depends on A. Consider the following hackup of files.ufs: There're diamonds (for example, ppp-deflate depends on ppp and zlib). Sure. But mostly there aren't. % grep ':.*,' sys/conf/files | wc -l 86 In this plan, what *.kmod will be generated? The ones declared? Or one big one, or one per source file, or whatever the blazes you want, actually... And how dependencies are represented? Um. I know perfectly well that config currently uses braces for something else. That's irrelevant. There's no need to use braces for grouping; it just happens to be readily comprehensible to passersby. There's an infinite number of possible other grouping symbols that can be used, ranging from to (! !) or even things like *( )*. Furthermore, the existing use of braces can just as easily be changed to something else if that seems desirable. I don't like unnecessary changes. There's a reason I said syntax like the above and if we can all agree on what it should be. That wasn't a concrete proposal, it wasn't meant to be a concrete proposal, no concrete proposal is complete without an analysis of whether the grammar remains unambiguous, and nitpicking it on those grounds is futile. You seem to be completely missing the point. So you're objecting my concrete proposal with your not-concrete proposal. All you've said is I don't like small files. If you have a concrete proposal, please post it as another thread. Masao
Re: config(5) break down
On Sat, Mar 20, 2010 at 2:42 PM, Masao Uebayashi uebay...@gmail.com wrote: audio(4) and tty(4) are very different from my view; audio(4) has the single entry and drivers implement backend. tty(4) is a common interface through which kernel accesses serial devices (correct me if wrong). I'm very curious which architecture looks better to you, because I have a vague idea to make tty(4) to something like audio(4). I've found that FreeBSD's tty(4) is already done so; serial drivers implement backend interfaces. System calls always go through tty(4)'s cdevsw. In NetBSD serial drivers have their own cdevsw... Masao
Re: config(5) break down
On Sun, Mar 21, 2010 at 1:01 AM, Matt Thomas m...@3am-software.com wrote: I'm talking about maj, min to device. How, as a user, do I know what actual tty does /dev/ttyXX open? If we make tty(4) a device, we can lookup its parent by drvctl(8) (extend it to return dv_parent). My question is - is this the only reason why each serial driver has its own device entry? Masao
Re: config(5) break down
On Mar 20, 2010, at 8:37 AM, Masao Uebayashi wrote: On Sun, Mar 21, 2010 at 12:14 AM, Matt Thomas m...@3am-software.com wrote: Which is fine if you have one type of serial port, but if you have a mix of devices all providing serial ports how do you know what tty is going to what serial port (especially with multiport serial devices)? Like puc(4)? It has an array of ports (sc_ports) where child com(4) device_t pointers are stored. com(4) has sc_tty. So you can lookup puc-com-tty. FreeBSD's tty(4) stores its serial device's softc as t_devswsoftc. I think we can make tty(4) as a device so that we can lookup tty-com-puc using dv_parent. I'm talking about maj, min to device. How, as a user, do I know what actual tty does /dev/ttyXX open?
Re: config(5) break down
On Sun, Mar 21, 2010 at 1:43 AM, Matt Thomas m...@3am-software.com wrote: On Mar 20, 2010, at 9:06 AM, Masao Uebayashi wrote: On Sun, Mar 21, 2010 at 1:01 AM, Matt Thomas m...@3am-software.com wrote: I'm talking about maj, min to device. How, as a user, do I know what actual tty does /dev/ttyXX open? If we make tty(4) a device, we can lookup its parent by drvctl(8) (extend it to return dv_parent). That is user unfriendly. The name in /dev should tell me enough to know what it is. I should not have to invoke another command to figure that out. You can use scripts and symlinks. My question is - is this the only reason why each serial driver has its own device entry? It is the primary reason. And it's a good reason. I disagree, but this is a matter of taste. I'm fine if you can outline tty/serial drivers structure and strategy of MP-safe locking. Masao
Re: config(5) break down
On Sat, Mar 20, 2010 at 09:43:21AM -0700, Matt Thomas wrote: On Mar 20, 2010, at 9:06 AM, Masao Uebayashi wrote: On Sun, Mar 21, 2010 at 1:01 AM, Matt Thomas m...@3am-software.com wrote: I'm talking about maj, min to device. How, as a user, do I know what actual tty does /dev/ttyXX open? If we make tty(4) a device, we can lookup its parent by drvctl(8) (extend it to return dv_parent). That is user unfriendly. The name in /dev should tell me enough to know what it is. I should not have to invoke another command to figure that out. When you say what it is, it's not clear whether you mean what kind or what instance. I have a machine with eleven dial-out /dev/ nodes: crw-rw 1 uucp dialer 8, 524288 Mar 20 13:35 /dev/dty00 crw-rw 1 uucp dialer 8, 524289 Mar 20 13:35 /dev/dty01 crw-rw 1 uucp dialer 8, 524290 Nov 9 2003 /dev/dty02 crw-rw 1 uucp dialer 8, 524291 Mar 10 18:48 /dev/dty03 crw-rw 1 uucp dialer 8, 524292 Mar 10 18:48 /dev/dty04 crw-rw 1 uucp dialer 8, 524293 Mar 10 18:48 /dev/dty05 crw-rw 1 uucp dialer 8, 524294 Mar 18 19:57 /dev/dty06 crw-rw 1 uucp dialer 8, 524295 Mar 10 18:47 /dev/dty07 crw-rw 1 uucp dialer 8, 524296 Mar 20 13:34 /dev/dty08 crw-rw 1 uucp dialer 8, 524297 Mar 10 18:46 /dev/dty09 crw-rw 1 uucp dialer 8, 524298 Mar 19 23:58 /dev/dty10 It's handy that I can type 'man dty' to see what kind they are, but beyond that, the listing is not user-friendly: apart from the stand-out modification time, nothing distinguishes /dev/dty02, the only node that has no hardware backing, from the rest. Nothing distinguishes dty00 - dty01, which are built into the motherboard, from dty03 - dty10, which reside on a puc(4). Luckily, dty03 - dty10 correspond directly to tentacles 1 - 8 of the puc(8). I think that your concern may be that a /dev/ node's name, major, and minor will change if we have, say, ttyP at comQ at pucR. It needn't be so---and, I say, it shouldn't. Let the (hypothetical) tty_attach_args convey either the type of (com, pty) and/or a minor/major (hmm...). I have to say that I really like the idea of splitting com from tty, with com providing a raw interface for changing (e.g.) RS-232 parameters, for reading, and for writing, and with tty providing the traditional TTY interface. Dave -- David Young OJC Technologies dyo...@ojctech.com Urbana, IL * (217) 278-3933
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 05:30:43PM -0400, Thor Lancelot Simon wrote: On Fri, Mar 19, 2010 at 09:23:35PM +, Herb Peyerl wrote: Last time I bought a cavium board it was $5k USD... An Octeon 3850 was $700 for 1521 piece part... I didn't think they had anything reasonable down below $500? (and as far as I remember, they already had FreeBSD running on the Octeons). Admittedly it's been a few years. Cavium and Raza (Now NetLogic) both have low-core-count parts in the way sub-$100 price range. Same basic architecture as the big parts (these aren't their cut-down 32-bit parts), just less cores and less goodies. And documentation? Dave -- David Young OJC Technologies dyo...@ojctech.com Urbana, IL * (217) 278-3933
MIPS SoC systems (was: Dead ports [Re: config(5) break down])
At Fri, 19 Mar 2010 21:23:35 +, Herb Peyerl hpey...@beer.org wrote: Subject: Re: Dead ports [Re: config(5) break down] On Fri, Mar 19, 2010 at 05:19:47PM -0400, Thor Lancelot Simon wrote: Have a look at http://www.rmicorp.com/assets/docs/2070SG_XLR_XLS_Product_Selection_Guide_2008-12-16.pdf specifically at the bottom few rows on the XLS chart. You're looking at parts that have 3 or 4 Gig-E interfaces, tons of useful hardware offload, and are, by published reports, way down in the sub-$50 range. You can get very similar stuff from Cavium. Last time I bought a cavium board it was $5k USD... An Octeon 3850 was $700 for 1521 piece part... I didn't think they had anything reasonable down below $500? (and as far as I remember, they already had FreeBSD running on the Octeons). Admittedly it's been a few years. FreeBSD is re-doing all its MIPS support, with quite a bit of work going into the Atheros and Cavium ports. Atheros is running, and some Cavium are running too, but not yet all the most interesting ones. Check out Warner Losh's postings: http://bsdimp.blogspot.com/search/label/mips I'm interested in bringing over some of those ports to NetBSD (though if I try to do it for my day job I'll need to bring over Netgraph first). Here's one company making Cavium-based systems at a reasonable price: http://www.portwell.com/products/detail.asp?CUSTCHAR1=CAM-0100 This one doesn't run FreeBSD yet, but someone is working on it and they are very close (it's not much different from the Cavium eval board Warner shows booting). They have a bunch of higher-end systems based on Cavium CPUs too (and some other CPUs too): http://www.portwell.com/products/MIPS.asp This company isn't as low-priced, but has similar devices. This one is just under $500, single unit: http://www.lannerinc.com/Network_Application_Platforms/Network_Processor_Platforms/Desktop_NPU_Platforms/MR-320 and they also have a wide product range: http://lannerinc.com/Network_Application_Platforms/Network_Processor_Platforms One of the cheapest Atheros boards is the Ubiquiti RouterStation series. You can get one in a case with power supply from various vendors now for just over $100, single unit pricing (the board is $80). http://www.ubnt.com/rspro This is one that FreeBSD runs on already, and I think adapting our AR53xx port to also work on its AR71xx SoC would be relatively easy. It's pretty snappy, but it has a poorly supported Ethernet switch chip that as yet limits it for use in my day job. When you start looking at what the GNU/Linux OpenWRT project supports, there are dozens of very interesting little systems available at relatively low prices. http://oldwiki.openwrt.org/TableOfHardware.html Routerboard.com (MikroTik) sell a bunch of interesting boards that even including their own proprietary GNU/Linux port licensing, are still quite cost effective. Most of the more powerful ones are AR71xx based. -- Greg A. Woods Planix, Inc. wo...@planix.com +1 416 218 0099http://www.planix.com/ pgpZAMKw7TNbn.pgp Description: PGP signature
Re: config(5) break down
On Thu, Mar 18, 2010 at 08:47:50PM -0400, Thor Lancelot Simon wrote: Sigh. Andy did; I didn't. I completely missed that he clearly excluded embedded platforms from his original comments. np. I *do* think it's a useful datapoint to note that sun2, pmax, algor, etc. are never, ever downloaded any more. Right, and these dead ports must be euthanized. The mountain of unused device drivers and core kernel code is a signficant hinderance to people working in the kernel. Random example: I had to abandon two paid efforts last year because I ran out of time owing to the huge amount of MD code, a lot of which is unused. That was TNF's money and my time down the toilet, and we missed out on multithreaded ttys and audio drivers (and a bucketload of bugfixes along with both).
Dead ports [Re: config(5) break down]
Andrew Doran a...@netbsd.org wrote: I *do* think it's a useful datapoint to note that sun2, pmax, algor, etc. are never, ever downloaded any more. Right, and these dead ports must be euthanized. The mountain of unused device drivers and core kernel code is a signficant hinderance to people working in the kernel. Fully agree (as one having this pain too). For those single developers who still have such machines, as a hardware replacement and moral compensation, TNF could buy some new ARM or MIPS board for hacking. Would be a win-win case, would not? -- Mindaugas
Re: Dead ports [Re: config(5) break down]
I *do* think it's a useful datapoint to note that sun2, pmax, algor, etc. are never, ever downloaded any more. Right, and these dead ports must be euthanized. The mountain of unused device drivers and core kernel code is a signficant hinderance to people working in the kernel. Fully agree (as one having this pain too). For those single developers who still have such machines, as a hardware replacement and moral compensation, TNF could buy some new ARM or MIPS board for hacking. Would be a win-win case, would not? Probably we should ask Core and Boards what's the best for TNF. I'm afraid there is no win-win case among many paranoiac geeks we have, but I won't hesitate to abandon all my machines and devices if it's better for TNF. No replacements are required. IMO, tiering ports might be better for us than best or die scheme. In that case you can leave unused ports and devices in dead state. --- Izumi Tsutsui
Re: Dead ports [Re: config(5) break down]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, On Mar 19, 2010, at 4:14 PM, David Young wrote: Regardless of what we do or do not do with sun2 et cetera, TNF could buy some ARM and MIPS boards for developers. ARM and MIPS boards, however, are not so precious as a sun2. In fact, they're abundant, and cheap. Nevertheless, there does not seem to be much interest in porting to them. Why is that? I don't care about hardware without graphics ;) *duck* have fun Michael -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iQEVAwUBS6PdwcpnzkX8Yg2nAQLEbAgAt7l14I8GJvIVxYHB9b16LsQuOdY+WaS+ Kf+4qupZ9ZbvgYtNYvtPT1yklMoiAIDxCZAJF80lQyXH0OcKdLajmYMShnV/wCww X1gBVLbcO+85UzaA9a9ekEIH6bpbPChJizVqNxvGKpoaPKNdl6nXEoly7P3DmmYb fa0sh5LdKjBKx1ZTnGR1E6TAmI0MqFg1RHjYjibBT5FiJQF/IMbbpepoEfbUi/MC gLEj9RyG6xw/vwHnxH1HdJKNWwtRJv/jJcgdmPytw6Si58yoAEI2UJowIynXQjyh bEyPOyUPwlkHRKLpJl3CqxgEck7qwbbL57TMd7EYDGAF0R5k1DqcSg== =uyR5 -END PGP SIGNATURE-
Re: Dead ports [Re: config(5) break down]
der Mouse mo...@rodents-montreal.org wrote: [...] sun2, pmax, algor, etc. [...] Right, and these dead ports must be euthanized. For those single developers who still have such machines, as a hardware replacement and moral compensation, TNF could buy some new ARM or MIPS board for hacking. Would be a win-win case, would not? Wow. Talk about missing the point. There is a reason why I put hardware replacement in quotes. Because of exactly what you described as fundamentally emotionally motivated. Given reason to think that $PORT does not actually have any users, perhaps. But, as various people have pointed out, lack of downloads of binary builds is hardly evidence of that. ... And I never said that downloads is justification for removal of any port. In fact, ARM and MIPS have zero point something of all downloads. They are nowhere near being unused, though. However, point being is that a *lot* of time is spent (when revamping certain interfaces) by MI kernel developers going through, for example, barely maintained m68k ports with a *massive* code duplication, acorn26 (in 3 or so months I could find only *one* person who could do testing on it), museum's sun2 (IIRC, Izumi was the only who cared?) and so on. Somewhere, I have a dusting clone of ZX Spectrum called Santaka 002, but I am not of the opinion that it should be supported by NetBSD. Not only because it is unrealistic. It would be a total maintenance pain (yet another one) for *others*. -- Mindaugas
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 03:14:57PM -0500, David Young wrote: Regardless of what we do or do not do with sun2 et cetera, TNF could buy some ARM and MIPS boards for developers. ARM and MIPS boards, however, are not so precious as a sun2. In fact, they're abundant, and cheap. Nevertheless, there does not seem to be much interest in porting to them. Why is that? Neither so abundant nor so cheap as you think, it seems to me, if you want something that's modern enough to really be useful and has a vendor friendly enough to make software development specifically for the platform an exercise anyone would want to do in his spare time...! That loongson-based laptop might be one cute exception to the rule. But eval boards for things like MIPS network processors are not exactly free, and documentation not trivial to come by without NDA. If you're going to buy 10,000 and make your own boards, sure, these platforms are cheaper than things like Soekris. But qty 1 it's a whole different ballpark, which in my opinion is why Soekris even exists at all! Thor
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 04:45:21PM -0400, Thor Lancelot Simon wrote: On Fri, Mar 19, 2010 at 03:14:57PM -0500, David Young wrote: Regardless of what we do or do not do with sun2 et cetera, TNF could buy some ARM and MIPS boards for developers. ARM and MIPS boards, however, are not so precious as a sun2. In fact, they're abundant, and cheap. Nevertheless, there does not seem to be much interest in porting to them. Why is that? Neither so abundant nor so cheap as you think, it seems to me, if you want something that's modern enough to really be useful and has a vendor friendly enough to make software development specifically for the platform an exercise anyone would want to do in his spare time...! I guess that it depends on your application whether these boards are useful or not, but these places sell inexpensive boards that have more-or-less open architecture/documentation: www.embeddedarm.com www.openplug.org www.routerboard.com www.ubnt.com In NetBSD, we have support for only a few of the boards at those links. I don't think that it's because TNF does not buy developers the boards. Dave -- David Young OJC Technologies dyo...@ojctech.com Urbana, IL * (217) 278-3933
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 05:19:47PM -0400, Thor Lancelot Simon wrote: Have a look at http://www.rmicorp.com/assets/docs/2070SG_XLR_XLS_Product_Selection_Guide_2008-12-16.pdf specifically at the bottom few rows on the XLS chart. You're looking at parts that have 3 or 4 Gig-E interfaces, tons of useful hardware offload, and are, by published reports, way down in the sub-$50 range. You can get very similar stuff from Cavium. Last time I bought a cavium board it was $5k USD... An Octeon 3850 was $700 for 1521 piece part... I didn't think they had anything reasonable down below $500? (and as far as I remember, they already had FreeBSD running on the Octeons). Admittedly it's been a few years.
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 09:23:35PM +, Herb Peyerl wrote: Last time I bought a cavium board it was $5k USD... An Octeon 3850 was $700 for 1521 piece part... I didn't think they had anything reasonable down below $500? (and as far as I remember, they already had FreeBSD running on the Octeons). Admittedly it's been a few years. Cavium and Raza (Now NetLogic) both have low-core-count parts in the way sub-$100 price range. Same basic architecture as the big parts (these aren't their cut-down 32-bit parts), just less cores and less goodies. Even the very bottom of the line parts typically still have multiple GigE interfaces on them and various other highly useful stuff. -- Thor Lancelot Simont...@rek.tjls.com All of my opinions are consistent, but I cannot present them all at once.-Jean-Jacques Rousseau, On The Social Contract
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 05:30:43PM -0400, Thor Lancelot Simon wrote: Cavium and Raza (Now NetLogic) both have low-core-count parts in the way sub-$100 price range. Same basic architecture as the big parts (these aren't their cut-down 32-bit parts), just less cores and less goodies. Even the very bottom of the line parts typically still have multiple GigE interfaces on them and various other highly useful stuff. FYI -- and I doubt you could really buy just one of these at this price -- http://avnetexpress.avnet.com/store/em/EMController/Network-Processor/Cavium-Networks/CN5220-500BG729-SCP-Y-G/_/R-8960980/A-8960980/An-0?action=partcatalogId=500201langId=-1storeId=500201 Some of the other things I've been talking about have lower pricing in the real world than this does. Thor
Re: Dead ports [Re: config(5) break down]
On Fri, Mar 19, 2010 at 06:19:27PM -0400, Thor Lancelot Simon wrote: FYI -- and I doubt you could really buy just one of these at this price -- http://avnetexpress.avnet.com/store/em/EMController/Network-Processor/Cavium-Networks/CN5220-500BG729-SCP-Y-G/_/R-8960980/A-8960980/An-0?action=partcatalogId=500201langId=-1storeId=500201 Some of the other things I've been talking about have lower pricing in the real world than this does. Nice. The 3850 was a really nice part.. I had a lot of fun writing code for it...
Re: Dead ports [Re: config(5) break down]
On Fri, 19 Mar 2010, Thor Lancelot Simon wrote: FYI -- and I doubt you could really buy just one of these at this price -- http://avnetexpress.avnet.com/store/em/EMController/Network-Processor/Cavium-Networks/CN5220-500BG729-SCP-Y-G/_/R-8960980/A-8960980/An-0?action=partcatalogId=500201langId=-1storeId=500201 I tried to get Avnet to provide a quote on an eval board for an Octeon and they never got back to me. (The eval board didn't have a Buy_Now button, only a Request_Quote button.) - | Paul Goyette | PGP DSS Key fingerprint: | E-mail addresses: | | Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com | | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net | | Kernel Developer | | pgoyette at netbsd.org | -
Re: config(5) break down
On Fri, Mar 19, 2010 at 11:49 PM, Andrew Doran a...@netbsd.org wrote: Random example: I had to abandon two paid efforts last year because I ran out of time owing to the huge amount of MD code, a lot of which is unused. That was TNF's money and my time down the toilet, and we missed out on multithreaded ttys and audio drivers (and a bucketload of bugfixes along with both). audio(4) and tty(4) are very different from my view; audio(4) has the single entry and drivers implement backend. tty(4) is a common interface through which kernel accesses serial devices (correct me if wrong). I'm very curious which architecture looks better to you, because I have a vague idea to make tty(4) to something like audio(4). Masao
Re: Dead ports [Re: config(5) break down]
On Mar 19, 2010, at 1:25 PM, Michael wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, On Mar 19, 2010, at 4:14 PM, David Young wrote: Regardless of what we do or do not do with sun2 et cetera, TNF could buy some ARM and MIPS boards for developers. ARM and MIPS boards, however, are not so precious as a sun2. In fact, they're abundant, and cheap. Nevertheless, there does not seem to be much interest in porting to them. Why is that? I don't care about hardware without graphics ;) I don't care about hardware with graphics. :)
Re: config(5) break down
On Tue, Mar 16, 2010 at 03:30:15PM -0400, Thor Lancelot Simon wrote: But bogus justifications involving who downloads what built binary releases from the FTP server are not really helpful to anyone. Huh??
Re: config(5) break down
On Thu, Mar 18, 2010 at 10:20:28AM +, Andrew Doran wrote: On Tue, Mar 16, 2010 at 03:30:15PM -0400, Thor Lancelot Simon wrote: But bogus justifications involving who downloads what built binary releases from the FTP server are not really helpful to anyone. Huh?? Did you read what I was responding to? -- Thor Lancelot Simont...@rek.tjls.com All of my opinions are consistent, but I cannot present them all at once.-Jean-Jacques Rousseau, On The Social Contract
Re: config(5) break down
On Thu, Mar 18, 2010 at 09:07:51AM -0400, Thor Lancelot Simon wrote: On Thu, Mar 18, 2010 at 10:20:28AM +, Andrew Doran wrote: On Tue, Mar 16, 2010 at 03:30:15PM -0400, Thor Lancelot Simon wrote: But bogus justifications involving who downloads what built binary releases from the FTP server are not really helpful to anyone. Huh?? Did you read what I was responding to? Sigh. Andy did; I didn't. I completely missed that he clearly excluded embedded platforms from his original comments. I *do* think it's a useful datapoint to note that sun2, pmax, algor, etc. are never, ever downloaded any more. It's with regard to things like sbmips that I think those data are highly likely to be misleading. Thor
Re: config(5) break down
On Tue, Mar 16, 2010 at 08:01:31PM +, David Holland wrote: But recompiling things isn't a complex unautomated procedure, it's a complex automated procedure, and not really that much different from other complex automated procedures like binary updates. The difference here is that a binary update is changing one particular machine and updating some other machine obviously won't have the intended effect, but recompiling things does the exact same thing regardless of where you do it, so having multiple people do it seems like a waste of time. Nor is it necessarily slow; building a kernel doesn't take any longer than booting Vista... Maybe on your machine. On mine it's still quite a bit slower than just editing a config file. eric
Re: config(5) break down
On Wed, Mar 17, 2010 at 11:10:59AM -0500, Eric Haszlakiewicz wrote: On Tue, Mar 16, 2010 at 08:01:31PM +, David Holland wrote: But recompiling things isn't a complex unautomated procedure, it's a complex automated procedure, and not really that much different from other complex automated procedures like binary updates. The difference here is that a binary update is changing one particular machine and updating some other machine obviously won't have the intended effect, but recompiling things does the exact same thing regardless of where you do it, so having multiple people do it seems like a waste of time. That's a red herring; applying a binary patch does the same thing everywhere, and recompiling updates one particular machine in exactly the same sense too. The difference is in what material is distributed and how and where it's processed. This is not something end users are going to care about much - at most they'll care about how long it takes. Which is a valid concern, of course, especially in extreme examples like building firefox on a sun3, but it's *not* a usability issue in the same sense that e.g. incomprehensible error messages are. Admittedly, neither CVS nor our build system is quite robust enough to make this really work, but in practice tools like apt-get and yum aren't quite, either. Anyhow, it seems to me that a blanket statement nobody should ever have to recompile anything requires some justification; however, people have been taking it as an axiom lately and that concerns me a bit. Nor is it necessarily slow; building a kernel doesn't take any longer than booting Vista... Maybe on your machine. On mine it's still quite a bit slower than just editing a config file. Well sure, but that just means we're way ahead of the competition, since in Windows editing that config file generally requires a reboot. :-) -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
On Tue, Mar 16, 2010 at 05:26:20AM +, David Holland wrote: After a fashion. Check how our LOCKDEBUG works. :-/ You mean crawls? Martin
Re: config(5) break down
On Tue, Mar 16, 2010 at 06:22:14AM +0100, Wojciech A. Koszek wrote:
You mean that you have a solution for:
struct mystruct {
#ifdef DEBUG_MYSTRUCT
int line;
char*file;
char*func;
void*another_pointer;
#endif
...
};
within a kernel code? That's the simpliest example, of course. There are
areas where you simply can't prevent this kind of #ifdef's.
Yes: don't code like that. :-)
That's not to say it doesn't happen because there is a culture
of using #ifdef to deal with things that should be taken care of at
runtime, and to optimize things that don't need optimizing.
Old habits are hard to break.
Re: config(5) break down
On Tue, Mar 16, 2010 at 06:42:29AM +0100, Wojciech A. Koszek wrote: coming from the same build and with the same set of critical options. Otherwise, if the struct mutex changes its size due to #define LOCK_DEBUGGING in the kernel, you'll going to get a page fault from the module's code with a useless trace message. FYI, that's not a problem for NetBSD. The ABI is insensitive to things like DIAGNOSTIC/DEBUG/LOCKDEBUG/-g/whatever. By design because it caused so much hassle in the past with userspace tools and was one of the major pain points with LKMs.
Re: config(5) break down
Julio Merino wrote: On Mon, Mar 15, 2010 at 11:22 PM, Johnny Billquist b...@softjar.se wrote: Julio Merino wrote: On Mon, Mar 15, 2010 at 10:08 PM, Masao Uebayashi uebay...@gmail.com wrote: Let me clarify: - NetBSD is used for many purposes. - The official binary should choose the sane default - FFS_EI enabled by default - XIP would be disabled by default - While leave developers freedom to customize NetBSD more. Err, no. BOTH developers and users should have the freedom to customize NetBSD, not only developers. And, of course, users should not have to rebuild anything, ever. If we can't provide such a binary-friendly system, we have failed as developers. And it's possible to provide such a system, as has been proven by Solaris, Windows and, increasingly, Linux. So what are you saying here? That our goal is to be like Solaris, Windows and Linux? I never said that. But last I read, NetBSD has never been targeted exclusively to people working on embedded systems. Hmm. So you mean that the only other group is embedded systems? No, you never said that NetBSD should be Linux, Windows or Solaris, you only used those as reference points on what could/should be done. The that's a fine difference. What is the point in that? There already exist several systems (as you note) who are that. Do you expect NetBSD to be in that same segment? I expect NetBSD to be as flexible as reasonably possible with the binaries we distribute. If we have to tell any user rebuild the software with option foo to get what they want, we have failed at that. And most users will run away. I would dare say that any user who decided to try NetBSD would cease use it, and not because of the build-you-own-kernel reason, but for the simple fact that there are a lot of alternatives out there which better fulfills his needs. The most important one being a lot of commercial or semi-commercial programs which don't run, or run poorly on NetBSD. So yes, most people are already running away. And the config system and kernel building isn't the reason. Let's instead ask us who, and why, some people do drift over to NetBSD? I would say that a large portion of those are people who for some reason or other have gotten tired of the magical modules, autoconfiguration, and magic tools that manage the system for you, and who wants to have better control and understanding of the system. Or else, who are curious about alternative ways of doing things, or possibly curious about how Unix systems used to do it. Thus, a big change to the philosophy of NetBSD here means we still would not attract the random users, but we would drive away the small following we do have. But hey. That is just my opinion, and I don't have any hard facts to back them up. Again, what I meant is: providing the source code of any application is not an excuse to having a deficient design that does not allow extensibility or customization without rebuilding. And that's orthogonal to being Solaris or whatever. But sometimes things do clash. I'm not against solutions that extend and refine what NetBSD do, but I do think it's the wrong way to try to redefine the system. And especially doing so with the argument that others do it that way. But, as I've said a number of times, I'm an old fart who is very conservative and backwards. Heck, I even run NetBSD on my VAXen, although I have not gone past NetBSD 3... Johnny
NetBSD binary [was Re: config(5) break down]
could you please use subject lines that somewhat reflect the content of the discussion please? I was surprised to find a discussion like this under that subject, or maybe you want to sneak it through? :) On Tue, Mar 16, 2010 at 11:39:07AM +0100, Johnny Billquist wrote: Julio Merino wrote: (...) I expect NetBSD to be as flexible as reasonably possible with the binaries we distribute. If we have to tell any user rebuild the software with option foo to get what they want, we have failed at that. And most users will run away. Well, if you tell them, run this script and reboot to configure your system for your needs, then most users would sign that. And that's all our (cross-)building is. Run a script. Now if the source is not properly maintained because someone keeps yelling we need binary kernel modules that don't install using said script... or build time options turn over to bitrot because you could as well keep and load all the relevant code... well, that's when NetBSD fails. but ... I would dare say that any user who decided to try NetBSD would cease use it, and not because of the build-you-own-kernel reason, but for the simple fact that there are a lot of alternatives out there which better fulfills his needs. The most important one being a lot of commercial or semi-commercial programs which don't run, or run poorly on NetBSD. Absolutely. That's the #1 reason linux guys who are initially quite charmed with NetBSD go away again. Hey! We don't have flash, skype, ... (and don't tell me about our emulator stuff. It's neat but it's not really a solution to the problem) So yes, most people are already running away. And the config system and kernel building isn't the reason. Actually the experience I had with my NetBSD advocacy in the windows and linux circles I used to do that was that everybody was quite positively impressed about the clarity of kernel config files, documentation and building. Let's instead ask us who, and why, some people do drift over to NetBSD? I would say that a large portion of those are people who for some reason or other have gotten tired of the magical modules, autoconfiguration, and magic tools that manage the system for you, and who wants to have better control and understanding of the system. Or else, who are curious about alternative ways of doing things, or possibly curious about how Unix systems used to do it. Absolutely! Thanks for these wise words, johnny. -Martin
Re: NetBSD binary [was Re: config(5) break down]
On Tue, 16 Mar 2010, Martin S. Weber wrote: Well, if you tell them, run this script and reboot to configure your system for your needs, then most users would sign that. And that's all our (cross-)building is. Run a script. Now if the source is not properly maintained because someone keeps yelling we need binary kernel modules that don't install using said script... or build time options turn over to bitrot because you could as well keep and load all the relevant code... well, that's when NetBSD fails. but ... I find that config and modules are at cross purposes. Config gives you the luxury of customizing the compilation process. Modules should allow you to replace indivcual binaries independent of the rest of the kernel. If I can't use a module someone else compiled without having to reconfigure and rebuild my kernel I fail to comprehend the purpose of having a modular kernel. If you want to reduce the kernel memory footprint, compile a custom kernel with only the code you want and you save yourself the overhead of the kernel linker and binary (in most cases ELF) headers and linkage goo. If you want to add functionality at runtime, we've had LKMs for a long time. With a modular kernel that uses config you not only need to build a custom kernel with information about what devices exist on the machine, but you also get the opportunity to lose the code at runtime. (Oops! I copied over a new kernel but forgot to copy over /modules!) Eduardo
Re: config(5) break down
2010/3/16 Julio Merino j...@netbsd.org: On Mon, Mar 15, 2010 at 10:08 PM, Masao Uebayashi uebay...@gmail.com wrote: Let me clarify: - NetBSD is used for many purposes. - The official binary should choose the sane default - FFS_EI enabled by default - XIP would be disabled by default - While leave developers freedom to customize NetBSD more. Err, no. BOTH developers and users should have the freedom to customize NetBSD, not only developers. And, of course, users should not have to rebuild anything, ever. If we can't provide such a binary-friendly system, we have failed as developers. And it's possible to provide such a system, as has been proven by Solaris, Windows and, increasingly, Linux. (That doesn't mean developers shouldn't have the ability to rebuild from source. Of course they should, that's the joy of open source! But being open source is not an excuse to provide a system that cannot be customized when distributed in binary form. Otherwise, that's no different than the no documentation because you can read the source policy...) -- Julio Merino I'm wholeheartedly behind Julio's statement. Users should never have to rebuild anything. And as Julio stated in the past, this applies to pkgsrc aswell, where we recently did a major step forward with pkgin. Zafer.
Re: NetBSD binary [was Re: config(5) break down]
Let's instead ask us who, and why, some people do drift over to NetBSD? I would say that a large portion of those are people who for some reason or other have gotten tired of the magical modules, autoconfiguration, and magic tools that manage the system for you, and who wants to have better control and understanding of the system. Absolutely! Thanks for these wise words, johnny. What they said. Thank you, Johnny, for putting it so eloquently. This is very much what bothers me about the directions NetBSD is headed, and has for quite a while: it appears to be trying to turn itself into Yet Another Unix Variant, different from Linux and Solaris and their ilk only in the brand name on the case. People say things like we'll never win over the Linux users without $FEATURE!, which makes me say why do you want to?. If I want Linux - or Solaris, or any of various other Unices - I know where to find it; if NetBSD succeeds in turning into a Linux - or whatever - clone, it will have lost the reasons it previously gave me to prefer it. But we're not trying to do that! Enough of you are that that's NetBSD's direction. You're trying to come up with point-and-drool installers. You're trying to fully support desktop environments. You're trying to make system administration easy for non-sysadmins. You are, in short, trying to give it the things I picked NetBSD to get away from, the things Johnny summarized so well: lots of magic code doing things users are not only not expected to understand but are expected to not understand (and under whose hood prying is not supported). /etc/rc.d/*. Modular kernels. Sysinst. build.sh. I don't want a system that has any code under whose hood prying is not supported; if the answer to I'm having trouble with $FOO is messing with $FOO is not supported; use our magic code instead, that's a bug as far as I'm concerned. I've seen it called elitist to prefer, for example, installing by hand. I can understand that point of view, but I think it misses the point fairly fundamentally. It's the difference between inclusive and exclusive - I don't want to keep hoi polloi out by demanding understanding before they are worthy to (say) install NetBSD; rather, I want to bring them in by imparting that understanding, with things like manual installation serving as teaching (and self-testing) tools. Nothing teaches like experience. Yes, this excludes the people who don't understand and don't want to. To steal a term from marketing, I don't think NetBSD should try to serve that market segment; it's already well-served by others, and I see no percentage in trying to join them. It doesn't serve them better (indeed, by adding yet another alternative they neither are nor want to be competent to choose among, it serves them worse) and it doesn't serve NetBSD (people who don't even want to understand are among the least likely to turn into developers and contribute back). So what's to like? /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: config(5) break down
At Tue, 16 Mar 2010 10:22:42 +, Andrew Doran a...@netbsd.org wrote: Subject: Re: config(5) break down Correctamundo. 95% of downloads in the week following the release of 5.0 were for x86. It doesn't say much about embedded but does tell us that a very large segment of the user population does commodity hardware. (What the figures also revealed was that a number of the ports had as close to zero downloads as matters. Which is, to be frank, a red flag for those that are not maintained.) Please do not even think about using downloads as a measure of which ports are used and how much they are used! That's a completely invalid measurement of how NetBSD might be used. Many of us just download the source. We don't tell you which parts of it that we use or don't use. Even port-* mailing list subscriptions aren't a truly valid hint of which ports are used or how much they are used. -- Greg A. Woods Planix, Inc. wo...@planix.com +1 416 218 0099http://www.planix.com/ pgpVTS2ubXFh7.pgp Description: PGP signature
Re: config(5) break down
On Tue, Mar 16, 2010 at 03:18:10PM -0400, Greg A. Woods wrote: Please do not even think about using downloads as a measure of which ports are used and how much they are used! That's a completely invalid measurement of how NetBSD might be used. No kidding. We'll ship thousands of units of products with NetBSD firmware on them in the next year. I doubt we download more than one or two copies of NetBSD a month, for miscellaneous use on developers' machines -- if that. I don't care what's done in NetBSD to make fancy all-singing, all-dancing desktop installations nicer *SO LONG AS IT DOESN'T GET IN MY WAY*. Bloating up the base system's libraries so I have to do more customization gets in my way. Making kernel configurations like I use a pain in the butt to keep working gets in my way. Lots of other things people want to do in NetBSD to support desktop use don't, and some are even positive for me. Please consider that many different user populations do many different things with NetBSD when proposing major changes, and I can't see how we cannot manage to sustain those many different populations in the future. But bogus justifications involving who downloads what built binary releases from the FTP server are not really helpful to anyone. Thor
Re: config(5) break down
On Tue, Mar 16, 2010 at 06:50:29PM +0100, Zafer Aydo?an wrote: I'm wholeheartedly behind Julio's statement. Users should never have to rebuild anything. Er, why? Users should never have to perform complex unautomated procedures, because such procedures can easily be screwed up and recovery becomes difficult or impossible. But recompiling things isn't a complex unautomated procedure, it's a complex automated procedure, and not really that much different from other complex automated procedures like binary updates. Nor is it necessarily slow; building a kernel doesn't take any longer than booting Vista... -- David A. Holland dholl...@netbsd.org
Re: NetBSD binary [was Re: config(5) break down]
Dear participants in this thread, If you feel the need to share your general opinions about NetBSD or the direction of development, please continue this on the netbsd-us...@netbsd.org mailing list. Please respect the users which have subscribed to the tech- mailing lists to participate in technical, specific and solution-oriented discussions. Many thanks and best regards, Mark Weinem
Re: config(5) break down
2010/3/15 Wojciech A. Koszek wkos...@freebsd.org: device X builds device X staticly into the kernel (and maybe does something device-specific), while: module X Only builds a KLD. I picked module, since it seems to be more-or-less an oposite of: static X which could build feature X, which is not a device staticly. I think your config(8) has this problem solved somehow, since you seem to have filesystem keyword as well. Nowadays, given that as you mentioned for NetBSD, in FreeBSD we also have no scoping for config(8), we must build all KLDs just in case someone needs them, since we don't know which file belongs to which module. Who writes these in what file? I was wondering how does Linux/Solaris kernel build system work in terms of opt_*.h files? Do they have some alternative solutions for #ifdef's based on what has been included into the kernel at configuration time? Without looking them, I don't think any infrastructural (== config(1) itself) change helps. Why not fix source code rather than improving config(1)? Masao
Re: config(5) break down
On Mon, Mar 15, 2010 at 11:50:09PM +0900, Masao Uebayashi wrote: [...] Without looking them, I don't think any infrastructural (== config(1) You know, saying stuff like without looking at [this], or i haven't read any of [that], while honest, is not a very good way of leading a conversation. People might just think you're condescending. itself) change helps. Why not fix source code rather than improving config(1)? Because, like it or not, config(1) has to change, because autoconf(9) has to change, too. I know it's not part of your own agenda, but it's a fact. Now, you can choose to dismiss any opinion that includes a change in config(1), but you might also want to think about how changing config(1) could help you. If some people are going to change it anyway, why wouldn't you consider the benefits of adding changes that serve your agenda? -- Quentin Garnier - c...@cubidou.net - c...@netbsd.org See the look on my face from staying too long in one place [...] every time the morning breaks I know I'm closer to falling KT Tunstall, Saving My Face, Drastic Fantastic, 2007. pgp4nfzzfgTl8.pgp Description: PGP signature
Re: config(5) break down
On Sun, 14 Mar 2010, Wojciech A. Koszek wrote: I was wondering how does Linux/Solaris kernel build system work in terms of opt_*.h files? Do they have some alternative solutions for #ifdef's based on what has been included into the kernel at configuration time? It's been a while since I poked around with Linux, but I think they have a single file that contains all that info. Solaris has no config. Period. You don't have any opt_*.h files. And you don't really need them. It's a fully dynamic kernel so the only things you can set are things like DEBUG and LOCKDEBUG. You don't worry about including specific modules, because they are all compiled separately and loaded on demand. Loading of modules is driven by config files in /etc that define what module is responsible for a specific device type (e.g. a specific PCI vendor and device ID combination.) Of course, if those config files get corrupted then the machine is unbootable. To recover you must boot from install media and likely reinstall the OS. And since everything is compiled separately you can often just replace one module with another one that is compiled with DEBUG turned on. Without rebooting the machine. (Certain inter-module interfaces are affected by DEBUG while others are not. YMMV.) Eduardo
Re: config(5) break down
On Tue, Mar 16, 2010 at 1:57 AM, Eduardo Horvath e...@netbsd.org wrote: On Sun, 14 Mar 2010, Wojciech A. Koszek wrote: I was wondering how does Linux/Solaris kernel build system work in terms of opt_*.h files? Do they have some alternative solutions for #ifdef's based on what has been included into the kernel at configuration time? It's been a while since I poked around with Linux, but I think they have a single file that contains all that info. My understanding is splitting opt_*.h into small files is just only to save rebuild time. Is this right? It's also same as GNU Autoconf + configure + #include config.h do. Solaris has no config. Period. You don't have any opt_*.h files. And you don't really need them. It's a fully dynamic kernel so the only things you can set are things like DEBUG and LOCKDEBUG. You don't worry about including specific modules, because they are all compiled separately and loaded on demand. Loading of modules is driven by config files in /etc that define what module is responsible for a specific device type (e.g. a specific PCI vendor and device ID combination.) Of course, if those config files get corrupted then the machine is unbootable. To recover you must boot from install media and likely reinstall the OS. Ah. Thanks for explaining this. This is pretty much expected. And this is what NetBSD should not be, IMO, because - one of NetBSD's targets is highly customized embedded purposes - let users customize as they want - NetBSD is free, unlike QNX. - users are not stupid - but those customized binaries are not supported And since everything is compiled separately you can often just replace one module with another one that is compiled with DEBUG turned on. Without rebooting the machine. (Certain inter-module interfaces are affected by DEBUG while others are not. YMMV.) Thanks, and this is also pretty much expected too. IIRC Windows did similar thing; providing foo.dll foodbg.dll. I think this strategy (== providing normal+debug binaries as official module binaries) would work for us too. Masao
Re: config(5) break down
On Tue, Mar 16, 2010 at 6:55 AM, Masao Uebayashi uebay...@gmail.com wrote: And this is what NetBSD should not be, IMO, because - one of NetBSD's targets is highly customized embedded purposes - let users customize as they want - NetBSD is free, unlike QNX. - users are not stupid - but those customized binaries are not supported Let me clarify: - NetBSD is used for many purposes. - The official binary should choose the sane default - FFS_EI enabled by default - XIP would be disabled by default - While leave developers freedom to customize NetBSD more. Masao
Re: config(5) break down
On Mon, Mar 15, 2010 at 10:08 PM, Masao Uebayashi uebay...@gmail.com wrote: Let me clarify: - NetBSD is used for many purposes. - The official binary should choose the sane default - FFS_EI enabled by default - XIP would be disabled by default - While leave developers freedom to customize NetBSD more. Err, no. BOTH developers and users should have the freedom to customize NetBSD, not only developers. And, of course, users should not have to rebuild anything, ever. If we can't provide such a binary-friendly system, we have failed as developers. And it's possible to provide such a system, as has been proven by Solaris, Windows and, increasingly, Linux. (That doesn't mean developers shouldn't have the ability to rebuild from source. Of course they should, that's the joy of open source! But being open source is not an excuse to provide a system that cannot be customized when distributed in binary form. Otherwise, that's no different than the no documentation because you can read the source policy...) -- Julio Merino
Re: config(5) break down
Julio Merino wrote: On Mon, Mar 15, 2010 at 10:08 PM, Masao Uebayashi uebay...@gmail.com wrote: Let me clarify: - NetBSD is used for many purposes. - The official binary should choose the sane default - FFS_EI enabled by default - XIP would be disabled by default - While leave developers freedom to customize NetBSD more. Err, no. BOTH developers and users should have the freedom to customize NetBSD, not only developers. And, of course, users should not have to rebuild anything, ever. If we can't provide such a binary-friendly system, we have failed as developers. And it's possible to provide such a system, as has been proven by Solaris, Windows and, increasingly, Linux. So what are you saying here? That our goal is to be like Solaris, Windows and Linux? What is the point in that? There already exist several systems (as you note) who are that. Do you expect NetBSD to be in that same segment? Just curious about what people think/want here... Johnny -- Johnny Billquist || I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip - B. Idol
Re: config(5) break down
On Tue, Mar 16, 2010 at 06:55:36AM +0900, Masao Uebayashi wrote: On Tue, Mar 16, 2010 at 1:57 AM, Eduardo Horvath e...@netbsd.org wrote: On Sun, 14 Mar 2010, Wojciech A. Koszek wrote: I was wondering how does Linux/Solaris kernel build system work in terms of opt_*.h files? Do they have some alternative solutions for #ifdef's based on what has been included into the kernel at configuration time? It's been a while since I poked around with Linux, but I think they have a single file that contains all that info. My understanding is splitting opt_*.h into small files is just only to save rebuild time. Is this right? It's also same as GNU Autoconf + configure + #include config.h do. I think it is also for narrowing the impact of particular options; it tends to act as a sort of layering and encapsulation. And saves a bit of confusion when a commiter enabled his debugging facility with DEBUG, which may not be unique. And since everything is compiled separately you can often just replace one module with another one that is compiled with DEBUG turned on. Without rebooting the machine. (Certain inter-module interfaces are affected by DEBUG while others are not. YMMV.) Thanks, and this is also pretty much expected too. IIRC Windows did similar thing; providing foo.dll foodbg.dll. I think this strategy (== providing normal+debug binaries as official module binaries) would work for us too. You mean that the delivery of two versions of each kernel module could eventually solve the problem for you? -- Wojciech A. Koszek wkos...@freebsd.org http://FreeBSD.czest.pl/~wkoszek/
Re: config(5) break down
On Thu, Mar 11, 2010 at 07:31:58PM +, David Holland wrote:
On Mon, Mar 08, 2010 at 06:33:04PM +0900, Masao Uebayashi wrote:
Well, first of all nothing says you can't read the whole file before
resolving dependencies; there's nothing inherently wrong with
define foo: bar
:
define bar: baz
:
I have no idea if config currently allows this but it's not exactly
difficult to arrange.
I don't think it's worth.
[..]
Mm... not strictly. That's only true when there are diamonds in the
dependency graph; otherwise, declaring B inside A just indicates that
B depends on A. Consider the following hackup of files.ufs:
file ufs/mfs/mfs_miniroot.c
module UFS {
option UFS_DIRHASH {
flag opt_ffs.h
file ufs/ufs/ufs_dirhash.c
}
[..]
module LFS : filesystem {
option LFS_KERNEL_RFW {
flag opt_lfs.h
file ufs/lfs/lfs_rfw.c
}
file ufs/lfs/lfs_alloc.c
file ufs/lfs/lfs_balloc.c
file ufs/lfs/lfs_bio.c
file ufs/lfs/lfs_cksum.c
file ufs/lfs/lfs_debug.c
file ufs/lfs/lfs_inode.c
file ufs/lfs/lfs_itimes.c
file ufs/lfs/lfs_segment.c
file ufs/lfs/lfs_subr.c
file ufs/lfs/lfs_syscalls.c
file ufs/lfs/lfs_vfsops.c
file ufs/lfs/lfs_vnops.c
}
}
The FreeBSD config(8) unsuprisingly suffers from the same problems. Syntax
presented above makes sense. In FreeBSD, we can also override per-file
compiler's flag, but files with custom compiler settings tend to come from the
same group.
Even though I don't like the idea of recursively embedded stuff, I think that:
group X {
require Y;
require Z;
file a.c
file b.c
}
Could work and make syntax a bit clearer. Anyway, this could even be more fully
featured:
group X {
require Y;
require Z;
if (option(X_DEBUG)) {
file X_debug.c;
}
}
I picked group, since I believe there'd be a demand on decreasing kernel
build time for developers. So:
device X
builds device X staticly into the kernel (and maybe does something
device-specific), while:
module X
Only builds a KLD. I picked module, since it seems to be more-or-less an
oposite of:
static X
which could build feature X, which is not a device staticly. I think your
config(8) has this problem solved somehow, since you seem to have filesystem
keyword as well. Nowadays, given that as you mentioned for NetBSD, in FreeBSD
we also have no scoping for config(8), we must build all KLDs just in case
someone needs them, since we don't know which file belongs to which module.
This could also let us to have one parser for files.*, options.* and kernel
configuration files, like GENERIC.
I was wondering how does Linux/Solaris kernel build system work in terms of
opt_*.h files? Do they have some alternative solutions for #ifdef's based on
what has been included into the kernel at configuration time?
--
Wojciech A. Koszek
wkos...@freebsd.org
http://FreeBSD.czest.pl/~wkoszek/
Re: config(5) break down
That line of reasoning only makes sense if splitting things into
multiple files provides some kind of scoping, or encapsulation, or
other form of abstraction.
One *.conf matches one module. We'll always build modules as single object
like intermediate *.a in userland. Expose only necessary symbols. Multiple
*.c are optimized by global optimizer. We can identify API - what symbols
it exposes or references. Modules are not only for dynamic loading.
Given that config language has no scoping at all right now, the first
order of business would be to add some of that, like
module vfs {
file kern/vfs_vnops.c
file kern/vfs_xattr.c
:
}
It can be written as:
define vfs
file kern/vfs_vnops.c vfs
file kern/vfs_xattr.c vfs
I don't see what's different.
but putting each one of those in its own file isn't going to serve any
purpose and it'll make everything that much harder to examine all at
once.
Have you ever examined sys/conf/files?
Meanwhile, I think trying to wipe out all the boolean dependency logic
in favor of a big graph of modules and submodules is also likely to
make a mess. What happens to e.g.
fileufs/ffs/ffs_bswap.c (ffs | mfs) ffs_ei
especially given that the ffs code is littered with FFS_EI conditional
compilation? You can make ffs_bswap its own module, but that doesn't
really serve any purpose. You could try making an FFS_EI module that
works by patching the ffs module on the fly or something, and then
include ffs_bswap.o into that, but that would be both very difficult
and highly gross. You could compile two copies each of ffs and mfs,
with and without FFS_EI support, but that wastes space. Or you could
make FFS_EI no longer optional, which would be a regression.
(FFS_EI isn't the only such option either, it's just one I happen to
have already banged heads with.)
Yes, we have to hunt all these dirtinesses.
One idea is to provide alternative modules. Modules that provide/require
identical API, but have different signature (-DXXX, #ifdef).
Masao
--
Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635
Re: config(5) break down
On Mon Mar 08 2010 at 07:09:07 +, David Holland wrote: Meanwhile, I think trying to wipe out all the boolean dependency logic in favor of a big graph of modules and submodules is also likely to make a mess. What happens to e.g. fileufs/ffs/ffs_bswap.c (ffs | mfs) ffs_ei especially given that the ffs code is littered with FFS_EI conditional compilation? You can make ffs_bswap its own module, but that doesn't really serve any purpose. You could try making an FFS_EI module that works by patching the ffs module on the fly or something, and then include ffs_bswap.o into that, but that would be both very difficult and highly gross. You could compile two copies each of ffs and mfs, with and without FFS_EI support, but that wastes space. Or you could make FFS_EI no longer optional, which would be a regression. (FFS_EI isn't the only such option either, it's just one I happen to have already banged heads with.) This one is easy, no need to make it difficult. The NetBSD-supplied module is always compiled with FFS_EI (if you don't like it, you can always compile your own just like you can compile your own kernel now). We don't care about mfs here, since it's not reasonable to want to mount a memory file system in the opposite byte order (technically I guess you could mmap an image instead of malloc+newfs and then mount(MOUNT_MFS), but you might just as well use ffs). Things like wapbl are currently an actual problem, since it is multiply owned (conf/files *and* ufs/files.ufs). The easy solution (and my vote) would be to make vfs_wapbl.c always included in the base kernel. If someone feels it's worth their salt to make it into two modules with all the dependency hum-haa, that would be a good place to start practicing instead of ffs_ei.
Re: config(5) break down
On Mon, Mar 08, 2010 at 10:53:16AM +0200, Antti Kantee wrote: (FFS_EI isn't the only such option either, it's just one I happen to have already banged heads with.) This one is easy, no need to make it difficult. Sure, but as I said it was just an example; what about the next one? Things like wapbl are currently an actual problem, since it is multiply owned (conf/files *and* ufs/files.ufs). I don't see why this is a problem either. The way things are right now, vfs_wapbl.c is conditional on wapbl the same as the rest; enforcing a hierarchical decomposition by source directory would break this but that's part of why such hierarchical decompositions are a bad idea. (I've also never fully understood why wapbl has to have so many tentacles hanging out of ffs, either.) -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
define foo: bar : define bar: baz : I have no idea if config currently allows this It doesn't allow, BTW /src/netbsd/src.TNF/sys/conf/files:289: undefined attribute `scsi_core' Masao
Re: config(5) break down
On Fri, 5 Mar 2010, Masao Uebayashi wrote:
I've found that the difficulty of understanding config(5) is due to its
flexibility; it can do one thing in many ways. You can define a collection
of sources with define, defflag, device, defpseudo{,dev}, devfs. OTOH you
can only write dependency on attributes (define). Another example is, you
can write interface with define, device, defpseudodev.
I'd propose to make a rule to simplify things (at the cost of a little
redundancy of config(5) files).
Allright. I have to ask:
If the plan is to go to a dynamically probed system with loadable modules,
why keep config around at all? It's only useful for custom kernels. Why
is it useful to give config a facelift instead of doing away with it
entirely?
Eduardo
Re: config(5) break down
On Mon, Mar 08, 2010 at 04:43:17PM +, Eduardo Horvath wrote:
On Fri, 5 Mar 2010, Masao Uebayashi wrote:
I've found that the difficulty of understanding config(5) is due to its
flexibility; it can do one thing in many ways. You can define a collection
of sources with define, defflag, device, defpseudo{,dev}, devfs. OTOH you
can only write dependency on attributes (define). Another example is, you
can write interface with define, device, defpseudodev.
I'd propose to make a rule to simplify things (at the cost of a little
redundancy of config(5) files).
Allright. I have to ask:
If the plan is to go to a dynamically probed system with loadable modules,
why keep config around at all? It's only useful for custom kernels. Why
is it useful to give config a facelift instead of doing away with it
entirely?
config(5) files carry information on what source file to build, under
what conditions as well as information about drivers and relationships
between devices. Why would you do away with it entirely? All that
information will have to be stored somewhere anyway.
--
Quentin Garnier - c...@cubidou.net - c...@netbsd.org
See the look on my face from staying too long in one place
[...] every time the morning breaks I know I'm closer to falling
KT Tunstall, Saving My Face, Drastic Fantastic, 2007.
pgph9qv88iLea.pgp
Description: PGP signature
Re: config(5) break down
On Mon, Mar 08, 2010 at 04:43:17PM +, Eduardo Horvath wrote: If the plan is to go to a dynamically probed system with loadable modules, why keep config around at all? It's only useful for custom kernels. Why is it useful to give config a facelift instead of doing away with it entirely? You can use config to generate the glue code that modules needs. eric
Re: config(5) break down
On Mon, 8 Mar 2010, Quentin Garnier wrote: On Mon, Mar 08, 2010 at 04:43:17PM +, Eduardo Horvath wrote: Allright. I have to ask: If the plan is to go to a dynamically probed system with loadable modules, why keep config around at all? It's only useful for custom kernels. Why is it useful to give config a facelift instead of doing away with it entirely? config(5) files carry information on what source file to build, under what conditions as well as information about drivers and relationships between devices. Why would you do away with it entirely? All that information will have to be stored somewhere anyway. Yes. It also specifies what drivers attach where. If you have a modular kernel and want to add a new driver, you could compile it and stick it into the module directory tree, but if the config file didn't have that module in it when the kernel was originally built, your driver can never be loaded and your new device is useless. config was a solution to the problem of needing to edit headers and source files to add a new device to the system. In a modular world I think it's a bit dated and there will be a need to figure all this stuff out dynamically. Eduardo
Re: config(5) break down
On Fri, Mar 05, 2010 at 01:14:50AM +0900, Masao Uebayashi wrote: Perhaps a first step would be using config(1) and files.* to generate the module makefiles instead of maintaining them by hand... cube@ said he did this part long time ago. The thing is that only fixing these tools don't solve all problems magically. We have to fix wrong instances around the tree. Maybe it should be merged then? Broken config(5) files will be named like module.conf, because files.* namespace is insufficient. For example pci.kmod can't use files.pci. Huh? I don't understand. Let's see the real examples. sys/conf/files has this: filenet/zlib.c (ppp ppp_deflate) | ipsec | opencrypto | vnd_compression This means [...] We should normalize this as [...] Now we define a module ppp_deflate which depends on ppp and zlib. To make dependency really work, the depended modules must be already defined. To make sure, we have to split files into pieces and include dependencies. net/zlib.conf See, this is the part that I don't understand. You're talking about normalizing logic, which is fine, and making shared files first-class entities, which is fine too though could get messy. But then suddenly you jump into splitting up files.* into lots of little tiny files and I don't see why or how that's connected to what you're trying to do. -- David A. Holland dholl...@netbsd.org
Re: config(5) break down
config(5) has 4 ways to define a collection of sources:
initialization interface dependency
configurable
device autoconf(9) bdev/cdev y y
defpseudodevxxxattach bdev/cdev n y
defpseudo xxxattach - n y
define - - y n
I think device part is fine. Device drivers are configured by autoconf(9),
and defined as device. Used with interface attributes (define xxx {}),
most of problems can be resolved. (Some complex ones like scsipi / wscons
are their problems, not config(5).)
In sys/net* and some sys/kern there're some code that can be modular. Most
of them are defined as either defpseudo or define. If it's user configurable,
it should be defpseudo. If it's simply a shared code, define.
There're lots of confusions in sys/conf/files; some are defflag or defparam.
Anything which _owns_ *.c, AND user-configurable should be defpseudo. We'll
have defpseudo inet or defpseudo ipsec eventually.
defpseudo lacks some points:
- No dependency.
- No detach.
I think defpseudo's initialization and module's one can be merged. Dependency
is also addressed naturally if defpseudo is always handled as modules.
config(1)
is taught about defpseudo's dependency. If config fragment like:
defpseudo inet
:
defpseudo gif
file if_gif.c gif
is passed, config(1) generates *.c and *.mk to build if_gif.kmod, which
depends on inet.kmod.
Masao
--
Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635
Re: config(5) break down
defpseudo inet : defpseudo gif file if_gif.c gif Should have been read as: defpseudo inet : defpseudo gif: inet --- file if_gif.c gif
Re: config(5) break down
On Wed, Mar 03, 2010 at 03:26:20AM +0900, Masao Uebayashi wrote: I want to slowly start breaking down config(5) files (sys/conf/files, sys/**/files.*) into small pieces. The goal is to clarify ownership of files; lines like file aaa.c bbb | ccc are to be changed into file aaa.c (ownership) and ccc: bbb (dependency). I'm not entirely convinced that makes sense in all cases, but I suppose it probably does in most. Because in the modular world one file belongs to one module. Perhaps a first step would be using config(1) and files.* to generate the module makefiles instead of maintaining them by hand... Broken config(5) files will be named like module.conf, because files.* namespace is insufficient. For example pci.kmod can't use files.pci. Huh? I don't understand. -- David A. Holland dholl...@netbsd.org
