Re: Next Generation kernel configuration?
On Wed, Jul 21, 2004 at 02:52:07PM +0200, Devon H. O'Dell wrote: I wholly disagree. I think using an excuse ``people will let everything else do the work for them and nobody will ever learn'' to discourage the development of automation tools is very poor. Try applying that argument to any utility that you use. You'd have to write your own bloody operating system because ``learning's in your best interest''. I'm sure this will become another bikeshed, so I suggest whoever came up with the idea to put up or shut up. People are interested in solutions, not suggestions. You confuse automation, with simplification. Automation tools are good for frequently re-run tasks. How often do you recompile your kernel? exactly. -- Avleen Vig Systems Administrator Personal: www.silverwraith.com EFnet:irc.mindspring.com (Earthlink user access only) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: disk recovery help
On Tue, 2004-Jul-20 14:01:06 -0400, Charles Sprickman wrote: On Tue, 20 Jul 2004, Peter Jeremy wrote: It's difficult to see how a sanely written RAID utility could totally screw up an array in a short time Upon reflection, one obvious way is to change the array layout. I don't know enough about your configuration and Adaptec's raidutil to know if this is likely. command does, but they are fairly certain that it writes it's config at the end of the disk, then zeros it from the outside in. Which puts an upper limit on the amount of damage done. The only difficulty with this is that (ISTR) your filesystem begins at the beginning of the array so the primary superblock should be the first thing over-written - and fsck would whinge loudly about that. grabbed the dd image before that. An fsck on the problem partition ran for 12 hours and I don't know how far along it was. Ctrl-T (aka SIGINFO) is your friend - fsck will tell you how far through its current phase it is. I looked at scan_ffs just now, and it looks like it works on the whole disk trying to find the label. Since I only have one partition, there's no label. scan_ffs searches the disk (or file) looking for UFS superblocks. The most common reason for needing this is to re-generate your partition tables. I was hoping it would also locate all the superblocks - which would let you verify that the structure looked reasonably sane. You might also try fsdb(8) - though I think it relies on the primary superblock being sane. Good luck. -- Peter Jeremy ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
On 21-Jul-2004 M. Warner Losh wrote: In message: [EMAIL PROTECTED] Murray Taylor [EMAIL PROTECTED] writes: : As an initial starting point for 'preloading' any menubased kernel : configurator, could the file /var/run/dmesg.boot be usefully parsed : as : a list of 'this is what is actually installed in this box, what else : do : you want to add? Of course any output developed on a run of the : configurator would/could/should be scanned as well to include : answers to : the question..What did I include last time? if devd could map, somehow, the pnp info into drivers to load, that would solve this problem. warner Interesting ideas. Saving all this stuff in my suggestion box. :-) -- Conrad J. Sabatier [EMAIL PROTECTED] -- In Unix veritas ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
On 21-Jul-2004 Devon H. O'Dell wrote: I'm sure this will become another bikeshed, so I suggest whoever came up with the idea to put up or shut up. People are interested in solutions, not suggestions. Agreed. And the original proponent of the idea was me. I just wanted to see if there was any willingness to even consider something like this before I went and did a lot of work for nothing. Seems the general concensus is that most people are OK with the idea, depending on the implementation. I'll be quiet now until/unless I can actually come up with something. :-) -- Conrad J. Sabatier [EMAIL PROTECTED] -- In Unix veritas ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Getting a fully-qualified path from a PID
On Wed, 2004-07-21 at 11:12, Dan Nelson wrote: In the last episode (Jul 20), Joe Marcus Clarke said: What is the canonical way for a userland application to get the fully-qualified path of an executable from its running PID? I know I can do a readlink(2) on /proc/pid/file, but procfs is deprecated on 5.X, correct? Is there a more appropriate way to do this? Thanks. realpath(argv[0]) works for commands not run from $PATH. Commands found through a PATH earch will just have the basename in argv[0] so you would have to check each PATH element until you found it. Note that /proc/pid/file won't work if vn_fullpath() fails (say the orignal file has been unlinked, or the filename has expired from the kernel's cache). If you are examining another process, you can use the kvm_getargv() and kvm_getenvv() functions to fetch argv[0] and PATH out of the target process. Okay, I was thinking about that. What I was specifically interested in was processes spawned from $PATH, so realpath isn't going to be much good to me there. I didn't know if there was a better way than getting the environ+argv with kvm, then searching each path element. Thanks for the clarification. Joe -- PGP Key : http://www.marcuscom.com/pgp.asc signature.asc Description: This is a digitally signed message part
Re: disk recovery help
Hi Charles, I sent this message. But SpamCop.net had listed my ISP address. I received undelivered message. So I will send again. Eitarou Eitarou Kamo wrote: Hi, How about this? 1. You mount dd.img as vn0 like this in your free space. #vnconfig vn0 dd.img # mount /dev/vn0c /mnt 2. archive or dump whole data of the /mnt. 3. restore archived data to /dev/ad3s1h after create file system. FSCK failed because you set the bs=1024 in the dd command, I guess. Charles Sprickman wrote: On Tue, 20 Jul 2004, Eitarou Kamo wrote: -- *** Eitarou Kamo Tel. +81 75 7035997 Fax +81 75 7035997 VoIP 050 10585997(domestic only) e-mail [EMAIL PROTECTED] For business: Feel free to mail me(above), please. Donation http://www.PayPal.Com GPG FingerPrint: 032D FDF9 D27B 23F7 9A81 BF4C 626C FBAA BC3A 9895 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
You confuse automation, with simplification. Automation tools are good for frequently re-run tasks. How often do you recompile your kernel? exactly. In my opinion, the least we could do is have it as a port/package. For the make check dependencies, that could be a great idea to commit because right now, unless your kernel doesn't compile our your computer doesn't start, there's no way to know you forgot something. The debate here is automation vs. simplification. Why we shouldn't simplificate the kernel compile ? Because our user base's average IQ will be lower ? We're not suggesting to have KDE installed by default here. Users would still to have to type the commands to compile the kernel by hand and do a little research about the options they enabled. XFree86 has ncurses/graphic configuration utilities. Why the kernel shouldn't ? -- Nicolas Bérard Nault ([EMAIL PROTECTED]) http://staff.xeatech.net/nicobn PGP public key: 0x64159509 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
Mmmm... I had something similar (much smaller) supplied with my Cromemco Z-80 CROMIX system... early 80's for the historical. You were prompted through a series of questions re the desired requirements for i/o devices etc. and then the last step would rebuild the 'kernel' to that configuration. It wasn't actually doing a full build as all the supplied stuff came out of linkable libraries, BUT one could use an assembler 'template' to create ones own device drivers for the homebrewed 'MummbleFrotz' device. These 'user-created drivers' were then assembled, loaded into your own library and became a linkable element from then onwards that you could add during any subsequent 'kernel rebuilds' If I remember correctly- the main elements of the template were - an entry 'jump table' for such things as Initialise (ENTRY+0), Open (ENTRY+2), Read (ENTRY+4) - a place to define major:minor numbers and the rules / options pertaining thereto, - and finally how to return data and driver status. Fun. mjt On Thu, 2004-07-22 at 06:41, Bruce R. Montague wrote: Hi, re, Next Generation kernel configuration: Years ago I had a job for a few years where I constantly built RSX-11 systems on PDP-11s. RSX-11 was always built from source and had a couple of hundred build-time options, many hardware build dependencies, and supported numerous other dynamic build-time functions; it was said that it was possible that no 2 RSX systems were really the same binaries. It may have been more combinatorially complex than FreeBSD. An RSX build could easily take a day. Although at the core the build was driven by a file of assembly macro defines, conceptually not unlike FreeBSD, the build process went through continuous evolution over the life of RSX. A comprehensive sysgen.bat script, or somesuch, evolved. Sysgen (which was a common industry term at the time) was a very large script that was intended to be run on a fast hard-copy decwriter; it printed out lists of possibilities and then asked you a question, you made a selection from the options, and so on. It conducted a 'scripted dialog' that reflected the options you made along the way. You wanted this on hard copy so you could go back and check things, keep it for next time, and so on. You could go back in the dialog and repeat a section, save the sysgen state and restart later, and so on. A sysgen dialog could easily take half-a-day (sometimes intermediate things had to be built and such), and then the build itself and install could take a number of hours... At the end of the sysgen dialog you could save the session, basically, and then the next time you did a session you could ask to use the saved session and essentially conduct a modification dialog. Working with sysgen often felt like taking part in an adventure game with an AI opponent; you had to know how to outsmart the script to get it to do exactly what you wanted. This might be a common failing of many pseudo AI type programs. On the one had this all worked and worked well. On the other hand if you can do it by simply editing flat files it's much better, because you don't have to become an expert on the sysgen script just to do a build. Back on the other hand, there may be a point of complexity (and lack of corresponding widespread sophistication) where a sysgen program is necessary. SO: If a sysgen-like program was built for FreeBSD that used a conversational, graphic, menu, or whatever interface, instead of actually doing anything to real files or the real system, could it just print out _what to do_, that is, it would output a list of instructions - in such-and-such a file, edit this option, then add this line... Or perhaps it would output diffs to files... or put the output in a candidate location. But in any case the program would be a SYSGEN ASSISTANT, not an actual sysgen program. Basically a kernel config checker, a smart build-lint, etc.. It could live in ports. If this program got to where it really worked, everyone liked the interface, and the system complexity was clearly at the point where it was needed, it could be used to directly generate system configurations. The dependency-rule evaluation and output part could be built independent of any user-interface, so a front-end back-end scheme might make sense. In any case, googling on RSX sysgen might produce some ideas of interest. BTW, I'm under the impression that for quite some time the largest rule-based AI application (real-time expert system) in the world was the OPS5 system implemented at DEC to configure VAX hardware, see links such as: http://encyclopedia.thefreedictionary.com/OPS5%20rule%20based It looks like there's a public domain system that compiles rules to C code, perhaps there are some interesting ideas there as well for things like general dependency rule evaluation in the backend and such:
Re: Next Generation kernel configuration?
The debate here is automation vs. simplification. Why we shouldn't simplificate the kernel compile ? Because our user base's average IQ will be lower ? We're not suggesting to have KDE installed by default here. Users would still to have to type the commands to compile the kernel by hand and do a little research about the options they enabled. XFree86 has ncurses/graphic configuration utilities. Why the kernel shouldn't ? Because configuring kernel by editing config files is, IMHO, the fastest and most convenient method. New FreeBSD users should get used to it from the beginning - this will save their time in future. There's nothing hard in editing the files, especially after few kernels for different machines have been created (they can be used as templates). Timestamp: 0x41005EBC [SorAlx] http://cydem.org.ua/ ridin' VN1500-B2 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
regarding timeout/untimeout kernel functions
HI all, i am working on a project , where i came across a situation where i need to execute a function when a timer expires ,exactly similar to functionality of the timeout() kernel function but i need this in userland(application), and the execution of the function is time sensitive, it should be run immediately when timer expires. i can't be using poll or select for timer becuse those will block the process untill the timer expires.for me the proess should not be blocked. and i also thought of taking the service of the timeout function by writing a system call and using signaling mechanism but i think this will become expensive when the number of timers to be checked increeses. i read the kern_timeout.c code that is very good implentation.with very less expensive. but i think user unable to enjoy that service. i will thankful if somebody can tell if there is any such a service or way provided by os( that i overlooked). thanks, -Pradeep - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: regarding timeout/untimeout kernel functions
If you're willing to take some precautions, you could run the timer code with select/usleep in a separate thread. However, since the callbacks would originate from that thread, you would need mutexes to protect any data that the function accesses that could also be accessed by the normal program flow. Joe pradeep reddy punnam wrote: HI all, i am working on a project , where i came across a situation where i need to execute a function when a timer expires ,exactly similar to functionality of the timeout() kernel function but i need this in userland(application), and the execution of the function is time sensitive, it should be run immediately when timer expires. i can't be using poll or select for timer becuse those will block the process untill the timer expires.for me the proess should not be blocked. and i also thought of taking the service of the timeout function by writing a system call and using signaling mechanism but i think this will become expensive when the number of timers to be checked increeses. i read the kern_timeout.c code that is very good implentation.with very less expensive. but i think user unable to enjoy that service. i will thankful if somebody can tell if there is any such a service or way provided by os( that i overlooked). thanks, -Pradeep - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
On Tue, 20 Jul 2004 19:39:31 -0500 (CDT) Conrad J. Sabatier [EMAIL PROTECTED] wrote: Just musing on an idea here: I've been thinking for a while now about trying to write a tool to make kernel configuration easier, sort of a make config (as in ports) for the kernel, similar to what's available on some of the Linux distros. Ideally, such a tool would be capable of automatically adapting itself to handle and present as choices, in an orderly and logical fashion, whatever devices, options, etc. were currently available, as defined by the files in /sys/conf et al. Hi, I gave this a brief shot on DragonFly with not much luck, but maybe some insight - here's my experience. The kernel config file should probably be replaced by a Makefile. This way, the user can say something like: MYKERNEL: device_i_want another_device_i_want ... ... And of course somewhere else (probably in an included Makefile) the dependencies would be specified a la device_i_want: another_device_you_also_need_for_it ... So, you'd never get a doomed kernel config that starts compiling but chokes halfway through the build, because any needed devices would be brought in automatically. That's the easy part. The hard part is discovering the dependencies. If you want to discover them automatically by looking through the kernel source code files - all I can say is, good luck! You'll either need a really, really smart relation-mining program, or more disciplined source code organization, or both. Alternatively, you could ascertain a set of dependencies manually (many of them are noted in GENERIC, LINT, NOTES, etc,) but then you'd also have to maintain that set when they change in the future. I'm not so sure that's much of a drawback (since currently src/sys/conf/files has to undergo that sort of maintenance anyway,) but it's less big of a win than having it all nicely, automatically generated from the inherent structure of the kernel. The major hurdle to overcome, it appears to me, is that the scheme currently employed to describe the available devices, options, etc. does not lend itself very easily at all to any kind of automatic parsing or other manipulations. Determining dependencies between components programmatically, for one thing, seems well near impossible. Precisely the conclusion I came to as well. -Chris ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: regarding timeout/untimeout kernel functions
HI joseph , i thought of threading with select before , but i belive that if the number of timers to be checked increases the number of the threads to be maintained increses,so the process may become very hevy. what do u think. i think ultimatley i am going to use the above thing. but in the process of my search i came across the timeout kernel function implemenation but i can not use that ( which i belive very efficient implementation of timers ), which user can not able to use it , so i just want to discuss it . thanks - pradeep Joseph M Link [EMAIL PROTECTED] wrote: If you're willing to take some precautions, you could run the timer code with select/usleep in a separate thread. However, since the callbacks would originate from that thread, you would need mutexes to protect any data that the function accesses that could also be accessed by the normal program flow. Joe pradeep reddy punnam wrote: HI all, i am working on a project , where i came across a situation where i need to execute a function when a timer expires ,exactly similar to functionality of the timeout() kernel function but i need this in userland(application), and the execution of the function is time sensitive, it should be run immediately when timer expires. i can't be using poll or select for timer becuse those will block the process untill the timer expires.for me the proess should not be blocked. and i also thought of taking the service of the timeout function by writing a system call and using signaling mechanism but i think this will become expensive when the number of timers to be checked increeses. i read the kern_timeout.c code that is very good implentation.with very less expensive. but i think user unable to enjoy that service. i will thankful if somebody can tell if there is any such a service or way provided by os( that i overlooked). thanks, -Pradeep - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: regarding timeout/untimeout kernel functions
In the last episode (Jul 22), pradeep reddy punnam said: i thought of threading with select before , but i belive that if the number of timers to be checked increases the number of the threads to be maintained increses,so the process may become very hevy. what do u think. Threads are very lightweight. You should be able to create hundreds of (mostly-sleeping) threads with no problem. You wouldn't even need to use select; just sleep (or nanosleep). i think ultimatley i am going to use the above thing. but in the process of my search i came across the timeout kernel function implemenation but i can not use that ( which i belive very efficient implementation of timers ), which user can not able to use it , so i just want to discuss it . You could also use the kqueue/kevent functions to queue up an arbitrary number of timer events in a single process. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Next Generation kernel configuration?
Hi, re rule-based configuration Chris Pressey noted: That's the easy part. The hard part is discovering the dependencies. My impression is that almost all rule-based expert systems of sufficient complexity that deal with a dynamic field have failed because of this, that is, due to the difficulty of determining current dependencies (rule discovery). Even the experts don't actually know; each will know some but nobody will know all, certainly not when the real dependencies are evolving all the time. Even worse, there may be many combinations of things that just don't work but nobody realizes it yet, new things that break a lot of old dependencies in an unknown way, etc. Even the experts will hit this and ask on an email list, I did this and look what happened, anybody got any ideas? So the experts will know how to solve the problem, but not in a way that can be automated. Unix has been pretty good over its life at resisting combinatorial complexity; RSX for instance had a relatively high degree of optional API sets and optional API features and similar things with kernel primitives, this introduced a very fine level of granularity that made for a bad dependency combinatorial explosion (part of this resulted from the old OS/360 mantra of one system that would scale across a very wide family, combined with paranoia about memory use). Feature sets selected for server components depended on other feature sets, kernel feature sets, API feature sets, driver features sets, etc and vice versa. My impression -dont know if it's true- is that the RSX experience made DEC say never again. One important reason was testing. Testing a system when few others would actually be built exactly like it raises issues... its good to know that it at least works... but how fragile is it wrt other build combinations? The large e-mail list as build expert-system of choice combined with a simple mechanism (flat files) to act as control knobs is likely a big advantage open source systems have over most proprietary systems. It would be interesting to know how many people world-wide are reasonably competent to build FreeBSD from source compared to how many actually know the same for NT. Maybe all the more reason to package something as an Assistant type educational, verification, or visualization tool for stable, well-known core dependencies. FreeBSD will be around for a long time and such a tool, if nothing else, might help get people on board w/o any impact wrt the current state of affairs. If nothing else, it's an interesting problem and systems complexity is not likely to go down anytime soon! - bruce ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: regarding timeout/untimeout kernel functions
On Thu, Jul 22, 2004 at 09:56:00PM -0500, Dan Nelson wrote: You could also use the kqueue/kevent functions to queue up an arbitrary number of timer events in a single process. I wrote a small routing daemon which uses kqueue/kevent to fire a period timer on a quantum which in turn calls into a timer list module I wrote, which can either use the kevent quantum, or multiplex on a single itimer. It's pretty basic and probably not foolproof, but it seems to work well. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: regarding timeout/untimeout kernel functions
I dont know anything about the kevent timer stuff, so you should probably look into that before re-inventing the wheel. However, you would only need 1 thread to implement this timer functionality. There are a couple data structures you could use, but it would probably be easiest to use a priorityq. You insert each timer event into the priorityq keyed on when it is due to fire. You look at first item in the queue and subtract the current time from it (gettimeofday()) to determine how long to sleep until the next one is ready. If use select, you'll need to create a pipe and select on it to wake up the thread when you insert new timer events. Joe pradeep reddy punnam wrote: HI joseph , i thought of threading with select before , but i belive that if the number of timers to be checked increases the number of the threads to be maintained increses,so the process may become very hevy. what do u think. i think ultimatley i am going to use the above thing. but in the process of my search i came across the timeout kernel function implemenation but i can not use that ( which i belive very efficient implementation of timers ), which user can not able to use it , so i just want to discuss it . thanks - pradeep Joseph M Link [EMAIL PROTECTED] wrote: If you're willing to take some precautions, you could run the timer code with select/usleep in a separate thread. However, since the callbacks would originate from that thread, you would need mutexes to protect any data that the function accesses that could also be accessed by the normal program flow. Joe pradeep reddy punnam wrote: HI all, i am working on a project , where i came across a situation where i need to execute a function when a timer expires ,exactly similar to functionality of the timeout() kernel function but i need this in userland(application), and the execution of the function is time sensitive, it should be run immediately when timer expires. i can't be using poll or select for timer becuse those will block the process untill the timer expires.for me the proess should not be blocked. and i also thought of taking the service of the timeout function by writing a system call and using signaling mechanism but i think this will become expensive when the number of timers to be checked increeses. i read the kern_timeout.c code that is very good implentation.with very less expensive. but i think user unable to enjoy that service. i will thankful if somebody can tell if there is any such a service or way provided by os( that i overlooked). thanks, -Pradeep - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] - Do you Yahoo!? Vote for the stars of Yahoo!'s next ad campaign! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
