Re: [Testers wanted] /dev/console cleanups
On Thu, Nov 20, 2008 at 8:31 AM, Freddie Cash [EMAIL PROTECTED] wrote: On November 20, 2008 02:42 am Nate Eldredge wrote: On Thu, 20 Nov 2008, Jeremy Chadwick wrote: On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: On Wed, 19 Nov 2008, Jeremy Chadwick wrote: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use dmesg to find out what the status of the system is. By status I don't mean from the point the kernel finished to now, I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. I forgot about that point. I can sympathize with those users; I feel the same way. It's a good way to learn about a system as a mere user (since usually sysadmins don't remember or bother to disable it). However, in my experience dmesg isn't really the best thing for that purpose; the kernel message buffer tends to get wiped out once the system has been up for a while. (It fills with ipfw logs, ethernet link state changes, etc.) Maybe a better approach would be to point them to /var/log/messages or whichever log file stores them permanently. I think what you are looking for is /var/run/dmesg.boot, which stores just the dmesg info from the initial boot. Nothing gets logged to this after the boot is complete. This file has been a life saver quite a few times since I discovered it, and is something I really miss when working with mis-behaving Linux systems. -- Freddie Cash [EMAIL PROTECTED] [EMAIL PROTECTED] ~]$ grep -rs dmesg.boot /usr/src/ /usr/src/contrib/ntp/scripts/freq_adj.in: open(DM, /var/run/dmesg.boot); /usr/src/etc/defaults/rc.conf:dmesg_enable=YES# Save dmesg(8) to /var/run/dmesg.boot /usr/src/etc/rc.d/dmesg:dmesg_file=/var/run/dmesg.boot /usr/src/sbin/dmesg/dmesg.8:.Bl -tag -width .Pa /var/run/dmesg.boot -compact /usr/src/sbin/dmesg/dmesg.8:.It Pa /var/run/dmesg.boot [EMAIL PROTECTED] ~]$ /etc/rc.d/dmesg does this. -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
Jeremy Chadwick [EMAIL PROTECTED] writes: Peter Jeremy [EMAIL PROTECTED] writes: This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. That doesn't sound deliberate at all -- it sounds like a quirk that people (you?) are relying on. No, it is deliberate. Just because you don't like it doesn't mean we're morons. You're under the mentality that the information is *always* available after a panic/reboot -- it isn't. I have 4 different Supermicro motherboards (all from different years) which will most of the time lose the msgbuf after rebooting from single-user -- but sometimes the msgbuf is retained. And no, bad hardware is not responsible for the randomness of the problem. We know there are systems where it doesn't work. That doesn't mean it's not useful when it *does* work. There are also systems where it works reliably, including most SoC and SFF systems. I think it's been discussed in the past how/why this can happen. It has to do with what each BIOS manufacturer chooses to do with some parts of memory during start-up. Mostly whether memory remains powered-on through the reset process. Speaking only of i386 / amd64 systems, there are several ways to reboot a computer: - Return to real mode and call the BIOS reboot subroutine. This will perform a cold boot (memory is cleared) or a warm boot (memory should not be cleared, but the BIOS may do so anyway) depending on the value stored at a specific location in memory. - On systems with an i8042 keyboard controller, one of the controller's output lines is usually wired to the CPU's reset pin. Writing a specific value to one of the keyboard controller's control registers causes it to pull that pin low. - Triple fault: set up an empty interrupt descriptor table and trigger an interrupt (e.g. divide by zero). The CPU will fail to invoke the corresponding interrupt descriptor, causing a double fault interrupt, which will fail to execute, causing a CPU reset. The latter two techniques can be used (and *have* been used) to switch back from protected mode to real mode, because they are faster than the documented method. Memory is *not* cleared, and the CPU will start executing whatever code is stored at 0xfff0 in real memory; this is normally a far jump into the BIOS reboot subroutine, so if you don't pull any tricks (e.g. jump to your own code that does whatever it was you needed to do in real mode, then return to protected mode), they are equivalent to the first technique. - ACPI reset: store the value specified by RESET_VALUE in the FADT into the register specified by RESET_REG in the FADT. According to the ACPI spec, this results in a cold boot. However, this part of the spec is optional, so not all motherboards support it. FreeBSD will only use ACPI reset if the hw.acpi.handle_reboot sysctl is not zero, which it is by default (mostly, if I recall correctly, because ACPI reset is unreliable). DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: On Wed, 19 Nov 2008, Jeremy Chadwick wrote: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use dmesg to find out what the status of the system is. By status I don't mean from the point the kernel finished to now, I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. 2) I don't understand how this would work (meaning, technically and literally: I do not understand). How do messages like CPU: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (2992.52-MHz K8-class CPU) get written to syslog when syslogd isn't even running (or any filesystems) mounted at that time? There must be some magic involved there (since syslog == libc, not syscall) when syslogd starts, but I don't know how it works. This way the buffer is cleared before any unprivileged users get to do anything. No kernel changes needed, just a little tweaking of the init scripts at most. If you should have a crash and suspect there is useful data in the buffer, you can boot to single-user mode (avoiding the clear) and retrieve it manually. Seems like this should make everyone happy. What I'm not understanding is the resistance towards Rink's patch, assuming the tunable defaults to disabled/off. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Thu, Nov 20, 2008 at 10:53:07AM +0100, Dag-Erling Smørgrav wrote: Jeremy Chadwick [EMAIL PROTECTED] writes: Peter Jeremy [EMAIL PROTECTED] writes: This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. That doesn't sound deliberate at all -- it sounds like a quirk that people (you?) are relying on. No, it is deliberate. Just because you don't like it doesn't mean we're morons. I said nothing about liking/disliking it, nor did I namecall or condescend. Thanks for being a complete prick, des. Jesus christ... -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Thu, 20 Nov 2008, Jeremy Chadwick wrote: On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: On Wed, 19 Nov 2008, Jeremy Chadwick wrote: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use dmesg to find out what the status of the system is. By status I don't mean from the point the kernel finished to now, I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. I forgot about that point. I can sympathize with those users; I feel the same way. It's a good way to learn about a system as a mere user (since usually sysadmins don't remember or bother to disable it). However, in my experience dmesg isn't really the best thing for that purpose; the kernel message buffer tends to get wiped out once the system has been up for a while. (It fills with ipfw logs, ethernet link state changes, etc.) Maybe a better approach would be to point them to /var/log/messages or whichever log file stores them permanently. Or, better yet, do some syslogd magic to make a logfile that can be appropriately readable but doesn't have any overly sensitive messages directed there (e.g. kernel yes, sshd no). 2) I don't understand how this would work (meaning, technically and literally: I do not understand). How do messages like CPU: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (2992.52-MHz K8-class CPU) get written to syslog when syslogd isn't even running (or any filesystems) mounted at that time? There must be some magic involved there (since syslog == libc, not syscall) when syslogd starts, but I don't know how it works. I think you're conflating a couple of things, and I also explained my idea poorly. As I understand it (from memory, which is a little vague), syslogd gets messages from two places: from the kernel via /dev/klog, and from other processes via a Unix domain socket in /var/run. These messages then get sent to the appropriate log files. The syslog(3) function of libc just connects and writes the message to the Unix domain socket. If syslogd isn't running to listen on that socket, syslog(3) won't work very well. Now /dev/klog should be a direct line to the kernel's message buffer. So when syslogd starts and reads /dev/klog for the first time, it will get everything that's accumulated so far, including the earliest boot messages. It should then write them to the appropriate log files. This already works, which is why /var/log/messages contains the kernel copyright message, etc. My idea is, after syslogd does this, but before the system goes multi-user, you should clear the kernel buffer. Early boot messages are already in the log files, so they won't be lost. Maybe the best thing would be to build this functionality into syslogd itself, to minimize the possibility of losing messages due to a race. This way the buffer is cleared before any unprivileged users get to do anything. No kernel changes needed, just a little tweaking of the init scripts at most. If you should have a crash and suspect there is useful data in the buffer, you can boot to single-user mode (avoiding the clear) and retrieve it manually. Seems like this should make everyone happy. What I'm not understanding is the resistance towards Rink's patch, assuming the tunable defaults to disabled/off. It seems reasonable to me. The only catch I can see is that if you have a crash and you want to look at the message buffer after rebooting, you'll have to remember to stop at the loader prompt and turn off that tunable. Which might be easy to forget in the heat of the moment. -- Nate Eldredge [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Nov 20, 2008, at 4:03 AM, Jeremy Chadwick wrote: This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use dmesg to find out what the status of the system is. By status I don't mean from the point the kernel finished to now, I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. Sorry for jumping in late, but... cat /var/run/dmesg.boot Is that acceptable? I know it depends on what end users want, but some of my old hosting customers really just wanted to see the specs of the box and nothing else. Making dmesg a shell script that just cats that file satisfied everyone who asked. Also: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick [EMAIL PROTECTED] wrote: There's a known issue with the kernel message buffer though: it's not NULL'd out upon reboot. This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. That doesn't sound deliberate at all -- it sounds like a quirk that people (you?) are relying on. I do not think any piece of the FreeBSD system (e.g. savecore, etc.) relies on this behaviour. You're under the mentality that the information is *always* available after a panic/reboot -- it isn't. I have 4 different Supermicro motherboards (all from different years) which will most of the time lose the msgbuf after rebooting from single-user -- but sometimes the msgbuf is retained. And no, bad hardware is not responsible for the randomness of the problem. I think it's been discussed in the past how/why this can happen. It has to do with what each BIOS manufacturer chooses to do with some parts of memory during start-up. I'm sure the Quick Boot (e.g. no extensive memory test, which really doesn't test anything these days) option plays a role, and that option is enabled by default on all motherboards I've used in the past 10 years. I've been involved with a few embedded systems, some BSD based some not. In a few cases we've used custom BIOSes on the motherboard. At least one BIOS SDK specifically describes this as a feature. What is exposed to the end user as Quick Boot is actually several options that the motherboard designer/BIOS configurer can select. One of which is specifying which chunks of memory should be preserved after a reboot, up to memory tests should be as non-destructive as possible. While you can't *rely* on it, with careful use of atomic writes and state checking you can pick up where you left off after a reboot on an embedded device that has no long-term storage if it was a warm boot. Or, gathering crash dumps and sending them off to the network. Here, the dmesg buffer is a simple ring buffer in the kernel. The start/end pointers and contents of the ring buffer are deliberately not cleared after a reboot in FreeBSD to at least make the information available if the BIOS didn't clobber it. This can be extremely useful in those a box on the other side of the world keeps rebooting with no panic, if you can tell the BIOS to skip the memory check. In one case I even (ab)used some extra video ram on the motherboard as a dmesg buffer since the video bios didn't wipe it on boot, but the system's bios insisted on doing a full memory test each time. If you have a BIOS that's sometimes but not always wiping the buffer, it's probably because a few bits are being lost while the motherboard turns the DRAM refresh off during the reboot. Some DIMMS can handle no refresh for tens of seconds, some start popping bit errors rather quickly. Take a look at sys/kern/subr_msgbuf.c:msgbuf_reinit. There's a magic number it looks for as well as a checksum on the whole buffer that's updated after every write. If anything changes, the kernel throws out the whole buffer and starts fresh. If you boot with -v you'll probably see msgbuf cksum mismatch. While I don't think any of the shipping FreeBSD tools rely on this behavior, I know I tried submitting a patch back in the FreeBSD-2.2(?) days to fix that overlooked uninitialized buffer and got educated by jkh pretty quickly. :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On November 20, 2008 02:42 am Nate Eldredge wrote: On Thu, 20 Nov 2008, Jeremy Chadwick wrote: On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: On Wed, 19 Nov 2008, Jeremy Chadwick wrote: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use dmesg to find out what the status of the system is. By status I don't mean from the point the kernel finished to now, I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. I forgot about that point. I can sympathize with those users; I feel the same way. It's a good way to learn about a system as a mere user (since usually sysadmins don't remember or bother to disable it). However, in my experience dmesg isn't really the best thing for that purpose; the kernel message buffer tends to get wiped out once the system has been up for a while. (It fills with ipfw logs, ethernet link state changes, etc.) Maybe a better approach would be to point them to /var/log/messages or whichever log file stores them permanently. I think what you are looking for is /var/run/dmesg.boot, which stores just the dmesg info from the initial boot. Nothing gets logged to this after the boot is complete. This file has been a life saver quite a few times since I discovered it, and is something I really miss when working with mis-behaving Linux systems. -- Freddie Cash [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Tue, Nov 18, 2008 at 7:34 PM, Ed Schouten [EMAIL PROTECTED] wrote: Hello Carlos, others, * Ed Schouten [EMAIL PROTECTED] wrote: About the /dev/console issues: Robert Watson and I discussed this some time ago on IRC and what I did in HEAD (not RELENG_7) was that I changed TIOCCONS not to take a look at the permissions of /dev/console, but we changed it to use priv_check(). This means that right now you can only call TIOCCONS as root. I can't really understand why the problem exists on RELENG_7. About making xconsole setuid: I've read the messages you mentioned, but I think we could just alter console to call TIOCCONS and just drop privileges. An even better solution would be to just get rid of TIOCCONS and invent a better solution to capture syslog messages. I can't really understand why we want to abuse TTY's to do this. So I can't say we're working on this, but at least I can confirm the issue. One solution would be to let xconsole just display /var/log/messages. There shouldn't be a valid reason to let syslogd print messages to /dev/console and capture them again using TIOCCONS. We could just instruct xconsole to read its data from the log files. If you save the attached patch as /usr/ports/x11/xconsole/files/ patch-xconsole.c (create directory first) and recompile xconsole, it will use the log file. I'll discuss this with others to decide if we should take such an approach. It is not necessary to patch xconsole to accomplish this. Using the -file command line argument would be enough. Be warned, however, that 1. messages sent straight to /dev/console will not show up at the xconsole window; 2. with large files it will become slow and consume lots of memory, because it will load the entire contents of /mar/log/messages to its text buffer; 3. it will show *all* messages, not only the urgent ones, which is not necessarily the desired behavior; 4. it will stop working upon log rotation. -- cd /usr/ports/sysutils/life make clean ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Wed, Nov 19, 2008 at 02:02:42AM -0800, Garrett Cooper wrote: On Tue, Nov 18, 2008 at 1:49 PM, David Wolfskill [EMAIL PROTECTED] wrote: On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: ... One solution would be to let xconsole just display /var/log/messages. Errr... it may be rather a pathological case, but you might want to check the content of /etc/syslog.conf on the local machine before getting too carried away with that approach. For example, on my firewall box at home (where I really do not want to log anything to local disk files, though I do have a serial console on it): janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf *.* @bunrab.catwhisker.org janus(6.4-P)[2] And then consider the fate of bunrab -- with stuff getting logged to /var/log/messages from various machines ... I'll discuss this with others to decide if we should take such an approach. I'm not trying to be obstructionist, here. If the above case is really too pathological to consider -- or if it's a case of me bringing that fate upon myself, I suppose -- that's actually something I can live with. It would be nice to be forwarned about it, though. :-} Peace, david Uh, I second that. /var/log/messages shouldn't necessarily be accessible by non-root users. Also, OSX 10.5 protects against non-root access to dmesg. Not saying we should go that far, but it's already being implemented, so I don't see any harm in hiding the contents of `messages', as required by the sysadmin. Footnote (not really applicable to the thread, but I want to point it out to users/admins reading): inhibiting users viewing the kernel message buffer (dmesg) can be accomplished by setting the security.bsd.unprivileged_read_msgbuf sysctl to 0. However, note that this can piss users off. We have numerous users on our system who rely on this information to see if anything weird is going on with the box. I set that sysctl one day (see below for why), and I got flames in my mailbox within 48 hours. Just something to keep in mind if you have technically-savvy users. There's a known issue with the kernel message buffer though: it's not NULL'd out upon reboot. Meaning, in some cases (depends on the BIOS or system), the kernel message buffer from single-user mode is retained even after a reboot! A user can then do dmesg and see all the nifty stuff you've done during single-user, which could include unencrypted passwords if mergemaster was tinkering with passwd/master.passwd, etc.. I've brought this up before, and people said Yeah, we know, moving on. Rink Springer created a patch where the kernel message buffer will start with NULL to keep this from happening, but it needs to be made into a loader.conf tunable. Also, /var/log/messages is explicitly set to 0644 in newsyslog.conf. If people want to debate that, be my guest. I'm not sure what security hole we'd be plugging if it was set to 0600, especially given that many userland programs use the LOG_NOTICE facility in syslog. If people want to debate those default perms, be my guest. I would rather people debate the default syslog.conf layout altogether; I'm surprised we haven't moved to syslog-ng (as part of the base system) by now. :-) -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Tue, Nov 18, 2008 at 1:49 PM, David Wolfskill [EMAIL PROTECTED] wrote: On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: ... One solution would be to let xconsole just display /var/log/messages. Errr... it may be rather a pathological case, but you might want to check the content of /etc/syslog.conf on the local machine before getting too carried away with that approach. For example, on my firewall box at home (where I really do not want to log anything to local disk files, though I do have a serial console on it): janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf *.* @bunrab.catwhisker.org janus(6.4-P)[2] And then consider the fate of bunrab -- with stuff getting logged to /var/log/messages from various machines ... I'll discuss this with others to decide if we should take such an approach. I'm not trying to be obstructionist, here. If the above case is really too pathological to consider -- or if it's a case of me bringing that fate upon myself, I suppose -- that's actually something I can live with. It would be nice to be forwarned about it, though. :-} Peace, david Uh, I second that. /var/log/messages shouldn't necessarily be accessible by non-root users. Also, OSX 10.5 protects against non-root access to dmesg. Not saying we should go that far, but it's already being implemented, so I don't see any harm in hiding the contents of `messages', as required by the sysadmin. -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick [EMAIL PROTECTED] wrote: There's a known issue with the kernel message buffer though: it's not NULL'd out upon reboot. This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. Meaning, in some cases (depends on the BIOS or system), the kernel message buffer from single-user mode is retained even after a reboot! A user can then do dmesg and see all the nifty stuff you've done during single-user, which could include unencrypted passwords if mergemaster was tinkering with passwd/master.passwd, etc.. There shouldn't be unencrypted passwords, though there might be encrypted passwords visible. Rink Springer created a patch where the kernel message buffer will start with NULL to keep this from happening, but it needs to be made into a loader.conf tunable. I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. pgp13Q2HNhDcL.pgp Description: PGP signature
Re: [Testers wanted] /dev/console cleanups
On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick [EMAIL PROTECTED] wrote: There's a known issue with the kernel message buffer though: it's not NULL'd out upon reboot. This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. That doesn't sound deliberate at all -- it sounds like a quirk that people (you?) are relying on. I do not think any piece of the FreeBSD system (e.g. savecore, etc.) relies on this behaviour. You're under the mentality that the information is *always* available after a panic/reboot -- it isn't. I have 4 different Supermicro motherboards (all from different years) which will most of the time lose the msgbuf after rebooting from single-user -- but sometimes the msgbuf is retained. And no, bad hardware is not responsible for the randomness of the problem. I think it's been discussed in the past how/why this can happen. It has to do with what each BIOS manufacturer chooses to do with some parts of memory during start-up. I'm sure the Quick Boot (e.g. no extensive memory test, which really doesn't test anything these days) option plays a role, and that option is enabled by default on all motherboards I've used in the past 10 years. Meaning, in some cases (depends on the BIOS or system), the kernel message buffer from single-user mode is retained even after a reboot! A user can then do dmesg and see all the nifty stuff you've done during single-user, which could include unencrypted passwords if mergemaster was tinkering with passwd/master.passwd, etc.. There shouldn't be unencrypted passwords, though there might be encrypted passwords visible. Sorry, that's what I meant. The point is that a lot of things can go on in single-user mode which can/will disclose information or data in files which users do not have access to. Once the system is rebooted, a non-root user can do dmesg -a and see this buffer, getting access to data he/she normally does not have access to. Do you and I agree that this is in fact a security risk/problem? Rink Springer created a patch where the kernel message buffer will start with NULL to keep this from happening, but it needs to be made into a loader.conf tunable. I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. I would like to see Rink's patch committed, as long as the loader tunable defaults to *off* (e.g. current/historic behaviour). I'll also ask Rink to chime in here with his thoughts/opinions. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
On Wed, 19 Nov 2008, Jeremy Chadwick wrote: On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This way the buffer is cleared before any unprivileged users get to do anything. No kernel changes needed, just a little tweaking of the init scripts at most. If you should have a crash and suspect there is useful data in the buffer, you can boot to single-user mode (avoiding the clear) and retrieve it manually. Seems like this should make everyone happy. -- Nate Eldredge [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Testers wanted] /dev/console cleanups
Hello Carlos, others,
* Ed Schouten [EMAIL PROTECTED] wrote:
About the /dev/console issues: Robert Watson and I discussed this some
time ago on IRC and what I did in HEAD (not RELENG_7) was that I changed
TIOCCONS not to take a look at the permissions of /dev/console, but we
changed it to use priv_check(). This means that right now you can only
call TIOCCONS as root. I can't really understand why the problem exists
on RELENG_7.
About making xconsole setuid: I've read the messages you mentioned, but
I think we could just alter console to call TIOCCONS and just drop
privileges. An even better solution would be to just get rid of TIOCCONS
and invent a better solution to capture syslog messages. I can't really
understand why we want to abuse TTY's to do this.
So I can't say we're working on this, but at least I can confirm the
issue.
One solution would be to let xconsole just display /var/log/messages.
There shouldn't be a valid reason to let syslogd print messages to
/dev/console and capture them again using TIOCCONS. We could just
instruct xconsole to read its data from the log files.
If you save the attached patch as /usr/ports/x11/xconsole/files/
patch-xconsole.c (create directory first) and recompile xconsole, it
will use the log file.
I'll discuss this with others to decide if we should take such an
approach.
--
Ed Schouten [EMAIL PROTECTED]
WWW: http://80386.nl/
--- xconsole.c
+++ xconsole.c
@@ -145,6 +145,11 @@
{-saveLines, *saveLines, XrmoptionSepArg, NULL},
};
+#ifdef __FreeBSD__
+#define USE_FILE
+#define FILE_NAME /var/log/messages
+#endif
+
#ifdef ultrix
#define USE_FILE
#define FILE_NAME /dev/xcons
@@ -252,7 +257,7 @@
if (!strcmp (app_resources.file, console))
{
/* must be owner and have read/write permission */
-#if !defined(__NetBSD__) !defined(__OpenBSD__) !defined(Lynx) !defined(__UNIXOS2__)
+#if !defined(__FreeBSD__) !defined(__NetBSD__) !defined(__OpenBSD__) !defined(Lynx) !defined(__UNIXOS2__)
struct stat sbuf;
# if !defined (linux)
if (!stat(/dev/console, sbuf)
@@ -266,9 +271,11 @@
if (!stat(FILE_NAME, sbuf))
# endif
input = fopen (FILE_NAME, r);
-# ifdef __UNIXOS2__
if (input)
{
+ struct stat sbuf;
+
+# ifdef __UNIXOS2__
ULONG arg = 1,arglen;
APIRET rc;
if ((rc=DosDevIOCtl(fileno(input), 0x76,0x4d,
@@ -278,8 +285,11 @@
fclose(input);
input = 0;
}
- }
# endif
+
+ if (!fstat(fileno(input), sbuf) S_ISREG(sbuf.st_mode))
+ regularFile = TRUE;
+ }
#endif
#ifdef USE_PTY
pgpt0FzOtlKLm.pgp
Description: PGP signature
Re: [Testers wanted] /dev/console cleanups
On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: ... One solution would be to let xconsole just display /var/log/messages. Errr... it may be rather a pathological case, but you might want to check the content of /etc/syslog.conf on the local machine before getting too carried away with that approach. For example, on my firewall box at home (where I really do not want to log anything to local disk files, though I do have a serial console on it): janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf *.* @bunrab.catwhisker.org janus(6.4-P)[2] And then consider the fate of bunrab -- with stuff getting logged to /var/log/messages from various machines ... I'll discuss this with others to decide if we should take such an approach. I'm not trying to be obstructionist, here. If the above case is really too pathological to consider -- or if it's a case of me bringing that fate upon myself, I suppose -- that's actually something I can live with. It would be nice to be forwarned about it, though. :-} Peace, david -- David H. Wolfskill [EMAIL PROTECTED] Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgpLDVAJGmZrP.pgp Description: PGP signature
Re: [Testers wanted] /dev/console cleanups
Ed, good day.
Tue, Oct 28, 2008 at 09:11:54AM +0100, Ed Schouten wrote:
It's nice to hear that the patch didn't break anything on your system. I
hope to receive more reviews, but I think I'll just commit it this
weekend (with small modifications).
Just a quick me too message: patchset from October, 29th works
flawlessly on my systems.
Thanks!
--
Eygene
____ _.--. #
\`.|\.....-'` `-._.-'_.-'` # Remember that it is hard
/ ' ` , __.--' # to read the on-line manual
)/' _/ \ `-_, /# while single-stepping the kernel.
`-' `\_ ,_.-;_.-\_ ', fsc/as #
_.-'_./ {_.' ; / #-- FreeBSD Developers handbook
{_.-``-' {_/#
pgp3Kf6n39Z2O.pgp
Description: PGP signature
Re: [Testers wanted] /dev/console cleanups
Hello Carlos, * Carlos A. M. dos Santos [EMAIL PROTECTED] wrote: The patched source builds and installs flawlessy. However I observed something that seems to be a regression. If I run either xconsole or xterm -C I only see kernel messages, even though my X startup (via XDM) changes the owner of /dev/console to the logged-in user. I mean, if I do some timg like echo OK /dev/console, the message is echoed on /dev/ttyv0, not by xconsole This is the same problem reported by Jeff Blank on RELENG_7: http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/044949.html http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/045885.html It's nice to hear that the patch didn't break anything on your system. I hope to receive more reviews, but I think I'll just commit it this weekend (with small modifications). About the /dev/console issues: Robert Watson and I discussed this some time ago on IRC and what I did in HEAD (not RELENG_7) was that I changed TIOCCONS not to take a look at the permissions of /dev/console, but we changed it to use priv_check(). This means that right now you can only call TIOCCONS as root. I can't really understand why the problem exists on RELENG_7. About making xconsole setuid: I've read the messages you mentioned, but I think we could just alter console to call TIOCCONS and just drop privileges. An even better solution would be to just get rid of TIOCCONS and invent a better solution to capture syslog messages. I can't really understand why we want to abuse TTY's to do this. So I can't say we're working on this, but at least I can confirm the issue. -- Ed Schouten [EMAIL PROTECTED] WWW: http://80386.nl/ pgpe3Z4oSjMVU.pgp Description: PGP signature
Re: [Testers wanted] /dev/console cleanups
On Sun, Oct 26, 2008 at 1:27 PM, Ed Schouten [EMAIL PROTECTED] wrote: Hello everyone, Most of you probably already know that I've been very busy improving our kernel's TTY implementation. I've committed the new MPSAFE TTY layer back in August. So far most of the things seem to work properly as far as I can see. There are always some small bugs, but I'm confident we'll get them fixed before 8.0. One of the things that I dislike about the code we have right now, is the way /dev/console is implemented. There is a small amount of complexity there, which is mainly because of the fact that our console code actually works on two different levels: - We've got kernel messages that are printed using very low-level routines and communicate directly with the drivers. - We've got user messages that are printed through /dev/console, which actually work on the TTY level, but make use of a similar device selection as the first set of routines. In an attempt to make /dev/console MPSAFE, I moved /dev/console into the TTY layer itself, which makes it a lot more simple than it is now. Well, to keep a long story short, it would be wonderful if some people could test the latest MPSAFE TTY patchset, just to make sure it doesn't wreck people's setups after I commit this. So just apply the patch and see if you can still boot your system, go into single user and multi user mode, use conscontrol(8), etc. I've stored the latest MPSAFE TTY patchsets at the usual location. Make sure you download the latest version. http://people.FreeBSD.org/~ed/mpsafetty/ The patchset also includes some other nice things, like some manual pages (not finished) and a port of snp(4) to the new TTY layer (also not finished). Thank you for your attention! The patched source builds and installs flawlessy. However I observed something that seems to be a regression. If I run either xconsole or xterm -C I only see kernel messages, even though my X startup (via XDM) changes the owner of /dev/console to the logged-in user. I mean, if I do some timg like echo OK /dev/console, the message is echoed on /dev/ttyv0, not by xconsole This is the same problem reported by Jeff Blank on RELENG_7: http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/044949.html http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/045885.html -- cd /usr/ports/sysutils/life make clean ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
[Testers wanted] /dev/console cleanups
Hello everyone, Most of you probably already know that I've been very busy improving our kernel's TTY implementation. I've committed the new MPSAFE TTY layer back in August. So far most of the things seem to work properly as far as I can see. There are always some small bugs, but I'm confident we'll get them fixed before 8.0. One of the things that I dislike about the code we have right now, is the way /dev/console is implemented. There is a small amount of complexity there, which is mainly because of the fact that our console code actually works on two different levels: - We've got kernel messages that are printed using very low-level routines and communicate directly with the drivers. - We've got user messages that are printed through /dev/console, which actually work on the TTY level, but make use of a similar device selection as the first set of routines. In an attempt to make /dev/console MPSAFE, I moved /dev/console into the TTY layer itself, which makes it a lot more simple than it is now. Well, to keep a long story short, it would be wonderful if some people could test the latest MPSAFE TTY patchset, just to make sure it doesn't wreck people's setups after I commit this. So just apply the patch and see if you can still boot your system, go into single user and multi user mode, use conscontrol(8), etc. I've stored the latest MPSAFE TTY patchsets at the usual location. Make sure you download the latest version. http://people.FreeBSD.org/~ed/mpsafetty/ The patchset also includes some other nice things, like some manual pages (not finished) and a port of snp(4) to the new TTY layer (also not finished). Thank you for your attention! -- Ed Schouten [EMAIL PROTECTED] WWW: http://80386.nl/ pgp5jhSTM0G8z.pgp Description: PGP signature
