Re: Automatic load of PCI kernel modules [WAS: [RFT] Automatic load of USB kernel modules]
On Jun 25, 2011, at 11:38 PM, Hans Petter Selasky wrote: > On Saturday 25 June 2011 19:07:01 Hans Petter Selasky wrote: >> On Saturday 25 June 2011 18:45:14 Warner Losh wrote: >>> On Jun 25, 2011, at 9:42 AM, Hans Petter Selasky wrote: On Saturday 25 June 2011 01:05:26 Jeremy Messenger wrote: > Jeremy Messenger Done. http://svn.freebsd.org/changeset/base/223536 >>> >>> Please move it back. It doesn't belong in /etc/defaults. It belongs in >>> /etc/devd if we're going to have it at all. If you have a full GENERIC >>> kernel, there will never be a NOMATCH line generated, so these entries >>> will just be ignored. >>> >>> Warner >> >> Done. >> >> http://svn.freebsd.org/changeset/base/223543 >> > > Hi, > > I see that a lot of PCI device drivers use code to check their ID's. > > One simple way to dump all the PCI device ID's is to compile or load all > modules and then make a dummy PCI device, which only calls device_probe on > the > full 32-bit space of the pci_get_devid(). This should take in matter of some > minutes to produce a complete list of PCI ID's. Then add these ID's to all > the > respective drivers like a generic device ID table: > > struct pci_device_id { > uint32_t device_id; > uint32_t card_id; > const char *description; > unsigned long driver_info; > } __aligned(32); > > Then export those automatically generated structures into the "pci_device_id" > section and have tools/bus_autoconf generate the matching rules. > > PCI format structure for bus_autoconf: > > "pci_device_id{256,:}" > "device_id[0]{" U32_XOR ",8}" > "device_id[1]{" U32_XOR ",8}" > "device_id[2]{" U32_XOR ",8}" > "device_id[3]{" U32_XOR ",8}" > > "card_id[0]{" U32_XOR ",8}" > "card_id[1]{" U32_XOR ",8}" > "card_id[2]{" U32_XOR ",8}" > "card_id[3]{" U32_XOR ",8}" I like the idea of having a standardized table. I've done this with PC Card and it really works well. It isn't the design pattern that all drivers use, and it may be hard to get everyone lined up on this. I tried PCI back after I did PC Card and met resistance. Most of the resistance was from people that are no longer active in the project, so I think that we could do this today. I suspect that some of the vendor drivers today might stand in the way of having PCI be completely uniform. The big advantage of USB is that it is uniform now. PCI isn't. It would take a lot of work to make it uniform. Also, like I was saying on IRC, I don't like the fact that we have to have different bus code in this automatic tool. It shouldn't know nor care about what the bus or drivers do. The drivers should export a table and a descriptor for the table that would allow it to either generate the devd.conf lines needed to do the matching, or for some other program to examine the pnp_string information from newbus and figure out which module(s) to load. Also, I don't find the above syntax easy to read or understand (sorry). Don't get me wrong. I love the USB autoload code you've done so far, but I consider it more of a prototype for the final system than the final system itself. Warner___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: kern.sync_on_panic
On Jun 25, 2011, at 8:49 AM, Andriy Gapon wrote: > Does anybody actually use kern.sync_on_panic tunable/sysctl? > If yes, then in what circumstances do you need it? > That is, why any other alternative doesn't work for you? > Like: > 1. remounting filesystems R/O before panic if you knowingly provoke it for > testing > 2. using netboot for your test system > 3. using su+j, gjournal or a different filesystem altogether > 4. using fsck after reboot > > It seems to me that syncing filesystems in panic context is an adventure. > And it > may become even more of an adventure if we introduce code that completely > stops > scheduler in and after panic. I've used it in the past when I was developing a device driver that was in the late stages of maturing. Since all the panics in the system were when the driver dereferenced NULL in that driver, sync was safe because all the data structures were sane except the aforementioned driver. (1) It was a production system, and everything that could be was already mounted r/w. However, some small, but every critical, amount of data was still r/w and it was very important to not lose this data. Production here likely should be in quotes, because it was in the late stages of testing/validation. The problem was without this sometimes the saved state of the GPS receiver and other hardware would wind up being zero, which meant that we'd have to do a cold start which cost us a few hours of time. At the time I was doing this, we saw zero files a couple times a day without this turned on. (2) netbooting wasn't an option since we were qualifying a non-netbooting system. (3) these weren't available at the time, but the goal was to prevent data loss, not to necessarily have to avoid fsck on boot. (4) Data loss without it. Now, I'll be the first to admit this has been a few years, and I haven't done a fresh evaluation to see if things are still safe. I'll also be the first to admit that this was a useful debugging setting late in development, and not in production. I'm also the first to admit this isn't what I'd call a very wide-spread case. But it did come in very handy when chasing a few bugs to be able to do 10 panic/reboot cycles an hour rather than 2 a day. Warner___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Automatic load of PCI kernel modules [WAS: [RFT] Automatic load of USB kernel modules]
On Saturday 25 June 2011 19:07:01 Hans Petter Selasky wrote: > On Saturday 25 June 2011 18:45:14 Warner Losh wrote: > > On Jun 25, 2011, at 9:42 AM, Hans Petter Selasky wrote: > > > On Saturday 25 June 2011 01:05:26 Jeremy Messenger wrote: > > >> Jeremy Messenger > > > > > > Done. > > > > > > http://svn.freebsd.org/changeset/base/223536 > > > > Please move it back. It doesn't belong in /etc/defaults. It belongs in > > /etc/devd if we're going to have it at all. If you have a full GENERIC > > kernel, there will never be a NOMATCH line generated, so these entries > > will just be ignored. > > > > Warner > > Done. > > http://svn.freebsd.org/changeset/base/223543 > Hi, I see that a lot of PCI device drivers use code to check their ID's. One simple way to dump all the PCI device ID's is to compile or load all modules and then make a dummy PCI device, which only calls device_probe on the full 32-bit space of the pci_get_devid(). This should take in matter of some minutes to produce a complete list of PCI ID's. Then add these ID's to all the respective drivers like a generic device ID table: struct pci_device_id { uint32_t device_id; uint32_t card_id; const char *description; unsigned long driver_info; } __aligned(32); Then export those automatically generated structures into the "pci_device_id" section and have tools/bus_autoconf generate the matching rules. PCI format structure for bus_autoconf: "pci_device_id{256,:}" "device_id[0]{" U32_XOR ",8}" "device_id[1]{" U32_XOR ",8}" "device_id[2]{" U32_XOR ",8}" "device_id[3]{" U32_XOR ",8}" "card_id[0]{" U32_XOR ",8}" "card_id[1]{" U32_XOR ",8}" "card_id[2]{" U32_XOR ",8}" "card_id[3]{" U32_XOR ",8}" --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Testing wanted] USB patch for HAL
On Sat, 25 Jun 2011, Andriy Gapon wrote: Hm, I have to ask, have you tried this without sg driver? Thanks for the suggestion. Hans Petter suggested the same thing in IRC earlier today. We're trying that and a few other things. -- Greg Rivers ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: GHC Port on 9-CURRENT
Hi Eric, On 06/26/11 02:13, Eric McCorkle wrote: > The GHC (Glasgow Haskell Compiler) was marked broken by pointyhat > sometime back in May, following its update to GHC 7.0.3_1. I'm not > sure if anyone has looked at this, but it doesn't appear anyone has. Yes, you are right, sorry, I have not had time to take a closer look at this, but there is a log [1] for it on pointyhat (that I could not also reproduce on a -CURRENT system). > The GCC build succeeds Sounds great. > With Clang, an error occurs in one of the configure scripts, because > Clang warns about unused command-line arguments, and the configure > script assumes that to be a compiler error. You can deal with this by > adding -Qunused-parameter to CFLAGS. Thanks for investigating this. > However, the stage 1 compiler's parser appears to be broken, as it > reports erroneous parse errors on the GHC compiler's source code (and > probably for any haskell program, but it never gets a chance to). I'm > not sure why this happens, but I will investigate more closely once I > deal with some more important issues I'm investigating. I would be happy to see the logs. > In any case, the port builds fine with GCC. Perhaps it would be > prudent to set CC=gcc and CXX=g++ in the makefile for the time being > and mark the port as working again? Yeah, that might be a solution, I will try it. [1] http://pointyhat.freebsd.org/errorlogs/amd64-errorlogs/a.9.20110516102328/ghc-7.0.3.log ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
GHC Port on 9-CURRENT
The GHC (Glasgow Haskell Compiler) was marked broken by pointyhat sometime back in May, following its update to GHC 7.0.3_1. I'm not sure if anyone has looked at this, but it doesn't appear anyone has. Sorry if any of this is repeated. I tried building the port yesterday, with both GCC and Clang. The GCC build succeeds, and the compiler seems to work just fine (it has no trouble compiling and installing the haskell-platform port, as well as many of my own haskell programs). With Clang, an error occurs in one of the configure scripts, because Clang warns about unused command-line arguments, and the configure script assumes that to be a compiler error. You can deal with this by adding -Qunused-parameter to CFLAGS. With this, it builds the bootstrap and stage 1 compilers. However, the stage 1 compiler's parser appears to be broken, as it reports erroneous parse errors on the GHC compiler's source code (and probably for any haskell program, but it never gets a chance to). I'm not sure why this happens, but I will investigate more closely once I deal with some more important issues I'm investigating. In any case, the port builds fine with GCC. Perhaps it would be prudent to set CC=gcc and CXX=g++ in the makefile for the time being and mark the port as working again? -- Eric McCorkle Computer Science Ph.D Student, University of Massachusetts Research Intern, IBM Research ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Testing wanted] USB patch for HAL
on 25/06/2011 21:53 Greg Rivers said the following: > Jun 25 13:15:45 blue kernel: ugen4.3: at usbus4 > Jun 25 13:15:45 blue kernel: umass0: Device, > class 0/0, rev 2.00/1.00, addr 3> on usbus4 > Jun 25 13:15:45 blue kernel: umass0: SCSI over Bulk-Only; quirks = 0x > Jun 25 13:15:47 blue kernel: umass0:0:0:-1: Attached to scbus0 > Jun 25 13:15:47 blue kernel: da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 > Jun 25 13:15:47 blue kernel: da0: Removable Direct > Access SCSI-2 device Jun 25 13:15:47 blue kernel: da0: 40.000MB/s transfers > Jun 25 13:15:47 blue kernel: da0: 983MB (2015231 512 byte sectors: 64H 32S/T > 983C) > Jun 25 13:15:47 blue kernel: (sg0:umass-sim0:0:0:0): > cam_periph_release_locked: > release 0xc47bfc00 when refcount is zero Hm, I have to ask, have you tried this without sg driver? -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Testing wanted] USB patch for HAL
On Fri, 24 Jun 2011, Hans Petter Selasky wrote: It appears there are some bugs in the USB2 HAL implementation. For example the parent USB device is not always correctly set and there are problems with dynamic attach/detach of USB devices in hald. For users of 9-current and 8-stable: Copy the attached file to /usr/ports/sysutils/hal/files/ Then rebuild HAL. Does it fix any USB/HAL related problems? For example related to multimedia/webcamd, lshal, mouse, keyboard etc. I tried your patch Hans (on 8.2-STABLE as of June 1), but it did not fix the problem I've been having on some machines with hald and umass devices. For reference, I first reported this problem[1] ~2 years ago. Basically what happens is this: when the first umass device is attached, I see the attach message and then a CAM error at ~1 second intervals (I assume this is the hald polling period) for the duration. When I pull the device I see the detach message, and it's over; hald is now unkillable, and the USB bus will no longer attach any other device, umass or otherwise. The only way out of this is a reboot. Here's an illustration: Jun 25 13:15:45 blue root: Unknown USB device: vendor 0x1307 product 0x0163 bus uhub4 Jun 25 13:15:45 blue kernel: ugen4.3: at usbus4 Jun 25 13:15:45 blue kernel: umass0: on usbus4 Jun 25 13:15:45 blue kernel: umass0: SCSI over Bulk-Only; quirks = 0x Jun 25 13:15:47 blue kernel: umass0:0:0:-1: Attached to scbus0 Jun 25 13:15:47 blue kernel: da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 Jun 25 13:15:47 blue kernel: da0: Removable Direct Access SCSI-2 device Jun 25 13:15:47 blue kernel: da0: 40.000MB/s transfers Jun 25 13:15:47 blue kernel: da0: 983MB (2015231 512 byte sectors: 64H 32S/T 983C) Jun 25 13:15:47 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:47 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:47 blue kernel: Jun 25 13:15:49 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:49 blue kernel: Jun 25 13:15:51 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:51 blue kernel: Jun 25 13:15:53 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:53 blue kernel: Jun 25 13:15:55 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:55 blue kernel: Jun 25 13:15:57 blue kernel: (sg0:umass-sim0:0:0:0): cam_periph_release_locked: release 0xc47bfc00 when refcount is zero Jun 25 13:15:57 blue kernel: Jun 25 13:15:58 blue kernel: ugen4.3: at usbus4 (disconnected) Jun 25 13:15:58 blue kernel: umass0: at uhub4, port 1, addr 3 (disconnected) Jun 25 13:16:07 blue kernel: (da0:umass-sim0:0:0:0): lost device Jun 25 13:16:07 blue kernel: (da0:umass-sim0:0:0:0): removing device entry At this point the USB bus is effectively wedged. It seems to be the detach that triggers the problem. If I do not detach anything, I can continue to add other devices and they attach fine. I'd be happy to continue troubleshooting and testing if you'd like to pursue this. Just let me know what you'd like me to do. Thanks for all your great work! [1] http://lists.freebsd.org/pipermail/freebsd-current/2009-August/010406.html -- Greg Rivers ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [Testing wanted] USB patch for HAL
On Saturday 25 June 2011 20:53:53 Greg Rivers wrote: > On Fri, 24 Jun 2011, Hans Petter Selasky wrote: > > It appears there are some bugs in the USB2 HAL implementation. For > > example the parent USB device is not always correctly set and there are > > problems with dynamic attach/detach of USB devices in hald. > > > > For users of 9-current and 8-stable: > > > > Copy the attached file to /usr/ports/sysutils/hal/files/ > > > > Then rebuild HAL. > > > > Does it fix any USB/HAL related problems? For example related to > > multimedia/webcamd, lshal, mouse, keyboard etc. > > I tried your patch Hans (on 8.2-STABLE as of June 1), but it did not fix > the problem I've been having on some machines with hald and umass devices. > For reference, I first reported this problem[1] ~2 years ago. > > Basically what happens is this: when the first umass device is attached, I > see the attach message and then a CAM error at ~1 second intervals (I > assume this is the hald polling period) for the duration. When I pull the > device I see the detach message, and it's over; hald is now unkillable, > and the USB bus will no longer attach any other device, umass or > otherwise. The only way out of this is a reboot. > > > At this point the USB bus is effectively wedged. It seems to be the > detach that triggers the problem. If I do not detach anything, I can > continue to add other devices and they attach fine. > > I'd be happy to continue troubleshooting and testing if you'd like to > pursue this. Just let me know what you'd like me to do. Thanks for all > your great work! Hi, Are you on #bsdusb on Efnet tonight or IM/Skype? --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CFT] patch to replace the regex code
Forgive me if I'm patronizing, but is there any surprise that a POSIX NFA implementation is slower than grep's DFA? Oh, of course an NFA implementation will always be slightly slower but the memory footprint will also be smaller, which is a big advantage for embedded systems. But in this case, the basis of the comparison was not the GNU DFA implementation but the old libc-regex. At the moment, BSD grep in the base system uses the GNU regex engine but I built a local version for testing purposes that used libc-regex. Gabor ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Thoughts on TMPFS no longer being considered "highly experimental"
On Fri, Jun 24, 2011 at 10:57:16PM +0300, Kostik Belousov wrote: > On Fri, Jun 24, 2011 at 06:20:03PM +0200, Peter Holm wrote: > > Got a "panic: Not a vnode object" quite fast: > > > > http://people.freebsd.org/~pho/stress/log/kostik441.txt > > Ah, yes, this is an assertion that was added in the r209702. > http://people.freebsd.org/~kib/misc/tmpfs.7.patch Looks good. The mmap(2) test doesn't panic any more, nor does any of the other TMPFS tests I have. - Peter ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT] Automatic load of USB kernel modules
On Saturday 25 June 2011 18:45:14 Warner Losh wrote: > On Jun 25, 2011, at 9:42 AM, Hans Petter Selasky wrote: > > On Saturday 25 June 2011 01:05:26 Jeremy Messenger wrote: > >> Jeremy Messenger > > > > Done. > > > > http://svn.freebsd.org/changeset/base/223536 > > Please move it back. It doesn't belong in /etc/defaults. It belongs in > /etc/devd if we're going to have it at all. If you have a full GENERIC > kernel, there will never be a NOMATCH line generated, so these entries > will just be ignored. > > Warner > Done. http://svn.freebsd.org/changeset/base/223543 --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT] Automatic load of USB kernel modules
On Jun 25, 2011, at 9:42 AM, Hans Petter Selasky wrote: > On Saturday 25 June 2011 01:05:26 Jeremy Messenger wrote: >> Jeremy Messenger > > Done. > > http://svn.freebsd.org/changeset/base/223536 Please move it back. It doesn't belong in /etc/defaults. It belongs in /etc/devd if we're going to have it at all. If you have a full GENERIC kernel, there will never be a NOMATCH line generated, so these entries will just be ignored. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT] Automatic load of USB kernel modules
On Jun 24, 2011, at 5:05 PM, Jeremy Messenger wrote: > On Fri, Jun 24, 2011 at 5:23 PM, Hans Petter Selasky wrote: >> On Saturday 25 June 2011 00:15:17 Warner Losh wrote: >>> Hey Hans, >>> >>> Given that all this stuff is really new and shiny, and we're really close >>> to the feature freeze, I'm not sure enabling it by default is the prudent >>> action. >>> >> >> Yes, you might be right. I'm not saying it should be enabled by default. >> >> BTW: How can we disable a .conf file from being picked up by devd? Or do I >> need to delete it from svn ? > > Maybe move it to src/etc/defaults/? No. I have no objection to usb.conf in /etc/devd. Just removing them all from the kernel just yet. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CFT] patch to replace the regex code
On 25 Jun 2011 15:08, "Gabor Kovesdan" wrote: > > Hi Folks, > > you may know that in the Summer of Code programme I'm working on replacing the old regex code with TRE, which is a BSD-licensed implementation. It supports wide characters, is POSIX-compliant and has a good performance compared to most of the open source implementations. Actually, I got mixed results. With sed, in the cases that I tested, the performance was more or less the same and in some few cases, TRE finished in half of the time. On the other hand, with grep sometimes it was significantly slower than the current regex code but grep has always been a complicated case. Forgive me if I'm patronizing, but is there any surprise that a POSIX NFA implementation is slower than grep's DFA? Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: LLVM: llvm-as, llvm-ld and so on not contained in FreeBSD core contrib?
On 06/25/11 10:10, Roman Divacky wrote: On Sat, Jun 25, 2011 at 09:57:52AM +0200, Hartmann, O. wrote: Hello. Just for my couriosity: I'm missing llvm-as, llvm-ld and other binutils from LLVM and was wondering why they are contained in the port's llvm collection but not in FreeBSD's source contribution. There's no use for these utilities in FreeBSD base system. I build FreeBSD 9 with CLANG. But as a missing llvm-as and llvm-ld (or llvm-ar) would imply, the binaries are generated via binutils from theGNU suite, aren't they? llvm-{as,ld,ar} are not replacements for those from binutils. llvm-* work on the llvm bitcode only and are of no use for normal object files. dim@ made a patch that adds those utilities if you really need them http://lists.freebsd.org/pipermail/freebsd-toolchain/2011-June/000216.html By default when you compile things with clang it uses its own assembler (ie. it goes directly from C -> .o) so typically only gnu ld is used in the compilation chain. roman Thank you very much. Patched and works. Oliver ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [RFT] Automatic load of USB kernel modules
On Saturday 25 June 2011 01:05:26 Jeremy Messenger wrote: > Jeremy Messenger Done. http://svn.freebsd.org/changeset/base/223536 --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Weird issue with hastd(8)
On Fri, Jun 3, 2011 at 11:26 AM, Maxim Sobolev wrote: > > I would also like to get your input on my two other patches - randomization > of the synchronization pattern and ad-hoc asynchronous more. Hastd appears > extremely useful to synchronize large virtual disks over slow links without > taking live virtual machine offline. For me the idea to send updates to secondary only via synchronization thread, starting it periodically looks interesting. Sure it should not be the replacement for "real" async mode, but having something like this in hast apart other synchronization modes might be useful. Comparing it with "real" async that is described in manual it has the following advantages: 1) It is much easier to implement. 2) If you have frequent updates of the same blocks, "real" async will send them all, while with sync thread approach we will skip many intermediate updates. Even if we don't run sync thread very frequently and HAST switches to failover it may sync dirty buffers from previous master. It might be useful for backuping volumes via WAN, instead of rsync or zfs send. There is a disadvantage -- instead of sending only one dirty block we synchronize the hole extent (see below how it may be improved though). But let me say about the problems with your patch: http://sobomax.sippysoft.com/primary.c.diff In your approach you still put the requests to the send thread but mark them there as failed so they are not actually sent and the extent is marked as need sync. You don't start sync thread. It starts in your case after reconnecting to secondary. You have frequent reconnects because of the following. Because there are requests in the send thread it does not send keep alive requests (it sends them only when it is idle) but actually the requests are not sent and the secondary exits by timeout not receiving any data from primary. Sure frequent reconnects are bad. Also the problem you described in "randomization" thread looks like is only possible with your patch. As the request "fails" in send thread the extent is marked as need sync, if at this time sync thread is running you may observe the effect when the same frequently updated extent is resent frequently. Without your patch an extent may be marked as need sync only when connection to secondary is lost, so synchronization is not running at that moment. I think the right approach could be: 1) Don't put the request to the send thread at all. 2) When returning the request to the kernel it still remains dirty in memmap. 3) periodically, the dirty (in memmap) extents are marked as need sync and the sync thread is waken up. Here is the patch that implements it: http://people.freebsd.org/~trociny/hast.async.patch The patch can not be considered as complete because of: 1) I think this mode should not be called async, because people would expect from it the behavior that was known from man (and how it works in DRBD it suppose). Also "real" async might be implemented in future too. Some other name should be thought out. 2) The synchronization thread is waked up in guard thread every HAST_KEEPALIVE seconds. I think it should be not so frequent and configurable. It can be improved but I would like to know Pawel's opinion first. He might know why this is completely wrong :-) Now about sending the hole extent when only small part of it is updated. It might be improved with checksum based synchronization. I have a patch that implements it -- when synchronizing an extent, before sending the chunk of MAXPHYS size, its checksum is send and if it matches the chunk is not sent. It is supposed to be useful when one needs to resync disks, e.g. after split brain, when most of the blocks on the nodes match. But apparently it should improve things in this case too. -- Mikolaj Golub ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
kern.sync_on_panic
Does anybody actually use kern.sync_on_panic tunable/sysctl? If yes, then in what circumstances do you need it? That is, why any other alternative doesn't work for you? Like: 1. remounting filesystems R/O before panic if you knowingly provoke it for testing 2. using netboot for your test system 3. using su+j, gjournal or a different filesystem altogether 4. using fsck after reboot It seems to me that syncing filesystems in panic context is an adventure. And it may become even more of an adventure if we introduce code that completely stops scheduler in and after panic. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: virtualbox-ose 4.0.8 fails
On Fri, 24 Jun 2011 16:11:27 -0400, Jung-uk Kim wrote: > On Friday 24 June 2011 02:58 pm, George Kontostanos wrote: >> On Fri, Jun 24, 2011 at 9:51 PM, Jung-uk Kim > wrote: >> >> Any ideas regarding the virtualbox itself ? >> > >> > I am rebuilding world/kernel now. �After that, I'll rebuild >> > virtualbox-ose and try to fix it unless someone beat me to it. >> > :-) >> > >> > Jung-uk Kim >> >> Brilliant !!! > > Please try this patch: > > http://people.freebsd.org/~jkim/patch-src-VBox-Main-src-server-freebsd-HostHardwareFreeBSD.cpp > > Just drop this in ports/emulators/virtualbox-ose/files and rebuild. Thanks a lot, they look good. Do you agree that those two patches are licensed under MIT License so that i can push them upstream? -- Bernhard Froehlich http://www.bluelife.at/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Fwd: stop scheduler in panic context
I would like to ask for testing of the following patch. Since the patch affects panic(9) context, then obviously its testing requires getting some sort of panic, and preferably some sort of "post-panic" activity too: doing something in kdb, dumping a core (via debugger command or in unattended mode), resetting a machine finally. At minimum I hope that no regressions are introduced. At maximum I hope that some things are improved like, e.g., crash dump succeeding where it failed before (PR amd64/139614). The patch is for recent head/CURRENT. It mostly affects SMP systems, but also has a smaller impact on UP systems. I can try to adapt it to stable/8, if sufficient interest arises. Please see the following message for the patch and some more details. Original Message I would like to present the following diff for review and discussion: http://people.freebsd.org/~avg/stop_scheduler_on_panic.diff The idea is to stop scheduler in a panic context and to provide a special environment for the only running thread, the one that called panic(9). I tried to make this diff as minimal as possible, it doesn't include changes that I consider to be useful improvements and [even] bug fixes, but which generated controversy in non-public discussions. If there is no negative feedback within next few days, then I plan to post the patch to current@ to solicit some testing. I will definitely wait for positive feedback before committing this change. I hope that I will be able to sneak it into the 9 release (unless there are objections to this). Thank you! -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
[CFT] patch to replace the regex code
Hi Folks, you may know that in the Summer of Code programme I'm working on replacing the old regex code with TRE, which is a BSD-licensed implementation. It supports wide characters, is POSIX-compliant and has a good performance compared to most of the open source implementations. Actually, I got mixed results. With sed, in the cases that I tested, the performance was more or less the same and in some few cases, TRE finished in half of the time. On the other hand, with grep sometimes it was significantly slower than the current regex code but grep has always been a complicated case and it has its own regex code, it was just used for testing here. I'm still working on some optimizations but apart from grep, the current performance may already be satisfying for normal cases. This is one thing that I would ask the interested testers to focus on: whether habitual scripts you execute finish later or sooner. I've also checked the POSIX-compliance and I found some cases when TRE is more permissive than the current implementation but that should not be a problem. The patch that I provide know probably can have a cleanup in the contrib area but it's just an early patch purely for testing purposes, so please avoid nitpicking for now and only report performance and/or functional problems. There's a code slush now so there's plenty of time to arrange this if it proves ready to go to 10-CURRENT. Thanks for all of you, who take the effort to give it a try. The patch is here: http://kovesdan.org/patches/tre-20110724.diff Regards, Gabor Kovesdan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Exactly that commit (was Re: Latest -current 100% hang at the late boot stage)
On Sat, Jun 25, 2011 at 07:27:20AM -0400, Justin T. Gibbs wrote: > > I use splitting by half method to find exact date which boots, then see > > the next commit above that date. Pre-commit kernel goes to multiuser and > > network is alive. I don't test CDs are working, I'll do that later and > > report it. > > So you know that revisions 223081, 223084, 223085, and 223089 all boot > just fine? I committed five revisions on that date. 223099 just happens > to be the last one for that day. OMG, I am very sorry to misleading you to wrong direction, and your time waste, give me pointy hat, please! It was r223081 which not boots. I.e. kernel cvsupped at src-sys date=2011.06.14.12.00.00 boots normally but cvsupped at src-sys date=2011.06.14.14.54.00 i.e. in the same minute after r223081 committed Tue Jun 14 14:53:17 2011 UTC hangs. I don't know how I may miss commits you mention(( -- http://ache.vniz.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Exactly that commit (was Re: Latest -current 100% hang at the late boot stage)
On 6/25/11 12:39 AM, Andrey Chernov wrote: On Fri, Jun 24, 2011 at 09:09:08PM -0400, Justin T. Gibbs wrote: >> No problem. I just set kern.geom.debugflags=4 in loader.conf and here is >> new photo (with recent kernel, no patches): >> http://img803.imageshack.us/img803/4679/25062011006.jpg >> I skip all noisy parts related to ada0 and ada1 partitions probes. >> As you can see, only 3 cd0-related geom call issued, right before cd1 >> probe shown. Strange thing is that I see no single cd1-related geom >> call, but it may be because of hang. > > The GEOM processing is serialized, so that is not unexpected. What your > logs are telling me is that the probe for CD0 is hanging. I don't know > why. Could you just postpone GEOM calls after any probe will be completed? It seems GEOM goes here even before probe and waits for probe forever. What probe waits in the same time is unclear for me (ccb_scan), but CD devices are slow and may not survive such multisleeping, missing some responses in the middle. The problem is not GEOM. It's not the thread waiting in ccb_scan - that thread is designed to wait there until an asynchronous device arrival/departure event occurs which is not the case here. The problem is in or below CAM, and that problem is causing the probe to never complete. > Are you positive it is this specific SVN revision that prevents cd0 > from probing properly and not one of my previous CAM commits? I use splitting by half method to find exact date which boots, then see the next commit above that date. Pre-commit kernel goes to multiuser and network is alive. I don't test CDs are working, I'll do that later and report it. So you know that revisions 223081, 223084, 223085, and 223089 all boot just fine? I committed five revisions on that date. 223099 just happens to be the last one for that day. -- Justin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: two issues after upgrading to a more up-to-date HEAD
On Fri Jun 17 11, Alexander Best wrote: > On Thu Jun 16 11, Alexander Best wrote: > > hi there, > > i reverted my kernel back to r222890. everything works fine now and both > issues > i mentioned below dissapeared. the previous issues (xpt_action_default and DIOCSKERNELDUMP) have been fixed. however this one, where a lot of apps fail with errno -128 when accessing the disk, still exists. this really needs to be fixed before 9.0! ...again...reverting to r222890 solves this completely. so it's not a problem with the disk or anything. ..i'm guessing this is ahci related, but i'm not sure. cheers. alex > > i also discovered another issue with the more recent kernel: > > i was getting errno -128 with a lot of apps. but only the first time i ran > them. e.g. with git: > > 1) -128 > 2) OK > 3) -128 > 4) OK > 5) ... > > the same with stuff like ./configure or ee(1). first time failures while > running or saving; second time it works. > > this as well was fixed by reverting back to r222890. > > cheers. > alex > > > > > i'm running HEAD on amd64. yesterday i updated my kernel to r223109. i'm now > > seeing two issues, which weren't there beforehand: > > > > 1) > > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > > ada0: ATA-7 SATA 2.x device > > ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > > ada0: Command Queueing enabled > > ada0: 238474MB (488395055 512 byte sectors: 16H 63S/T 16383C) > > ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 > > ada1: ATA-8 SATA 2.x device > > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > > ada1: Command Queueing enabled > > ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) > > xpt_action_default: CCB type 0xe not supported > > xpt_action_default: CCB type 0xe not supported > > xpt_action_default: CCB type 0xe not supported > > cd0 at ahcich2 bus 0 scbus2 target 0 lun 0 > > cd0: Removable CD-ROM SCSI-0 device > > SMP: AP CPU #1 Launched! > > cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) > > cd0: cd present [2291664 x 2048 byte records] > > > > ^^ i suspect the xpt_action_default messages have been introduced by the > > recent > > CAM changes. they appear to be harmless though. > > > > 2) > > dumpon: ioctl(DIOCSKERNELDUMP): Device not configured > > /etc/rc: WARNING: unable to specify /dev/label/swapfs as a dump device > > > > /dev/label/swapfs is a perfect swap partition and worked without any > > problems > > beforehand. specifying the device node instead of the label makes no > > difference: > > > > taku% dumpon /dev/ada1p1 > > dumpon: ioctl(DIOCSKERNELDUMP): Device not configured > > otaku% gpart show > > => 34 488394988 ada0 GPT (232G) > > 34128 1 freebsd-boot (64k) > > 162 16777216- free - (8.0G) > >16777378 471617644 3 freebsd-ufs (224G) > > > > =>34 1953525101 ada1 GPT (931G) > > 3420971520 1 freebsd-swap (10G) > > 20971554 4194304 2 freebsd-ufs (2.0G) > > 25165858 1928359277 3 freebsd-ufs (919G) > > > > otaku% glabel status > > Name Status Components > > label/boot N/A ada0p1 > > gptid/e52df583-e446-11de-bb92-000fb58207c8 N/A ada0p1 > > ufs/rootfs N/A ada0p3 > > label/swapfs N/A ada1p1 > > ufs/varfs N/A ada1p2 > > ufs/usrfs N/A ada1p3 > > iso9660/Futurama Season 6 Ep 1-8 720p N/A cd0 > > > > cheers. > > alex > > > > -- > > a13x > > -- > a13x ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: LLVM: llvm-as, llvm-ld and so on not contained in FreeBSD core contrib?
On Sat, Jun 25, 2011 at 09:57:52AM +0200, Hartmann, O. wrote: > Hello. > Just for my couriosity: I'm missing llvm-as, llvm-ld and other binutils > from LLVM and was wondering why they are contained in the port's llvm > collection but not in FreeBSD's source contribution. There's no use for these utilities in FreeBSD base system. > I build FreeBSD 9 with CLANG. But as a missing llvm-as and llvm-ld (or > llvm-ar) would imply, the binaries are generated via binutils from > theGNU suite, aren't they? llvm-{as,ld,ar} are not replacements for those from binutils. llvm-* work on the llvm bitcode only and are of no use for normal object files. dim@ made a patch that adds those utilities if you really need them http://lists.freebsd.org/pipermail/freebsd-toolchain/2011-June/000216.html By default when you compile things with clang it uses its own assembler (ie. it goes directly from C -> .o) so typically only gnu ld is used in the compilation chain. roman ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
LLVM: llvm-as, llvm-ld and so on not contained in FreeBSD core contrib?
Hello. Just for my couriosity: I'm missing llvm-as, llvm-ld and other binutils from LLVM and was wondering why they are contained in the port's llvm collection but not in FreeBSD's source contribution. I build FreeBSD 9 with CLANG. But as a missing llvm-as and llvm-ld (or llvm-ar) would imply, the binaries are generated via binutils from theGNU suite, aren't they? Sorry for the noise. O. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"