Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment)
On Sat, 08 Dec 2012 19:52:34 -0800 Ronald F. Guilmette r...@tristatelogic.com wrote: analysis skipped. As regards to the Native Command Queuing all I can say is Crap! I wasn't aware...until now... that FreeBSD did not support that. That really is a rather entirely serious issue. But I do think that the performance hit from that would be dwarfed by the performance hit that could be caused by the AF misaligment problem. ada0 at ahcich1 bus 0 scbus2 target 0 lun 0 ada0: ST3750330AS SD1A ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 715404MB (1465149168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ahcich3 bus 0 scbus4 target 0 lun 0 ada1: ST3750330AS SD1A ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 715404MB (1465149168 512 byte sectors: 16H 63S/T 16383C) Crap! I must be dreaming then. -- Alexander Kabaev signature.asc Description: PGP signature
Re: System is flooded with failed read(2) calls: Resource temporarily unavailable (errno=35) coming from xorg unix socket
On Fri, 29 Jun 2012 13:16:47 -0700 Yuri y...@rawbw.com wrote: SKIP # Declare DTrace script # if ($COUNT) { # aggregate style $dtrace = END; /usr/sbin/dtrace -n ' #pragma D option quiet syscall:::return /errno != 0 pid != \$pid $FILTER/ { \@Errs[execname, probefunc, errno] = count(); } dtrace:::END { printa(%s %s %d %\@d\\n, \@Errs); }' END Pardon my possibly naive question, but isn't using errno to detect whether the syscall is succesful a wrong techique? Syscall will NOT change errno unless unless it actually failed, so unless dtrace's errno emulation is more magic than I thought, your script will mistakenly attribute error code coming from a distant past to syscalls just complete with no errors? -- Alexander Kabaev signature.asc Description: PGP signature
Re: Import crt{begin,end}.S from NetBSD
On Thu, 14 Jun 2012 14:54:28 -0400 Richard Yao r...@gentoo.org wrote: NetBSD has replacements for GCC's crt{begin,end}.S: http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/csu/arch/?only_with_tag=MAIN This would complement compiler-rt and libstdc++. We intend to import it in downstream Gentoo FreeBSD. Could this be imported into FreeBSD-CURRENT? Apart from licensing, what others reasons are there to do that? -- Alexander Kabaev signature.asc Description: PGP signature
Re: Import crt{begin,end}.S from NetBSD
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 14 Jun 2012 22:00:18 -0400 Richard Yao r...@gentoo.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/14/12 20:51, Alexander Kabaev wrote: On Thu, 14 Jun 2012 14:54:28 -0400 Richard Yao r...@gentoo.org wrote: NetBSD has replacements for GCC's crt{begin,end}.S: http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/csu/arch/?only_with_tag=MAIN This would complement compiler-rt and libstdc++. We intend to import it in downstream Gentoo FreeBSD. Could this be imported into FreeBSD-CURRENT? Apart from licensing, what others reasons are there to do that? These components should not be tied to a specific compiler. If GCC is going to be deprecated, then they should be replaced. Anyway, having this tied to GCC has caused headaches for Clang integration in Gentoo. In particular, we let the user pick the toolchain that he uses, so we cannot place GCC's crt{begin,end}.o in the same location that FreeBSD uses. This makes it difficult for Clang to find the correct crt{begin,end}.o. We will likely import the NetBSD crt{begin,end}.S code to rectify this, but it would be preferable to do this in upstreamFreeBSD. Assuming NetBSD version is a direct plugin for crtbegin/end provided by GCC, I see no reason why we cannot do that. Are you are willing to do the work and submit the patch, or would like to wait for someone on our side? - -- Alexander Kabaev -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFP2pzlQ6z1jMm+XZYRAj9DAKDiYhGiRDL9Ow8/fkcBW+EOX1DrJwCfdJH7 bL9t1FXvMhua6bu2Sv5BwGE= =DbLg -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: ARM + CACHE_LINE_SIZE + DMA
On Tue, 22 May 2012 07:56:42 +0200 Hans Petter Selasky hsela...@c2i.net wrote: On Tuesday 22 May 2012 01:35:48 Alexander Kabaev wrote: On Thu, 17 May 2012 11:01:34 -0500 Mark Tinguely marktingu...@gmail.com wrote: On Thu, May 17, 2012 at 8:20 AM, Svatopluk Kraus onw...@gmail.com wrote: Hi, I'm working on DMA bus implementation for ARM11mpcore platform. I've looked at implementation in ARM tree, but IMHO it only works with some assumptions. There is a problem with DMA on memory block which is not aligned on CACHE_LINE_SIZE (start and end) if memory is not coherent. Let's have a buffer for DMA which is no aligned on CACHE_LINE_SIZE. Then first cache line associated with the buffer can be divided into two parts, A and B, where A is a memory we know nothing about it and B is buffer memory. The same stands for last cache line associatted with the buffer. We have no problem if a memory is coherent. Otherwise it depends on memory attributes. 1. [no cache] attribute No problem as memory is coherent. 2. [write throught] attribute The part A can be invalidated without loss of any data. It's not problem too. 3. [write back] attribute In general, there is no way how to keep both parts consistent. At the start of DMA transaction, the cache line is written back and invalidated. However, as we know nothing about memory associated with part A of the cache line, the cache line can be filled again at any time and messing up DMA transaction if flushed. Even if the cache line is only filled but not flushed during DMA transaction, we must make it coherent with memory after that. There is a trick with saving part A of the line into temporary buffer, invalidating the line, and restoring part A in current ARM (MIPS) implementation. However, if somebody is writting to memory associated with part A of the line during this trick, the part A will be messed up. Moreover, the part A can be part of another DMA transaction. To safely use DMA with no coherent memory, a memory with [no cache] or [write throught] attributes can be used without problem. A memory with [write back] attribute must be aligned on CACHE_LINE_SIZE. However, for example mbuf, a buffer for DMA can be part of a structure which can be aligned on CACHE_LINE_SIZE, but not the buffer itself. We can know that nobody will write to the structure during DMA transaction, so it's safe to use the buffer event if it's not aligned on CACHE_LINE_SIZE. So, in practice, if DMA buffer is not aligned on CACHE_LINE_SIZE and we want to avoid bounce pages overhead, we must support additional information to DMA transaction. It should be easy to support the information about drivers data buffers. However, what about OS data buffers like mentioned mbufs? The question is following. Is or can be guaranteed for all or at least well-known OS data buffers which can be part of DMA access that the not CACHE_LINE_SIZE aligned buffers are surrounded by data which belongs to the same object as the buffer and the data is not written by OS when given to a driver? Any answer is appreciated. However, 'bounce pages' is not an answer. Thanks, Svata Sigh. A several ideas by several people, but a good answer has not been created yet. SMP will make this worse. To make things worse, there are drivers that use memory from the stack as DMA buffers. I was hoping that mbufs are pretty well self-contained, unless you expect to modify them while DMA is happening. This is on my to-do list. --Mark. Drivers that do DMA from memory that was not allocated by proper busdma methods or load buffers for DMA using not properly constrained busdma tags are broken drivers. We did not have a busdma tag inheritance from parent bus to child devices before, but now we should just take advantage of that and just make cache line alignment a requirement for the platform. USB is firmly in that 'broken' category btw and is currently being worked around by the USB_HOST_ALIGN hack on MIPS, which suffers from the very same cache coherency issues you describe. Hi, Drivers do not always use the same buffer format. That mean two entities exchanging data using different buffer allocations must either: 1) Copy the data 2) Negotiate parameters for zero copy Many USB protocols have headers which are designed without any thought about ARM's and CACHE alignment. That means byte access via DMA must be supported, else you end up having to copy the data en-mass. The USB_HOST_ALIGN is not a hack. It is coherently implemented across EHCI, OHCI, UHCI and XHCI drivers, which are currently the only USB drivers using DMA. BUSDMA must instruct use of bounce buffers for case 1
Re: ARM + CACHE_LINE_SIZE + DMA
On Thu, 17 May 2012 11:01:34 -0500 Mark Tinguely marktingu...@gmail.com wrote: On Thu, May 17, 2012 at 8:20 AM, Svatopluk Kraus onw...@gmail.com wrote: Hi, I'm working on DMA bus implementation for ARM11mpcore platform. I've looked at implementation in ARM tree, but IMHO it only works with some assumptions. There is a problem with DMA on memory block which is not aligned on CACHE_LINE_SIZE (start and end) if memory is not coherent. Let's have a buffer for DMA which is no aligned on CACHE_LINE_SIZE. Then first cache line associated with the buffer can be divided into two parts, A and B, where A is a memory we know nothing about it and B is buffer memory. The same stands for last cache line associatted with the buffer. We have no problem if a memory is coherent. Otherwise it depends on memory attributes. 1. [no cache] attribute No problem as memory is coherent. 2. [write throught] attribute The part A can be invalidated without loss of any data. It's not problem too. 3. [write back] attribute In general, there is no way how to keep both parts consistent. At the start of DMA transaction, the cache line is written back and invalidated. However, as we know nothing about memory associated with part A of the cache line, the cache line can be filled again at any time and messing up DMA transaction if flushed. Even if the cache line is only filled but not flushed during DMA transaction, we must make it coherent with memory after that. There is a trick with saving part A of the line into temporary buffer, invalidating the line, and restoring part A in current ARM (MIPS) implementation. However, if somebody is writting to memory associated with part A of the line during this trick, the part A will be messed up. Moreover, the part A can be part of another DMA transaction. To safely use DMA with no coherent memory, a memory with [no cache] or [write throught] attributes can be used without problem. A memory with [write back] attribute must be aligned on CACHE_LINE_SIZE. However, for example mbuf, a buffer for DMA can be part of a structure which can be aligned on CACHE_LINE_SIZE, but not the buffer itself. We can know that nobody will write to the structure during DMA transaction, so it's safe to use the buffer event if it's not aligned on CACHE_LINE_SIZE. So, in practice, if DMA buffer is not aligned on CACHE_LINE_SIZE and we want to avoid bounce pages overhead, we must support additional information to DMA transaction. It should be easy to support the information about drivers data buffers. However, what about OS data buffers like mentioned mbufs? The question is following. Is or can be guaranteed for all or at least well-known OS data buffers which can be part of DMA access that the not CACHE_LINE_SIZE aligned buffers are surrounded by data which belongs to the same object as the buffer and the data is not written by OS when given to a driver? Any answer is appreciated. However, 'bounce pages' is not an answer. Thanks, Svata Sigh. A several ideas by several people, but a good answer has not been created yet. SMP will make this worse. To make things worse, there are drivers that use memory from the stack as DMA buffers. I was hoping that mbufs are pretty well self-contained, unless you expect to modify them while DMA is happening. This is on my to-do list. --Mark. Drivers that do DMA from memory that was not allocated by proper busdma methods or load buffers for DMA using not properly constrained busdma tags are broken drivers. We did not have a busdma tag inheritance from parent bus to child devices before, but now we should just take advantage of that and just make cache line alignment a requirement for the platform. USB is firmly in that 'broken' category btw and is currently being worked around by the USB_HOST_ALIGN hack on MIPS, which suffers from the very same cache coherency issues you describe. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Where do the elf32_obj_loadfile, elf32_loadfile, elf64_obj_loadfile and elf64_loadfile symbols live?
On Sun, 29 Apr 2012 12:25:22 -0400 Richard Yao r...@cs.stonybrook.edu wrote: Dear Everyone, I tried compiling zfsloader from the FreeBSD 9.0-RELEASE tree on Gentoo Linux, but I encountered issues due to missing symbols: /var/tmp/portage/sys-boot/gptzfsloader-9.0/work/sys/boot/i386/zfsloader/../libi386/libi386.a(elf32_freebsd.o):(.data+0x0): undefined reference to `elf32_obj_loadfile' /var/tmp/portage/sys-boot/gptzfsloader-9.0/work/sys/boot/i386/zfsloader/../libi386/libi386.a(elf32_freebsd.o):(.data+0x8): undefined reference to `elf32_loadfile' /var/tmp/portage/sys-boot/gptzfsloader-9.0/work/sys/boot/i386/zfsloader/../libi386/libi386.a(elf64_freebsd.o):(.data+0x0): undefined reference to `elf64_obj_loadfile' /var/tmp/portage/sys-boot/gptzfsloader-9.0/work/sys/boot/i386/zfsloader/../libi386/libi386.a(elf64_freebsd.o):(.data+0x8): undefined reference to `elf64_loadfile' I searched the sources using grep, but I cannot find where the functions implementing those symbols are declared. Does anyone know where I can find them? Yours truly, Richard Yao Hi, please look at sys/elf_generic.c and macros it defines, namely __elfN. -- Alexander Kabaev signature.asc Description: PGP signature
Re: tftpd: avoid logging error for pxeboot option negotiation?
On Mon, 20 Feb 2012 15:09:10 -0500 Ed Maste ema...@freebsd.org wrote: After upgrading a diskless boot server from FreeBSD 6 to 8 I see an error message logged each time a diskless client boots: Feb 20 00:56:38 TPC-D4-35 tftpd[55229]: Got ERROR packet: TFTP Aborted It turns out that the pxeboot client (from Intel) first performs a TFTP read request with the tsize option to which it receives an acknowledgement containing the size of the file to be transferred. Then it sends back an error response to abort the transfer, and sends the read request again (without the tsize option). The sequence of packets is: 1. C-S TFTP Read Request, File: pxeboot, Transfer type: octet, tsize=0 2. S-C TFTP Option Acknowledgement, tsize=239616 3. C-S TFTP Error Code, Code: Not defined, Message: TFTP Aborted 4. C-S TFTP Read Request, File: pxeboot, Transfer type: octet, blksize=1456 5. S-C TFTP Option Acknowledgement, blksize=1456 6. C-S TFTP Acknowledgement, Block: 0 7. S-C TFTP Data Packet, Block: 1 ... I'd like to avoid logging the error here, for the sake of this pxeboot client and any other tftp clients that might check options without actually starting a transfer. Anyone opposed to removing it? (A proposed patch is at http://people.freebsd.org/~emaste/tftpd.diff). -Ed ___ IIRC, PXE sends an error packet with zero error code. Could you supress the error message in that case and avoid propagation of the 'ignore error' flag? PXE client is not alone doing that - custom TFTP implementation in pxeloader does that as well now, so suppressing errors in this case is a good idea. -- Alexander Kabaev signature.asc Description: PGP signature
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On Sat, 19 Nov 2011 12:01:50 +0200 Gleb Kurtsou gleb.kurt...@gmail.com wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ It should take about ten times less time than this thread took already to isolate _short_ test case demonstrating the problem, yet nothing of the sort has shown up yet from anyone involved. Am I missing something? -- Alexander Kabaev signature.asc Description: PGP signature
Re: rtld and noexec
On Fri, 2 Dec 2011 18:22:57 +0100 joris dedieu joris.ded...@gmail.com wrote: Hi, Here is a patch I use to prevent loading a shared object from a noexec mountpoint. It's an easy way, I found, after the last root exploit ((http://seclists.org/fulldisclosure/2011/Nov/452), to enhance the security of my web servers (with /home, /tmp and /var/tmp mounted with noexec). - the last ftpd/porftpd (libc ?) exploit does not work (indirect use of rtld via nsswitch) - the previous rtld security issue should have been more difficult to use in a noexec context. - It may help to prevent some miscellaneous usage of common softwares using dlopen like apache or php. I think it also makes sens because loading a shared object sounds like a kind of execution. What do you think about this patch and the opportunity to open a PR on this subject? Cheers Joris --- libexec/rtld-elf/rtld.c.orig2011-12-02 12:09:40.0 +0100 +++ libexec/rtld-elf/rtld.c 2011-12-02 13:45:18.0 +0100 @@ -1123,32 +1123,50 @@ { char *pathname; char *name; +struct statfs mnt; if (strchr(xname, '/') != NULL) { /* Hard coded pathname */ + name = NULL; if (xname[0] != '/' !trust) { _rtld_error(Absolute pathname required for shared object \%s\, xname); return NULL; } if (refobj != NULL refobj-z_origin) - return origin_subst(xname, refobj-origin_path); + pathname = origin_subst(xname, refobj-origin_path); else - return xstrdup(xname); + pathname = xstrdup(xname); +} +else { /* xname is not a path */ + if (libmap_disable || (refobj == NULL) || + (name = lm_find(refobj-path, xname)) == NULL) + name = (char *)xname; + + dbg( Searching for \%s\, name); + + pathname = search_library_path(name, ld_library_path); + if (pathname == NULL refobj != NULL) +pathname = search_library_path(name, refobj-rpath); + if (pathname == NULL) +pathname = search_library_path(name, gethints()); + if (pathname == NULL) +pathname = search_library_path(name, STANDARD_LIBRARY_PATH); +} + +if (pathname != NULL) { /* noexec mountpoint in pathname */ + if (statfs(pathname, mnt) != 0) +free(pathname); + else { +if (mnt.f_flags MNT_NOEXEC) { + _rtld_error(noexec violation for shared object \%s\, pathname); + free(pathname); + return NULL; +} +else + return pathname; + } } -if (libmap_disable || (refobj == NULL) || - (name = lm_find(refobj-path, xname)) == NULL) - name = (char *)xname; - -dbg( Searching for \%s\, name); - -if ((pathname = search_library_path(name, ld_library_path)) != NULL || - (refobj != NULL - (pathname = search_library_path(name, refobj-rpath)) != NULL) || - (pathname = search_library_path(name, gethints())) != NULL || - (pathname = search_library_path(name, STANDARD_LIBRARY_PATH)) != NULL) - return pathname; - if(refobj != NULL refobj-path != NULL) { _rtld_error(Shared object \%s\ not found, required by \%s\, name, basename(refobj-path)); ___ 1. There is a race using statfs and then loading the file. 2. We already have the check in do_load_object -- Alexander Kabaev signature.asc Description: PGP signature
Re: .eh_frame, .eh_frame_hdr - how to remove that trash
On Fri, 21 Oct 2011 00:54:32 -0700 Garrett Cooper yaneg...@gmail.com wrote: On Fri, Oct 21, 2011 at 12:51 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: on entry into each function, which is different from usual x86 convention. Asynchronous unwind info (yeah, same stuff you keep referring to as crap), is the only way you can debug your program or get anything remotely close to usable backtrace, by default. i understand but i DO NOT called compiler with -g option which clearly means that i do not want to debug isn't it? It seems like a binutils bug (or somewhere in that immediate neighborhood) because all debugging related sections should be stripped out by strip including unwind, correct? Thanks, -Garrett NO. Unwind info is a part of the ABI, and its relationship to debug flags as well as to wishes of the OP is very remote. -- Alexander Kabaev signature.asc Description: PGP signature
Re: .eh_frame, .eh_frame_hdr - how to remove that trash
On Fri, 21 Oct 2011 01:13:59 +0200 (CEST) Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: -fno-asynchronous-unwind-tables should get rid of unwind information, a.k.a. 'crap'. and this worked. found it just before getting your mail ;) yes and this is crap... possibly it is needed for some cases and some languages and i would not call it crap if it would not be included by default! It is part of the amd64 ABI, which defaults to not pushing stack frames on entry into each function, which is different from usual x86 convention. Asynchronous unwind info (yeah, same stuff you keep referring to as crap), is the only way you can debug your program or get anything remotely close to usable backtrace, by default. -- Alexander Kabaev signature.asc Description: PGP signature
Re: .eh_frame, .eh_frame_hdr - how to remove that trash
On Fri, 21 Oct 2011 00:20:52 +0200 (CEST) Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: i both don't use C++ and don't want to debug when i am linking final binary. how to turn this off? Which compiler do you use? supplied with FreeBSD 8.2 [wojtek@wojtek ~]$ cc -v Using built-in specs. Target: amd64-undermydesk-freebsd Configured with: FreeBSD/amd64 system compiler Thread model: posix gcc version 4.2.2 20070831 prerelease [FreeBSD] -fno-asynchronous-unwind-tables should get rid of unwind information, a.k.a. 'crap'. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Report Kernel log
On Mon, 22 Aug 2011 20:33:48 + elman elman_s...@yahoo.com wrote: Hai hacker, I have received the kernel log on freebsd 8.2 kernel log messages: +++ /tmp/security.qHlmbCRv2011-08-23 03:05:25.0 +0700 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x05 Depth 122 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x05 Depth 122 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x05 Depth 122 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x05 Depth 122 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x06 Depth 119 +mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x06 Depth 119 What the issue? Ceers. Elman Powered by Telkomsel BlackBerry® The device is telling us back that it cannot hold as many open tags as we are trying to feed it. This is not fatal, CAM will scale back and will settle on some smaller value that will keep the device happy. -- Alexander Kabaev signature.asc Description: PGP signature
Re: MIPS toolchain
On Fri, 29 Jul 2011 11:12:35 -0400 James Jones ja...@freedomnet.co.nz wrote: Does anyone have a prebuilt MIPS tool chain? There's so many unanswered details in this loaded question one cannot possibly hope to answer it correctly. Are you asking for what mips ISA and ABIs? What is to be used as a host? In general, toolchain is generally quite easy to build as cross-tool, assuming FreeBSD supports the target. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] fake pre-processor macros when building on non-FreeBSD system
On Tue, 12 Jul 2011 22:00:56 +0200 Robert Millan r...@debian.org wrote: 2011/7/12 Oliver Pinter oliver.p...@gmail.com: +.if ${OPSYS} != FreeBSD and what's the status example on NetBSD? I didn't think of it. There are some instances of __NetBSD__ and also __OpenBSD__ in the kernel tree, and the same problem can be fixed for these two systems with minimal effort. Here's a new version of the patch, which also adds -U__NetBSD__ and -U__OpenBSD__ to CFLAGS. -- Robert Millan Whatever happened to using a proper cross-tool to do the job? Why is this hack needed? -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] fake pre-processor macros when building on non-FreeBSD system
On Tue, 12 Jul 2011 23:06:12 +0200 Robert Millan r...@debian.org wrote: 2011/7/12 Alexander Kabaev kab...@gmail.com: Whatever happened to using a proper cross-tool to do the job? Why would one need to build a cross-compiler in order to compile userland-agnostic code for the same CPU architecture? This would be like requiring a cross-compiler in order to build things like GRUB or SeaBIOS. Why is this hack needed? The kernel tree expects flags like __linux__ or __FreeBSD__ to have a different meaning when compiling for kernel space. Instead of we're building code that will run on $foo, they mean we're building $foo itself. This assumption is correct most of the time, but not always so. My patch addresses some of the situations in which the assumption fails. -- Robert Millan The fact that Linux compiler with manually undefined and re-defined platform macros can compile is a coincidence and is not guaranteed to work and certainly is not a goal of FreeBSD project so this can be broken at any moment. Relying on that is unwise, putting hacks into FreeBSD sources to legitimize the practice is not the move I would support as well. Traditionally, IMHO. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] fake pre-processor macros when building on non-FreeBSD system
On Tue, 12 Jul 2011 23:59:05 +0200 Robert Millan r...@debian.org wrote: 2011/7/12 Alexander Kabaev kab...@gmail.com: The fact that Linux compiler with manually undefined and re-defined platform macros can compile is a coincidence and is not guaranteed to work and certainly is not a goal of FreeBSD project so this can be broken at any moment. There must be some missunderstanding, I never asked the FreeBSD project to garantee this works, or that it won't break in the future. I'm not holding anyone responsible in case it breaks, or anything similar to that. Once it works, however, it is not a coincidence: We've been working to archieve this (most of my other patches sent to this list go in this direction). But it is the coincidence. See the other email by Nathan, I could not have said it better. Just defining __FreeBSD__ in Linux compiler does not magically turn it into compiler for FreeBSD. That aside, it seems there's some interest in FreeBSD community about building the kernel on GNU/Linux. In particular, MIPS porters have it in their TODO list: http://wiki.freebsd.org/FreeBSD/mips/todo Not sure what they mean there. There is a general desire for FreeBSD to be able to build the first set of bootstrap tools on other OSes (Linux, MAC OS X) that has come up several times in recent years. Host tools were to be used to build cross-toolchain only in all discussions I've been part of so far. putting hacks into FreeBSD sources to legitimize the practice is not the move I would support as well. Traditionally, IMHO. I try to take a neutral stance and merely follow on what the source tree expects. If the source tree does things like: #ifdef __FreeBSD__ // code that expects to be built as part of the FreeBSD kernel tree // (this chunk of code is invariably what we want, no matter the compiler we're using) #endif #ifdef __linux__ // code that expects to be built as part of the Linux tree // (this chunk of code depends on out-of-tree headers and will never build, no matter the compiler we're using) #endif I naively read this as when building kernel code, we assume these macros refer to *what* we're building rather than to what are we building it *for*. See Nathan's email again. __FreeBSD__ also means that code can expect certain ABI or some other detail to match established FreeBSD practice, which are not necessarily adhered to by the Linux compiler. Take your own example and imagine using the same assumption with Mac OS X toolchain. *what* and *for* both apply here, if I understand your dilemma correctly. My naive mind can't tell if this assumption is correct or not, it merely tries to adapt to existing practice. If the assumption is not correct, then I would propose a different kind of solution to the problem. So before we proceed, could you clarify whether the assumption is correct or not? -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sun, 3 Jul 2011 17:47:02 +0200 Robert Millan r...@debian.org wrote: 2011/7/3 Robert Millan r...@debian.org: 2011/7/3 Alexander Kabaev kab...@gmail.com: Not really, unless you have way of sticking this definition into past compiler releases. There is one way, but it's slow. It basically involves waiting for long enough that any past release of any compiler you care about includes the macros you need, before starting to use them. :-) However, in the case of FreeBSD, where the compilers are imported into your codebase, isn't it enough if support is present in the FreeBSD version of these compilers (for production and development releases of FreeBSD), plus the latest release of the upstream version of each of those compilers? -- Robert Millan The slow way would be the right way if you were inclined to really take it. Once old releases of tools that can be broken by the new macro use are long and forgotten we can start on relying on said macro, but not before. I disagree with the notion that we supporting only the imported compiler version is sufficient in FreeBSD. This was a long time deficiency of our development policy and it did more harm than good in the long term. We do want FSF source releases to work work out of the box, and when they do not, it is our loss. I know for a fact that several of our bigger customers are using compilers built for them by a third party and do not rely on the toolchain we supply in source tree. Breaking them would be bad. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sun, 3 Jul 2011 17:37:30 +0200 Robert Millan r...@debian.org wrote: 2011/7/3 Alexander Kabaev kab...@gmail.com: __linux__ is exactly what __FreeBSD__ is and dies not identify kernel but rather Linux as whole OS, whatever that might be these days. There does not appear to be an universal macro that identifies environment as using Linux kernel regardless of the rest of components used (say, to identify Android and Ubuntu or something embedded with ucLibc as running Linux kernel with different userland implementations). I think I'll cowardly try to stick to a practical POV and avoid the discussion about names that tends to get people excited about :-) So from a purely practical perspective: - If __linux__ is defined, this implies you're using a specific kernel. - If you're using that specific kernel, this implies __linux__ is defined. which explains why using __linux__ to identify the kernel is useful and reliable. Can __linux__ be used to identify more things? Most notably, can __linux__ be used to identify userland API? - If __linux__ is defined, this does not imply you're using GNU libc. - If you're using GNU libc, this does not imply __linux__ is defined. which explains why using __linux__ to identify libc is a bad idea. However, it can be useful to identify a set of different C libraries (Glibc, Bionic, uclibc, etc) if they share the property you're interested in. -- Robert Millan I agree with all of the above reasons, but none of them change the fact that __linux__ is used left and right to identify both kernel and userland environments just as __FreeBSD__ is. You choose to disable __FreeBSD__ in GNU/kFreeBSD presumably because it makes your life porting software easier and are asking FreeBSD project to cope with the decision. This breaks compiles of new software with older compilers than do not define the macro, takes __FreeBSD__ out of symmetry with __linux__ and other similarly defined platform macros and forces a race to update every other compiler out there to follow the suit. I fail to see the benefits out-weighting the drawbacks in this scenario. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Tue, 5 Jul 2011 21:04:41 +0200 Robert Millan r...@debian.org wrote: 2011/7/5 Alexander Kabaev kab...@gmail.com: I agree with all of the above reasons, but none of them change the fact that __linux__ is used left and right to identify both kernel and userland environments Yes, __linux__ is used to identify userland. And almost 100% of the times it is the wrong macro. We've spent several years fixing this kind of bug. To give just one example among hundreds, here's the latest incarnation of __linux__ misuse, reported 3 days ago: http://lists.debian.org/debian-bsd/2011/07/msg00013.html just as __FreeBSD__ is. Not exactly. For example, when you see __FreeBSD__, you know what libc you're dealing with. Only because there was no GNU/kFreeBSD before and people got lazy. Using __FreeBSD__ to identify userland can be considered just as wrong practice as using __linux__ for the same purpose, yet several years have been spent fixing this on Linux and quick hack is somehow appropriate for FreeBSD. Why do you keep arguing that intended use of __FreeBSD__ is any different than one of __linux__? It is not and both should be fixed when misused. You choose to disable __FreeBSD__ in GNU/kFreeBSD presumably because it makes your life porting software easier If Debian GNU/kFreeBSD enabled __FreeBSD__, software would expect that it is being built on FreeBSD, and make lots of wrong assumptions. Debian GNU/kFreeBSD is not FreeBSD. It is a derivative that builds on the kernel of FreeBSD and some of its kernel-specific libraries. It's not surprising that asserting we're FreeBSD via the compiler results in breakage (to begin with, even GCC headers wouldn't work properly). Then be a proper OS and have __kFreeBSD__ or what not defined in your toolchain. (similar statements can be made for e.g. Darwin, which I think you're familiar with) Darwin _is_ a proper first-class OS, per above. Last time I checked we were not forced to change out toolchain in any way to cater to their uses of components they borrow from FreeBSD. and are asking FreeBSD project to cope with the decision. I'm asking FreeBSD project to make life easier for a derivative that is, one way or another, part of its ecosystem, at no cost other than the time spent discussing the proposal. Not true, the change does break backward compatibility with older software if the new macro were to be used as you propose, to enable the code that is specific to FreeBSD kernel. This breaks compiles of new software with older compilers than do not define the macro, As far as I'm concerned, new software isn't supposed to rely on this macro unconditionally untill it is deemed reasonable to do so. takes __FreeBSD__ out of symmetry with __linux__ I could argue that __FreeBSD_kernel__ would be in symmetry with __linux__. But that's wordplay. I think we agreed to leave it out of the list :-) See above. Why inventing your own symbol as opposed fixing the use of existing one is more appropriate on FreeBSD and not on Linux? and other similarly defined platform macros and forces a race to update every other compiler out there to follow the suit. There's no race, and nobody forcing it. Predefined symbols are only useful if all compilers are consistent in their use. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sat, 2 Jul 2011 20:05:12 -0700 Garrett Cooper yaneg...@gmail.com wrote: On Sat, Jul 2, 2011 at 7:08 PM, Ed Maste ema...@freebsd.org wrote: On Sat, Jul 02, 2011 at 07:37:24PM -0400, Alexander Kabaev wrote: On Sat, 2 Jul 2011 17:41:03 +0200 Robert Millan r...@debian.org wrote: My request is that FreeBSD also defines __FreeBSD_kernel__. If this happens, life would be made a bit easier on both sides, as it'd be more natural for porters of either system to support both using a single macro [1]. I think this is a good idea, especially if it means that a single change can make it into upstream projects to support both FreeBSD and Debian kFreeBSD. I do not think this belongs in GCC at all. You should already have a defined symbol to identify your OS and that should be used in cases where it matters. I suspect the proposed patch put it in GCC based on the fact that we already have __FreeBSD__ in contrib/gcc/config/freebsd-spec.h: builtin_define_with_int_value (__FreeBSD__, FBSD_MAJOR); \ Alternatively, you should provide the symbol in similar way in which we provide __FreeBSD_version, through well-known header like sys/param.h and not pollute GCC. I suspect this is probably a reasonable alternative, but may mean software will have to pick up an additional #include. Out of curiosity, what is the canonical way for software to identify a Linux kernel -- __linux__ or some variant? Where is it defined? linux is most often reliably defined value based on my personal experience and it's defined in gcc [look for `builtin_define.*(linux);' (note: this is a regexp..)]. Example: $ echo '' | gcc -E -xc -dM -c - 21 | grep linux #define __linux 1 #define __linux__ 1 #define __gnu_linux__ 1 #define linux 1 HTH, -Garrett __linux__ is exactly what __FreeBSD__ is and dies not identify kernel but rather Linux as whole OS, whatever that might be these days. There does not appear to be an universal macro that identifies environment as using Linux kernel regardless of the rest of components used (say, to identify Android and Ubuntu or something embedded with ucLibc as running Linux kernel with different userland implementations). -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sun, 3 Jul 2011 12:41:44 +0200 Robert Millan r...@debian.org wrote: 2011/7/3 Alexander Kabaev kab...@gmail.com: GCC is on the way to be pushed out into ports in FreeBSD and it will not the the only usable compiler before long. Your proposal will force similar changes in Clang, Path64 and PCC, what else, to be really universal which is not practical. Would my proposal be more welcome if it included a patch for Clang, Path64 and PCC as well? -- Robert Millan Not really, unless you have way of sticking this definition into past compiler releases. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sun, 3 Jul 2011 15:38:41 +0100 Chris Rees cr...@freebsd.org wrote: 2011/7/3 Alexander Kabaev kab...@gmail.com: __linux__ is exactly what __FreeBSD__ is and dies not identify kernel but rather Linux as whole OS, whatever that might be these days. There does not appear to be an universal macro that identifies environment as using Linux kernel regardless of the rest of components used (say, to identify Android and Ubuntu or something embedded with ucLibc as running Linux kernel with different userland implementations). Please-- Linux is not and has never been an OS. __linux__ applies to the kernel, the OS is called GNU/Linux. Call me pedantic, but that was a nonsensical statement. Chris How about you spare Linux vs. GNU/Linux wordplay for advocacy lists, where they belong? -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] __FreeBSD_kernel__
On Sat, 2 Jul 2011 17:41:03 +0200 Robert Millan r...@debian.org wrote: Since their inception, GNU/kFreeBSD systems had defined __FreeBSD_kernel__ as builtin macro to indicate this is a system that uses the kernel of FreeBSD. We couldn't define __FreeBSD__ because this implies a full FreeBSD system, and a lot of software checks for this macro when it is concerned with userland (usually libc). As a result of this, and of the considerable porting effort that followed, many 3rd party programs with kernel-specific extensions have been ported to recognize __FreeBSD_kernel__ as well. E.g.: #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) // code for FreeBSD kernel #endif My request is that FreeBSD also defines __FreeBSD_kernel__. If this happens, life would be made a bit easier on both sides, as it'd be more natural for porters of either system to support both using a single macro [1]. [1] When porting software to support FreeBSD and systems with kernel of FreeBSD myself (as I did with e.g. GRUB), I generally took care to ensure both macros are checked for, but this isn't always the case. Having a unified macro would make it easier for developers of both systems to cooperate. -- Robert Millan I do not think this belongs in GCC at all. You should already have a defined symbol to identify your OS and that should be used in cases where it matters. Alternatively, you should provide the symbol in similar way in which we provide __FreeBSD_version, through well-known header like sys/param.h and not pollute GCC. GCC is on the way to be pushed out into ports in FreeBSD and it will not the the only usable compiler before long. Your proposal will force similar changes in Clang, Path64 and PCC, what else, to be really universal which is not practical. All IMHO. -- Alexander Kabaev signature.asc Description: PGP signature
Re: sizeof(function pointer)
On Tue, 31 May 2011 21:09:10 -0700 Marcel Moolenaar mar...@xcllnt.net wrote: On May 31, 2011, at 5:06 PM, Alexander Kabaev wrote: Usually it is different only on segmented architectures like 16-bit x86. Not so on ia64, where they have special function descriptor type. Actually, no. On ia64 a function pointer has the same size as a data pointer. It's just that a function pointer does not point to the actual function (i.e. the first instruction of a function), but to a function descriptor. The function descriptor contains the address of the actual function and the value of the GP register that needs to be set before entering the function. As such, only virtual functions in C++ are impacted by this. The function descriptor needs to be stored in the object instead of the function pointer in that case. FYI, -- Oh, you are correct. I forgot the double indirection we do to support that in dlsym, where we are maintain our own 'virtual table' of function descriptors within rtld itself. -- Alexander Kabaev signature.asc Description: PGP signature
Re: sizeof(function pointer)
On Tue, 31 May 2011 17:18:16 -0600 Warner Losh i...@bsdimp.com wrote: On May 31, 2011, at 5:07 PM, m...@freebsd.org wrote: I am looking into potentially MFC'ing r212367 and related, that adds drains to sbufs. The reason for MFC is that several pieces of new code in CURRENT are using the drain functionality and it would make MFCing those changes much easier. The problem is that r212367 added a pointer to a drain function in the sbuf (it replaced a pointer to void). The C standard doesn't guarantee that a void * and a function pointer have the same size, though its true on amd64, i386 and I believe PPC. What I'm wondering is, though not guaranteed by the standard, is it *practically* true that sizeof(void *) == sizeof(int(*)(void)), such that an MFC won't break binary compatibility for any supported architecture? (The standard does guarantee, though not in words, that all function pointers have the same size, since it guarantees that pointers to functions can be cast to other pointers to functions and back without changing the value). Another possibility is to malloc a blob that is sizeof(int(*)(void)) and store that in a renamed s_unused; this is a bit messier but guaranteed to work. I'd just rather the code be an MCF instead of a partial re-write. It is the same on MIPS too for all three ABIs that we support (and all ABIs that I know about). It is true on ARM as well. Usually it is different only on segmented architectures like 16-bit x86. Not so on ia64, where they have special function descriptor type. -- Alexander Kabaev signature.asc Description: PGP signature
Re: no KLD symbols in dtrace?
On Fri, 22 Apr 2011 07:27:30 -0700 Chuck Tuffli ctuf...@gmail.com wrote: On Thu, Apr 21, 2011 at 7:16 PM, Alexander Kabaev kab...@gmail.com wrote: ... There is an omission on our .mk files which prevents CTF info to be generated for kld modules, regardless of WITH_CTF flag. I had discovered this at work just recently and have been using the following patch for the time being: http://people.freebsd.org/~kan/kmod-dtrace.diff If you can confirm it works for you too, I'll get it committed. I patched /usr/src/sys/conf/kmod.mk and rebuilt my kld. It looks like ctfmerge is called at the end of the build, but dtrace still shows just the address. Maybe I missed a step? ---chuck Do ctfdump -tf on your kld and verify that what it outputs actually makes sense. Do kldload and see if fbt provider knows about your module and functions. There is nothing else special that I can think off. Maybe make sure that you compile your sources with -g in the first place? DWARF debug info is what CTF generation utils use to figure out types and function prototypes. -- Alexander Kabaev signature.asc Description: PGP signature
Re: no KLD symbols in dtrace?
On Thu, 21 Apr 2011 18:37:24 -0700 Chuck Tuffli ctuf...@gmail.com wrote: (Note I'm new to DTrace, so this may be ignorance on my part) I have re-built a stock 8.2 kernel and enabled dtrace as per the handbook (including WITH_CTF=1) and have built a kld also using WITH_CTF=1. When I run the following dtrace -nbus_release_resource:entry { stack(); } the output looks like ... 0 33759 bus_release_resource:entry 0x813db04e 0x813db091 kernel`device_detach+0x84 kernel`driver_module_handler+0x37c kernel`module_unload+0x49 kernel`linker_file_unload+0x178 kernel`kern_kldunload+0x117 kernel`syscallenter+0x23d kernel`syscall+0x4b kernel`0x808c5572 where the 0x813dbXXX addresses correspond to my kld. But I was expecting to see symbolic names instead of addresses. Is there something else I need to do to add the kld's symbols to the system? TIA. There is an omission on our .mk files which prevents CTF info to be generated for kld modules, regardless of WITH_CTF flag. I had discovered this at work just recently and have been using the following patch for the time being: http://people.freebsd.org/~kan/kmod-dtrace.diff If you can confirm it works for you too, I'll get it committed. Thanks, -- Alexander Kabaev signature.asc Description: PGP signature
Re: puzzled: fork +libthr
On Sun, 17 Apr 2011 17:39:43 +0300 Andriy Gapon a...@freebsd.org wrote: on 16/04/2011 14:46 Andriy Gapon said the following: The second puzzle is the EPERM return value itself, on stable/8. From what I seem chromium does a bunch of forks before it gets to the place of interest. My debugging shows that those forks are single-threaded (i.e. code in thr_fork.c is not called). And then in a process/thread that makes that pthread_cond_wait call I see that libthr and kernel have different opinions about what current TID is. Userland part uses what is actually a kernel TID of its parent thread (the one that called fork). And given how the work is divided between userland and kernel in libthr, that mismatch leads to serious consequences. So my question is why libthr doesn't see its actual TID. Maybe some initialization code is not invoked. BTW, chromium is linked to both libc and libthr (per ldd). But it seems that there are no pthread calls up the fork chain until that pthread_cond_wait call. The second problem seems to be caused by chrome binary being linked to libc and libthr in incorrect order, libc comes before libthr in ldd output. My debugging shows that fork is resolved from libc, not from libthr. Not sure what to blame here: - build toolchain for putting libc before libthr - rtld for not preferring libthr over libc - libc/libthr for being split into two pieces in the current way Why rtld would make any special allowances for libthr?! It does exactly what it is told to do, just as it should. -- Alexander Kabaev signature.asc Description: PGP signature
Re: puzzled: fork +libthr
On Sun, 17 Apr 2011 18:56:52 +0300 Andriy Gapon a...@freebsd.org wrote: on 17/04/2011 18:21 Daniel Eischen said the following: On Sun, 17 Apr 2011, Andriy Gapon wrote: on 16/04/2011 14:46 Andriy Gapon said the following: The second puzzle is the EPERM return value itself, on stable/8. From what I seem chromium does a bunch of forks before it gets to the place of interest. My debugging shows that those forks are single-threaded (i.e. code in thr_fork.c is not called). And then in a process/thread that makes that pthread_cond_wait call I see that libthr and kernel have different opinions about what current TID is. Userland part uses what is actually a kernel TID of its parent thread (the one that called fork). And given how the work is divided between userland and kernel in libthr, that mismatch leads to serious consequences. So my question is why libthr doesn't see its actual TID. Maybe some initialization code is not invoked. BTW, chromium is linked to both libc and libthr (per ldd). But it seems that there are no pthread calls up the fork chain until that pthread_cond_wait call. The second problem seems to be caused by chrome binary being linked to libc and libthr in incorrect order, libc comes before libthr in ldd output. My debugging shows that fork is resolved from libc, not from libthr. Not sure what to blame here: - build toolchain for putting libc before libthr - rtld for not preferring libthr over libc - libc/libthr for being split into two pieces in the current way - The build procedure for chromium. libc/[libc_r, libpthread, libthr] have always behaved that way since the libc/libc_r split. Well, I wouldn't blame it so expressly: -pthread is the first option on the linkage command line, there is -lc there also. I would expect that that would do the right thing, but it doesn't. And that's a PITA for porting. I would blame it, and expressly at that. -pthread is a shortcut for {-lpthread -lc} instead of just {-lc} in the place where implied libc is provided by the compiler driver and has no magic properties you want it it to have. If chromium build infrastructure circumvents that, it is only said build infrastructure to blame. -- Alexander Kabaev signature.asc Description: PGP signature
Re: rtld optimizations
noise. Pressing ^C certainly not precise enough for that. -- Alexander Kabaev signature.asc Description: PGP signature
Re: rtld optimizations
On Fri, 28 Jan 2011 00:47:37 +0100 Joerg Sonnenberger jo...@britannica.bec.de wrote: On Thu, Jan 27, 2011 at 06:24:18PM -0500, Alexander Kabaev wrote: For starters, the number of libraries given binary is linked too is completely and utterly irrelevant :) The change NetBSD guys claims to revolutionize his application startup times only applies to programs that dlopen (read - load dynamically) libraries with long largely identical dependency chains and calls dlsym on them many, many thousand times. I do not think you will find any real app out there that fits this description close enough to actually demonstrate the effect of the change that is distinguishable from the statistical noise. Pressing ^C certainly not precise enough for that. (1) The program itself needs to have a large enough number of linked libraries. That is wrong, of course. symlook_needed is only called when dlsym(real object handle, symbol name) is called and the number of objects on main and global DAGS is very much an irrelevant detail hereirrelevant here. (2) The program needs to dlopen() modules. Yep. (3) The program needs to search for missing symbols often enough. ... by means of explicit dlsym() call. From memory, the real world example that fulfilled all three cases was Evolution. (1) and (2) are pretty typical though. Never claimed otherwise. It is just ones with nested trees deep enough to matter is what in very short supply. I would be very interested in seeing the Evolution benchmarked, but I dount very much than even Evolution will manage to break through statistical noise. -- Alexander Kabaev signature.asc Description: PGP signature
Re: rtld optimizations
On Wed, 26 Jan 2011 09:25:27 -0600 Mark Felder f...@feld.me wrote: On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev kab...@gmail.com wrote: The only extra quirk that said commit does is an optimization of a dlsym() call, which is hardly ever in critical performance path. It's really not my place to say, but it seems strange that if an optimization is available people would ignore it because they don't think it's important enough. I don't understand this mentality; if it's not going to break anything and it obviously can improve performance in certain use cases, why not merge it and make FreeBSD even better? The link to patch with said optimization is provided and one would hope the next email on this thread provides something more alike to hard numbers proving that it actually helps, instead of mentality discussions. -- Alexander Kabaev signature.asc Description: PGP signature
Re: rtld optimizations
On Tue, 25 Jan 2011 21:40:42 -0500 Mark Saad nones...@longcount.org wrote: Hello Hackers The NetBSD folks have a nice improvement with the rtld-elf subsystem, known as Negative Symbol Cache . http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative Roy Marples roy@ has a simple write up of the change. I took the basic idea from FreeBSD, but improved the performance drastically. Basically, the huge win is by caching both breadth and depth of the needed/weak symbol lookup. Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row where we cache both rows and columns. Has anyone looked into porting the changes back to FreeBSD ? The improvement on load time for things like firefox, openoffice, and java is huge on NetBSD. It looks like this change could improve load times on FreeBSD in the same ways. This is a second time someone posts this to public mailing list and curiously enough is a second time it suggested that someone else is to do the investigation. From the quick look, the commit in question is more or less a direct rip-off of Donelists we had for ages and as such is completely over-hyped. The only extra quirk that said commit does is an optimization of a dlsym() call, which is hardly ever in critical performance path. Said optimization is trivial and easy to try. Here you have it: http://people.freebsd.org/~kan/rtld-symlook-depth.diff Since it only applies to dlsym, it only affects programs that are heavy plugin users, which I suppose is the category OpenOffice and firefox both fall into. Care to do some benchmarks with and without the patch and report the results? I frankly doubt that you'll see any noticeable difference compared to our stock rtld's performance. -- Alexander Kabaev signature.asc Description: PGP signature
Re: [PATCH] Add -lssp_nonshared to GCC's LIB_SPEC unconditionally
On Fri, 5 Nov 2010 22:39:06 +0100 Jeremie Le Hen jere...@le-hen.org wrote: Hi Kib, On Tue, Oct 05, 2010 at 08:18:04PM +0200, Jeremie Le Hen wrote: On Mon, Sep 27, 2010 at 06:44:57PM +0300, Kostik Belousov wrote: Hardcoding /usr/lib as the path to the library in the script looks problematic. For the buidlworld, you are linking resulting binaries with the host library, instead of the buildworld-produced one. For lib32, it makes non-working combination of 32/64 bit. Sorry for the late reply, but I had to collect various evidences for my sayings and my development machine is reaaally slow. In fact it seems the toolchain built for buildworld contains a ld(1) binary which invariably bases lookups for libraries in ${WORLDTMP}, even in case of an absolute path. I have two evidences of this: - Putting /usr/obj/usr/src/tmp/usr/lib/libssp_nonshared.a in /usr/obj/usr/src/tmp/usr/lib/libc.ld leads toolchain's ld(1) to use /usr/obj/usr/src/tmp/usr/obj/usr/src/tmp/usr/lib/libssp_nonshared.a; - I also verified this with a hand-wrought opensnoop-like DTrace script. I dare to remind you about my patch. Do you have any other concerns? Thanks. Regards, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche Hmm, I thought I did approve this patch already a long time agi, but since you asked: +.if defined(SHLIB_LDSCRIPT) exists(${.CURDIR}/${SHLIB_LDSCRIPT}) this should be: +.if defined(SHLIB_LDSCRIPT) ditto for all other similar places. Otherwise I do not think we should hold the patch in queue ans should unleash it on unsuspecting public. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Why I can't trace linux process's childs with truss?
On Fri, 10 Sep 2010 14:15:17 -0700 Garrett Cooper gcoo...@freebsd.org wrote: On Fri, Sep 10, 2010 at 12:07 PM, Yuri y...@rawbw.com wrote: I am trying to get the log of all system calls that skype makes with truss -f /usr/local/share/skype/skype For some reason the resulting log only has the leading process calls and nothing from it's 8 childs. Truss doesn't show any 'cloned' processes. Is this a bug in truss that it doesn't follow 'cloned' processes? Is there any workaround or other way I can debug skype? strace doesn't work on amd64. I am primarily interested why it can't read /dev/video0 device, created by webcamd. It doesn't look like ptrace support has been added into the linuxulator, yet.. but I could be wrong. -Garrett You are. ptrace is supported by linuxulator for a while now. The originator problem is likely because he is trying truss, which is not Linux-aware. ktrace/linux_kdump combo should work. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
while (nops 0 ps[nops - 1] == 0) (gdb) bt #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 #1 0x407f4314 in malloc_init () at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5418 #2 0x407f67d8 in malloc (size=32) at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5932 #3 0x001069ac in clone_setdefcallback (ifprefix=0x11b8a8 wlan, p=0x10a1a0 wlan_create) at /usr/home/marius/co/head/src/sbin/ifconfig/ifclone.c:106 #4 0x00119864 in __do_global_ctors_aux () #5 0x0010243c in _init () #6 0x00102508 in _start () #7 0x4022719c in .rtld_start () at /usr/home/marius/co/head/src/libexec/rtld-elf/sparc64/rtld_start.S:59 #8 0x4022719c in .rtld_start () at /usr/home/marius/co/head/src/libexec/rtld-elf/sparc64/rtld_start.S:59 Previous frame identical to this frame (corrupt stack?) All faults I've looked at died the same why. Thank you. I think I found the reason, which was an unitialized variable. I also fixed a sillyness with osrelver. In the patched tree, there is tools/test/auxinfo that could be used to quick-check the system. Updated patch is available at http://people.freebsd.org/~kib/misc/rtld_auxinfo.1.patch I can confirm that this versions works on sparc64. After the discussion with Alexander, I made the change to allow libc to access aux vectors, instead of providing rtld interface to fetch values. I decided to keep the single function that fetch the values, because it is convenient place to do digesting of the vector. http://people.freebsd.org/~kib/misc/rtld_auxinfo.2.patch No objections here. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Add -lssp_nonshared to GCC's LIB_SPEC unconditionally
On Tue, 3 Aug 2010 17:05:46 +0200 Jeremie Le Hen jere...@le-hen.org wrote: Hi, Currenty, -lssp_nonshared is appended at the link stage only when -fstack-protector is used on the gcc command-line. The problem I would like to fix is a library (static or shared) compiled with -fstack-protector being linked in without using the same flag: mygeeto# cc -I /usr/local/include -L /usr/local/lib -lintl conftest.c /usr/local/lib/libintl.so: undefined reference to `__stack_chk_fail_local' Since world is now compiled by default with -fstack-protector, this may happen to everyone naively compiling a source file like above. I therefore propose the following change to always link in libssp_nonshared.a. I think this change is harmless when the symbol is not needed in one of the objects linked together since the linker won't pull in the library member ssp-local.o in the target object. Index: freebsd-spec.h === RCS file: /data/repos/freebsd-cvsroot/src/contrib/gcc/config/freebsd-spec.h,v retrieving revision 1.26.2.2 diff -u -r1.26.2.2 freebsd-spec.h --- freebsd-spec.h 27 Dec 2009 20:39:58 - 1.26.2.2 +++ freebsd-spec.h 3 Aug 2010 10:18:08 - @@ -168,7 +168,7 @@ %{pg: %{pthread:-lpthread_p} -lc_p}} \ %{shared: \ %{pthread:-lpthread} -lc} \ - %{fstack-protector|fstack-protector-all:-lssp_nonshared} \ + -lssp_nonshared \ #endif #endif This change is also important because I've submitted a PR (138228) to compile ports with SSP. Of course many of them (although relatively few) break. If we eventually want this feature to reach the ports tree, it is necessary to fix as much problems as possible. I've already provided a few patches in the PR, but sometimes it is exaggeratedly difficult to fix the problem. For instance, devel/p5-Locale-gettext tests the existence of libintl.so with a naively compiled source file which doesn't link because of this error. The easiest way after the one proposed above would be to apply a patch conditionnaly. Thank you. Regards, -- Jeremie Le Hen Coluche I have no objection, but think we should cave in and investigate the possibility of using linker script wrapping libc.so in FreeBSD-9.0: Below is Linux' counterpart: /* GNU ld script Use the shared library, but some functions are only in the static library, so try that secondarily. */ OUTPUT_FORMAT(elf32-i386) GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.2 ) ) -- Alexander Kabaev signature.asc Description: PGP signature
Re: [DTRACE] ctfconvert .bss data: Invalid section descriptor +
On Thu, 15 Jul 2010 20:02:01 -0400 jhell jh...@dataix.net wrote: I should probably note that this is on stable/8 i386 r210115. And that this was before the 210145:210147 commits. This is a message from ctfconvert bogusly trying to read non-existent data from .bss. I have fix for this which I forwarded to our libelf author. The bug if harmless otherwise. -- Alexander Kabaev PS. I usually avoid responding to messages from strangely named entities, but there's always a place for exception... signature.asc Description: PGP signature
Re: *sigpause hanging on 8.x+
On Sun, 11 Jul 2010 15:59:05 -0700 Garrett Cooper yaneg...@gmail.com wrote: + if (!_SIG_VALID(how)) + return (-EINVAL); -EINVAL? Smells too much of Linux, try returning EINVAL instead. -- Alexander Kabaev signature.asc Description: PGP signature
Re: kernel patch needed for wine?
On Wed, 30 Jun 2010 14:42:47 -0700 Garrett Cooper yanef...@gmail.com wrote: On Wed, Jun 30, 2010 at 2:22 PM, Sam Fourman Jr. sfour...@gmail.com wrote: On Wed, Jun 30, 2010 at 11:26 AM, Garrett Cooper yanef...@gmail.com wrote: On Wed, Jun 30, 2010 at 8:43 AM, Sam Fourman Jr. sfour...@gmail.com wrote: Which patch ? icebp generates the SIGTRAP on latest 8-stable, verified by the following trivival assembler program: .text .globl main main: .byte 0xf1 xorl %edi,%edi call exit Here is the C program that the linux people used as a test case. *** #include stdio.h #include signal.h void trap_handler(int sig) { printf(trapped\n); } /* * icebp * ret */ char icebp_func[] = \xf1\xc3; typedef void (*icebp_call)(void); int main(int argc, char **argv) { icebp_call func = (icebp_call)icebp_func; signal(SIGTRAP, trap_handler); func(); return 0; } *** My question is why doe the above code not print trapped on amd64? FreeBSD 8.1 i386 this code prints Trapped as intended FreeBSD 8.1 amd64 this code prints Segmentation fault: 11 FreeBSD 8.1 amd64 chrooted to 32bit prints Segmentation fault I did verify that from Linux amd64 this works and prints Trapped uname -a Linux workstation 2.6.32-23-generic #37-Ubuntu SMP Fri Jun 11 08:03:28 UTC 2010 x86_64 GNU/Linux Hmmm... I've seen similar whackiness with Linux and signals, but that's a different thing entirely (it was rt signals vs non-rt signals). Here's a modified version of the testcase (wanted to make sure that things were sane): $ cat test_sigtrap.c #include err.h #include signal.h #include stdio.h int trapped = 0; void trap_handler(int sig) { trapped = 1; } /* * icebp * ret */ char icebp_func[] = \xf1\xc3; typedef void (*icebp_call)(void); int main(int argc, char **argv) { icebp_call func = (icebp_call)icebp_func; if (signal(SIGTRAP, trap_handler) == SIG_ERR) err(1, signal); func(); if (trapped) printf(Admiral Ackbar: it's a trap!\n); return 0; } Ran it and it segfaulted on CURRENT: Now make icebp_func const and observe the program start working. The test case is broken as written, because icebp_func array is writable, so in ends up in a non-const part of .bss, which is not marked as executable and rightfully causes SIGSEGV when jumped to. -- Alexander Kabaev signature.asc Description: PGP signature
Re: kernel patch needed for wine?
On Wed, 30 Jun 2010 16:45:18 -0700 Garrett Cooper yanef...@gmail.com wrote: Now make icebp_func const and observe the program start working. The test case is broken as written, because icebp_func array is writable, so in ends up in a non-const part of .bss, which is not marked as executable and rightfully causes SIGSEGV when jumped to. Which means that Linux is broken in this regard because it's loading data as text, not data as data and text as text? Thanks, Nope, I think this is i386 vs. amd64 difference. NX page protection is enforced in long mode, or in 32-bit with PAE, if I remember things correctly. -- Alexander Kabaev signature.asc Description: PGP signature
Re: bus_space_tag, bus_space_handle
On Thu, 11 Mar 2010 08:14:30 -0500 John Baldwin j...@freebsd.org wrote: On Thursday 11 March 2010 4:04:52 am Cole wrote: Hi. Im currently needing to write to a few registers for a PCI device. The driver is provided, but it does not contain support for writing specific registers itself. I also do not with to modify the driver. I would like to know what the best method would be for writing to these registers, the best way to go about getting bus_space_tag, and handle for the connected pci device. Im currently using FreeBSD 7.x. You will most likely need to modify the driver. The PCI bus code only hands out valid resources for a given BAR to the device_t for a given device. -- John Baldwin I used /dev/mem and dd in the past for this purpose. Not pretty and custom tool beats it any day, but it does work. -- Alexander Kabaev signature.asc Description: PGP signature
Re: NetBSD 5.0 statistics
On Thu, 30 Apr 2009 20:32:00 +0100 (BST) Robert Watson rwat...@freebsd.org wrote: On Thu, 30 Apr 2009, Oliver Pinter wrote: Is the FreeBSD's FS management so slow? http://www.netbsd.org/~ad/50/img15.html Or so big is the difference between the two cpu scheduler? Also, there's a known and serious performance regression in CAM relating to tgged queueing, and the generic disk sort routine, introduced 7.1, which will be fixed in 7.2. I can't speak more generally to the benchmarks -- we'll need to run them in a controlled environment and see if we can reproduce the results. Well, there is also r185801 of vfs_cache.c which all but destroys the usefulness of negative name caching and surely can account for a large percentage of reported performance drop in 7.1, especially with build.sh benchmarks. The bug was fixed in stable/7 soon after 7.1 was released. -- Alexander Kabaev signature.asc Description: PGP signature
Re: vfs_cache panic, 7.2-prerelease (stable)
How recent are your sources? There were a number of bugs introduced and then fixed in releng/7.2 and stable/7 and line number you post does not match anything interesting in either. Please make sure you have latest vfs_cache.c file at the least. -- Alexander Kabaev signature.asc Description: PGP signature
Re: vfs_cache panic, 7.2-prerelease (stable)
On Fri, 1 May 2009 20:21:13 +0100 xorquew...@googlemail.com wrote: Hello. Checking back through my sent mail from the DRI thread, this version of -STABLE was checked out and compiled on Fri, 10 Apr 2009 16:42:51 +0100. The machine was so new that I hadn't even set the clock, so the kernel timestamps appear to be Jan 1. I'll make an attempt to check out today's -STABLE and compile it but I'm not optimistic that the machine will actually be able to build world without reverting to a release first. You can always try to bypass namecache altogether while you are building the new kernel: do sysctl debug.vfscache=0 -- Alexander Kabaev signature.asc Description: PGP signature
Re: Kernel Module - GCC Requires memmove
On Thu, 22 Jan 2009 00:44:23 +0100 Dimitry Andric dimi...@andric.com wrote: On 2009-01-21 13:12, Andrew Brampton wrote: The .ii file (post-processed source) did NOT mention memmove at all. So I found it very odd that an undefined symbol existed in the object file. So then I looked in the .s file (asm), and it was clearing making a single call to memmove. This can (amongst others) occur if you assign structs, e.g.: int test(void) { struct foo { char bar[100]; } a, b; b = a; } Compile this with gcc -O0 -S, and you'll see it generates a call to memcpy(). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org From GCC's info pages: Most of the compiler support routines used by GCC are present in `libgcc', but there are a few exceptions. GCC requires the freestanding environment provide `memcpy', `memmove', `memset' and `memcmp'. /end quote We do not provide all necessary functions in kernel and mostly depend on luck for the kernel to link. Your luck apparently ran out :( -- Alexander Kabaev signature.asc Description: PGP signature
Re: Kernel Module - GCC Requires memmove
On Thu, 22 Jan 2009 00:52:13 + Andrew Brampton brampton+freebsd-hack...@gmail.com wrote: 2009/1/21 Alexander Kabaev kab...@gmail.com: From GCC's info pages: Most of the compiler support routines used by GCC are present in `libgcc', but there are a few exceptions. GCC requires the freestanding environment provide `memcpy', `memmove', `memset' and `memcmp'. /end quote We do not provide all necessary functions in kernel and mostly depend on luck for the kernel to link. Your luck apparently ran out :( Thanks for the info, good thing I'm not a gambling man. Anyway I also read that part of the GCC manual, so my next question is: If code can be generated with those four functions, why are they not exported by the kernel? Surely another kernel module will at some point also be hit by this? thanks Andrew Very good question and the answer is simple: we do not export these functions because nobody needed them yet :) Historically we have grown these functions on an 'as needed' basis. I am sure the patch to add missing functions would get committed if it were made available. hint -- Alexander Kabaev signature.asc Description: PGP signature
Re: remapping kernel buffer in VMS of user process
On Mon, 1 Dec 2008 02:38:51 +0100 Alexej Sokolov [EMAIL PROTECTED] wrote: Hello, I would like to remap some buffers allocated in kernel space to memory space of certain process. The simplest way is to expose this buffer through device pager. Implement the driver callback and let userland to simply mmap the page. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Why does adding /usr/lib32 to LD_LIBRARY_PATH break 64-bit binaries?
On Sat, 25 Oct 2008 08:49:19 -0400 Alexander Sack [EMAIL PROTECTED] wrote: Is this a bug or not in FreeBSD's rtld? -aps It is not. In case it was not clear before, I maintain that you _ask_ rtld for wrong behaviour and you get back what you asked for, down to the letter. 'Tasting' libraries just because someone somewhere want to screw up their configuration does not seem right to me at all. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Why does adding /usr/lib32 to LD_LIBRARY_PATH break 64-bit binaries?
On Sat, 25 Oct 2008 13:10:53 -0400 Alexander Sack [EMAIL PROTECTED] wrote: On Sat, Oct 25, 2008 at 9:57 AM, Alexander Kabaev [EMAIL PROTECTED] wrote: On Sat, 25 Oct 2008 08:49:19 -0400 Alexander Sack [EMAIL PROTECTED] wrote: Is this a bug or not in FreeBSD's rtld? -aps It is not. In case it was not clear before, I maintain that you _ask_ rtld for wrong behaviour and you get back what you asked for, down to the letter. 'Tasting' libraries just because someone somewhere want to screw up their configuration does not seem right to me at all. I maintain that rtld should not load 32-bit libraries for a 64-bit binary. That is WRONG anyway you look at it. And again, if it checked the arch type and skipped libutil.so.5 in /usr/lib32 it would fall back to checking /lib and things would work. Moreover, if /usr/lib had major number links just like /usr/lib32 has, this would again have worked without issue. I believe this will be fixed on the other side of the fence (not setting LD_LIBRARY_PATH to include /usr/lib32 to begin wtih) but still, my point still stands. -aps It doesn't. Stop feeding 32 bit libraries and it won't try to load them. It is as simple as that. For complex scenarious we do provide LD_32_ family of environment variables and if you refuse using them and insist on sticking with clearly broken configuration, it your problem, not rtld's. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Why does adding /usr/lib32 to LD_LIBRARY_PATH break 64-bit binaries?
On Thu, 23 Oct 2008 21:48:47 -0400 Alexander Sack [EMAIL PROTECTED] wrote: Thanks, comments most appreciated. Damn, I was looking for someone to go a ha, you can't do this because Alright, let me see why rtld on 6.1-amd64 is picking up /usr/lib32 stuff for a native 64-bit binary via debugging techniques. This seems very very wrong to me. I mean if /usr/lib is in my LD_LIBRARY_PATH and it comes before /usr/lib the /usr/lib32 *should* be innocuous, right? Feel free to use that last statement on my epitaph! :D LD_LIBRARY_PATH is for native 64bit rtld. If you want a specific path added for use by 32-bit ld-elf.so.1 only, use LD_32_LIBRARY_PATH. Said that, your problem is likely caused by the fact that there is no /lib32, only /usr/lib32. So if 64-bit library lives in /lib, your LD_LIBRARY_PATH will cause loader to find its 32-bit equivalent in /usr/lib32 first. Try LD_LIBRARY_PATH=/lib:/usr/lib:/usr/lib32:/usr/lib64 for better results. -- Alexander Kabaev signature.asc Description: PGP signature
Re: g++ associative containers
On Mon, 16 Jun 2008 20:13:01 +0300 Vlad GALU [EMAIL PROTECTED] wrote: On 6/16/08, Alexander Kabaev [EMAIL PROTECTED] wrote: On Mon, 16 Jun 2008 16:58:10 +0300 Vlad GALU [EMAIL PROTECTED] wrote: Hello list, I didn't have a clue where else to post this so I figured this place was the right one. I also CCed Alex Kabaev, who did the gcc 4 import. I'd like to use a Patricia container in the new libstdc++. The issue is that /usr/include/c++/4.2/ext/pb_ds/assoc_container.hpp has two erroneous #include directive on lines 52 and 53. Are there any plans to cover that part of the new libstdc++, or is there any workaround? Thanks, Vlad File a PR plaase and assign it to me. Hi Alexander, I submitted the PR, but I've no clue how to assign it to you. Neither send-pr or the web interface (which I ended up using) seemed to allow me to do that. This is probably a PEBKAC :) Never mind then Just mail me the PR number. -- Alexander Kabaev signature.asc Description: PGP signature
Re: g++ associative containers
On Mon, 16 Jun 2008 16:58:10 +0300 Vlad GALU [EMAIL PROTECTED] wrote: Hello list, I didn't have a clue where else to post this so I figured this place was the right one. I also CCed Alex Kabaev, who did the gcc 4 import. I'd like to use a Patricia container in the new libstdc++. The issue is that /usr/include/c++/4.2/ext/pb_ds/assoc_container.hpp has two erroneous #include directive on lines 52 and 53. Are there any plans to cover that part of the new libstdc++, or is there any workaround? Thanks, Vlad File a PR plaase and assign it to me. -- Alexander Kabaev signature.asc Description: PGP signature
Re: C++ exceptions are broken in FreeBSD with gcc-compiled code?
On Wed, 09 Apr 2008 09:39:09 -0700 Yuri [EMAIL PROTECTED] wrote: I am unable to make a C++ program to catch an exception using the the system g++ compiler. % c++ -o exc exc.c % ./exc Caught an exception String % c++ -O2 -pthread -o exc exc.c % ./exc Caught an exception String % c++42 -O2 -pthread -o exc exc.c % ./exc Caught an exception String % c++42 -O2 -o exc exc.c % ./exc Caught an exception String % cat exc.c #include iostream #include string using namespace std; int main() { try { throw string(String); } catch (string s) { cout Caught an exception \ s \\n; } return 0; } % uname -a FreeBSD kan.dnsalias.net 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Sun Apr 6 14:22:23 EDT 2008 ... Same on RELENG_6 (do not have 7.0 around, but 8.0 and 7.0 are identical compiler-wise. Same on 8.0/amd64. BTW, do you have . in your PATH? g++ -fexceptions -o exc exc.C exc should it be ./exc ? Exception raised: Memory allocation failure! ^^^ Not in your sample code. -- Alexander Kabaev signature.asc Description: PGP signature
Re: dlopen(), atexit() crash on FreeBSD (testcase included)
On Mon, 31 Dec 2007 19:01:23 -0800 Tim Kientzle [EMAIL PROTECTED] wrote: Markus Hoenicka wrote: Alexander Kabaev writes: As designed. atexit should not be used by shared objects that do not expect themselves to live until actual exit() happens. ELF provides proper _init/_fini sections to support shared object initialization/destruction. That is, the only real solution to this problem is to convince the Firebird folks to remove their atexit() calls from the client libraries? I suspect they never considered that their dynamic library might be used via dlopen()/dlclose(). The real question is whether they're interesting in supporting this model. If the Firebird folks aren't interested in having their library be accessible in that fashion, then you have little choice but to simply forgo unloading this particular library. It's a bit unfortunate that there is no standard way to remove an atexit() registration. It would probably be easier to convince the Firebird folks to remove the registration as part of their cleanup routines (and you could then invoke those cleanup routines manually for that case). Also, I'm wondering how other OSes handle this. I don't see this code crash on Linux, contrary to its design as you say. I would be curious to see the results of running your sample program (with lots of extra fprint(stderr...) calls, of course) on Linux to see whether it calls the registered exit function at dlclose time or never. Linux pulls hidden atexit symbol into every binary that uses it by means of linking in libc_nonshared.a into every glibc consumer. Having local function allows for reliable determination of who has called the atexit function. Linux calls atexit entries at object unload time. Solaris implements a libc callback from ld.so.1 to cleanup dangling pointers from objects being unloaded. This resets sigaction entries, atfork and atexit callbacks. Solaris calls atexit callback when removing it too. I guess we better follow the suit, if anyone wants to that. I prefer Solaris approach myself, but see the reason for Linux one too. -- Alexander Kabaev signature.asc Description: PGP signature
Re: dlopen(), atexit() crash on FreeBSD (testcase included)
On Mon, 31 Dec 2007 17:35:10 +0100 Markus Hoenicka [EMAIL PROTECTED] wrote: Hi, I've been redirected by Giorgos Keramidas to this list after reporting a problem on the freebsd-questions list. I'd greatly appreciate if you could have a look at the following problem. Apparently programs are doomed to segfault on FreeBSD if dlopen()ed modules install exit handlers via atexit(). Similar problem reports have cropped up before, see e.g. http://www.imagemagick.org/pipermail/magick-developers/2006-March/002523.html My system runs: FreeBSD yeti.mininet 6.1-RELEASE FreeBSD 6.1-RELEASE #1: Mon Aug 28 22:24:48 CEST 2006 [EMAIL PROTECTED]:/usr/src/sys/i386/compile/YETI i386 I'm one of the developers of libdbi, a database abstraction layer for C, see http://libdbi.sourceforge.net. libdbi is a library for programs which are supposed to be able to access different database engines with a unified API. libdbi essentially maps generic API calls to the specific database client library calls of a particular database engine. To do this, libdbi loads available database drivers at runtime via dlopen() calls. Each of these drivers is linked against one database client library. E.g. the Firebird driver is linked against libfbclient.so. When libdbi is properly shut down, it unloads all loaded drivers by calling dlclose() on each of them. This design works well on all supported platforms and with all supported database engines, with one exception: the Firebird driver on FreeBSD invariably causes a segfault when the application linked against libdbi exits: #0 0x28514fe4 in ?? () #1 0x281507c3 in __cxa_finalize () from /lib/libc.so.6 #2 0x281503fe in exit () from /lib/libc.so.6 #3 0x0804a40f in main (argc=1, argv=0xbfbfe754) at test_dbi.c:419 The reason appears to be that the Firebird client libraries install exit handlers via atexit(). Remember that due to libdbi's design to load all available drivers whether or not they are used later, libdbi will cause a crash even if no Firebird database is accessed - it is sufficient that the driver has been loaded. As per Giorgos' suggestion it is simple to circumvent this segfault by avoiding the call to dlclose() before exiting, but I wonder whether there is a more robust solution for this problem. The attached minimal testcase is sufficient to illustrate the problem. atexitmod.c defines a module which is loaded by datest.c Make sure to fix the hardcoded path in datest.c before building the app. To build the test program and watch it crash, do the following: gcc -shared -o atexitmod.so atexitmod.c gcc -o datest datest.c ./datest Commenting out either the atexit() call in atexitmod.c or the dlclose() call in datest.c prevent the segfault. If you find some solution, please cc me as I'm not subscribed to freebsd-hackers. regards, Markus As designed. atexit should not be used by shared objects that do not expect themselves to live until actual exit() happens. ELF provides proper _init/_fini sections to support shared object initialization/destruction. -- Alexander Kabaev signature.asc Description: PGP signature
Re: dlopen(), atexit() crash on FreeBSD (testcase included)
On Mon, 31 Dec 2007 14:20:51 -0800 Julian Elischer [EMAIL PROTECTED] wrote: Markus Hoenicka wrote: Alexander Kabaev writes: As designed. atexit should not be used by shared objects that do not expect themselves to live until actual exit() happens. ELF provides proper _init/_fini sections to support shared object initialization/destruction. That is, the only real solution to this problem is to convince the Firebird folks to remove their atexit() calls from the client libraries? I'll try, but I'm not very confident this is going to work. Also, I'm wondering how other OSes handle this. I don't see this code crash on Linux, contrary to its design as you say. Thanks anyway to you, and to Jason Evans, for your replies. Not sure but I'd guess that each library calls its at_exit entrypoints as part of its unload handling. This is not possible in general case. The object that calls atexit() is not necessarily object that contains clean-up routine, so when atexit() is to be called: when object that contains the routine itself is being unloaded or when the object that registered it goes away? Also, calling atexit hooks at that time is semantically wrong for atexit() as it is defined, dlclose(3) is NOT exit(2). C++ ABI defines a special __cxa_atexit function that does have proper semantics to work around atexit() deficiencies. We can hack rtld/libc to pull dangling atexit off the list entries when shared objects are being unloaded or to add extra references to libraries pointed to by atexit entries, but that will just paper over the general issue of Firebird (and like) folks using the API improperly. -- Alexander Kabaev signature.asc Description: PGP signature
Re: dlopen: resolving external library symbols to calling program
On Fri, 30 Nov 2007 22:19:48 +0200 Kostik Belousov [EMAIL PROTECTED] wrote: On Fri, Nov 30, 2007 at 04:40:33PM -0300, Alejandro Pulver wrote: On Fri, 30 Nov 2007 19:02:01 +0200 Kostik Belousov [EMAIL PROTECTED] wrote: On Fri, Nov 30, 2007 at 01:28:58PM -0300, Alejandro Pulver wrote: Hello. When I was updating the games/deng port, I found it failed at runtime with the following error: % doomsday While opening dynamic library /usr/local/lib/libdropengl.so: /usr/local/lib/libdropengl.so: Undefined symbol ArgExists DD_InitDGL: Loading of libdropengl.so failed. (null). The function is defined in m_args.c which is included in both doomsday and libdropengl.so. But nm(1) reports it as undefined for libdropengl.so. Also, it is loaded with RTLD_NOW. % nm `which doomsday` | grep ArgExists 080d9ef0 T ArgExists You are looking at the wrong symbol table. ELF objects have the dynamic symbol table that is used during run-time linking, and symbol table used by the static linker ld. The former table is shown by nm -D. I suspect that you need to link the doomsday binary with the --export-dynamic flag. See the info ld for details. It worked, thank you very much. I am reading some books that explain the basics of COFF/ELF formats (like Write Great Code Volume 2: Thinking Low-Level, Writing High-Level), but didn't know about the dynamic symbol table. I found the following article which briefly describes it (though it's for Solaris): http://blogs.sun.com/ali/entry/inside_elf_symbol_tables The ELF specification is freely available, and contains at least the neccessary minimum of information. The object format has evolved since then. Sun' Linkers and Loaders Guide also contain a lot of useful information, applicable to any ELF platform. Now that I remember, the games/quakeforge port had the same problem. But someone fixed it by referencing the symbol (it was only one function) with a function pointer so it got exported in the dynamic I remember it was me. table. In this case, could that be done with -u symbol when linking the executable, or it isn't possible to export a symbol with linker parameters? I do not know of any such facility in GNU ld. You may limit the exported symbols by using the version script. But, by default, no symbols are exported for executable (for shared object, reverse is true, all symbols are exported). Just a small correction: by default, linker _does_ export symbols from main application, just not all of them. Only symbols that satisfy references from shared libraries passed to linker on command line at binary creation time are exported. -- Alexander Kabaev -- Alexander Kabaev signature.asc Description: PGP signature
HEADS UP: OpenSSL problems after GCC 4.2 upgrade
Hi all, there were several reports of OpenSSL being broken when compiled with GCC 4.2. It turns out OpenSSL uses function casting feature that was aggressively de-supported by GCC 4.2 and GCC goes as far as inserting invalid instructions ON PURPOSE to discourage the practice. Consequently, OpenSSL need the patch similar to attached one to work. Just in case mailing list will eat the attachment, the patch can be found at http://people.freebsd.org/~kan/openssl-gcc42.diff Unfortunately, our OpenSSL maintainer(s) are currently en-route from BSDCan and cannot attend to the matters. Once we figure the best way to fix the code and to integrate the fix into OpenSSL, we will check the fix info CVS. People are advised to patch their sources locally until then. -- Alexander Kabaev Index: openssl/crypto/asn1/asn1.h === RCS file: /home/ncvs/src/crypto/openssl/crypto/asn1/asn1.h,v retrieving revision 1.1.1.8 diff -u -r1.1.1.8 asn1.h --- openssl/crypto/asn1/asn1.h 29 Jul 2006 19:10:16 - 1.1.1.8 +++ openssl/crypto/asn1/asn1.h 20 May 2007 05:01:40 - @@ -903,22 +903,22 @@ /* Used to implement other functions */ void *ASN1_dup(i2d_of_void *i2d, d2i_of_void *d2i, char *x); #define ASN1_dup_of(type,i2d,d2i,x) \ - ((type *(*)(I2D_OF(type),D2I_OF(type),type *))openssl_fcast(ASN1_dup))(i2d,d2i,x) + ((type *)ASN1_dup((i2d_of_void *)(i2d), (d2i_of_void *)(d2i), (char *)(x))) #define ASN1_dup_of_const(type,i2d,d2i,x) \ - ((type *(*)(I2D_OF_const(type),D2I_OF(type),type *))openssl_fcast(ASN1_dup))(i2d,d2i,x) + ((type *)ASN1_dup((i2d_of_void *)(i2d), (d2i_of_void *)(d2i), (char *)(x))) void *ASN1_item_dup(const ASN1_ITEM *it, void *x); #ifndef OPENSSL_NO_FP_API void *ASN1_d2i_fp(void *(*xnew)(void), d2i_of_void *d2i, FILE *in, void **x); #define ASN1_d2i_fp_of(type,xnew,d2i,in,x) \ - ((type *(*)(type *(*)(void),D2I_OF(type),FILE *,type **))openssl_fcast(ASN1_d2i_fp))(xnew,d2i,in,x) + ((type *)ASN1_d2i_fp((void *(*)(void))(xnew), (d2i_of_void *)(d2i), (in), (void **)(x))) void *ASN1_item_d2i_fp(const ASN1_ITEM *it, FILE *in, void *x); int ASN1_i2d_fp(i2d_of_void *i2d,FILE *out,void *x); #define ASN1_i2d_fp_of(type,i2d,out,x) \ - ((int (*)(I2D_OF(type),FILE *,type *))openssl_fcast(ASN1_i2d_fp))(i2d,out,x) + (ASN1_i2d_fp((i2d_of_void *)(i2d), (out), (x))) #define ASN1_i2d_fp_of_const(type,i2d,out,x) \ - ((int (*)(I2D_OF_const(type),FILE *,type *))openssl_fcast(ASN1_i2d_fp))(i2d,out,x) + (ASN1_i2d_fp((i2d_of_void *)(i2d), (out), (x))) int ASN1_item_i2d_fp(const ASN1_ITEM *it, FILE *out, void *x); int ASN1_STRING_print_ex_fp(FILE *fp, ASN1_STRING *str, unsigned long flags); #endif @@ -928,13 +928,13 @@ #ifndef OPENSSL_NO_BIO void *ASN1_d2i_bio(void *(*xnew)(void), d2i_of_void *d2i, BIO *in, void **x); #define ASN1_d2i_bio_of(type,xnew,d2i,in,x) \ - ((type *(*)(type *(*)(void),D2I_OF(type),BIO *,type **))openssl_fcast(ASN1_d2i_bio))(xnew,d2i,in,x) + ((type *)ASN1_d2i_bio( (void *(*)(void))(xnew), (d2i_of_void *)(d2i), (in), (void **)(x))) void *ASN1_item_d2i_bio(const ASN1_ITEM *it, BIO *in, void *x); int ASN1_i2d_bio(i2d_of_void *i2d,BIO *out, unsigned char *x); #define ASN1_i2d_bio_of(type,i2d,out,x) \ - ((int (*)(I2D_OF(type),BIO *,type *))openssl_fcast(ASN1_i2d_bio))(i2d,out,x) + (ASN1_i2d_bio((i2d_of_void *)(i2d), (out), (void *)(x))) #define ASN1_i2d_bio_of_const(type,i2d,out,x) \ - ((int (*)(I2D_OF_const(type),BIO *,const type *))openssl_fcast(ASN1_i2d_bio))(i2d,out,x) + (ASN1_i2d_bio((i2d_of_void *)(i2d), (out), (void *)(x))) int ASN1_item_i2d_bio(const ASN1_ITEM *it, BIO *out, void *x); int ASN1_UTCTIME_print(BIO *fp,ASN1_UTCTIME *a); int ASN1_GENERALIZEDTIME_print(BIO *fp,ASN1_GENERALIZEDTIME *a); @@ -978,7 +978,7 @@ ASN1_STRING *ASN1_pack_string(void *obj, i2d_of_void *i2d, ASN1_OCTET_STRING **oct); #define ASN1_pack_string_of(type,obj,i2d,oct) \ - ((ASN1_STRING *(*)(type *,I2D_OF(type),ASN1_OCTET_STRING **))openssl_fcast(ASN1_pack_string))(obj,i2d,oct) + (ASN1_pack_string((obj), (i2d_of_void *)(i2d), (oct))) ASN1_STRING *ASN1_item_pack(void *obj, const ASN1_ITEM *it, ASN1_OCTET_STRING **oct); void ASN1_STRING_set_default_mask(unsigned long mask); Index: openssl/crypto/ocsp/ocsp.h === RCS file: /home/ncvs/src/crypto/openssl/crypto/ocsp/ocsp.h,v retrieving revision 1.1.1.2 diff -u -r1.1.1.2 ocsp.h --- openssl/crypto/ocsp/ocsp.h 29 Jul 2006 19:10:18 - 1.1.1.2 +++ openssl/crypto/ocsp/ocsp.h 20 May 2007 05:13:06 - @@ -469,7 +469,7 @@ ASN1_STRING *ASN1_STRING_encode(ASN1_STRING *s, i2d_of_void *i2d, void *data, STACK_OF(ASN1_OBJECT) *sk); #define ASN1_STRING_encode_of(type,s,i2d,data,sk) \ -((ASN1_STRING *(*)(ASN1_STRING *,I2D_OF(type),type *,STACK_OF(ASN1_OBJECT) *))openssl_fcast(ASN1_STRING_encode))(s,i2d,data,sk) +(ASN1_STRING_encode((s), (i2d_of_void *)(i2d), (data), (STACK_OF(ASN1_OBJECT) *)(sk
Re: HEADS UP: GCC 4.2.0 is coming
On Fri, 18 May 2007 19:20:07 -0400 Alexander Kabaev [EMAIL PROTECTED] wrote: HEADS UP: I will start importing GCC 4.2.0 bits in about one hour and plan to finish in a couple of hours after that. The src/ tree will be utterly broken meanwhile. I'll send an 'all clear' message when done. Done. -- Alexander Kabaev signature.asc Description: PGP signature
Re: HEADS UP: GCC 4.2.0 is coming
On Sat, 19 May 2007 09:31:51 +0200 Jeremie Le Hen [EMAIL PROTECTED] wrote: On Sat, May 19, 2007 at 02:20:16AM -0400, Alexander Kabaev wrote: On Fri, 18 May 2007 19:20:07 -0400 Alexander Kabaev [EMAIL PROTECTED] wrote: HEADS UP: I will start importing GCC 4.2.0 bits in about one hour and plan to finish in a couple of hours after that. The src/ tree will be utterly broken meanwhile. I'll send an 'all clear' message when done. Done. Many thanks for this work, this is invaluable! Best regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org NOTE: At this time upgrades from pre-symver enabled systems appear to be broken. You need to build and install world for any time past 2007-05-13 14:12:41 UTC and only then jump to 4.2. The fix will be committed tomorrow. -- Alexander Kabaev signature.asc Description: PGP signature
HEADS UP: GCC 4.2.0 is coming
HEADS UP: I will start importing GCC 4.2.0 bits in about one hour and plan to finish in a couple of hours after that. The src/ tree will be utterly broken meanwhile. I'll send an 'all clear' message when done. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Thread local storage not working with -fPIC and shared objects
On Sat, 31 Mar 2007 04:26:23 +0200 Pieter de Goeje [EMAIL PROTECTED] wrote: Hello List, I have these files: --- loader.cpp --- #include stdio.h #include tls.h int main() { tls = 0; printf(%d\n, tls); } --- tls.cpp --- #include tls.h int __thread tls; --- tls.h --- extern __thread int tls; When I compile them like this: c++ -fPIC -o tls.so tls.cpp -shared c++ -fPIC -o loader loader.cpp tls.so And run the resulting program, I get: [EMAIL PROTECTED]:~/projects/misc/tls ./loader /libexec/ld-elf.so.1: ./loader: Unsupported relocation type 37 in non-PLT relocations When I omit -fPIC, it runs fine. But I need fPIC for the shared object on amd64 arch. I've tried it on Linux/i386 (gcc 4.1) and it ran fine (with fPIC). Much to my surprise however, a particularly large application I'm working on did compile run on FreeBSD/amd64 using -fpic (lowercase) and gcc 4.3. Trying -fpic on FreeBSD/i386 resulted in failure. FYI, I need tls to work because I'm using OpenMP's tls (#pragma omp threadprivate()) support in gcc 4.3. The workaround I found on FreeBSD/amd64 was linking the main executable with -fno-PIC, or building everything with -fpic. (both workarounds didn't work on FreeBSD/i386) I would be grateful if someone could shed some light on this. There is no reason whatsoever to compile main binary code with -f[pP]ic. Executables are not shared libraries and stuffing position independent code into them makes no sense. -- Alexander Kabaev signature.asc Description: PGP signature
Re: some symbols of libc may be resolved by RTLD internal entities.
On Tue, 30 Jan 2007 18:00:46 +0900 [EMAIL PROTECTED] wrote: Hi, It seems that some symbols in libc is resolved by libc entities which is linked with RTLD to implement it. % nm -D ld-elf.so.1 ... 000158ec T mmap c4fc W mprotect c4dc W munmap ... It doesn't. rtld is a special beast and its symbols availability to user programs is controlled by a special code in rtld. Look up static func_ptr_type exports[] in rtld.c and see how it is used. -- Alexander Kabaev P.S. The proper way to control symbol visibility is to use version script file when linking rtld-elf.so.1 in order to force all unintended symbols to local scope. exports array is there for historical reasons. signature.asc Description: PGP signature
Re: unresolved symbol for C++ class dtor
On Wed, 27 Dec 2006 18:09:41 +0100 Gergely CZUCZY [EMAIL PROTECTED] wrote: SKIP original email Executables only export symbols required by shared libraries known at link time by default. You want --export-dynamic on linker command line or ether -rdynamic or -Wl,--export-dynamic on CC command line. -- Alexander Kabaev signature.asc Description: PGP signature
Re: Tools for FreeBSD development
On Sat, 2 Dec 2006 18:28:57 -0500 Vishal Patil [EMAIL PROTECTED] wrote: I have recently moved over from Linux to FreeBSD and would like to if there is something similar to UML (User Mode Linux) for doing kernel development for FreeBSD. Reading different mailing lists, wikis etc it seems that qemu seems to be the best option. Is this tool used by most of the FreeBSD developers? Thanks. I personally think that having a dedicated box in disk-less configuration is the best option out there. The ability to quickly go through series of hands/reboots without any associated fsck runs and without the risk of terminally damaging any local FS is priceless. If qemu can be tricked into disk-less booting, it should be just as good though. -- Alexander Kabaev signature.asc Description: PGP signature
Re: gdb able to debug both fbsd and linux binaries
On Sun, 9 Jul 2006 11:46:53 +0200 Divacky Roman [EMAIL PROTECTED] wrote: hi, is it able to somehow make gdb be able to debug fbsd and linux binaries at the same time? I mean.. alter somehow the way its built in buildworld or something thnx roman -- www.liberalnistrana.cz Just use Linux gdb to debug Linux binaries. Unless something broke recently, it should work fine. -- Alexander Kabaev signature.asc Description: PGP signature
Re: NVIDIA FreeBSD kernel feature requests
On Thu, Jun 29, 2006 at 09:32:42AM -0700, Kip Macy wrote: IIRC lack of per instance cdevs also limits Freebsd to one vmware instance. -Kip On 6/29/06, Oleksandr Tymoshenko [EMAIL PROTECTED] wrote: Christian Zander wrote: Hi all, # Task:implement mechanism to allow character drivers to maintain per-open instance data (e.g. like the Linux kernel's 'struct file *'). Motivation: allows per thread NVIDIA notification delivery; also reduces CPU overhead for notification delivery from the NVIDIA kernel module to the X driver and to OpenGL. Priority:should translate to improved X/OpenGL performance. Status: has not been started. I've stumbled across this issue a while ago. Actually it can be partially solved using EVENTHANDLER_REGISTER of dev_clone event with keeping state structure in si_drv1 or si_drv2 fields. I'm not sure it's the best solution but it works for me though it smells like hack, and looks like hack :) Anyway, having legitimate per-open instance data structures of cdevs is a great assistance in porting linux drivers to FreeBSD. Just my $0.02. WHY it smells like a hack? It was designed precisely to do that. I am using cloned devices in our product with great success. Every client opening 'magic' device gets its own exclusive cloned device instance and everything works like a charm. I am yet to hear any single coherent description of what Linux's approach has over device cloning in FreeBSD. I wouldn't mind being educated on this. -- Alexander Kabaev pgpQ6tQSpB5xt.pgp Description: PGP signature
Re: [patch] Re: dlopen() and dlclose() are not MT-safe? YES, esp. for libthr
On Fri, 24 Mar 2006 10:48:34 +0200 Kostik Belousov [EMAIL PROTECTED] wrote: I did understand the purpose of the thread mask code in libexec/rtld/rtld_lock.c, or, more precisely, the condition where this code works (for the context, see the mails with same subject on freebsd-hackers). Look, that code assumes that blocking async signals would stop thread scheduler from doing preemption of the current thread. This works for libc_r, but fails in libpthread and libthr cases. libpthread provides implementation of the locks for rtld. But libthr does not ! As result, rtld exhibit races when used with libthr. In other words, libthr needs code to do proper locking. Do you agree ? Does somebody already planned to do this work ? Best regards, Kostik Belousov The thread mask only makes sense when flags are per-thread. I meant to use it to detect PLT recursions from locking primitives exported to rtld by the threads library as those are not allowed and threads implementations are required to take special care to provide only self-contained locks. The 'default' lock implementation will not work with any library other than libc_r, and even that holds true only for some definition of work. The dynamic loader never had a reliable locking and furthermore, there was no way to make it work better without threading library cooperation. This is why we came up with a set of callbacks rtld expects every threading library to provide. libpthread was the first where these callbacks were implemented. It comes as a surprise that libthr did not have them, because David Xu was the one who did most of the work on rtld locking callbacks in libpthread. The def_thread_set_flag function use is racy and should be fixed. -- Alexander Kabaev ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why is our symbol lookup the way it is?
On Wed, Sep 07, 2005 at 02:06:44AM -0400, Joe Marcus Clarke wrote: This is something that's been bothering me for a while, ever since I fixed the symbol conflicts in Mozilla with -Bsymbolic. Why do we not look in the referencing object first by default? I'm referring to the great comments in the symlook_default() function in rtld.c. We only check the referencing object first when -Bsymbolic is passed to the linker. Number of reasons. Programs should be able to override symbols from dynamim libraries, for instance. C++ exceptions won't work with -Bsymbolic when exceptions are thrown across shared library boundaries, as thrower and hander will use their own typeinfo structures and the catch clause in handler block will simply not recognize the exception, etc. The main reason I'm asking is that I just tracked down another symbol conflict in a dlopen'd library that plagues us but not Linux. I assume there was a good reason for resolving symbols in the order we do, but admittedly, I don't understand enough of the linker to know what it is. Thanks. I would suggest providing an exect scenario that is biting you. I fixed most incompatibilities in symbol lookup order between us and Linux for when Martin Blapp was working on initial OpenOffice port and if there is another, I would like to know about it. -- Alexander Kabaev ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: mutual exclusion in vkbd
On Tue, May 31, 2005 at 09:41:18AM -0700, Maksim Yevmenkin wrote: Norbert, I am currently trying to backport vkbd to FreeBSD 4. ok Maksim Yevmenkin uses mtx_lock()/mtx_unlock() for protecting access to data structures under FreeBSD 5/6 between the device functions and the kernel thread. How should I best do this under FreeBSD 4? Would something like splhigh() work in that context? Or should I use lockmgr with LK_EXCLUSIVE/LK_RELEASE? Is there any (pseudo)process context inside a kernel task? spltty() is what you probably need to use. you could just adjust the following defines like #define VKBD_LOCK_DECLint #define VKBD_LOCK_INIT(s) /* noop */ #define VKBD_LOCK_DESTROY(s) /* noop */ #define VKBD_LOCK(s) (s)-ks_lock = spltty() #define VKBD_UNLOCK(s)splx((s)-ks_lock) #define VKBD_LOCK_ASSERT(s, w) #define VKBD_SLEEP(s, f, d, t) \ tsleep((s)-f, PCATCH | (PZERO + 1), d, t) The code above will probably crash the kernel in many spectacular and unpredictable ways. You will need to save interrupt flags locally to each VKBD_LOCK caller or they will end up restoring each other's flags. -- Alexander Kabaev ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 5.2 v/s FreeBSD 4.9 MFLOPS performance (gcc3.3.3v/sgcc2.9.5)
On Mon, 16 Feb 2004 16:36:35 -0500 Juan Tumani [EMAIL PROTECTED] wrote: Thanks Matt for picking up on the linker problem. Patching the kernel would, to me, be masking the real problem. What other improvements does gcc333 have over gcc295 that might explain why it's linked products run in a half-fast mode (take twice+ as long)? JT From: Matthew Dillon [EMAIL PROTECTED] To: Juan Tumani [EMAIL PROTECTED] CC: [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: FreeBSD 5.2 v/s FreeBSD 4.9 MFLOPS performance (gcc3.3.3v/sgcc2.9.5) Date: Mon, 16 Feb 2004 13:12:15 -0800 (PST) I'm surprised Bruce hasn't chimed in here yet. I guess he's tired of repeating himself. In 4.9, libcsu, which generates crt1.o (which is the start code for C programs which the linker links in automatically) has this line in it: andl$~0xf, %%esp# align stack to 16-byte boundary So anything linked with 4.9 is going to align the stack on a 16 byte boundary no matter WHAT the kernel does. FreeBSD-5 does not have this alignment in its crt1.o because GCC3 automatically aligns the stack on a per-procedure basis. Or at least it is supposed to. Maybe it's broke? :-) -Matt Quite possibly. I run the same test using the latest GCC snapshot configured as system compiler and did not see such a massive slowdown. GCC 3.3.3 wins slightly on most tests and loses only on module #2 of the flops.c program posted here. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Subversion/CVS experiment summary
cvs-to-perforce scripts use DB files to keep an information on related commits while they scan CVS repo. I didn't try FreeBSD CVS, but whole SGI Linux tree with full history was processed quite effortlessly, without running out of memory. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Subversion/CVS experiment summary
On Mon, 09 Feb 2004 20:43:05 +0100 [EMAIL PROTECTED] (Dag-Erling Sm_rgrav) wrote: Alexander Kabaev [EMAIL PROTECTED] writes: cvs-to-perforce scripts use DB files to keep an information on related commits while they scan CVS repo. I didn't try FreeBSD CVS, but whole SGI Linux tree with full history was processed quite effortlessly, without running out of memory. Perforce uses CVS as backing store, so I expect matters are a little different than for SVN. There are two versions of cvs2p4 script and one of them populates repository using p4 commands. Either way, both versions require full scan of source repository to locate related changes and group them together. So matters are not that different for SVN. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Filesystem marker.
Look for ffsfind.c elsewhere on Internet. I used one when I incidentally relabeled a device from under a host on our SAN array. Had to modify it slightly to recognize superblocks UFS2 on FreeBSD side, but on a bright side, it worked pretty much unchanged on Solaris box. Just in any case, I saved my copy at http://people.freebsd.org/~kan/ffsfind.c On Wed, 14 Jan 2004 10:48:45 -0500 David Gilbert [EMAIL PROTECTED] wrote: Is there a set of bytes at some offset in a block that is common to any instance of a BSD ufs filesystem? I ask because recently my home machine erased it's fdisk block _and_ the bsdlabel with it. It certainly didn't have time to erase the whole disk, but I'm having trouble guessing where the partitions are. /usr/ports/sysutils/gpart will look for partitions on a disk ... but it only knows to look for bsd disklabels ... not bsd filesystems. Ideally, I'd like to make a bsd filesystem module for gpart with some pointers from the group. Dave. -- = ===|David Gilbert, Independent Contractor. | Two things can only be ||Mail: [EMAIL PROTECTED]| equal if and only if they ||http://daveg.ca | are precisely opposite. |=GLO ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: patch: portable dirhash
On Tue, 16 Dec 2003 22:12:08 -0500 (EST) Ted Unangst [EMAIL PROTECTED] wrote: can somebody please review/commit this to freebsd? it is most of the differences to permit openbsd to use the code. it should not change the code in any functional way. I do not think there is any point in this code ever hitting FreeBSD CVS repository. Rather, OpenBSD should just take cleaned-out copy of this code and be done with it. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: C++ code in a kernel module?
On Mon, 8 Sep 2003 11:35:37 -0600 John Giacomoni [EMAIL PROTECTED] wrote: I was planning on using the macro __cplusplus to toggle using extern C { }, however the bsd.kmod.mk style Makefiles seem to force the language to -std=c99 even when compiling with c++ . my initial steps have been as follows: take a functioning C based kernel module and rename to .cc added extern C around the includes. #defined key words such as new to xxx_new recompiled the new .cc file by hand without -std=c99, but keeping all the flags as the Makefile set them. then linked using the Makefile and finally loaded the module. -fno-rtti -fno-exceptions is probably a must unless you want to bring a whole libsupc++ library into the kernel. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: C++ code in a kernel module?
On Mon, 8 Sep 2003 23:02:33 -0400 Matthew Emmerton [EMAIL PROTECTED] wrote: I've been silently following this thread, and unless I missed something, has anyone asked John why he wants/needs to use C++ in the kernel? Tools, not policy :) -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dumping a core from inside of process
Look for abort() or SIGABRT. On 21 Aug 2003 21:57:41 + Artem 'Zazoobr' Ignatjev [EMAIL PROTECTED] wrote: Hello, hackers I'm writing some program, which dlopens() a lot of shared objects, and can do nasty things to it's own memory. Some day I decided to trap fatal memory signals, like SIGILL, SIGBUS and SIGSEGV, and wrote a handler for these, which swears with bad words into syslog, dlcloses() all that objects, and quits. But today I found that it's very useful - to have coredump handy, since its eases debug a lot. What is the (correct) way to make a coredump of your own memory (and, it'll be nice to have all that stack frames and registers written as they were when the signal did occured, not what they were when we are already in signal handler) -- Artem 'Zazoobr' Ignatjev [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Ok, IPC jailed.
Sure, send PRs over :) On Wed, 19 Feb 2003 19:43:19 +0100 Pawel Jakub Dawidek [EMAIL PROTECTED] wrote: Hello hackers... I prepared patches against 4.7-STABLE for secure IPC in jail. Main host and every jail has separated IPC memory zones, etc. I have added some sysctls and set-gid on group kmem for ipcs(1) is nomore needed. I could port those patches against 5.x if someone will review and commit it maybe. So, any volunteers?:) Patches are attached. -- Pawel Jakub Dawidek UNIX Systems Administrator http://garage.freebsd.pl Am I Evil? Yes, I Am. -- Alexander Kabaev To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: Ok, IPC jailed.
On Wed, 19 Feb 2003 19:45:52 +0100 Pawel Jakub Dawidek [EMAIL PROTECTED] wrote: On Wed, Feb 19, 2003 at 07:43:19PM +0100, Pawel Jakub Dawidek wrote: + Patches are attached. Now!:) Please resubmit your patch as PR to ensure it will not get lost. -- Alexander Kabaev To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: filesytem bsize=64k causes libc/db hash crash
On Thu, 12 Sep 2002 15:23:21 -0700 Terry Lambert [EMAIL PROTECTED] wrote: + if (b.psize 32768) +b.psize = 32868; 32768 vs. 32868 intentional here? -- Alexander Kabaev To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: gcc -O broken in CURRENT
This is a case of exception context register getting clobbered in shared library function call. GCC does not reload it when needed and this ultimately leads to semi-random word in program memory decremented by the __cp_pop_exception function. The bug is only triggered under very specific circumstances involving inline functions and nested degenerate exception handlers, that's why it existed unnoticed for quite some time. On Wed, 13 Mar 2002 22:53:48 -0800 Terry Lambert [EMAIL PROTECTED] wrote: M. Warner Losh wrote: In message: [EMAIL PROTECTED] Ed Hall [EMAIL PROTECTED] writes: : Exception-handling is broken with -O in -stable, and has been for years.: FreeBSD is one of the few systems that use setjmp/longjmp stack unwinds: to implement exceptions, so when the GCC folks broke that path, it was: never fixed. There are supposedly patches floating around that fix the: problem, but they either didn't work as advertised or the ball got dropped. H, C++ exceptions work in -stable with -O and have for at least a year. At least they are working for us in our environment. What's busted? Per thread exception stacks? THat's where I'd look... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: gcc -O broken in CURRENT
Do you have a patch for this ? I do not fully understand the parts of GCC involved, so I need some time to verify my initial diagnosis and to create a patch. In other words - not yet :) -- Alexander Kabaev To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: gcc -O broken in CURRENT
2) Bug is in os delivered gcc but not in port gcc. a) port has more or less patches / os gcc has been modified -- Didn't someone told they are the same? GCC from ports uses DWARF2 exception unwinding while GCC in src tree uses sjlj exceptions. The exception handling code generated by these two compilers is very different as a result. b) other options were set at compile time -- Why dont change to the same in the port? Leads it to a broken world? If the only difference is the lost of binary compatibility, i would say, ok... do it now and we'll need to compile or ports... Pretty much each and every C++ binary and shared library will have to be recompiled. Massive binary compatibility breakage is not an option for -STABLE, one can hope. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message