Re: Communication between kernel and userspace via local socket
On 11/17/11 11:40 AM, Maxim Ignatenko wrote: Julian Elischer wrote: On 11/16/11 12:55 AM, Ed Schouten wrote: * Maxim Ignatenkogelraen...@gmail.com, 2015 21:18: I'm currently inventing the wheel^W^W^Wwriting a firewall from scratch and looking for most convenient way to establish communication between userspace processes and kernel part. Communication pattern best fits to listening PF_LOCAL socket opened from kernel and userspace processes connecting to it. What's wrong with a character device? you can't easily have a different character device depending on which jail you are in.. (well, you can but it gets tricky).. see the problem with /dev/pflog and vimages. Maxim, look at the usage of sockets with netgraph ng_socket node.. also divert sockets. Did you meant ng_ksocket? I've looked on it, but in case of ng_ksocket connections accepted upon receiving control message NGM_KSOCKET_ACCEPT, but I need to accept connections without such punch. As far as I understand, I need to spawn kernel process or thread which will listen for incoming connections and respond to requests, just like normal network daemon does, but I don't know how to do this. divert(4) will not do the job, since packets written to divert socket goes to IP stack. No I meant ng_socket.. you wanted to communicate between userland and kernel. that ng_socket is the interface between kernel and userland for netgraph. I also meant, look at how the divert sockets create the sockets, not that you should use divert. ng_ksocket is something else. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Communication between kernel and userspace via local socket
On сб, 19 лис 2011 11:02:47 Julian Elischer wrote: On 11/17/11 11:40 AM, Maxim Ignatenko wrote: Julian Elischer wrote: On 11/16/11 12:55 AM, Ed Schouten wrote: * Maxim Ignatenkogelraen...@gmail.com, 2015 21:18: I'm currently inventing the wheel^W^W^Wwriting a firewall from scratch and looking for most convenient way to establish communication between userspace processes and kernel part. Communication pattern best fits to listening PF_LOCAL socket opened from kernel and userspace processes connecting to it. What's wrong with a character device? you can't easily have a different character device depending on which jail you are in.. (well, you can but it gets tricky).. see the problem with /dev/pflog and vimages. Maxim, look at the usage of sockets with netgraph ng_socket node.. also divert sockets. Did you meant ng_ksocket? I've looked on it, but in case of ng_ksocket connections accepted upon receiving control message NGM_KSOCKET_ACCEPT, but I need to accept connections without such punch. As far as I understand, I need to spawn kernel process or thread which will listen for incoming connections and respond to requests, just like normal network daemon does, but I don't know how to do this. divert(4) will not do the job, since packets written to divert socket goes to IP stack. No I meant ng_socket.. you wanted to communicate between userland and kernel. that ng_socket is the interface between kernel and userland for netgraph. Thanks! Creating new domain is, probably, overkill, but should work :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On Sat Nov 19 11, Gleb Kurtsou wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy does the issue also come up with '-fno-builtin' in addition to those flags? is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist when using -O1 and -O3? cheers. alex I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (19/11/2011 12:25), Alexander Best wrote: On Sat Nov 19 11, Gleb Kurtsou wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy does the issue also come up with '-fno-builtin' in addition to those flags? is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist when using -O1 and -O3? -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK So, it's -finline-functions that makes it work. But I'm afraid it just masks the real problem. The rest of CFLAGS is warnings and includes: -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 -Wno-error=strict-aliasing cheers. alex I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou gleb.kurt...@gmail.com wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. How small are the structs? gcc has an optimization for structs that are no larger than a register, but it's buggy in 4.2 and we disabled it at $WORK. I can dig up the patch if this is the problem. Thanks, matthew ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On (19/11/2011 07:26), m...@freebsd.org wrote: On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou gleb.kurt...@gmail.com wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. How small are the structs? gcc has an optimization for structs that are no larger than a register, but it's buggy in 4.2 and we disabled it at $WORK. I can dig up the patch if this is the problem. struct sockaddr_in in this particular test. 16 bytes. Register size structs are rather common, e.g. struct in_addr. I could test the patch. Adding -finline-functions seems to fix the issue for me. Thanks, Gleb. Thanks, matthew ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On Sat, Nov 19, 2011 at 8:19 AM, Gleb Kurtsou gleb.kurt...@gmail.com wrote: On (19/11/2011 07:26), m...@freebsd.org wrote: On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou gleb.kurt...@gmail.com wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. How small are the structs? gcc has an optimization for structs that are no larger than a register, but it's buggy in 4.2 and we disabled it at $WORK. I can dig up the patch if this is the problem. struct sockaddr_in in this particular test. 16 bytes. Register size structs are rather common, e.g. struct in_addr. I could test the patch. Adding -finline-functions seems to fix the issue for me. I can't find the thing I'm thinking of. The only potentially relevant patch I see in our gcc sources is this: Index: opts.c === --- opts.c (.../vendor.branches/freebsd/stable/7/src/contrib/gcc/opts.c) (revision 211574) +++ opts.c (.../head/src/contrib/gcc/opts.c) (revision 211574) @@ -457,11 +457,7 @@ flag_tree_dse = 1; flag_tree_ter = 1; flag_tree_live_range_split = 1; + /** + * 7dot1MERGE: tree-sra in gcc 4.2.x is buggy and + * breaks bitfield structs. + */ + flag_tree_sra = 0; - flag_tree_sra = 1; flag_tree_copyrename = 1; flag_tree_fre = 1; flag_tree_copy_prop = 1; Thanks, matthew ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
On Sat Nov 19 11, Gleb Kurtsou wrote: On (19/11/2011 12:25), Alexander Best wrote: On Sat Nov 19 11, Gleb Kurtsou wrote: Hi, I was lucky to write a bit of code which gcc 4.2 fails to compile correctly with -O2. Too keep long story short the code fails for gcc from base system and last gcc 4.2 snapshot from ports. It works with gcc 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and -Os optimization levels are fine (I've tried with all -f* flags mentioned in documentation) -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I presume i386 should be fine. These options are also used for compilation of kernel (with debugging enabled) and modules. I'm not able to share the code, but have a test case reproducing the bug. I've encountered the issue over a week ago and tried narrowing it down to a simple test I could share but without much success. The code itself is very common: initialize two structs on stack, call a function with pointers to those stucts as arguments. A number of inlined assertion functions. gcc fails to correctly optimize struct assignments with -fno-omit-frame-pointer, I have a number of small structs assigned, gcc decides not to use data coping but to assign fields directly. I've tried disabling sra, tweaking sra parameters -- no luck in forcing it to copy data. Replacing one particular assignment with memcpy produces correct code, but that's not a solution. -O2 -fno-omit-frame-pointer -fno-inline is buggy -O2 -fno-omit-frame-pointer -frename-registers is buggy does the issue also come up with '-fno-builtin' in addition to those flags? is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist when using -O1 and -O3? -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK btw: for any optimisation -O1, -fno-strict-aliasing isn't needed, since it gets added automatically (see sys/conf/kern.pre.mk). cheers. alex So, it's -finline-functions that makes it work. But I'm afraid it just masks the real problem. The rest of CFLAGS is warnings and includes: -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 -Wno-error=strict-aliasing cheers. alex I found similar issue with gcc 4.6, but I'm not able to reproduce it with gcc test case: https://bugzilla.redhat.com/show_bug.cgi?id=679924 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 I'll be glad to help debugging it and will be hanging on #bsddev during weekend as glk. Thanks, Gleb. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: easy way to determine if a stream or fd is seekable
On Nov 18, 2011, at 2:10 PM, Juergen Lock wrote: On Fri, Nov 18, 2011 at 12:00:07PM -0800, Tim Kientzle wrote: On Nov 17, 2011, at 12:55 PM, Juergen Lock wrote: After a few experiments, bsdtar stopped using lseek() on FreeBSD for anything other than regular files and block devices. Ah is that the reason why my patch never made it into FreeBSD 9? …. Cheers, Juergen (who would still like to see a faster tar tfv /dev/cd0... :) I would like to see that as well. Take a look at http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c Ah so it's `just' a slow release cycle? % grep DIOCGMEDIASIZE /home/ncvs/src/lib/libarchive/archive_read_open_filename.c,v % When will we see this code in FreeBSD? 10.0? 9.1? 8.3? :) Definitely in FreeBSD 10. Unfortunately libarchive 3.0 breaks the API and ABI, so it can't be imported into 9.1 or 8.3 wholesale. It should be possible to back port select pieces, though. Tim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: easy way to determine if a stream or fd is seekable
On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: On Fri Nov 18 11, Tim Kientzle wrote: Take a look at http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c Especially the comments about detecting disk-like devices. I rewrote a bunch of this code to introduce an explicit notion of strategy so that we could optimize access to a variety of different devices. This code has a notion of disk-like file descriptors and some optimizations for such. There are some comments in there outlining similar optimizations that could be made for tape-like or socket-like devices. great you posted that file as reference. i believe most of the stuff done there should actually be done within lseek(). Libarchive runs on a lot of systems other than FreeBSD. FreeBSD is not the only Unix-like system with this issue, so that code isn't going to go out of libarchive regardless. If you think those same ideas can be used in dd or hd to speed them up, please send your patches. The key point: You cannot unconditionally call lseek() to skip over data. Instead, treat lseek() as an optimization that can be used under some circumstances. The question then becomes one of figuring out when that optimization can be enabled. Tim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
