Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 5/11/12 1:17 PM, Bruce Evans wrote: However, they may be some some boot programs that save space by not bzeroing the bss. Not being a C program in this respect apparently caused the following farcal churn in i386/boot2: - the opts variable started with a bogus explicit initialization to 0. Long ago (in 2003), but probably after the default of -fzero-initialized-in-bss turned this into just a style bug, the explicit initializaton was removed. - zeroing of the bss was moved from i386/boot2/boot1.s ro rbx in 2004. It hopefully still worked. - in 2011, the kname variable was changed from char [1024] to a pointer. The pointer was explicitly initialized to NULL. This was not just a style bug, since the bss was apparently no longer zeroed. - initialization of the opts variable was apparently broken in the same way some time before this (whenever the zeroing was optimized away). - in the next commit in 2011, it was claimed that the BSS is not zeroed, and the initialization of kname was moved into main() to fix this. bss was spelled BSS, so my greps didn't find this immediately. - in a later commit in 2012, the initialization of opts was moved into main(), presumably for the same reason, but neither bss nor BSS was mentioned in the commit log, so this took longer to find. - these changes broke at least kname. Apparently, it can be initialized to non-null before main() in some cases. This was fixed in the most recent commit to boot2.c, in 2012, by moving the initialization of kname out of main(). opts was moved similary. The commit message doesn't mention bss or BSS, and says that this is to make clange build again. But boot2 is compiled without -fzero-initialized-in-bss, so initializing kname and opts outside of main() has no effect. They are now uninitialized again, assuming that the 2011 commit message is correct. Even the old char array for kname probably needed kname[0] to be zero. We apparently depend on most memory being 0 when booting, so that the chance of it being nonzero for these 2 variables is tiny. kname and opts are the only 2 global variables in i386/boot2 that need to be initialized to 0 in some way (the others either need to be initialized to non-0, so they never get put in the bss, or their initial value doesn't matter. After reducing kname to a pointer, these variables have total size 8. Perhaps the zeroing code would now take more than 8 bytes. However, in the 2004 version when the zeroing code was in boot1.S, it size seems to be 6 or 7 bytes (the code for this is smart). Thus the savings might be negative, but they are more likely to be a whole 4-8 bytes. There may be safer ways to save space. So I finally figured this out recently while looking at avg@'s recent patches. boot2's BSS is in fact zero'd by sys/boot/i386/btx/lib/btxcsu.S. This same code also zero's the BSS of /boot/loader. I have a patch to revert boot2 back to depending on the zero'd BSS in a tree, just need to commit it. -- John Baldwin ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
svn commit: r235267 - in head/usr.bin/sort: . nls
Author: gabor
Date: Fri May 11 12:37:16 2012
New Revision: 235267
URL: http://svn.freebsd.org/changeset/base/235267
Log:
Add a BSD-licensed sort rewrite that was started by me and later completed
with the major functionality and optimizations by Oleg Moskalenko.
It is compatible with the latest version of POSIX and the current GNU sort
version that we have in base. Beside this, it implements all the
functionality introduced in later versions of GNU sort. For now, it will
be installed as bsdsort, keeping GNU sort as the default sort
implementation.
Added:
head/usr.bin/sort/
head/usr.bin/sort/Makefile (contents, props changed)
head/usr.bin/sort/bwstring.c (contents, props changed)
head/usr.bin/sort/bwstring.h (contents, props changed)
head/usr.bin/sort/coll.c (contents, props changed)
head/usr.bin/sort/coll.h (contents, props changed)
head/usr.bin/sort/file.c (contents, props changed)
head/usr.bin/sort/file.h (contents, props changed)
head/usr.bin/sort/mem.c (contents, props changed)
head/usr.bin/sort/mem.h (contents, props changed)
head/usr.bin/sort/nls/
head/usr.bin/sort/nls/C.msg (contents, props changed)
head/usr.bin/sort/nls/hu_HU.ISO8859-2.msg (contents, props changed)
head/usr.bin/sort/radixsort.c (contents, props changed)
head/usr.bin/sort/radixsort.h (contents, props changed)
head/usr.bin/sort/sort.1.in (contents, props changed)
head/usr.bin/sort/sort.c (contents, props changed)
head/usr.bin/sort/sort.h (contents, props changed)
head/usr.bin/sort/vsort.c (contents, props changed)
head/usr.bin/sort/vsort.h (contents, props changed)
Added: head/usr.bin/sort/Makefile
==
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/usr.bin/sort/Makefile Fri May 11 12:37:16 2012(r235267)
@@ -0,0 +1,45 @@
+# $FreeBSD$
+
+.include bsd.own.mk
+
+.if ${MK_BSD_SORT} == yes
+PROG= sort
+.else
+PROG= bsdsort
+CLEANFILES+= bsdsort.1
+
+bsdsort.1: sort.1
+ cp ${.ALLSRC} ${.TARGET}
+.endif
+
+SRCS= bwstring.c coll.c file.c mem.c radixsort.c sort.c vsort.c
+
+WARNS= 6
+
+sort.1: sort.1.in
+ /usr/bin/sed ${MAN_SUB} ${.ALLSRC} ${.TARGET}
+
+CLEANFILES+= sort.1
+
+.if !defined(WITHOUT_THREADS)
+CFLAGS+= -DSORT_THREADS
+LDFLAGS+= -lpthread -lmd
+MAN_SUB+= -e 's|%%THREADS%%||g'
+.else
+LDFLAGS+= -lmd
+MAN_SUB+= -e 's|%%THREADS%%|\.\\|g'
+.endif
+
+.if !defined(WITHOUT_NLS)
+NLS+= hu_HU.ISO8859-2
+NLSSRCFILES= ${NLS:S@$@.msg@}
+MAN_SUB+= -e 's|%%NLS%%||g'
+.for lang in ${NLS}
+NLSSRCDIR_${lang}= ${.CURDIR}/nls
+.endfor
+.else
+CFLAGS+= -DWITHOUT_NLS
+MAN_SUB+= -e 's|%%THREADS%%|\.\\|g'
+.endif
+
+.include bsd.prog.mk
Added: head/usr.bin/sort/bwstring.c
==
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/usr.bin/sort/bwstring.cFri May 11 12:37:16 2012
(r235267)
@@ -0,0 +1,1138 @@
+/*-
+ * Copyright (C) 2009 Gabor Kovesdan ga...@freebsd.org
+ * Copyright (C) 2012 Oleg Moskalenko oleg.moskale...@citrix.com
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include sys/cdefs.h
+__FBSDID($FreeBSD$);
+
+#include ctype.h
+#include errno.h
+#include err.h
+#include langinfo.h
+#include math.h
+#include stdlib.h
+#include string.h
+#include wchar.h
+#include wctype.h
+
+#include bwstring.h
+#include sort.h
+
+bool byte_sort = false;
+
+static wchar_t **wmonths = NULL;
+static unsigned char **cmonths = NULL;
+
+/* initialise months */
+
+void
+initialise_months(void)
+{
+ const nl_item item[12] = { ABMON_1, ABMON_2, ABMON_3,
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: Author: gabor Date: Fri May 11 12:37:16 2012 New Revision: 235267 URL: http://svn.freebsd.org/changeset/base/235267 +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. pgp8tPZnUT7G9.pgp Description: PGP signature
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 11 May 2012, at 08:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: Author: gabor Date: Fri May 11 12:37:16 2012 New Revision: 235267 URL: http://svn.freebsd.org/changeset/base/235267 +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. Really? The C specification requires all globals and statics that are not explicitly initialised to be set to their zero value, so this initialisation has no effect on the resulting binary[1]. These are placed in the BSS section, irrespective of whether the initialisation is implicit or explicit and the loader is responsible for allocating space for them - all that is stored in the binary is the size. For local variables, initialisation like this has no effect even at low optimisation levels - dead stores will be removed. It does, however, make it more difficult for the compiler to distinguish between initialised and initialised sensibly in some cases. David [1] In a standards-compliant compiler. Apparently a few shipping compilers for embedded systems fail to respect this, but everything FreeBSD is compiled with does.___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 05/11/12 05:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. I just tested this hypothesis, and found no change in binary size using either clang or gcc46. Presumably they're smart enough to ignore explicit (and unnecessary) initializations of statics to zero. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 2012.05.11. 15:02, Colin Percival wrote: On 05/11/12 05:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. I just tested this hypothesis, and found no change in binary size using either clang or gcc46. Presumably they're smart enough to ignore explicit (and unnecessary) initializations of statics to zero. Thanks Colin and thanks Konstantin for raising this doubt. I will clean these up later for better style, anyway. Gabor ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On Fri, May 11, 2012 at 08:57:29AM -0400, David Chisnall wrote: On 11 May 2012, at 08:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: Author: gabor Date: Fri May 11 12:37:16 2012 New Revision: 235267 URL: http://svn.freebsd.org/changeset/base/235267 +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. Really? The C specification requires all globals and statics that are not explicitly initialised to be set to their zero value, so this initialisation has no effect on the resulting binary[1]. These are placed in the BSS section, irrespective of whether the initialisation is implicit or explicit and the loader is responsible for allocating space for them - all that is stored in the binary is the size. The initialized variables are placed in .data and not .bss. Apparently, some compilers do an optimiziation and put zero-initialized objects into .bss, as Colin noted, but this is not guaranteed behaviour. If placed in .data, they do consume disk space. Redundand initialization is not encouraged by style as well. For local variables, initialisation like this has no effect even at low optimisation levels - dead stores will be removed. It does, however, make it more difficult for the compiler to distinguish between initialised and initialised sensibly in some cases. local variables are irrelevant there. David [1] In a standards-compliant compiler. Apparently a few shipping compilers for embedded systems fail to respect this, but everything FreeBSD is compiled with does. pgpu5zwYzr0XY.pgp Description: PGP signature
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 2012-05-11 15:02, Colin Percival wrote: On 05/11/12 05:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. I just tested this hypothesis, and found no change in binary size using either clang or gcc46. Presumably they're smart enough to ignore explicit (and unnecessary) initializations of statics to zero. This is default behaviour, which can be toggled with gcc's (and clang's) command line option -fno-zero-initialized-in-bss: http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fno_002dzero_002dinitialized_002din_002dbss-744 ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On May 11, 2012, at 9:02 AM, Colin Percival wrote: On 05/11/12 05:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. I just tested this hypothesis, and found no change in binary size using either clang or gcc46. Presumably they're smart enough to ignore explicit (and unnecessary) initializations of statics to zero. How did you test this? size(1) or ls(1)? If ls, then you may be running into the page rounding of the .text and .data sections... Warner ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r235267 - in head/usr.bin/sort: . nls
On 05/11/12 13:08, Warner Losh wrote: On May 11, 2012, at 9:02 AM, Colin Percival wrote: On 05/11/12 05:48, Konstantin Belousov wrote: On Fri, May 11, 2012 at 12:37:16PM +, Gabor Kovesdan wrote: +bool byte_sort = false; + +static wchar_t **wmonths = NULL; +static unsigned char **cmonths = NULL; Such initializations are useless. You only increase the size of the binary on the disk as the consequence. I just tested this hypothesis, and found no change in binary size using either clang or gcc46. Presumably they're smart enough to ignore explicit (and unnecessary) initializations of statics to zero. How did you test this? size(1) or ls(1)? If ls, then you may be running into the page rounding of the .text and .data sections... size(1), ls(1)-before-strip(1), and ls(1)-after-strip(1). -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
