CVS commit: src/lib/libc/locale
Module Name:src
Committed By: riastradh
Date: Fri Jun 7 13:53:02 UTC 2024
Modified Files:
src/lib/libc/locale: global_locale.c
Log Message:
libc/locale/global_locale.c: Nix confusing NBCHAR_MAX.
NBCHAR_MAX is used only in the usr.bin/mklocale build, but that
doesn't use global_locale.c. This was added when inserting char
casts, but using the mklocale's hook just to insert a char cast is
unnecessarily confusing -- and the char casts are unnecessary anyway.
No functional change intended. No binary change to aarch64 libc.
Raised in PR lib/58319.
To generate a diff of this commit:
cvs rdiff -u -r1.25 -r1.26 src/lib/libc/locale/global_locale.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/lib/libc/locale/global_locale.c
diff -u src/lib/libc/locale/global_locale.c:1.25 src/lib/libc/locale/global_locale.c:1.26
--- src/lib/libc/locale/global_locale.c:1.25 Fri Apr 29 16:26:48 2016
+++ src/lib/libc/locale/global_locale.c Fri Jun 7 13:53:02 2024
@@ -1,4 +1,4 @@
-/* $NetBSD: global_locale.c,v 1.25 2016/04/29 16:26:48 joerg Exp $ */
+/* $NetBSD: global_locale.c,v 1.26 2024/06/07 13:53:02 riastradh Exp $ */
/*-
* Copyright (c)2008 Citrus Project,
@@ -28,7 +28,7 @@
#include
#if defined(LIBC_SCCS) && !defined(lint)
-__RCSID("$NetBSD: global_locale.c,v 1.25 2016/04/29 16:26:48 joerg Exp $");
+__RCSID("$NetBSD: global_locale.c,v 1.26 2024/06/07 13:53:02 riastradh Exp $");
#endif /* LIBC_SCCS and not lint */
#include
@@ -43,10 +43,6 @@ __RCSID("$NetBSD: global_locale.c,v 1.25
#include "runetype_local.h"
#include "setlocale_local.h"
-#ifndef NBCHAR_MAX
-#define NBCHAR_MAX (char)CHAR_MAX
-#endif
-
static const _MessagesLocale _DefaultMessagesLocale = {
"^[Yy]",
"^[Nn]",
@@ -124,20 +120,20 @@ __dso_hidden const struct _locale_cache_
.mon_grouping = __UNCONST(""),
.positive_sign = __UNCONST(""),
.negative_sign = __UNCONST(""),
- .int_frac_digits = NBCHAR_MAX,
- .frac_digits = NBCHAR_MAX,
- .p_cs_precedes = NBCHAR_MAX,
- .p_sep_by_space = NBCHAR_MAX,
- .n_cs_precedes = NBCHAR_MAX,
- .n_sep_by_space = NBCHAR_MAX,
- .p_sign_posn = NBCHAR_MAX,
- .n_sign_posn = NBCHAR_MAX,
- .int_p_cs_precedes = NBCHAR_MAX,
- .int_n_cs_precedes = NBCHAR_MAX,
- .int_p_sep_by_space = NBCHAR_MAX,
- .int_n_sep_by_space = NBCHAR_MAX,
- .int_p_sign_posn = NBCHAR_MAX,
- .int_n_sign_posn = NBCHAR_MAX,
+ .int_frac_digits = CHAR_MAX,
+ .frac_digits = CHAR_MAX,
+ .p_cs_precedes = CHAR_MAX,
+ .p_sep_by_space = CHAR_MAX,
+ .n_cs_precedes = CHAR_MAX,
+ .n_sep_by_space = CHAR_MAX,
+ .p_sign_posn = CHAR_MAX,
+ .n_sign_posn = CHAR_MAX,
+ .int_p_cs_precedes = CHAR_MAX,
+ .int_n_cs_precedes = CHAR_MAX,
+ .int_p_sep_by_space = CHAR_MAX,
+ .int_n_sep_by_space = CHAR_MAX,
+ .int_p_sign_posn = CHAR_MAX,
+ .int_n_sign_posn = CHAR_MAX,
},
.monetary_name = _lc_C_locale_name,
.numeric_name = _lc_C_locale_name,
CVS commit: src/lib/libc/locale
Module Name:src Committed By: riastradh Date: Fri Jun 7 13:53:02 UTC 2024 Modified Files: src/lib/libc/locale: global_locale.c Log Message: libc/locale/global_locale.c: Nix confusing NBCHAR_MAX. NBCHAR_MAX is used only in the usr.bin/mklocale build, but that doesn't use global_locale.c. This was added when inserting char casts, but using the mklocale's hook just to insert a char cast is unnecessarily confusing -- and the char casts are unnecessary anyway. No functional change intended. No binary change to aarch64 libc. Raised in PR lib/58319. To generate a diff of this commit: cvs rdiff -u -r1.25 -r1.26 src/lib/libc/locale/global_locale.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: src/lib/libc/locale
Module Name:src Committed By: rillig Date: Sat Feb 3 10:55:38 UTC 2024 Modified Files: src/lib/libc/locale: mblen.3 Log Message: mblen.3: clean up wording and grammar To generate a diff of this commit: cvs rdiff -u -r1.6 -r1.7 src/lib/libc/locale/mblen.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/lib/libc/locale/mblen.3 diff -u src/lib/libc/locale/mblen.3:1.6 src/lib/libc/locale/mblen.3:1.7 --- src/lib/libc/locale/mblen.3:1.6 Sat Oct 14 07:51:01 2006 +++ src/lib/libc/locale/mblen.3 Sat Feb 3 10:55:38 2024 @@ -1,4 +1,4 @@ -.\" $NetBSD: mblen.3,v 1.6 2006/10/14 07:51:01 wiz Exp $ +.\" $NetBSD: mblen.3,v 1.7 2024/02/03 10:55:38 rillig Exp $ .\" .\" Copyright (c)2002 Citrus Project, .\" All rights reserved. @@ -24,7 +24,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd February 3, 2002 +.Dd February 3, 2024 .Dt MBLEN 3 .Os .\" -- @@ -47,18 +47,20 @@ function usually determines the number o a multibyte character pointed to by .Fa s and returns it. -This function shall only examine max n bytes of the array beginning from +This function examines at most +.Fa n +bytes of the array beginning from .Fa s . .Pp In state-dependent encodings, .Fa s -may point the special sequence bytes to change the shift-state. +may point to the special sequence bytes to change the shift-state. Although such sequence bytes corresponds to no individual wide-character code, the .Fn mblen -changes the own state by them and treats them -as if they are a part of the subsequent multibyte character. +function changes its own state by them and treats them +as if they were part of the subsequent multibyte character. .Pp Unlike .Xr mbrlen 3 , @@ -106,8 +108,7 @@ determines whether the current encoding This function returns 0 if the encoding is state-independent, otherwise non-zero. .It "n == 0" -In this case, -the first +The first .Fa n bytes of the array pointed to by .Fa s @@ -128,11 +129,11 @@ points to a nul byte .Pq Sq \e0 . .It "positive" The value returned is -a number of bytes for the valid multibyte character pointed to by +the number of bytes for the valid multibyte character pointed to by .Fa s . -There are no cases that this value is greater than -.Fa n -or the value of the +The value returned is at most +.Fa n , +and at most the value of the .Dv MB_CUR_MAX macro. .It "-1" @@ -140,7 +141,7 @@ macro. points to an invalid or incomplete multibyte character. The .Fn mblen -also sets +function also sets .Va errno to indicate the error. .El @@ -151,7 +152,7 @@ is equal to .Dv NULL , the .Fn mblen -returns: +function returns: .Bl -tag -width 0123456789 .It "0" The current encoding is state-independent.
CVS commit: src/lib/libc/locale
Module Name:src Committed By: rillig Date: Sat Feb 3 10:55:38 UTC 2024 Modified Files: src/lib/libc/locale: mblen.3 Log Message: mblen.3: clean up wording and grammar To generate a diff of this commit: cvs rdiff -u -r1.6 -r1.7 src/lib/libc/locale/mblen.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: src/lib/libc/locale
Module Name:src
Committed By: mlelstv
Date: Sun Apr 16 20:37:59 UTC 2023
Modified Files:
src/lib/libc/locale: newlocale.c
Log Message:
Fix parsing a locale string with multiple components.
Also check for truncation of a long locale string.
To generate a diff of this commit:
cvs rdiff -u -r1.3 -r1.4 src/lib/libc/locale/newlocale.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/lib/libc/locale/newlocale.c
diff -u src/lib/libc/locale/newlocale.c:1.3 src/lib/libc/locale/newlocale.c:1.4
--- src/lib/libc/locale/newlocale.c:1.3 Fri Sep 13 13:13:32 2013
+++ src/lib/libc/locale/newlocale.c Sun Apr 16 20:37:59 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: newlocale.c,v 1.3 2013/09/13 13:13:32 joerg Exp $ */
+/* $NetBSD: newlocale.c,v 1.4 2023/04/16 20:37:59 mlelstv Exp $ */
/*-
* Copyright (c)2008, 2011 Citrus Project,
@@ -27,7 +27,7 @@
*/
#include
-__RCSID("$NetBSD: newlocale.c,v 1.3 2013/09/13 13:13:32 joerg Exp $");
+__RCSID("$NetBSD: newlocale.c,v 1.4 2023/04/16 20:37:59 mlelstv Exp $");
#include "namespace.h"
#include
@@ -57,7 +57,10 @@ newlocale(int mask, const char *name, lo
if (src == NULL)
src = _current_locale();
memcpy(dst, src, sizeof(*src));
- strlcpy([0], name, sizeof(head));
+ if (strlcpy([0], name, sizeof(head)) >= sizeof(head)) {
+ free(dst);
+ return (locale_t)NULL;
+ }
tokens[0] = (const char *)[0];
tail = strchr(tokens[0], '/');
if (tail == NULL) {
@@ -77,6 +80,7 @@ newlocale(int mask, const char *name, lo
}
if (howmany-- > 0) {
for (i = 1; i < howmany; ++i) {
+*tail++ = '\0';
tokens[i] = (const char *)tail;
tail = strchr(tokens[i], '/');
if (tail == NULL) {
@@ -84,6 +88,7 @@ newlocale(int mask, const char *name, lo
return NULL;
}
}
+ *tail++ = '\0';
tokens[howmany] = tail;
tail = strchr(tokens[howmany], '/');
if (tail != NULL) {
CVS commit: src/lib/libc/locale
Module Name:src Committed By: mlelstv Date: Sun Apr 16 20:37:59 UTC 2023 Modified Files: src/lib/libc/locale: newlocale.c Log Message: Fix parsing a locale string with multiple components. Also check for truncation of a long locale string. To generate a diff of this commit: cvs rdiff -u -r1.3 -r1.4 src/lib/libc/locale/newlocale.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: src/lib/libc/locale
Module Name:src Committed By: nia Date: Sat Dec 31 14:35:56 UTC 2022 Modified Files: src/lib/libc/locale: setlocale.3 Log Message: setlocale.3: Reflect state of NetBSD locale support in the 21st century Don't use LC_COLLATE in a real code example, since a real call to set LC_COLLATE will fail. Mention this. Inconsistencies noticed by khw on IRC. To generate a diff of this commit: cvs rdiff -u -r1.22 -r1.23 src/lib/libc/locale/setlocale.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/lib/libc/locale/setlocale.3 diff -u src/lib/libc/locale/setlocale.3:1.22 src/lib/libc/locale/setlocale.3:1.23 --- src/lib/libc/locale/setlocale.3:1.22 Sun Oct 24 17:30:19 2021 +++ src/lib/libc/locale/setlocale.3 Sat Dec 31 14:35:56 2022 @@ -1,4 +1,4 @@ -.\" $NetBSD: setlocale.3,v 1.22 2021/10/24 17:30:19 gutteridge Exp $ +.\" $NetBSD: setlocale.3,v 1.23 2022/12/31 14:35:56 nia Exp $ .\" .\" Copyright (c) 1993 .\" The Regents of the University of California. All rights reserved. @@ -32,7 +32,7 @@ .\" .\" @(#)setlocale.3 8.1 (Berkeley) 6/9/93 .\" -.Dd October 24, 2021 +.Dd December 31, 2022 .Dt SETLOCALE 3 .Os .Sh NAME @@ -314,6 +314,14 @@ which may be altered by later calls to .Fn setlocale or .Fn localeconv . +.Pp +Currently, +.Fn setlocale +returns +.Dv NULL +and fails to change the locale when +.Dv LC_COLLATE +is modified independently of other values. .Sh EXAMPLES The following code illustrates how a program can initialize the international environment for one language, while selectively @@ -322,7 +330,7 @@ string operations can be applied to text language: .Bd -literal setlocale(LC_ALL, "de"); - setlocale(LC_COLLATE, "fr"); + setlocale(LC_NUMERIC, "C"); .Ed .Pp When a process is started, its current locale is set to the C or POSIX @@ -379,21 +387,13 @@ and functions first appeared in .Bx 4.4 . .Sh BUGS -The current implementation supports only the -.Li "\&""C"" -and -.Li "\&""POSIX"" -locales for all but the -.Dv LC_CTYPE -locale. -.Pp In spite of the gnarly currency support in .Fn localeconv , the standards don't include any functions for generalized currency formatting. .Pp .Dv LC_COLLATE -does not make sense for many languages. +is unimplemented (but does not make sense for many languages). Use of .Dv LC_MONETARY could lead to misleading results until we have a real time currency
CVS commit: src/lib/libc/locale
Module Name:src Committed By: nia Date: Sat Dec 31 14:35:56 UTC 2022 Modified Files: src/lib/libc/locale: setlocale.3 Log Message: setlocale.3: Reflect state of NetBSD locale support in the 21st century Don't use LC_COLLATE in a real code example, since a real call to set LC_COLLATE will fail. Mention this. Inconsistencies noticed by khw on IRC. To generate a diff of this commit: cvs rdiff -u -r1.22 -r1.23 src/lib/libc/locale/setlocale.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
CVS commit: src/lib/libc/locale
Module Name:src Committed By: gutteridge Date: Sun Oct 24 17:30:19 UTC 2021 Modified Files: src/lib/libc/locale: setlocale.3 Log Message: setlocale.3: minor updates to reflect the current implementation Reflect a couple of amendments noted by John Marino on tech-userlevel: http://mail-index.netbsd.org/tech-userlevel/2021/10/19/msg013091.html To generate a diff of this commit: cvs rdiff -u -r1.21 -r1.22 src/lib/libc/locale/setlocale.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/lib/libc/locale/setlocale.3 diff -u src/lib/libc/locale/setlocale.3:1.21 src/lib/libc/locale/setlocale.3:1.22 --- src/lib/libc/locale/setlocale.3:1.21 Sat Jan 24 16:58:54 2004 +++ src/lib/libc/locale/setlocale.3 Sun Oct 24 17:30:19 2021 @@ -1,4 +1,4 @@ -.\" $NetBSD: setlocale.3,v 1.21 2004/01/24 16:58:54 wiz Exp $ +.\" $NetBSD: setlocale.3,v 1.22 2021/10/24 17:30:19 gutteridge Exp $ .\" .\" Copyright (c) 1993 .\" The Regents of the University of California. All rights reserved. @@ -32,7 +32,7 @@ .\" .\" @(#)setlocale.3 8.1 (Berkeley) 6/9/93 .\" -.Dd May 30, 2003 +.Dd October 24, 2021 .Dt SETLOCALE 3 .Os .Sh NAME @@ -81,9 +81,6 @@ functions. This controls recognition of upper and lower case, alphabetic or non-alphabetic characters, and so on. -The real work is done by the -.Fn setrunelocale -function. .It Dv LC_MESSAGES Set a locale for message catalogs. This controls the selection of message catalogs by the @@ -336,7 +333,9 @@ following manner before using any of the .Bd -literal setlocale(LC_ALL, ""); .Ed -.\" .Sh FILES XXX +.Sh FILES +The use of multibyte locales requires shared libraries located in +.Pa /usr/lib/i18n . .\" .Bl -tag -width /usr/share/locale/locale/category -compact XXX .\" .It Pa $PATH_LOCALE/\fIlocale\fP/\fIcategory\fP XXX .\" .It Pa /usr/share/locale/\fIlocale\fP/\fIcategory\fP XXX
CVS commit: src/lib/libc/locale
Module Name:src Committed By: gutteridge Date: Sun Oct 24 17:30:19 UTC 2021 Modified Files: src/lib/libc/locale: setlocale.3 Log Message: setlocale.3: minor updates to reflect the current implementation Reflect a couple of amendments noted by John Marino on tech-userlevel: http://mail-index.netbsd.org/tech-userlevel/2021/10/19/msg013091.html To generate a diff of this commit: cvs rdiff -u -r1.21 -r1.22 src/lib/libc/locale/setlocale.3 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
Re: CVS commit: src/lib/libc/locale
On Mon, Feb 15, 2021 at 04:37:15PM +0100, Thomas Klausner wrote: > Hi! > > Thanks for the man pages. > > There is none for uselocale(3), which is heavily referenced from > these, nor do I find a prototype in /usr/include... > so how does one use these? :) We don't support uselocale. You use this functions by passing the locale_t to the *_l functions directly. Joerg
Re: CVS commit: src/lib/libc/locale
Hi!
Thanks for the man pages.
There is none for uselocale(3), which is heavily referenced from
these, nor do I find a prototype in /usr/include...
so how does one use these? :)
Thomas
On Mon, Feb 15, 2021 at 09:35:04AM -0500, Christos Zoulas wrote:
> Module Name: src
> Committed By: christos
> Date: Mon Feb 15 14:35:04 UTC 2021
>
> Modified Files:
> src/lib/libc/locale: Makefile.inc
> Added Files:
> src/lib/libc/locale: duplocale.3 freelocale.3 newlocale.3
>
> Log Message:
> Add missing man pages (from FreeBSD)
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.64 -r1.65 src/lib/libc/locale/Makefile.inc
> cvs rdiff -u -r0 -r1.1 src/lib/libc/locale/duplocale.3 \
> src/lib/libc/locale/freelocale.3 src/lib/libc/locale/newlocale.3
>
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>
> Modified files:
>
> Index: src/lib/libc/locale/Makefile.inc
> diff -u src/lib/libc/locale/Makefile.inc:1.64
> src/lib/libc/locale/Makefile.inc:1.65
> --- src/lib/libc/locale/Makefile.inc:1.64 Sun Aug 18 16:03:48 2013
> +++ src/lib/libc/locale/Makefile.inc Mon Feb 15 09:35:04 2021
> @@ -1,5 +1,5 @@
> #from: @(#)Makefile.inc 5.1 (Berkeley) 2/18/91
> -#$NetBSD: Makefile.inc,v 1.64 2013/08/18 20:03:48 joerg Exp $
> +#$NetBSD: Makefile.inc,v 1.65 2021/02/15 14:35:04 christos Exp $
>
> # locale sources
> .PATH: ${ARCHDIR}/locale ${.CURDIR}/locale
> @@ -21,7 +21,7 @@ CPPFLAGS.runetable.c+= -I${LIBCDIR}/cit
> CPPFLAGS.multibyte_c90.c+= -I${LIBCDIR}/citrus
> CPPFLAGS.multibyte_amd1.c+= -I${LIBCDIR}/citrus
>
> -MAN+=setlocale.3 nl_langinfo.3
> +MAN+=duplocale.3 freelocale.3 newlocale.3 setlocale.3 nl_langinfo.3
>
> MAN+=mbtowc.3 mbstowcs.3 wctomb.3 wcstombs.3 mblen.3 \
>
>
> Added files:
>
> Index: src/lib/libc/locale/duplocale.3
> diff -u /dev/null src/lib/libc/locale/duplocale.3:1.1
> --- /dev/null Mon Feb 15 09:35:04 2021
> +++ src/lib/libc/locale/duplocale.3 Mon Feb 15 09:35:04 2021
> @@ -0,0 +1,80 @@
> +.\" $NetBSD: duplocale.3,v 1.1 2021/02/15 14:35:04 christos Exp $
> +.\" Copyright (c) 2011 The FreeBSD Foundation
> +.\" All rights reserved.
> +.\"
> +.\" This documentation was written by David Chisnall under sponsorship from
> +.\" the FreeBSD Foundation.
> +.\"
> +.\" Redistribution and use in source and binary forms, with or without
> +.\" modification, are permitted provided that the following conditions
> +.\" are met:
> +.\" 1. Redistributions of source code must retain the above copyright
> +.\"notice, this list of conditions and the following disclaimer.
> +.\" 2. Redistributions in binary form must reproduce the above copyright
> +.\"notice, this list of conditions and the following disclaimer in the
> +.\"documentation and/or other materials provided with the distribution.
> +.\"
> +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
> PURPOSE
> +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> CONSEQUENTIAL
> +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> STRICT
> +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> +.\" SUCH DAMAGE.
> +.\"
> +.\" $FreeBSD: head/lib/libc/locale/duplocale.3 281925 2015-04-24 10:17:55Z
> theraven $
> +.\"
> +.Dd September 17, 2011
> +.Dt DUPLOCALE 3
> +.Os
> +.Sh NAME
> +.Nm duplocale
> +.Nd duplicate an locale
> +.Sh LIBRARY
> +.Lb libc
> +.Sh SYNOPSIS
> +.In locale.h
> +.Ft locale_t
> +.Fn duplocale "locale_t locale"
> +.Sh DESCRIPTION
> +Duplicates an existing
> +.Fa locale_t
> +returning a new
> +.Fa locale_t
> +that refers to the same locale values but has an independent internal state.
> +Various functions, such as
> +.Xr mblen 3
> +require a persistent state.
> +These functions formerly used static variables and calls to them from
> multiple
> +threads had undefined behavior.
> +They now use fields in the
> +.Fa locale_t
> +associated with the current thread by
> +.Xr uselocale 3 .
> +These calls are therefore only thread safe on threads with a unique
> per-thread
> +locale.
> +The locale returned by this call must be freed with
> +.Xr freelocale 3 .
> +.Sh SEE ALSO
> +.Xr freelocale 3 ,
> +.Xr localeconv 3 ,
> +.Xr newlocale 3 ,
> +.\" .Xr querylocale 3 ,
> +.Xr uselocale 3 ,
> +.\" .Xr xlocale 3
> +.Sh STANDARDS
> +This function conforms to
> +.St -p1003.1-2008 .
> +.Sh BUGS
> +Ideally,
> +.Xr uselocale 3
> +should make a
Re: CVS commit: src/lib/libc/locale
On Mon, Apr 22, 2013 at 01:45:52AM +, YAMAMOTO Takashi wrote: On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. It adds more cost on the caller side. So far, there are three mechanisms available (especially for libraries): (1) Adhoc access to internal locale state. This is used with glibc. (2) Explicit newlocale(). (3) Implicit access via 0 argument to *_l. The first one is clearly a hack and not acceptable. Portable code can always conditionally use (2), but it requires additional setup and storage cost. (3) is used by Apple (which is where a large part of the *_l interface originates from) and FreeBSD. It is orthogonal to (2) and certainly easier to use. Exposing it via a special library call is also possible and effectively a way to implement (2) by a static wrapper. It still adds more cost to every caller and this is a classic case where there are typically much more callers. Joerg (3) adds small costs to every calls of *_l, even when the extension is not used. i sounds worse than a one-time cost of (2) to me. (2) still needs to load the address (instead of a constant), so it isn't free either. Given that this is very popular as functionality and at least on modern CPUs often implementable as conditional-move, it sounds like a much better trade off. very popular? i'm not aware of a single software which uses this extension. can you provide examples? The very popular specifically applies to the need for accessing the C locale. Most bigger libraries need it one way or another. I am currently talking with Apple about providing a visible macro (LC_C_LOCALE) to address the testability issue. Joerg
Re: CVS commit: src/lib/libc/locale
On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. It adds more cost on the caller side. So far, there are three mechanisms available (especially for libraries): (1) Adhoc access to internal locale state. This is used with glibc. (2) Explicit newlocale(). (3) Implicit access via 0 argument to *_l. The first one is clearly a hack and not acceptable. Portable code can always conditionally use (2), but it requires additional setup and storage cost. (3) is used by Apple (which is where a large part of the *_l interface originates from) and FreeBSD. It is orthogonal to (2) and certainly easier to use. Exposing it via a special library call is also possible and effectively a way to implement (2) by a static wrapper. It still adds more cost to every caller and this is a classic case where there are typically much more callers. Joerg (3) adds small costs to every calls of *_l, even when the extension is not used. i sounds worse than a one-time cost of (2) to me. (2) still needs to load the address (instead of a constant), so it isn't free either. Given that this is very popular as functionality and at least on modern CPUs often implementable as conditional-move, it sounds like a much better trade off. very popular? i'm not aware of a single software which uses this extension. can you provide examples? YAMAMOTO Takashi Joerg
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. It adds more cost on the caller side. So far, there are three mechanisms available (especially for libraries): (1) Adhoc access to internal locale state. This is used with glibc. (2) Explicit newlocale(). (3) Implicit access via 0 argument to *_l. The first one is clearly a hack and not acceptable. Portable code can always conditionally use (2), but it requires additional setup and storage cost. (3) is used by Apple (which is where a large part of the *_l interface originates from) and FreeBSD. It is orthogonal to (2) and certainly easier to use. Exposing it via a special library call is also possible and effectively a way to implement (2) by a static wrapper. It still adds more cost to every caller and this is a classic case where there are typically much more callers. Joerg (3) adds small costs to every calls of *_l, even when the extension is not used. i sounds worse than a one-time cost of (2) to me. YAMAMOTO Takashi
Re: CVS commit: src/lib/libc/locale
On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. It adds more cost on the caller side. So far, there are three mechanisms available (especially for libraries): (1) Adhoc access to internal locale state. This is used with glibc. (2) Explicit newlocale(). (3) Implicit access via 0 argument to *_l. The first one is clearly a hack and not acceptable. Portable code can always conditionally use (2), but it requires additional setup and storage cost. (3) is used by Apple (which is where a large part of the *_l interface originates from) and FreeBSD. It is orthogonal to (2) and certainly easier to use. Exposing it via a special library call is also possible and effectively a way to implement (2) by a static wrapper. It still adds more cost to every caller and this is a classic case where there are typically much more callers. Joerg (3) adds small costs to every calls of *_l, even when the extension is not used. i sounds worse than a one-time cost of (2) to me. (2) still needs to load the address (instead of a constant), so it isn't free either. Given that this is very popular as functionality and at least on modern CPUs often implementable as conditional-move, it sounds like a much better trade off. Joerg
Re: CVS commit: src/lib/libc/locale
On 2013-04-16 00:55, YAMAMOTO Takashi wrote: On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote: POSIX2008 spec say, *_l func with invalid locale handle may EINVAL. NULL or (locale_t)0 is invalid locale handle. Note that POSIX Issue 7 TC1 was recently published [1], and there are some relevant locale fixes. Specifically, [2] removes the may fail EINVAL error from *_l(), and adds the following paragraph: The behavior is undefined if the locale argument to *_l() is the special locale object LC_GLOBAL_LOCALE or is not a valid locale object handle. -Richard [1] http://article.gmane.org/gmane.comp.standards.posix.austin.general/7004 [2] http://austingroupbugs.net/view.php?id=283
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote: On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. It adds more cost on the caller side. So far, there are three mechanisms available (especially for libraries): (1) Adhoc access to internal locale state. This is used with glibc. (2) Explicit newlocale(). (3) Implicit access via 0 argument to *_l. The first one is clearly a hack and not acceptable. Portable code can always conditionally use (2), but it requires additional setup and storage cost. (3) is used by Apple (which is where a large part of the *_l interface originates from) and FreeBSD. It is orthogonal to (2) and certainly easier to use. Exposing it via a special library call is also possible and effectively a way to implement (2) by a static wrapper. It still adds more cost to every caller and this is a classic case where there are typically much more callers. Joerg
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 04:55:01AM +, YAMAMOTO Takashi wrote: hi, On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote: POSIX2008 spec say, *_l func with invalid locale handle may EINVAL. NULL or (locale_t)0 is invalid locale handle. why are you think fallback to C locale? That's what Apple and FreeBSD provide and which is actually quite useful. Wanting access to the C locale is the most common case for many locale issues. Making that easy sounds like a good idea in general. is there consumers of this extension in the field? Yes. an application can have a global c_locale = newlocale(C) if it wants an easy access to the C locale. ...which means keeping a copy around in various libraries etc. It is possible, but wastes memory. Joerg
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: before implementing such non-portable *broken* stuff. At the very least it is a conforming extension. ...which means keeping a copy around in various libraries etc. It is possible, but wastes memory. most of application in the world doesn't require multi-locale stuff. so that the struct _locale __C_locale in libc is much more wasteful. Most applications can/should enable the user locales. That means that as soon as they also have to deal with accessing data from non-internalized sources, they have to deal with both the user and the C locale. The very same reasoning applies to libraries. Joerg
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: That's what Apple and FreeBSD provide and which is actually quite useful. useful? don't say that. I have heard same word from who spreads broken iconv(3) usage, (such as non-portable //TRANSLIT features that annoying pkgsrc peoples). http://mail-index.netbsd.org/tech-userlevel/2006/04/03/.html //TRANSLIT is quite a different deal because it is a workaround for the bad default behavior of GNU iconv... Joerg
Re: CVS commit: src/lib/libc/locale
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote: so that the struct _locale __C_locale in libc is much more wasteful. I should add that it is an internal detail and the way the composed C locale is stored can and likely will change later. So the way it is essentially a copy of (old) global locale is just a way to be minimally intrusive. Joerg i care the API. if you really want it be in libc, how about having libc provide a locale_t get_static_c_locale(); style API rather than using NULL? it's better because 1) less code in *_l, 2) autoconf-like can detect the extension easily, and 3) a portable application can trivially have a fallback implementation using newlocale+pthread_once. YAMAMOTO Takashi
Re: CVS commit: src/lib/libc/locale
I can' t understand why this change is needed. POSIX2008 spec say, *_l func with invalid locale handle may EINVAL. NULL or (locale_t)0 is invalid locale handle. why are you think fallback to C locale? http://pubs.opengroup.org/onlinepubs/9699919799/functions/isalpha.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/nl_langinfo.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/newlocale.html glibc2(*_l function's origin) does more simpler, they don't check NULL and cause Segmentation Fault. I think glibc2 is reasonable, because POSIX2008 does change for the worse. as far as is*_l functions, return value is non-zero(=matches class) or zero(=not matches class) only. we can't know the error from it. 2013/4/15 Joerg Sonnenberger jo...@netbsd.org: Module Name:src Committed By: joerg Date: Sun Apr 14 23:44:54 UTC 2013 Modified Files: src/lib/libc/locale: Makefile.inc setlocale_local.h Added Files: src/lib/libc/locale: c_locale.c Log Message: Provide a const copy of global_locale for libc-internal use. This will be used by *_l when a NULL pointer is given. To generate a diff of this commit: cvs rdiff -u -r1.60 -r1.61 src/lib/libc/locale/Makefile.inc cvs rdiff -u -r0 -r1.1 src/lib/libc/locale/c_locale.c cvs rdiff -u -r1.10 -r1.11 src/lib/libc/locale/setlocale_local.h Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. -- Takehiko NOZAKItakehiko.noz...@gmail.com
Re: CVS commit: src/lib/libc/locale
On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote: POSIX2008 spec say, *_l func with invalid locale handle may EINVAL. NULL or (locale_t)0 is invalid locale handle. why are you think fallback to C locale? That's what Apple and FreeBSD provide and which is actually quite useful. Wanting access to the C locale is the most common case for many locale issues. Making that easy sounds like a good idea in general. Joerg
Re: CVS commit: src/lib/libc/locale
hi, On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote: POSIX2008 spec say, *_l func with invalid locale handle may EINVAL. NULL or (locale_t)0 is invalid locale handle. why are you think fallback to C locale? That's what Apple and FreeBSD provide and which is actually quite useful. Wanting access to the C locale is the most common case for many locale issues. Making that easy sounds like a good idea in general. is there consumers of this extension in the field? an application can have a global c_locale = newlocale(C) if it wants an easy access to the C locale. YAMAMOTO Takashi Joerg
Re: CVS commit: src/lib/libc/locale
I was just wondering why this change was made? It seems to be a backwards step to me? If we need exact-width interger we should use uintNN_t but I don't think exact width is required in locale. As other guys say we can't assume uint8_t == unsigned char implicitly: http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/arch/pdp10/include/Attic/int_types.h?rev=1.4 (though pdp10 has been removed) --- Izumi Tsutsui
Re: CVS commit: src/lib/libc/locale
On Sun, 06 Jun 2010, Izumi Tsutsui wrote: If we need exact-width interger we should use uintNN_t but I don't think exact width is required in locale. The [u]int_leastN_t types may be useful in this sort of code. For example, [u]int_least32_t is required to exist, and on a 36-bit machine it would presumably be a 36-bit type; whereas [u]int32_t would not exist on a 36-bit machine (because C99 requires the [u]intN_t types to have exact sizes with no padding bits). --apb (Alan Barrett)
Re: CVS commit: src/lib/libc/locale
hi, all
I was just wondering why this change was made? It seems to be a backwards
step to me?
my intension is quite paranoia.
changing uint8_t - unsigned char / int16_t - short is because
_ctype_ / _tolower_tab_ / _toupper_tab_ in ctype.h (now splitted
sys/ctype_bits.h)
declared as unsigned char / short.
ctype.h(rev1.29) -- http://tinyurl.com/365ycfq
extern const unsigned char *_ctype_;
extern const short *_tolower_tab_;
extern const short *_toupper_tab_;
but ctypeio.c read them from LC_CTYPE as uint8_t / int16_t and network endian
and directly cast to unsigned char/short.
ctypeio.c(rev1.11) -- http://tinyurl.com/22my4t8
if (fread((void *)ptr, sizeof(uint8_t), len, fp) != len)
...
if (fread((void *)ptr, sizeof(int16_t), len, fp) != len)
...
if (fread((void *)ptr, sizeof(int16_t), len, fp) != len)
...
#if BYTE_ORDER == LITTLE_ENDIAN
for (i = 1; i = len; i++) {
new_toupper[i] = ntohs(new_toupper[i]);
new_tolower[i] = ntohs(new_tolower[i]);
}
#endif
...
data-ctype_tab = (const unsigned char *)new_ctype;
data-toupper_tab = (const short *)new_toupper;
data-tolower_tab = (const short *)new_tolower;
if the case, someone port to the machine that sizeof(unsigned char) !=
sizeof(uint8_t)
or sizeof(short) != sizeof(int16_t), is*/to* function doesn't work correctlly.
# such as NetBSD/pdp10(9bit char), or cray1(64bit char)
If you ever port to a 36-bit machine, you are going to have to lie and
define int32_t as 'int' anyway
intN_t is not portable type, these are optimal:
ISO/IEC9899:1999 7.18.1.1
3 These types are optional. However, if an implementation provides
integer types with widths of 8, 16, 32, or 64 bits, no padding bits,
and (for the signed types) that have a two’s complement representation,
it shall define the corresponding typedef names.
NetBSD/pdp10 seems that providing them by using gcc extension
__attribute__((size N)).
The [u]int_leastN_t types may be useful in this sort of code.
exactly, but it require C99 stdint.h(or sys/types.h).
ctype.h should keep C90 namespace, i'm not willing to include it.
P.S.
i'm now attempting increase _ctype_ bits 8 - 16 before branching netbsd-6.
because these ancient code(delived from 4BSD) have two bugs:
1. _B bit desn't means isblank(3) but isprint(3) !isgraph(3).
so we have to provide real isblank bit, but there is no bit space
left.
2. isprint(3) doesn't recognize some japanese half-width character
and vietnamese's phonogram character, we have to provide
phonogram bit too.
i'll post patch to tech-userlevel@, this weekend or next week.
very truly yours.
--
Takehiko NOZAKI tnoz...@netbsd.org
Re: CVS commit: src/lib/libc/locale
Hi Nozaki-san, On Wed, Jun 02, 2010 at 04:04:52PM +, Takehiko NOZAKI wrote: Module Name: src Committed By: tnozaki Date: Wed Jun 2 16:04:52 UTC 2010 Modified Files: src/lib/libc/locale: bsdctype.c Log Message: uint8_t - unsigned char, int16_t - short. As ever, thank you for looking after our locale code. I was just wondering why this change was made? It seems to be a backwards step to me? Thanks, Alistair
Re: CVS commit: src/lib/libc/locale
On Sat, Jun 05, 2010 at 11:53:32PM +0200, Alistair Crooks wrote: Module Name: src Committed By: tnozaki Date: Wed Jun 2 16:04:52 UTC 2010 Modified Files: src/lib/libc/locale: bsdctype.c Log Message: uint8_t - unsigned char, int16_t - short. As ever, thank you for looking after our locale code. I was just wondering why this change was made? It seems to be a backwards step to me? I looked at it and it appears to me that it's a change to use non-sized types (that are guaranteed to be large enough) in the in-memory structures. If we ever do a port to a 36-bit machine or whatever it's probably desirable. -- David A. Holland dholl...@netbsd.org
Re: CVS commit: src/lib/libc/locale
In message: 20100606041254.gb19...@netbsd.org David Holland dholland-sourcechan...@netbsd.org writes: : On Sat, Jun 05, 2010 at 11:53:32PM +0200, Alistair Crooks wrote: :Module Name: src :Committed By:tnozaki :Date:Wed Jun 2 16:04:52 UTC 2010 : :Modified Files: : src/lib/libc/locale: bsdctype.c : :Log Message: :uint8_t - unsigned char, int16_t - short. : : As ever, thank you for looking after our locale code. : : I was just wondering why this change was made? It seems to be a backwards : step to me? : : I looked at it and it appears to me that it's a change to use : non-sized types (that are guaranteed to be large enough) in the : in-memory structures. If we ever do a port to a 36-bit machine or : whatever it's probably desirable. If you ever port to a 36-bit machine, you are going to have to lie and define int32_t as 'int' anyway Warner
Re: CVS commit: src/lib/libc/locale
On Sat, Jun 05, 2010 at 11:28:31PM -0600, M. Warner Losh wrote: : I looked at it and it appears to me that it's a change to use : non-sized types (that are guaranteed to be large enough) in the : in-memory structures. If we ever do a port to a 36-bit machine or : whatever it's probably desirable. If you ever port to a 36-bit machine, you are going to have to lie and define int32_t as 'int' anyway Well, maybe... it would be sort of interesting to try, actually. -- David A. Holland dholl...@netbsd.org
CVS commit: src/lib/libc/locale
Module Name:src Committed By: wiz Date: Sun Jan 17 23:12:30 UTC 2010 Modified Files: src/lib/libc/locale: ctypeio.c Log Message: Close file handles in error cases. Found by cppcheck. To generate a diff of this commit: cvs rdiff -u -r1.10 -r1.11 src/lib/libc/locale/ctypeio.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
