Re: Is --enable-utf8 working everywhere?
On Wed, Jul 17, 2002 at 05:39:52PM -0400, Jim Jagielski wrote: Ulrich Drepper wrote: On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() =20 I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? I don't think Ulrich is concerned with *correctness*. His comments are about resulting code size and performance. I think Ulrich hasn't been clear in that regard, but you (Jim) are also (seemingly) tending to be a bit obstinate in wanting to do it your way :-) Cheers, -g -- Greg Stein, http://www.lyra.org/
Re: Is --enable-utf8 working everywhere?
I just checked in a change to apr-util/xlate/xlate.c following Ulrich's recommend pattern of copy/test. If any APR developer has strong feelings for the *other* way, then it's on your head to fix it. I see no point to continuing this conversation, and a lot of reason to simply have a frickin' fix in the code. Cheers, -g On Wed, Jul 17, 2002 at 05:09:55PM -0700, Greg Stein wrote: On Wed, Jul 17, 2002 at 05:39:52PM -0400, Jim Jagielski wrote: Ulrich Drepper wrote: On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() =20 I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? I don't think Ulrich is concerned with *correctness*. His comments are about resulting code size and performance. I think Ulrich hasn't been clear in that regard, but you (Jim) are also (seemingly) tending to be a bit obstinate in wanting to do it your way :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Greg Stein, http://www.lyra.org/
Re: Is --enable-utf8 working everywhere?
Greg Stein wrote: Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? I don't think Ulrich is concerned with *correctness*. His comments are about resulting code size and performance. I think Ulrich hasn't been clear in that regard, but you (Jim) are also (seemingly) tending to be a bit obstinate in wanting to do it your way :-) No, Ulrich was *not* clear on what the problem was, and that's the reason why I was being so obstinate, not so much for my way but *why* he wanted/needed it the other way. Now that we all know what he was driving at, we're in better shape to address it. For my part, if the use of a temp variable to avoid too many errno calls results in measureable differences, then why not. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
On Wed, 17 Jul 2002, Greg Stein wrote: On Wed, Jul 17, 2002 at 05:39:52PM -0400, Jim Jagielski wrote: Ulrich Drepper wrote: On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() =20 I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? I don't think Ulrich is concerned with *correctness*. His comments are about resulting code size and performance. I think Ulrich hasn't been clear in that regard, but you (Jim) are also (seemingly) tending to be a bit obstinate in wanting to do it your way :-) Yeah, I definately didn't understand Ulrich's concerns until he specifically stated what he was trying to solve. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper wrote:
On Tue, 2002-07-16 at 15:48, Blair Zajac wrote:
As a real patch use
--- apr/i18n/unix/xlate.c Tue Apr 16 15:37:07 2002
+++ apr/i18n/unix/xlate.c-new Tue Jul 16 17:21:07 2002
@@ -255,7 +255,8 @@
if (!found) {
new-ich = iconv_open(topage, frompage);
if (new-ich == (iconv_t)-1) {
-return errno;
+int e = errno;
+return e ? e : EINVAL;
}
found = 1;
check_sbcs(new);
Karl, can you help out here and check in a fix for this?
We don't need to make a copy of errno. It's not clear to me which
error value we should pick. Is EINVAL appropriate when the OS
doesn't set errno when it should? Maybe we should use something
else that signifies we don't know what the real errno is. The
Solaris man pages for iconv_open() says it can return EMFILE,
ENFILE, ENOMEM and EINVAL.
diff -u -r1.28 xlate.c
--- i18n/unix/xlate.c 15 Jul 2002 19:21:01 - 1.28
+++ i18n/unix/xlate.c 17 Jul 2002 00:45:19 -
@@ -269,7 +269,7 @@
if (!found) {
new-ich = iconv_open(topage, frompage);
if (new-ich == (iconv_t)-1) {
-return errno;
+return errno ? errno : EINVAL;
}
found = 1;
check_sbcs(new);
Best,
Blair
--
Blair Zajac [EMAIL PROTECTED]
Web and OS performance plots - http://www.orcaware.com/orca/
Re: Is --enable-utf8 working everywhere?
Right. Here's the patch that adds support for apr_xlate_* on Win32. I'd
like to get some feedback and independent testing before I check this in.
For APR:
* cd to the top of the APR sources
* unzip apr-winiconv.zip (new dirs and files there)
* apply apr-winiconv.patch
For Apache:
* cd to the top of the httpd-2.0 sources
* apply httpd-winiconv.patch to Makefile.win (this is just so that
iconv.dll gets installed in Apache's bin directory).
To enable the apr_xlate_* stuff, drop the libiconv sources into
i18n/win32/libiconv (I'm testing this with libiconv-1.8), and rebuild
APR/httpd/whatever.
Karl, about the file name conversions in Subversion: remember what I
said about APR using UTF-8 file names directly on some platforms? Well,
the conversions in SVN should probably be coded like this:
char *utf8_filename = (something);
char *native_filename;
#ifdef WIN32
# if APR_HAS_UNICODE_FS
IF_WIN_OS_IS_UNICODE
{
native_filename = utf8_filename;
}
# endif
# if APR_HAS_ANSI_FS
ELSE_WIN_OS_IS_ANSI
{
native_filename = convert_utf8_to_locale_charset(utf8_filename);
}
# endif
#else /* !WIN32 */
native_filename = convert_utf8_to_locale_charset(utf8_filename);
#endif /* !WIN32 */
--
Brane ibej [EMAIL PROTECTED] http://www.xbc.nu/brane/
Index: apr.dsp
===
RCS file: /home/cvs/apr/apr.dsp,v
retrieving revision 1.107
diff -u -p -r1.107 apr.dsp
--- apr.dsp 11 Jul 2002 15:32:17 - 1.107
+++ apr.dsp 17 Jul 2002 00:46:12 -
@@ -41,7 +41,7 @@ RSC=rc.exe
# PROP Intermediate_Dir LibR
# PROP Target_Dir
# ADD BASE CPP /nologo /MD /W3 /O2 /D WIN32 /D NDEBUG /D _WINDOWS /FD /c
-# ADD CPP /nologo /MD /W3 /O2 /I ./include /I ./include/arch /I
./include/arch/win32 /I ./include/arch/unix /D NDEBUG /D
APR_DECLARE_STATIC /D WIN32 /D _WINDOWS /FdLibR\apr /FD /c
+# ADD CPP /nologo /MD /W3 /O2 /I ./include /I ./include/arch /I
./include/arch/win32 /I ./include/arch/unix /I
./i18n/win32/libiconv/include /D NDEBUG /D APR_DECLARE_STATIC /D WIN32
/D _WINDOWS /FdLibR\apr /FD /c
# ADD BASE RSC /l 0x409
# ADD RSC /l 0x409
BSC32=bscmake.exe
@@ -65,7 +65,7 @@ LIB32=link.exe -lib
# PROP Ignore_Export_Lib 0
# PROP Target_Dir
# ADD BASE CPP /nologo /MDd /W3 /GX /Zi /Od /D WIN32 /D _DEBUG /D
_WINDOWS /FD /c
-# ADD CPP /nologo /MDd /W3 /GX /Zi /Od /I ./include /I ./include/arch /I
./include/arch/win32 /I ./include/arch/unix /D _DEBUG /D
APR_DECLARE_STATIC /D WIN32 /D _WINDOWS /FdLibD\apr /FD /c
+# ADD CPP /nologo /MDd /W3 /GX /Zi /Od /I ./include /I ./include/arch /I
./include/arch/win32 /I ./include/arch/unix /I
./i18n/win32/libiconv/include /D _DEBUG /D APR_DECLARE_STATIC /D WIN32
/D _WINDOWS /FdLibD\apr /FD /c
# ADD BASE RSC /l 0x409
# ADD RSC /l 0x409
BSC32=bscmake.exe
@@ -157,6 +157,35 @@ SOURCE=.\file_io\win32\seek.c
# PROP Default_Filter
# Begin Source File
+SOURCE=.\i18n\win32\libiconv.c
+
+!IF $(CFG) == apr - Win32 Release
+
+# PROP Ignore_Default_Tool 1
+# Begin Custom Build - Building iconv.lib
+InputPath=.\i18n\win32\libiconv.c
+
+.\LibR\iconv.lib : $(SOURCE) $(INTDIR) $(OUTDIR)
+ call .\build\win32iconv.bat genlib librel
+
+# End Custom Build
+
+!ELSEIF $(CFG) == apr - Win32 Debug
+
+# PROP Ignore_Default_Tool 1
+# Begin Custom Build - Building iconv.lib
+InputPath=.\i18n\win32\libiconv.c
+
+.\LibD\iconv.lib : $(SOURCE) $(INTDIR) $(OUTDIR)
+ call .\build\win32iconv.bat genlib libdbg
+
+# End Custom Build
+
+!ENDIF
+
+# End Source File
+# Begin Source File
+
SOURCE=.\i18n\unix\utf8_ucs2.c
# End Source File
# Begin Source File
@@ -470,6 +499,7 @@ SOURCE=.\include\apr.hw
!IF $(CFG) == apr - Win32 Release
+USERDEP__APR_H=.\include\apr_config_iconv.h
# Begin Custom Build - Creating apr.h from apr.hw
InputPath=.\include\apr.hw
@@ -480,6 +510,7 @@ InputPath=.\include\apr.hw
!ELSEIF $(CFG) == apr - Win32 Debug
+USERDEP__APR_H=.\include\apr_config_iconv.h
# Begin Custom Build - Creating apr.h from apr.hw
InputPath=.\include\apr.hw
@@ -494,6 +525,33 @@ InputPath=.\include\apr.hw
# Begin Source File
SOURCE=.\include\apr_compat.h
+# End Source File
+# Begin Source File
+
+SOURCE=.\include\apr_config_iconv.hw
+
+!IF $(CFG) == apr - Win32 Release
+
+# Begin Custom Build - Creating apr_config_iconv.h from apr_config_iconv.hw
+InputPath=.\include\apr_config_iconv.hw
+
+.\include\apr_config_iconv.h : $(SOURCE) $(INTDIR) $(OUTDIR)
+ call .\build\win32iconv.bat genhdr
+
+# End Custom Build
+
+!ELSEIF $(CFG) == apr - Win32 Debug
+
+# Begin Custom Build - Creating apr_config_iconv.h from apr_config_iconv.hw
+InputPath=.\include\apr_config_iconv.hw
+
+.\include\apr_config_iconv.h : $(SOURCE) $(INTDIR) $(OUTDIR)
+ call .\build\win32iconv.bat genhdr
+
+# End Custom Build
+
+!ENDIF
+
# End Source File
# Begin Source File
Index: libapr.dsp
===
RCS file:
Re: Is --enable-utf8 working everywhere?
At 08:50 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: Right. Here's the patch that adds support for apr_xlate_* on Win32. I'd like to get some feedback and independent testing before I check this in. Can we start from the top here? What iconv sources are you planning to build? From where? The apr-iconv port in-progress? Am I misinterpreting that this is a dummy module? Finally, it's libapriconv or libiconv. I'm reviewing the code now. But I am confused. Please enlighten me before you go checking this in... thanks, Bill
Re: Is --enable-utf8 working everywhere?
William A. Rowe, Jr. wrote: At 08:50 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: Right. Here's the patch that adds support for apr_xlate_* on Win32. I'd like to get some feedback and independent testing before I check this in. Can we start from the top here? What iconv sources are you planning to build? From where? I did say I was using the libiconv-1.8 sources. It's from www.gnu.org. The apr-iconv port in-progress? Not yet. Am I misinterpreting that this is a dummy module? Yes and no. If you don't have the libiconv sources, it creates a dummy (empty) iconv.lib and doesn't define APR_HAS_XLATE, etc. That's because of MSVC project file idiosyncracies. However, if you drop the libiconv sources into i18n/win32/libiconv, build/win32iconv.bat will a) modify apr_config_iconv.h so that APR_HAS_XLATE, HAVE_ICONV_H and HAVE_ICONV are defined in the right places, and the _real_ iconv.lib/dll gets built and linked in. Yes, I know that smells of hackishness, but it's the best I could come up with on short notice without special tools (configure, etc) available. And we really need this support in Subversion ASAP, it has to go into our Alpha. Finally, it's libapriconv or libiconv. Hm? I don't understand. What is libapriconv or libiconv? I'm reviewing the code now. But I am confused. Please enlighten me before you go checking this in... thanks, Hope this is enlightening enough. -- Brane ibej [EMAIL PROTECTED] http://www.xbc.nu/brane/
Re: Is --enable-utf8 working everywhere?
At 09:37 PM 7/16/2002, Brane wrote: Can we start from the top here? What iconv sources are you planning to build? From where? I did say I was using the libiconv-1.8 sources. It's from www.gnu.org. Ok, one to three problems here. First, apr-iconv is from BSD - it's not under gnu license. Are we planning to distribute it with sources? As binaries? This is dripping into muddy waters. And this with a project that claims the ASL isn't even a compatible license. But it becomes hugely mucky waters if you start building our library with their library. They are pretty clear (assuming this is LGPL) that other apps can link to it. But other libraries? Second, we're invested in apr-iconv, most of the porting effort was done... The apr-iconv port in-progress? Not yet. Sad that we aren't investing the effort here. But there was the .dll tangle, apr-iconv needing apr, but apr being built upon iconv. The only thing that needed it, according to my grep of apr_xlate*, was md5. That just moved. iconv is next. Expect minor breakage for just a bit. Then we can include apr-iconv between building apr and building apr-util. In fact, my preference is to build apr-iconv as apr-util/xlate/iconv/... much like /xml/expat. I don't intend to break the ability to link to an installed local iconv. This is a -supplement- for those who don't have an iconv handy. Am I misinterpreting that this is a dummy module? Yes and no. If you don't have the libiconv sources, it creates a dummy (empty) iconv.lib and doesn't define APR_HAS_XLATE, etc. That's because of MSVC project file idiosyncracies. I'm not too happy about them. I just finished changing over a number of functions, xlate amoung them, to always export the symbols, and then correctly return ENOTIMPL. Why the extra mad effort here? It's an on/off switch. However, if you drop the libiconv sources into i18n/win32/libiconv, build/win32iconv.bat will a) modify apr_config_iconv.h so that APR_HAS_XLATE, HAVE_ICONV_H and HAVE_ICONV are defined in the right places, and the _real_ iconv.lib/dll gets built and linked in. Yes, I know that smells of hackishness, but it's the best I could come up with on short notice without special tools (configure, etc) available. And we really need this support in Subversion ASAP, it has to go into our Alpha. Fine, then lets take the time to get this right. I'm willing to help, I simply asked for breathing room till Wed. Cool. I had to get this support finished anyways. Finally, it's libapriconv or libiconv. Hm? I don't understand. What is libapriconv or libiconv? libiconv.dll - never iconv.dll please. We are trying to stay somewhat in-sync with some unix conventions here so folks don't get as confused. I'm reviewing the code now. But I am confused. Please enlighten me before you go checking this in... thanks, Hope this is enlightening enough. Very, thanks. I'm 100% behind requiring win32 users to download apr-iconv as part of their apr build. Be done with the custom tweakish build stuff already, this is 2002. We need xlate :-) Bill
Re: Is --enable-utf8 working everywhere?
Blair Zajac [EMAIL PROTECTED] writes: Karl, can you help out here and check in a fix for this? Hey, Blair, just an `ACK'. Thanks for the work; I'm following this thread as a background process, will take a closer look as soon as I'm done with issue #797, which is top priority right now (as it can lead to data corruption). Fortunately I'm not the only Subversion developer who's also an APR developer :-), so this doesn't need to bottleneck on me. And I see it's not, judging by the thread so far, good... -K
Re: Is --enable-utf8 working everywhere?
On Tue, 2002-07-16 at 15:48, Blair Zajac wrote:
As a real patch use
--- apr/i18n/unix/xlate.c Tue Apr 16 15:37:07 2002
+++ apr/i18n/unix/xlate.c-new Tue Jul 16 17:21:07 2002
@@ -255,7 +255,8 @@
if (!found) {
new-ich = iconv_open(topage, frompage);
if (new-ich == (iconv_t)-1) {
-return errno;
+int e = errno;
+return e ? e : EINVAL;
}
found = 1;
check_sbcs(new);
--
---. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `
signature.asc
Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
Branko ibej [EMAIL PROTECTED] writes:
Karl, about the file name conversions in Subversion: remember what I
said about APR using UTF-8 file names directly on some platforms?
Well, the conversions in SVN should probably be coded like this:
Yeah, remember that. But does the platform-specific conditional code
have to go in Subversion? I understand that APR rightly shouldn't do
any conversion here, since the output would be the same as the input,
but perhaps APR could somehow signal this fact back? (The fact that
no conversion is necessary, that is). After all, it is our
portability layer... Then Subversion, on receiving this signal, could
simply dup the input in the right pool and return it, without having
any platform-specific tests in its own code.
(Btw we *do* have to dup into the right pool, can't just assign back,
as Marcus and I learned the hard way :-) ).
char *utf8_filename = (something);
char *native_filename;
#ifdef WIN32
# if APR_HAS_UNICODE_FS
IF_WIN_OS_IS_UNICODE
{
native_filename = utf8_filename;
}
# endif
# if APR_HAS_ANSI_FS
ELSE_WIN_OS_IS_ANSI
{
native_filename = convert_utf8_to_locale_charset(utf8_filename);
}
# endif
#else /* !WIN32 */
native_filename = convert_utf8_to_locale_charset(utf8_filename);
#endif /* !WIN32 */
-K
Re: Is --enable-utf8 working everywhere?
William A. Rowe, Jr. wrote: At 09:37 PM 7/16/2002, Brane wrote: Can we start from the top here? What iconv sources are you planning to build? From where? I did say I was using the libiconv-1.8 sources. It's from www.gnu.org. Ok, one to three problems here. First, apr-iconv is from BSD - it's not under gnu license. Are we planning to distribute it with sources? As binaries? This is dripping into muddy waters. And this with a project that claims the ASL isn't even a compatible license. We're not planning to distribute anything. My patch only adds for Windows something that we've been providing on Unix for ages -- namely, support to let the _user_ link apr with libiconv. On Unix, it's in the configury; on Windows, it must be done some other way. But it becomes hugely mucky waters if you start building our library with their library. They are pretty clear (assuming this is LGPL) that other apps can link to it. But other libraries? Not our problem. The LGPL only kicks in if we _distribute_ the libiconv sources, which we don't/won't. Second, we're invested in apr-iconv, most of the porting effort was done... The apr-iconv port in-progress? Not yet. Sad that we aren't investing the effort here. But there was the .dll tangle, apr-iconv needing apr, but apr being built upon iconv. The only thing that needed it, according to my grep of apr_xlate*, was md5. That just moved. iconv is next. Expect minor breakage for just a bit. Then we can include apr-iconv between building apr and building apr-util. In fact, my preference is to build apr-iconv as apr-util/xlate/iconv/... much like /xml/expat. Sure, I'm all for that. But, notice that even getting apr-iconv building correctly on Windows would take more time than it took me to support libiconv. Like I said, we _need_ this for Subversion Alpha, in two days' time. I don't think I could get apr-iconv into shape by then. (I notice it relies on symlinks for charset aliases -- bah!) I don't intend to break the ability to link to an installed local iconv. This is a -supplement- for those who don't have an iconv handy. There's no such ability on Windows today, and my patch touches nothing else (I hope). Am I misinterpreting that this is a dummy module? Yes and no. If you don't have the libiconv sources, it creates a dummy (empty) iconv.lib and doesn't define APR_HAS_XLATE, etc. That's because of MSVC project file idiosyncracies. I'm not too happy about them. I just finished changing over a number of functions, xlate amoung them, to always export the symbols, and then correctly return ENOTIMPL. Why the extra mad effort here? It's an on/off switch. The problem is that you can't teach the .dsp files to conditionally link with a library. The only way to simulate conditional linking is to provide a dummy library if the real one isn't available. And, since it's a dummy, it doesn't provide support for apr_xlate, so APR_HAS_XLATE doesn't get defined in this case. (The changes I made to xlate.c were necessary because the symbols would _not_ get exported correclty in Windows #if APR_HAS_XLATE, that's all.) However, if you drop the libiconv sources into i18n/win32/libiconv, build/win32iconv.bat will a) modify apr_config_iconv.h so that APR_HAS_XLATE, HAVE_ICONV_H and HAVE_ICONV are defined in the right places, and the _real_ iconv.lib/dll gets built and linked in. Yes, I know that smells of hackishness, but it's the best I could come up with on short notice without special tools (configure, etc) available. And we really need this support in Subversion ASAP, it has to go into our Alpha. Fine, then lets take the time to get this right. I'm willing to help, I simply asked for breathing room till Wed. Cool. I had to get this support finished anyways. Like I said, time is somethig we don't have if we want to get Subversion Alpha out the door by Thursday. I'm all for getting apr-iconv integrated, of course. Finally, it's libapriconv or libiconv. Hm? I don't understand. What is libapriconv or libiconv? libiconv.dll - never iconv.dll please. We are trying to stay somewhat in-sync with some unix conventions here so folks don't get as confused. The build scripts that come with libiconv-1.8 create iconv.lib/dll, and I just copy that around. But it can obviously be renamed, no problem there. I'm reviewing the code now. But I am confused. Please enlighten me before you go checking this in... thanks, Hope this is enlightening enough. Very, thanks. I'm 100% behind requiring win32 users to download apr-iconv as part of their apr build. Be done with the custom tweakish build stuff already, this is 2002. We need xlate :-) I dunno. You can use apr without apr-util; you should be able to use it without apr-iconv, too. And use apr-iconv without apr-util. And apr-util without apr-iconv (apr_md5 can work without iconv, after all). -- Brane ibej [EMAIL PROTECTED] http://www.xbc.nu/brane/
Re: Is --enable-utf8 working everywhere?
At 10:42 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: William A. Rowe, Jr. wrote: I'm 100% behind requiring win32 users to download apr-iconv as part of their apr build. Be done with the custom tweakish build stuff already, this is 2002. We need xlate :-) I dunno. You can use apr without apr-util; you should be able to use it without apr-iconv, too. And use apr-iconv without apr-util. And apr-util without apr-iconv (apr_md5 can work without iconv, after all). Seriously? Nah. If a win32 user wants to build apr-util, let them build apr-iconv in with it. You are only asking for a STUB, right? Doesn't have to do anything? So if we have the base libapriconv.dll working tommorow, you are happy? No codepages need be loaded for your alpha 1? Nobody seriously expected Win32 to users to have iconv.dll installed on their PC? Bill
Re: Is --enable-utf8 working everywhere?
At 10:57 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: BTW; I just noticed that the apr_filepath_* functions on Windows can potentially fail horribly if the paths are not UTF-8 (so, not IF_WIN_OS_IS_UNICODE) and the locale uses Shift-JIS, because '\' can be the second byte in a SJIS doublebyte char. Talk about fun. That's exactly correct. apr on win32 is a utf8 only filesystem. It was the only natural way to map the entire unicode filesystem into apr, in a platform-neutral way. If you have a non-utf8 name to work with, apr_xlate it (after we've finished up this iconv work.) Bill
Re: Is --enable-utf8 working everywhere?
William A. Rowe, Jr. wrote: At 10:42 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: William A. Rowe, Jr. wrote: I'm 100% behind requiring win32 users to download apr-iconv as part of their apr build. Be done with the custom tweakish build stuff already, this is 2002. We need xlate :-) I dunno. You can use apr without apr-util; you should be able to use it without apr-iconv, too. And use apr-iconv without apr-util. And apr-util without apr-iconv (apr_md5 can work without iconv, after all). Seriously? Nah. If a win32 user wants to build apr-util, let them build apr-iconv in with it. You are only asking for a STUB, right? Doesn't have to do anything? No no no no! So if we have the base libapriconv.dll working tommorow, you are happy? No codepages need be loaded for your alpha 1? Nobody seriously expected Win32 to users to have iconv.dll installed on their PC? We need full apr_xlate functionality, that was the whole point of the exercise. I seriously expect people to get libiconv-1.8 sources and drop them into their APR source tree f they want to build Subversion Alpha. -- Brane ibej [EMAIL PROTECTED] http://www.xbc.nu/brane/
Re: Is --enable-utf8 working everywhere?
At 11:31 PM 7/16/2002, Brane wrote: William A. Rowe, Jr. wrote: At 10:57 PM 7/16/2002, =?UTF-8?B?QnJhbmtvIMSMaWJlag==?= wrote: BTW; I just noticed that the apr_filepath_* functions on Windows can potentially fail horribly if the paths are not UTF-8 (so, not IF_WIN_OS_IS_UNICODE) and the locale uses Shift-JIS, because '\' can be the second byte in a SJIS doublebyte char. Talk about fun. That's exactly correct. apr on win32 is a utf8 only filesystem. It was the only natural way to map the entire unicode filesystem into apr, in a platform-neutral way. O.K., that's fine. But then, when do the ELSE_WIN_OS_IS_ANSI conditions kick in? On what platforms? (Just curious, that's all.) NEVER. Sorry for the confusion, that should probably be better hidden. The ANSI stuff is magic to make the entire blocks of code evaporate when we toggle a /D WINNT build. One major problem with the APR_HAS_UNICODE_FS garbage ... it isn't strictly true. On Win9x [when compiled for WIN32 - both NT and 9x] that macro will be set, but it lies. I have -absolutely-no- objection to having a function that returns the local codepage (utf-8 on NT, the lcs on 9x, and the appropriate tag from the c locale on other platforms) if we want to identify the codepage of the filesystem. I've just never gotten that far. Bill
Re: Is --enable-utf8 working everywhere?
At 11:42 PM 07/16/2002, Brane wrote: The problem is that you can't teach the .dsp files to conditionally link with a library. The only way to simulate conditional linking is to provide a dummy library if the real one isn't available. That's not quite true. You can do conditional linking using: #if SOME_CONDITION #pragma comment(lib, libname.lib) #endif inside a source file in the project instead of putting the lib in the .dsp. I'm not 100% sure that this would work in this particular case. -- Greg Marr [EMAIL PROTECTED] We thought you were dead. I was, but I'm better now. - Sheridan, The Summoning
Re: Is --enable-utf8 working everywhere?
On Tue, 2002-07-16 at 17:49, Blair Zajac wrote: We don't need to make a copy of errno. Oh yes, we do. When it comes to interaction with the C library you can believe me. errno is no simple variable on almost all systems. So, use the patch as I wrote it, please. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
RE: Is --enable-utf8 working everywhere?
From: Ulrich Drepper [mailto:[EMAIL PROTECTED] On Tue, 2002-07-16 at 17:49, Blair Zajac wrote: We don't need to make a copy of errno. Oh yes, we do. When it comes to interaction with the C library you can believe me. errno is no simple variable on almost all systems. So, use the patch as I wrote it, please. You most definitely do not need to make a copy of errno. As proof, we don't make copies of it anywhere in APR, and it works on every platform we have tried. While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. Ryan
RE: Is --enable-utf8 working everywhere?
From: Ryan Bloom [mailto:[EMAIL PROTECTED] From: Ulrich Drepper [mailto:[EMAIL PROTECTED] On Tue, 2002-07-16 at 17:49, Blair Zajac wrote: We don't need to make a copy of errno. Oh yes, we do. When it comes to interaction with the C library you can believe me. errno is no simple variable on almost all systems. So, use the patch as I wrote it, please. You most definitely do not need to make a copy of errno. As proof, we don't make copies of it anywhere in APR, and it works on every platform we have tried. While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. BTW, I thought of this after I hit send. Errno MUST be treated as a simple variable, or all of the older apps that used it as such would break with new compilers. Ryan
Re: Is --enable-utf8 working everywhere?
Ryan Bloom wrote: You most definitely do not need to make a copy of errno. As proof, we don't make copies of it anywhere in APR, and it works on every platform we have tried. While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. I am almost positive that errno, in use, is always treated as an integral value. No matter what the underlying implementation is, -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
On Tue, 2002-07-16 at 21:14, Branko ibej wrote: Strictly speaking, Ulrich's patch is better, because there's no guarantee that errno is a simple global variable. It could be translated into a function call, for all you know -- depends on the platform, linc implementation, threading support, etc. etc. So making a copy makes sense. So it's a function call. So what? It still returns a single stable value. If you're worried about speed, that's ridiculous. This is an error case; a few cycles here or there isn't going to matter. (And it's not like errno is ever a slow function call, when it's a function call.)
RE: Is --enable-utf8 working everywhere?
On Wed, 2002-07-17 at 08:44, Ryan Bloom wrote: While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. Wrong. errno is in fact on most systems a hidden function call and the C language does not have any provisions to say that a function call can be eliminated because it always returns the same value and has no side effects. We have such a feature in gcc but it's not evailable everywhere. And the fact that you don't make copies in APR anywhere does not mean anything. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
Greg Hudson wrote: So it's a function call. So what? It still returns a single stable value. If you're worried about speed, that's ridiculous. This is an error case; a few cycles here or there isn't going to matter. (And it's not like errno is ever a slow function call, when it's a function call.) Agreed. Checking errno is a cornerstone of defensive programming. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper wrote: While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. Wrong. errno is in fact on most systems a hidden function call and the C language does not have any provisions to say that a function call can be eliminated because it always returns the same value and has no side effects. We have such a feature in gcc but it's not evailable everywhere. If 'errno' is a function call, how the heck does 'errno = 0;' do anything? If it works because the compiler/system treats errno as a integral value, then everything else is moot. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper [EMAIL PROTECTED] writes: Wrong. errno is in fact on most systems a hidden function call and the C language does not have any provisions to say that a function call can be eliminated because it always returns the same value and has no side effects. Could you explain what the impact of this is? Is the concern that we'll have a different result between calls to the hidden function call, or that it's somewhat wasteful to call it multiple times, or something else? I think I and other people understand that it can be a call, but don't see why that matters in this instance. -- Alan Shutko [EMAIL PROTECTED] - In a variety of flavors! Computers will become self-aware entities if struck by lightning.
Re: Is --enable-utf8 working everywhere?
If 'errno' is a function call, how the heck does 'errno = 0;' do anything? If it works because the compiler/system treats errno as a integral value, then everything else is moot. int *func_call(); #define errno *func_call() I don't see the problem with return errno; though. - Sascha
Re: Is --enable-utf8 working everywhere?
Sascha Schumann wrote: If 'errno' is a function call, how the heck does 'errno = 0;' do anything? If it works because the compiler/system treats errno as a integral value, then everything else is moot. int *func_call(); #define errno *func_call() Exactly... we're still back to integral values. Of course, returning a *int and an int are different (and therefore the above can't be a full implemetation), but the concept still implies that we're integral. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
Exactly... we're still back to integral values. Of course, returning a *int and an int are different (and therefore the above can't be a full implemetation), but the concept still implies that we're integral. Actually, that is conceptually the full interface definition. The function returns a pointer to an integer in thread-local storage, so that each thread can maintain its own errno instance. In real, it looks like this on Linux: extern int *__errno_location (void) __THROW __attribute__ ((__const__)); # define errno (*__errno_location ()) - Sascha
Re: Is --enable-utf8 working everywhere?
On Wed, 2002-07-17 at 09:36, Jim Jagielski wrote: If 'errno' is a function call, how the heck does 'errno = 0;' do anything? If it works because the compiler/system treats errno as a integral value, then everything else is moot. Come on people, just look at the implementation of a decent system: #define errno (*__errno_location ()) is what glibc does. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
On Wed, 2002-07-17 at 09:35, Alan Shutko wrote:
Could you explain what the impact of this is? Is the concern that
we'll have a different result between calls to the hidden function
call, or that it's somewhat wasteful to call it multiple times, or
something else?
Compile this code:
extern int *foo (void);
int
bar (void)
{
return *foo () ? *foo () : 1;
}
This is what effectively corresponds to
return errno ? errno : 1;
You will always end up with two calls to 'foo' since the compiler is not
allowed to eliminate them. It has no means to determine that 'foo' has
no side effects and a constant return value.
With gcc the story is different since we can declare 'foo' as
extern int *foo (void) __attribute__ ((const));
This is why you see only one function can. But this doesn't work with a
compiler which only implements ISO C or which has no support in the
system's headers.
--
---. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `
signature.asc
Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper wrote: If 'errno' is a function call, how the heck does 'errno =3D 0;' do anything? If it works because the compiler/system treats errno as a integral value, then everything else is moot. Come on people, just look at the implementation of a decent system: #define errno (*__errno_location ()) is what glibc does. This issue was that we needed to jump through all kinds of hoops because errno could be anything. The counter argument was that as far as usage and implementation, errno is seen as a scalar integer value which significantly bounds the problem. How errno is really implemented is immaterial. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere?
On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper wrote: On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() =20 I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: Is --enable-utf8 working everywhere
On Wed, 2002-07-17 at 14:37, Greg Hudson wrote: I'll try once more, having been seemingly ignored last time: What, precisely, is the problem with calling __error() twice? Speed and much more importantly size. On x86 the code sequence with using a variable is 14 bytes shorter and no conditional jump is needed. Multiply this with all the places you have to perform defensive programming and you'll end up with a large number. The error handling code also effects the non-error case since icache gets polluted with unnecessary code. I have never suggested to change every place where errno isn't used up to this standard (even though this would be good). But if you get a patch which does it like that no argument like that's not how we did it so far must outweigh the benefits. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
On Wed, 2002-07-17 at 14:39, Jim Jagielski wrote: Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? No, just read what I wrote before. But try to tell the compiler that the function it calls is going to return the same value. Just try it before asking another question. It's not possible in ISO C and therefore portable code should always be written with this in mind. -- ---. ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com ` signature.asc Description: This is a digitally signed message part
Re: Is --enable-utf8 working everywhere?
Ulrich Drepper wrote:
On Tue, 2002-07-16 at 10:41, Blair Zajac wrote:
#0 0x000318c4 in apr_xlate_conv_buffer (convset=0x0, inbuf=0xffbef380
stringtest.tmp, inbytes_left=0xffbef0ac, outbuf=0x558f0 twice as
long,
outbytes_left=0xffbef0a8) at xlate.c:308
308 if (convset-ich != (iconv_t)-1) {
(gdb) bt
#0 0x000318c4 in apr_xlate_conv_buffer (convset=0x0, inbuf=0xffbef380
stringtest.tmp, inbytes_left=0xffbef0ac, outbuf=0x558f0 twice as
long,
outbytes_left=0xffbef0a8) at xlate.c:308
#1 0x0002116c in convert_to_stringbuf (convset=0x0, src_data=0xffbef380
stringtest.tmp, src_length=14, dest=0xffbef134, pool=0x55708)
at subversion/libsvn_subr/utf.c:164
#2 0x0002161c in svn_utf_cstring_from_utf8 (dest=0xffbef1b4, src=0xffbef380
stringtest.tmp, pool=0x55708) at subversion/libsvn_subr/utf.c:375
I don't have access to the sources so take this with a grain of salt.
It looks as if the function svn_utf_cstring_from_utf8 calls to get a
iconv descriptor returns NULL as a sign that no such converstion is
possible. iconv_open normally signals this with (iconv_t)-1. So this
can be a simple mismatch. The error might not be visible on other
platforms because they actually do have a decent amount of converters
available.
[cc'ing [EMAIL PROTECTED]
It looks like a problem with iconv_open on my Solaris 8 system where
iconv_open returns (iconv_t)-1, but errno is 0.
Here's a way to demonstrate this using Apr only. Apply the following
patch to apr and run
% test/testmd5 626 UTF-8
Amongst the other print out, I get
iconv_open failed: returned errno is 0
Best,
Blair
--
Blair Zajac [EMAIL PROTECTED]
Web and OS performance plots - http://www.orcaware.com/orca/
Index: i18n/unix/xlate.c
===
RCS file: /home/cvspublic/apr/i18n/unix/xlate.c,v
retrieving revision 1.28
diff -u -r1.28 xlate.c
--- i18n/unix/xlate.c 15 Jul 2002 19:21:01 - 1.28
+++ i18n/unix/xlate.c 16 Jul 2002 22:46:53 -
@@ -52,6 +52,7 @@
* http://www.apache.org/.
*/
+#include stdio.h
#include apr_private.h
#include apr_lib.h
@@ -269,7 +270,9 @@
if (!found) {
new-ich = iconv_open(topage, frompage);
if (new-ich == (iconv_t)-1) {
-return errno;
+const int e = errno;
+fprintf(stderr, iconv_open failed: returned %p errno is %d\n, new-ich, e);
+return e;
}
found = 1;
check_sbcs(new);
Index: test/testmd5.c
===
RCS file: /home/cvspublic/apr/test/testmd5.c,v
retrieving revision 1.14
diff -u -r1.14 testmd5.c
--- test/testmd5.c 10 Jul 2002 06:01:13 - 1.14
+++ test/testmd5.c 16 Jul 2002 22:46:53 -
@@ -98,6 +98,7 @@
STD_TEST_NEQ(apr_md5_init, apr_md5_init(context))
if (xlate) {
+#define APR_HAS_XLATE 1
#if APR_HAS_XLATE
STD_TEST_NEQ(apr_md5_set_xlate,
apr_md5_set_xlate(context, xlate))
