[jira] Commented: (STDCXX-491) string::push_back() slow
[ https://issues.apache.org/jira/browse/STDCXX-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517675 ] Martin Sebor commented on STDCXX-491: - Here are the timings for gcc on the same machine for comparison. Interesting that the gcc-generated code is faster than Intel's own (although we're still more than 30% faster)... $ gcc --version && g++ -m64 -O2 -D_REENTRANT -pthread t.cpp && time ./a.out 0x gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3) Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. real0m48.376s user0m45.259s sys 0m3.104s > string::push_back() slow > > > Key: STDCXX-491 > URL: https://issues.apache.org/jira/browse/STDCXX-491 > Project: C++ Standard Library > Issue Type: Bug > Components: 21. Strings >Affects Versions: 4.1.3 > Environment: gcc 4.1.2, Linux/x86_64 >Reporter: Mark Brown >Assignee: Martin Sebor > Fix For: 4.2 > > > According to my timings string::push_back() in stdcxx 4.1.3 is more than > twice as slow than the same function in gcc 4.1.2 on Linux x86_64: > $ time ./push_back-stdcxx 1 > real0m2.175s > user0m2.004s > sys 0m0.172s > $ time ./push_back-gcc 1 > real0m0.924s > user0m0.760s > sys 0m0.164s > #include > #include > #include > int main (int argc, char *argv[]) > { > const int N = argc < 2 ? 1 : std::atoi (argv [1]); > std::string str; > for (int i = 0; i < N; ++i) > str.push_back ('x'); > assert (str.size () == std::size_t (N)); > } > Comparing the generated assembly, the gcc push_back() is mostly inline but > the stdcxx push_back() is not: > stdcxx: > _Z8pushbackRSsc: > .LFB449: > movq(%rdi), %rax > movl%esi, %edx > movl$1, %ecx > movsbl %dl,%r8d > xorl%edx, %edx > movq-8(%rax), %rsi > jmp _ZNSs7replaceEmmmc > gcc: > _Z8pushbackRSsc: > .LFB904: > movq%rbp, -16(%rsp) > .LCFI0: > movq%r12, -8(%rsp) > .LCFI1: > movq%rdi, %rbp > movq%rbx, -24(%rsp) > .LCFI2: > subq$24, %rsp > .LCFI3: > movq(%rdi), %rax > movl%esi, %r12d > subq$24, %rax > movq(%rax), %rbx > addq$1, %rbx > cmpq8(%rax), %rbx > ja .L2 > movl16(%rax), %eax > testl %eax, %eax > jg .L2 > .L4: > movq(%rbp), %rdx > movq-24(%rdx), %rax > movb%r12b, (%rdx,%rax) > movq(%rbp), %rax > subq$24, %rax > movl$0, 16(%rax) > movq%rbx, (%rax) > movb$0, 24(%rax,%rbx) > movq(%rsp), %rbx > movq8(%rsp), %rbp > movq16(%rsp), %r12 > addq$24, %rsp > ret > .p2align 4,,7 > .L2: > movq%rbx, %rsi > movq%rbp, %rdi > call_ZNSs7reserveEm > jmp .L4 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (STDCXX-491) string::push_back() slow
[ https://issues.apache.org/jira/browse/STDCXX-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Sebor closed STDCXX-491. --- Resolution: Fixed Fix Version/s: 4.2 Here are the new timings obtained with Intel C++ 9.1 on Xeon E5345. We're almost twice as fast than the native C++ Standard Library (libstdc++). $ grep "model name" /proc/cpuinfo | head -n 1 && icc --version model name : Intel(R) Xeon(R) CPU E5345 @ 2.33GHz icc (ICC) 9.1 20070320 Copyright (C) 1985-2007 Intel Corporation. All rights reserved. $ cat t.cpp && make t && icc -D_REENTRANT -O2 t.cpp -pthread && time ./t 0x && time ./a.out 0x #include #include #include int main (int argc, char *argv[]) { const unsigned long N = argc < 2 ? 1 : std::strtoul (argv [1], 0, 0); const unsigned long N0 = 100 < N ? N / 100 : 1; const unsigned long N1 = N % 100; for (unsigned long j = 0; j < N0; ++j) { std::string str; for (unsigned long i = 0; i < N1; ++i) { str.push_back ('x'); } assert (str.size () == std::size_t (N1)); } } icc -c -I/amd/devco/sebor/stdcxx/include/ansi -D_REENTRANT -I/amd/devco/sebor/stdcxx/include -I/build/sebor/stdcxx-icc-9.1.049-12D/include -I/amd/devco/sebor/stdcxx/examples/include -cxxlib-nostd -O2 -w1 t.cpp icc t.o -o t -cxxlib-nostd -lpthread -L/build/sebor/stdcxx-icc-9.1.049-12D/lib -Wl,-R/build/sebor/stdcxx-icc-9.1.049-12D/lib -lstd12D -lcxaguard -lsupc++ -lm real0m35.937s user0m30.230s sys 0m5.696s real0m57.159s user0m53.995s sys 0m3.145s > string::push_back() slow > > > Key: STDCXX-491 > URL: https://issues.apache.org/jira/browse/STDCXX-491 > Project: C++ Standard Library > Issue Type: Bug > Components: 21. Strings >Affects Versions: 4.1.3 > Environment: gcc 4.1.2, Linux/x86_64 >Reporter: Mark Brown >Assignee: Martin Sebor > Fix For: 4.2 > > > According to my timings string::push_back() in stdcxx 4.1.3 is more than > twice as slow than the same function in gcc 4.1.2 on Linux x86_64: > $ time ./push_back-stdcxx 1 > real0m2.175s > user0m2.004s > sys 0m0.172s > $ time ./push_back-gcc 1 > real0m0.924s > user0m0.760s > sys 0m0.164s > #include > #include > #include > int main (int argc, char *argv[]) > { > const int N = argc < 2 ? 1 : std::atoi (argv [1]); > std::string str; > for (int i = 0; i < N; ++i) > str.push_back ('x'); > assert (str.size () == std::size_t (N)); > } > Comparing the generated assembly, the gcc push_back() is mostly inline but > the stdcxx push_back() is not: > stdcxx: > _Z8pushbackRSsc: > .LFB449: > movq(%rdi), %rax > movl%esi, %edx > movl$1, %ecx > movsbl %dl,%r8d > xorl%edx, %edx > movq-8(%rax), %rsi > jmp _ZNSs7replaceEmmmc > gcc: > _Z8pushbackRSsc: > .LFB904: > movq%rbp, -16(%rsp) > .LCFI0: > movq%r12, -8(%rsp) > .LCFI1: > movq%rdi, %rbp > movq%rbx, -24(%rsp) > .LCFI2: > subq$24, %rsp > .LCFI3: > movq(%rdi), %rax > movl%esi, %r12d > subq$24, %rax > movq(%rax), %rbx > addq$1, %rbx > cmpq8(%rax), %rbx > ja .L2 > movl16(%rax), %eax > testl %eax, %eax > jg .L2 > .L4: > movq(%rbp), %rdx > movq-24(%rdx), %rax > movb%r12b, (%rdx,%rax) > movq(%rbp), %rax > subq$24, %rax > movl$0, 16(%rax) > movq%rbx, (%rax) > movb$0, 24(%rax,%rbx) > movq(%rsp), %rbx > movq8(%rsp), %rbp > movq16(%rsp), %r12 > addq$24, %rsp > ret > .p2align 4,,7 > .L2: > movq%rbx, %rsi > movq%rbp, %rdi > call_ZNSs7reserveEm > jmp .L4 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: num_get thread safety tests
Travis Vitek wrote: Martin Sebor wrote: I suppose that would be okay just so long as it's possible to still exercise one facet without the other. Which I suspect might defeat your goal of integrating the testing of both facets into the same program. The problem with creating a dependency between the two facets is that a bug in one might prevent the other from ever being tested or might cause false positives in the other, making debugging the problem more difficult. At the same time, exercising both of the facets simultaneously would be valuable in its own right so it would be nice to be able to do both. Okay, I think I'm going to settle for having separate tests for the get and put. I believe this is what you had originally intended, and I hope that is still acceptable. It's definitely the easier of the two approaches and I agree it's the way to go if you don't want to get bogged down in fixing the process API in the test driver. We will need to fix it at some point but now is not the time. An idea that would let us have it both ways is to integrate the tests for both (all three when you count numpunct) facets into the same program, still keeping the option of running them independently of one another, exercising them all at the same time by default and, when one fails, invoking the program again and exercising them independently, one after another. This approach would involve at least two processes: one to drive and monitor the tests, and another one or more running the tests themselves. The infrastructure for this approach has already been put in place: see the rw_process_create() API in the test driver. Unfortunately, the tests for the API have been failing on a number of platforms so there are issues with it that remain to be resolved. That is more trouble than I'm wanting to chew on at this time. This seems rather complicated, and since I'm not yet familiar with how the testing infrastructure works, I'm worried that I would end up spending time trying to get this approach to work and not writing tests. Agreed. Martin
Re: running tests linked with stdcxx.dll on Windows
Travis Vitek wrote: Martin, There is a build configuration option that for generate.bat that appears to setup the project files to copy the dll into the executable directory so that it can be found at runtime. If you didn't specify that option when generating the solution and project files, you will either need to copy the .dll into the directory of the excutable, or make sure that it can be found in your PATH. Ah, yes, I forgot about the /COPYDLL option. Thanks! On the UNIX side of things, we use the -rpath linker option (or whatever it's called) to hardwire $BUILDDIR/lib into the tests and examples so that the dynamic loader finds the lib even w/o it being in LD_LIBRARY_PATH. It doesn't look like there is an equivalent option in the Microsoft linker: http://msdn2.microsoft.com/en-us/library/y0zzbyt4(VS.80).aspx so copying the DLL or setting PATH are the only available options. I wonder if there is a way to programmatically set PATH in the IDE, either when the solution is loaded or when a project is run. I found that if I enabled the option when generating the solution it added a post-build rule to copy the dlls into the executable directory, but the post-build step always failed to copy the rwtest.dll into place. That's not good, although I wonder how come it works in automated builds. I suspect we'll need to get Farid to look into what's going on here. Martin Here is a paste of the post-build rule. I've added the 'echo' lines to expose the problem that I was seeing echo copying $(SolutionDir)15d\lib\libstd15d.dll to $(OutDir)\libstd15d.dll if exist "$(SolutionDir)15d\lib\libstd15d.dll" ( del "$(OutDir)\libstd15d.dll" copy /Y "$(SolutionDir)15d\lib\libstd15d.dll" "$(OutDir)\libstd15d.dll" ) echo copying $(SolutionDir)15d\tests\rwtest.dll to $(OutDir)\rwtest.dll if exist "$(SolutionDir)15d\tests\rwtest.dll" ( del "$(OutDir)\rwtest.dll" copy /Y "$(SolutionDir)15d\tests\rwtest.dll" "$(OutDir)\rwtest.dll" ) Here is the output when attempt to build/run the 0.inputiter test after building the .rwtest project. Performing Post-Build Event... copying C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\lib\libstd15d.dll to C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\libstd15d.dll 1 file(s) copied. copying C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\rwtest.dll to C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\rwtest.dll The system cannot find the file specified. The output that I get indicates that '$(SolutionDir)\15d\tests' is the same as '$(OutDir)'. So the post build rule is actually deleting the rwtest.dll and then trying to copy the deleted file. Maybe the projects.js script should be updated to print the names of the files that are being manipulated and avoid deleting the destination file if it is the same as the source. Travis -Original Message- From: Martin Sebor [mailto:[EMAIL PROTECTED] Sent: Friday, August 03, 2007 12:22 PM To: stdcxx-dev@incubator.apache.org Subject: running tests linked with stdcxx.dll on Windows When I try to run stdcxx tests linked against the stdcxx.dll using the msvc-8.0_tst.sln solution on Windows I get an error from the loader that the DLL isn't in PATH. Is there something special I need to do to make this work? I expected the project/solution to automatically set things like this up for me. Thanks Martin
RE: num_get thread safety tests (was: Re: next steps)
>Martin Sebor wrote: > >I suppose that would be okay just so long as it's possible to still >exercise one facet without the other. Which I suspect might defeat >your goal of integrating the testing of both facets into the same >program. > >The problem with creating a dependency between the two facets is >that a bug in one might prevent the other from ever being tested >or might cause false positives in the other, making debugging the >problem more difficult. At the same time, exercising both of the >facets simultaneously would be valuable in its own right so it >would be nice to be able to do both. Okay, I think I'm going to settle for having separate tests for the get and put. I believe this is what you had originally intended, and I hope that is still acceptable. > >An idea that would let us have it both ways is to integrate the >tests for both (all three when you count numpunct) facets into >the same program, still keeping the option of running them >independently of one another, exercising them all at the same >time by default and, when one fails, invoking the program again >and exercising them independently, one after another. This >approach would involve at least two processes: one to drive and >monitor the tests, and another one or more running the tests >themselves. The infrastructure for this approach has already been >put in place: see the rw_process_create() API in the test driver. >Unfortunately, the tests for the API have been failing on a number >of platforms so there are issues with it that remain to be resolved. That is more trouble than I'm wanting to chew on at this time. This seems rather complicated, and since I'm not yet familiar with how the testing infrastructure works, I'm worried that I would end up spending time trying to get this approach to work and not writing tests. > >This approach makes the assumption that the implementations of >the facet specializations for charT* are the same as those for >the streambuf_iterators. This happens to be a safe assumption >to make at the moment but it might change in the future. The >other "issue" with this approach is that the specializations >that are used in the vast majority of cases are those on >the streambuf iterators and not those on pointers. Yes, that is the reason that it didn't appeal to me in the first place. The reason I like it is that it doesn't add a dependency on the streambuf or the streambuf_iterator types. If there is a mt issue in either one of these, then the problem would manifest as a failure in the locale facet test, which just seems odd. I realize that the liklihood of a mt issue existing in either the basic_streambuf or the streambuf_iterator is minimal at the moment but it might change in the future. :) > >So I would suggest to roll your own very simple iterator derived >from streambuf_iterator that operates on a plain character buffer. >There should be examples of how to do this in the test suite, >including the test driver itself (see the header rw_streambuf.h). Yes, I realize that. I was just trying to eliminate as much non-facet related code as I could from the test. Obviously I'm trying so hard as to not be testing the cases that I should be. > >Martin > Travis
RE: running tests linked with stdcxx.dll on Windows
Martin, There is a build configuration option that for generate.bat that appears to setup the project files to copy the dll into the executable directory so that it can be found at runtime. If you didn't specify that option when generating the solution and project files, you will either need to copy the .dll into the directory of the excutable, or make sure that it can be found in your PATH. I found that if I enabled the option when generating the solution it added a post-build rule to copy the dlls into the executable directory, but the post-build step always failed to copy the rwtest.dll into place. Here is a paste of the post-build rule. I've added the 'echo' lines to expose the problem that I was seeing echo copying $(SolutionDir)15d\lib\libstd15d.dll to $(OutDir)\libstd15d.dll if exist "$(SolutionDir)15d\lib\libstd15d.dll" ( del "$(OutDir)\libstd15d.dll" copy /Y "$(SolutionDir)15d\lib\libstd15d.dll" "$(OutDir)\libstd15d.dll" ) echo copying $(SolutionDir)15d\tests\rwtest.dll to $(OutDir)\rwtest.dll if exist "$(SolutionDir)15d\tests\rwtest.dll" ( del "$(OutDir)\rwtest.dll" copy /Y "$(SolutionDir)15d\tests\rwtest.dll" "$(OutDir)\rwtest.dll" ) Here is the output when attempt to build/run the 0.inputiter test after building the .rwtest project. Performing Post-Build Event... copying C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\lib\libstd15d.dll to C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\libstd15d.dll 1 file(s) copied. copying C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\rwtest.dll to C:\build\stdcxx\win32.vc8x\msvc-8.0\15d\tests\rwtest.dll The system cannot find the file specified. The output that I get indicates that '$(SolutionDir)\15d\tests' is the same as '$(OutDir)'. So the post build rule is actually deleting the rwtest.dll and then trying to copy the deleted file. Maybe the projects.js script should be updated to print the names of the files that are being manipulated and avoid deleting the destination file if it is the same as the source. Travis >-Original Message- >From: Martin Sebor [mailto:[EMAIL PROTECTED] >Sent: Friday, August 03, 2007 12:22 PM >To: stdcxx-dev@incubator.apache.org >Subject: running tests linked with stdcxx.dll on Windows > >When I try to run stdcxx tests linked against the stdcxx.dll >using the msvc-8.0_tst.sln solution on Windows I get an error >from the loader that the DLL isn't in PATH. > >Is there something special I need to do to make this work? >I expected the project/solution to automatically set things >like this up for me. > >Thanks >Martin >
[jira] Commented: (STDCXX-491) string::push_back() slow
[ https://issues.apache.org/jira/browse/STDCXX-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517641 ] Martin Sebor commented on STDCXX-491: - See the following thread for the patch and a discussion: http://www.mail-archive.com/stdcxx-dev@incubator.apache.org/msg03993.html > string::push_back() slow > > > Key: STDCXX-491 > URL: https://issues.apache.org/jira/browse/STDCXX-491 > Project: C++ Standard Library > Issue Type: Bug > Components: 21. Strings >Affects Versions: 4.1.3 > Environment: gcc 4.1.2, Linux/x86_64 >Reporter: Mark Brown >Assignee: Martin Sebor > > According to my timings string::push_back() in stdcxx 4.1.3 is more than > twice as slow than the same function in gcc 4.1.2 on Linux x86_64: > $ time ./push_back-stdcxx 1 > real0m2.175s > user0m2.004s > sys 0m0.172s > $ time ./push_back-gcc 1 > real0m0.924s > user0m0.760s > sys 0m0.164s > #include > #include > #include > int main (int argc, char *argv[]) > { > const int N = argc < 2 ? 1 : std::atoi (argv [1]); > std::string str; > for (int i = 0; i < N; ++i) > str.push_back ('x'); > assert (str.size () == std::size_t (N)); > } > Comparing the generated assembly, the gcc push_back() is mostly inline but > the stdcxx push_back() is not: > stdcxx: > _Z8pushbackRSsc: > .LFB449: > movq(%rdi), %rax > movl%esi, %edx > movl$1, %ecx > movsbl %dl,%r8d > xorl%edx, %edx > movq-8(%rax), %rsi > jmp _ZNSs7replaceEmmmc > gcc: > _Z8pushbackRSsc: > .LFB904: > movq%rbp, -16(%rsp) > .LCFI0: > movq%r12, -8(%rsp) > .LCFI1: > movq%rdi, %rbp > movq%rbx, -24(%rsp) > .LCFI2: > subq$24, %rsp > .LCFI3: > movq(%rdi), %rax > movl%esi, %r12d > subq$24, %rax > movq(%rax), %rbx > addq$1, %rbx > cmpq8(%rax), %rbx > ja .L2 > movl16(%rax), %eax > testl %eax, %eax > jg .L2 > .L4: > movq(%rbp), %rdx > movq-24(%rdx), %rax > movb%r12b, (%rdx,%rax) > movq(%rbp), %rax > subq$24, %rax > movl$0, 16(%rax) > movq%rbx, (%rax) > movb$0, 24(%rax,%rbx) > movq(%rsp), %rbx > movq8(%rsp), %rbp > movq16(%rsp), %r12 > addq$24, %rsp > ret > .p2align 4,,7 > .L2: > movq%rbx, %rsi > movq%rbp, %rdi > call_ZNSs7reserveEm > jmp .L4 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (STDCXX-491) string::push_back() slow
[ https://issues.apache.org/jira/browse/STDCXX-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Sebor reassigned STDCXX-491: --- Assignee: Martin Sebor > string::push_back() slow > > > Key: STDCXX-491 > URL: https://issues.apache.org/jira/browse/STDCXX-491 > Project: C++ Standard Library > Issue Type: Bug > Components: 21. Strings >Affects Versions: 4.1.3 > Environment: gcc 4.1.2, Linux/x86_64 >Reporter: Mark Brown >Assignee: Martin Sebor > > According to my timings string::push_back() in stdcxx 4.1.3 is more than > twice as slow than the same function in gcc 4.1.2 on Linux x86_64: > $ time ./push_back-stdcxx 1 > real0m2.175s > user0m2.004s > sys 0m0.172s > $ time ./push_back-gcc 1 > real0m0.924s > user0m0.760s > sys 0m0.164s > #include > #include > #include > int main (int argc, char *argv[]) > { > const int N = argc < 2 ? 1 : std::atoi (argv [1]); > std::string str; > for (int i = 0; i < N; ++i) > str.push_back ('x'); > assert (str.size () == std::size_t (N)); > } > Comparing the generated assembly, the gcc push_back() is mostly inline but > the stdcxx push_back() is not: > stdcxx: > _Z8pushbackRSsc: > .LFB449: > movq(%rdi), %rax > movl%esi, %edx > movl$1, %ecx > movsbl %dl,%r8d > xorl%edx, %edx > movq-8(%rax), %rsi > jmp _ZNSs7replaceEmmmc > gcc: > _Z8pushbackRSsc: > .LFB904: > movq%rbp, -16(%rsp) > .LCFI0: > movq%r12, -8(%rsp) > .LCFI1: > movq%rdi, %rbp > movq%rbx, -24(%rsp) > .LCFI2: > subq$24, %rsp > .LCFI3: > movq(%rdi), %rax > movl%esi, %r12d > subq$24, %rax > movq(%rax), %rbx > addq$1, %rbx > cmpq8(%rax), %rbx > ja .L2 > movl16(%rax), %eax > testl %eax, %eax > jg .L2 > .L4: > movq(%rbp), %rdx > movq-24(%rdx), %rax > movb%r12b, (%rdx,%rax) > movq(%rbp), %rax > subq$24, %rax > movl$0, 16(%rax) > movq%rbx, (%rax) > movb$0, 24(%rax,%rbx) > movq(%rsp), %rbx > movq8(%rsp), %rbp > movq16(%rsp), %r12 > addq$24, %rsp > ret > .p2align 4,,7 > .L2: > movq%rbx, %rsi > movq%rbp, %rdi > call_ZNSs7reserveEm > jmp .L4 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
Farid Zaripov wrote: -Original Message- From: Andrew Black [mailto:[EMAIL PROTECTED] Sent: Friday, August 03, 2007 6:36 PM To: stdcxx-dev@incubator.apache.org Subject: Re: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h Something in this patch appears to cause the exec utility to hang on Windows. I haven't had much time to dig into the cause, but I thought I'd give everyone a heads up. Farid, if you have a chance, could you look into this? The exec process deadlocked in (exec.cpp, line 1055): - warn_last_error ("Opening child input stream"); - after the CreateFile (options->infname) has failed (because options->infname == "/dev/null"). The proposed fix below, but deadlocking is the another issue to be resolved: Thanks! I went ahead and committed it myself to get the builds going again but I forgot to attribute it to you -- hope you're okay with it. http://svn.apache.org/viewvc?view=rev&rev=562580 Martin Index: util.cpp === --- util.cpp(revision 562407) +++ util.cpp(working copy) @@ -200,9 +200,15 @@ free (fname); } +#ifndef _WIN32 +#define DEV_NULL "/dev/null" +#else +#define DEV_NULL "NUL" +#endif + /* If we didn't find a source file, use /dev/null */ -fname = (char*)RW_MALLOC (sizeof "/dev/null"); -strcpy (fname, "/dev/null"); +fname = (char*)RW_MALLOC (sizeof DEV_NULL); +strcpy (fname, DEV_NULL); return fname; } Farid.
[jira] Created: (STDCXX-509) RW libstd is sensitive to link order
RW libstd is sensitive to link order Key: STDCXX-509 URL: https://issues.apache.org/jira/browse/STDCXX-509 Project: C++ Standard Library Issue Type: Improvement Components: Build Affects Versions: 4.1.2 Environment: gcc3.2, linux AS3.0 Reporter: Ravi K Inampudi Fix For: 4.1.2 A customer discovered a problem with std::numeric_limits in RW libstd. Placing RW libstd on linkline *before* libFoo results in the program printing "0" instead of "inf". The problem doesn't happen with native gcc STL. We reproduced the problem with the latest version of standard library downloaded from Apache. From our understanding, when creating the libFoo.so and using the native standard library is creating an implicit dependency on the libsupc++ (compiler dependent). This implicit dependency is initializing the static variable when libFoo.so is loaded. If we use RW libstd instread of libstdc++(native gcc stl), the program will print "0" instead of "inf". # make shared lib gcc -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/include/ansi -pthread -D_RWSTD_USE_CONFIG -I/nfs/homes/dean/temp/stdlib-cxx/12d/include -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/include -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/examples/include -pedantic -nostdinc++ -O2 -Wall -W -Wcast-qual -Winline -Wshadow -Wwrite-strings -Wno-long-long -Wcast-align -fPIC -shared -o libFooRW-link.so foo.C # make binary. gcc -pthread -o mainRW main.C -L/nfs/homes/dean/temp/stdlib-cxx/12d/lib -lstd12d -lsupc++ -lm -lFooRW However, using the RW standard library, if such dependency is created explicitly then it prints "inf". For example, the following set of commands fixes the problem: # make shared lib gcc -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/include/ansi -pthread -D_RWSTD_USE_CONFIG -I/nfs/homes/dean/temp/stdlib-cxx/12d/include -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/include -I/amd/homes/dean/temp/stdlib-cxx/stdcxx-4.1.3/examples/include -pedantic -nostdinc++ -O2 -Wall -W -Wcast-qual -Winline -Wshadow -Wwrite-strings -Wno-long-long -Wcast-align -fPIC -shared -o libFooRW-link.so foo.C -lstd12d # make binary. gcc -pthread -o mainRW main.C -L/nfs/homes/dean/temp/stdlib-cxx/12d/lib -lstd12d -lsupc++ -lm -lFooRW But the customer never links their shared libs(i.e libFoo in this example) with RW libstd. They only link binaries with RW libstd! But it makes RW libstd sensitive to link order. And they think the problem is in limits.cpp file: > > static union { > char _C_bits [sizeof (double)]; > double _C_inf; > } __rw_dbl_inf_bits = { _RWSTD_DBL_INF_BITS }; > > _RWSTD_EXPORT extern const double __rw_dbl_infinity = > __rw_dbl_inf_bits._C_inf; > > __rw_dbl_infinity ends up in the uninitialized data section of > libstd_gcc32.so > > nm -C libstd_gcc32.so| grep __rw_dbl_infinity > 000937f8 B __rw::__rw_dbl_infinity > If the symbol was initialized in data section, the link order wouldn't matter. Environment: RWSP6, gcc3.2, linux AS3.0 Martin Sebor's Comments: Because the initializer of __rw_dbl_infinity is not a constant expresssion [expr.const] the symbol is initialized dynamically (at runtime) rather than statically (i.e., at load time). It seems that the compiler should be able to initialize it statically anyway, even if it's not required to. Regardless, we should avoid making the assumption that it will and change the initialization of the extern constants to use constant expressions instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
running tests linked with stdcxx.dll on Windows
When I try to run stdcxx tests linked against the stdcxx.dll using the msvc-8.0_tst.sln solution on Windows I get an error from the loader that the DLL isn't in PATH. Is there something special I need to do to make this work? I expected the project/solution to automatically set things like this up for me. Thanks Martin
Re: num_get thread safety tests
Martin Sebor wrote: [redirected to stdcxx-dev from a private thread] Travis Vitek wrote: As mentioned, I'm already started on the locale tests. I'm actually thinking it might make more sense to have the new test [22.locale.num.get.mt.cpp] verify the correctness of the output as well as the mt safety of the get operations. The existing 22.locale.num.put.mt.cpp test could remain as-is, or maybe I should just roll both the get and put tests into one? I suppose that would be okay just so long as it's possible to still exercise one facet without the other. Which I suspect might defeat your goal of integrating the testing of both facets into the same program. The problem with creating a dependency between the two facets is that a bug in one might prevent the other from ever being tested or might cause false positives in the other, making debugging the problem more difficult. At the same time, exercising both of the facets simultaneously would be valuable in its own right so it would be nice to be able to do both. An idea that would let us have it both ways is to integrate the tests for both (all three when you count numpunct) facets into the same program, still keeping the option of running them independently of one another, exercising them all at the same time by default and, when one fails, invoking the program again and exercising them independently, one after another. This approach would involve at least two processes: one to drive and monitor the tests, and another one or more running the tests themselves. The infrastructure for this approach has already been put in place: see the rw_process_create() API in the test driver. Unfortunately, the tests for the API have been failing on a number of platforms so there are issues with it that remain to be resolved. So here is a quick one for you. The locale facets are all templatized on the iterator type. For simplicity I'm thinking that I should just instantiate the facets using character pointers instead of the default [i,o]streambuf_iterator template. This will avoid testing the stream buf iterators in each of the facet tests. Do you have any issues with that? This approach makes the assumption that the implementations of the facet specializations for charT* are the same as those for the streambuf_iterators. This happens to be a safe assumption to make at the moment but it might change in the future. The other "issue" with this approach is that the specializations that are used in the vast majority of cases are those on the streambuf iterators and not those on pointers. So I would suggest to roll your own very simple iterator derived from streambuf_iterator I meant streambuf here, not iterator of course. Martin that operates on a plain character buffer. There should be examples of how to do this in the test suite, including the test driver itself (see the header rw_streambuf.h). Martin
Re: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
Farid Zaripov wrote: -Original Message- From: Farid Zaripov [mailto:[EMAIL PROTECTED] Sent: Friday, August 03, 2007 7:53 PM To: stdcxx-dev@incubator.apache.org Subject: RE: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h The exec process deadlocked in (exec.cpp, line 1055): - warn_last_error ("Opening child input stream"); - after the CreateFile (options->infname) has failed (because options->infname == "/dev/null"). The proposed fix below, but deadlocking is the another issue to be resolved: The deadlock fixed thus: http://svn.apache.org/viewvc?view=rev&rev=562537 Great! I'm working on resolving the regression in exec.cpp. Martin
RE: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
> -Original Message- > From: Farid Zaripov [mailto:[EMAIL PROTECTED] > Sent: Friday, August 03, 2007 7:53 PM > To: stdcxx-dev@incubator.apache.org > Subject: RE: svn commit: r562224 - in > /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp > runall.cpp target.h util.cpp util.h > > The exec process deadlocked in (exec.cpp, line 1055): > - > warn_last_error ("Opening child input stream"); > - > after the CreateFile (options->infname) has failed (because > options->infname == "/dev/null"). > > The proposed fix below, but deadlocking is the another issue to be > resolved: The deadlock fixed thus: http://svn.apache.org/viewvc?view=rev&rev=562537 Farid.
num_get thread safety tests (was: Re: next steps)
[redirected to stdcxx-dev from a private thread] Travis Vitek wrote: As mentioned, I'm already started on the locale tests. I'm actually thinking it might make more sense to have the new test [22.locale.num.get.mt.cpp] verify the correctness of the output as well as the mt safety of the get operations. The existing 22.locale.num.put.mt.cpp test could remain as-is, or maybe I should just roll both the get and put tests into one? I suppose that would be okay just so long as it's possible to still exercise one facet without the other. Which I suspect might defeat your goal of integrating the testing of both facets into the same program. The problem with creating a dependency between the two facets is that a bug in one might prevent the other from ever being tested or might cause false positives in the other, making debugging the problem more difficult. At the same time, exercising both of the facets simultaneously would be valuable in its own right so it would be nice to be able to do both. An idea that would let us have it both ways is to integrate the tests for both (all three when you count numpunct) facets into the same program, still keeping the option of running them independently of one another, exercising them all at the same time by default and, when one fails, invoking the program again and exercising them independently, one after another. This approach would involve at least two processes: one to drive and monitor the tests, and another one or more running the tests themselves. The infrastructure for this approach has already been put in place: see the rw_process_create() API in the test driver. Unfortunately, the tests for the API have been failing on a number of platforms so there are issues with it that remain to be resolved. So here is a quick one for you. The locale facets are all templatized on the iterator type. For simplicity I'm thinking that I should just instantiate the facets using character pointers instead of the default [i,o]streambuf_iterator template. This will avoid testing the stream buf iterators in each of the facet tests. Do you have any issues with that? This approach makes the assumption that the implementations of the facet specializations for charT* are the same as those for the streambuf_iterators. This happens to be a safe assumption to make at the moment but it might change in the future. The other "issue" with this approach is that the specializations that are used in the vast majority of cases are those on the streambuf iterators and not those on pointers. So I would suggest to roll your own very simple iterator derived from streambuf_iterator that operates on a plain character buffer. There should be examples of how to do this in the test suite, including the test driver itself (see the header rw_streambuf.h). Martin
[PATCH] rw_create_catalog()
Attached is a patch, moving generate_catalog() function from 22.locale.messages.cpp into rwtest library. ChangeLog: * rw_locale.h (rw_create_catalog): New function to create message catalog. * locale.cpp: Ditto. * 22.locale.messages.cpp: Removed generate_catalog(), used rw_create_catalog(). Farid. Index: include/rw_locale.h === --- include/rw_locale.h (revision 562407) +++ include/rw_locale.h (working copy) @@ -120,4 +120,22 @@ _TEST_EXPORT int rw_opt_setlocales (int, char*[]); +enum { +RW_MAX_SETS = 5, +RW_MAX_MESSAGES = 5 +}; + +#ifndef _WIN32 +# define RW_CAT_EXT ".cat" +#else +# define RW_CAT_EXT ".dll" +#endif + +// creates message file and invokes gencat to create message catalog +// then removes the message file +// returns 0 in success +_TEST_EXPORT int +rw_create_catalog (const char *, + const char* const [RW_MAX_SETS][RW_MAX_MESSAGES]); + #endif // RW_LOCALE_H_INCLUDED Index: localization/22.locale.messages.cpp === --- localization/22.locale.messages.cpp (revision 562407) +++ localization/22.locale.messages.cpp (working copy) @@ -40,7 +40,7 @@ #include// for strlen() #include// for getcwd(), getenv() -#include // for FILE, fopen(), fprintf() +#include // for remove() #include// for LC_ALL #include // for mbsinit() @@ -206,10 +206,7 @@ #define NLS_CAT_NAME "rwstdmessages" -#define MAX_SETS 5 -#define MAX_MESSAGES 5 - int msg_id (int set, int id) { #ifdef _WIN32 @@ -227,7 +224,7 @@ /***/ static const char* const -messages [MAX_SETS][MAX_MESSAGES] = { +messages [RW_MAX_SETS][RW_MAX_MESSAGES] = { { "First set, first message", "First set, second message", "First set, third message", @@ -260,76 +257,6 @@ } }; - -void generate_catalog (const char *msg_name, - const char* const text [MAX_SETS][MAX_MESSAGES]) -{ -std::FILE* const f = std::fopen (msg_name, "w"); - -if (!f) -return; - -#ifndef _WIN32 - -for (int i = 0; i < MAX_SETS; ++i) { -std::fprintf (f, "$set %d This is Set %d\n", i+1, i+1); -for (int j = 0; j < MAX_MESSAGES; ++j) { -std::fprintf (f, "%d %s\n", j + 1, text [i][j]); -} -} - -#else // if defined (_WIN32) - -std::fprintf (f, "STRINGTABLE\nBEGIN\n"); -for (int i = 0; i < MAX_SETS; ++i) { -for (int j = 0; j < MAX_MESSAGES; ++j) { -const int msgid = msg_id (i + 1, j + 1); -std::fprintf (f, "%d \"%s\"\n", msgid, text[i][j]); -} -} - -std::fprintf (f, "END\n"); - -#endif // _WIN32 - -std::fclose (f); - -char *cat_name = new char [std::strlen (msg_name) + 1]; -const char *dot = std::strrchr (msg_name, '.'); -std::strncpy (cat_name, msg_name, dot - msg_name); -*(cat_name + (dot - msg_name)) = '\0'; - -#ifndef _WIN32 - -rw_system ("gencat %s.cat %s", cat_name, msg_name); - -#else // if defined (_WIN32) - -char cpp_name [128]; - -std::sprintf (cpp_name, "%s.cpp", cat_name); - -std::FILE* const cpp_file = std::fopen (cpp_name, "w"); -std::fprintf (cpp_file, "void foo () { }"); -std::fclose (cpp_file); - -rw_system ( "rc -r %s.rc " - "&& cl -nologo -c %s" - "&& link -nologo /DLL /OUT:%s.dll %s.obj %s.res", - cat_name, - cpp_name, - cat_name, cat_name, cat_name); - -rw_system (SHELL_RM_F "%s %s.rc %s.res %s.obj", - cpp_name, cat_name, cat_name, cat_name); - -#endif // _WIN32 - -delete[] cat_name; - -std::remove (msg_name); -} - /***/ @@ -590,9 +517,9 @@ typedef std::allocator Allocator; typedef std::basic_string String; -for (int setId = 1; setId < MAX_SETS; ++setId) { +for (int setId = 1; setId < RW_MAX_SETS; ++setId) { -for (int msgId = 1; msgId < MAX_MESSAGES; ++msgId) { +for (int msgId = 1; msgId < RW_MAX_MESSAGES; ++msgId) { const int id = msg_id (setId, msgId); const String got = msgs.get (cat, setId, id, def); @@ -777,16 +704,12 @@ std::sprintf (msg_name, "rwstdmessages_%d.rc", int (i)); #endif -generate_catalog (msg_name, messages); +rw_create_catalog (msg_name, messages); const char* const dot = std::strrchr (msg_name, '.'); std::strncpy (catalog_names[i], msg_name, dot - msg_name); *(catalog_names[i] + (dot - msg_name)) = '\0'; -#ifdef _WIN32 -std::strcat (catalog_names[i], ".dll"); -#endif // _WIN32 - // open each catalog (expect success) cats [i] = open_catalog (msgs, catalog_names [i], loc, 0, cname, __
RE: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
> -Original Message- > From: Andrew Black [mailto:[EMAIL PROTECTED] > Sent: Friday, August 03, 2007 6:36 PM > To: stdcxx-dev@incubator.apache.org > Subject: Re: svn commit: r562224 - in > /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp > runall.cpp target.h util.cpp util.h > > Something in this patch appears to cause the exec utility to > hang on Windows. I haven't had much time to dig into the > cause, but I thought I'd give everyone a heads up. Farid, if > you have a chance, could you look into this? The exec process deadlocked in (exec.cpp, line 1055): - warn_last_error ("Opening child input stream"); - after the CreateFile (options->infname) has failed (because options->infname == "/dev/null"). The proposed fix below, but deadlocking is the another issue to be resolved: Index: util.cpp === --- util.cpp(revision 562407) +++ util.cpp(working copy) @@ -200,9 +200,15 @@ free (fname); } +#ifndef _WIN32 +#define DEV_NULL "/dev/null" +#else +#define DEV_NULL "NUL" +#endif + /* If we didn't find a source file, use /dev/null */ -fname = (char*)RW_MALLOC (sizeof "/dev/null"); -strcpy (fname, "/dev/null"); +fname = (char*)RW_MALLOC (sizeof DEV_NULL); +strcpy (fname, DEV_NULL); return fname; } Farid.
Re: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
Andrew Black wrote: Greetings Martin. Something in this patch appears to cause the exec utility to hang on Windows. I haven't had much time to dig into the cause, but I thought I'd give everyone a heads up. Farid, if you have a chance, could you look into this? Aww, crap! Windows strikes again. We really need to minimize the amount of platform-specific code here. Let me do a build and see what's going on. Thanks Martin --Andrew Black [EMAIL PROTECTED] wrote: Author: sebor Date: Thu Aug 2 12:04:26 2007 New Revision: 562224 URL: http://svn.apache.org/viewvc?view=rev&rev=562224 Log: 2007-07-26 Martin Sebor <[EMAIL PROTECTED]> * display.h (print_footer): Added an argument for the total number of programs processed by the utility. * display.cpp (print_target_verbose): Print stdin, stdout, and stderr redirectiopn. (print_status_verbose): Justified output. (print_footer_plain): Printed the total number of programs processed by the utility and avoided printing assertion totals unless they're valid. (print_footer_verbose): Added an argument. * target.h (target_opts): Added infname and outfname members. * util.h (input_name): Declared. * util.cpp (input_name): Defined to parallel output_name(). * exec.cpp (open_input): Removed. (exec_file): Used target_opts::infname and target_opts::outfname. * runall.cpp (run_target): Called input_name() and output_name() to set the names of files to redirect input and output from and to, respectively. Avoided printing out assertion totals when they're not valid. Modified: incubator/stdcxx/trunk/util/display.cpp incubator/stdcxx/trunk/util/display.h incubator/stdcxx/trunk/util/exec.cpp incubator/stdcxx/trunk/util/runall.cpp incubator/stdcxx/trunk/util/target.h incubator/stdcxx/trunk/util/util.cpp incubator/stdcxx/trunk/util/util.h [Change snipped]
Re: svn commit: r562224 - in /incubator/stdcxx/trunk/util: display.cpp display.h exec.cpp runall.cpp target.h util.cpp util.h
Greetings Martin. Something in this patch appears to cause the exec utility to hang on Windows. I haven't had much time to dig into the cause, but I thought I'd give everyone a heads up. Farid, if you have a chance, could you look into this? --Andrew Black [EMAIL PROTECTED] wrote: > Author: sebor > Date: Thu Aug 2 12:04:26 2007 > New Revision: 562224 > > URL: http://svn.apache.org/viewvc?view=rev&rev=562224 > Log: > 2007-07-26 Martin Sebor <[EMAIL PROTECTED]> > > * display.h (print_footer): Added an argument for the total number > of programs processed by the utility. > * display.cpp (print_target_verbose): Print stdin, stdout, and stderr > redirectiopn. > (print_status_verbose): Justified output. > (print_footer_plain): Printed the total number of programs processed > by the utility and avoided printing assertion totals unless they're > valid. > (print_footer_verbose): Added an argument. > * target.h (target_opts): Added infname and outfname members. > * util.h (input_name): Declared. > * util.cpp (input_name): Defined to parallel output_name(). > * exec.cpp (open_input): Removed. > (exec_file): Used target_opts::infname and target_opts::outfname. > * runall.cpp (run_target): Called input_name() and output_name() > to set the names of files to redirect input and output from and > to, respectively. > Avoided printing out assertion totals when they're not valid. > > Modified: > incubator/stdcxx/trunk/util/display.cpp > incubator/stdcxx/trunk/util/display.h > incubator/stdcxx/trunk/util/exec.cpp > incubator/stdcxx/trunk/util/runall.cpp > incubator/stdcxx/trunk/util/target.h > incubator/stdcxx/trunk/util/util.cpp > incubator/stdcxx/trunk/util/util.h > [Change snipped]
Re: [PATCH] gencat.cpp
Farid Zaripov wrote: -Original Message- From: Martin Sebor [mailto:[EMAIL PROTECTED] Sent: Friday, August 03, 2007 2:03 AM To: stdcxx-dev@incubator.apache.org Subject: Re: [PATCH] gencat.cpp (was: RE: svn commit: r555006 - /incubator/stdcxx/trunk/src/catalog.cpp) This looks good, although it could use a few more comments :) Where does it live? In $TOPDIR/util/? Yes. Does the conditional #error cause any problems on UNIX? We can exclude this file from compilation (the same way as excluded rwstdmessages.cpp in makefile.exm). Would it be safer to implement it on UNIX as well by having it invoke the system gencat utility? Yes we can do it, but with renaming the utility to, for example, rwgencat to avoid invoking the utility recursively. Or by specifying the full pathname when invoking the system gencat utility. Or simply by removing $BUILDDIR/bin from PATH if it's there. If you think this approach will work I would favor it over changing the makefiles and avoiding building the utility on UNIX. Does anyone have any concerns or suggestions? Andrew? Travis? Martin
RE: [PATCH] gencat.cpp (was: RE: svn commit: r555006 - /incubator/stdcxx/trunk/src/catalog.cpp)
> -Original Message- > From: Martin Sebor [mailto:[EMAIL PROTECTED] > Sent: Friday, August 03, 2007 2:03 AM > To: stdcxx-dev@incubator.apache.org > Subject: Re: [PATCH] gencat.cpp (was: RE: svn commit: r555006 > - /incubator/stdcxx/trunk/src/catalog.cpp) > > This looks good, although it could use a few more comments :) > > Where does it live? In $TOPDIR/util/? Yes. > Does the conditional #error cause any problems on UNIX? We can exclude this file from compilation (the same way as excluded rwstdmessages.cpp in makefile.exm). > Would it be safer to implement it on UNIX as well by having it > invoke the system gencat utility? Yes we can do it, but with renaming the utility to, for example, rwgencat to avoid invoking the utility recursively. Farid.
RE: [PATCH] fix for STDCXX-507
> -Original Message- > From: Martin Sebor [mailto:[EMAIL PROTECTED] > Sent: Friday, August 03, 2007 12:30 AM > To: stdcxx-dev@incubator.apache.org > Subject: Re: [PATCH] fix for STDCXX-507 > > A few questions: > > 1. Where does the script come from and are there any licensing > restrictions on incorporating it into stdcxx? I don't think so. I get it by link on page http://kde-cygwin.sourceforge.net/kde3/faq.php#0xc05 Here (http://lists-archives.org/mingw-users/07369-the-application-failed-to-i nitializeproperly-0xc005.html) I found that this script included into mingw in /mingw/usr/lib/ldscripts/. > 2. Is the script necessary? I.e., is there a simpler solution, > such as conditionally changing the source code for CygWin > and making the problematic constants non-const? Yes, but how to find this constants? > 3. If the answer to (2) is yes, shouldn't the script be used > only for gcc prior to 3.4? From the thread you referenced > in the patch it sounded like the starting with 3.4 gcc > handles this case correctly. Or have I misunderstood? I have gcc 3.4.4 here and problem still exist. And I think it's is impossible to detect which constants should be located in .data section by compiler because the compiler don't knows will be the library linked statically or dynamicaly. Farid.