Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Jean-Marc Lasgouttes wrote:

Koji == Koji Yokota [EMAIL PROTECTED] writes:



Koji Then, is a sort of modification as the attached patch
Koji acceptable? It seems it solved the problem and is working fine!

This particular patch does not look good, but we'll find one around
this idea. Isn't there some define that we can test against to detect
the faulty freebsd versions?
  
I'm not quite sure whether _GLIBCPP_USE_WCHAR_T is set in forthcoming 
7.0-release, but as far as judging from the documentation which comes 
with the newest gcc-4.2 which 7.0-release will adopt, it probably is 
not. If it is the case, all versions of FreeBSD at this moment do not 
support it. So simply look at the define __FreeBSD__?


Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Jean-Marc Lasgouttes wrote:

"Koji" == Koji Yokota <[EMAIL PROTECTED]> writes:



Koji> Then, is a sort of modification as the attached patch
Koji> acceptable? It seems it solved the problem and is working fine!

This particular patch does not look good, but we'll find one around
this idea. Isn't there some define that we can test against to detect
the faulty freebsd versions?
  
I'm not quite sure whether _GLIBCPP_USE_WCHAR_T is set in forthcoming 
7.0-release, but as far as judging from the documentation which comes 
with the newest gcc-4.2 which 7.0-release will adopt, it probably is 
not. If it is the case, all versions of FreeBSD at this moment do not 
support it. So simply look at the define __FreeBSD__?


Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:
What a shame. This problem is known since 2003, and still not fixed? Does 
nobody care for such a standard violation?
  
It seems the fact that FreeBSD had an old implementation of a wchar_t 
equivalence before wchar_t was introduced made the problem complicated. 
I wish it is solved in the next major release.
Anyway, since hacking libstdc++ is not an option for LyX, there is only one 
possible solution: Modify the configure check that checks for a usable 
wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code 
that is used on cygwin will be used on freebsd. That code is not as good 
as the native gcc code, but it is at least better than nothing.


BTW, disabling boost::format would not help. That might fix the bformat 
problem, but the locale stuff that fails in boost::format is needed at 
other places, too (e.g. docstringstream). You can be lucky that 
boost::format crashed, the docstringstream would probably simply corrupt 
some data and proceed happily.
  
Then, is a sort of modification as the attached patch acceptable? It 
seems it solved the problem and is working fine!


Koji


patch-m4-wchar_t.m4
Description: application/m4

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Koji == Koji Yokota [EMAIL PROTECTED] writes:

Koji Then, is a sort of modification as the attached patch
Koji acceptable? It seems it solved the problem and is working fine!

This particular patch does not look good, but we'll find one around
this idea. Isn't there some define that we can test against to detect
the faulty freebsd versions?

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:
What a shame. This problem is known since 2003, and still not fixed? Does 
nobody care for such a standard violation?
  
It seems the fact that FreeBSD had an old implementation of a wchar_t 
equivalence before wchar_t was introduced made the problem complicated. 
I wish it is solved in the next major release.
Anyway, since hacking libstdc++ is not an option for LyX, there is only one 
possible solution: Modify the configure check that checks for a usable 
wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code 
that is used on cygwin will be used on freebsd. That code is not as good 
as the native gcc code, but it is at least better than nothing.


BTW, disabling boost::format would not help. That might fix the bformat 
problem, but the locale stuff that fails in boost::format is needed at 
other places, too (e.g. docstringstream). You can be lucky that 
boost::format crashed, the docstringstream would probably simply corrupt 
some data and proceed happily.
  
Then, is a sort of modification as the attached patch acceptable? It 
seems it solved the problem and is working fine!


Koji


patch-m4-wchar_t.m4
Description: application/m4

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes:

Koji> Then, is a sort of modification as the attached patch
Koji> acceptable? It seems it solved the problem and is working fine!

This particular patch does not look good, but we'll find one around
this idea. Isn't there some define that we can test against to detect
the faulty freebsd versions?

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Koji Yokota wrote:


Georg Baum wrote:

If you want to debug this fdurther it might be a good idea to write a
small standalone program that simply calls boost::format with the
problematic input.

boost::basic_format() itself seems working if it is called with
ordinary strings:

  #include iostream
  #include boost/format.hpp
 
  using namespace std;
 
  main()
  {
  cout  boost::basic_formatchar (test %1$s \n) % OK;
  }

which prints out

test OK


That is expected, but does boost::basic_formatwchar_tit also do that?


No, it doesn't compile. It complains that there is no matching function, 
which probablly means that in boost/boost/format/format_fwd.hpp:



#if !defined(BOOST_NO_STD_WSTRING)   !defined(BOOST_NO_STD_WSTREAMBUF) \
 !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM)
typedef basic_formatwchar_t   wformat;
#endif


one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or 
BOOST_FORMAT_IGNORE_STRINGSTREAM is defined.



  (gdb) p *start
  $38 = (const wchar_t ) @0x8b1e9c0: 49


49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.

That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.


Thank you. I will try to investigate it further.

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Koji Yokota wrote:
No, it doesn't compile. It complains that there is no matching function, 
which probablly means that in boost/boost/format/format_fwd.hpp:


#if !defined(BOOST_NO_STD_WSTRING)   
!defined(BOOST_NO_STD_WSTREAMBUF) \

 !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM)
typedef basic_formatwchar_t   wformat;
#endif


one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or 
BOOST_FORMAT_IGNORE_STRINGSTREAM is defined.


There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, 
which may still exist(?) as shown here:


http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9

I see that in 
/usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h


#define _GLIBCXX_USE_WCHAR_T 1

is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for 
system gcc3.4).


Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Am Dienstag, 12. Juni 2007 19:05 schrieb Koji Yokota:

 There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, 
 which may still exist(?) as shown here:

This did never occur as an option to me. Good detective work!

   http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9
 
 I see that in 
 /usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h
 
 #define _GLIBCXX_USE_WCHAR_T 1
 
 is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for 
 system gcc3.4).

What a shame. This problem is known since 2003, and still not fixed? Does 
nobody care for such a standard violation?

Anyway, since hacking libstdc++ is not an option for LyX, there is only one 
possible solution: Modify the configure check that checks for a usable 
wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code 
that is used on cygwin will be used on freebsd. That code is not as good 
as the native gcc code, but it is at least better than nothing.

BTW, disabling boost::format would not help. That might fix the bformat 
problem, but the locale stuff that fails in boost::format is needed at 
other places, too (e.g. docstringstream). You can be lucky that 
boost::format crashed, the docstringstream would probably simply corrupt 
some data and proceed happily.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Koji Yokota wrote:


Georg Baum wrote:

If you want to debug this fdurther it might be a good idea to write a
small standalone program that simply calls boost::format with the
problematic input.

boost::basic_format() itself seems working if it is called with
"ordinary" strings:

 > #include 
 > #include 
 >
 > using namespace std;
 >
 > main()
 > {
 > cout << boost::basic_format("test %1$s \n") % "OK";
 > }

which prints out

test OK


That is expected, but does boost::basic_formatit also do that?


No, it doesn't compile. It complains that there is no matching function, 
which probablly means that in boost/boost/format/format_fwd.hpp:



#if !defined(BOOST_NO_STD_WSTRING)  && !defined(BOOST_NO_STD_WSTREAMBUF) \
&& !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM)
typedef basic_format  wformat;
#endif


one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or 
BOOST_FORMAT_IGNORE_STRINGSTREAM is defined.



 > (gdb) p *start
 > $38 = (const wchar_t &) @0x8b1e9c0: 49


49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.

That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.


Thank you. I will try to investigate it further.

Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Koji Yokota wrote:
No, it doesn't compile. It complains that there is no matching function, 
which probablly means that in boost/boost/format/format_fwd.hpp:


#if !defined(BOOST_NO_STD_WSTRING)  && 
!defined(BOOST_NO_STD_WSTREAMBUF) \

&& !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM)
typedef basic_format  wformat;
#endif


one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or 
BOOST_FORMAT_IGNORE_STRINGSTREAM is defined.


There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, 
which may still exist(?) as shown here:


http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9

I see that in 
/usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h


#define _GLIBCXX_USE_WCHAR_T 1

is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for 
system gcc3.4).


Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Am Dienstag, 12. Juni 2007 19:05 schrieb Koji Yokota:

> There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, 
> which may still exist(?) as shown here:

This did never occur as an option to me. Good detective work!

>   http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9
> 
> I see that in 
> /usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h
> 
> #define _GLIBCXX_USE_WCHAR_T 1
> 
> is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for 
> system gcc3.4).

What a shame. This problem is known since 2003, and still not fixed? Does 
nobody care for such a standard violation?

Anyway, since hacking libstdc++ is not an option for LyX, there is only one 
possible solution: Modify the configure check that checks for a usable 
wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code 
that is used on cygwin will be used on freebsd. That code is not as good 
as the native gcc code, but it is at least better than nothing.

BTW, disabling boost::format would not help. That might fix the bformat 
problem, but the locale stuff that fails in boost::format is needed at 
other places, too (e.g. docstringstream). You can be lucky that 
boost::format crashed, the docstringstream would probably simply corrupt 
some data and proceed happily.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

 Georg Baum wrote:
 If you want to debug this fdurther it might be a good idea to write a
 small standalone program that simply calls boost::format with the
 problematic input.
 
 boost::basic_format() itself seems working if it is called with
 ordinary strings:
 
   #include iostream
   #include boost/format.hpp
  
   using namespace std;
  
   main()
   {
   cout  boost::basic_formatchar (test %1$s \n) % OK;
   }
 
 which prints out
 
 test OK

That is expected, but does boost::basic_formatwchar_tit also do that?

 And what is the value of *start? And how does the whole format string
 look like?
 
 They are as follows:
 
   (gdb) p *start
   $38 = (const wchar_t ) @0x8b1e9c0: 49

49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.

That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

 So, this may have to be modified too.
 
   (gdb) p *start
   $38 = (const wchar_t ) @0x8b1e9c0: 49
 
   (gdb) x/5s 0x8b1e9c0
   0x8b1e9c0:  1
   0x8b1e9c2:  
   0x8b1e9c3:  
   0x8b1e9c4:  $
   0x8b1e9c6:  
   0x8b1e9c7:  
   0x8b1e9c8:  s
   0x8b1e9ca:  
   0x8b1e9cb:  
   0x8b1e9cc:  \r0
   0x8b1e9cf:  
   0x8b1e9d0:  \2220
   0x8b1e9d3:  
   0x8b1e9d4:  \\O
   0x8b1e9d7:  
   0x8b1e9d8:  \020b
   0x8b1e9db:  
   0x8b1e9dc:  W0
   0x8b1e9df:  
   0x8b1e9e0:  f0
   0x8b1e9e3:  
   0x8b1e9e4:  D0
   0x8b1e9e7:  
   0x8b1e9e8:  ~0
   0x8b1e9eb:  
 
 where string size is
 
   (gdb) p fstring_size
   $54 = 25

This looks a bit strange (can gdb not print wstrings?), but it seems to be a
correct format string.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

> Georg Baum wrote:
>> If you want to debug this fdurther it might be a good idea to write a
>> small standalone program that simply calls boost::format with the
>> problematic input.
> 
> boost::basic_format() itself seems working if it is called with
> "ordinary" strings:
> 
>  > #include 
>  > #include 
>  >
>  > using namespace std;
>  >
>  > main()
>  > {
>  > cout << boost::basic_format("test %1$s \n") % "OK";
>  > }
> 
> which prints out
> 
> test OK

That is expected, but does boost::basic_formatit also do that?

>> And what is the value of *start? And how does the whole format string
>> look like?
> 
> They are as follows:
> 
>  > (gdb) p *start
>  > $38 = (const wchar_t &) @0x8b1e9c0: 49

49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.

That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

> So, this may have to be modified too.
> 
>>  > (gdb) p *start
>>  > $38 = (const wchar_t &) @0x8b1e9c0: 49
> 
>  > (gdb) x/5s 0x8b1e9c0
>  > 0x8b1e9c0:  "1"
>  > 0x8b1e9c2:  ""
>  > 0x8b1e9c3:  ""
>  > 0x8b1e9c4:  "$"
>  > 0x8b1e9c6:  ""
>  > 0x8b1e9c7:  ""
>  > 0x8b1e9c8:  "s"
>  > 0x8b1e9ca:  ""
>  > 0x8b1e9cb:  ""
>  > 0x8b1e9cc:  "\r0"
>  > 0x8b1e9cf:  ""
>  > 0x8b1e9d0:  "\2220"
>  > 0x8b1e9d3:  ""
>  > 0x8b1e9d4:  "\\O"
>  > 0x8b1e9d7:  ""
>  > 0x8b1e9d8:  "\020b"
>  > 0x8b1e9db:  ""
>  > 0x8b1e9dc:  "W0"
>  > 0x8b1e9df:  ""
>  > 0x8b1e9e0:  "f0"
>  > 0x8b1e9e3:  ""
>  > 0x8b1e9e4:  "D0"
>  > 0x8b1e9e7:  ""
>  > 0x8b1e9e8:  "~0"
>  > 0x8b1e9eb:  ""
> 
> where string size is
> 
>  > (gdb) p fstring_size
>  > $54 = 25

This looks a bit strange (can gdb not print wstrings?), but it seems to be a
correct format string.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

 I'm not quite sure about the data structure used here, but arguments fmt
 and arg1 in bformat() seem to contain no data fields:

Oh, I should assume fmt and arg1 contain pointers. Sorry.

 (gdb) p fmt
 $44 = (const docstring *) 0xbfbfe588
 (gdb) p arg1
 $46 = (docstring *) 0xbfbfe580
 (gdb) x/s 0xbfbfe588
 0xbfbfe588: [EMAIL PROTECTED],%\bD
 (gdb) x/s 0xbfbfe580
 0xbfbfe580:	 
[EMAIL PROTECTED],%\bD


Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Also,


template class Ch, class Tr, class Alloc
basic_formatCh, Tr, Alloc:: basic_format(const string_type s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
  exceptions_(io::all_error_bits)
{
parse(s);  }


it contains:

 (gdb) p s
 $47 = (
 const 
std::basic_stringwchar_t,std::char_traitswchar_t,std::allocatorwchar_t 
 *) 0xbfbfe588


 (gdb) x/s 0xbfbfe588
 0xbfbfe588: [EMAIL PROTECTED],%\bD

and in


bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), items_[cur_item], fac, i1, exceptions());



we have the following argument data:

 (gdb) p items_[cur_item]
 $48 = (
const 
boost::io::detail::format_itemwchar_t,std::char_traitswchar_t,std::allocatorwchar_t 
 *) 0x8af4a80


 (gdb) x/s 0x8af4a80
 0x8af4a80:  \b\b


Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

So, this may have to be modified too.


  (gdb) p *start
  $38 = (const wchar_t ) @0x8b1e9c0: 49


 (gdb) x/5s 0x8b1e9c0
 0x8b1e9c0:  1
 0x8b1e9c2:  
 0x8b1e9c3:  
 0x8b1e9c4:  $
 0x8b1e9c6:  
 0x8b1e9c7:  
 0x8b1e9c8:  s
 0x8b1e9ca:  
 0x8b1e9cb:  
 0x8b1e9cc:  \r0
 0x8b1e9cf:  
 0x8b1e9d0:  \2220
 0x8b1e9d3:  
 0x8b1e9d4:  \\O
 0x8b1e9d7:  
 0x8b1e9d8:  \020b
 0x8b1e9db:  
 0x8b1e9dc:  W0
 0x8b1e9df:  
 0x8b1e9e0:  f0
 0x8b1e9e3:  
 0x8b1e9e4:  D0
 0x8b1e9e7:  
 0x8b1e9e8:  ~0
 0x8b1e9eb:  

where string size is

 (gdb) p fstring_size
 $54 = 25


Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

> I'm not quite sure about the data structure used here, but arguments fmt
> and arg1 in bformat() seem to contain no data fields:

Oh, I should assume  and  contain pointers. Sorry.

> (gdb) p 
> $44 = (const docstring *) 0xbfbfe588
> (gdb) p 
> $46 = (docstring *) 0xbfbfe580
> (gdb) x/s 0xbfbfe588
> 0xbfbfe588: "[EMAIL PROTECTED],%\bD"
> (gdb) x/s 0xbfbfe580
> 0xbfbfe580:	 
"[EMAIL PROTECTED],%\bD"


Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Also,


template< class Ch, class Tr, class Alloc>
basic_format:: basic_format(const string_type& s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
  exceptions_(io::all_error_bits)
{
parse(s);  }


it contains:

> (gdb) p 
> $47 = (
> const 
std::basic_string *) 0xbfbfe588

>
> (gdb) x/s 0xbfbfe588
> 0xbfbfe588: "[EMAIL PROTECTED],%\bD"

and in


bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), _[cur_item], fac, i1, exceptions());



we have the following argument data:

> (gdb) p _[cur_item]
> $48 = (
const 
boost::io::detail::format_item *) 0x8af4a80

>
> (gdb) x/s 0x8af4a80
> 0x8af4a80:  "\b\b"


Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

So, this may have to be modified too.


 > (gdb) p *start
 > $38 = (const wchar_t &) @0x8b1e9c0: 49


> (gdb) x/5s 0x8b1e9c0
> 0x8b1e9c0:  "1"
> 0x8b1e9c2:  ""
> 0x8b1e9c3:  ""
> 0x8b1e9c4:  "$"
> 0x8b1e9c6:  ""
> 0x8b1e9c7:  ""
> 0x8b1e9c8:  "s"
> 0x8b1e9ca:  ""
> 0x8b1e9cb:  ""
> 0x8b1e9cc:  "\r0"
> 0x8b1e9cf:  ""
> 0x8b1e9d0:  "\2220"
> 0x8b1e9d3:  ""
> 0x8b1e9d4:  "\\O"
> 0x8b1e9d7:  ""
> 0x8b1e9d8:  "\020b"
> 0x8b1e9db:  ""
> 0x8b1e9dc:  "W0"
> 0x8b1e9df:  ""
> 0x8b1e9e0:  "f0"
> 0x8b1e9e3:  ""
> 0x8b1e9e4:  "D0"
> 0x8b1e9e7:  ""
> 0x8b1e9e8:  "~0"
> 0x8b1e9eb:  ""

where string size is

> (gdb) p fstring_size
> $54 = 25


Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

If you want to debug this fdurther it might be a good idea to write a small
standalone program that simply calls boost::format with the problematic
input.


boost::basic_format() itself seems working if it is called with 
ordinary strings:


 #include iostream
 #include boost/format.hpp

 using namespace std;

 main()
 {
 cout  boost::basic_formatchar (test %1$s \n) % OK;
 }

which prints out

test OK


And what is the value of *start? And how does the whole format string look
like?


They are as follows:

 (gdb) p *start
 $38 = (const wchar_t ) @0x8b1e9c0: 49

 (gdb) p start
 $39 = (
 __gnu_cxx::__normal_iteratorconst
 wchar_t*,std::basic_stringwchar_t, std::char_traitswchar_t,
 std::allocatorwchar_t   ) @0xbfbfe2cc: {
   _M_current = 0x8b1e9c0}

 (gdb) x/25s 0xbfbfe2cc
 0xbfbfe2cc: \300\351\261\b\030\033\177)\r
 0xbfbfe2d6: 
 0xbfbfe2d7: 
 0xbfbfe2d8: \214\351\261\bp\352\261\b
 0xbfbfe2e1: 
 0xbfbfe2e2: 
 0xbfbfe2e3: 
 0xbfbfe2e4: 
 0xbfbfe2e5: 
 0xbfbfe2e6: 
 0xbfbfe2e7: 
 0xbfbfe2e8: 
 0xbfbfe2e9: 
 0xbfbfe2ea: 
 0xbfbfe2eb: 
 0xbfbfe2ec:	 
\234\v\177)\314;\177)8\344\277\277(\343\277\277f|v)\354;\177)\001

 0xbfbfe306: 
 0xbfbfe307: 
 0xbfbfe308:	 
([EMAIL PROTECTED] 
\037\177)%

 0xbfbfe32a: 
 0xbfbfe32b: 
 0xbfbfe32c: \016\312\023\001\377\377\377\377
 0xbfbfe335: 
 0xbfbfe336: 
 0xbfbfe337: 


I am now pretty sure that the
problem is not in any locale stuff, but in boost::format itself, because
normally this part of the code should only be reached with valid format
characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid
format character).


I'm not quite sure about the data structure used here, but arguments fmt 
and arg1 in bformat() seem to contain no data fields:



src/support/lstrings.cpp line 875



template
docstring bformatdocstring(docstring const  fmt, docstring arg1)
{
return (boost::basic_formatchar_type(fmt) % arg1).str();
}


Data:
fmt  = $26 = (const docstring ) @0xbfbfe588: {static npos = 4294967295, 
  _M_dataplus = {std::allocatorwchar_t = {__gnu_cxx::new_allocatorwchar_t = {No data fields}, No data fields}, _M_p = 0x8b1e98c}}
arg1 = $27 = (docstring ) @0xbfbfe580: {static npos = 4294967295, 
  _M_dataplus = {std::allocatorwchar_t = {__gnu_cxx::new_allocatorwchar_t = {No data fields}, No data fields}, _M_p = 0x8b1e70c}}


And, subsequently call in boost/format/format_implementation.hpp line 60


template class Ch, class Tr, class Alloc
basic_formatCh, Tr, Alloc:: basic_format(const string_type s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
  exceptions_(io::all_error_bits)
{
parse(s);  
}


Data:

s = $28 = (
const std::basic_stringwchar_t,std::char_traitswchar_t,std::allocatorwchar_t  ) @0xbfbfe588: {static npos = 4294967295, 
  _M_dataplus = {std::allocatorwchar_t = {__gnu_cxx::new_allocatorwchar_t = {No data fields}, No data fields}, _M_p = 0x8b1e98c}}


I also attach the data of arguments passed to parse_printf_directive() 
in boost/format/parsing.hpp which is called later:



boost/format/parsing.hpp line 437



bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), items_[cur_item], fac, i1, exceptions());

(gdb) p it
$29 = {_M_current = 0x8b1e9c0}
(gdb) p buf.end()
$30 = {_M_current = 0x8b1ea70}
(gdb) p items_[cur_item]
$31 = (
const class 
boost::io::detail::format_itemwchar_t,std::char_traitswchar_t,std::allocatorwchar_t
  *) 0x8af4a80
(gdb) p items_[cur_item]
$35 = (
const 
boost::io::detail::format_itemwchar_t,std::char_traitswchar_t,std::allocatorwchar_t
  ) @0x8af4a80: {argN_ = -1, res_ = {
static npos = 4294967295, 
_M_dataplus = {std::allocatorwchar_t = {__gnu_cxx::new_allocatorwchar_t = {No data fields}, No data fields}, _M_p = 0x8a5a7a8}}, 
  appendix_ = {static npos = 4294967295, 
_M_dataplus = {std::allocatorwchar_t = {__gnu_cxx::new_allocatorwchar_t = {No data fields}, No data fields}, _M_p = 0x8a5a7a8}}, 
  fmtstate_ = {width_ = 0, precision_ = 6, fill_ = 32, flags_ = 4098, 
rdstate_ = std::_S_goodbit, exceptions_ = std::_S_goodbit, 
loc_ = {boost::optional_detail::optional_basestd::locale = {boost::optional_detail::optional_tag = {No data fields}, m_initialized = false, 
m_storage = {dummy_ = {data = \000\000\000, 
aligner_ = incomplete type}}}, No data fields}}, 
  truncate_ = 2147483647, pad_scheme_ = 0}

(gdb) p fac
$32 = (const struct std::ctypewchar_t ) @0x297f1f20: incomplete type
(gdb) p i1
$33 = 13
(gdb) p exceptions()
$34 = 255 '\377'


Are these passed data OK?

Thank you,

Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

If you want to debug this fdurther it might be a good idea to write a small
standalone program that simply calls boost::format with the problematic
input.


boost::basic_format() itself seems working if it is called with 
"ordinary" strings:


> #include 
> #include 
>
> using namespace std;
>
> main()
> {
> cout << boost::basic_format("test %1$s \n") % "OK";
> }

which prints out

test OK


And what is the value of *start? And how does the whole format string look
like?


They are as follows:

> (gdb) p *start
> $38 = (const wchar_t &) @0x8b1e9c0: 49
>
> (gdb) p start
> $39 = (
> __gnu_cxx::__normal_iterator wchar_t*,std::basic_string std::allocator > > &) @0xbfbfe2cc: {
>   _M_current = 0x8b1e9c0}
>
> (gdb) x/25s 0xbfbfe2cc
> 0xbfbfe2cc: "\300\351\261\b\030\033\177)\r"
> 0xbfbfe2d6: ""
> 0xbfbfe2d7: ""
> 0xbfbfe2d8: "\214\351\261\bp\352\261\b"
> 0xbfbfe2e1: ""
> 0xbfbfe2e2: ""
> 0xbfbfe2e3: ""
> 0xbfbfe2e4: ""
> 0xbfbfe2e5: ""
> 0xbfbfe2e6: ""
> 0xbfbfe2e7: ""
> 0xbfbfe2e8: ""
> 0xbfbfe2e9: ""
> 0xbfbfe2ea: ""
> 0xbfbfe2eb: ""
> 0xbfbfe2ec:	 
"\234\v\177)\314;\177)8\344\277\277(\343\277\277f|v)\354;\177)\001"

> 0xbfbfe306: ""
> 0xbfbfe307: ""
> 0xbfbfe308:	 
"([EMAIL PROTECTED] 
\037\177)%"

> 0xbfbfe32a: ""
> 0xbfbfe32b: ""
> 0xbfbfe32c: "\016\312\023\001\377\377\377\377"
> 0xbfbfe335: ""
> 0xbfbfe336: ""
> 0xbfbfe337: ""


I am now pretty sure that the
problem is not in any locale stuff, but in boost::format itself, because
normally this part of the code should only be reached with valid format
characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid
format character).


I'm not quite sure about the data structure used here, but arguments fmt 
and arg1 in bformat() seem to contain no data fields:



src/support/lstrings.cpp line 875

>

template<>
docstring bformat(docstring const & fmt, docstring arg1)
{
return (boost::basic_format(fmt) % arg1).str();
}


Data:
fmt  = $26 = (const docstring &) @0xbfbfe588: {static npos = 4294967295, 
  _M_dataplus = { = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e98c}}
arg1 = $27 = (docstring &) @0xbfbfe580: {static npos = 4294967295, 
  _M_dataplus = { = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e70c}}


And, subsequently call in boost/format/format_implementation.hpp line 60


template< class Ch, class Tr, class Alloc>
basic_format:: basic_format(const string_type& s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
  exceptions_(io::all_error_bits)
{
parse(s);  
}


Data:

s = $28 = (
const std::basic_string &) @0xbfbfe588: {static npos = 4294967295, 
  _M_dataplus = { = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e98c}}


I also attach the data of arguments passed to parse_printf_directive() 
in boost/format/parsing.hpp which is called later:



boost/format/parsing.hpp line 437

>

bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), _[cur_item], fac, i1, exceptions());

(gdb) p it
$29 = {_M_current = 0x8b1e9c0}
(gdb) p buf.end()
$30 = {_M_current = 0x8b1ea70}
(gdb) p _[cur_item]
$31 = (
const class 
boost::io::detail::format_item *) 0x8af4a80
(gdb) p items_[cur_item]
$35 = (
const 
boost::io::detail::format_item &) @0x8af4a80: {argN_ = -1, res_ = {
static npos = 4294967295, 
_M_dataplus = { = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8a5a7a8}}, 
  appendix_ = {static npos = 4294967295, 
_M_dataplus = { = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8a5a7a8}}, 
  fmtstate_ = {width_ = 0, precision_ = 6, fill_ = 32, flags_ = 4098, 
rdstate_ = std::_S_goodbit, exceptions_ = std::_S_goodbit, 
loc_ = { = { = {}, m_initialized = false, 
m_storage = {dummy_ = {data = "\000\000\000", 
aligner_ = }}}, }}, 
  truncate_ = 2147483647, pad_scheme_ = 0}

(gdb) p fac
$32 = (const struct std::ctype &) @0x297f1f20: 
(gdb) p i1
$33 = 13
(gdb) p exceptions()
$34 = 255 '\377'


Are these passed data OK?

Thank you,

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

 Georg Baum wrote:
 Then Koji could fire up a debugger, set a breakpoint in
 parse_printf_directive() and report what the result of the narrow() call
 is.
 
 Do you mean the return value of wrap_narrow() call? If so, the result of
 gdb was:

Yes, I meant that.

   $1 = -65 '\277'
 
 It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259
 and both gave the same result.

And what is the value of *start? And how does the whole format string look
like? Unfortunately I forgot to ask for that. I am now pretty sure that the
problem is not in any locale stuff, but in boost::format itself, because
normally this part of the code should only be reached with valid format
characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid
format character).

If you want to debug this fdurther it might be a good idea to write a small
standalone program that simply calls boost::format with the problematic
input.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Koji Yokota wrote:

> Georg Baum wrote:
>> Then Koji could fire up a debugger, set a breakpoint in
>> parse_printf_directive() and report what the result of the narrow() call
>> is.
> 
> Do you mean the return value of wrap_narrow() call? If so, the result of
> gdb was:

Yes, I meant that.

>  > $1 = -65 '\277'
> 
> It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259
> and both gave the same result.

And what is the value of *start? And how does the whole format string look
like? Unfortunately I forgot to ask for that. I am now pretty sure that the
problem is not in any locale stuff, but in boost::format itself, because
normally this part of the code should only be reached with valid format
characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid
format character).

If you want to debug this fdurther it might be a good idea to write a small
standalone program that simply calls boost::format with the problematic
input.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Then Koji could fire up a debugger, set a breakpoint in
parse_printf_directive() and report what the result of the narrow() call
is.


Do you mean the return value of wrap_narrow() call? If so, the result of 
gdb was:


 $1 = -65 '\277'

It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259 
and both gave the same result.


Thanks,

Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Then Koji could fire up a debugger, set a breakpoint in
parse_printf_directive() and report what the result of the narrow() call
is.


Do you mean the return value of wrap_narrow() call? If so, the result of 
gdb was:


> $1 = -65 '\277'

It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259 
and both gave the same result.


Thanks,

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Jean-Marc Lasgouttes wrote:

- how can we test whether freebsd's wchat_t support is good enough?

First of all is wchar_t really 32 bit on freebsd? I know that some unices
have 16bit wchar_t, IIRC AIX. This could be tested with a small test
program that simply prints sizeof(wchar_t). I know that the configure test
is supposed to find that out, but who knows.


I tried this and the result was sizeof(wchar_t) = 4.


I also see that it is installed in /usr/local. If that means that he
compiled himself from source then I would guess that something went wrong
with the configuration of the compiler. gcc has tons of configuration
options, and with many combinations you can get an unusable result (BTDT).


This gcc 4.1 was installed via ports (a package management system) as 
the default gcc used by the system is gcc 3.4.6. Since it is tested by 
the port maintainer and used by many people in the same setup, I think 
it is fairly reliable (although it may not be as solid as the system gcc).



Another thing to try is to force Enricos facet implementation by applying
the attached patch. Does that change anything?


Unfortunately, this didn't change anything.

 Then Koji could fire up a debugger, set a breakpoint in
 parse_printf_directive() and report what the result of the narrow() call
 is.

I will try this and post the result soon.

Thanks!

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Am Dienstag, 5. Juni 2007 19:11 schrieb Koji Yokota:
 Georg Baum wrote:
  Jean-Marc Lasgouttes wrote:
  - how can we test whether freebsd's wchat_t support is good enough?
  First of all is wchar_t really 32 bit on freebsd? I know that some 
unices
  have 16bit wchar_t, IIRC AIX. This could be tested with a small test
  program that simply prints sizeof(wchar_t). I know that the configure 
test
  is supposed to find that out, but who knows.
 
 I tried this and the result was sizeof(wchar_t) = 4.

I expected that, but it is always better to be 100% sure.

  I also see that it is installed in /usr/local. If that means that he
  compiled himself from source then I would guess that something went 
wrong
  with the configuration of the compiler. gcc has tons of configuration
  options, and with many combinations you can get an unusable result 
(BTDT).
 
 This gcc 4.1 was installed via ports (a package management system) as 
 the default gcc used by the system is gcc 3.4.6. Since it is tested by 
 the port maintainer and used by many people in the same setup, I think 
 it is fairly reliable (although it may not be as solid as the system 
gcc).

I tend to agree. Just for completeness, could you try to compile LyX with 
the system gcc please? gcc 3.4.x is supposed to be good enough as well, 
and since it uses the same abi as gcc 4.1 you can even use it with 
libraries compiled with g++ 4.1.

  Another thing to try is to force Enricos facet implementation by 
applying
  the attached patch. Does that change anything?
 
 Unfortunately, this didn't change anything.

That means that the gcc locale facets are probably not the problem.

   Then Koji could fire up a debugger, set a breakpoint in
   parse_printf_directive() and report what the result of the narrow() 
call
   is.
 
 I will try this and post the result soon.

I believe now that this is the only way to find out the problem. Don't be 
surprised if you seem to see wild jumps in the debugger, when I debugged 
the locale code last time there was a lot of inlined stuff that gdb could 
not relate to the correct line in the correct file.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Georg Baum wrote:

Jean-Marc Lasgouttes wrote:

- how can we test whether freebsd's wchat_t support is good enough?

First of all is wchar_t really 32 bit on freebsd? I know that some unices
have 16bit wchar_t, IIRC AIX. This could be tested with a small test
program that simply prints sizeof(wchar_t). I know that the configure test
is supposed to find that out, but who knows.


I tried this and the result was sizeof(wchar_t) = 4.


I also see that it is installed in /usr/local. If that means that he
compiled himself from source then I would guess that something went wrong
with the configuration of the compiler. gcc has tons of configuration
options, and with many combinations you can get an unusable result (BTDT).


This gcc 4.1 was installed via ports (a package management system) as 
the default gcc used by the system is gcc 3.4.6. Since it is tested by 
the port maintainer and used by many people in the same setup, I think 
it is fairly reliable (although it may not be as solid as the system gcc).



Another thing to try is to force Enricos facet implementation by applying
the attached patch. Does that change anything?


Unfortunately, this didn't change anything.

> Then Koji could fire up a debugger, set a breakpoint in
> parse_printf_directive() and report what the result of the narrow() call
> is.

I will try this and post the result soon.

Thanks!

Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Am Dienstag, 5. Juni 2007 19:11 schrieb Koji Yokota:
> Georg Baum wrote:
> > Jean-Marc Lasgouttes wrote:
> >> - how can we test whether freebsd's wchat_t support is good enough?
> > First of all is wchar_t really 32 bit on freebsd? I know that some 
unices
> > have 16bit wchar_t, IIRC AIX. This could be tested with a small test
> > program that simply prints sizeof(wchar_t). I know that the configure 
test
> > is supposed to find that out, but who knows.
> 
> I tried this and the result was sizeof(wchar_t) = 4.

I expected that, but it is always better to be 100% sure.

> > I also see that it is installed in /usr/local. If that means that he
> > compiled himself from source then I would guess that something went 
wrong
> > with the configuration of the compiler. gcc has tons of configuration
> > options, and with many combinations you can get an unusable result 
(BTDT).
> 
> This gcc 4.1 was installed via ports (a package management system) as 
> the default gcc used by the system is gcc 3.4.6. Since it is tested by 
> the port maintainer and used by many people in the same setup, I think 
> it is fairly reliable (although it may not be as solid as the system 
gcc).

I tend to agree. Just for completeness, could you try to compile LyX with 
the system gcc please? gcc 3.4.x is supposed to be good enough as well, 
and since it uses the same abi as gcc 4.1 you can even use it with 
libraries compiled with g++ 4.1.

> > Another thing to try is to force Enricos facet implementation by 
applying
> > the attached patch. Does that change anything?
> 
> Unfortunately, this didn't change anything.

That means that the gcc locale facets are probably not the problem.

>  > Then Koji could fire up a debugger, set a breakpoint in
>  > parse_printf_directive() and report what the result of the narrow() 
call
>  > is.
> 
> I will try this and post the result soon.

I believe now that this is the only way to find out the problem. Don't be 
surprised if you seem to see wild jumps in the debugger, when I debugged 
the locale code last time there was a lot of inlined stuff that gdb could 
not relate to the correct line in the correct file.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Tue, May 29, 2007 at 11:20:45AM +0900, Koji Yokota wrote:
 Enrico Forestieri wrote:
  I don't understand. Seems that you are still using wchar_t ...
  This crash is very similar to the one I was getting on cygwin after the
  switch to unicode. I remember that someway I was able to fix it by
  using the STLport library, before the real fix by Georg. Have a look
  at this thread (may be you will get some ideas on how to proceed):
  http://thread.gmane.org/gmane.editors.lyx.devel/68855
 
 Enrico, I'm now checking this but before that I send a quick result of 
 the patches. I applied the attached patches but unfortunately there was 
 no improvement. I stated that the initialization succeeded in the 
 previous mail, but it was due to the fact that I already had a user 
 setting directory $HOME/.lyx-svn . Removing it draws back to the same 
 result as before.

That is because of autoloading of the splash.lyx document.
I am running out of ideas here. I think that you will need advice from
a real expert such as Georg.

 # Does RC1 go with the following modification suggested before?
 
 --- src/support/lstrings.cpp(revision 18398)
 +++ src/support/lstrings.cpp(working copy)
 @@ -21,6 +21,7 @@
   #include boost/tokenizer.hpp
   #include boost/assert.hpp
 
 +#undef USE_BOOST_FORMAT
   #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
   #if USE_BOOST_FORMAT
   #include boost/format.hpp

 --- src/support/docstring.h.orig  Sun May 27 17:26:19 2007
 +++ src/support/docstring.h   Sun May 27 17:27:27 2007
 @@ -98,7 +98,7 @@
  } // namespace lyx
  
  
 -#if SIZEOF_WCHAR_T != 4  defined(__GNUC__)  defined(__GNUC_MINOR__)  
 __GNUC__ == 3  __GNUC_MINOR__  4
 +#if 1
  // Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
  namespace std {
  
 --- src/support/docstream.cpp.origSun May 27 17:40:11 2007
 +++ src/support/docstream.cpp Sun May 27 17:40:11 2007
 @@ -322,7 +322,7 @@
  
  }
  
 -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
 +#if 1
  // We get undefined references to these virtual methods. This looks like
  // a bug in gcc. The implementation here does not do anything useful, since
  // it is overriden in iconv_codecvt_facet.
 --- src/support/docstring.cpp.origSun May 27 17:40:35 2007
 +++ src/support/docstring.cpp Sun May 27 17:40:35 2007
 @@ -217,7 +217,7 @@
  
  } // namespace lyx
  
 -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
 +#if 1
  
  // gcc does not have proper locale facets for lyx::char_type if
  // sizeof(wchar_t) == 2, so we have to implement them on our own.
 --- src/support/types.h.orig  Sun May 27 17:40:58 2007
 +++ src/support/types.h   Sun May 27 17:40:58 2007
 @@ -23,7 +23,7 @@
  namespace lyx {
  
   /// The type used to hold characters in paragraphs
 -#if defined(HAVE_WCHAR_T)  SIZEOF_WCHAR_T == 4
 +#if 0
   // Prefer this if possible because GNU libstdc++ has usable
   // std::ctypewchar_t locale facets but not
   // std::ctypeboost::uint32_t. gcc older than 3.4 is also missing

 --- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
 +++ configure.ac2007-05-28 17:11:04.599949500 +0200
 @@ -356,11 +356,7 @@
  #endif
  #endif
  
 -#if defined(HAVE_OSTREAM)  defined(HAVE_LOCALE)  defined(HAVE_SSTREAM)
 -#  define USE_BOOST_FORMAT 1
 -#else
  #  define USE_BOOST_FORMAT 0
 -#endif
  
  #define BOOST_USER_CONFIG config.h


-- 
Enrico

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Jean-Marc Lasgouttes wrote:

 - do you know of any problem related to boost::format and wchat_t?

No. I remember some problems, but they were eventually solved, I believe by
switching from boost::uint32_t to wchar_t.

 - how can we test whether freebsd's wchat_t support is good enough?

First of all is wchar_t really 32 bit on freebsd? I know that some unices
have 16bit wchar_t, IIRC AIX. This could be tested with a small test
program that simply prints sizeof(wchar_t). I know that the configure test
is supposed to find that out, but who knows.

Then, by looking at the compiler and libstdc++ version. I see that Koji uses
gcc 4.1, this version works perfectly on linux and should have all needed
stuff in libstdc++.
I also see that it is installed in /usr/local. If that means that he
compiled himself from source then I would guess that something went wrong
with the configuration of the compiler. gcc has tons of configuration
options, and with many combinations you can get an unusable result (BTDT).

Then Koji could fire up a debugger, set a breakpoint in
parse_printf_directive() and report what the result of the narrow() call
is.

Another thing to try is to force Enricos facet implementation by applying
the attached patch. Does that change anything?

 - if it is not, what shall we do?

Develop a suitable test that can be used instead of the #if 1 in the
attached patch. But I doubt that this is the case (or if it is the case
then it is a bug of gcc and/or freebsd that should be reported).


GeorgIndex: src/support/docstring.cpp
===
--- src/support/docstring.cpp	(Revision 18558)
+++ src/support/docstring.cpp	(Arbeitskopie)
@@ -217,7 +217,8 @@ lyx::docstring  operator+=(lyx::docstri
 
 } // namespace lyx
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
+//#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
+#if 1
 
 // gcc does not have proper locale facets for lyx::char_type if
 // sizeof(wchar_t) == 2, so we have to implement them on our own.

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Tue, May 29, 2007 at 11:20:45AM +0900, Koji Yokota wrote:
> Enrico Forestieri wrote:
> > I don't understand. Seems that you are still using wchar_t ...
> > This crash is very similar to the one I was getting on cygwin after the
> > switch to unicode. I remember that someway I was able to fix it by
> > using the STLport library, before the real fix by Georg. Have a look
> > at this thread (may be you will get some ideas on how to proceed):
> > http://thread.gmane.org/gmane.editors.lyx.devel/68855
> 
> Enrico, I'm now checking this but before that I send a quick result of 
> the patches. I applied the attached patches but unfortunately there was 
> no improvement. I stated that the initialization succeeded in the 
> previous mail, but it was due to the fact that I already had a user 
> setting directory $HOME/.lyx-svn . Removing it draws back to the same 
> result as before.

That is because of autoloading of the splash.lyx document.
I am running out of ideas here. I think that you will need advice from
a real expert such as Georg.

> # Does RC1 go with the following modification suggested before?
> 
> --- src/support/lstrings.cpp(revision 18398)
> +++ src/support/lstrings.cpp(working copy)
> @@ -21,6 +21,7 @@
>   #include 
>   #include 
> 
> +#undef USE_BOOST_FORMAT
>   #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
>   #if USE_BOOST_FORMAT
>   #include 

> --- src/support/docstring.h.orig  Sun May 27 17:26:19 2007
> +++ src/support/docstring.h   Sun May 27 17:27:27 2007
> @@ -98,7 +98,7 @@
>  } // namespace lyx
>  
>  
> -#if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) && 
> __GNUC__ == 3 && __GNUC_MINOR__ < 4
> +#if 1
>  // Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
>  namespace std {
>  
> --- src/support/docstream.cpp.origSun May 27 17:40:11 2007
> +++ src/support/docstream.cpp Sun May 27 17:40:11 2007
> @@ -322,7 +322,7 @@
>  
>  }
>  
> -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
> +#if 1
>  // We get undefined references to these virtual methods. This looks like
>  // a bug in gcc. The implementation here does not do anything useful, since
>  // it is overriden in iconv_codecvt_facet.
> --- src/support/docstring.cpp.origSun May 27 17:40:35 2007
> +++ src/support/docstring.cpp Sun May 27 17:40:35 2007
> @@ -217,7 +217,7 @@
>  
>  } // namespace lyx
>  
> -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
> +#if 1
>  
>  // gcc does not have proper locale facets for lyx::char_type if
>  // sizeof(wchar_t) == 2, so we have to implement them on our own.
> --- src/support/types.h.orig  Sun May 27 17:40:58 2007
> +++ src/support/types.h   Sun May 27 17:40:58 2007
> @@ -23,7 +23,7 @@
>  namespace lyx {
>  
>   /// The type used to hold characters in paragraphs
> -#if defined(HAVE_WCHAR_T) && SIZEOF_WCHAR_T == 4
> +#if 0
>   // Prefer this if possible because GNU libstdc++ has usable
>   // std::ctype locale facets but not
>   // std::ctype. gcc older than 3.4 is also missing

> --- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
> +++ configure.ac2007-05-28 17:11:04.599949500 +0200
> @@ -356,11 +356,7 @@
>  #endif
>  #endif
>  
> -#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM)
> -#  define USE_BOOST_FORMAT 1
> -#else
>  #  define USE_BOOST_FORMAT 0
> -#endif
>  
>  #define BOOST_USER_CONFIG 


-- 
Enrico

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Jean-Marc Lasgouttes wrote:

> - do you know of any problem related to boost::format and wchat_t?

No. I remember some problems, but they were eventually solved, I believe by
switching from boost::uint32_t to wchar_t.

> - how can we test whether freebsd's wchat_t support is good enough?

First of all is wchar_t really 32 bit on freebsd? I know that some unices
have 16bit wchar_t, IIRC AIX. This could be tested with a small test
program that simply prints sizeof(wchar_t). I know that the configure test
is supposed to find that out, but who knows.

Then, by looking at the compiler and libstdc++ version. I see that Koji uses
gcc 4.1, this version works perfectly on linux and should have all needed
stuff in libstdc++.
I also see that it is installed in /usr/local. If that means that he
compiled himself from source then I would guess that something went wrong
with the configuration of the compiler. gcc has tons of configuration
options, and with many combinations you can get an unusable result (BTDT).

Then Koji could fire up a debugger, set a breakpoint in
parse_printf_directive() and report what the result of the narrow() call
is.

Another thing to try is to force Enricos facet implementation by applying
the attached patch. Does that change anything?

> - if it is not, what shall we do?

Develop a suitable test that can be used instead of the #if 1 in the
attached patch. But I doubt that this is the case (or if it is the case
then it is a bug of gcc and/or freebsd that should be reported).


GeorgIndex: src/support/docstring.cpp
===
--- src/support/docstring.cpp	(Revision 18558)
+++ src/support/docstring.cpp	(Arbeitskopie)
@@ -217,7 +217,8 @@ lyx::docstring & operator+=(lyx::docstri
 
 } // namespace lyx
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+//#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+#if 1
 
 // gcc does not have proper locale facets for lyx::char_type if
 // sizeof(wchar_t) == 2, so we have to implement them on our own.

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Mon, May 28, 2007 at 10:20:57AM +0900, Koji Yokota wrote:

 Peter Kümmel wrote:
  Hasn't Abdel fixed it today? Maybe updating helps.
 
 Yeah, right. With a newer version, Lyx at least proceeds with a 
 successful initialization and its GUI appears. However, trial of File - 
 New, File - Open, or start of lyx with a parameter like 'lyx -dbg any' 
 leads to the same crash as before:
 
   terminate called after throwing an instance of
   'boost::io::bad_format_string'
 what():  boost::bad_format_string: format-string is ill-formed
  
   Program received signal SIGABRT, Aborted.
   0x29934897 in kill () from /lib/libc.so.6
 
 When it crashed with File - New, the backtrace is as follows:

I don't understand. Seems that you are still using wchar_t ...
This crash is very similar to the one I was getting on cygwin after the
switch to unicode. I remember that someway I was able to fix it by
using the STLport library, before the real fix by Georg. Have a look
at this thread (may be you will get some ideas on how to proceed):
http://thread.gmane.org/gmane.editors.lyx.devel/68855

You could also try forcing USE_BOOST_FORMAT to 0 by hacking configure.ac
as follows:

--- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
+++ configure.ac2007-05-28 17:11:04.599949500 +0200
@@ -356,11 +356,7 @@
 #endif
 #endif
 
-#if defined(HAVE_OSTREAM)  defined(HAVE_LOCALE)  defined(HAVE_SSTREAM)
-#  define USE_BOOST_FORMAT 1
-#else
 #  define USE_BOOST_FORMAT 0
-#endif
 
 #define BOOST_USER_CONFIG config.h


-- 
Enrico

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Enrico Forestieri wrote:

I don't understand. Seems that you are still using wchar_t ...
This crash is very similar to the one I was getting on cygwin after the
switch to unicode. I remember that someway I was able to fix it by
using the STLport library, before the real fix by Georg. Have a look
at this thread (may be you will get some ideas on how to proceed):
http://thread.gmane.org/gmane.editors.lyx.devel/68855


Enrico, I'm now checking this but before that I send a quick result of 
the patches. I applied the attached patches but unfortunately there was 
no improvement. I stated that the initialization succeeded in the 
previous mail, but it was due to the fact that I already had a user 
setting directory $HOME/.lyx-svn . Removing it draws back to the same 
result as before.


Koji

# Does RC1 go with the following modification suggested before?

--- src/support/lstrings.cpp(revision 18398)
+++ src/support/lstrings.cpp(working copy)
@@ -21,6 +21,7 @@
 #include boost/tokenizer.hpp
 #include boost/assert.hpp

+#undef USE_BOOST_FORMAT
 #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
 #if USE_BOOST_FORMAT
 #include boost/format.hpp
--- src/support/docstring.h.origSun May 27 17:26:19 2007
+++ src/support/docstring.h Sun May 27 17:27:27 2007
@@ -98,7 +98,7 @@
 } // namespace lyx
 
 
-#if SIZEOF_WCHAR_T != 4  defined(__GNUC__)  defined(__GNUC_MINOR__)  
__GNUC__ == 3  __GNUC_MINOR__  4
+#if 1
 // Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
 namespace std {
 
--- src/support/docstream.cpp.orig  Sun May 27 17:40:11 2007
+++ src/support/docstream.cpp   Sun May 27 17:40:11 2007
@@ -322,7 +322,7 @@
 
 }
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
+#if 1
 // We get undefined references to these virtual methods. This looks like
 // a bug in gcc. The implementation here does not do anything useful, since
 // it is overriden in iconv_codecvt_facet.
--- src/support/docstring.cpp.orig  Sun May 27 17:40:35 2007
+++ src/support/docstring.cpp   Sun May 27 17:40:35 2007
@@ -217,7 +217,7 @@
 
 } // namespace lyx
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4)  defined(__GNUC__)
+#if 1
 
 // gcc does not have proper locale facets for lyx::char_type if
 // sizeof(wchar_t) == 2, so we have to implement them on our own.
--- src/support/types.h.origSun May 27 17:40:58 2007
+++ src/support/types.h Sun May 27 17:40:58 2007
@@ -23,7 +23,7 @@
 namespace lyx {
 
/// The type used to hold characters in paragraphs
-#if defined(HAVE_WCHAR_T)  SIZEOF_WCHAR_T == 4
+#if 0
// Prefer this if possible because GNU libstdc++ has usable
// std::ctypewchar_t locale facets but not
// std::ctypeboost::uint32_t. gcc older than 3.4 is also missing
--- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
+++ configure.ac2007-05-28 17:11:04.599949500 +0200
@@ -356,11 +356,7 @@
 #endif
 #endif
 
-#if defined(HAVE_OSTREAM)  defined(HAVE_LOCALE)  defined(HAVE_SSTREAM)
-#  define USE_BOOST_FORMAT 1
-#else
 #  define USE_BOOST_FORMAT 0
-#endif
 
 #define BOOST_USER_CONFIG config.h

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Mon, May 28, 2007 at 10:20:57AM +0900, Koji Yokota wrote:

> Peter Kümmel wrote:
> > Hasn't Abdel fixed it today? Maybe updating helps.
> 
> Yeah, right. With a newer version, Lyx at least proceeds with a 
> successful initialization and its GUI appears. However, trial of File -> 
> New, File -> Open, or start of lyx with a parameter like 'lyx -dbg any' 
> leads to the same crash as before:
> 
>  > terminate called after throwing an instance of
>  > 'boost::io::bad_format_string'
>  >   what():  boost::bad_format_string: format-string is ill-formed
>  >
>  > Program received signal SIGABRT, Aborted.
>  > 0x29934897 in kill () from /lib/libc.so.6
> 
> When it crashed with File -> New, the backtrace is as follows:

I don't understand. Seems that you are still using wchar_t ...
This crash is very similar to the one I was getting on cygwin after the
switch to unicode. I remember that someway I was able to fix it by
using the STLport library, before the real fix by Georg. Have a look
at this thread (may be you will get some ideas on how to proceed):
http://thread.gmane.org/gmane.editors.lyx.devel/68855

You could also try forcing USE_BOOST_FORMAT to 0 by hacking configure.ac
as follows:

--- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
+++ configure.ac2007-05-28 17:11:04.599949500 +0200
@@ -356,11 +356,7 @@
 #endif
 #endif
 
-#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM)
-#  define USE_BOOST_FORMAT 1
-#else
 #  define USE_BOOST_FORMAT 0
-#endif
 
 #define BOOST_USER_CONFIG 


-- 
Enrico

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Enrico Forestieri wrote:

I don't understand. Seems that you are still using wchar_t ...
This crash is very similar to the one I was getting on cygwin after the
switch to unicode. I remember that someway I was able to fix it by
using the STLport library, before the real fix by Georg. Have a look
at this thread (may be you will get some ideas on how to proceed):
http://thread.gmane.org/gmane.editors.lyx.devel/68855


Enrico, I'm now checking this but before that I send a quick result of 
the patches. I applied the attached patches but unfortunately there was 
no improvement. I stated that the initialization succeeded in the 
previous mail, but it was due to the fact that I already had a user 
setting directory $HOME/.lyx-svn . Removing it draws back to the same 
result as before.


Koji

# Does RC1 go with the following modification suggested before?

--- src/support/lstrings.cpp(revision 18398)
+++ src/support/lstrings.cpp(working copy)
@@ -21,6 +21,7 @@
 #include 
 #include 

+#undef USE_BOOST_FORMAT
 #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
 #if USE_BOOST_FORMAT
 #include 
--- src/support/docstring.h.origSun May 27 17:26:19 2007
+++ src/support/docstring.h Sun May 27 17:27:27 2007
@@ -98,7 +98,7 @@
 } // namespace lyx
 
 
-#if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) && 
__GNUC__ == 3 && __GNUC_MINOR__ < 4
+#if 1
 // Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
 namespace std {
 
--- src/support/docstream.cpp.orig  Sun May 27 17:40:11 2007
+++ src/support/docstream.cpp   Sun May 27 17:40:11 2007
@@ -322,7 +322,7 @@
 
 }
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+#if 1
 // We get undefined references to these virtual methods. This looks like
 // a bug in gcc. The implementation here does not do anything useful, since
 // it is overriden in iconv_codecvt_facet.
--- src/support/docstring.cpp.orig  Sun May 27 17:40:35 2007
+++ src/support/docstring.cpp   Sun May 27 17:40:35 2007
@@ -217,7 +217,7 @@
 
 } // namespace lyx
 
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+#if 1
 
 // gcc does not have proper locale facets for lyx::char_type if
 // sizeof(wchar_t) == 2, so we have to implement them on our own.
--- src/support/types.h.origSun May 27 17:40:58 2007
+++ src/support/types.h Sun May 27 17:40:58 2007
@@ -23,7 +23,7 @@
 namespace lyx {
 
/// The type used to hold characters in paragraphs
-#if defined(HAVE_WCHAR_T) && SIZEOF_WCHAR_T == 4
+#if 0
// Prefer this if possible because GNU libstdc++ has usable
// std::ctype locale facets but not
// std::ctype. gcc older than 3.4 is also missing
--- configure.ac.orig   2007-05-28 17:10:24.952949000 +0200
+++ configure.ac2007-05-28 17:11:04.599949500 +0200
@@ -356,11 +356,7 @@
 #endif
 #endif
 
-#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM)
-#  define USE_BOOST_FORMAT 1
-#else
 #  define USE_BOOST_FORMAT 0
-#endif
 
 #define BOOST_USER_CONFIG 

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Jean-Marc Lasgouttes wrote:

Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:


Abdelrazak Maybe Koji could try to enable that unconditionally:

Abdelrazak - #if SIZEOF_WCHAR_T != 4  defined(__GNUC__) 
Abdelrazak defined(__GNUC_MINOR__)  __GNUC__ == 3  __GNUC_MINOR__
Abdelrazak  4 + #if 1

Koji, could you try that?


OK, I just send the result. I tried this modification on the files below:


src/support/docstream.cpp
src/support/docstring.cpp
src/support/docstring.h
src/support/types.h


However, lyx (today's svn) crashed even before it reaches to main() 
function with the following message:


 Program received signal SIGSEGV, Segmentation fault.
 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
 ostream.tcc:55

Backtrace at the crash time seems corrupt, but initial layers contain:

 #0  0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
 ostream.tcc:55
 #1  0x0807c972 in std::operator std::char_traitschar  (
 Ba[EMAIL PROTECTED], __s=0x879fcd1 Assertion triggered in )
 at ostream.tcc:738
 #2  0x08088a48 in boost::assertion_failed (expr=0x87d28ed
 singleton_,
 function=0x87d4680 static lyx::LyX lyx::LyX::ref(),
 file=0x87d2860 LyX.cpp, line=211) at boost.cpp:54
 #3  0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211
 #4  0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546
 #5  0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34
 #6  0x0845b170 in __static_initialization_and_destruction_0 (
 __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117
 #7  0x08467ded in global constructors keyed to
 _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE ()
 at InsetListingsParams.cpp:658
 #8  0x0879e0f5 in __do_global_ctors_aux ()
 #9  0x0806c891 in _init ()
 #10 0x0206 in ?? ()
 #11 0x0001 in ?? ()... and so on

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Koji Yokota wrote:
 Jean-Marc Lasgouttes wrote:
 Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:

 Abdelrazak Maybe Koji could try to enable that unconditionally:

 Abdelrazak - #if SIZEOF_WCHAR_T != 4  defined(__GNUC__) 
 Abdelrazak defined(__GNUC_MINOR__)  __GNUC__ == 3  __GNUC_MINOR__
 Abdelrazak  4 + #if 1

 Koji, could you try that?
 
 OK, I just send the result. I tried this modification on the files below:
 
 src/support/docstream.cpp
 src/support/docstring.cpp
 src/support/docstring.h
 src/support/types.h
 
 However, lyx (today's svn) crashed even before it reaches to main()
 function with the following message:

Hasn't Abdel fixed it today? Maybe updating helps.

 
 Program received signal SIGSEGV, Segmentation fault.
 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
 ostream.tcc:55
 
 Backtrace at the crash time seems corrupt, but initial layers contain:
 
 #0  0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
 ostream.tcc:55
 #1  0x0807c972 in std::operator std::char_traitschar  (
 Ba[EMAIL PROTECTED], __s=0x879fcd1 Assertion triggered in )
 at ostream.tcc:738
 #2  0x08088a48 in boost::assertion_failed (expr=0x87d28ed
 singleton_,
 function=0x87d4680 static lyx::LyX lyx::LyX::ref(),
 file=0x87d2860 LyX.cpp, line=211) at boost.cpp:54
 #3  0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211
 #4  0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546
 #5  0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34
 #6  0x0845b170 in __static_initialization_and_destruction_0 (
 __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117
 #7  0x08467ded in global constructors keyed to
 _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE ()
 at InsetListingsParams.cpp:658
 #8  0x0879e0f5 in __do_global_ctors_aux ()
 #9  0x0806c891 in _init ()
 #10 0x0206 in ?? ()
 #11 0x0001 in ?? ()... and so on
 
 Koji
 


-- 
Peter Kümmel

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Peter Kümmel wrote:

Hasn't Abdel fixed it today? Maybe updating helps.


Yeah, right. With a newer version, Lyx at least proceeds with a 
successful initialization and its GUI appears. However, trial of File - 
New, File - Open, or start of lyx with a parameter like 'lyx -dbg any' 
leads to the same crash as before:


 terminate called after throwing an instance of
 'boost::io::bad_format_string'
   what():  boost::bad_format_string: format-string is ill-formed

 Program received signal SIGABRT, Aborted.
 0x29934897 in kill () from /lib/libc.so.6

When it crashed with File - New, the backtrace is as follows:

#0  0x29934897 in kill () from /lib/libc.so.6
#1  0x298474f7 in raise () from /lib/libpthread.so.2
#2  0x299332fb in abort () from /lib/libc.so.6
#3  0x297e9dc4 in __gnu_cxx::__verbose_terminate_handler ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#4  0x297e773a in __cxxabiv1::__terminate ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#5  0x297e7782 in std::terminate ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#6  0x297e78ba in __cxa_throw () from 
/usr/local/lib/gcc-4.1.3/libstdc++.so.6
#7  0x08142c5c in boost::throw_exceptionboost::io::bad_format_string 
(e=Could not find the frame base for void 
boost::throw_exceptionboost::io::bad_format_string(boost::io::bad_format_string 
const).

)
at throw_exception.hpp:39
#8  0x08142e65 in boost::io::detail::maybe_throw_exception (
exceptions=255 '', pos=7, size=10) at parsing.hpp:106
#9  0x081439a6 in boost::io::detail::parse_printf_directivewchar_t, 
std::char_traitswchar_t, std::allocatorwchar_t, 
__gnu_cxx::__normal_iteratorwchar_t const*, std::basic_stringwchar_t, 
std::char_traitswchar_t, std::allocatorwchar_t  , 
std::ctypewchar_t  ([EMAIL PROTECTED], [EMAIL PROTECTED],

fpar=0x8fda700, [EMAIL PROTECTED], offset=7, exceptions=255 '')
at parsing.hpp:333
#10 0x0814585f in boost::basic_formatwchar_t, 
std::char_traitswchar_t, std::allocatorwchar_t ::parse 
(this=0xbfbfbf94, [EMAIL PROTECTED])

at parsing.hpp:437
#11 0x08145c76 in basic_format (this=0xbfbfbf94, [EMAIL PROTECTED])
at format_implementation.hpp:64
#12 0x0875c167 in lyx::support::bformatstd::basic_stringwchar_t, 
std::char_traitswchar_t, std::allocatorwchar_t   ([EMAIL PROTECTED], 
[EMAIL PROTECTED])

at lstrings.cpp:875
#13 0x08309eeb in lyx::Text::currentState (this=0x8f375e0, [EMAIL PROTECTED])
at Text.cpp:1759
#14 0x08179b70 in lyx::Cursor::currentState (this=0x8af3e08)
at Cursor.cpp:1224
#15 0x0826c775 in lyx::LyXFunc::viewStatusMessage (this=0x8aca000)
at LyXFunc.cpp:2126
#16 0x0856c147 in lyx::frontend::GuiView::updateStatusBar (this=0x8cb3200)
at GuiView.cpp:590
#17 0x0850d2c6 in lyx::LyXView::setBuffer (this=0x8cb3214, b=0x8f0d200)
at LyXView.cpp:157
#18 0x082705d1 in lyx::LyXFunc::menuNew (this=0x8aca000, [EMAIL PROTECTED],
fromTemplate=false) at LyXFunc.cpp:1911
#19 0x082741d2 in lyx::LyXFunc::dispatch (this=0x8aca000, [EMAIL PROTECTED])
at LyXFunc.cpp:875
#20 0x0824f5e9 in lyx::dispatch ([EMAIL PROTECTED]) at LyX.cpp:1463
#21 0x0850b037 in lyx::LyXView::dispatch (this=0x8cb3214, [EMAIL PROTECTED])
at LyXView.cpp:448
#22 0x0869edab in lyx::frontend::Action::action (this=0x8db9580)
at Action.cpp:90
#23 0x0869ee2e in lyx::frontend::Action::qt_metacall (this=0x8db9580,
_c=QMetaObject::InvokeMetaMethod, _id=1, _a=0xbfbfd6b8)
at Action_moc.cpp:69
#24 0x292f0d43 in QMetaObject::activate (sender=0x8db9580,
from_signal_index=5, to_signal_index=6, argv=0xbfbfd6b8)
at kernel/qobject.cpp:2937
#25 0x292f0ec3 in QMetaObject::activate (sender=0x0, m=0x0,
from_local_signal_index=1, to_local_signal_index=2, argv=0x14487)
at kernel/qobject.cpp:2992
#26 0x28b607b3 in QAction::triggered (this=0x0, _t1=false)
at moc_qaction.cpp:208
#27 0x28b60b5c in QAction::activate (this=0x8db9580, event=QAction::Trigger)
at kernel/qaction.cpp:1075
#28 0x28e6bcca in QMenuPrivate::activateAction (this=0x8cf8a00,
action=0x8db9580, action_e=QAction::Trigger) at widgets/qmenu.cpp:756
#29 0x28e6e7fd in QMenu::keyPressEvent (this=0x8ce4680, e=0xbfbfdf50)
at widgets/qmenu.cpp:2266
#30 0x28bac58d in QWidget::event (this=0x8ce4680, event=0xbfbfdf50)
at kernel/qwidget.cpp:5753
#31 0x28e699e4 in QMenu::event (this=0x8ce4680, e=0xbfbfdf50)
at widgets/qmenu.cpp:1901
#32 0x28b665f4 in QApplicationPrivate::notify_helper (this=0x8acb000,
receiver=0x8ce4680, e=0xbfbfdf50) at kernel/qapplication.cpp:3439
#33 0x28b66dc7 in QApplication::notify (this=0x8ad1000, receiver=0x8ce4680,
e=0xbfbfdf50) at kernel/qapplication.cpp:3067
#34 0x0855d691 in lyx::frontend::GuiApplication::notify (this=0x8ad1000,
receiver=0x8ce4680, event=0xbfbfdf50) at GuiApplication.cpp:237
#35 0x28bb33c7 in qt_sendSpontaneousEvent (receiver=0x14487, event=0x14487)
at qcoreapplication.h:186
#36 0x28be256c in QKeyMapper::sendKeyEvent (keyWidget=0x8ce4680, 
grab=false,
type=QEvent::KeyPress, code=78, modifiers=Error 

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Jean-Marc Lasgouttes wrote:

"Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:


Abdelrazak> Maybe Koji could try to enable that unconditionally:

Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) &&
Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__
Abdelrazak> < 4 + #if 1

Koji, could you try that?


OK, I just send the result. I tried this modification on the files below:


src/support/docstream.cpp
src/support/docstring.cpp
src/support/docstring.h
src/support/types.h


However, lyx (today's svn) crashed even before it reaches to main() 
function with the following message:


> Program received signal SIGSEGV, Segmentation fault.
> 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
> ostream.tcc:55

Backtrace at the crash time seems corrupt, but initial layers contain:

> #0  0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
> ostream.tcc:55
> #1  0x0807c972 in std::operator<<  (
> Ba[EMAIL PROTECTED], __s=0x879fcd1 "Assertion triggered in ")
> at ostream.tcc:738
> #2  0x08088a48 in boost::assertion_failed (expr=0x87d28ed
> "singleton_",
> function=0x87d4680 "static lyx::LyX& lyx::LyX::ref()",
> file=0x87d2860 "LyX.cpp", line=211) at boost.cpp:54
> #3  0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211
> #4  0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546
> #5  0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34
> #6  0x0845b170 in __static_initialization_and_destruction_0 (
> __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117
> #7  0x08467ded in global constructors keyed to
> _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE ()
> at InsetListingsParams.cpp:658
> #8  0x0879e0f5 in __do_global_ctors_aux ()
> #9  0x0806c891 in _init ()
> #10 0x0206 in ?? ()
> #11 0x0001 in ?? ()... and so on

Koji

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Koji Yokota wrote:
> Jean-Marc Lasgouttes wrote:
>>> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:
>>
>> Abdelrazak> Maybe Koji could try to enable that unconditionally:
>>
>> Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) &&
>> Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__
>> Abdelrazak> < 4 + #if 1
>>
>> Koji, could you try that?
> 
> OK, I just send the result. I tried this modification on the files below:
> 
>> src/support/docstream.cpp
>> src/support/docstring.cpp
>> src/support/docstring.h
>> src/support/types.h
> 
> However, lyx (today's svn) crashed even before it reaches to main()
> function with the following message:

Hasn't Abdel fixed it today? Maybe updating helps.

> 
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
>> ostream.tcc:55
> 
> Backtrace at the crash time seems corrupt, but initial layers contain:
> 
>> #0  0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at
>> ostream.tcc:55
>> #1  0x0807c972 in std::operator<<  (
>> Ba[EMAIL PROTECTED], __s=0x879fcd1 "Assertion triggered in ")
>> at ostream.tcc:738
>> #2  0x08088a48 in boost::assertion_failed (expr=0x87d28ed
>> "singleton_",
>> function=0x87d4680 "static lyx::LyX& lyx::LyX::ref()",
>> file=0x87d2860 "LyX.cpp", line=211) at boost.cpp:54
>> #3  0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211
>> #4  0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546
>> #5  0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34
>> #6  0x0845b170 in __static_initialization_and_destruction_0 (
>> __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117
>> #7  0x08467ded in global constructors keyed to
>> _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE ()
>> at InsetListingsParams.cpp:658
>> #8  0x0879e0f5 in __do_global_ctors_aux ()
>> #9  0x0806c891 in _init ()
>> #10 0x0206 in ?? ()
>> #11 0x0001 in ?? ()... and so on
> 
> Koji
> 


-- 
Peter Kümmel

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Peter Kümmel wrote:

Hasn't Abdel fixed it today? Maybe updating helps.


Yeah, right. With a newer version, Lyx at least proceeds with a 
successful initialization and its GUI appears. However, trial of File -> 
New, File -> Open, or start of lyx with a parameter like 'lyx -dbg any' 
leads to the same crash as before:


> terminate called after throwing an instance of
> 'boost::io::bad_format_string'
>   what():  boost::bad_format_string: format-string is ill-formed
>
> Program received signal SIGABRT, Aborted.
> 0x29934897 in kill () from /lib/libc.so.6

When it crashed with File -> New, the backtrace is as follows:

#0  0x29934897 in kill () from /lib/libc.so.6
#1  0x298474f7 in raise () from /lib/libpthread.so.2
#2  0x299332fb in abort () from /lib/libc.so.6
#3  0x297e9dc4 in __gnu_cxx::__verbose_terminate_handler ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#4  0x297e773a in __cxxabiv1::__terminate ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#5  0x297e7782 in std::terminate ()
   from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
#6  0x297e78ba in __cxa_throw () from 
/usr/local/lib/gcc-4.1.3/libstdc++.so.6
#7  0x08142c5c in boost::throw_exception 
(e=Could not find the frame base for "void 
boost::throw_exception(boost::io::bad_format_string 
const&)".

)
at throw_exception.hpp:39
#8  0x08142e65 in boost::io::detail::maybe_throw_exception (
exceptions=255 '', pos=7, size=10) at parsing.hpp:106
#9  0x081439a6 in boost::io::detail::parse_printf_directive >, 
std::ctype > ([EMAIL PROTECTED], [EMAIL PROTECTED],

fpar=0x8fda700, [EMAIL PROTECTED], offset=7, exceptions=255 '')
at parsing.hpp:333
#10 0x0814585f in boost::basic_format::parse 
(this=0xbfbfbf94, [EMAIL PROTECTED])

at parsing.hpp:437
#11 0x08145c76 in basic_format (this=0xbfbfbf94, [EMAIL PROTECTED])
at format_implementation.hpp:64
#12 0x0875c167 in lyx::support::bformat > ([EMAIL PROTECTED], 
[EMAIL PROTECTED])

at lstrings.cpp:875
#13 0x08309eeb in lyx::Text::currentState (this=0x8f375e0, [EMAIL PROTECTED])
at Text.cpp:1759
#14 0x08179b70 in lyx::Cursor::currentState (this=0x8af3e08)
at Cursor.cpp:1224
#15 0x0826c775 in lyx::LyXFunc::viewStatusMessage (this=0x8aca000)
at LyXFunc.cpp:2126
#16 0x0856c147 in lyx::frontend::GuiView::updateStatusBar (this=0x8cb3200)
at GuiView.cpp:590
#17 0x0850d2c6 in lyx::LyXView::setBuffer (this=0x8cb3214, b=0x8f0d200)
at LyXView.cpp:157
#18 0x082705d1 in lyx::LyXFunc::menuNew (this=0x8aca000, [EMAIL PROTECTED],
fromTemplate=false) at LyXFunc.cpp:1911
#19 0x082741d2 in lyx::LyXFunc::dispatch (this=0x8aca000, [EMAIL PROTECTED])
at LyXFunc.cpp:875
#20 0x0824f5e9 in lyx::dispatch ([EMAIL PROTECTED]) at LyX.cpp:1463
#21 0x0850b037 in lyx::LyXView::dispatch (this=0x8cb3214, [EMAIL PROTECTED])
at LyXView.cpp:448
#22 0x0869edab in lyx::frontend::Action::action (this=0x8db9580)
at Action.cpp:90
#23 0x0869ee2e in lyx::frontend::Action::qt_metacall (this=0x8db9580,
_c=QMetaObject::InvokeMetaMethod, _id=1, _a=0xbfbfd6b8)
at Action_moc.cpp:69
#24 0x292f0d43 in QMetaObject::activate (sender=0x8db9580,
from_signal_index=5, to_signal_index=6, argv=0xbfbfd6b8)
at kernel/qobject.cpp:2937
#25 0x292f0ec3 in QMetaObject::activate (sender=0x0, m=0x0,
from_local_signal_index=1, to_local_signal_index=2, argv=0x14487)
at kernel/qobject.cpp:2992
#26 0x28b607b3 in QAction::triggered (this=0x0, _t1=false)
at moc_qaction.cpp:208
#27 0x28b60b5c in QAction::activate (this=0x8db9580, event=QAction::Trigger)
at kernel/qaction.cpp:1075
#28 0x28e6bcca in QMenuPrivate::activateAction (this=0x8cf8a00,
action=0x8db9580, action_e=QAction::Trigger) at widgets/qmenu.cpp:756
#29 0x28e6e7fd in QMenu::keyPressEvent (this=0x8ce4680, e=0xbfbfdf50)
at widgets/qmenu.cpp:2266
#30 0x28bac58d in QWidget::event (this=0x8ce4680, event=0xbfbfdf50)
at kernel/qwidget.cpp:5753
#31 0x28e699e4 in QMenu::event (this=0x8ce4680, e=0xbfbfdf50)
at widgets/qmenu.cpp:1901
#32 0x28b665f4 in QApplicationPrivate::notify_helper (this=0x8acb000,
receiver=0x8ce4680, e=0xbfbfdf50) at kernel/qapplication.cpp:3439
#33 0x28b66dc7 in QApplication::notify (this=0x8ad1000, receiver=0x8ce4680,
e=0xbfbfdf50) at kernel/qapplication.cpp:3067
#34 0x0855d691 in lyx::frontend::GuiApplication::notify (this=0x8ad1000,
receiver=0x8ce4680, event=0xbfbfdf50) at GuiApplication.cpp:237
#35 0x28bb33c7 in qt_sendSpontaneousEvent (receiver=0x14487, event=0x14487)
at qcoreapplication.h:186
#36 0x28be256c in QKeyMapper::sendKeyEvent (keyWidget=0x8ce4680, 
grab=false,
type=QEvent::KeyPress, code=78, modifiers=Error accessing memory 
address 0x0: Bad address.

)
at kernel/qkeymapper_x11.cpp:1580
#37 0x28be289b in QKeyMapperPrivate::translateKeyEvent 

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Jean-Marc Lasgouttes wrote:
 Peter == Peter Kümmel [EMAIL PROTECTED] writes:
 
 Peter We could just wait for 1.34.1, upgrading to 1.34 is nearly a
 Peter noop:
 
 Peter http://www.lyx.org/trac/changeset/18458
 
 I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
 released.
 
 JMarc
 

OK, I start updating ...

Peter Kümmel

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Peter Kümmel wrote:
 Jean-Marc Lasgouttes wrote:
 Peter == Peter Kümmel [EMAIL PROTECTED] writes:
 Peter We could just wait for 1.34.1, upgrading to 1.34 is nearly a
 Peter noop:

 Peter http://www.lyx.org/trac/changeset/18458

 I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
 released.

 JMarc

 
 OK, I start updating ...
 

Done, all files (also filesystem) are now from 1.34.

Peter

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Peter == Peter Kümmel [EMAIL PROTECTED] writes:

Peter Done, all files (also filesystem) are now from 1.34.

Thanks.

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Jean-Marc Lasgouttes wrote:
>> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:
> 
> Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a
> Peter> noop:
> 
> Peter> http://www.lyx.org/trac/changeset/18458
> 
> I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
> released.
> 
> JMarc
> 

OK, I start updating ...

Peter Kümmel

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Peter Kümmel wrote:
> Jean-Marc Lasgouttes wrote:
>>> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:
>> Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a
>> Peter> noop:
>>
>> Peter> http://www.lyx.org/trac/changeset/18458
>>
>> I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
>> released.
>>
>> JMarc
>>
> 
> OK, I start updating ...
> 

Done, all files (also filesystem) are now from 1.34.

Peter

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:

Peter> Done, all files (also filesystem) are now from 1.34.

Thanks.

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:

Abdelrazak Maybe Koji could try to enable that unconditionally:

Abdelrazak - #if SIZEOF_WCHAR_T != 4  defined(__GNUC__) 
Abdelrazak defined(__GNUC_MINOR__)  __GNUC__ == 3  __GNUC_MINOR__
Abdelrazak  4 + #if 1

Koji, could you try that?

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Thu, May 24, 2007 at 02:15:35PM +0200, Jean-Marc Lasgouttes wrote:
  Abdelrazak == Abdelrazak Younes [EMAIL PROTECTED] writes:
 
 Abdelrazak Maybe Koji could try to enable that unconditionally:
 
 Abdelrazak - #if SIZEOF_WCHAR_T != 4  defined(__GNUC__) 
 Abdelrazak defined(__GNUC_MINOR__)  __GNUC__ == 3  __GNUC_MINOR__
 Abdelrazak  4 + #if 1
 
 Koji, could you try that?

I think that he has to tweak several files, namely:
src/support/docstream.cpp
src/support/docstring.cpp
src/support/docstring.h
src/support/types.h

-- 
Enrico

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Georg Baum wrote:
 Jean-Marc Lasgouttes wrote:
 
 It was Georg who wrote

   Note that the iostreams library in boost 1.34.0 is an outdated
   version (older than the version in boost 1.33.1) with a memory leak,
   so watch for iostreams updates in the boost 1.34 branch, too.

 I am afraid I do not know what he means. Georg?
 
 See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265.
 This (and the very late 1.34.0 release) is a sign that boost currently has
 severe problems with testing/the release procedure. I hope they can sort
 this out for the next release.
 
 
 Georg
 

We could just wait for 1.34.1, upgrading to 1.34 is nearly a noop:

http://www.lyx.org/trac/changeset/18458

Peter

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Peter == Peter Kümmel [EMAIL PROTECTED] writes:

Peter We could just wait for 1.34.1, upgrading to 1.34 is nearly a
Peter noop:

Peter http://www.lyx.org/trac/changeset/18458

I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
released.

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:

Abdelrazak> Maybe Koji could try to enable that unconditionally:

Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) &&
Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__
Abdelrazak> < 4 + #if 1

Koji, could you try that?

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Thu, May 24, 2007 at 02:15:35PM +0200, Jean-Marc Lasgouttes wrote:
> > "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes:
> 
> Abdelrazak> Maybe Koji could try to enable that unconditionally:
> 
> Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) &&
> Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__
> Abdelrazak> < 4 + #if 1
> 
> Koji, could you try that?

I think that he has to tweak several files, namely:
src/support/docstream.cpp
src/support/docstring.cpp
src/support/docstring.h
src/support/types.h

-- 
Enrico

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Georg Baum wrote:
> Jean-Marc Lasgouttes wrote:
> 
>> It was Georg who wrote
>>
>>   Note that the iostreams library in boost 1.34.0 is an outdated
>>   version (older than the version in boost 1.33.1) with a memory leak,
>>   so watch for iostreams updates in the boost 1.34 branch, too.
>>
>> I am afraid I do not know what he means. Georg?
> 
> See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265.
> This (and the very late 1.34.0 release) is a sign that boost currently has
> severe problems with testing/the release procedure. I hope they can sort
> this out for the next release.
> 
> 
> Georg
> 

We could just wait for 1.34.1, upgrading to 1.34 is nearly a noop:

http://www.lyx.org/trac/changeset/18458

Peter

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:

Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a
Peter> noop:

Peter> http://www.lyx.org/trac/changeset/18458

I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is
released.

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Michael Gerz]

Jean-Marc Lasgouttes schrieb:

Peter After the release of 1.5, I assume.

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.
  

+1

Michael

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Peter == Peter Kümmel [EMAIL PROTECTED] writes:

Peter I've updated to boost 1.34 in a branch, and it's not that hard,
Peter it compiles and runs without problems. The changes are:

Very good, thanks.

Peter Should I update the header files? JMarc says in bugzilla that
Peter we should not update filesystem?

It was Georg who wrote 

  Note that the iostreams library in boost 1.34.0 is an outdated
  version (older than the version in boost 1.33.1) with a memory leak,
  so watch for iostreams updates in the boost 1.34 branch, too.

I am afraid I do not know what he means. Georg?

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Jean-Marc Lasgouttes wrote:

 It was Georg who wrote
 
   Note that the iostreams library in boost 1.34.0 is an outdated
   version (older than the version in boost 1.33.1) with a memory leak,
   so watch for iostreams updates in the boost 1.34 branch, too.
 
 I am afraid I do not know what he means. Georg?

See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265.
This (and the very late 1.34.0 release) is a sign that boost currently has
severe problems with testing/the release procedure. I hope they can sort
this out for the next release.


Georg

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Koji == Koji Yokota [EMAIL PROTECTED] writes:

Koji I attach the backtrace when the program is crashed as additional
Koji information.

Looking at the backtrace, the problem happens at line 333 of
boost/format/parsing.hpp:

switch ( wrap_narrow(fac, *start, 0) ) {
case 'X':
fpar-fmtstate_.flags_ |= std::ios_base::uppercase;
[...]
case 'n' :  
fpar-argN_ = format_item_t::argN_ignored;
break;
default: 
maybe_throw_exception(exceptions, start-start0+offset, 
fstring_size);  /// -- line 333 is here
}

This seems to point to a problem with using widen/narrow on a facet
like std::ctypewchar_t. Of course, I know next to nothing about that.
Enrico, would you have some ideas about whether freebsd has the needed
support? 

Was it freebsd that did not even have a wchar_t type some years ago?

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Wed, May 23, 2007 at 03:40:08PM +0200, Jean-Marc Lasgouttes wrote:

 This seems to point to a problem with using widen/narrow on a facet
 like std::ctypewchar_t. Of course, I know next to nothing about that.
 Enrico, would you have some ideas about whether freebsd has the needed
 support? 

I really don't know. I still have NetBSD installed on an old MC68040
machine but I don't have it handy now, and furthermore I haven't
updated it in the last 7 or 8 years...

 Was it freebsd that did not even have a wchar_t type some years ago?

When sizeof(wchar_t) != 4 or wchar_t is not present, then our facets
implementation should kick in. I use it with no problems in both cygwin
and mingw, don't know about other platforms.

-- 
Enrico

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Enrico == Enrico Forestieri [EMAIL PROTECTED] writes:

Enrico When sizeof(wchar_t) != 4 or wchar_t is not present, then our
Enrico facets implementation should kick in. I use it with no
Enrico problems in both cygwin and mingw, don't know about other
Enrico platforms.

Do you have a simple program to test whether the facet works? I seem
to remember that you did it at the time.

And can we force to use our implementation in some cases?

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Abdelrazak Younes]

Jean-Marc Lasgouttes wrote:

Enrico == Enrico Forestieri [EMAIL PROTECTED] writes:


Enrico When sizeof(wchar_t) != 4 or wchar_t is not present, then our
Enrico facets implementation should kick in. I use it with no
Enrico problems in both cygwin and mingw, don't know about other
Enrico platforms.

Do you have a simple program to test whether the facet works? I seem
to remember that you did it at the time.

And can we force to use our implementation in some cases?


The test is in docstring.h:101

#if SIZEOF_WCHAR_T != 4  defined(__GNUC__)  defined(__GNUC_MINOR__) 
 __GNUC__ == 3  __GNUC_MINOR__  4


Maybe Koji could try to enable that unconditionally:

- #if SIZEOF_WCHAR_T != 4  defined(__GNUC__)  
defined(__GNUC_MINOR__)  __GNUC__ == 3  __GNUC_MINOR__  4

+ #if 1

Abdel.

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Wed, May 23, 2007 at 05:20:52PM +0200, Jean-Marc Lasgouttes wrote:
  Enrico == Enrico Forestieri [EMAIL PROTECTED] writes:
 
 Enrico When sizeof(wchar_t) != 4 or wchar_t is not present, then our
 Enrico facets implementation should kick in. I use it with no
 Enrico problems in both cygwin and mingw, don't know about other
 Enrico platforms.
 
 Do you have a simple program to test whether the facet works? I seem
 to remember that you did it at the time.

I think that you are confusing me with Georg here. He had written
some tests at the time. In this field, he is the master, I'm only
an apprentice ;-)

http://thread.gmane.org/gmane.editors.lyx.devel/66786

 And can we force to use our implementation in some cases?

I think so, but the implementation must match the compiler internals,
that's why an explicit check for the gcc version is also made.

-- 
Enrico

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Michael Gerz]

Jean-Marc Lasgouttes schrieb:

Peter> After the release of 1.5, I assume.

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.
  

+1

Michael

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:

Peter> I've updated to boost 1.34 in a branch, and it's not that hard,
Peter> it compiles and runs without problems. The changes are:

Very good, thanks.

Peter> Should I update the header files? JMarc says in bugzilla that
Peter> we should not update filesystem?

It was Georg who wrote 

  Note that the iostreams library in boost 1.34.0 is an outdated
  version (older than the version in boost 1.33.1) with a memory leak,
  so watch for iostreams updates in the boost 1.34 branch, too.

I am afraid I do not know what he means. Georg?

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Georg Baum]

Jean-Marc Lasgouttes wrote:

> It was Georg who wrote
> 
>   Note that the iostreams library in boost 1.34.0 is an outdated
>   version (older than the version in boost 1.33.1) with a memory leak,
>   so watch for iostreams updates in the boost 1.34 branch, too.
> 
> I am afraid I do not know what he means. Georg?

See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265.
This (and the very late 1.34.0 release) is a sign that boost currently has
severe problems with testing/the release procedure. I hope they can sort
this out for the next release.


Georg

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes:

Koji> I attach the backtrace when the program is crashed as additional
Koji> information.

Looking at the backtrace, the problem happens at line 333 of
boost/format/parsing.hpp:

switch ( wrap_narrow(fac, *start, 0) ) {
case 'X':
fpar->fmtstate_.flags_ |= std::ios_base::uppercase;
[...]
case 'n' :  
fpar->argN_ = format_item_t::argN_ignored;
break;
default: 
maybe_throw_exception(exceptions, start-start0+offset, 
fstring_size);  /// <-- line 333 is here
}

This seems to point to a problem with using widen/narrow on a facet
like std::ctype. Of course, I know next to nothing about that.
Enrico, would you have some ideas about whether freebsd has the needed
support? 

Was it freebsd that did not even have a wchar_t type some years ago?

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Wed, May 23, 2007 at 03:40:08PM +0200, Jean-Marc Lasgouttes wrote:

> This seems to point to a problem with using widen/narrow on a facet
> like std::ctype. Of course, I know next to nothing about that.
> Enrico, would you have some ideas about whether freebsd has the needed
> support? 

I really don't know. I still have NetBSD installed on an old MC68040
machine but I don't have it handy now, and furthermore I haven't
updated it in the last 7 or 8 years...

> Was it freebsd that did not even have a wchar_t type some years ago?

When sizeof(wchar_t) != 4 or wchar_t is not present, then our facets
implementation should kick in. I use it with no problems in both cygwin
and mingw, don't know about other platforms.

-- 
Enrico

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes:

Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our
Enrico> facets implementation should kick in. I use it with no
Enrico> problems in both cygwin and mingw, don't know about other
Enrico> platforms.

Do you have a simple program to test whether the facet works? I seem
to remember that you did it at the time.

And can we force to use our implementation in some cases?

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Abdelrazak Younes]

Jean-Marc Lasgouttes wrote:

"Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes:


Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our
Enrico> facets implementation should kick in. I use it with no
Enrico> problems in both cygwin and mingw, don't know about other
Enrico> platforms.

Do you have a simple program to test whether the facet works? I seem
to remember that you did it at the time.

And can we force to use our implementation in some cases?


The test is in docstring.h:101

#if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) 
&& __GNUC__ == 3 && __GNUC_MINOR__ < 4


Maybe Koji could try to enable that unconditionally:

- #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && 
defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ < 4

+ #if 1

Abdel.

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Enrico Forestieri]

On Wed, May 23, 2007 at 05:20:52PM +0200, Jean-Marc Lasgouttes wrote:
> > "Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes:
> 
> Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our
> Enrico> facets implementation should kick in. I use it with no
> Enrico> problems in both cygwin and mingw, don't know about other
> Enrico> platforms.
> 
> Do you have a simple program to test whether the facet works? I seem
> to remember that you did it at the time.

I think that you are confusing me with Georg here. He had written
some tests at the time. In this field, he is the master, I'm only
an apprentice ;-)

http://thread.gmane.org/gmane.editors.lyx.devel/66786

> And can we force to use our implementation in some cases?

I think so, but the implementation must match the compiler internals,
that's why an explicit check for the gcc version is also made.

-- 
Enrico

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Peter == Peter Kümmel [EMAIL PROTECTED] writes:

 We should probably update boost first to 1.34 final, but I do not
 know who wants to do it.

Peter After the release of 1.5, I assume.

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[José Matos]

On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:
 Are we sure? The 1.5 branch would be better with the real 1.34, rather
 than a prerelease.

  I tend to agree with Jean-Marc here.

 JMarc

-- 
José Abílio

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Abdelrazak Younes]

José Matos wrote:

On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.


  I tend to agree with Jean-Marc here.


Me too.

Abdel.

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Abdelrazak Younes wrote:
 José Matos wrote:
 On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:
 Are we sure? The 1.5 branch would be better with the real 1.34, rather
 than a prerelease.

   I tend to agree with Jean-Marc here.
 
 Me too.
 
 Abdel.
 

I've updated to boost 1.34 in a branch, and it's not that hard,
it compiles and runs without problems.
The changes are:

$ svn ci -mcheck in changes of already exisiting files
Sendingboost/boost/bind/storage.hpp
Sendingboost/boost/config/no_tr1/utility.hpp
Sendingboost/boost/filesystem/cerrno.hpp
Sendingboost/boost/iostreams/detail/streambuf/indirect_streambuf.hpp
Sendingboost/boost/mpl/aux_/lambda_spec.hpp
Sendingboost/boost/optional/optional.hpp
Sendingboost/boost/preprocessor/config/config.hpp
Sendingboost/boost/regex/v4/cpp_regex_traits.hpp
Sendingboost/boost/regex/v4/regex_traits.hpp
Sendingboost/boost/spirit/utility/loops.hpp
Sendingboost/boost/type_traits/is_convertible.hpp

Sendingboost/libs/filesystem/src/operations.cpp
Sendingboost/libs/filesystem/src/path.cpp
Sendingboost/libs/filesystem/src/portability.cpp

and

renaming none_t.hpp to boost/boost/none.hpp


Should I update the header files? JMarc says in bugzilla
that we should not update filesystem?


Peter

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes:

>> We should probably update boost first to 1.34 final, but I do not
>> know who wants to do it.

Peter> After the release of 1.5, I assume.

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[José Matos]

On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:
> Are we sure? The 1.5 branch would be better with the real 1.34, rather
> than a prerelease.

  I tend to agree with Jean-Marc here.

> JMarc

-- 
José Abílio

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Abdelrazak Younes]

José Matos wrote:

On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:

Are we sure? The 1.5 branch would be better with the real 1.34, rather
than a prerelease.


  I tend to agree with Jean-Marc here.


Me too.

Abdel.

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Abdelrazak Younes wrote:
> José Matos wrote:
>> On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote:
>>> Are we sure? The 1.5 branch would be better with the real 1.34, rather
>>> than a prerelease.
>>
>>   I tend to agree with Jean-Marc here.
> 
> Me too.
> 
> Abdel.
> 

I've updated to boost 1.34 in a branch, and it's not that hard,
it compiles and runs without problems.
The changes are:

$ svn ci -m"check in changes of already exisiting files"
Sendingboost/boost/bind/storage.hpp
Sendingboost/boost/config/no_tr1/utility.hpp
Sendingboost/boost/filesystem/cerrno.hpp
Sendingboost/boost/iostreams/detail/streambuf/indirect_streambuf.hpp
Sendingboost/boost/mpl/aux_/lambda_spec.hpp
Sendingboost/boost/optional/optional.hpp
Sendingboost/boost/preprocessor/config/config.hpp
Sendingboost/boost/regex/v4/cpp_regex_traits.hpp
Sendingboost/boost/regex/v4/regex_traits.hpp
Sendingboost/boost/spirit/utility/loops.hpp
Sendingboost/boost/type_traits/is_convertible.hpp

Sendingboost/libs/filesystem/src/operations.cpp
Sendingboost/libs/filesystem/src/path.cpp
Sendingboost/libs/filesystem/src/portability.cpp

and

renaming none_t.hpp to boost/boost/none.hpp


Should I update the header files? JMarc says in bugzilla
that we should not update filesystem?


Peter

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Angus Leeming]

Peter Kümmel [EMAIL PROTECTED] writes:
  But we could disable boost format, so please try attached patch.

  Peter, with this patch, it seems the problem has resolved. lyx
  successfully finishes the initialization and starts up.

 Great.
 Hope you get not other problems because of iconv.

 We should integrate the disabling of boost's
 format code for FreeBSD into the build system.

Peter,

could you go the extra mile and ascertain exactly what boost dislikes here and 
then work with the boost team to fix their code. We make great use of boost 
and should feel honour bound to help squash those bugs we find.

Angus

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Angus == Angus Leeming [EMAIL PROTECTED] writes:

Angus could you go the extra mile and ascertain exactly what boost
Angus dislikes here and then work with the boost team to fix their
Angus code. We make great use of boost and should feel honour bound
Angus to help squash those bugs we find.

We should probably update boost first to 1.34 final, but I do not know
who wants to do it.

JMarc

PS: Feel like coming to Finland in August?

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Angus Leeming]

Jean-Marc Lasgouttes [EMAIL PROTECTED] writes:
 PS: Feel like coming to Finland in August?

Not really. Finland in August is much like Scotland in August; full of nasty, 
biting insects!

Angus (ducking from the wrath of M.V. ;-))

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

 Angus == Angus Leeming [EMAIL PROTECTED] writes:

Angus Jean-Marc Lasgouttes [EMAIL PROTECTED] writes:
 PS: Feel like coming to Finland in August?

Angus Not really. Finland in August is much like Scotland in August;
Angus full of nasty, biting insects!

You have a point there. And don't forget nasty, biting developers.

JMarc

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Martin Vermeer]

On Mon, May 21, 2007 at 09:32:22PM +, Angus Leeming wrote:
 Jean-Marc Lasgouttes [EMAIL PROTECTED] writes:
  PS: Feel like coming to Finland in August?
 
 Not really. Finland in August is much like Scotland in August; full of nasty, 
 biting insects!
 
 Angus (ducking from the wrath of M.V. ;-))

FUD, FUD, FUD... (how does one flame in Fahrenheit?)

- Martin
 

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Jean-Marc Lasgouttes wrote:
 Angus == Angus Leeming [EMAIL PROTECTED] writes:
 
 Angus could you go the extra mile and ascertain exactly what boost
 Angus dislikes here and then work with the boost team to fix their
 Angus code. We make great use of boost and should feel honour bound
 Angus to help squash those bugs we find.
 

The problem is that I couldn't reproduce it here, but when Koji Yokota
is willed to help me a bit I could give it a try.
But I'm still not 100% sure that it is a boost bug.

 We should probably update boost first to 1.34 final, but I do not know
 who wants to do it.

After the release of 1.5, I assume.

 
 JMarc
 
 PS: Feel like coming to Finland in August?
 
 
 
 
 


-- 
Peter Kümmel

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Angus Leeming]

Peter Kümmel <[EMAIL PROTECTED]> writes:
> >> But we could disable boost format, so please try attached patch.

> > Peter, with this patch, it seems the problem has resolved. lyx
> > successfully finishes the initialization and starts up.

> Great.
> Hope you get not other problems because of iconv.

> We should integrate the disabling of boost's
> format code for FreeBSD into the build system.

Peter,

could you go the extra mile and ascertain exactly what boost dislikes here and 
then work with the boost team to fix their code. We make great use of boost 
and should feel honour bound to help squash those bugs we find.

Angus

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes:

Angus> could you go the extra mile and ascertain exactly what boost
Angus> dislikes here and then work with the boost team to fix their
Angus> code. We make great use of boost and should feel honour bound
Angus> to help squash those bugs we find.

We should probably update boost first to 1.34 final, but I do not know
who wants to do it.

JMarc

PS: Feel like coming to Finland in August?

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Angus Leeming]

Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes:
> PS: Feel like coming to Finland in August?

Not really. Finland in August is much like Scotland in August; full of nasty, 
biting insects!

Angus (ducking from the wrath of M.V. ;-))

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Jean-Marc Lasgouttes]

> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes:

Angus> Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes:
>> PS: Feel like coming to Finland in August?

Angus> Not really. Finland in August is much like Scotland in August;
Angus> full of nasty, biting insects!

You have a point there. And don't forget nasty, biting developers.

JMarc

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Martin Vermeer]

On Mon, May 21, 2007 at 09:32:22PM +, Angus Leeming wrote:
> Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes:
> > PS: Feel like coming to Finland in August?
> 
> Not really. Finland in August is much like Scotland in August; full of nasty, 
> biting insects!
> 
> Angus (ducking from the wrath of M.V. ;-))

FUD, FUD, FUD... (how does one flame in Fahrenheit?)

- Martin
 

Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Jean-Marc Lasgouttes wrote:
>> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes:
> 
> Angus> could you go the extra mile and ascertain exactly what boost
> Angus> dislikes here and then work with the boost team to fix their
> Angus> code. We make great use of boost and should feel honour bound
> Angus> to help squash those bugs we find.
> 

The problem is that I couldn't reproduce it here, but when Koji Yokota
is willed to help me a bit I could give it a try.
But I'm still not 100% sure that it is a boost bug.

> We should probably update boost first to 1.34 final, but I do not know
> who wants to do it.

After the release of 1.5, I assume.

> 
> JMarc
> 
> PS: Feel like coming to Finland in August?
> 
> 
> 
> 
> 


-- 
Peter Kümmel

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Peter Kümmel wrote:

lyxerr  LyX: Creating directory %1$s  endl,
lyxerr  package().user_support().absFilename()  endl;
docstring s1 = _(LyX: Creating directory %1$s);
docstring s2 = from_utf8(package().user_support().absFilename());

//docstring s3 = bformat(_(LyX directory, %1$s. \n), /home/dir/);


Peter, OK, all tests succeeded up to this point, but


docstring s4= bformat(_(LyX directory, %1$s. \n), 
from_utf8(/home/dir/));


fails (and all the rest).

In this test, after the evaluation of from_utf8() part, it jumps to

docstring const _(string const  str)

in gettext.cpp (L34) and this function returns a value and successfully 
finishes. Then, it goes to


  docstring bformatdocstring(docstring const  fmt, docstring arg1)

and crashes when it evaluates the return line (L875 in lstrings.cpp).

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

We still don't know which string is wrong
so here the next code:

docstring format = _(LyX directory, %1$s. \n);
lyxerr  format =   to_utf8(format)  endl;

docstring path = from_utf8(/home/dir/);
lyxerr  path =   to_utf8(path)  endl;

docstring s4= bformat(format, path);


Here I get this output:

format = LyX directory, %1$s.

path = /home/dir/

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Koji Yokota]

Peter Kümmel wrote:

We still don't know which string is wrong
so here the next code:

docstring format = _(LyX directory, %1$s. \n);
lyxerr  format =   to_utf8(format)  endl;

docstring path = from_utf8(/home/dir/);
lyxerr  path =   to_utf8(path)  endl;

docstring s4= bformat(format, path);


OK, stderr is exactly the same:


format = LyX directory, %1$s.

path = /home/dir/


so seems from/to utf is working though I don't know the intermediate 
product is correct.


But

docstring s4= bformat(format, path);

still fails...

Koji

Re: UTF8-UCS4 failure on FreeBSD 6.2-RELEASE

[Peter Kümmel]

Koji Yokota wrote:
 Peter Kümmel wrote:
 We still don't know which string is wrong
 so here the next code:

 docstring format = _(LyX directory, %1$s. \n);
 lyxerr  format =   to_utf8(format)  endl;

 docstring path = from_utf8(/home/dir/);
 lyxerr  path =   to_utf8(path)  endl;

 docstring s4= bformat(format, path);
 
 OK, stderr is exactly the same:
 
 format = LyX directory, %1$s.

 path = /home/dir/
 
 so seems from/to utf is working though I don't know the intermediate
 product is correct.
 
 But
 
 docstring s4= bformat(format, path);

bformat is just a call to boost:

boost::basic_formatchar_type(fmt) % arg1).str();

So it is a boost bug when checking the format string,
or our string is somehow invalid. This does not help much.

But we could disable boost format, so please try attached patch.


 
 still fails...
 
 Koji
 


-- 
Peter Kümmel
Index: src/support/lstrings.cpp
===
--- src/support/lstrings.cpp(revision 18398)
+++ src/support/lstrings.cpp(working copy)
@@ -21,6 +21,7 @@
 #include boost/tokenizer.hpp
 #include boost/assert.hpp
 
+#undef USE_BOOST_FORMAT
 #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
 #if USE_BOOST_FORMAT
 #include boost/format.hpp