Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Then, is a sort of modification as the attached patch Koji> acceptable? It seems it solved the problem and is working fine! This particular patch does not look good, but we'll find one around this idea. Isn't there some define that we can test against to detect the faulty freebsd versions? I'm not quite sure whether _GLIBCPP_USE_WCHAR_T is set in forthcoming 7.0-release, but as far as judging from the documentation which comes with the newest gcc-4.2 which 7.0-release will adopt, it probably is not. If it is the case, all versions of FreeBSD at this moment do not support it. So simply look at the define __FreeBSD__? Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Then, is a sort of modification as the attached patch Koji> acceptable? It seems it solved the problem and is working fine! This particular patch does not look good, but we'll find one around this idea. Isn't there some define that we can test against to detect the faulty freebsd versions? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: What a shame. This problem is known since 2003, and still not fixed? Does nobody care for such a standard violation? It seems the fact that FreeBSD had an old implementation of a wchar_t equivalence before wchar_t was introduced made the problem complicated. I wish it is solved in the next major release. Anyway, since hacking libstdc++ is not an option for LyX, there is only one possible solution: Modify the configure check that checks for a usable wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code that is used on cygwin will be used on freebsd. That code is not as good as the native gcc code, but it is at least better than nothing. BTW, disabling boost::format would not help. That might fix the bformat problem, but the locale stuff that fails in boost::format is needed at other places, too (e.g. docstringstream). You can be lucky that boost::format crashed, the docstringstream would probably simply corrupt some data and proceed happily. Then, is a sort of modification as the attached patch acceptable? It seems it solved the problem and is working fine! Koji patch-m4-wchar_t.m4 Description: application/m4
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Am Dienstag, 12. Juni 2007 19:05 schrieb Koji Yokota: > There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, > which may still exist(?) as shown here: This did never occur as an option to me. Good detective work! > http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9 > > I see that in > /usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h > > #define _GLIBCXX_USE_WCHAR_T 1 > > is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for > system gcc3.4). What a shame. This problem is known since 2003, and still not fixed? Does nobody care for such a standard violation? Anyway, since hacking libstdc++ is not an option for LyX, there is only one possible solution: Modify the configure check that checks for a usable wchar_t to not set HAVE_WCHAR_T on freebsd. Then the same replacement code that is used on cygwin will be used on freebsd. That code is not as good as the native gcc code, but it is at least better than nothing. BTW, disabling boost::format would not help. That might fix the bformat problem, but the locale stuff that fails in boost::format is needed at other places, too (e.g. docstringstream). You can be lucky that boost::format crashed, the docstringstream would probably simply corrupt some data and proceed happily. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote: No, it doesn't compile. It complains that there is no matching function, which probablly means that in boost/boost/format/format_fwd.hpp: #if !defined(BOOST_NO_STD_WSTRING) && !defined(BOOST_NO_STD_WSTREAMBUF) \ && !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM) typedef basic_format wformat; #endif one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or BOOST_FORMAT_IGNORE_STRINGSTREAM is defined. There seems to be a problem in libstdc++ for FreeBSD to handle wchar_t, which may still exist(?) as shown here: http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#3_9 I see that in /usr/local/lib/gcc-4.1.3/include/c++/i386-portbld-freebsd6.2/bits/c++config.h #define _GLIBCXX_USE_WCHAR_T 1 is defined, but _GLIBCPP_USE_WCHAR_T is not found there (same holds for system gcc3.4). Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote:
Koji Yokota wrote:
Georg Baum wrote:
If you want to debug this fdurther it might be a good idea to write a
small standalone program that simply calls boost::format with the
problematic input.
boost::basic_format() itself seems working if it is called with
"ordinary" strings:
> #include
> #include
>
> using namespace std;
>
> main()
> {
> cout << boost::basic_format("test %1$s \n") % "OK";
> }
which prints out
test OK
That is expected, but does boost::basic_formatit also do that?
No, it doesn't compile. It complains that there is no matching function,
which probablly means that in boost/boost/format/format_fwd.hpp:
#if !defined(BOOST_NO_STD_WSTRING) && !defined(BOOST_NO_STD_WSTREAMBUF) \
&& !defined(BOOST_FORMAT_IGNORE_STRINGSTREAM)
typedef basic_format wformat;
#endif
one of BOOST_NO_STD_WSTRING, BOOST_NO_STD_WSTREAMBUF or
BOOST_FORMAT_IGNORE_STRINGSTREAM is defined.
> (gdb) p *start
> $38 = (const wchar_t &) @0x8b1e9c0: 49
49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.
That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.
Thank you. I will try to investigate it further.
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote: > So, this may have to be modified too. > >> > (gdb) p *start >> > $38 = (const wchar_t &) @0x8b1e9c0: 49 > > > (gdb) x/5s 0x8b1e9c0 > > 0x8b1e9c0: "1" > > 0x8b1e9c2: "" > > 0x8b1e9c3: "" > > 0x8b1e9c4: "$" > > 0x8b1e9c6: "" > > 0x8b1e9c7: "" > > 0x8b1e9c8: "s" > > 0x8b1e9ca: "" > > 0x8b1e9cb: "" > > 0x8b1e9cc: "\r0" > > 0x8b1e9cf: "" > > 0x8b1e9d0: "\2220" > > 0x8b1e9d3: "" > > 0x8b1e9d4: "\\O" > > 0x8b1e9d7: "" > > 0x8b1e9d8: "\020b" > > 0x8b1e9db: "" > > 0x8b1e9dc: "W0" > > 0x8b1e9df: "" > > 0x8b1e9e0: "f0" > > 0x8b1e9e3: "" > > 0x8b1e9e4: "D0" > > 0x8b1e9e7: "" > > 0x8b1e9e8: "~0" > > 0x8b1e9eb: "" > > where string size is > > > (gdb) p fstring_size > > $54 = 25 This looks a bit strange (can gdb not print wstrings?), but it seems to be a correct format string. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote:
> Georg Baum wrote:
>> If you want to debug this fdurther it might be a good idea to write a
>> small standalone program that simply calls boost::format with the
>> problematic input.
>
> boost::basic_format() itself seems working if it is called with
> "ordinary" strings:
>
> > #include
> > #include
> >
> > using namespace std;
> >
> > main()
> > {
> > cout << boost::basic_format("test %1$s \n") % "OK";
> > }
>
> which prints out
>
> test OK
That is expected, but does boost::basic_formatit also do that?
>> And what is the value of *start? And how does the whole format string
>> look like?
>
> They are as follows:
>
> > (gdb) p *start
> > $38 = (const wchar_t &) @0x8b1e9c0: 49
49 is the ASCII value of the character '1'. That means that wrap_narrow() is
definitely wrong here, since it should simply return the input in this case
and not -65. It also means that boost::format is probably correct, since
the 1 is a part of the format string that the code we are lloking at needs
to parse.
That means that indeed something is wrong with your locale stuff, but I
don't understand at all why this does not change if you use the replacement
facets from Enrico that definitely work for ASCII input. The only way to
track this down is to step through the wrap_narrow() function with a
debugger. Then you will eventually see where the 49 is converted to -65.
Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
So, this may have to be modified too. > (gdb) p *start > $38 = (const wchar_t &) @0x8b1e9c0: 49 > (gdb) x/5s 0x8b1e9c0 > 0x8b1e9c0: "1" > 0x8b1e9c2: "" > 0x8b1e9c3: "" > 0x8b1e9c4: "$" > 0x8b1e9c6: "" > 0x8b1e9c7: "" > 0x8b1e9c8: "s" > 0x8b1e9ca: "" > 0x8b1e9cb: "" > 0x8b1e9cc: "\r0" > 0x8b1e9cf: "" > 0x8b1e9d0: "\2220" > 0x8b1e9d3: "" > 0x8b1e9d4: "\\O" > 0x8b1e9d7: "" > 0x8b1e9d8: "\020b" > 0x8b1e9db: "" > 0x8b1e9dc: "W0" > 0x8b1e9df: "" > 0x8b1e9e0: "f0" > 0x8b1e9e3: "" > 0x8b1e9e4: "D0" > 0x8b1e9e7: "" > 0x8b1e9e8: "~0" > 0x8b1e9eb: "" where string size is > (gdb) p fstring_size > $54 = 25 Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Also,
template< class Ch, class Tr, class Alloc>
basic_format:: basic_format(const string_type& s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
exceptions_(io::all_error_bits)
{
parse(s); }
it contains:
> (gdb) p &s
> $47 = (
> const
std::basic_string,std::allocator
> *) 0xbfbfe588
>
> (gdb) x/s 0xbfbfe588
> 0xbfbfe588: "[EMAIL PROTECTED],%\bD"
and in
bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), &items_[cur_item], fac, i1, exceptions());
we have the following argument data:
> (gdb) p &items_[cur_item]
> $48 = (
const
boost::io::detail::format_item,std::allocator
> *) 0x8af4a80
>
> (gdb) x/s 0x8af4a80
> 0x8af4a80: "\b\b"
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> I'm not quite sure about the data structure used here, but arguments fmt > and arg1 in bformat() seem to contain no data fields: Oh, I should assume &fmt and &arg1 contain pointers. Sorry. > (gdb) p &fmt > $44 = (const docstring *) 0xbfbfe588 > (gdb) p &arg1 > $46 = (docstring *) 0xbfbfe580 > (gdb) x/s 0xbfbfe588 > 0xbfbfe588: "[EMAIL PROTECTED],%\bD" > (gdb) x/s 0xbfbfe580 > 0xbfbfe580: "[EMAIL PROTECTED],%\bD" Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote:
If you want to debug this fdurther it might be a good idea to write a small
standalone program that simply calls boost::format with the problematic
input.
boost::basic_format() itself seems working if it is called with
"ordinary" strings:
> #include
> #include
>
> using namespace std;
>
> main()
> {
> cout << boost::basic_format("test %1$s \n") % "OK";
> }
which prints out
test OK
And what is the value of *start? And how does the whole format string look
like?
They are as follows:
> (gdb) p *start
> $38 = (const wchar_t &) @0x8b1e9c0: 49
>
> (gdb) p start
> $39 = (
> __gnu_cxx::__normal_iterator wchar_t*,std::basic_string,
> std::allocator > > &) @0xbfbfe2cc: {
> _M_current = 0x8b1e9c0}
>
> (gdb) x/25s 0xbfbfe2cc
> 0xbfbfe2cc: "\300\351\261\b\030\033\177)\r"
> 0xbfbfe2d6: ""
> 0xbfbfe2d7: ""
> 0xbfbfe2d8: "\214\351\261\bp\352\261\b"
> 0xbfbfe2e1: ""
> 0xbfbfe2e2: ""
> 0xbfbfe2e3: ""
> 0xbfbfe2e4: ""
> 0xbfbfe2e5: ""
> 0xbfbfe2e6: ""
> 0xbfbfe2e7: ""
> 0xbfbfe2e8: ""
> 0xbfbfe2e9: ""
> 0xbfbfe2ea: ""
> 0xbfbfe2eb: ""
> 0xbfbfe2ec:
"\234\v\177)\314;\177)8\344\277\277(\343\277\277f|v)\354;\177)\001"
> 0xbfbfe306: ""
> 0xbfbfe307: ""
> 0xbfbfe308:
"([EMAIL PROTECTED]
\037\177)%"
> 0xbfbfe32a: ""
> 0xbfbfe32b: ""
> 0xbfbfe32c: "\016\312\023\001\377\377\377\377"
> 0xbfbfe335: ""
> 0xbfbfe336: ""
> 0xbfbfe337: ""
I am now pretty sure that the
problem is not in any locale stuff, but in boost::format itself, because
normally this part of the code should only be reached with valid format
characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid
format character).
I'm not quite sure about the data structure used here, but arguments fmt
and arg1 in bformat() seem to contain no data fields:
src/support/lstrings.cpp line 875
>
template<>
docstring bformat(docstring const & fmt, docstring arg1)
{
return (boost::basic_format(fmt) % arg1).str();
}
Data:
fmt = $26 = (const docstring &) @0xbfbfe588: {static npos = 4294967295,
_M_dataplus = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e98c}}
arg1 = $27 = (docstring &) @0xbfbfe580: {static npos = 4294967295,
_M_dataplus = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e70c}}
And, subsequently call in boost/format/format_implementation.hpp line 60
template< class Ch, class Tr, class Alloc>
basic_format:: basic_format(const string_type& s)
: style_(0), cur_arg_(0), num_args_(0), dumped_(false),
exceptions_(io::all_error_bits)
{
parse(s);
}
Data:
s = $28 = (
const std::basic_string,std::allocator > &) @0xbfbfe588: {static npos = 4294967295,
_M_dataplus = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8b1e98c}}
I also attach the data of arguments passed to parse_printf_directive()
in boost/format/parsing.hpp which is called later:
boost/format/parsing.hpp line 437
>
bool parse_ok = io::detail::parse_printf_directive(
it, buf.end(), &items_[cur_item], fac, i1, exceptions());
(gdb) p it
$29 = {_M_current = 0x8b1e9c0}
(gdb) p buf.end()
$30 = {_M_current = 0x8b1ea70}
(gdb) p &items_[cur_item]
$31 = (
const class
boost::io::detail::format_item,std::allocator
> *) 0x8af4a80
(gdb) p items_[cur_item]
$35 = (
const
boost::io::detail::format_item,std::allocator
> &) @0x8af4a80: {argN_ = -1, res_ = {
static npos = 4294967295,
_M_dataplus = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8a5a7a8}},
appendix_ = {static npos = 4294967295,
_M_dataplus = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x8a5a7a8}},
fmtstate_ = {width_ = 0, precision_ = 6, fill_ = 32, flags_ = 4098,
rdstate_ = std::_S_goodbit, exceptions_ = std::_S_goodbit,
loc_ = {> = { = {}, m_initialized = false,
m_storage = {dummy_ = {data = "\000\000\000",
aligner_ = }}}, }},
truncate_ = 2147483647, pad_scheme_ = 0}
(gdb) p fac
$32 = (const struct std::ctype &) @0x297f1f20:
(gdb) p i1
$33 = 13
(gdb) p exceptions()
$34 = 255 '\377'
Are these passed data OK?
Thank you,
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote: > Georg Baum wrote: >> Then Koji could fire up a debugger, set a breakpoint in >> parse_printf_directive() and report what the result of the narrow() call >> is. > > Do you mean the return value of wrap_narrow() call? If so, the result of > gdb was: Yes, I meant that. > > $1 = -65 '\277' > > It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259 > and both gave the same result. And what is the value of *start? And how does the whole format string look like? Unfortunately I forgot to ask for that. I am now pretty sure that the problem is not in any locale stuff, but in boost::format itself, because normally this part of the code should only be reached with valid format characters (and signed -65 == unsigned 191 == 0277 == 0xbf is not a valid format character). If you want to debug this fdurther it might be a good idea to write a small standalone program that simply calls boost::format with the problematic input. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: Then Koji could fire up a debugger, set a breakpoint in parse_printf_directive() and report what the result of the narrow() call is. Do you mean the return value of wrap_narrow() call? If so, the result of gdb was: > $1 = -65 '\277' It appeared twice in boost/boost/format/parsing.hpp, in line 181 and 259 and both gave the same result. Thanks, Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Am Dienstag, 5. Juni 2007 19:11 schrieb Koji Yokota: > Georg Baum wrote: > > Jean-Marc Lasgouttes wrote: > >> - how can we test whether freebsd's wchat_t support is good enough? > > First of all is wchar_t really 32 bit on freebsd? I know that some unices > > have 16bit wchar_t, IIRC AIX. This could be tested with a small test > > program that simply prints sizeof(wchar_t). I know that the configure test > > is supposed to find that out, but who knows. > > I tried this and the result was sizeof(wchar_t) = 4. I expected that, but it is always better to be 100% sure. > > I also see that it is installed in /usr/local. If that means that he > > compiled himself from source then I would guess that something went wrong > > with the configuration of the compiler. gcc has tons of configuration > > options, and with many combinations you can get an unusable result (BTDT). > > This gcc 4.1 was installed via ports (a package management system) as > the default gcc used by the system is gcc 3.4.6. Since it is tested by > the port maintainer and used by many people in the same setup, I think > it is fairly reliable (although it may not be as solid as the system gcc). I tend to agree. Just for completeness, could you try to compile LyX with the system gcc please? gcc 3.4.x is supposed to be good enough as well, and since it uses the same abi as gcc 4.1 you can even use it with libraries compiled with g++ 4.1. > > Another thing to try is to force Enricos facet implementation by applying > > the attached patch. Does that change anything? > > Unfortunately, this didn't change anything. That means that the gcc locale facets are probably not the problem. > > Then Koji could fire up a debugger, set a breakpoint in > > parse_printf_directive() and report what the result of the narrow() call > > is. > > I will try this and post the result soon. I believe now that this is the only way to find out the problem. Don't be surprised if you seem to see wild jumps in the debugger, when I debugged the locale code last time there was a lot of inlined stuff that gdb could not relate to the correct line in the correct file. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: Jean-Marc Lasgouttes wrote: - how can we test whether freebsd's wchat_t support is good enough? First of all is wchar_t really 32 bit on freebsd? I know that some unices have 16bit wchar_t, IIRC AIX. This could be tested with a small test program that simply prints sizeof(wchar_t). I know that the configure test is supposed to find that out, but who knows. I tried this and the result was sizeof(wchar_t) = 4. I also see that it is installed in /usr/local. If that means that he compiled himself from source then I would guess that something went wrong with the configuration of the compiler. gcc has tons of configuration options, and with many combinations you can get an unusable result (BTDT). This gcc 4.1 was installed via ports (a package management system) as the default gcc used by the system is gcc 3.4.6. Since it is tested by the port maintainer and used by many people in the same setup, I think it is fairly reliable (although it may not be as solid as the system gcc). Another thing to try is to force Enricos facet implementation by applying the attached patch. Does that change anything? Unfortunately, this didn't change anything. > Then Koji could fire up a debugger, set a breakpoint in > parse_printf_directive() and report what the result of the narrow() call > is. I will try this and post the result soon. Thanks! Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: > - do you know of any problem related to boost::format and wchat_t? No. I remember some problems, but they were eventually solved, I believe by switching from boost::uint32_t to wchar_t. > - how can we test whether freebsd's wchat_t support is good enough? First of all is wchar_t really 32 bit on freebsd? I know that some unices have 16bit wchar_t, IIRC AIX. This could be tested with a small test program that simply prints sizeof(wchar_t). I know that the configure test is supposed to find that out, but who knows. Then, by looking at the compiler and libstdc++ version. I see that Koji uses gcc 4.1, this version works perfectly on linux and should have all needed stuff in libstdc++. I also see that it is installed in /usr/local. If that means that he compiled himself from source then I would guess that something went wrong with the configuration of the compiler. gcc has tons of configuration options, and with many combinations you can get an unusable result (BTDT). Then Koji could fire up a debugger, set a breakpoint in parse_printf_directive() and report what the result of the narrow() call is. Another thing to try is to force Enricos facet implementation by applying the attached patch. Does that change anything? > - if it is not, what shall we do? Develop a suitable test that can be used instead of the #if 1 in the attached patch. But I doubt that this is the case (or if it is the case then it is a bug of gcc and/or freebsd that should be reported). GeorgIndex: src/support/docstring.cpp === --- src/support/docstring.cpp (Revision 18558) +++ src/support/docstring.cpp (Arbeitskopie) @@ -217,7 +217,8 @@ lyx::docstring & operator+=(lyx::docstri } // namespace lyx -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__) +//#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__) +#if 1 // gcc does not have proper locale facets for lyx::char_type if // sizeof(wchar_t) == 2, so we have to implement them on our own.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Enrico Forestieri wrote: >> I don't understand. Seems that you are still using wchar_t ... This >> crash is very similar to the one I was getting on cygwin after the >> switch to unicode. I remember that someway I was able to fix it by >> using the STLport library, before the real fix by Georg. Have a >> look at this thread (may be you will get some ideas on how to >> proceed): http://thread.gmane.org/gmane.editors.lyx.devel/68855 Koji> Enrico, I'm now checking this but before that I send a quick Koji> result of the patches. I applied the attached patches but Koji> unfortunately there was no improvement. I stated that the Koji> initialization succeeded in the previous mail, but it was due to Koji> the fact that I already had a user setting directory Koji> $HOME/.lyx-svn . Removing it draws back to the same result as Koji> before. Hello Georg, We could use some of your help here. It seems that boost::format crashes on FreeBSD when parsing special tokens. The backtrace is here: http://article.gmane.org/gmane.editors.lyx.devel/84094 and my analysis is there: http://thread.gmane.org/gmane.editors.lyx.devel/85096 Since then, many things have been tried to compile with LyX included std::ctype or to try to disable boost::format, but with no luck. So my question is threefold: - do you know of any problem related to boost::format and wchat_t? - how can we test whether freebsd's wchat_t support is good enough? - if it is not, what shall we do? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Tue, May 29, 2007 at 11:20:45AM +0900, Koji Yokota wrote:
> Enrico Forestieri wrote:
> > I don't understand. Seems that you are still using wchar_t ...
> > This crash is very similar to the one I was getting on cygwin after the
> > switch to unicode. I remember that someway I was able to fix it by
> > using the STLport library, before the real fix by Georg. Have a look
> > at this thread (may be you will get some ideas on how to proceed):
> > http://thread.gmane.org/gmane.editors.lyx.devel/68855
>
> Enrico, I'm now checking this but before that I send a quick result of
> the patches. I applied the attached patches but unfortunately there was
> no improvement. I stated that the initialization succeeded in the
> previous mail, but it was due to the fact that I already had a user
> setting directory $HOME/.lyx-svn . Removing it draws back to the same
> result as before.
That is because of autoloading of the splash.lyx document.
I am running out of ideas here. I think that you will need advice from
a real expert such as Georg.
> # Does RC1 go with the following modification suggested before?
>
> --- src/support/lstrings.cpp(revision 18398)
> +++ src/support/lstrings.cpp(working copy)
> @@ -21,6 +21,7 @@
> #include
> #include
>
> +#undef USE_BOOST_FORMAT
> #ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
> #if USE_BOOST_FORMAT
> #include
> --- src/support/docstring.h.orig Sun May 27 17:26:19 2007
> +++ src/support/docstring.h Sun May 27 17:27:27 2007
> @@ -98,7 +98,7 @@
> } // namespace lyx
>
>
> -#if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) &&
> __GNUC__ == 3 && __GNUC_MINOR__ < 4
> +#if 1
> // Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
> namespace std {
>
> --- src/support/docstream.cpp.origSun May 27 17:40:11 2007
> +++ src/support/docstream.cpp Sun May 27 17:40:11 2007
> @@ -322,7 +322,7 @@
>
> }
>
> -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
> +#if 1
> // We get undefined references to these virtual methods. This looks like
> // a bug in gcc. The implementation here does not do anything useful, since
> // it is overriden in iconv_codecvt_facet.
> --- src/support/docstring.cpp.origSun May 27 17:40:35 2007
> +++ src/support/docstring.cpp Sun May 27 17:40:35 2007
> @@ -217,7 +217,7 @@
>
> } // namespace lyx
>
> -#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
> +#if 1
>
> // gcc does not have proper locale facets for lyx::char_type if
> // sizeof(wchar_t) == 2, so we have to implement them on our own.
> --- src/support/types.h.orig Sun May 27 17:40:58 2007
> +++ src/support/types.h Sun May 27 17:40:58 2007
> @@ -23,7 +23,7 @@
> namespace lyx {
>
> /// The type used to hold characters in paragraphs
> -#if defined(HAVE_WCHAR_T) && SIZEOF_WCHAR_T == 4
> +#if 0
> // Prefer this if possible because GNU libstdc++ has usable
> // std::ctype locale facets but not
> // std::ctype. gcc older than 3.4 is also missing
> --- configure.ac.orig 2007-05-28 17:10:24.952949000 +0200
> +++ configure.ac2007-05-28 17:11:04.599949500 +0200
> @@ -356,11 +356,7 @@
> #endif
> #endif
>
> -#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM)
> -# define USE_BOOST_FORMAT 1
> -#else
> # define USE_BOOST_FORMAT 0
> -#endif
>
> #define BOOST_USER_CONFIG
--
Enrico
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Enrico Forestieri wrote:
I don't understand. Seems that you are still using wchar_t ...
This crash is very similar to the one I was getting on cygwin after the
switch to unicode. I remember that someway I was able to fix it by
using the STLport library, before the real fix by Georg. Have a look
at this thread (may be you will get some ideas on how to proceed):
http://thread.gmane.org/gmane.editors.lyx.devel/68855
Enrico, I'm now checking this but before that I send a quick result of
the patches. I applied the attached patches but unfortunately there was
no improvement. I stated that the initialization succeeded in the
previous mail, but it was due to the fact that I already had a user
setting directory $HOME/.lyx-svn . Removing it draws back to the same
result as before.
Koji
# Does RC1 go with the following modification suggested before?
--- src/support/lstrings.cpp(revision 18398)
+++ src/support/lstrings.cpp(working copy)
@@ -21,6 +21,7 @@
#include
#include
+#undef USE_BOOST_FORMAT
#ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
#if USE_BOOST_FORMAT
#include
--- src/support/docstring.h.origSun May 27 17:26:19 2007
+++ src/support/docstring.h Sun May 27 17:27:27 2007
@@ -98,7 +98,7 @@
} // namespace lyx
-#if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) &&
__GNUC__ == 3 && __GNUC_MINOR__ < 4
+#if 1
// Missing char_traits methods in gcc 3.3 and older. Taken from gcc 4.2svn.
namespace std {
--- src/support/docstream.cpp.orig Sun May 27 17:40:11 2007
+++ src/support/docstream.cpp Sun May 27 17:40:11 2007
@@ -322,7 +322,7 @@
}
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+#if 1
// We get undefined references to these virtual methods. This looks like
// a bug in gcc. The implementation here does not do anything useful, since
// it is overriden in iconv_codecvt_facet.
--- src/support/docstring.cpp.orig Sun May 27 17:40:35 2007
+++ src/support/docstring.cpp Sun May 27 17:40:35 2007
@@ -217,7 +217,7 @@
} // namespace lyx
-#if (!defined(HAVE_WCHAR_T) || SIZEOF_WCHAR_T != 4) && defined(__GNUC__)
+#if 1
// gcc does not have proper locale facets for lyx::char_type if
// sizeof(wchar_t) == 2, so we have to implement them on our own.
--- src/support/types.h.origSun May 27 17:40:58 2007
+++ src/support/types.h Sun May 27 17:40:58 2007
@@ -23,7 +23,7 @@
namespace lyx {
/// The type used to hold characters in paragraphs
-#if defined(HAVE_WCHAR_T) && SIZEOF_WCHAR_T == 4
+#if 0
// Prefer this if possible because GNU libstdc++ has usable
// std::ctype locale facets but not
// std::ctype. gcc older than 3.4 is also missing
--- configure.ac.orig 2007-05-28 17:10:24.952949000 +0200
+++ configure.ac2007-05-28 17:11:04.599949500 +0200
@@ -356,11 +356,7 @@
#endif
#endif
-#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM)
-# define USE_BOOST_FORMAT 1
-#else
# define USE_BOOST_FORMAT 0
-#endif
#define BOOST_USER_CONFIG
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Mon, May 28, 2007 at 10:20:57AM +0900, Koji Yokota wrote: > Peter Kümmel wrote: > > Hasn't Abdel fixed it today? Maybe updating helps. > > Yeah, right. With a newer version, Lyx at least proceeds with a > successful initialization and its GUI appears. However, trial of File -> > New, File -> Open, or start of lyx with a parameter like 'lyx -dbg any' > leads to the same crash as before: > > > terminate called after throwing an instance of > > 'boost::io::bad_format_string' > > what(): boost::bad_format_string: format-string is ill-formed > > > > Program received signal SIGABRT, Aborted. > > 0x29934897 in kill () from /lib/libc.so.6 > > When it crashed with File -> New, the backtrace is as follows: I don't understand. Seems that you are still using wchar_t ... This crash is very similar to the one I was getting on cygwin after the switch to unicode. I remember that someway I was able to fix it by using the STLport library, before the real fix by Georg. Have a look at this thread (may be you will get some ideas on how to proceed): http://thread.gmane.org/gmane.editors.lyx.devel/68855 You could also try forcing USE_BOOST_FORMAT to 0 by hacking configure.ac as follows: --- configure.ac.orig 2007-05-28 17:10:24.952949000 +0200 +++ configure.ac2007-05-28 17:11:04.599949500 +0200 @@ -356,11 +356,7 @@ #endif #endif -#if defined(HAVE_OSTREAM) && defined(HAVE_LOCALE) && defined(HAVE_SSTREAM) -# define USE_BOOST_FORMAT 1 -#else # define USE_BOOST_FORMAT 0 -#endif #define BOOST_USER_CONFIG -- Enrico
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote: Hasn't Abdel fixed it today? Maybe updating helps. Yeah, right. With a newer version, Lyx at least proceeds with a successful initialization and its GUI appears. However, trial of File -> New, File -> Open, or start of lyx with a parameter like 'lyx -dbg any' leads to the same crash as before: > terminate called after throwing an instance of > 'boost::io::bad_format_string' > what(): boost::bad_format_string: format-string is ill-formed > > Program received signal SIGABRT, Aborted. > 0x29934897 in kill () from /lib/libc.so.6 When it crashed with File -> New, the backtrace is as follows: #0 0x29934897 in kill () from /lib/libc.so.6 #1 0x298474f7 in raise () from /lib/libpthread.so.2 #2 0x299332fb in abort () from /lib/libc.so.6 #3 0x297e9dc4 in __gnu_cxx::__verbose_terminate_handler () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #4 0x297e773a in __cxxabiv1::__terminate () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #5 0x297e7782 in std::terminate () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #6 0x297e78ba in __cxa_throw () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #7 0x08142c5c in boost::throw_exception (e=Could not find the frame base for "void boost::throw_exception(boost::io::bad_format_string const&)". ) at throw_exception.hpp:39 #8 0x08142e65 in boost::io::detail::maybe_throw_exception ( exceptions=255 '', pos=7, size=10) at parsing.hpp:106 #9 0x081439a6 in boost::io::detail::parse_printf_directivestd::char_traits, std::allocator, __gnu_cxx::__normal_iteratorstd::char_traits, std::allocator > >, std::ctype > ([EMAIL PROTECTED], [EMAIL PROTECTED], fpar=0x8fda700, [EMAIL PROTECTED], offset=7, exceptions=255 '') at parsing.hpp:333 #10 0x0814585f in boost::basic_formatstd::char_traits, std::allocator >::parse (this=0xbfbfbf94, [EMAIL PROTECTED]) at parsing.hpp:437 #11 0x08145c76 in basic_format (this=0xbfbfbf94, [EMAIL PROTECTED]) at format_implementation.hpp:64 #12 0x0875c167 in lyx::support::bformatstd::char_traits, std::allocator > > ([EMAIL PROTECTED], [EMAIL PROTECTED]) at lstrings.cpp:875 #13 0x08309eeb in lyx::Text::currentState (this=0x8f375e0, [EMAIL PROTECTED]) at Text.cpp:1759 #14 0x08179b70 in lyx::Cursor::currentState (this=0x8af3e08) at Cursor.cpp:1224 #15 0x0826c775 in lyx::LyXFunc::viewStatusMessage (this=0x8aca000) at LyXFunc.cpp:2126 #16 0x0856c147 in lyx::frontend::GuiView::updateStatusBar (this=0x8cb3200) at GuiView.cpp:590 #17 0x0850d2c6 in lyx::LyXView::setBuffer (this=0x8cb3214, b=0x8f0d200) at LyXView.cpp:157 #18 0x082705d1 in lyx::LyXFunc::menuNew (this=0x8aca000, [EMAIL PROTECTED], fromTemplate=false) at LyXFunc.cpp:1911 #19 0x082741d2 in lyx::LyXFunc::dispatch (this=0x8aca000, [EMAIL PROTECTED]) at LyXFunc.cpp:875 #20 0x0824f5e9 in lyx::dispatch ([EMAIL PROTECTED]) at LyX.cpp:1463 #21 0x0850b037 in lyx::LyXView::dispatch (this=0x8cb3214, [EMAIL PROTECTED]) at LyXView.cpp:448 #22 0x0869edab in lyx::frontend::Action::action (this=0x8db9580) at Action.cpp:90 #23 0x0869ee2e in lyx::frontend::Action::qt_metacall (this=0x8db9580, _c=QMetaObject::InvokeMetaMethod, _id=1, _a=0xbfbfd6b8) at Action_moc.cpp:69 #24 0x292f0d43 in QMetaObject::activate (sender=0x8db9580, from_signal_index=5, to_signal_index=6, argv=0xbfbfd6b8) at kernel/qobject.cpp:2937 #25 0x292f0ec3 in QMetaObject::activate (sender=0x0, m=0x0, from_local_signal_index=1, to_local_signal_index=2, argv=0x14487) at kernel/qobject.cpp:2992 #26 0x28b607b3 in QAction::triggered (this=0x0, _t1=false) at moc_qaction.cpp:208 #27 0x28b60b5c in QAction::activate (this=0x8db9580, event=QAction::Trigger) at kernel/qaction.cpp:1075 #28 0x28e6bcca in QMenuPrivate::activateAction (this=0x8cf8a00, action=0x8db9580, action_e=QAction::Trigger) at widgets/qmenu.cpp:756 #29 0x28e6e7fd in QMenu::keyPressEvent (this=0x8ce4680, e=0xbfbfdf50) at widgets/qmenu.cpp:2266 #30 0x28bac58d in QWidget::event (this=0x8ce4680, event=0xbfbfdf50) at kernel/qwidget.cpp:5753 #31 0x28e699e4 in QMenu::event (this=0x8ce4680, e=0xbfbfdf50) at widgets/qmenu.cpp:1901 #32 0x28b665f4 in QApplicationPrivate::notify_helper (this=0x8acb000, receiver=0x8ce4680, e=0xbfbfdf50) at kernel/qapplication.cpp:3439 #33 0x28b66dc7 in QApplication::notify (this=0x8ad1000, receiver=0x8ce4680, e=0xbfbfdf50) at kernel/qapplication.cpp:3067 #34 0x0855d691 in lyx::frontend::GuiApplication::notify (this=0x8ad1000, receiver=0x8ce4680, event=0xbfbfdf50) at GuiApplication.cpp:237 #35 0x28bb33c7 in qt_sendSpontaneousEvent (receiver=0x14487, event=0x14487) at qcoreapplication.h:186 #36 0x28be256c in QKeyMapper::sendKeyEvent (keyWidget=0x8ce4680, grab=false, type=QEvent::KeyPress, code=78, modifiers=Error accessing memory address 0x0: Bad address. ) at kernel/qkeymapper_x11.cpp:1580 #37 0x28be289b in QKeyMapperPrivate::translateKeyEvent (this=0x8aefc80, keyWidget=0x8ce4680, event=0xbfbfe
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote: > Jean-Marc Lasgouttes wrote: >>> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes: >> >> Abdelrazak> Maybe Koji could try to enable that unconditionally: >> >> Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && >> Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ >> Abdelrazak> < 4 + #if 1 >> >> Koji, could you try that? > > OK, I just send the result. I tried this modification on the files below: > >> src/support/docstream.cpp >> src/support/docstring.cpp >> src/support/docstring.h >> src/support/types.h > > However, lyx (today's svn) crashed even before it reaches to main() > function with the following message: Hasn't Abdel fixed it today? Maybe updating helps. > >> Program received signal SIGSEGV, Segmentation fault. >> 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at >> ostream.tcc:55 > > Backtrace at the crash time seems corrupt, but initial layers contain: > >> #0 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at >> ostream.tcc:55 >> #1 0x0807c972 in std::operator<< > ( >> Ba[EMAIL PROTECTED], __s=0x879fcd1 "Assertion triggered in ") >> at ostream.tcc:738 >> #2 0x08088a48 in boost::assertion_failed (expr=0x87d28ed >> "singleton_", >> function=0x87d4680 "static lyx::LyX& lyx::LyX::ref()", >> file=0x87d2860 "LyX.cpp", line=211) at boost.cpp:54 >> #3 0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211 >> #4 0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546 >> #5 0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34 >> #6 0x0845b170 in __static_initialization_and_destruction_0 ( >> __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117 >> #7 0x08467ded in global constructors keyed to >> _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE () >> at InsetListingsParams.cpp:658 >> #8 0x0879e0f5 in __do_global_ctors_aux () >> #9 0x0806c891 in _init () >> #10 0x0206 in ?? () >> #11 0x0001 in ?? ()... and so on > > Koji > -- Peter Kümmel
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes: Abdelrazak> Maybe Koji could try to enable that unconditionally: Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ Abdelrazak> < 4 + #if 1 Koji, could you try that? OK, I just send the result. I tried this modification on the files below: src/support/docstream.cpp src/support/docstring.cpp src/support/docstring.h src/support/types.h However, lyx (today's svn) crashed even before it reaches to main() function with the following message: > Program received signal SIGSEGV, Segmentation fault. > 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at > ostream.tcc:55 Backtrace at the crash time seems corrupt, but initial layers contain: > #0 0x0807c732 in sentry (this=0xbfbfe6c0, [EMAIL PROTECTED]) at > ostream.tcc:55 > #1 0x0807c972 in std::operator<< > ( > Ba[EMAIL PROTECTED], __s=0x879fcd1 "Assertion triggered in ") > at ostream.tcc:738 > #2 0x08088a48 in boost::assertion_failed (expr=0x87d28ed > "singleton_", > function=0x87d4680 "static lyx::LyX& lyx::LyX::ref()", > file=0x87d2860 "LyX.cpp", line=211) at boost.cpp:54 > #3 0x0824f3e6 in lyx::LyX::ref () at LyX.cpp:211 > #4 0x082593cf in lyx::getGuiMessages () at LyX.cpp:1546 > #5 0x081d3d39 in lyx::_ ([EMAIL PROTECTED]) at gettext.cpp:34 > #6 0x0845b170 in __static_initialization_and_destruction_0 ( > __initialize_p=1, __priority=65535) at InsetListingsParams.cpp:117 > #7 0x08467ded in global constructors keyed to > _ZN3lyx36_GLOBAL__N_._pch.h__5FBB75A917allowed_languagesE () > at InsetListingsParams.cpp:658 > #8 0x0879e0f5 in __do_global_ctors_aux () > #9 0x0806c891 in _init () > #10 0x0206 in ?? () > #11 0x0001 in ?? ()... and so on Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: Peter> Done, all files (also filesystem) are now from 1.34. Thanks. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote: > Jean-Marc Lasgouttes wrote: >>> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: >> Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a >> Peter> noop: >> >> Peter> http://www.lyx.org/trac/changeset/18458 >> >> I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is >> released. >> >> JMarc >> > > OK, I start updating ... > Done, all files (also filesystem) are now from 1.34. Peter
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: >> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: > > Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a > Peter> noop: > > Peter> http://www.lyx.org/trac/changeset/18458 > > I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is > released. > > JMarc > OK, I start updating ... Peter Kümmel
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: Peter> We could just wait for 1.34.1, upgrading to 1.34 is nearly a Peter> noop: Peter> http://www.lyx.org/trac/changeset/18458 I'd suggest to upgrade to 1.34 now, and consider 1.34.1 when it is released. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: > Jean-Marc Lasgouttes wrote: > >> It was Georg who wrote >> >> Note that the iostreams library in boost 1.34.0 is an outdated >> version (older than the version in boost 1.33.1) with a memory leak, >> so watch for iostreams updates in the boost 1.34 branch, too. >> >> I am afraid I do not know what he means. Georg? > > See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265. > This (and the very late 1.34.0 release) is a sign that boost currently has > severe problems with testing/the release procedure. I hope they can sort > this out for the next release. > > > Georg > We could just wait for 1.34.1, upgrading to 1.34 is nearly a noop: http://www.lyx.org/trac/changeset/18458 Peter
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Thu, May 24, 2007 at 02:15:35PM +0200, Jean-Marc Lasgouttes wrote: > > "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes: > > Abdelrazak> Maybe Koji could try to enable that unconditionally: > > Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && > Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ > Abdelrazak> < 4 + #if 1 > > Koji, could you try that? I think that he has to tweak several files, namely: src/support/docstream.cpp src/support/docstring.cpp src/support/docstring.h src/support/types.h -- Enrico
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Abdelrazak" == Abdelrazak Younes <[EMAIL PROTECTED]> writes: Abdelrazak> Maybe Koji could try to enable that unconditionally: Abdelrazak> - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && Abdelrazak> defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ Abdelrazak> < 4 + #if 1 Koji, could you try that? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Wed, May 23, 2007 at 05:20:52PM +0200, Jean-Marc Lasgouttes wrote: > > "Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes: > > Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our > Enrico> facets implementation should kick in. I use it with no > Enrico> problems in both cygwin and mingw, don't know about other > Enrico> platforms. > > Do you have a simple program to test whether the facet works? I seem > to remember that you did it at the time. I think that you are confusing me with Georg here. He had written some tests at the time. In this field, he is the master, I'm only an apprentice ;-) http://thread.gmane.org/gmane.editors.lyx.devel/66786 > And can we force to use our implementation in some cases? I think so, but the implementation must match the compiler internals, that's why an explicit check for the gcc version is also made. -- Enrico
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: "Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes: Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our Enrico> facets implementation should kick in. I use it with no Enrico> problems in both cygwin and mingw, don't know about other Enrico> platforms. Do you have a simple program to test whether the facet works? I seem to remember that you did it at the time. And can we force to use our implementation in some cases? The test is in docstring.h:101 #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ < 4 Maybe Koji could try to enable that unconditionally: - #if SIZEOF_WCHAR_T != 4 && defined(__GNUC__) && defined(__GNUC_MINOR__) && __GNUC__ == 3 && __GNUC_MINOR__ < 4 + #if 1 Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Enrico" == Enrico Forestieri <[EMAIL PROTECTED]> writes: Enrico> When sizeof(wchar_t) != 4 or wchar_t is not present, then our Enrico> facets implementation should kick in. I use it with no Enrico> problems in both cygwin and mingw, don't know about other Enrico> platforms. Do you have a simple program to test whether the facet works? I seem to remember that you did it at the time. And can we force to use our implementation in some cases? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Wed, May 23, 2007 at 03:40:08PM +0200, Jean-Marc Lasgouttes wrote: > This seems to point to a problem with using widen/narrow on a facet > like std::ctype. Of course, I know next to nothing about that. > Enrico, would you have some ideas about whether freebsd has the needed > support? I really don't know. I still have NetBSD installed on an old MC68040 machine but I don't have it handy now, and furthermore I haven't updated it in the last 7 or 8 years... > Was it freebsd that did not even have a wchar_t type some years ago? When sizeof(wchar_t) != 4 or wchar_t is not present, then our facets implementation should kick in. I use it with no problems in both cygwin and mingw, don't know about other platforms. -- Enrico
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes:
Koji> I attach the backtrace when the program is crashed as additional
Koji> information.
Looking at the backtrace, the problem happens at line 333 of
boost/format/parsing.hpp:
switch ( wrap_narrow(fac, *start, 0) ) {
case 'X':
fpar->fmtstate_.flags_ |= std::ios_base::uppercase;
[...]
case 'n' :
fpar->argN_ = format_item_t::argN_ignored;
break;
default:
maybe_throw_exception(exceptions, start-start0+offset,
fstring_size); /// <-- line 333 is here
}
This seems to point to a problem with using widen/narrow on a facet
like std::ctype. Of course, I know next to nothing about that.
Enrico, would you have some ideas about whether freebsd has the needed
support?
Was it freebsd that did not even have a wchar_t type some years ago?
JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: > It was Georg who wrote > > Note that the iostreams library in boost 1.34.0 is an outdated > version (older than the version in boost 1.33.1) with a memory leak, > so watch for iostreams updates in the boost 1.34 branch, too. > > I am afraid I do not know what he means. Georg? See this thread: http://thread.gmane.org/gmane.comp.lib.boost.devel/159265. This (and the very late 1.34.0 release) is a sign that boost currently has severe problems with testing/the release procedure. I hope they can sort this out for the next release. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: Peter> I've updated to boost 1.34 in a branch, and it's not that hard, Peter> it compiles and runs without problems. The changes are: Very good, thanks. Peter> Should I update the header files? JMarc says in bugzilla that Peter> we should not update filesystem? It was Georg who wrote Note that the iostreams library in boost 1.34.0 is an outdated version (older than the version in boost 1.33.1) with a memory leak, so watch for iostreams updates in the boost 1.34 branch, too. I am afraid I do not know what he means. Georg? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes schrieb: Peter> After the release of 1.5, I assume. Are we sure? The 1.5 branch would be better with the real 1.34, rather than a prerelease. +1 Michael
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Abdelrazak Younes wrote: > José Matos wrote: >> On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote: >>> Are we sure? The 1.5 branch would be better with the real 1.34, rather >>> than a prerelease. >> >> I tend to agree with Jean-Marc here. > > Me too. > > Abdel. > I've updated to boost 1.34 in a branch, and it's not that hard, it compiles and runs without problems. The changes are: $ svn ci -m"check in changes of already exisiting files" Sendingboost/boost/bind/storage.hpp Sendingboost/boost/config/no_tr1/utility.hpp Sendingboost/boost/filesystem/cerrno.hpp Sendingboost/boost/iostreams/detail/streambuf/indirect_streambuf.hpp Sendingboost/boost/mpl/aux_/lambda_spec.hpp Sendingboost/boost/optional/optional.hpp Sendingboost/boost/preprocessor/config/config.hpp Sendingboost/boost/regex/v4/cpp_regex_traits.hpp Sendingboost/boost/regex/v4/regex_traits.hpp Sendingboost/boost/spirit/utility/loops.hpp Sendingboost/boost/type_traits/is_convertible.hpp Sendingboost/libs/filesystem/src/operations.cpp Sendingboost/libs/filesystem/src/path.cpp Sendingboost/libs/filesystem/src/portability.cpp and renaming none_t.hpp to boost/boost/none.hpp Should I update the header files? JMarc says in bugzilla that we should not update filesystem? Peter
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
José Matos wrote: On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote: Are we sure? The 1.5 branch would be better with the real 1.34, rather than a prerelease. I tend to agree with Jean-Marc here. Me too. Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Tuesday 22 May 2007 8:52:30 am Jean-Marc Lasgouttes wrote: > Are we sure? The 1.5 branch would be better with the real 1.34, rather > than a prerelease. I tend to agree with Jean-Marc here. > JMarc -- José Abílio
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Peter" == Peter Kümmel <[EMAIL PROTECTED]> writes: >> We should probably update boost first to 1.34 final, but I do not >> know who wants to do it. Peter> After the release of 1.5, I assume. Are we sure? The 1.5 branch would be better with the real 1.34, rather than a prerelease. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: >> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes: > > Angus> could you go the extra mile and ascertain exactly what boost > Angus> dislikes here and then work with the boost team to fix their > Angus> code. We make great use of boost and should feel honour bound > Angus> to help squash those bugs we find. > The problem is that I couldn't reproduce it here, but when Koji Yokota is willed to help me a bit I could give it a try. But I'm still not 100% sure that it is a boost bug. > We should probably update boost first to 1.34 final, but I do not know > who wants to do it. After the release of 1.5, I assume. > > JMarc > > PS: Feel like coming to Finland in August? > > > > > -- Peter Kümmel
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Mon, May 21, 2007 at 09:32:22PM +, Angus Leeming wrote: > Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes: > > PS: Feel like coming to Finland in August? > > Not really. Finland in August is much like Scotland in August; full of nasty, > biting insects! > > Angus (ducking from the wrath of M.V. ;-)) FUD, FUD, FUD... (how does one flame in Fahrenheit?) - Martin
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes: Angus> Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes: >> PS: Feel like coming to Finland in August? Angus> Not really. Finland in August is much like Scotland in August; Angus> full of nasty, biting insects! You have a point there. And don't forget nasty, biting developers. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes <[EMAIL PROTECTED]> writes: > PS: Feel like coming to Finland in August? Not really. Finland in August is much like Scotland in August; full of nasty, biting insects! Angus (ducking from the wrath of M.V. ;-))
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Angus" == Angus Leeming <[EMAIL PROTECTED]> writes: Angus> could you go the extra mile and ascertain exactly what boost Angus> dislikes here and then work with the boost team to fix their Angus> code. We make great use of boost and should feel honour bound Angus> to help squash those bugs we find. We should probably update boost first to 1.34 final, but I do not know who wants to do it. JMarc PS: Feel like coming to Finland in August?
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel <[EMAIL PROTECTED]> writes: > >> But we could disable boost format, so please try attached patch. > > Peter, with this patch, it seems the problem has resolved. lyx > > successfully finishes the initialization and starts up. > Great. > Hope you get not other problems because of iconv. > We should integrate the disabling of boost's > format code for FreeBSD into the build system. Peter, could you go the extra mile and ascertain exactly what boost dislikes here and then work with the boost team to fix their code. We make great use of boost and should feel honour bound to help squash those bugs we find. Angus
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote: Have fun! Do you know that beta3 is out? Yes, now I'm going to try it. Hope you get not other problems because of iconv. I will report if there is any. Thanks, Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote: > Peter Kümmel wrote: >> bformat is just a call to boost: >> >> boost::basic_format(fmt) % arg1).str(); >> >> So it is a boost bug when checking the format string, >> or our string is somehow invalid. This does not help much. >> >> But we could disable boost format, so please try attached patch. > > > Peter, with this patch, it seems the problem has resolved. lyx > successfully finishes the initialization and starts up. Great. > > Now, I can try this new product :) Have fun! Do you know that beta3 is out? > > Many thanks to all, > > Koji > Hope you get not other problems because of iconv. We should integrate the disabling of boost's format code for FreeBSD into the build system. -- Peter Kümmel
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote: bformat is just a call to boost: boost::basic_format(fmt) % arg1).str(); So it is a boost bug when checking the format string, or our string is somehow invalid. This does not help much. But we could disable boost format, so please try attached patch. Peter, with this patch, it seems the problem has resolved. lyx successfully finishes the initialization and starts up. Now, I can try this new product :) Many thanks to all, Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote:
> Peter Kümmel wrote:
>> We still don't know which string is wrong
>> so here the next code:
>>
>> docstring format = _("LyX directory, %1$s. \n");
>> lyxerr << "format = " << to_utf8(format) << endl;
>>
>> docstring path = from_utf8("/home/dir/");
>> lyxerr << "path = " << to_utf8(path) << endl;
>>
>> docstring s4= bformat(format, path);
>
> OK, stderr is exactly the same:
>
>> format = LyX directory, %1$s.
>>
>> path = /home/dir/
>
> so seems from/to utf is working though I don't know the intermediate
> product is correct.
>
> But
>
>> docstring s4= bformat(format, path);
bformat is just a call to boost:
boost::basic_format(fmt) % arg1).str();
So it is a boost bug when checking the format string,
or our string is somehow invalid. This does not help much.
But we could disable boost format, so please try attached patch.
>
> still fails...
>
> Koji
>
--
Peter Kümmel
Index: src/support/lstrings.cpp
===
--- src/support/lstrings.cpp(revision 18398)
+++ src/support/lstrings.cpp(working copy)
@@ -21,6 +21,7 @@
#include
#include
+#undef USE_BOOST_FORMAT
#ifndef I_AM_NOT_AFRAID_OF_HEADER_LIBRARIES
#if USE_BOOST_FORMAT
#include
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote:
We still don't know which string is wrong
so here the next code:
docstring format = _("LyX directory, %1$s. \n");
lyxerr << "format = " << to_utf8(format) << endl;
docstring path = from_utf8("/home/dir/");
lyxerr << "path = " << to_utf8(path) << endl;
docstring s4= bformat(format, path);
OK, stderr is exactly the same:
format = LyX directory, %1$s.
path = /home/dir/
so seems from/to utf is working though I don't know the intermediate
product is correct.
But
>docstring s4= bformat(format, path);
still fails...
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
We still don't know which string is wrong
so here the next code:
docstring format = _("LyX directory, %1$s. \n");
lyxerr << "format = " << to_utf8(format) << endl;
docstring path = from_utf8("/home/dir/");
lyxerr << "path = " << to_utf8(path) << endl;
docstring s4= bformat(format, path);
Here I get this output:
format = LyX directory, %1$s.
path = /home/dir/
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote:
lyxerr << "LyX: Creating directory %1$s" << endl,
lyxerr << package().user_support().absFilename() << endl;
docstring s1 = _("LyX: Creating directory %1$s");
docstring s2 = from_utf8(package().user_support().absFilename());
//docstring s3 = bformat(_("LyX directory, %1$s. \n"), "/home/dir/");
Peter, OK, all tests succeeded up to this point, but
docstring s4= bformat(_("LyX directory, %1$s. \n"),
from_utf8("/home/dir/"));
fails (and all the rest).
In this test, after the evaluation of from_utf8() part, it jumps to
docstring const _(string const & str)
in gettext.cpp (L34) and this function returns a value and successfully
finishes. Then, it goes to
docstring bformat(docstring const & fmt, docstring arg1)
and crashes when it evaluates the "return" line (L875 in lstrings.cpp).
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
there is no bformat which takes a char*:
lyxerr << "LyX: Creating directory %1$s" << endl,
lyxerr << package().user_support().absFilename() << endl;
docstring s1 = _("LyX: Creating directory %1$s");
docstring s2 = from_utf8(package().user_support().absFilename());
//docstring s3 = bformat(_("LyX directory, %1$s. \n"), "/home/dir/");
docstring s4= bformat(_("LyX directory, %1$s. \n"),
from_utf8("/home/dir/"));
docstring s5 = bformat(_("LyX directory, %1$s. \n"),
from_utf8(package().user_support().absFilename()));
docstring s6 = bformat(_("LyX directory, %1$s. \n"),
from_utf8(package().user_support().absFilename()));
docstring s7 = bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own
configuration."),
from_utf8("/home/dir/"));
docstring s8 = bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own configuration."),
from_utf8(package().user_support().absFilename()));
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
横田 宏治 wrote:
> As I think I'd better continue to throw additional informations on this
> problem, I'll do so gradually.
>
Yes this is a good idea.
First we must locate the reason for the crash.
Is it the format string or the path string. And crashes it really
because of iconv or has it nothing to do with iconv?
Could you copy this
lyxerr << "LyX: Creating directory %1$s" << endl,
lyxerr << package().user_support().absFilename() << endl;
docstring s1 = _("LyX: Creating directory %1$s");
docstring s2 = from_utf8(package().user_support().absFilename());
docstring s3 = bformat(_("LyX directory, %1$s. \n"), "/home/dir/");
docstring s4= bformat(_("LyX directory, %1$s. \n"),
from_utf8("/home/dir/"));
docstring s5 = bformat(_("LyX directory, %1$s. \n"),
from_utf8(package().user_support().absFilename()));
docstring s6 = bformat(_("LyX directory, %1$s. \n"),
from_utf8(package().user_support().absFilename()));
docstring s7 = bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own
configuration."),
from_utf8("/home/dir/"));
docstring s8 = bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own configuration."),
from_utf8(package().user_support().absFilename()));
to the function and check on which line it crashes?
> Before the call of from_utf8 in lyx_main.C:
>
>> lyxerr << to_utf8(bformat(_("LyX: Creating directory %1$s"),
>> from_utf8(package().user_support().absFilename(
>><< endl;
>
> I tried to issue the following command on gdb:
>
> (gdb) print lyx::support::package().user_support().absFilename()
>
> then I got:
>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x2942672a in std::basic_string,
>> std::allocator >::basic_string () from
>> /usr/local/lib/gcc-4.1.3/libstdc++.so.6
>> The program being debugged was signaled while in a function called
>> from GDB.
>> GDB remains in the frame where the signal was received.
>> To change this behavior use "set unwindonsignal on"
>> Evaluation of the expression containing the function
>> (lyx::support::FileName::absFilename() const) will be abandoned.
>
> Can this related to the problem? However, ordinary execution of the lyx
> program issues no problem in getting absFilename() and proceeds to the
> evaluation of from_utf8().
So it crashes because of the debugger? I don't know gdb but absfilename
return a std::string, so gdb should know how to handle it.
std::string const absFilename() const { return name_; }
>
> Koji
>
> Koji Yokota wrote:
>> Peter Kümmel wrote:
>>> Could you have a look at the sting passes to boost::basic_format?
>>> support/lstrings.C line 875
>>
>> OK, it's as follows:
>>
>> > (gdb) p fmt
>> > $18 = (const docstring &) @0xbfbfe5a8: {static npos = 4294967295,
>> > _M_dataplus = {> =
>> > {<__gnu_cxx::new_allocator> = {}, > > fields>}, _M_p = 0x87ebb0c}}
>> > (gdb) p/c 0xbfbfe5a8
>> > $19 = -88 '\250'
>> > (gdb) p arg1
>> > $20 = (docstring &) @0xbfbfe5a0: {static npos = 4294967295,
>> > _M_dataplus = {> =
>> > {<__gnu_cxx::new_allocator> = {}, > > fields>}, _M_p = 0x880a20c}}
>> > (gdb) p/c 0xbfbfe5a0
>> > $21 = -96 '\240'
>>
>> So "something" is in both fmt and arg1 (I believe fmt is UCS-4LE
>> format of string "/home/hge/.lyx-devel". Should this be visible on
>> screen?).
>>
>>> bformat(_("You have specified a non-existent user "
>>>"LyX directory, %1$s.\n"
>>>"It is needed to keep your own configuration."),
>>
>> No, lyx dies before the function boost::basic_format() returns a
>> value, i.e. before it evaluates bformat(_( ..., ...)), I think. The
>> trace after this point seems to go out of lyx sources.
>>
>> I don't know whether it is useful information or not, but before
>> bad_format_string() in exception.hpp is called after the above call
>> and right before crash, maybe_throw_exception() in
>> boost/format/paring.hpp contains:
>>
>> boost::io::detail::maybe_throw_exception (exceptions=255 '\377',
>> pos=25, size=28)
>>
>> Thanks,
>>
>> Koji
>>
>
--
Peter Kümmel
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
As I think I'd better continue to throw additional informations on this
problem, I'll do so gradually.
Before the call of from_utf8 in lyx_main.C:
> lyxerr << to_utf8(bformat(_("LyX: Creating directory %1$s"),
> from_utf8(package().user_support().absFilename(
><< endl;
I tried to issue the following command on gdb:
(gdb) print lyx::support::package().user_support().absFilename()
then I got:
> Program received signal SIGSEGV, Segmentation fault.
> 0x2942672a in std::basic_string,
> std::allocator >::basic_string () from
> /usr/local/lib/gcc-4.1.3/libstdc++.so.6
> The program being debugged was signaled while in a function called
> from GDB.
> GDB remains in the frame where the signal was received.
> To change this behavior use "set unwindonsignal on"
> Evaluation of the expression containing the function
> (lyx::support::FileName::absFilename() const) will be abandoned.
Can this related to the problem? However, ordinary execution of the lyx
program issues no problem in getting absFilename() and proceeds to the
evaluation of from_utf8().
Koji
Koji Yokota wrote:
Peter Kümmel wrote:
Could you have a look at the sting passes to boost::basic_format?
support/lstrings.C line 875
OK, it's as follows:
> (gdb) p fmt
> $18 = (const docstring &) @0xbfbfe5a8: {static npos = 4294967295,
> _M_dataplus = {> =
> {<__gnu_cxx::new_allocator> = {}, fields>}, _M_p = 0x87ebb0c}}
> (gdb) p/c 0xbfbfe5a8
> $19 = -88 '\250'
> (gdb) p arg1
> $20 = (docstring &) @0xbfbfe5a0: {static npos = 4294967295,
> _M_dataplus = {> =
> {<__gnu_cxx::new_allocator> = {}, fields>}, _M_p = 0x880a20c}}
> (gdb) p/c 0xbfbfe5a0
> $21 = -96 '\240'
So "something" is in both fmt and arg1 (I believe fmt is UCS-4LE format
of string "/home/hge/.lyx-devel". Should this be visible on screen?).
bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own configuration."),
No, lyx dies before the function boost::basic_format() returns a value,
i.e. before it evaluates bformat(_( ..., ...)), I think. The trace after
this point seems to go out of lyx sources.
I don't know whether it is useful information or not, but before
bad_format_string() in exception.hpp is called after the above call and
right before crash, maybe_throw_exception() in boost/format/paring.hpp
contains:
boost::io::detail::maybe_throw_exception (exceptions=255 '\377', pos=25,
size=28)
Thanks,
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Peter Kümmel wrote:
Could you have a look at the sting passes to boost::basic_format?
support/lstrings.C line 875
OK, it's as follows:
> (gdb) p fmt
> $18 = (const docstring &) @0xbfbfe5a8: {static npos = 4294967295,
> _M_dataplus = {> =
> {<__gnu_cxx::new_allocator> = {}, fields>}, _M_p = 0x87ebb0c}}
> (gdb) p/c 0xbfbfe5a8
> $19 = -88 '\250'
> (gdb) p arg1
> $20 = (docstring &) @0xbfbfe5a0: {static npos = 4294967295,
> _M_dataplus = {> =
> {<__gnu_cxx::new_allocator> = {}, fields>}, _M_p = 0x880a20c}}
> (gdb) p/c 0xbfbfe5a0
> $21 = -96 '\240'
So "something" is in both fmt and arg1 (I believe fmt is UCS-4LE format
of string "/home/hge/.lyx-devel". Should this be visible on screen?).
bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own configuration."),
No, lyx dies before the function boost::basic_format() returns a value,
i.e. before it evaluates bformat(_( ..., ...)), I think. The trace after
this point seems to go out of lyx sources.
I don't know whether it is useful information or not, but before
bad_format_string() in exception.hpp is called after the above call and
right before crash, maybe_throw_exception() in boost/format/paring.hpp
contains:
boost::io::detail::maybe_throw_exception (exceptions=255 '\377', pos=25,
size=28)
Thanks,
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Koji Yokota wrote:
> I attach the backtrace when the program is crashed as additional
> information.
>
> Koji
>
> ---
> Program received signal SIGABRT, Aborted.
> 0x2958a897 in kill () from /lib/libc.so.6
> (gdb) bt
> #0 0x2958a897 in kill () from /lib/libc.so.6
> #1 0x2949d4f7 in raise () from /lib/libpthread.so.2
> #2 0x295892fb in abort () from /lib/libc.so.6
> #3 0x29448dc4 in __gnu_cxx::__verbose_terminate_handler ()
>from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
> #4 0x2944673a in __cxxabiv1::__terminate ()
>from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
> #5 0x29446782 in std::terminate ()
>from /usr/local/lib/gcc-4.1.3/libstdc++.so.6
> #6 0x294468ba in __cxa_throw () from
> /usr/local/lib/gcc-4.1.3/libstdc++.so.6
> #7 0x0808d15e in boost::throw_exception
> (e=Could not find the frame base for "void
> boost::throw_exception(boost::io::bad_format_string
> const&)".
> )
> at throw_exception.hpp:39
> #8 0x0808d367 in boost::io::detail::maybe_throw_exception (
> exceptions=255 '', pos=25, size=28) at parsing.hpp:106
> #9 0x0808dea8 in boost::io::detail::parse_printf_directive std::char_traits, std::allocator,
> __gnu_cxx::__normal_iterator std::char_traits, std::allocator > >,
> std::ctype > ([EMAIL PROTECTED], [EMAIL PROTECTED],
> fpar=0x87f1800, [EMAIL PROTECTED], offset=25, exceptions=255 '')
> at parsing.hpp:333
> #10 0x0808ec0b in boost::basic_format std::char_traits, std::allocator >::parse
> (this=0xbfbfe404, [EMAIL PROTECTED])
> at parsing.hpp:437
> #11 0x0808f020 in basic_format (this=0xbfbfe404, [EMAIL PROTECTED])
> at format_implementation.hpp:64
> #12 0x0855f043 in lyx::support::bformat std::char_traits, std::allocator > > ([EMAIL PROTECTED],
> [EMAIL PROTECTED])
> at lstrings.C:875
Could you have a look at the sting passes to boost::basic_format?
support/lstrings.C line 875
> #13 0x081a31f1 in lyx::LyX::queryUserLyXDir (this=0xbfbfe834,
> explicit_userdir=false) at lyx_main.C:1095
If the above string is not empty goto lyx_main.C:1095
and check if boost doesn't like the format string:
bformat(_("You have specified a non-existent user "
"LyX directory, %1$s.\n"
"It is needed to keep your own configuration."),
Maybe removing \n or merging the lines helps.
Peter
> #14 0x081a45e6 in lyx::LyX::init (this=0xbfbfe834) at lyx_main.C:845
> #15 0x081a584f in lyx::LyX::init (this=0xbfbfe834, [EMAIL PROTECTED],
> argv=0xbfbfe884) at lyx_main.C:535
> #16 0x081a60b9 in lyx::LyX::exec (this=0xbfbfe834, [EMAIL PROTECTED],
> argv=0xbfbfe884) at lyx_main.C:439
> #17 0x080697c7 in main (argc=1, argv=0xbfbfe884) at main.C:48
>
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
ps.
The last call of processor.convert() is called by getEnv("LC_ALL") in
function Messages::get() in messages.C when it handles strings "LyX:
Creating directory %1$s".
Koji Yokota wrote:
When it is called just before crash:
(1) Before
(gdb) p inbuf:$29 = 0x87f22ec "C"
(gdb) p inbytesleft:$30 = 4
(gdb) p outbuf:$31 = 0x8750e60
"/home/ujis/print/lyx-devel2/work/lyx-1.5.0beta2/src/lyx-qt4"
(gdb) p outsize:$32 = 32768
(2) After
(gdb) p inbuf:$33 = 0x87f22ec "C"
(gdb) p inbytesleft:$34 = 4
(gdb) p outbuf:$35 = 0x8750e60
"Chome/ujis/print/lyx-devel2/work/lyx-1.5.0beta2/src/lyx-qt4"
(gdb) p outsize:$36 = 32768
(gdb) p bytes:$37 = 1
Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
I attach the backtrace when the program is crashed as additional information. Koji --- Program received signal SIGABRT, Aborted. 0x2958a897 in kill () from /lib/libc.so.6 (gdb) bt #0 0x2958a897 in kill () from /lib/libc.so.6 #1 0x2949d4f7 in raise () from /lib/libpthread.so.2 #2 0x295892fb in abort () from /lib/libc.so.6 #3 0x29448dc4 in __gnu_cxx::__verbose_terminate_handler () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #4 0x2944673a in __cxxabiv1::__terminate () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #5 0x29446782 in std::terminate () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #6 0x294468ba in __cxa_throw () from /usr/local/lib/gcc-4.1.3/libstdc++.so.6 #7 0x0808d15e in boost::throw_exception (e=Could not find the frame base for "void boost::throw_exception(boost::io::bad_format_string const&)". ) at throw_exception.hpp:39 #8 0x0808d367 in boost::io::detail::maybe_throw_exception ( exceptions=255 '', pos=25, size=28) at parsing.hpp:106 #9 0x0808dea8 in boost::io::detail::parse_printf_directive, std::allocator, __gnu_cxx::__normal_iterator, std::allocator > >, std::ctype > ([EMAIL PROTECTED], [EMAIL PROTECTED], fpar=0x87f1800, [EMAIL PROTECTED], offset=25, exceptions=255 '') at parsing.hpp:333 #10 0x0808ec0b in boost::basic_format, std::allocator >::parse (this=0xbfbfe404, [EMAIL PROTECTED]) at parsing.hpp:437 #11 0x0808f020 in basic_format (this=0xbfbfe404, [EMAIL PROTECTED]) at format_implementation.hpp:64 #12 0x0855f043 in lyx::support::bformat, std::allocator > > ([EMAIL PROTECTED], [EMAIL PROTECTED]) at lstrings.C:875 #13 0x081a31f1 in lyx::LyX::queryUserLyXDir (this=0xbfbfe834, explicit_userdir=false) at lyx_main.C:1095 #14 0x081a45e6 in lyx::LyX::init (this=0xbfbfe834) at lyx_main.C:845 #15 0x081a584f in lyx::LyX::init (this=0xbfbfe834, [EMAIL PROTECTED], argv=0xbfbfe884) at lyx_main.C:535 #16 0x081a60b9 in lyx::LyX::exec (this=0xbfbfe834, [EMAIL PROTECTED], argv=0xbfbfe884) at lyx_main.C:439 #17 0x080697c7 in main (argc=1, argv=0xbfbfe884) at main.C:48
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: What is the value of "out"? This is the important one. Uh, is this mean encodings of output? If it is, it's "UCS-4LE". Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: Koji, I am slowly looking at this again. It is normal that the value of outbuf is changed by iconv, since it is incremented to point to the end of the converted string. I see. I am not sure. What are the values of the variables before and after the call int bytes = processor.convert(inbuf, inbytesleft, outbuf, outsize); in iconv_convert? What is the return value? It seems this function is called multiple times, initially succeeds and fails later. When the call was successful: (1) Before the call of processor.convert() (gdb) p inbuf: $3 = 0x878850c "T" (gdb) p inbytesleft:$4 = 36 (gdb) p outbuf: $5 = 0x8750e60 "The owner" (gdb) p outsize:$6 = 32768 (2) After (gdb) p inbuf: $7 = 0x878850c "T" (gdb) p inbytesleft:$8 = 36 (gdb) p outbuf: $9 = 0x8750e60 "The owner" (gdb) p outsize:$10 = 32768 (gdb) p bytes: $11 = 9 When it is called just before crash: (1) Before (gdb) p inbuf: $29 = 0x87f22ec "C" (gdb) p inbytesleft:$30 = 4 (gdb) p outbuf: $31 = 0x8750e60 "/home/ujis/print/lyx-devel2/work/lyx-1.5.0beta2/src/lyx-qt4" (gdb) p outsize:$32 = 32768 (2) After (gdb) p inbuf: $33 = 0x87f22ec "C" (gdb) p inbytesleft:$34 = 4 (gdb) p outbuf: $35 = 0x8750e60 "Chome/ujis/print/lyx-devel2/work/lyx-1.5.0beta2/src/lyx-qt4" (gdb) p outsize:$36 = 32768 (gdb) p bytes: $37 = 1 Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Can the attached unicode.C.patch be a solution? With the patch, Koji> I can compile lyx (but I got a crash in another place, sigh...). I am not sure. What are the values of the variables before and after the call int bytes = processor.convert(inbuf, inbytesleft, outbuf, outsize); in iconv_convert? What is the return value? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
>> I think the 'in' and 'out' are iso8859-1 because the passed strings >> are a directory name. When iconv is called in unicode.C: >>int res = iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf, >> &outbytesleft); >> output of gdb before call is: >> inbuf : $3 = 0x8a79d0c "/home/hge/.lyx-devel/" inbytesleft : $4 = >> 21 outbuf : $5 = 0x8b01d8c "" outbytesleft : $6 = 84 >> and after: >> inbuf : $8 = 0x8a79d21 "" inbytesleft : $10 = 0 outbuf : $9 = >> 0x8b01de0 "" outbytesleft : $11 = 0 res : $12 = 0 This is normal. >> Here, return value is zero. This too. What is the value of "out"? This is the important one. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Hi, I'm investigating in this problem and it seems the address Koji> of outbuf changes before and after of the call of Koji> iconv((pimpl_->cd, &inbuf, &inbytesleft, &outbuf, Koji> &outbytesleft); Koji, I am slowly looking at this again. It is normal that the value of outbuf is changed by iconv, since it is incremented to point to the end of the converted string. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> Jean-Marc Lasgouttes wrote: >> What architecture is it? Koji> Sorry, architecture is all i386 (one is amd64 running in i386 Koji> mode). So it is not a problem of endianness. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote: What architecture is it? Sorry, architecture is all i386 (one is amd64 running in i386 mode). Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean-Marc Lasgouttes wrote:
Does the behaviour depend from current locale settings?
No, it doesn't seem so. The same thing happens under LC_ALL=C.
What architecture is it?
All are FreeBSD-6.2-RELEASE with current ports programs.
Can you reproduce it with a minimal C(++) file?
Yeah, I tried and there may be a problem in reference to the output of
libiconv conversion in src/support/unicode.C. Please check if it is
correct or not as I'm an amateur in this matter.
I wrote a small test program in C (test.c) and got a result as in
result.txt (both attached). All conversions ISO-8859-1 <-> UCS etc. by
libiconv seem to work fine, but the output is stored not in outbuf but
pointed address of outbuf (i.e. the variable &out_str in test.c gets the
outcome but not &outbuf).
Can the attached unicode.C.patch be a solution? With the patch, I can
compile lyx (but I got a crash in another place, sigh...). That is, in
unicode.C,
> int IconvProcessor::convert(char const * buf, size_t buflen,
> char * outbuf, size_t maxoutsize)
> {
>...
>char ICONV_CONST * inbuf = const_cast(buf);
> + char * outbuf2 = outbuf;
>size_t inbytesleft = buflen;
>size_t outbytesleft = maxoutsize;
>
> - int res = iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf,
&outbytesleft);
> + int res = iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf2,
&outbytesleft);
I added lines with + in stead of lines with -.
Thanks,
Koji
// cc -liconv -I /usr/local/include -L /usr/local/lib test.c
#include
#include
#include
main()
{
iconv_t cd;
char in_str[22] = "ABCDE";
char out_str[22] = "";
char out_str2[22] = "";
char const * inbuf = in_str;
char * outbuf = out_str;
size_t inbytesleft = 5;
size_t outbytesleft = 50;
size_t res;
int i;
int arg=1;
char fromcode[11] = "ISO-8859-1";
char tocode[7][12];
strcpy(tocode[0],"UTF-8");
strcpy(tocode[1],"ISO-2022-JP");
strcpy(tocode[2],"CP932");
strcpy(tocode[3],"EUC-JP");
strcpy(tocode[4],"UCS-4LE");
strcpy(tocode[5],"UCS-4BE");
strcpy(tocode[6],"UCS-4");
for(i=0;i<=6;i++){
printf("\n* Start conversion: %s -> %s -> %s
*\n",fromcode,tocode[i],fromcode);
printf("* before 1st conversion: %s -> %s *\n",fromcode,tocode[i]);
printf("inbuf= %s\t\toutbuf = %s\n", inbuf, outbuf);
cd = iconv_open(tocode[i], fromcode);
res = iconv(cd, &inbuf, &inbytesleft, &outbuf, &outbytesleft);
iconv_close(cd);
printf("* after 1st conversion *\n");
printf("inbuf= %s\t\tinbytesleft = %d\n", inbuf, inbytesleft);
printf("outbuf = %s\t\toutbytesleft = %d\n", outbuf, outbytesleft);
printf("out_str = %s\t\tres = %d\n", out_str, res);
inbuf = out_str;
outbuf = out_str2;
inbytesleft = 50 - outbytesleft;
outbytesleft = 5;
printf("* before 2nd conversion: %s -> %s *\n",tocode[i],fromcode);
printf("inbuf= %s\t\toutbuf = %s\n", inbuf, outbuf);
cd = iconv_open(fromcode, tocode[i]);
res = iconv(cd, &inbuf, &inbytesleft, &outbuf, &outbytesleft);
iconv_close(cd);
printf("* after 2nd conversion *\n");
printf("inbuf= %s\t\tinbytesleft = %d\n", inbuf, inbytesleft);
printf("outbuf = %s\t\toutbytesleft = %d\n", outbuf, outbytesleft);
printf("out_str = %s\t\tres = %d\n", out_str2, res);
strcpy(out_str,"");
strcpy(out_str2,"");
inbuf = in_str;
outbuf = out_str;
inbytesleft=5;
outbytesleft=50;
}
}
* Start conversion: ISO-8859-1 -> UTF-8 -> ISO-8859-1 *
* before 1st conversion: ISO-8859-1 -> UTF-8 *
inbuf= ABCDEoutbuf =
* after 1st conversion *
inbuf= inbytesleft = 0
outbuf = outbytesleft = 45
out_str = ABCDEres = 0
* before 2nd conversion: UTF-8 -> ISO-8859-1 *
inbuf= ABCDEoutbuf =
* after 2nd conversion *
inbuf= inbytesleft = 0
outbuf = outbytesleft = 0
out_str = ABCDEres = 0
* Start conversion: ISO-8859-1 -> ISO-2022-JP -> ISO-8859-1 *
* before 1st conversion: ISO-8859-1 -> ISO-2022-JP *
inbuf= ABCDEoutbuf =
* after 1st conversion *
inbuf= inbytesleft = 0
outbuf = outbytesleft = 45
out_str = ABCDEres = 0
* before 2nd conversion: ISO-2022-JP -> ISO-8859-1 *
inbuf= ABCDEoutbuf =
* after 2nd conversion *
inbuf= inbytesleft = 0
outbuf = outbytesleft = 0
out_str = ABCDEres = 0
* Start conversion: ISO-8859-1 -> CP932 -> ISO-8859-1 *
* before 1st conversion: ISO-8859-1 -> CP932 *
inbuf= ABCDEoutbuf =
* after 1st conversion *
inbuf= inbytesleft = 0
outbuf =
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Koji> I confirmed the same bug arises in at least three freebsd boxes Koji> (all stable release version). Does the behaviour depend from current locale settings? What architecture is it? Can you reproduce it with a minimal C(++) file? JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Correction. iconv conversion is from UTF-8 to UCS4-LE. 1) Initially, IconvProcessor::init() is called from IconvProcessor::convert(). 2) Then, since pimpl_->cd = invalid_cd, iconv_open() is called. 3) arguments of iconv_open(tocode_.c_str(),fromcode_.c_str()) is: tocode_.c_str() : 0x8a6dfcc "UCS-4LE" fromcode_.c_str() : 0x8a8006c "UTF-8" 4) Then pimpl_->cd != invalid_cd, so IconvProcessor::init() returns true. 5) Then, iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf, &outbytesleft) is called and returns zero with wrong results. On the same system, using binary "iconv" which comes with libiconv-1.9.2, COMMAND LINE> echo /home/hge/.lyx-devel/ | iconv -f UTF-8 -t UCS-4LE STDOUT> /home/hge/.lyx-devel/ which works fine. I confirmed the same bug arises in at least three freebsd boxes (all stable release version). Koji Koji Yokota wrote: Jean and Abdel, libiconv is compiled from the "port" package management and its version is 1.9.2, so I can say that the problem occurs under a fairly standard setting of FreeBSD (6.2-(stable)release with updated ports). It is installed under /usr/local/ because, in FreeBSD, all files which come from "port" system or pre-compiled packages are guided to install there. I think the 'in' and 'out' are iso8859-1 because the passed strings are a directory name. When iconv is called in unicode.C: int res = iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf, &outbytesleft); output of gdb before call is: inbuf: $3 = 0x8a79d0c "/home/hge/.lyx-devel/" inbytesleft : $4 = 21 outbuf : $5 = 0x8b01d8c "" outbytesleft : $6 = 84 and after: inbuf: $8 = 0x8a79d21 "" inbytesleft : $10 = 0 outbuf : $9 = 0x8b01de0 "" outbytesleft : $11 = 0 res : $12 = 0 Here, return value is zero. On the other hand, the man page of libiconv-1.9.2 says: > SYNOPSIS >#include > >size_t iconv (iconv_t cd, > const char* * inbuf, size_t * inbytesleft, > char* * outbuf, size_t * outbytesleft); Since the man page is borrowed from linux --- its title says "Linux Programmer's Manual" ---, there is a risk that it is not correctly modified to suit FreeBSD, but as far as I see in iconv.h, it is appropriately modified. I attach the /usr/local/include/iconv.h file (there is no /usr/include/iconv.h file). Koji
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Jean and Abdel, libiconv is compiled from the "port" package management and its version is 1.9.2, so I can say that the problem occurs under a fairly standard setting of FreeBSD (6.2-(stable)release with updated ports). It is installed under /usr/local/ because, in FreeBSD, all files which come from "port" system or pre-compiled packages are guided to install there. I think the 'in' and 'out' are iso8859-1 because the passed strings are a directory name. When iconv is called in unicode.C: int res = iconv(pimpl_->cd, &inbuf, &inbytesleft, &outbuf, &outbytesleft); output of gdb before call is: inbuf: $3 = 0x8a79d0c "/home/hge/.lyx-devel/" inbytesleft : $4 = 21 outbuf : $5 = 0x8b01d8c "" outbytesleft : $6 = 84 and after: inbuf: $8 = 0x8a79d21 "" inbytesleft : $10 = 0 outbuf : $9 = 0x8b01de0 "" outbytesleft : $11 = 0 res : $12 = 0 Here, return value is zero. On the other hand, the man page of libiconv-1.9.2 says: > SYNOPSIS >#include > >size_t iconv (iconv_t cd, > const char* * inbuf, size_t * inbytesleft, > char* * outbuf, size_t * outbytesleft); Since the man page is borrowed from linux --- its title says "Linux Programmer's Manual" ---, there is a risk that it is not correctly modified to suit FreeBSD, but as far as I see in iconv.h, it is appropriately modified. I attach the /usr/local/include/iconv.h file (there is no /usr/include/iconv.h file). Koji Abdelrazak Younes wrote: Jean-Marc Lasgouttes wrote: "Koji" == Koji Yokota <[EMAIL PROTECTED]> writes: Abdelrazak> Ah, I misread this report. I guess iconv is the real Abdelrazak> problem here. Jean-Marc> Yes there seems to be a problem with the autoconf test (a Jean-Marc> missing const). Could I get to see the whole config.log? I am not so sure that the iconv test fails after all. Koji> Yes, here I attach the config.log file. Thanks. As far as I can see the tests for iconv work perfectly. The second time it fails is the just the indication that a 'const' is needed for the second argument, and this is correctly detected: configure:25076: result: extern size_t iconv (iconv_t cd, const char * *inbuf, size_t *inbytesleft, char * *outbuf, size_t *outbytesleft); So the problem seems to be that iconv does not work as expected. Koji, can you give more details? What is the return value from iconv() when it does not work? What are the 'in' and 'out' encodings in this case? Can you check whether they are supported by your version of iconv? Also, is this a locally compiled version of iconv? I ask because it is located in /usr/local/ which AFAIK is not populated by system packages. I am also worried about the 'const' requirement which is a windows think IIRC. I guess FreeBSD have a proper version of iconv in its "port" package management, why don't you try with that? Abdel. -- 横田 宏治 ([EMAIL PROTECTED]) 小樽商科大学 商学部 経済学科 /* Copyright (C) 1999-2003 Free Software Foundation, Inc. This file is part of the GNU LIBICONV Library. The GNU LIBICONV Library is free software; you can redistribute it and/or modify it under the terms of the GNU Library General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. The GNU LIBICONV Library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public License for more details. You should have received a copy of the GNU Library General Public License along with the GNU LIBICONV Library; see the file COPYING.LIB. If not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */ /* When installed, this file is called "iconv.h". */ #ifndef _LIBICONV_H #define _LIBICONV_H #define _LIBICONV_VERSION 0x0109/* version number: (major<<8) + minor */ extern int _libiconv_version; /* Likewise */ /* We would like to #include any system header file which could define iconv_t, 1. in order to eliminate the risk that the user gets compilation errors because some other system header file includes /usr/include/iconv.h which defines iconv_t or declares iconv after this file, 2. when compiling for LIBICONV_PLUG, we need the proper iconv_t type in order to produce binary compatible code. But gcc's #include_next is not portable. Thus, once libiconv's iconv.h has been installed in /usr/local/include, there is no way any more to include the original /usr/include/iconv.h. We simply have to get away without it. Ad 1. The risk that a system header file does #include "iconv.h" or #include_next "iconv.h" is small. They all do #include . Ad 2. The iconv_t type is a pointer type in all cases I have seen. (It has to be a scalar type because (iconv_t)(
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Fri, May 04, 2007 at 07:47:06PM +0200, Abdelrazak Younes wrote: > Andre Poenitz wrote: > >On Fri, May 04, 2007 at 06:49:32PM +0200, Andre Poenitz wrote: > >>>About this particular one, I proposed to make the method pure virtual, > >>>any objection? > >>Unless you do not end up with 85 implementations, that's fine with me. ^^^^^ Neg. Neg. > >Negation is considered a projection... > > What do you mean? If for an operator P the equality P o P = P holds true (i.e. basically $P(P(x)) = P(x) \forall x \in \Omega$), it is called a 'projection'. To make a double negation mean the same as a single negation (and only a single negation is needed for wat I was trying to say originally), it would be sufficient, if negation was a projection... Now, of course you could argue that it isn't, but then I simply won't listen ;-} Andre'
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Andre Poenitz wrote: On Fri, May 04, 2007 at 06:49:32PM +0200, Andre Poenitz wrote: About this particular one, I proposed to make the method pure virtual, any objection? Unless you do not end up with 85 implementations, that's fine with me. Negation is considered a projection... What do you mean? Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Fri, May 04, 2007 at 06:49:32PM +0200, Andre Poenitz wrote: > > About this particular one, I proposed to make the method pure virtual, > > any objection? > > Unless you do not end up with 85 implementations, that's fine with me. Negation is considered a projection... Andre'
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
On Fri, May 04, 2007 at 02:41:55PM +0200, Abdelrazak Younes wrote: > Georg Baum wrote: > >And then look at the very recent marginal bug: AFAIK (correct me if I am > >wrong) this happened because of missing review of the insetName -> name > >change. > > Small correction: this bug appeared in the InsetOld -> Inset transfer > with the removal of setInsetName(). Nothin to do with the insetName -> > name change. > > >A simple and safe looking change caused a serious bug. > > About this particular one, I proposed to make the method pure virtual, > any objection? Unless you do not end up with 85 implementations, that's fine with me. Andre'
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: Abdelrazak Younes wrote: Georg Baum wrote: And then look at the very recent marginal bug: AFAIK (correct me if I am wrong) this happened because of missing review of the insetName -> name change. Small correction: this bug appeared in the InsetOld -> Inset transfer with the removal of setInsetName(). Nothin to do with the insetName -> name change. Thanks for the correction. It does not change much: I just looked it up, that rename also went in without review. I think I've warned that I will do it. There was not much to review IMHO, but anyway point taken. A simple and safe looking change caused a serious bug. About this particular one, I proposed to make the method pure virtual, any objection? Looks good provided you are 100% sure that the names are correct. JMarc is not in favour so I postpone this to 1.6. I've reviewed the InsetOld removing commit and I don't think there was any other error than the "Marginal" one. Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Abdelrazak Younes wrote: > Georg Baum wrote: >> And then look at the very recent marginal bug: AFAIK (correct me if I am >> wrong) this happened because of missing review of the insetName -> name >> change. > > Small correction: this bug appeared in the InsetOld -> Inset transfer > with the removal of setInsetName(). Nothin to do with the insetName -> > name change. Thanks for the correction. It does not change much: I just looked it up, that rename also went in without review. >> A simple and safe looking change caused a serious bug. > > About this particular one, I proposed to make the method pure virtual, > any objection? Looks good provided you are 100% sure that the names are correct. Georg
lots of unamed insets! (was Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
Abdelrazak Younes wrote: Georg Baum wrote: And then look at the very recent marginal bug: AFAIK (correct me if I am wrong) this happened because of missing review of the insetName -> name change. Small correction: this bug appeared in the InsetOld -> Inset transfer with the removal of setInsetName(). Nothin to do with the insetName -> name change. A simple and safe looking change caused a serious bug. About this particular one, I proposed to make the method pure virtual, any objection? If I do that, here is the list of unamed insets, could someone review them please? 'lyx::InsetVSpace' : cannot instantiate abstract class 'lyx::InsetUrl' : cannot instantiate abstract class 'lyx::InsetTOC' : cannot instantiate abstract class 'lyx::InsetNewline' : cannot instantiate abstract class 'lyx::InsetText' : cannot instantiate abstract class 'lyx::InsetTabular' : cannot instantiate abstract class 'lyx::InsetSpecialChar' : cannot instantiate abstract class 'lyx::InsetSpace' : cannot instantiate abstract class 'lyx::InsetRef' : cannot instantiate abstract class 'lyx::InsetPagebreak' : cannot instantiate abstract class 'lyx::InsetClearPage' : cannot instantiate abstract class 'lyx::InsetClearDoublePage' : cannot instantiate abstract class 'lyx::InsetNomencl' : cannot instantiate abstract class 'lyx::InsetPrintNomencl' : cannot instantiate abstract class 'lyx::InsetNewline' : cannot instantiate abstract class 'lyx::InsetLine' : cannot instantiate abstract class 'lyx::InsetLabel' : cannot instantiate abstract class 'lyx::InsetIndex' : cannot instantiate abstract class 'lyx::InsetPrintIndex' : cannot instantiate abstract class 'lyx::InsetInclude' : cannot instantiate abstract class 'lyx::InsetHFill' : cannot instantiate abstract class 'lyx::InsetGraphics' : cannot instantiate abstract class 'lyx::InsetFloatList' : cannot instantiate abstract class 'lyx::InsetExternal' : cannot instantiate abstract class 'lyx::InsetCitation' : cannot instantiate abstract class 'lyx::InsetCaption' : cannot instantiate abstract class 'lyx::InsetBibtex' : cannot instantiate abstract class 'lyx::InsetBibitem' : cannot instantiate abstract class Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: And then look at the very recent marginal bug: AFAIK (correct me if I am wrong) this happened because of missing review of the insetName -> name change. Small correction: this bug appeared in the InsetOld -> Inset transfer with the removal of setInsetName(). Nothin to do with the insetName -> name change. A simple and safe looking change caused a serious bug. About this particular one, I proposed to make the method pure virtual, any objection? Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Georg" == Georg Baum <[EMAIL PROTECTED]> writes: Georg> And then look at the very recent marginal bug: AFAIK (correct Georg> me if I am wrong) this happened because of missing review of Georg> the insetName -> name change. A simple and safe looking change Georg> caused a serious bug. I am convinced that more bugs of this Georg> sort will be found after 1.5.0 is released, because the Georg> practice of committing without review is followed since some Georg> time. Could not agree more. JMarc
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Georg Baum wrote: Abdelrazak Younes wrote: Georg Baum wrote: think I need to reiterate all of them, one example can be seen in my other mail of today. Georg, you know very well that you have to get involved in order to lead. Uwe is simply not as competent as you in this field, I guess none of us is. And that coming from the guy who wrote I should not write "you should" so often :-) Sorry, but leading as in "imposing my view on everybody else" with very little action on my side is nothing I am interested in. And as you found out yourself it does not work either ;-) Indeed :-) But leading in your area of expertise would work I think... Another one is too little focus on 1.5.0 release. That's the reason why I proposed to release it soon and impose a short period of hard freeze for bug fixing. I am all for short release cycles. Unfortunately bugs don't fix themselves, and as it looks now some are set to stay forever. Only until someone fix them. There are an impressive number of old bugs that were fixed in the 1.5 cycle. The funny thing is that LyX is supposed to be in hard freeze since October, but nobody cares. I don't think nobody cares (I do). Anyway, now that we learned from it, let's just forget the past. Since it would be arrogant to demand from all of you to adjust to my wishes for the very little amount I could contribute I chose the other alternative: Don't help with a product that I cannot endorse. What about this third alternative: contribute to your area expertise and focus on that. You don't have to endorse all of 1.5, only the changes that you authored. 1.5.0 will be a damn good release, the initial reaction from the users list shows. You are probably the biggest contributor in 1.5 and you should feel proud of it. The armenian stuff is a perfect example why this does not work: Uwe did the work and has a different opinion than me. He sees one minor quirk, I see several minor and one big one. This is my area of expertise, so I could fix all the issues after Uwe is finished, and I know he won't oppose. But this way, others are dictating what I would do, and all fun is gone. You could have said a bit louder that this Armenian support is just not ready. Like we are all trying to say to Bo that InsetListing is probably not going to make it for 1.6. Of course I could simply ignore what others are doing and work on some pet projects. Then I could have some fun, but that would be unfair to everybody doing bug fixing, and the overall result would still be something I don't like, because it would contain stuff like the armenian support. Once 1.5 is out you will have fun again, promised :-) Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Michael Gerz wrote: > I looked up the meaning of "endorse" and I must confess that I am sorry > that you cannot endorse a product on which you have spent sooo much > spare time. > > Apart from the Armenian support and the recent InsetListings discussion, > what exactly is missing to make LyX 1.5.0 a great product - better than > any other release we have published before? It will no doubt be a great product, but not better than any other release. 1.3.7 was the best release I know. Of course I know that a .0 release can never compete with a .7 release WRT stability, so we should not assert that. 1.5.0 will also transport a certain message to the users: "Hey, we ripped everything apart before 1.4.0, put together some of the most important things again for 1.4.0, put together some more for 1.5.0, added many new and exciting features for 1.5.0, but the rest will stay ripped apart forever and nobody cares". I don't want to be connected to such a message in any way, and therefore I wrote that I cannot endorse 1.5.0. Don't get me wrong, I agree that this ripping apart was necessary, but IMO it is also necessary to put things back together again. And since until now nobody was interested in doing so the chance is very low that somebody will become interested after 1.5.0. > And a software that eases > _your_ life? I don't think I will switch to 1.5 for my thesis. I somehow adjusted to the quirks of 1.4, put some additional stuff that eases my life in my personal branch, and I prefer living with the known problems over the risk of unknown ones. Some of them (math speed problems) are fixed by brute force: I have a fast machine. By the time I would switch (1.5.2 or so) I am hopefully finished. > Could you point us to some bug reports that you consider > critical for the success of LyX 1.5.0? I guess, the whole LyX community > is eager to make you happy! The bugs that annoy me most are - unusable scrollbar for long insets - the math regressions WRT 1.3: wrong font for unknown commands, unlogical cursor movement, wrong color for some symbols - 1973 - 2555 - 2556 (although I got used to the workaround with the space) - 3291 There might be some more that I don't remember currently, but that means that they are probably not too important. And then look at the very recent marginal bug: AFAIK (correct me if I am wrong) this happened because of missing review of the insetName -> name change. A simple and safe looking change caused a serious bug. I am convinced that more bugs of this sort will be found after 1.5.0 is released, because the practice of committing without review is followed since some time. > As you know, I promised to give some gifts to the most busy bees of the > LyX 1.5.0 development cycle. No matter happens, a prizes will go to you > and Abdel (third prize is still vacant). Thank you for that. Georg
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Abdelrazak Younes wrote: > Georg Baum wrote: >> think I need to reiterate all of them, one example can be seen in my >> other mail of today. > > Georg, you know very well that you have to get involved in order to > lead. Uwe is simply not as competent as you in this field, I guess none > of us is. And that coming from the guy who wrote I should not write "you should" so often :-) Sorry, but leading as in "imposing my view on everybody else" with very little action on my side is nothing I am interested in. And as you found out yourself it does not work either ;-) >> Another one is too little focus on 1.5.0 release. > > That's the reason why I proposed to release it soon and impose a short > period of hard freeze for bug fixing. I am all for short release cycles. Unfortunately bugs don't fix themselves, and as it looks now some are set to stay forever. The funny thing is that LyX is supposed to be in hard freeze since October, but nobody cares. >> Since it would be arrogant to demand from all of you to adjust to my >> wishes for the very little amount I could contribute I chose the other >> alternative: Don't help with a product that I cannot endorse. > > What about this third alternative: contribute to your area expertise and > focus on that. You don't have to endorse all of 1.5, only the changes > that you authored. 1.5.0 will be a damn good release, the initial > reaction from the users list shows. You are probably the biggest > contributor in 1.5 and you should feel proud of it. The armenian stuff is a perfect example why this does not work: Uwe did the work and has a different opinion than me. He sees one minor quirk, I see several minor and one big one. This is my area of expertise, so I could fix all the issues after Uwe is finished, and I know he won't oppose. But this way, others are dictating what I would do, and all fun is gone. Of course I could simply ignore what others are doing and work on some pet projects. Then I could have some fun, but that would be unfair to everybody doing bug fixing, and the overall result would still be something I don't like, because it would contain stuff like the armenian support. Georg
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
Bo Peng wrote:
Ouch... this is a big, non obvious, patch Bo... I am not sure we will
find the time to properly review it and add the missing bits before
1.5.0 :-(
InsetListings ~= InsetERT so you do not have to pay much attention to
it, other than the InsetListingsParams part.
Than why don't you merge the two instead? Or inherit from it? I
personally really dislike code repetition, this is asking for trouble
later on.
Many other small changes are startard methods to add dialog, add
require('listings') and add the inset itself. They can be ignored as
well.
I'll try to have a look this week-end. But do we really need this dialog
now?
What I need from you is the qt4/QListings.* and qt4/ui/ListingUi.ui
part. You can get the parameters and put them to the listWidget, when
one of the items is clicked, its value goes to the editbox and when
the focus leaves the editbox, its value should be saved in the
listwidget. Then, the parameters are returned to the inset. If you are
not sure how parameters are passed, I am not sure either, but I copied
the code from QWrap. :-) InsetWrap has something like InsetWrapParams
as well.
Why don't you copy the code architecture from one of the new dialogs
instead: View source is easy enough and you'll find your way in it.
If you can find a clever way to put this dialog to insert -> include
(as the fourth file type), then we can easily have \lstinputlisting.
If you can put it also to file -> settings, then we can have proper
\lstset in the preamble.
I have to apply and test the patch to understand what you mean. Not
before the week-end. I do not promise anything though because I already
have some bug-fixing patches in the pipe and these have higher priority.
Abdel.
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
...
> Index: src/insets/Inset.cpp
> ===
> --- src/insets/Inset.cpp (revision 18187)
> +++ src/insets/Inset.cpp (working copy)
> @@ -76,6 +76,7 @@
> InsetName("bibtex", Inset::BIBTEX_CODE),
> InsetName("text", Inset::TEXT_CODE),
> InsetName("ert", Inset::ERT_CODE),
> + InsetName("listings", Inset::ERT_CODE),
Shouldn't this have its own code? Naive question
> InsetName("foot", Inset::FOOT_CODE),
> InsetName("margin", Inset::MARGIN_CODE),
> InsetName("float", Inset::FLOAT_CODE),
ERT_CODE dictates verbatim output which is exactly what InsetListings
needs. I do not see a need for Inset::LISTINGS_CODE right now but this
may turn out to be wrong.
Cheers,
Bo
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
On Thu, May 03, 2007 at 03:07:58PM -0500, Bo Peng wrote:
> >
> > What do you need?
>
> I am attaching what I have right now. Many parts are missing but the
> main points are there:
>
> What I have got:
...
> Index: src/insets/Inset.cpp
> ===
> --- src/insets/Inset.cpp (revision 18187)
> +++ src/insets/Inset.cpp (working copy)
> @@ -76,6 +76,7 @@
> InsetName("bibtex", Inset::BIBTEX_CODE),
> InsetName("text", Inset::TEXT_CODE),
> InsetName("ert", Inset::ERT_CODE),
> + InsetName("listings", Inset::ERT_CODE),
Shouldn't this have its own code? Naive question
> InsetName("foot", Inset::FOOT_CODE),
> InsetName("margin", Inset::MARGIN_CODE),
> InsetName("float", Inset::FLOAT_CODE),
- Martin
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
Ouch... this is a big, non obvious, patch Bo... I am not sure we will
find the time to properly review it and add the missing bits before
1.5.0 :-(
InsetListings ~= InsetERT so you do not have to pay much attention to
it, other than the InsetListingsParams part.
Many other small changes are startard methods to add dialog, add
require('listings') and add the inset itself. They can be ignored as
well.
I'll try to have a look this week-end. But do we really need this dialog
now?
What I need from you is the qt4/QListings.* and qt4/ui/ListingUi.ui
part. You can get the parameters and put them to the listWidget, when
one of the items is clicked, its value goes to the editbox and when
the focus leaves the editbox, its value should be saved in the
listwidget. Then, the parameters are returned to the inset. If you are
not sure how parameters are passed, I am not sure either, but I copied
the code from QWrap. :-) InsetWrap has something like InsetWrapParams
as well.
If you can find a clever way to put this dialog to insert -> include
(as the fourth file type), then we can easily have \lstinputlisting.
If you can put it also to file -> settings, then we can have proper
\lstset in the preamble.
It sounds like you are doing most of the work, but I am really not
familiar with qt4 to do them. Maybe Peter can also help?
Regarding file format changes, there will be
1. InsetListings...
\begin_inset Listings
language Python
xleftmargin 10pt
status open
\begin_layout Standard
program
\end_layout
\end_inset
There will be MANY listings options. (see later)
2. inputlisting
\begin_inset Include \lstinputlisting{file.py}
language Python
startnumber 10
\end_inset
3. preamble changes if we add a listings panel in document settings dialog.
\language Python
\
Due to the large number of options, I propose that we combine all of
them to something like:
\lstparams language=Python,float,whatever=value
in InsetListings, include and preamble. In this way, all the
adjustments of parameters can be done without breaking lyx format.
Cheers,
Bo
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> "Bo" == Bo Peng <[EMAIL PROTECTED]> writes: >> > Seemingly, scons also fails to find libiconv: >> >> Bo, you might want to improve scons FreeBSD support. iconv is >> apparently properly detected by autotools... Bo> FreeBSD and macOS are left out because I can not test on them. Bo> Maybe I will install a FreeBSD virtual machine soon. For iconv, the autoconf test checks whether the second argument needs a const. JMarc
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
On Thu, 3 May 2007, Michael Gerz wrote: José Matos schrieb: On Thursday 03 May 2007 19:40:06 [EMAIL PROTECTED] wrote: > Good points. Does this mean you've approved the schedule above? > The calendar is optimistic, and I am not know for being pessimistic. ;-) You should, if possible, try to have some weekly list of the most critical bugs. Think in something like "the 10 most wanted ...". For doing this I will need help, I know that both you, Abdel and Michael had already examined the bug list. IMHO Uwe is the man to set up "the 10 most wanted bugs". He continually scans all bug reports and does a good for this. Moreover, he seems to read the user list as well. I agree with Michael, Uwe is the man for this. My contribution was really just helping Uwe on the wiki side. That doesn't mean we other can help prepare the list, especially with some guidance/pointers from Uwe. /Christian -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
On Thu, 3 May 2007, Bo Peng wrote:
we have to be very careful with InsetListings! If we add a half-baked
solution, people will be nagging all the time, i.e., we will be flooded
by countless bug reports ("Listing does not support this, Listing does
not support that").
I am using a list of string method to handle all listings options. It
will be good to have something like qt designer peroperty editor but
this listing method is much easier to implement. I still need some help
from Abdel on this though.
Will the users really be a problem if we say that the feature is
"experimental"?
Will it be possible to improve on it later without changing the file
format? (If so, then it can be improved in the 1.5-series - with user
feedback...)
/C
--
Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
Bo Peng wrote: What do you need? I am attaching what I have right now. Many parts are missing but the main points are there: Ouch... this is a big, non obvious, patch Bo... I am not sure we will find the time to properly review it and add the missing bits before 1.5.0 :-( What I have got: 1. add InsetListings in file InsetListings.h and InsetListings.cpp, this is a slightly modified copy of InsetERT. It already works in the sense that you can insert -> program code, enter code in the box and compile. What about Georg suggestion of using InsetInclude? 2. add InsetListingsParams, with all listings options roughly there. 3. add a dialog (ControlListings, QListings, ListingsUi) that allow user to alter listings options. This dialog will display all listings parameters (default values) and allow users to change their values. Abdel may have some better idea. I'll try to have a look this week-end. But do we really need this dialog now? Abdel.
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
> Seemingly, scons also fails to find libiconv: Bo, you might want to improve scons FreeBSD support. iconv is apparently properly detected by autotools... FreeBSD and macOS are left out because I can not test on them. Maybe I will install a FreeBSD virtual machine soon. Bo
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
I am attaching what I have right now. Many parts are missing but the
main points are there:
Sorry, last patch does not have InsetListingsParams.h/cpp.
Bo
Index: src/LyXAction.cpp
===
--- src/LyXAction.cpp (revision 18187)
+++ src/LyXAction.cpp (working copy)
@@ -371,6 +371,7 @@
{ LFUN_NOMENCL_PRINT, "nomencl-print", Noop },
{ LFUN_CLEARPAGE_INSERT, "clearpage-insert", Noop },
{ LFUN_CLEARDOUBLEPAGE_INSERT, "cleardoublepage-insert", Noop },
+ { LFUN_LISTING_INSERT, "listing-insert", Noop },
{ LFUN_NOACTION, "", Noop }
};
Index: src/insets/Inset.cpp
===
--- src/insets/Inset.cpp (revision 18187)
+++ src/insets/Inset.cpp (working copy)
@@ -76,6 +76,7 @@
InsetName("bibtex", Inset::BIBTEX_CODE),
InsetName("text", Inset::TEXT_CODE),
InsetName("ert", Inset::ERT_CODE),
+ InsetName("listings", Inset::ERT_CODE),
InsetName("foot", Inset::FOOT_CODE),
InsetName("margin", Inset::MARGIN_CODE),
InsetName("float", Inset::FLOAT_CODE),
Index: src/insets/InsetListings.h
===
--- src/insets/InsetListings.h (revision 0)
+++ src/insets/InsetListings.h (revision 0)
@@ -0,0 +1,116 @@
+// -*- C++ -*-
+/**
+ * \file InsetListings.h
+ * This file is part of LyX, the document processor.
+ * Licence details can be found in the file COPYING.
+ *
+ * \author Jürgen Vigna
+ * \author Lars Gullik Bjønnes
+ *
+ * Full author contact details are available in file CREDITS.
+ */
+
+#ifndef INSETLISTINGS_H
+#define INSETLISTINGS_H
+
+#include "LaTeXFeatures.h"
+#include "InsetCollapsable.h"
+#include "InsetListingsParams.h"
+#include "Length.h"
+#include "MailInset.h"
+
+
+namespace lyx {
+
+/** A collapsable text inset for program listings.
+*/
+
+
+class InsetListings : public InsetCollapsable {
+public:
+ ///
+ InsetListings(BufferParams const &, CollapseStatus status = Open);
+ ///
+ ~InsetListings();
+ ///
+ Inset::Code lyxCode() const { return Inset::ERT_CODE; }
+ ///
+ docstring name() const { return from_ascii("Listings"); }
+ ///
+ void write(Buffer const & buf, std::ostream & os) const;
+ ///
+ void read(Buffer const & buf, Lexer & lex);
+ ///
+ virtual docstring const editMessage() const;
+ ///
+ bool insetAllowed(Inset::Code code) const;
+ ///
+ int latex(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ int plaintext(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ int docbook(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ void validate(LaTeXFeatures &) const;
+ ///
+ bool metrics(MetricsInfo &, Dimension &) const;
+ ///
+ void draw(PainterInfo & pi, int x, int y) const;
+ ///
+ bool showInsetDialog(BufferView *) const;
+ ///
+ void getDrawFont(Font &) const;
+ ///
+ bool forceDefaultParagraphs(idx_type) const { return true; }
+ /// should paragraph indendation be ommitted in any case?
+ bool neverIndent(Buffer const &) const { return true; }
+ ///
+ InsetListingsParams const & params() const { return params_; }
+protected:
+ InsetListings(InsetListings const &);
+ ///
+ virtual void doDispatch(Cursor & cur, FuncRequest & cmd);
+ ///
+ bool getStatus(Cursor & cur, FuncRequest const & cmd, FuncStatus &) const;
+private:
+ virtual std::auto_ptr doClone() const;
+ ///
+ void init();
+ ///
+ void setButtonLabel();
+ ///
+ bool allowSpellCheck() const { return false; }
+ ///
+ InsetListingsParams params_;
+};
+
+
+class InsetListingsMailer : public MailInset {
+public:
+ ///
+ InsetListingsMailer(InsetListings & inset);
+ ///
+ virtual Inset & inset() const { return inset_; }
+ ///
+ virtual std::string const & name() const { return name_; }
+ ///
+ virtual std::string const inset2string(Buffer const &) const;
+ ///
+ static void string2params(std::string const &,
+ InsetListingsParams &);
+ ///
+ static std::string const params2string(InsetListingsParams const &);
+private:
+ ///
+ static std::string const name_;
+ ///
+ InsetListings & inset_;
+};
+
+
+} // namespace lyx
+
+#endif
Index: src/insets/InsetListingsParams.h
===
--- src/insets/InsetListingsParams.h (revision 0)
+++ src/insets/InsetListingsParams.h (revision 0)
@@ -0,0 +1,203 @@
+// -*- C++ -*-
+/**
+ * \file InsetListings.h
+ * This file is part of LyX, the document processor.
+ * Licence details can be found in the file COPYING.
+ *
+ * \author Jürgen Vigna
+ * \author Lars Gullik Bjønnes
+ *
+ * Full author contact details are available in file CREDITS.
+ */
+
+#ifndef INSETLISTINGSPARAMS_H
+#define INSETLISTINGSPARAMS_H
+
+#include
+#include "Lexer.h"
+
+namespace lyx {
+
+struct listings_param {
+ std::string name;
+ std::string value;
+ // for option with value "true", "false",
+ // if onoff is true,
+ // "true": option
+ // "false":
+ // false:
+ // "true": option=true
+ // "false": o
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
What do you need?
I am attaching what I have right now. Many parts are missing but the
main points are there:
What I have got:
1. add InsetListings in file InsetListings.h and InsetListings.cpp,
this is a slightly modified copy of InsetERT. It already works in the
sense that you can insert -> program code, enter code in the box and
compile.
2. add InsetListingsParams, with all listings options roughly there.
3. add a dialog (ControlListings, QListings, ListingsUi) that allow
user to alter listings options. This dialog will display all listings
parameters (default values) and allow users to change their values.
Abdel may have some better idea.
What are missing:
1. lyx2lyx support for InsetListings.
2. The Qt part is largely missing. I guess Abdel can get the idea of
the dialog and add the missing parts rather quickly.
3. If this dialog can be put or copied to insert->include (lstinclude
?) and file->settings (global settings) without much difficulty, the
listings feature will be complete. The list-of-string option is less
userfriendly than Herbert's large dialog, but it is a good way to
provide *all* options to users.
4. autotools and cmake support are missing (should be easy to add files).
Cheers,
Bo
Index: src/LyXAction.cpp
===
--- src/LyXAction.cpp (revision 18187)
+++ src/LyXAction.cpp (working copy)
@@ -371,6 +371,7 @@
{ LFUN_NOMENCL_PRINT, "nomencl-print", Noop },
{ LFUN_CLEARPAGE_INSERT, "clearpage-insert", Noop },
{ LFUN_CLEARDOUBLEPAGE_INSERT, "cleardoublepage-insert", Noop },
+ { LFUN_LISTING_INSERT, "listing-insert", Noop },
{ LFUN_NOACTION, "", Noop }
};
Index: src/insets/Inset.cpp
===
--- src/insets/Inset.cpp (revision 18187)
+++ src/insets/Inset.cpp (working copy)
@@ -76,6 +76,7 @@
InsetName("bibtex", Inset::BIBTEX_CODE),
InsetName("text", Inset::TEXT_CODE),
InsetName("ert", Inset::ERT_CODE),
+ InsetName("listings", Inset::ERT_CODE),
InsetName("foot", Inset::FOOT_CODE),
InsetName("margin", Inset::MARGIN_CODE),
InsetName("float", Inset::FLOAT_CODE),
Index: src/insets/InsetListings.h
===
--- src/insets/InsetListings.h (revision 0)
+++ src/insets/InsetListings.h (revision 0)
@@ -0,0 +1,116 @@
+// -*- C++ -*-
+/**
+ * \file InsetListings.h
+ * This file is part of LyX, the document processor.
+ * Licence details can be found in the file COPYING.
+ *
+ * \author Jürgen Vigna
+ * \author Lars Gullik Bjønnes
+ *
+ * Full author contact details are available in file CREDITS.
+ */
+
+#ifndef INSETLISTINGS_H
+#define INSETLISTINGS_H
+
+#include "LaTeXFeatures.h"
+#include "InsetCollapsable.h"
+#include "InsetListingsParams.h"
+#include "Length.h"
+#include "MailInset.h"
+
+
+namespace lyx {
+
+/** A collapsable text inset for program listings.
+*/
+
+
+class InsetListings : public InsetCollapsable {
+public:
+ ///
+ InsetListings(BufferParams const &, CollapseStatus status = Open);
+ ///
+ ~InsetListings();
+ ///
+ Inset::Code lyxCode() const { return Inset::ERT_CODE; }
+ ///
+ docstring name() const { return from_ascii("Listings"); }
+ ///
+ void write(Buffer const & buf, std::ostream & os) const;
+ ///
+ void read(Buffer const & buf, Lexer & lex);
+ ///
+ virtual docstring const editMessage() const;
+ ///
+ bool insetAllowed(Inset::Code code) const;
+ ///
+ int latex(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ int plaintext(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ int docbook(Buffer const &, odocstream &,
+ OutputParams const &) const;
+ ///
+ void validate(LaTeXFeatures &) const;
+ ///
+ bool metrics(MetricsInfo &, Dimension &) const;
+ ///
+ void draw(PainterInfo & pi, int x, int y) const;
+ ///
+ bool showInsetDialog(BufferView *) const;
+ ///
+ void getDrawFont(Font &) const;
+ ///
+ bool forceDefaultParagraphs(idx_type) const { return true; }
+ /// should paragraph indendation be ommitted in any case?
+ bool neverIndent(Buffer const &) const { return true; }
+ ///
+ InsetListingsParams const & params() const { return params_; }
+protected:
+ InsetListings(InsetListings const &);
+ ///
+ virtual void doDispatch(Cursor & cur, FuncRequest & cmd);
+ ///
+ bool getStatus(Cursor & cur, FuncRequest const & cmd, FuncStatus &) const;
+private:
+ virtual std::auto_ptr doClone() const;
+ ///
+ void init();
+ ///
+ void setButtonLabel();
+ ///
+ bool allowSpellCheck() const { return false; }
+ ///
+ InsetListingsParams params_;
+};
+
+
+class InsetListingsMailer : public MailInset {
+public:
+ ///
+ InsetListingsMailer(InsetListings & inset);
+ ///
+ virtual Inset & inset() const { return inset_; }
+ ///
+ virtual std::string const & name() const { return name_; }
+ ///
+ virtual std::string const inset2string(Buffer const &) const;
+ ///
+ static void string2params(std::string const &,
+ Inset
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
On Thursday 03 May 2007 20:38:00 Michael Gerz wrote: > Point taken. But all hobby-horses have to be stopped by beta 3, even if > it hurts (no matter how important or smart they are). IMHO things like > icons rescaling falls into this category, too. I agree and I count with every developer to implement this policy. > If May 11 is too early, we spend add another week (i.e. May 18) but not > more than that. I agree. To have a release date is very important, but there are other assets. > But, hey, I am not the project leader. José, speak to us. I am sure > nobody will object! I am not the project leader, I am the release manager. :-) I am here to release our potential synergies. ;-) > Michael -- José Abílio
Re: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE
Abdelrazak Younes schrieb: Since it would be arrogant to demand from all of you to adjust to my wishes for the very little amount I could contribute I chose the other alternative: Don't help with a product that I cannot endorse. Georg, I looked up the meaning of "endorse" and I must confess that I am sorry that you cannot endorse a product on which you have spent sooo much spare time. Apart from the Armenian support and the recent InsetListings discussion, what exactly is missing to make LyX 1.5.0 a great product - better than any other release we have published before? And a software that eases _your_ life? Could you point us to some bug reports that you consider critical for the success of LyX 1.5.0? I guess, the whole LyX community is eager to make you happy! As you know, I promised to give some gifts to the most busy bees of the LyX 1.5.0 development cycle. No matter happens, a prizes will go to you and Abdel (third prize is still vacant). Best wishes, Michael
Re: Towards 1.5.0? (Was: UTF8->UCS4 failure on FreeBSD 6.2-RELEASE)
On Thursday 03 May 2007 15:49:06 Bo Peng wrote: > > You know of course that you have the power to implement everything you > > want locally don't you? :-) > > But another guy in my group will collaborate with me on this document > and it would be easier for him if it is in lyx.1.5.0. Since this issue involves a file format change any local change would be quite difficult to implement locally. > > Will this feature involves a format change? If yes, how much time do you > > need in order to implement the very minimal change to support it? > > I have done more than minimal for this feature. I can polish and > submit a patch for review later today. I need some help on lyx2lyx > though. What do you need? > Cheers, > Bo -- José Abílio
