Re: iconv error with listing + Spanish

[Scott Kostyshak]

On Wed, Mar 01, 2017 at 05:35:45PM +, Guenter Milde wrote:

> This confirms it as an instance of #9871 (or related to it).

Good, I added the information to the ticket.

Scott


signature.asc
Description: PGP signature

Re: iconv error with listing + Spanish

[Guenter Milde]

On 2017-03-01, Kornel Benko wrote:
> Am Mittwoch, 1. März 2017 um 11:35:45, schrieb Guenter Milde 
> 
>> On 2017-03-01, Scott Kostyshak wrote:

>> >> > Note that this affects lib/examples/es/linguistics.lyx
>> >> > 
>> >> > Can anyone else reproduce? Is the problem obvious?
>> >> > 
>> >> > I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
>> >> > 2.1.0. I can do a git bisect if someone thinks the time would be worth
>> >> > it.

>> >> Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
>> >> I inverted the affected test.

>> Could you try whether export works with pdftex if the input encoding is set
>> to ASCII?

> Does not work here. Even lualatex cannot compile if encoding is set to ascii.

This confirms it as an instance of #9871 (or related to it).

> Setting to utf8 works with pdflatex and lualatex but not with xelatex.

Yes, because xelatex ignores the inputenc setting and forces ascii.

Thanks,

Günter

Re: iconv error with listing + Spanish

[Scott Kostyshak]

On Wed, Mar 01, 2017 at 11:35:45AM +, Guenter Milde wrote:

> 36b9645b is part of the fix to #9740 (XeTeX with TeX fonts).
> (BTW: how do you get the content/description of a change-set given this
> strange number only?)

git show 36b9645b

Does that do what you want? The number is referred to as the "short
hash" of the git commit.

Scott


signature.asc
Description: PGP signature

Re: iconv error with listing + Spanish

[Scott Kostyshak]

On Wed, Mar 01, 2017 at 11:41:50AM +, Guenter Milde wrote:
> On 2017-02-28, Scott Kostyshak wrote:
> > On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:
> 
> >> Note that this affects lib/examples/es/linguistics.lyx
> 
> >> Can anyone else reproduce? Is the problem obvious?
> 
> >> I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
> >> 2.1.0. I can do a git bisect if someone thinks the time would be worth
> >> it.
> 
> > Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
> > I inverted the affected test.
> 
> > But that issue is not labeled as a regression. Why does the MWE work
> > with 2.1.0?
> 
> This is not a regression, as it always was a problem if the input encoding
> is ASCII. 
> 
> The "regression" is due to the fact, that XeTeX with TeX fonts requires
> ASCII input encoding and LyX enforces it since 2.2.
> Try the file with 2.1 and inputenc manually set to ASCII.

Thanks for this explanation,

Scott


signature.asc
Description: PGP signature

Re: iconv error with listing + Spanish

[Kornel Benko]

Am Mittwoch, 1. März 2017 um 11:35:45, schrieb Guenter Milde 

> On 2017-03-01, Scott Kostyshak wrote:
> > On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote:
> >> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:
> 
> >> > Note that this affects lib/examples/es/linguistics.lyx
> >> > 
> >> > Can anyone else reproduce? Is the problem obvious?
> >> > 
> >> > I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
> >> > 2.1.0. I can do a git bisect if someone thinks the time would be worth
> >> > it.
> 
> >> Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
> >> I inverted the affected test.
> 
> >> But that issue is not labeled as a regression. Why does the MWE work
> >> with 2.1.0?
> 
> > The commit that changed the behavior was 36b9645b.
> 
> > Günter, can you confirm that the export just worked by chance (perhaps
> > even bad output?) before, so that 36b9645b did not really break the
> > export? I have a vague memory that this is the case, but I want to
> > double-check.
> 
> 36b9645b is part of the fix to #9740 (XeTeX with TeX fonts).
> (BTW: how do you get the content/description of a change-set given this
> strange number only?)

# git show 36b9645b

> Possible regressions due to the restrictions of XeTeX with TeX-fonts are
> expected (and should be eventually handled) but don't outweigh the
> benefits of the change. No reason to restore 2.1 behaviour.
> 
> See also the similar discussion in https://www.lyx.org/trac/ticket/10199
> 
> Could you try whether export works with pdftex if the input encoding is set
> to ASCII?

Does not work here. Even lualatex cannot compile if encoding is set to ascii.
Setting to utf8 works with pdflatex and lualatex but not with xelatex.

> Günter

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: iconv error with listing + Spanish

[Guenter Milde]

On 2017-02-28, Scott Kostyshak wrote:
> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:

>> Note that this affects lib/examples/es/linguistics.lyx

>> Can anyone else reproduce? Is the problem obvious?

>> I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
>> 2.1.0. I can do a git bisect if someone thinks the time would be worth
>> it.

> Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
> I inverted the affected test.

> But that issue is not labeled as a regression. Why does the MWE work
> with 2.1.0?

This is not a regression, as it always was a problem if the input encoding
is ASCII. 

The "regression" is due to the fact, that XeTeX with TeX fonts requires
ASCII input encoding and LyX enforces it since 2.2.
Try the file with 2.1 and inputenc manually set to ASCII.

Günter

Re: iconv error with listing + Spanish

[Guenter Milde]

On 2017-03-01, Scott Kostyshak wrote:
> On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote:
>> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:

>> > Note that this affects lib/examples/es/linguistics.lyx
>> > 
>> > Can anyone else reproduce? Is the problem obvious?
>> > 
>> > I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
>> > 2.1.0. I can do a git bisect if someone thinks the time would be worth
>> > it.

>> Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
>> I inverted the affected test.

>> But that issue is not labeled as a regression. Why does the MWE work
>> with 2.1.0?

> The commit that changed the behavior was 36b9645b.

> Günter, can you confirm that the export just worked by chance (perhaps
> even bad output?) before, so that 36b9645b did not really break the
> export? I have a vague memory that this is the case, but I want to
> double-check.

36b9645b is part of the fix to #9740 (XeTeX with TeX fonts).
(BTW: how do you get the content/description of a change-set given this
strange number only?)

Possible regressions due to the restrictions of XeTeX with TeX-fonts are
expected (and should be eventually handled) but don't outweigh the
benefits of the change. No reason to restore 2.1 behaviour.

See also the similar discussion in https://www.lyx.org/trac/ticket/10199

Could you try whether export works with pdftex if the input encoding is set
to ASCII?

Günter

Re: iconv error with listing + Spanish

[Scott Kostyshak]

On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote:
> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:
> 
> > Note that this affects lib/examples/es/linguistics.lyx
> > 
> > Can anyone else reproduce? Is the problem obvious?
> > 
> > I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
> > 2.1.0. I can do a git bisect if someone thinks the time would be worth
> > it.
> 
> Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
> I inverted the affected test.
> 
> But that issue is not labeled as a regression. Why does the MWE work
> with 2.1.0?

The commit that changed the behavior was 36b9645b.

Günter, can you confirm that the export just worked by chance (perhaps
even bad output?) before, so that 36b9645b did not really break the
export? I have a vague memory that this is the case, but I want to
double-check.

Scott


signature.asc
Description: PGP signature

Re: iconv error with listing + Spanish

[Scott Kostyshak]

On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote:

> Note that this affects lib/examples/es/linguistics.lyx
> 
> Can anyone else reproduce? Is the problem obvious?
> 
> I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
> 2.1.0. I can do a git bisect if someone thinks the time would be worth
> it.

Perhaps this is http://www.lyx.org/trac/ticket/9871 ?
I inverted the affected test.

But that issue is not labeled as a regression. Why does the MWE work
with 2.1.0?

Scott


signature.asc
Description: PGP signature

iconv error with listing + Spanish

[Scott Kostyshak]

To reproduce, either use the attached MWE, or do the following:

1. create a new program listing.
2. enter "abc"
3. In Document > Settings > Language change to "Spanish".

Export to XeLaTeX (either the TeX or the PDF).

I get messages like the following in the terminal:

Error 84 returned from iconv when converting from UCS-4LE to ascii: Invalid or 
incomplete multibyte or wide character
Converted input: 0x005c 0x0062 0x0061 0x0074 0x0063 0x0068 0x006d 0x006f 0x0064 
0x0065 0x000a 0x005c 0x006d 0x0061 0x006b 0x0065 0x0061 0x0074 0x006c 0x0065 
0x0074 0x0074 0x0065 0x0072 0x000a 0x005c 0x0064 0x0065 0x0066 0x005c 0x0069 
0x006e 0x0070 0x0075 0x0074 0x0040 0x0070 0x0061 0x0074 0x0068 0x007b 0x007b 
0x002f 0x0068 0x006f 0x006d 0x0065 0x002f 0x0073 0x0063 0x006f 0x0074 0x0074 
0x002f 0x0044 0x0065 0x0073 0x006b 0x0074 0x006f 0x0070 0x002f 0x006d 0x0077 
0x0065 0x002f 0x007d 0x007d 0x000a 0x005c 0x006d 0x0061 0x006b 0x0065 0x0061 
0x0074 0x006f 0x0074 0x0068 0x0065 0x0072 0x000a 0x005c 0x0064 0x006f 0x0063 
0x0075 0x006d 0x0065 0x006e 0x0074 0x0063 0x006c 0x0061 0x0073 0x0073 0x005b 
0x0073 0x0070 0x0061 0x006e 0x0069 0x0073 0x0068 0x005d 0x007b 0x0061 0x0072 
0x0074 0x0069 0x0063 0x006c 0x0065 0x007d 0x000a 0x005c 0x0075 0x0073 0x0065 
0x0070 0x0061 0x0063 0x006b 0x0061 0x0067 0x0065 0x005b 0x0054 0x0031 0x005d 
0x007b 0x0066 0x006f 0x006e 0x0074 0x0065 0x006e 0x0063 0x007d 0x000a 0x005c 
0x0075 0x0073 0x0065 0x0070 0x0061 0x0063 0x006b 0x0061 0x0067 0x0065 0x007b 
0x0062 0x0061 0x0062 0x0065 0x006c 0x007d 0x000a 0x005c 0x0061 0x0064 0x0064 
0x0074 0x006f 0x005c 0x0073 0x0068 0x006f 0x0072 0x0074 0x0068 0x0061 0x006e 
0x0064 0x0073 0x0073 0x0070 0x0061 0x006e 0x0069 0x0073 0x0068 0x007b 0x005c 
0x0073 0x0070 0x0061 0x006e 0x0069 0x0073 0x0068 0x0064 0x0065 0x0061 0x0063 
0x0074 0x0069 0x0076 0x0061 0x0074 0x0065 0x007b 0x007e 0x003c 0x003e 0x007d 
0x007d 0x000a 0x000a 0x005c 0x0075 0x0073 0x0065 0x0070 0x0061 0x0063 0x006b 
0x0061 0x0067 0x0065 0x007b 0x006c 0x0069 0x0073 0x0074 0x0069 0x006e 0x0067 
0x0073 0x007d 0x000a 0x005c 0x0072 0x0065 0x006e 0x0065 0x0077 0x0063 0x006f 
0x006d 0x006d 0x0061 0x006e 0x0064 0x007b 0x005c 0x006c 0x0073 0x0074 0x006c 
0x0069 0x0073 0x0074 0x0069 0x006e 0x0067 0x006e 0x0061 0x006d 0x0065 0x007d 
0x007b 0x004c 0x0069 0x0073 0x0074 0x0061 0x0064 0x006f 0x0020 0x0064 0x0065 
0x0020 0x0063
Stopped at: 0x00f3
Unconverted input: 0x0064 0x0069 0x0067 0x006f 0x007d 0x000a 0x000a 0x005c 
0x0062 0x0065 0x0067 0x0069 0x006e 0x007b 0x0064 0x006f 0x0063 0x0075 0x006d 
0x0065 0x006e 0x0074 0x007d 0x000a

Note that this affects lib/examples/es/linguistics.lyx

Can anyone else reproduce? Is the problem obvious?

I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on
2.1.0. I can do a git bisect if someone thinks the time would be worth
it.

Scott
#LyX 2.3 created this file. For more info see http://www.lyx.org/
\lyxformat 474
\begin_document
\begin_header
\textclass article
\use_default_options true
\maintain_unincluded_children false
\language spanish
\language_package default
\inputencoding auto
\fontencoding global
\font_roman default
\font_sans default
\font_typewriter default
\font_math auto
\font_default_family default
\use_non_tex_fonts false
\font_sc false
\font_osf false
\font_sf_scale 100
\font_tt_scale 100
\graphics default
\default_output_format default
\output_sync 1
\bibtex_command default
\index_command default
\paperfontsize default
\spacing single
\use_hyperref false
\papersize default
\use_geometry false
\use_package amsmath 1
\use_package amssymb 1
\use_package cancel 1
\use_package esint 1
\use_package mathdots 1
\use_package mathtools 1
\use_package mhchem 1
\use_package stackrel 1
\use_package stmaryrd 1
\use_package undertilde 1
\cite_engine basic
\cite_engine_type default
\biblio_style plain
\use_bibtopic false
\use_indices false
\paperorientation portrait
\suppress_date false
\justification true
\use_refstyle 1
\index Index
\shortcut idx
\color #008000
\end_index
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\paragraph_indentation default
\quotes_language french
\papercolumns 1
\papersides 1
\paperpagestyle default
\tracking_changes false
\output_changes false
\html_math_output 0
\html_css_as_file 0
\html_be_strict false
\end_header

\begin_body

\begin_layout Standard
\begin_inset listings
inline false
status open

\begin_layout Plain Layout

abc
\end_layout

\end_inset


\end_layout

\end_body
\end_document


signature.asc
Description: PGP signature