[
https://issues.apache.org/jira/browse/XERCESC-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Boris Kolpackov updated XERCESC-1846:
-------------------------------------
Component/s: (was: Non-Validating Parser)
> We should not let the ICU use a replacement character that we know will
> result in a document that's not well-formed.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: XERCESC-1846
> URL: https://issues.apache.org/jira/browse/XERCESC-1846
> Project: Xerces-C++
> Issue Type: Bug
> Components: Utilities
> Affects Versions: 2.8.0, 3.0.0
> Environment: ICU 4.0, 3.8
> Xerces 3.0, 2.8. Probably will be common problem for earlier versions too.
> HP-UX B.11.23 U 9000/800
> aCC compier, cc
> Reporter: Jan Suchy
> Assignee: David Bertoni
>
> Jan Suchý wrote:
> > Hi Jesse,
> > thank you for your answer and ideas.
> > I have found one kind of solution to patch the transcoder wrap class:
> > src\xercesc\util\Transcoders\ICU\ICUTransService.cpp
> >
> > adding there to constructor of ICUTranscoder::ICUTranscoder these lines:
> >
> > UErrorCode uerr = U_ZERO_ERROR;
> > ucnv_setSubstChars(toAdopt, "?", 1, &uerr);
> > ...
> >
> > Than, the "?" character is used as replacement char, when using icu.
> > This is ICU specific solutions and is not clear, because there is necessary
> > to rebuild xerces lib. I would like to see some possible switch around
> > XMLFormatter class, but there is unknown UConverter form ICU which will be
> > used next, because there is nothing to know which transcoder will be called
> > later.
> Please create a Jira issue because this is a bug. We should not let the
> ICU use a replacement character that we know will result in a document
> that's not well-formed.
> Dave
> HISTORY:
> ------------------------------------------------------------------------------------------------------------------------
> Hi Jesse,
> thank you for your answer and ideas.
> I have found one kind of solution to patch the transcoder wrap class:
> src\xercesc\util\Transcoders\ICU\ICUTransService.cpp
> adding there to constructor of ICUTranscoder::ICUTranscoder these lines:
> UErrorCode uerr = U_ZERO_ERROR;
> ucnv_setSubstChars(toAdopt, "?", 1, &uerr);
> ...
> Than, the "?" character is used as replacement char, when using icu.
> This is ICU specific solutions and is not clear, because there is necessary
> to rebuild xerces lib. I would like to see some possible switch around
> XMLFormatter class, but there is unknown UConverter form ICU which will be
> used next, because there is nothing to know which transcoder will be called
> later.
> Not optimal, but works.
> thank you,
> jan
> > ------------ Původní zpráva ------------
> > Od: Jesse Pelton <j...@pkc.com>
> > Předmět: RE: xerces/ICU unicode alias for weak encoding when
> > serializing/converting to CP
> > Datum: 16.12.2008 15:31:01
> > ----------------------------------------
> > I'm not an expert on this area, but the transcoders included with Xerces do
> > not
> > provide any way to specify the replacement character, and ICU may be the
> > same.
> > Even if ICU gives you a way to do so, I'm not sure how you'd get access to a
> > transcoder instance to alter.
> >
> > Note, though, that 0x1A is not a legal character in an XML document.
> > (Oracle's
> > parser is correct in rejecting it.) I think it's safe to assume in your
> > scenario
> > that any such characters in a serialized document are replacements for
> > unrepresentable characters. You should therefore be able to post-process the
> > serialization output and replace 0x1A with one or more characters of your
> > choosing. If you don't want to post-process the whole document, you could
> > derive
> > an XMLFormatTarget that replaces the replacement character in each chunk of
> > data
> > handed to it. Neither option is exactly elegant, but I'd probably do the
> > latter;
> > it'll work regardless of your format target, where the former approach
> > requires
> > serializing to memory.
> >
> >
> > -----Original Message-----
> > From: Jan Suchý [mailto:zu...@post.cz]
> > Sent: Tuesday, December 16, 2008 5:37 AM
> > To: c-us...@xerces.apache.org
> > Subject: RE: xerces/ICU unicode alias for weak encoding when
> > serializing/converting to CP
> >
> > Hello again,
> > i have tried to use class:
> >
> > http://xerces.apache.org/xerces-c/apiDocs-2/classXMLFormatter.html#_details
> >
> > with attributes: NoEscapes , UnRep_Replace
> >
> > and the problematic char was replaced by:
> > ^Z
> >
> > But it is still not solving problem with Oracle DB XML parser to parse this
> > xml.
> > I have got this error:
> >
> > ORA-31011: XML parsing failed
> > ORA-19202: Error occurred in XML processing
> > LPX-00216: invalid character 26 (0x1A)
> > Error at line 22
> >
> > I would like to replace unknown character with my own character, which will
> > be
> > parseable (for example char "?" or "_").
> > How can I change replacement character, which is used as default?
> >
> > Thank anybody for any idea.
> >
> > Have a nice day,
> > Jan
> >
> >
> > > ------------ Původní zpráva ------------
> > > Od: Jan Suchý <zu...@post.cz>
> > > Předmět: RE: xerces/ICU unicode alias for weak encoding when
> > > serializing/converting to CP
> > > Datum: 16.12.2008 09:35:40
> > > ----------------------------------------
> > > Hello Jesse,
> > > thank you for your answer :-) it seems to be promising. I'll look at it.
> > > Jan
> > >
> > >
> > > > ------------ Původní zpráva ------------
> > > > Od: Jesse Pelton <j...@pkc.com>
> > > > Předmět: RE: xerces/ICU unicode alias for weak encoding when
> > > > serializing/converting to CP
> > > > Datum: 15.12.2008 18:15:49
> > > > ----------------------------------------
> > > > The constructors for the Xerces XMLFormatter object all take an
> > > > UnRepFlags
> > > > argument that allows you to specify how to handle unrepresentable
> > characters.
> > >
> > > > So does XMLFormatter::formatBuf(). It appears that the transcoder gets
> > > > to
> > > > decide what character to replace unrepresentable characters with.
> > > >
> > > > Hope that helps.
> > > >
> > > > -----Original Message-----
> > > > From: Jan Suchý [mailto:zu...@post.cz]
> > > > Sent: Monday, December 15, 2008 4:25 AM
> > > > To: c-us...@xerces.apache.org
> > > > Subject: xerces/ICU unicode alias for weak encoding when
> > > serializing/converting
> > > > to CP
> > > >
> > > > Hello all,
> > > > I need to obtain output XML in iso-8859-2 encoding.
> > > > I am using UTF-8 as input encoding.
> > > > There is some character, in UTF-8 xml, which is not representable in
> > > > iso-8859-2.
> > > > I am using ICU 3.8, xerces 2.8 and Xqilla svn 702.
> > > >
> > > > After serializing XML to iso-8859-2 the problematic character is
> > > > serialized
> > > by
> > > > ICU/xerces/xq to:
> > > >
> > > > –
> > > >
> > > > The problem is, that if I will send message in iso-8859-2 with character
> > > > – inside to Oracle DB, the Oracle parser
> > > >
> > > > does not like this character and this error is obtained:
> > > >
> > > > ORA-31011: XML parsing failed, LPX-00217: invalid character 8211
> > > > (U+2013)
> > > >
> > > > So, what I am looking for is some method, how to say to the ICU or to
> > Xerces
> > > or
> > > > to XQ, that the Unicode character, must
> > > >
> > > > not be included in result and must be for example replaced by character
> > "?",
> > > to
> > > > avoid Oracle parser to process it.
> > > >
> > > > I would like to find clear solution, like saying to ICU not calling
> > callback
> > > > function or define own alias or behavior on
> > > >
> > > > this situation. Is it possible?
> > > > Any ideas?
> > > > Thank you
> > > > Jan Suchy
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org
