Re: [HACKERS] possible encoding issues with libxml2 functions

2017-11-11 Thread Pavel Stehule
2017-11-11 21:19 GMT+01:00 Noah Misch : > On Sun, Nov 05, 2017 at 06:10:04PM +0100, Pavel Stehule wrote: > > Hi > > > > 2017-11-05 4:07 GMT+01:00 Noah Misch : > > > > > On Tue, Oct 17, 2017 at 06:06:40AM +0200, Pavel Stehule wrote: > > > > Please, if you can, try it write. I am little bit lost :)

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-11-11 Thread Noah Misch
On Sun, Nov 05, 2017 at 06:10:04PM +0100, Pavel Stehule wrote: > Hi > > 2017-11-05 4:07 GMT+01:00 Noah Misch : > > > On Tue, Oct 17, 2017 at 06:06:40AM +0200, Pavel Stehule wrote: > > > Please, if you can, try it write. I am little bit lost :) > > > > I'm attaching the patch I desired. Please re

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-11-05 Thread Pavel Stehule
Hi 2017-11-05 4:07 GMT+01:00 Noah Misch : > On Tue, Oct 17, 2017 at 06:06:40AM +0200, Pavel Stehule wrote: > > Please, if you can, try it write. I am little bit lost :) > > I'm attaching the patch I desired. Please review. This will probably miss > this week's minor releases. If there's signif

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-11-04 Thread Noah Misch
On Tue, Oct 17, 2017 at 06:06:40AM +0200, Pavel Stehule wrote: > Please, if you can, try it write. I am little bit lost :) I'm attaching the patch I desired. Please review. This will probably miss this week's minor releases. If there's significant support, I could instead push before the wrap.

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-10-16 Thread Pavel Stehule
2017-10-17 1:57 GMT+02:00 Noah Misch : > On Sun, Aug 20, 2017 at 10:37:10PM +0200, Pavel Stehule wrote: > > > We have xpath-bugfix.patch and xpath-parsing-error-fix.patch. Both are > > > equivalent under supported use cases (xpath in UTF8 databases). Among > > > non-supported use cases, they eac

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-10-16 Thread Noah Misch
On Sun, Aug 20, 2017 at 10:37:10PM +0200, Pavel Stehule wrote: > > We have xpath-bugfix.patch and xpath-parsing-error-fix.patch. Both are > > equivalent under supported use cases (xpath in UTF8 databases). Among > > non-supported use cases, they each make different things better and > > different

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-10-15 Thread Pavel Stehule
2017-08-21 6:25 GMT+02:00 Pavel Stehule : > > >> xpath-bugfix.patch affected only xml values containing an xml declaration >> with >> "encoding" attribute. In UTF8 databases, this latest proposal >> (xpath-parsing-error-fix.patch) is equivalent to xpath-bugfix.patch. In >> non-UTF8 databases, xp

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-20 Thread Pavel Stehule
> > xpath-bugfix.patch affected only xml values containing an xml declaration > with > "encoding" attribute. In UTF8 databases, this latest proposal > (xpath-parsing-error-fix.patch) is equivalent to xpath-bugfix.patch. In > non-UTF8 databases, xpath-parsing-error-fix.patch affects all xml values

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-20 Thread Pavel Stehule
Hi > xpath-bugfix.patch affected only xml values containing an xml declaration > with > "encoding" attribute. In UTF8 databases, this latest proposal > (xpath-parsing-error-fix.patch) is equivalent to xpath-bugfix.patch. In > non-UTF8 databases, xpath-parsing-error-fix.patch affects all xml val

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-20 Thread Noah Misch
On Sun, Aug 20, 2017 at 10:54:57AM +0200, Pavel Stehule wrote: > 2017-08-20 9:21 GMT+02:00 Noah Misch : > > On Mon, Aug 07, 2017 at 05:10:14PM -0700, Noah Misch wrote: > > > On Wed, Apr 05, 2017 at 10:53:39PM +0200, Pavel Stehule wrote: > > > > One possible fix - and similar technique is used more

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-20 Thread Pavel Stehule
2017-08-20 9:21 GMT+02:00 Noah Misch : > On Sun, Aug 20, 2017 at 08:46:03AM +0200, Pavel Stehule wrote: > > 2017-08-20 4:17 GMT+02:00 Noah Misch : > > > On Sat, Aug 19, 2017 at 10:53:19PM +0200, Pavel Stehule wrote: > > > > I am sending some POC - it does support XPATH and XMLTABLE for not > UTF8

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-20 Thread Noah Misch
On Sun, Aug 20, 2017 at 08:46:03AM +0200, Pavel Stehule wrote: > 2017-08-20 4:17 GMT+02:00 Noah Misch : > > On Sat, Aug 19, 2017 at 10:53:19PM +0200, Pavel Stehule wrote: > > > I am sending some POC - it does support XPATH and XMLTABLE for not UTF8 > > > server encoding. > > > > > > In this case,

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-19 Thread Pavel Stehule
2017-08-20 4:17 GMT+02:00 Noah Misch : > On Fri, Aug 18, 2017 at 11:43:19PM +0200, Pavel Stehule wrote: > > yes, probably libXML2 try to do check from utf8 encoding to header > > specified encoding. > > Yes. That has been the topic of this thread. > > > a) all values created by xml_in iterface a

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-19 Thread Noah Misch
On Fri, Aug 18, 2017 at 11:43:19PM +0200, Pavel Stehule wrote: > yes, probably libXML2 try to do check from utf8 encoding to header > specified encoding. Yes. That has been the topic of this thread. > a) all values created by xml_in iterface are in database encoding - input > string is stored w

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-19 Thread Pavel Stehule
Hi 2017-08-19 22:53 GMT+02:00 Pavel Stehule : > Hi > > I am sending some POC - it does support XPATH and XMLTABLE for not UTF8 > server encoding. > > In this case, all strings should be converted to UTF8 before call libXML2 > functions, and result should be converted back from UTF8. > > I found

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-19 Thread Pavel Stehule
Hi I am sending some POC - it does support XPATH and XMLTABLE for not UTF8 server encoding. In this case, all strings should be converted to UTF8 before call libXML2 functions, and result should be converted back from UTF8. I found some previous experiments https://marc.info/?l=pgsql-bug s&m=12

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-19 Thread Pavel Stehule
> > Isn't the most correct solution to call xml_parse function? > I am reply to self. Probably not. Now, I am thinking so I found a reason of this issue. The document processed in xpath_internal is passed to libXML2 by doc = xmlCtxtReadMemory(ctxt, (char *) string, len, NULL, NULL, 0); We d

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-18 Thread Pavel Stehule
Hi 2017-08-08 2:10 GMT+02:00 Noah Misch : > On Wed, Apr 05, 2017 at 10:53:39PM +0200, Pavel Stehule wrote: > > 2017-03-17 4:23 GMT+01:00 Noah Misch : > > > On Sun, Mar 12, 2017 at 10:26:33PM +0100, Pavel Stehule wrote: > > > > 2017-03-12 21:57 GMT+01:00 Noah Misch : > > > > > On Sun, Mar 12, 2017

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-08-07 Thread Noah Misch
On Wed, Apr 05, 2017 at 10:53:39PM +0200, Pavel Stehule wrote: > 2017-03-17 4:23 GMT+01:00 Noah Misch : > > On Sun, Mar 12, 2017 at 10:26:33PM +0100, Pavel Stehule wrote: > > > 2017-03-12 21:57 GMT+01:00 Noah Misch : > > > > On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > > > > > 2

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-04-05 Thread Pavel Stehule
2017-03-17 4:23 GMT+01:00 Noah Misch : > On Sun, Mar 12, 2017 at 10:26:33PM +0100, Pavel Stehule wrote: > > 2017-03-12 21:57 GMT+01:00 Noah Misch : > > > On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > > > > 2017-03-12 0:56 GMT+01:00 Noah Misch : > > > Please add a test case. > I

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-16 Thread Noah Misch
On Sun, Mar 12, 2017 at 10:26:33PM +0100, Pavel Stehule wrote: > 2017-03-12 21:57 GMT+01:00 Noah Misch : > > On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > > > 2017-03-12 0:56 GMT+01:00 Noah Misch : > > Please add a test case. > > It needs a application - currently there is not p

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-12 Thread Pavel Stehule
2017-03-12 22:26 GMT+01:00 Pavel Stehule : > > > 2017-03-12 21:57 GMT+01:00 Noah Misch : > >> On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: >> > 2017-03-12 0:56 GMT+01:00 Noah Misch : >> > > On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: >> > > > There are possible

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-12 Thread Pavel Stehule
2017-03-12 21:57 GMT+01:00 Noah Misch : > On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > > 2017-03-12 0:56 GMT+01:00 Noah Misch : > > > On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: > > > > There are possible two fixes > > > > > > > > a) clean decl on input - the

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-12 Thread Noah Misch
On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > 2017-03-12 0:56 GMT+01:00 Noah Misch : > > On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: > > > There are possible two fixes > > > > > > a) clean decl on input - the encoding info can be removed from decl part > > > >

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-12 Thread Pavel Stehule
2017-03-12 0:56 GMT+01:00 Noah Misch : > On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: > > Today I played with xml_recv function and with xml processing functions. > > > > xml_recv function ensures correct encoding from document encoding to > server > > encoding. But the decl sect

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-11 Thread Noah Misch
On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote: > Today I played with xml_recv function and with xml processing functions. > > xml_recv function ensures correct encoding from document encoding to server > encoding. But the decl section holds original encoding info - that should > be