Re: [xml] xmlDocDump() on Windows

[Trappel Martin]

Daniel Corbe wrote:
> /* Begin code snippet */
> xmlDocPtr mtp3;
> FILE *xml;
>
> mtp3 = xmlReadFile(config->MTP3filename, NULL, 0); xml =
> fopen(config->MTP3filename, FOPEN_MODE_OVERWRITE); if (!xml ||
> ferror(xml)) {
>
> outputf("ERROR WRITING MTP3 XML File: %s\n", strerror(errno)); } else
> { xmlDocDump(xml, mtp3); fclose(xml); }
(...)
> I've tried building my app on both debug and release, and the same thing 
> happens.
> You're saying there may be something wrong with my build environment?
(...)
>   libxml2-2.6.30 (binary) as gotten from:
>   http://www.zlatkovic.com/libxml.en.html


This cannot work.
You cannot reliably work with FILE* as a parameter to any dll unless you are 
100% sure that it uses _exactly_ the same runtime version.

Now, the prebuilt version of libxml2 links against MSVCRT.dll (as it is built 
using VC6, afaik)

You however, using VC8, are linking your program against an exact version of 
MSVCR80?.dll which you can find in the WINDOWS/WinSxS directory.

If you want to use any libxml2 function that relies on runtime lib structures 
like FILE* you have to compile it yourself against the same msvcr* version as 
your application.

Personally, I would avoid these functions and use something like 
xmlDocDumpFormatMemory to write the tree to memory and save that to a file.

Br,
Martin


best regards / mit freundlichen Grüßen,
Martin Trappel
-
Kristl, Seibt & Co Ges.m.b.H. / A-8052 Graz
Tel.: (+43 316) 5995 - 1581
mailto:[EMAIL PROTECTED]
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] Xslt Transform Capacity

[Alex Neblett]

Hi,

Is there a limit on the size of xml files that can be transformed using
libexslt/libxslt?

On a Win32 platform, when transforming files and using parameters, I reach a
limit of roughly 43MB. When transforming strings rather than files, the
limit is closer to 32MB for the same xml document/transformation.

I have created a dll in c/c++ which does the transform. I would be more than
happy to share the sourcecode to my exslt.dll. Perhaps it would help others
as well as provide an opportunity to improve my code. It was based off a
combination of the dated tutorial and xsltproc. Potentially, an improved
version of the code could become part of an updated tutorial.

Thanks,

Alex


___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] Parser error: html entities not defined

[Bruno Dilly]

Thank you very much, Daniel

On 9/10/07, Daniel Veillard <[EMAIL PROTECTED]> wrote:
> On Thu, Sep 06, 2007 at 07:22:21PM -0300, Bruno Dilly wrote:
> > Indeed, the rss is not-well-formed. Is it possible to load an external
> > dtd not included in the rss?
>
>   Yes separately to validate a document. See the documentation.
> What you can't or should NOT try to do is to process something which
> is not well-formed to make it work if it's not XML.
>   If an RSS feed is broken, DROP IT, then people will fix it !
> If you don't I think you make a disservice to the users, and you have
> no garantee from me that what you did to make it work with libxml2
> will continue to work in the future.
>
> > For example, can I load
> > http://my.netscape.com/publish/formats/rss-0.91.dtd before parse the
> > file? And is possible to load it from a local file? How could I do it?
>
>  What do you want to do ? You can use a separated DTD to validate an
> already parsed well-formed XML file. That's possible in the API. What you
> can't do is to modify the parsing to fake a non-existent DTD.
> If you want to  have the DTD local see the catalog support, there is a
> page describing it, and it's a standard.
>http://xmlsoft.org/catalog.html
>
> Daniel
>
> --
> Red Hat Virtualization group http://redhat.com/virtualization/
> Daniel Veillard  | virtualization library  http://libvirt.org/
> [EMAIL PROTECTED]  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
> http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/
>
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmllint as minimal non-validating parser?

[Fred Drake]

On Sep 13, 2007, at 5:33 PM, Chuck Bearden wrote:
> Is there a way to make xmllint do no more than check documents
> against the well-formedness constraints, to emulate a minimal
> non-validating processor?

I hate to sound like advertising for the competition, but  
"xmlwf" (distributed with the Expat library) will handle this.

What's interesting is that xmllint exits with a return code of 0,  
though the man page claims that means no error (as expected),  
suggesting that this message is just a warning.  Using --nowarning  
doesn't suppress the message, however.  Perhaps this is a bug in the  
--nowarning support?


   -Fred

-- 
Fred Drake   
"Chaos is the score upon which reality is written." --Henry Miller



___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Nix]

On 13 Sep 2007, Daniel Corbe stated:
> Why would I run into heap corruption issues unless there's something
> blatantly wrong with xmlDocDump.
>
> My understanding of HEAP vs STACK memory is that local variables come from
> the stack and global variables along with anything malloc() comes off of the
> heap.

This is correct from the point of view of the C abstract machine (except
that it doesn't call the stack anything specifically; it's just
automatic allocation). It's incorrect from the point of view of what the
machine actually does, where (on a Linux box and many/most other ELF
systems):

 - stack variables come off the stack. This is generally strictly limited
   in size, and extremely limited on 32-bit threaded environments because
   of the need to fit all the stacks for all the threads into the address
   space.

 - initialized static variables (both global and local) come from the .data
   section of the executable, which is privately mmap()ped and modifiable.
   This is limited in size only by available memory and address space.

 - uninitialized static variables (both global and local) come from the
   .bss section, which is allocated by the dynamic linker and filled in with
   zeroes. This too is limited in size only by available memory and address
   space.

 - heap allocations are satisfied on nearly all Unixes from an arena
   maintained by the C library, raised and shrunk on demand via the brk()
   system call. Because this is a single contiguous arena, it can suffer
   from fragmentation and overruns. Most C libraries store housekeeping data
   for a block before the start of that block, so underrunning a block can
   corrupt the arena and crash programs on later malloc() or free() calls.
   This is theoretically limited only by available memory and address space,
   but alignment constraints, housekeeping data, and especially fragmentation
   can reduce its effective size substantially as a program runs. On Linux/
   glibc and some other systems, large malloc() allocations are satisfied
   via mmap() directly from the OS, mostly to reduce heap fragmentation.
   (The definition of `large' is changeable by the application and on modern
   glibc versions varies dynamically). Overruns in these areas might cause
   segfaults but will not corrupt other state or cause later crashes in
   other calls.

(Windows's memory allocation models are profoundly different and the last time
I had to deal with them was in the Windows 3.1 days, so anything I could say
would be more misleading than useful. If anyone else wants to describe the
Windows model, feel free.

>Heap memory is essentially limited only by the amount of physical RAM
> and virtual memory in your machine

With modern RAM volumes, address space is a more serious constraint on many
applications. I doubt that Windows apps can allocate anything like as much
as 4Gb on a 32-bit platform.

> whereas your call stack is generally
> limited to 1Mbit per thread by default on Windows.

The amount on Linux 32-bit platforms has varied with time and is
customizable; the default is generally somewhere betwen four and eight
megabytes, IIRC.

> The default stack size in most Linux distributions is unlimited or some very
> high number

stack size  (kbytes, -s) 8192

> so I could easily see how I may have missed a stack issue.
>  Issues with the heap tend to be more visible (in the form of crashes) and
> obvious (dereferencing null or uninitialized pointers, reading/writing
> out-of-bounds, etc)

If this app runs on Linux too, you might want to try to valgrind it and
see if that spots anything. (valgrind is *very* good at detecting
overruns on the heap, although less good at detecting stack
problems. GCC 4's -fmudflap option might also be useful.

-- 
`Some people don't think performance issues are "real bugs", and I think 
such people shouldn't be allowed to program.' --- Linus Torvalds
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Rob Richards]

Hi Daniel,

Daniel Corbe wrote:
> Rob,
>
> I've tried building my app on both debug and release, and the same 
> thing happens.  You're saying there may be something wrong with my 
> build environment?  I'm not entirely convinced of that but I won't 
> rule it out as a possibility.
Yes. You need to either compile libxml2 in statically or make sure that 
your app is using the same run-time (/MD since you are using a 
downloaded one).
>
> I'm running Visual C++ Express 2005 (which is basically VC8, if you 
> actually paid for a copy)
> Microsoft Windows 2003 Server R2 Platform SDK
I have built and tested your example using both VC8 (Vista and XP) and 
VC6 (XP), but you have the additional wildcard in there which I haven't 
tried yet (I believe the libxml2 libs you are using are linked to 
msvcrt.dll). I build my own dlls so things here have a common 
environment. Just an FYI so you don't think I am blowing smoke here, but 
I do get the access violation (within the fwrite call) to occur when i 
compile the executable with a different runtime setting than the libxml2 
dll.

When you call fopen, it creates the file pointer from msvcrt8.dll 
(because you are using VS 2005), while fwrite is being called from the 
msvcrt.dll. So even if you are using the same runtime flags, it is 
probably an issue from the different run time dlls. I will test that 
tomorrow to confirm.

If you are going to distribute the app, I would suggest that you either 
use alternative functions that do not require a FILE pointer passed in 
(unless you are getting it was already created from within the libxml2 
dll), or build and distribute with your own libxml2.dll, or compile in 
the libxml2 functionality statically. Those are the only way you are 
going to be able to guarantee there is no issue there.



I cut out the rest because what I meant about a HEAP issue is that it 
most like was an access issue due to run time problems and not a 
specific problem in code.

Rob
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmllint as minimal non-validating parser?

[Chuck Bearden]

Aack! I forgot to specify my platform and libxml2 version:

  Linux (Ubuntu 7.04)
  xmllint: using libxml version 20627

Chuck

On Thu, Sep 13, 2007 at 04:33:37PM -0500, Chuck Bearden wrote:
> My reading of the XML Recommendation:
> The well-formedness constraint "Entity Declared" [1] does not apply
> to an XML document with an external DTD subset and which does not
> have a standalone declaration of 'no', since on-validating processors
> are not required to read external DTD subsets.  Such a document may
> contain internal general entity references that aren't defined in an
> internal DTD subset and nonetheless be well-formed.
> 
> The attached well-formed, valid document contains a reference to an
> entity defined in the external DTD subset.  However, I can't find a
> way to make xmllint treat it as well-formed:
> 
>   $ xmllint --noout --noent Briantest.xml
>   Briantest.xml:20: parser error : Entity 'plus' not defined
>   +
>   ^
>   $ xmllint --noout Briantest.xml
>   Briantest.xml:20: parser error : Entity 'plus' not defined
>   +
>   ^
>   $
> 
> Is there a way to make xmllint do no more than check documents
> against the well-formedness constraints, to emulate a minimal
> non-validating processor?
> 
> [1] 
> 
> Thanks in advance!
> Chuck
> --
> Chuck Bearden ([EMAIL PROTECTED] ; 713.348.3661)
> XML Engineer, Connexions
> http://cnx.org/


> ___
> xml mailing list, project page  http://xmlsoft.org/
> xml@gnome.org
> http://mail.gnome.org/mailman/listinfo/xml

___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

[xml] xmllint as minimal non-validating parser?

[Chuck Bearden]

My reading of the XML Recommendation:
The well-formedness constraint "Entity Declared" [1] does not apply
to an XML document with an external DTD subset and which does not
have a standalone declaration of 'no', since on-validating processors
are not required to read external DTD subsets.  Such a document may
contain internal general entity references that aren't defined in an
internal DTD subset and nonetheless be well-formed.

The attached well-formed, valid document contains a reference to an
entity defined in the external DTD subset.  However, I can't find a
way to make xmllint treat it as well-formed:

  $ xmllint --noout --noent Briantest.xml
  Briantest.xml:20: parser error : Entity 'plus' not defined
  +
  ^
  $ xmllint --noout Briantest.xml
  Briantest.xml:20: parser error : Entity 'plus' not defined
  +
  ^
  $

Is there a way to make xmllint do no more than check documents
against the well-formedness constraints, to emulate a minimal
non-validating processor?

[1] 

Thanks in advance!
Chuck
--
Chuck Bearden ([EMAIL PROTECTED] ; 713.348.3661)
XML Engineer, Connexions
http://cnx.org/


Briantest.xml
Description: application/xml
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Daniel Corbe]

#ifdef __WIN32__
# define OPEN_MODE_OVERWRITE "wb"# define OPEN_MODE_WRITE "w+b"
# define OPEN_MODE_READ "rb"
#else
# define OPEN_MODE_OVERWRITE "w"
# define OPEN_MODE_WRITE "w+b"
# define OPEN_MODE_READ "rb"
#endif /* __WIN32__ */

Because windows' fopen() by default opens files in "text mode" which causes
fread() to do character translations -- for examle CRLF = LF, \0 = EOF, etc.
 Since this code tries to be portable, we define these macros to make
windows fopen() and fread()  behave more like fopen() and fread() in
POSIX environments.

On 9/13/07, Daniel Corbe <[EMAIL PROTECTED]> wrote:
>
> Rob,
>
> I've tried building my app on both debug and release, and the same thing
> happens.  You're saying there may be something wrong with my
> build environment?  I'm not entirely convinced of that but I won't rule it
> out as a possibility.
>
>
> I'm running Visual C++ Express 2005 (which is basically VC8, if you
> actually paid for a copy)
> Microsoft Windows 2003 Server R2 Platform SDK
>
>
> It's all basically the latest and greatest non-beta stuff that Microsoft
> has out right now.
>
>
> libxml2-2.6.30 (binary) as gotten from:
> http://www.zlatkovic.com/libxml.en.html
>
>
> Why would I run into heap corruption issues unless there's something
> blatantly wrong with xmlDocDump.
>
>
> My understanding of HEAP vs STACK memory is that local variables come from
> the stack and global variables along with anything malloc() comes off of the
> heap.  Heap memory is essentially limited only by the amount of physical RAM
> and virtual memory in your machine whereas your call stack is generally
> limited to 1Mbit per thread by default on Windows.
>
>
> If anything I would say that there's a stack corruption issue here, not a
> heap corruption issue, and that could easily be caused somewhere else in the
> code (my code) without me noticing.
>
>
> The default stack size in most Linux distributions is unlimited or some
> very high number so I could easily see how I may have missed a stack issue.
>  Issues with the heap tend to be more visible (in the form of crashes) and
> obvious (dereferencing null or uninitialized pointers, reading/writing
> out-of-bounds, etc)
>
>
> That's a semantic point at any rate.
>
>
> It's beginning to look like I'm going to need to fetch the source and
> compile it myself so that I can give you more useful debugging information
> :(
>
>
> xmlDocDump is the cleaner approach in this case than xmlSaveFile.  The
> reason being is that the xmlSaveFile documentation clearly states that
> compression is used by default if it is compiled in to the library and this
> is a bad thing for general deployment.
>
>
> -Daniel
>
>
> On 9/13/07, Rob Richards <[EMAIL PROTECTED]> wrote:
> >
> > Daniel Corbe wrote:
> > > Sorry but I must reject this notion.  You're trying to say that it's
> > > broken the way I'm using it, but just on Windows?  Because like I said
> > > on Linux it runs fine.
> > I am saying that it was either that or you are mixing release and debug
> > versions and probably running into a problem on the HEAP somewhere.
> > Just to make sure this wasn't something new, I did try your example code
> > and it worked flawlessly (hardcoding path names of course and using "wb"
> > for mode as I have no idea what you defined OPEN_MODE_OVERWRITE as).
> >
> > I used both a debug version of libxml2 and my calling executable. They
> > were both built with VS 2005 and I'll test again later building the
> > executeable against a VC6 version of libxml2.
> >
> > >
> > > I'm trying to do something EXTREMELY simple here.  See below.  In my
> > > experience if xmlReadFile fails I get a bunch of things to stderr
> > > about why and my program then exists.
> > >
> > > And I have also verified with a debugger that mtp3 is indeed a valid
> > > pointer.  So if mtp3 is a valid and parsed and my FILE*xml is also a
> > > valid pointer,  xmlDocDump would have no reason to fail.
> > >
> > > I have attached some screenshots highlighting the failure.
> > >
> > > /* Begin code snippet */
> > > xmlDocPtr mtp3;
> > > FILE *xml;
> > >
> > > mtp3 = xmlReadFile(config->MTP3filename, NULL, 0);
> > > xml = fopen(config->MTP3filename, FOPEN_MODE_OVERWRITE);
> > > if (!xml || ferror(xml))
> > > {
> > >
> > > outputf("ERROR WRITING MTP3 XML File: %s\n", strerror(errno));
> > > }
> > > else
> > > {
> > > xmlDocDump(xml, mtp3);
> > > fclose(xml);
> > > }
> > Try a debug version of libxml2 and if it crashes then, you should be
> > able to find out why at least because I cannot reproduce your crash.
> >
> > Rob
> >
>
>
>
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Daniel Corbe]

Rob,
I've tried building my app on both debug and release, and the same thing
happens.  You're saying there may be something wrong with my
build environment?  I'm not entirely convinced of that but I won't rule it
out as a possibility.

I'm running Visual C++ Express 2005 (which is basically VC8, if you actually
paid for a copy)
Microsoft Windows 2003 Server R2 Platform SDK

It's all basically the latest and greatest non-beta stuff that Microsoft has
out right now.

libxml2-2.6.30 (binary) as gotten from:
http://www.zlatkovic.com/libxml.en.html


Why would I run into heap corruption issues unless there's something
blatantly wrong with xmlDocDump.

My understanding of HEAP vs STACK memory is that local variables come from
the stack and global variables along with anything malloc() comes off of the
heap.  Heap memory is essentially limited only by the amount of physical RAM
and virtual memory in your machine whereas your call stack is generally
limited to 1Mbit per thread by default on Windows.

If anything I would say that there's a stack corruption issue here, not a
heap corruption issue, and that could easily be caused somewhere else in the
code (my code) without me noticing.

The default stack size in most Linux distributions is unlimited or some very
high number so I could easily see how I may have missed a stack issue.
 Issues with the heap tend to be more visible (in the form of crashes) and
obvious (dereferencing null or uninitialized pointers, reading/writing
out-of-bounds, etc)

That's a semantic point at any rate.

It's beginning to look like I'm going to need to fetch the source and
compile it myself so that I can give you more useful debugging information
:(

xmlDocDump is the cleaner approach in this case than xmlSaveFile.  The
reason being is that the xmlSaveFile documentation clearly states that
compression is used by default if it is compiled in to the library and this
is a bad thing for general deployment.


-Daniel


On 9/13/07, Rob Richards <[EMAIL PROTECTED]> wrote:
>
> Daniel Corbe wrote:
> > Sorry but I must reject this notion.  You're trying to say that it's
> > broken the way I'm using it, but just on Windows?  Because like I said
> > on Linux it runs fine.
> I am saying that it was either that or you are mixing release and debug
> versions and probably running into a problem on the HEAP somewhere.
> Just to make sure this wasn't something new, I did try your example code
> and it worked flawlessly (hardcoding path names of course and using "wb"
> for mode as I have no idea what you defined OPEN_MODE_OVERWRITE as).
>
> I used both a debug version of libxml2 and my calling executable. They
> were both built with VS 2005 and I'll test again later building the
> executeable against a VC6 version of libxml2.
>
> >
> > I'm trying to do something EXTREMELY simple here.  See below.  In my
> > experience if xmlReadFile fails I get a bunch of things to stderr
> > about why and my program then exists.
> >
> > And I have also verified with a debugger that mtp3 is indeed a valid
> > pointer.  So if mtp3 is a valid and parsed and my FILE*xml is also a
> > valid pointer,  xmlDocDump would have no reason to fail.
> >
> > I have attached some screenshots highlighting the failure.
> >
> > /* Begin code snippet */
> > xmlDocPtr mtp3;
> > FILE *xml;
> >
> > mtp3 = xmlReadFile(config->MTP3filename, NULL, 0);
> > xml = fopen(config->MTP3filename, FOPEN_MODE_OVERWRITE);
> > if (!xml || ferror(xml))
> > {
> >
> > outputf("ERROR WRITING MTP3 XML File: %s\n", strerror(errno));
> > }
> > else
> > {
> > xmlDocDump(xml, mtp3);
> > fclose(xml);
> > }
> Try a debug version of libxml2 and if it crashes then, you should be
> able to find out why at least because I cannot reproduce your crash.
>
> Rob
>
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Rob Richards]

Daniel Corbe wrote:
> Sorry but I must reject this notion.  You're trying to say that it's 
> broken the way I'm using it, but just on Windows?  Because like I said 
> on Linux it runs fine.
I am saying that it was either that or you are mixing release and debug 
versions and probably running into a problem on the HEAP somewhere.
Just to make sure this wasn't something new, I did try your example code 
and it worked flawlessly (hardcoding path names of course and using "wb" 
for mode as I have no idea what you defined OPEN_MODE_OVERWRITE as).

I used both a debug version of libxml2 and my calling executable. They 
were both built with VS 2005 and I'll test again later building the 
executeable against a VC6 version of libxml2.

>
> I'm trying to do something EXTREMELY simple here.  See below.  In my 
> experience if xmlReadFile fails I get a bunch of things to stderr 
> about why and my program then exists.  
>
> And I have also verified with a debugger that mtp3 is indeed a valid 
> pointer.  So if mtp3 is a valid and parsed and my FILE*xml is also a 
> valid pointer,  xmlDocDump would have no reason to fail.
>
> I have attached some screenshots highlighting the failure.
>
> /* Begin code snippet */
> xmlDocPtr mtp3;
> FILE *xml;
>
> mtp3 = xmlReadFile(config->MTP3filename, NULL, 0);
> xml = fopen(config->MTP3filename, FOPEN_MODE_OVERWRITE);
> if (!xml || ferror(xml))
> {
>
> outputf("ERROR WRITING MTP3 XML File: %s\n", strerror(errno));
> }
> else
> {
> xmlDocDump(xml, mtp3);
> fclose(xml);
> }
Try a debug version of libxml2 and if it crashes then, you should be 
able to find out why at least because I cannot reproduce your crash.

Rob
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml

Re: [xml] xmlDocDump() on Windows

[Rob Richards]

Daniel Corbe wrote:
> Hi,
>
> As of 2.6.30 this issue doesn't seem to be resolved and going back in 
> the mailing list archives others have brought it up with no success.
>
>  
> On windows only (code has been tested under Linux and Mac OS X as 
> well) xmlDocDump() and family crashes.  
>
> Is there an alternative I can use in the Tree API to write the file to 
> disk or a memory segment (so I can write it to disk myself?)
>
> Can anyone confirm this?  Is this a known issue?  Is there a work-around?
What issue in particular are you referring to? xmlDocDump() and the rest 
all work perfectly fine on windows, so it's more likely something with 
your usage causing the crash.

Rob
___
xml mailing list, project page  http://xmlsoft.org/
xml@gnome.org
http://mail.gnome.org/mailman/listinfo/xml