Hi Adrian,
I have not time at the moment, but yes a page in the Wiki with a link to a Jira
issue sounds good
Thanks
Jacques
From: "Adrian Crum" <adrian.c...@yahoo.com>
Okay, I did some work on this purely as a learning experience for me. I wanted to learn SAX parsing, so I tried converting the
screen widgets to SAX parsing.
I found a small public-domain framework that makes the whole process very easy. Since all of the model screen widgets subclass a
single base class, I was able to hook them into the parsing framework by just having the base class subclass one of the framework
classes. Model widgets that don't have sub-widgets just needed a new constructor. Model widgets that have sub-widgets needed a
little extra code to handle the sub-widgets, but it was no more code than what already exists to handle the DOM version of the
sub-widgets.
Overall, it was pretty easy and I was surprised when it worked the very first
time I tried it.
If anyone is interested, I would be happy to post the POC code in Jira. Just
let me know.
-Adrian
--- On Sat, 4/25/09, Adrian Crum <adrian.c...@yahoo.com> wrote:
From: Adrian Crum <adrian.c...@yahoo.com>
Subject: Re: Discussion: XML file parsing improvement
To: dev@ofbiz.apache.org
Date: Saturday, April 25, 2009, 10:28 PM
Adam and David,
Thank you for your comments! I'll look into the entity
import code some more.
Personally, I don't have an issue with importing large
XML files. I see it come up from time to time on the mailing
lists. I remember BJ Freeman had to write his own import
code because of some OFBiz limitation.
I'll accept the widget files, scripts, and config files
are too small to optimize. Having event-driven parsing for
those might be an interesting experiment though.
-Adrian
--- On Sat, 4/25/09, Adam Heath
<doo...@brainfood.com> wrote:
> From: Adam Heath <doo...@brainfood.com>
> Subject: Re: Discussion: XML file parsing improvement
> To: dev@ofbiz.apache.org
> Date: Saturday, April 25, 2009, 9:52 PM
> Adrian Crum wrote:
> > OFBiz uses a lot of XML files. When each XML file
is
> > read, it is first parsed into a DOM Document,
then the
> > DOM Document is parsed into OFBiz Java objects.
This
> > two-step process consumes a lot of memory, and it
> > takes more time than it should.
> >
> > There is an alternative - what is called
event-driven
> > parsing. The XML parser can be set up to convert
XML
> > elements directly to the OFBiz Java objects -
> > bypassing the DOM Document build and parse steps.
> > Theoretically, this could provide a huge
performance
> > boost, and it would use less memory. In addition,
it
> > would solve the problem of huge XML files maxing
out
> > server memory during the parse process - like
with
> entity XML
> import/export.
> >
> > Has anyone else considered this? Do you think it
is
> > worth pursuing?
>
> What files are you talking about, that are so huge,
they
> can't be
> parsed with the simpler DOM model?
>
> entity data files are sax based already.
>
> widget files, scripts, config files are small, so
it's
> better to keep
> the simpler algo, as David suggested.
>
> additionally, I already did some memory profiling a
while
> back, and
> interned the long-lived strings from parsed xml. This
> actually
> reduced memory usage.
>
> Another thing, the widgets, scripts, config files are
read
> very
> infrequently, then cached. The time it takes to parse
them
> is not
> really a performance consideration.
>
> As an aside, how much swap do you have on your server?
> Any? Is it
> being used? Then you don't have enough ram. If
your
> work-load is
> causing swap to be used, then you haven't
correctly
> identified your
> work load usage requirements.
>
> The same can be said for java maximum memory
allocation.
