Re: [jibx-users] Object Streaming using JIBX

Luca Buraggi Wed, 13 May 2009 02:03:26 -0700

Hi,
I also am facing with a similar problem. More precisely it is the symmetric problem: I must unmarshall some huge (sometimes greater than 500 Mbytes) XML files composed by a sequence of the same complex element. Something similar to this:
---
<list>
<element> ... </element>
<element> ... </element>
<element> ... </element>
   ...
   ...
</list>
---
While unmarshalling each single <element> is perfectly feasible, any attempt to unmarshall the whole file fails due to lack of heap space.
I am not an expert JIBX user, I am still exploring the framework. But I examined other popular XML binding framework (JAXB and Castor) and I think JIBX architecture is better and more powerful if applied to such a task.
I am now prototyping and testing a solution based on using the API of the UnmarshallingContext class (I can't use the IUnmarshallingContext interface because it hides some methods that are needed, and this is definitely not an optimal solution, but I have no chance)
This is the scenario:
---
IBindingFactory bfact = BindingDirectory.getFactory(Element.class);
// Very ugly, but ...
UnmarshallingContext uctx = (UnmarshallingContext) bfact.createUnmarshallingContext();
uctx.setDocument(new FileInputStream("HugeDataFile.xml"), null);

boolean done = false;
// This loop scans the whole input data file
while (done == false) {
if (uctx.isAt(null, "element") {
    // The current event is a <element> start tag
    Element el = (Element) uctx.unmarshalElement();
    // Now process the el object
    ...
    ...
} else {
    // End of file ?
    if (uctx.next() == IXMLReader.END_DOCUMENT) {
      done = true;
    }
}
}
---

I also will of course appreciate any suggestion or comment ...

Regards,

Luca Buraggi

------------------------------------
Anshuk Pal Chaudhuri ha scritto:

Hi,

I have been working on jibx for quite some time and really appreciate the philosophy went into building it and how the byte code enhancements helps in faster marshalling of the objects into XML. I was thinking to evangelize more into it and was trying to figure out how could the java objects be marshaled/streamed one at a time rather than marshalling the whole list of objects. I have defined the scenario details in the below section:

A query is fetched to hit the database which would fetch say 10000 records (in some defined batches say 500); each record when fetched is populated into a custom object (some custom value object). Rather than setting the objects (10000 in this case) into a parent object’s some list or some array list, I was thinking whether I can marshal the custom value object straight way into the xml and as more records are fetched, every time the populated object is marshaled into the same xml.

So in a way, the complete list of objects (one big parent object) which could have been passed into the jibx marshal only in one time (which would have consume more memory), every a single object would be marshaled and append in the same file every time it is being fetched from the object.

I think there are some apis which can be used to achieve something like the above scenario; like IMarshallingContext. startDocument, IXMLWriter. startTagClosed, IXMLWriter. endTag

Can anybody let me know your thoughts on this.

Any help is appreciated.

Regards,

Anshuk Pal Chaudhuri
**************** CAUTION - Disclaimer ***************** This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions are unlawful. This e-mail may contain viruses. Infosys has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. Infosys reserves the right to monitor and review the content of all messages sent to or from this e-mail address. Messages sent to or from this e-mail address may be stored on the Infosys e-mail system. ***INFOSYS******** End of Disclaimer ********INFOSYS***
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
jibx-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jibx-users
  

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com

_______________________________________________
jibx-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jibx-users

Re: [jibx-users] Object Streaming using JIBX

Reply via email to