If I may also add a voice to the other replies, suggesting that you
perhaps look at a different protocol for transmitting such a large list
of objects?
If there are no data dependencies between processing individual objects
(whatever "processing" means - I have no idea what is in these objects,
but it seems related to running reports?) I would submit them in smaller
chunks, or - if you want to process them in parallel over multiple
machines, submit them into a JavaSpace, and have multiple workers
process them.
I can't think of any scenario where submitting 1,000,000 non-trivial
objects in a single ArrayList over the network using Jini would be the
correct approach - even if you tune your memory settings, you can
perhaps then process double the amount. What if you need to process 500
times the volume? A different, de-coupled, streaming approach - as also
suggested by Bryan - would solve most of your problems.
warm regards,
Dawid Loubser
On 11/09/2015 18:38, Palash Ray wrote:
> Hi,
>
> I have a Jini server, and I am doing a lookup from a Jini registry and
> then making a call on the remote.
>
> The client code is:
> LookupLocator lookupLocator = new LookupLocator(jiniRegistryUrl);
> return (Remote) lookupLocator.getRegistrar().lookup(new
> ServiceTemplate(null,
> new Class[]{serviceInterfaceClass}, new Entry[]{new
> Name(serviceName)}));
> We are transmitting pretty heavy objects: ArrayList having a million+
> rows. It works fine for most part. However, when the list has over 10
> million rows, the server is still fine. But, the client starts
> behaving weird and throwing java.io.EOFException. I am pasting the
> full stack trace.
>
> Any help would be appreciated.
>
> Thanks,
> Palash.
>
>
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.imsi.iss.portiss.jasper.query.RmiQueryExecutor.queryReportData(RmiQueryExecutor.java:112)
> at
> com.imsi.iss.portiss.jasper.query.PortissQueryExecutor.createDatasource(PortissQueryExecutor.java:41)
> ... 16 more
> Caused by: java.rmi.UnmarshalException: exception unmarshalling
> response; nested exception is:
> java.io.EOFException
> at
> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:847)
> at
> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659)
> at
> net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528)
> at com.sun.proxy.$Proxy176.getPartnerStatementStrategicValue(Unknown
> Source)
> ... 22 more
> Caused by: java.io.EOFException
> at
> java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.HashMap.readObject(HashMap.java:1155)
> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.ArrayList.readObject(ArrayList.java:733)
> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.HashMap.readObject(HashMap.java:1155)
> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.ArrayList.readObject(ArrayList.java:733)
> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.HashMap.readObject(HashMap.java:1155)
> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at java.util.ArrayList.readObject(ArrayList.java:733)
> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at
> com.sun.jini.jeri.internal.runtime.Util.unmarshalValue(Util.java:221)
> at
> net.jini.jeri.BasicInvocationHandler.unmarshalReturn(BasicInvocationHandler.java:1242)
> at
> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:825)
> ... 25 more
