The other detail to look at is the ObjectOutputStream cache. If a persistent/long-lived version of that class is used to write all the objects across the network, it can present a problem. That class caches objects and sends object ids for duplicates instead of the entire object, so that object graphs are recreated with the duplicate references as references instead of as new objects. It may be necessary to use the “reset” method on that class to cause the object cache to not grow unbounded or unnecessarily. If you know there are no duplicate objects being sent, that makes it easy to know when to call reset. If there are duplicate objects being sent, but you have no execution boundary to line up with the objects being sent, you may have to design a different way of transporting the data so that you can minimize caching or other unbounded memory use by ObjectOutputStream.
Gregg Wonderly > On Sep 11, 2015, at 12:03 PM, Bryan Thompson <br...@systap.com> wrote: > > One suggestion is to separate the messages from the payload and use a > different protocol for the payload. For example, having the receiver reach > back across the network when it is ready to read the payload. This has > several advantages: > > - The receiver can impose flow control on the heavy messages by deciding > when it wants the data. > - You can use one network for the lighter messages that coordinate activity > and another network for the bandwidth intensive data transfers. > - You can avoid Java serialization for heavy if you have simple objects > such as arrays. > > Bryan > > ---- > Bryan Thompson > Chief Scientist & Founder > SYSTAP, LLC > 4501 Tower Road > Greensboro, NC 27410 > br...@systap.com > http://blazegraph.com > http://blog.bigdata.com <http://bigdata.com> > http://mapgraph.io > > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints > APIs. Blazegraph is now available with GPU acceleration using our disruptive > technology to accelerate data-parallel graph analytics and graph query. > > CONFIDENTIALITY NOTICE: This email and its contents and attachments are > for the sole use of the intended recipient(s) and are confidential or > proprietary to SYSTAP. Any unauthorized review, use, disclosure, > dissemination or copying of this email or its contents or attachments is > prohibited. If you have received this communication in error, please notify > the sender by reply email and permanently delete all copies of the email > and its contents and attachments. > > On Fri, Sep 11, 2015 at 12:38 PM, Palash Ray <paa...@gmail.com> wrote: > >> Hi, >> >> I have a Jini server, and I am doing a lookup from a Jini registry and >> then making a call on the remote. >> >> The client code is: >> LookupLocator lookupLocator = new LookupLocator(jiniRegistryUrl); >> return (Remote) lookupLocator.getRegistrar().lookup(new >> ServiceTemplate(null, >> new Class[]{serviceInterfaceClass}, new Entry[]{new >> Name(serviceName)})); >> We are transmitting pretty heavy objects: ArrayList having a million+ >> rows. It works fine for most part. However, when the list has over 10 >> million rows, the server is still fine. But, the client starts >> behaving weird and throwing java.io.EOFException. I am pasting the >> full stack trace. >> >> Any help would be appreciated. >> >> Thanks, >> Palash. >> >> >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> com.imsi.iss.portiss.jasper.query.RmiQueryExecutor.queryReportData(RmiQueryExecutor.java:112) >> at >> com.imsi.iss.portiss.jasper.query.PortissQueryExecutor.createDatasource(PortissQueryExecutor.java:41) >> ... 16 more >> Caused by: java.rmi.UnmarshalException: exception unmarshalling >> response; nested exception is: >> java.io.EOFException >> at >> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:847) >> at >> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659) >> at >> net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528) >> at >> com.sun.proxy.$Proxy176.getPartnerStatementStrategicValue(Unknown >> Source) >> ... 22 more >> Caused by: java.io.EOFException >> at >> java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.HashMap.readObject(HashMap.java:1155) >> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.ArrayList.readObject(ArrayList.java:733) >> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.HashMap.readObject(HashMap.java:1155) >> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.ArrayList.readObject(ArrayList.java:733) >> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.HashMap.readObject(HashMap.java:1155) >> at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at java.util.ArrayList.readObject(ArrayList.java:733) >> at sun.reflect.GeneratedMethodAccessor291.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at >> com.sun.jini.jeri.internal.runtime.Util.unmarshalValue(Util.java:221) >> at >> net.jini.jeri.BasicInvocationHandler.unmarshalReturn(BasicInvocationHandler.java:1242) >> at >> net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:825) >> ... 25 more >>
