Yeah, I did some guarantee to keep the data structure imutable just now.
But I still cannot emit my data structure.
There must be some problem in my structure, but I don't know what kind of lapse
would cause such error.
I got a data structure like this:
public class Counter implements Serializable{
private Counter left = null;
private Counter right = null;
/*
the rest
*/
}
Is this you what referred to recursive reference?
Thank you very much.
yang...@bupt.edu.cn
From: Nathan Leung
Date: 2015-05-21 10:25
To: yang...@bupt.edu.cn
CC: user
Subject: Re: Re: StackOverFlow
If the data structure is not immutable then you cannot guarantee that it's the
same as when your partner emitted it. :) It's hard to say without code, and
even then, if it's a race condition between bolts I'm guessing it won't be easy
to diagnose. If you have a fanout (bolt b and bolt c both subscribe to bolt
a), then you could be in trouble (if tuple stays in the same worker for both b
and c). If bolt a emits a tuple, then modifies the object, you could be in
trouble. And whether you get into trouble or not, and how much trouble,
depends on what you are doing with the object.
On Wed, May 20, 2015 at 9:21 PM, yang...@bupt.edu.cn <yang...@bupt.edu.cn>
wrote:
Nathan,
You are right ! I'm trying to emitting some data structure.
and I'm pretty sure the problem is caused by this.
but I can't figure out what is wrong with my data structure...
You said that the data structure may be cyclic, but my partner emitted it
perfectly,using the same data structure.
Do I have to do some serialization myself?
Or is some rules I overlooked?
If the data structure is not kept imutable, will this error happen too?
thank you~
yang...@bupt.edu.cn
From: Nathan Leung
Date: 2015-05-20 21:12
To: user
Subject: Re: StackOverFlow
It looks like he's in a recursive loop in a java serialization routine
(ObjectOutputStream). This leads me to believe that he is serializing some
data structure (perhaps by emitting it in the framework) and since the data
structure is cyclic, the stack ends up blown.
On Wed, May 20, 2015 at 9:03 AM, Jeffery Maass <maas...@gmail.com> wrote:
Nathan, could you expand on what you mean by "your data structure".
yang...@bupt.edu.cn, could you try turning off
Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION ?
Config conf = new backtype.storm.Config();
conf.setFallBackOnJavaSerialization(false);
StormSubmitter.submitTopology(topo_name, conf, builder.createTopology());
If what I think is happening is true, you will receive a different error in
your worker.
Thank you for your time!
+++++++++++++++++++++
Jeff Maass
linkedin.com/in/jeffmaass
stackoverflow.com/users/373418/maassql
+++++++++++++++++++++
On Tue, May 19, 2015 at 10:23 PM, Nathan Leung <ncle...@gmail.com> wrote:
Looks like you have a reference loop in your data structure
On May 19, 2015 11:07 PM, "yang...@bupt.edu.cn" <yang...@bupt.edu.cn> wrote:
hi,
I encountered the following error exception:
java.lang.StackOverflowError at
java.io.ObjectStreamClass$FieldReflector.getPrimFieldValues(ObjectStreamClass.java:1930)
at java.io.ObjectStreamClass.getPrimFieldValues(ObjectStreamClass.java:1233)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1532)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
…………
I tried to locate the error, but I could hardly figure it out.
I checked all my for loop and while loop, and there is no infinite loop.
all tuples are 'newed' and kept imutable.
how can I figure this out?
thanks ...
ps. this happened both in clusters and local mode.
yang...@bupt.edu.cn
