There is some information available here: http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
If the object is Java serializable, you can set the stream codec or wrap into KryoJdkContainer: https://github.com/apache/apex-malhar/tree/master/library/src/main/java/com/datatorrent/lib/codec On Mon, Aug 22, 2016 at 11:42 AM, Mukkamula, Suryavamshivardhan (CWM-NR) < [email protected]> wrote: > Hi Tushar, > > In Our case, Generic Record fields are generated at run time from > database. I cannot convert into a predefined POJO to pass through output > port. > Is it mandatory that Generic Record class must have no-arg constructor for > kryo serialization ? > > Regards, > Surya Vamshi > -----Original Message----- > From: Tushar Gosavi [mailto:[email protected]] > Sent: 2016, August, 20 2:33 AM > To: [email protected] > Subject: Re: kryo Serealization Exception > > Hi > > Another option is to create your own Java object and populate the fields > you need for further processing from GenericRecord, and send it on the > output port. You can use this approach if you can not put the operators in > single container, because 1) you need to shuffle based on key or 2) > resource constraints. > > -Tushar. > > > On Sat, Aug 20, 2016 at 3:23 AM, Devendra Tagare < > [email protected]> wrote: > > Hi, > > > > You can set the Locality of the parser and the writer to Container local. > > > > This will ensure that Generic Record from the parser does not get > > serialized between containers. > > > > Thanks, > > Dev > > > > On Fri, Aug 19, 2016 at 2:21 PM, Mukkamula, Suryavamshivardhan > > (CWM-NR) <[email protected]> wrote: > >> > >> Hi, > >> > >> Can you please help resolve the below issue? > >> > >> In our project we are using ‘org.apache.avro.generic.GenericRecord’ > >> as Tuple writing to a parquet file and we are using avro schema for > >> each record. We are getting the below exception, I suppose > >> GenericRecord does not have no-arg constructor, and looking for some > ideas to solve this problem. > >> > >> ######### Exception ################################## > >> > >> 2016-08-19 16:29:12,845 [5/silverFileOut:AvroFileOutputOperator] > >> ERROR codec.Def aultStatefulStreamCodec fromDataStatePair - > >> Catastrophic Error: Execution halted due to Kryo exception! > >> com.esotericsoftware.kryo.KryoException: Class cannot be created > >> (missing no-arg > >> constructor): org.apache.avro.generic.GenericData$Record > >> at > >> com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstant > >> iatorOf(Kryo.java:1228) > >> at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo. > java:1049) > >> at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > >> at > >> com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSer > >> ializer.java:547) > >> at > >> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSeria > >> lizer.java:523) > >> at > >> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > >> at > >> com.datatorrent.stram.codec.DefaultStatefulStreamCodec.fromDataStateP > >> air(DefaultStatefulStreamCodec.java:99) > >> at > >> com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.p > >> rocessPayload(BufferServerSubscriber.java:364) > >> at > >> com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.s > >> weep(BufferServerSubscriber.java:316) > >> at > >> com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:252) > >> at > >> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContai > >> ner.java:1382) > >> 2016-08-19 16:30:09,336 [main] INFO stram.StreamingContainerManager > >> updateCheck > >> > >> Regards, > >> Surya Vamshi > >> > >> > >> _____________________________________________________________________ > >> __ > >> > >> If you received this email in error, please advise the sender (by > >> return email or otherwise) immediately. You have consented to receive > >> the attached electronically at the above-noted email address; please > >> retain a copy of this confirmation for future reference. > >> > >> Si vous recevez ce courriel par erreur, veuillez en aviser > >> l'expéditeur immédiatement, par retour de courriel ou par un autre > >> moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) > >> par voie électronique à l'adresse courriel indiquée ci-dessus; > >> veuillez conserver une copie de cette confirmation pour les fins de > reference future. > > > > > > _______________________________________________________________________ > If you received this email in error, please advise the sender (by return > email or otherwise) immediately. You have consented to receive the attached > electronically at the above-noted email address; please retain a copy of > this confirmation for future reference. > > Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur > immédiatement, par retour de courriel ou par un autre moyen. Vous avez > accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à > l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de > cette confirmation pour les fins de reference future. >
