Hi Tushar, In Our case, Generic Record fields are generated at run time from database. I cannot convert into a predefined POJO to pass through output port. Is it mandatory that Generic Record class must have no-arg constructor for kryo serialization ?
Regards, Surya Vamshi -----Original Message----- From: Tushar Gosavi [mailto:[email protected]] Sent: 2016, August, 20 2:33 AM To: [email protected] Subject: Re: kryo Serealization Exception Hi Another option is to create your own Java object and populate the fields you need for further processing from GenericRecord, and send it on the output port. You can use this approach if you can not put the operators in single container, because 1) you need to shuffle based on key or 2) resource constraints. -Tushar. On Sat, Aug 20, 2016 at 3:23 AM, Devendra Tagare <[email protected]> wrote: > Hi, > > You can set the Locality of the parser and the writer to Container local. > > This will ensure that Generic Record from the parser does not get > serialized between containers. > > Thanks, > Dev > > On Fri, Aug 19, 2016 at 2:21 PM, Mukkamula, Suryavamshivardhan > (CWM-NR) <[email protected]> wrote: >> >> Hi, >> >> Can you please help resolve the below issue? >> >> In our project we are using ‘org.apache.avro.generic.GenericRecord’ >> as Tuple writing to a parquet file and we are using avro schema for >> each record. We are getting the below exception, I suppose >> GenericRecord does not have no-arg constructor, and looking for some ideas >> to solve this problem. >> >> ######### Exception ################################## >> >> 2016-08-19 16:29:12,845 [5/silverFileOut:AvroFileOutputOperator] >> ERROR codec.Def aultStatefulStreamCodec fromDataStatePair - >> Catastrophic Error: Execution halted due to Kryo exception! >> com.esotericsoftware.kryo.KryoException: Class cannot be created >> (missing no-arg >> constructor): org.apache.avro.generic.GenericData$Record >> at >> com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstant >> iatorOf(Kryo.java:1228) >> at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) >> at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) >> at >> com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSer >> ializer.java:547) >> at >> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSeria >> lizer.java:523) >> at >> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) >> at >> com.datatorrent.stram.codec.DefaultStatefulStreamCodec.fromDataStateP >> air(DefaultStatefulStreamCodec.java:99) >> at >> com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.p >> rocessPayload(BufferServerSubscriber.java:364) >> at >> com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.s >> weep(BufferServerSubscriber.java:316) >> at >> com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:252) >> at >> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContai >> ner.java:1382) >> 2016-08-19 16:30:09,336 [main] INFO stram.StreamingContainerManager >> updateCheck >> >> Regards, >> Surya Vamshi >> >> >> _____________________________________________________________________ >> __ >> >> If you received this email in error, please advise the sender (by >> return email or otherwise) immediately. You have consented to receive >> the attached electronically at the above-noted email address; please >> retain a copy of this confirmation for future reference. >> >> Si vous recevez ce courriel par erreur, veuillez en aviser >> l'expéditeur immédiatement, par retour de courriel ou par un autre >> moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) >> par voie électronique à l'adresse courriel indiquée ci-dessus; >> veuillez conserver une copie de cette confirmation pour les fins de >> reference future. > > _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
