Re: How to deserialize the object without avro schema?

[Ratha v]

Thanks..I try to follow that trick..

On 19 April 2016 at 17:10,  wrote:

>
>
> Hi, avro Schemas imply a pretty big overhead, if you would include them in
> every message. It's good practice to include a schema id with the
> message... Then you need a schema repository to lookup the matching schema
> based on the id.
>
> Have a look at confluent.io. They offer a schema repo among other Kafka
> related tools.
>
> Regards
>
> Jan
>
>
>
> Sent from my iPhone
> > On 19 Apr 2016, at 08:02, Ratha v  wrote:
> >
> > Hi all;
> >
> > I try to publish/consume my java objects to kafka. I use Avro schema.
> >
> > My basic program works fine. In my program i use my schema in the
> producer
> > (for encoding) and consumer (decoding).
> >
> > If i publish different objects to different topics( eg: 100 topics)at the
> > receiver, i do not know, what type of message i received. I would like to
> > get the avro schema from the received byte and would like to use that for
> > decoding. Is that right? If so, how can i retrieve from the received
> object?
> > Or is there any better approach?
> >
> > Thanks.
> > --
> > -Ratha
> > http://vvratha.blogspot.com/
>



-- 
-Ratha
http://vvratha.blogspot.com/

Re: How to deserialize the object without avro schema?

[jan . omar]

Hi, avro Schemas imply a pretty big overhead, if you would include them in 
every message. It's good practice to include a schema id with the message... 
Then you need a schema repository to lookup the matching schema based on the id.

Have a look at confluent.io. They offer a schema repo among other Kafka related 
tools.

Regards 

Jan



Sent from my iPhone
> On 19 Apr 2016, at 08:02, Ratha v  wrote:
> 
> Hi all;
> 
> I try to publish/consume my java objects to kafka. I use Avro schema.
> 
> My basic program works fine. In my program i use my schema in the producer
> (for encoding) and consumer (decoding).
> 
> If i publish different objects to different topics( eg: 100 topics)at the
> receiver, i do not know, what type of message i received. I would like to
> get the avro schema from the received byte and would like to use that for
> decoding. Is that right? If so, how can i retrieve from the received object?
> Or is there any better approach?
> 
> Thanks.
> -- 
> -Ratha
> http://vvratha.blogspot.com/

Re: How to deserialize the object without avro schema?

[Liquan Pei]

Hi Ratha,

Do you want to give a try to the schema registry and Avro serializers using
it?
http://docs.confluent.io/2.0.1/schema-registry/docs/serializer-formatter.html#serializer

Thanks,
Liquan

On Mon, Apr 18, 2016 at 11:02 PM, Ratha v  wrote:

> Hi all;
>
> I try to publish/consume my java objects to kafka. I use Avro schema.
>
> My basic program works fine. In my program i use my schema in the producer
> (for encoding) and consumer (decoding).
>
> If i publish different objects to different topics( eg: 100 topics)at the
> receiver, i do not know, what type of message i received. I would like to
> get the avro schema from the received byte and would like to use that for
> decoding. Is that right? If so, how can i retrieve from the received
> object?
> Or is there any better approach?
>
> Thanks.
> --
> -Ratha
> http://vvratha.blogspot.com/
>



-- 
Liquan Pei
Software Engineer, Confluent Inc