1. We use kryo with snappy compression
2. Why do you say that Bolt should be used? What would be the wrong
thing to do?
On 20/06/2017 08:37, Nikhil Ranjan Silsarma wrote:
Hi,
Here are my five cents for this.
1. you should use Bolt to deserialize the message, which will make
things clean and clear
2. Kryo or simple java byte[] is good enough
https://community.hortonworks.com/articles/18388/storm-serialization-with-avro-using-kryo-serialize.html
Thanks,
Nikhil
On 20 June 2017 at 05:51, pradeep s <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I am receiving avro messages from Kafka spout in below schema .
{
"type": "record",
"name": "generic_wrapper",
"namespace": "oracle.goldengate",
"fields": [{
"name": "table_name",
"type": "string"
}, {
"name": "schema_fingerprint",
"type": "long"
}, {
"name": "*payload*",
"type": "*bytes*"
}]
}
This is deserialized using Kafk Spout in Storm 1.0.0 using below
config in topology
/Builder<String, MdpData> builder = new
KafkaSpoutConfig.Builder<String, MdpData>(kafkaBootstrapServers,/
/ StringDeserializer.class, AvroDeserializer.class,
topicName);/
/
/
/ KafkaSpoutConfig<String, MdpData> spoutConfig =/
/
builder.setGroupId(kafkaGroupId).setProp(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG,
"false")/
/.setFirstPollOffsetStrategy(FirstPollOffsetStrategy.UNCOMMITTED_EARLIEST).build();/
/ KafkaSpout<String, MdpData> kafkaSpout = new
KafkaSpout<String, MdpData>(spoutConfig);/
There is another schema associated to payload field in the above
schema and i need to do one deserialization to retrieve the
payload bytes to a java POJO.
Can i use java deserialization in the bolt to acheive this . There
is only one bolt.
In docs , its mentioned java deserialization is not good in terms
of performance .
Whats the other option to deserialize tje payload bytes .
Regards
Pradeep S
--
My THALES email is [email protected].
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France
