Right. You can do the deserialization directly in the input, but if you
pass it on to a decoder (which will be running in its own separate
goroutine) that frees up the input to go back to the network to grab
the next record, keeping the pipeline full. Won't matter much when
maxprocs=1, but might allow for higher throughput if you're using
multiple cores.
As always, YMMV, it's better to measure than guess. :)
-r
On Mon 03 Feb 2014 10:08:17 AM PST, Michael Trinkala wrote:
For use in an input plugin 'b' should be written into
pipeline.PipelinePack.MsgBytes and the pack should be sent to a
ProtobufDecoder (which will perform the de-serialization below)
Trink
------------------------------------------------------------------------
*From: *"David Birdsong" <david.birds...@gmail.com>
*To: *"Michael Trinkala" <tr...@mozilla.com>
*Cc: *"heka" <heka@mozilla.org>
*Sent: *Friday, January 31, 2014 11:35:05 PM
*Subject: *Re: [heka] initializing msg from a Message.proto dumped
string
That is a quick way, thanks!
On Jan 31, 2014 10:27 PM, "Michael Trinkala" <tr...@mozilla.com
<mailto:tr...@mozilla.com>> wrote:
import "code.google.com/p/goprotobuf/proto
<http://code.google.com/p/goprotobuf/proto>"
import "github.com/mozilla-services/heka/message
<http://github.com/mozilla-services/heka/message>"
b := // []byte value from your code
msg := new(message.Message)
err := proto.Unmarshal(b, msg)
Trink
------------------------------------------------------------------------
*From: *"David Birdsong" <david.birds...@gmail.com
<mailto:david.birds...@gmail.com>>
*To: *"heka" <heka@mozilla.org <mailto:heka@mozilla.org>>
*Sent: *Friday, January 31, 2014 7:07:19 PM
*Subject: *[heka] initializing msg from a Message.proto
dumped string
What's the easiest way to initialize a message that's been
previously serialized? I'm writing an input module from
kafka that takes care of message framing. So I've got a
[]byte value that could be directly instantiated into a
message object. I'm looking at other input modules for an
example, but so many of them deal with some sort of fd and
parsing the input stream to fully build up a message object.
_______________________________________________
Heka mailing list
Heka@mozilla.org <mailto:Heka@mozilla.org>
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
