Hey Rob, thanks a lot for your input. It clears up a lot of doubts regarding my code. I have rewritten it according to your suggestions and it works. The ticker now fires at regular intervals as expected, with the added bonus that the plugin is able to shut down cleanly upon killing the process. I have add other features like gzip compression as well. My company will be using this plugin in production soon. Thanks all for your help.
On Thu, May 14, 2015 at 6:06 AM, Rob Miller <rmil...@mozilla.com> wrote: > Your code is put together very strangely. The ticker channel is firing > correctly, but once it fires the first time you immediately get trapped in > the for loop that is over the range of the inChan. So you aren't getting > any of the input messages until the first tick fires, but after it fires > you never escape. > > Also, your Stop method will never be called. Stop methods are only for > input plugins, output plugins are signaled to exit by closing the inChan. > > You want to use a select on the inChan and the tickerChan, both of them > enclosed in a loop that will exit when the inChan is closed. The simplest > plugin I can think of that demonstrates this pattern is the CounterFilter: > > > https://github.com/mozilla-services/heka/blob/dev/pipeline/counter_filter.go > > Hope this helps, > > -r > > > > On 05/13/2015 03:42 AM, Alex Jiao wrote: > >> Hi, I'm writing an S3 output plugin for heka. As I am new to golang and >> heka, I'm not too sure what's the best way to go about writing the >> plugin. Hence I wish to seek you guys' help to resolve some issues that >> are plaguing my code. >> >> heka-s3 plugin code: >> https://github.com/uohzxela/heka-s3/blob/master/s3.go >> >> My plugin needs to send message packs from the pipeline at regular >> intervals as specified in the .toml file. It writes to a buffer during >> each interval and upload to a S3 bucket at the next tick. The interval >> is specified by the 'ticker_interval' option. However, I'm not too sure >> whether you need to take care of the ticker logic in the for loop or >> that the output Run function will be invoked at the specified ticker >> interval by Heka service itself. >> >> I have implemented the first case as seen in my code and it led to a >> weird issue. The plugin can be loaded upon starting the Heka service, >> however the ticker can only run once after the elapsed time interval. >> It's only after I sent a TERM signal to the Heka process using kill >> (kill <heka-pid>), that the ticker starts to work. >> >> Could you guys shed light on resolving this issue? Why does the ticker >> not work upon starting the plugin? Does the programmer have to take care >> of the ticker logic or the output runner will automatically be invoked >> at regular intervals by the Heka service? I'd appreciate it as well if >> you can point me to some output plugins that have the ticker >> functionality. >> >> Thank you. >> >> -- >> Alex >> >> >> _______________________________________________ >> Heka mailing list >> Heka@mozilla.org >> https://mail.mozilla.org/listinfo/heka >> >> -- Alex
_______________________________________________ Heka mailing list Heka@mozilla.org https://mail.mozilla.org/listinfo/heka
