Also, the fact that you're seeing this error is a bug in the default buffer config settings that has recently been fixed (https://github.com/mozilla-services/heka/pull/1685). The fix will be in the next release.
-r On 08/28/2015 12:52 PM, Giordano, J C. wrote:
Ming, Buffering has it’s own documentation page: http://hekad.readthedocs.org/en/v0.10.0b1/buffering.html Here’s an example for the TCPOutput: [TcpOutput] message_matcher = "Type !~ /^heka/" address = "upstream.example.com <http://upstream.example.com>:5565" keep_alive = true use_buffering = true [TcpOutput.buffering] max_file_size = 268435456 # 256MiB max_buffer_size = 1073741824 # 1GiB full_action = "block" cursor_update_count = 100 Chris > On Aug 28, 2015, at 1:09 AM, Ho, Ming <[email protected] > <mailto:[email protected]>> wrote: > > Heka community, > I try the following config file to send memstats data to Elasticsearch > using [ElasticSearchOutput]. > [MemStats] > type = "FilePollingInput" > ticker_interval = 5 > file_path = "/proc/meminfo" > decoder = "MemStatsDecoder" > [MemStatsDecoder] > type = "SandboxDecoder" > filename = "lua_decoders/linux_memstats.lua" > [ESJsonEncoder] > index = "test" > es_index_from_timestamp = true > type_name = "%{Type}" > [ElasticSearchOutput] > server = "http://10.211.62.132:9200 <http://10.211.62.132:9200/>" > message_matcher = "Type == 'heka.memstat'" > encoder = "ESJsonEncoder" > flush_interval = 10 > But got “ ElasticSearchOutput can’t initialize buffer” error as shown > below when I run heka. I don’t see max_file_size is a configurable > parameter in the heka doc. Appreciate any help on this > 2015/08/27 21:49:14 Pre-loading: [MemStatsDecoder] > 2015/08/27 21:49:14 Pre-loading: [ESJsonEncoder] > 2015/08/27 21:49:14 Pre-loading: [ElasticSearchOutput] > 2015/08/27 21:49:14 Pre-loading: [MemStats] > 2015/08/27 21:49:14 Pre-loading: [ProtobufDecoder] > 2015/08/27 21:49:14 Loading: [ProtobufDecoder] > 2015/08/27 21:49:14 Pre-loading: [ProtobufEncoder] > 2015/08/27 21:49:14 Loading: [ProtobufEncoder] > 2015/08/27 21:49:14 Pre-loading: [TokenSplitter] > 2015/08/27 21:49:14 Loading: [TokenSplitter] > 2015/08/27 21:49:14 Pre-loading: [HekaFramingSplitter] > 2015/08/27 21:49:14 Loading: [HekaFramingSplitter] > 2015/08/27 21:49:14 Pre-loading: [NullSplitter] > 2015/08/27 21:49:14 Loading: [NullSplitter] > 2015/08/27 21:49:14 Loading: [MemStatsDecoder] > 2015/08/27 21:49:14 Loading: [ESJsonEncoder] > 2015/08/27 21:49:14 Loading: [MemStats] > 2015/08/27 21:49:14 Loading: [ElasticSearchOutput] > 2015/08/27 21:49:14 Starting hekad... > 2015/08/27 21:49:14 Output 'ElasticSearchOutput' failed to start: > can't initialize buffer: `max_file_size` must be greater than maximum > record size of 65794 > 2015/08/27 21:49:14 MessageRouter started. > 2015/08/27 21:49:14 Input started: MemStats > 2015/08/27 21:49:14 Shutdown initiated. > 2015/08/27 21:49:14 Stop message sent to input 'MemStats' > 2015/08/27 21:49:14 Waiting for decoders shutdown > 2015/08/27 21:49:14 Decoders shutdown complete > 2015/08/27 21:49:14 Stop message sent to output 'ElasticSearchOutput' > 2015/08/27 21:49:14 Shutdown complete. > Ming > _______________________________________________ > Heka mailing list > [email protected] <mailto:[email protected]> > https://mail.mozilla.org/listinfo/heka _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
