[heka] Heka ETL scopes

2015-01-20 Thread Monosij Dutta-Roy
Hello Rob and Heka users - I was exploring Heka (on Ubuntu VM) and had a few questions. i will try to keep them brief. ... 1. Can Heka be used, being used as an ETL tool to load data from various files into DBMS? or is it mainly for log processing and/or stream processing? 2. To me it seems that

Re: [heka] Remove Sensitive

2015-01-20 Thread Michael Giannini
Thanks! I ended up creating a lua decoder: require "string" function process_message () local message = read_message("Payload") local matches = 0 message, matches = string.gsub(message, 'password=[%a%d]+', "password=[filtered]") m

Re: [heka] Remove Sensitive

2015-01-20 Thread Michael Trinkala
You can strip it in a SandboxDecoder (on the way into Heka) or in a SandboxEncoder (on the way out) depending on your requirements. You would just do a search and replace on the request_uri (if you are using the common log format grammars). On another note, having sensitive information in the URI