Re: [heka] Heka master compilation error

2016-01-06 Thread Emily Gu
Hi Rob, It helped. Thanks for the details. Thanks, Emily On Wed, Jan 6, 2016 at 2:18 PM, Rob Miller wrote: > On 01/06/2016 01:53 PM, Emily Gu wrote: > >> Thanks Rob for your quick response. >> >> I used "git clone..." to clone the Mozilla Heka under my gvm pkgset >> src/github/mozilla-service

Re: [heka] Heka master compilation error

2016-01-06 Thread Emily Gu
I created "plugin_loader.cmake" file with the following content, add_external_plugin(/Users/xxx/.gvm/pkgsets/go1.5/heka-input/src/github.com//heka-input :local) source build.sh gave the following error. CMake Error at cmake/plugin_loader.cmake:1 (add_external_plugin): add_external_plugin F

Re: [heka] Heka master compilation error

2016-01-06 Thread Rob Miller
On 01/06/2016 01:53 PM, Emily Gu wrote: Thanks Rob for your quick response. I used "git clone..." to clone the Mozilla Heka under my gvm pkgset src/github/mozilla-service directory. Then ran "source build.sh" and "make install". In my own project, when I reference Heka installs itself into it

Re: [heka] Heka master compilation error

2016-01-06 Thread Emily Gu
Thanks Rob for your quick response. I used "git clone..." to clone the Mozilla Heka under my gvm pkgset src/github/mozilla-service directory. Then ran "source build.sh" and "make install". In my own project, when I reference "github.com/mozilla-services/heka/pipeline" It gave the error. Were y

Re: [heka] Heka master compilation error

2016-01-06 Thread Rob Miller
Not sure how you're trying to compile Heka. The recommended way, as noted in the docs (http://hekad.readthedocs.org/en/v0.10.0/installing.html#building-hekad-with-external-plugins), is to put your code in an `externals` folder and add a `cmake/plugin_loader.cmake` file that uses the special `:

Re: [heka] Space Delimited File Containing Values with Spaces

2016-01-06 Thread Rob Miller
You clearly won't be able to use spaces as a delimiter if the value you're capturing can contain spaces. For the values, you'll probably want to consider the end delimiter to be either the next key (i.e. some alphanumeric chars followed by an `=`) or EOL. And you'll need to be careful to not ad

[heka] Heka master compilation error

2016-01-06 Thread Emily Gu
Dear Hekas, I'm writing a Heka plugin for our project. After clone Heka master and follow the installation instruction to build it, I got the following compilation error when import "github.com/mozilla-services/heka/pipeline" in myplugin.go file. # github.com/mozilla-services/heka/message ../../

Re: [heka] Geo IP

2016-01-06 Thread Rob Miller
Right, MultiDecoder is what you want. Also note that the release binaries don't include the GeoIPDecoder. You'll want to build from source with GeoIP.h in your include path, and make sure you have the libGeoIP library on the target machines. -r On 01/06/2016 09:47 AM, Robbie Jacka wrote: Yo

Re: [heka] Logstreamer fails if log directory does not exist

2016-01-06 Thread Tony Cesaro
This sounds like something that could easily be solved with a configuration management solution that builds Heka's configuration from templates based on logic defined within it. You could check whether the host being deployed to has the directory you are wanting to check for the existence of and t

Re: [heka] Geo IP

2016-01-06 Thread Robbie Jacka
You should look into using a MultiDecoder, with a cascade strategy of 'all'. [ProtobufGeoIpMultiDecoder] type = "MultiDecoder" cascade_strategy = "all" log_sub_errors = false subs = [ 'ProtobufDecoder', 'GeoIpDecoder' ] [GeoIpDecoder] type = "GeoIpDecoder" db_file="/etc/geoip/GeoLiteCity.dat" sou

[heka] Geo IP

2016-01-06 Thread Ramin Ali Dousti
Hi, I have an array of heka servers that would parse my application logs and use TcpOutput with ProtobufEncoder to send the produced Heka Messages to a centralized heka daemon. The message would have a field that represents an IP address. On this centralized heka daemon I'd like to do a goe IP loo

Re: [heka] Logstreamer fails if log directory does not exist

2016-01-06 Thread Eric LEMOINE
On Tue, Jan 5, 2016 at 12:09 AM, Rob Miller wrote: > I could maybe be convinced, but I'm leaning negative on this one. > Logstreamer is already a pretty complicated feature, with a lot of fiddly > code. I'm loathe to add yet more features / complexity unless the win is > big. It doesn't seem to te