Re: Example needed for Perl Search::Elasticsearch
Clinton,
I get that, but for some reason it's not that easy to map them for a novice
to the API. I'm used to using kibana and have recently just started
messing with the Perl API to produce some automated reports. I find the
API to be very robust and has lots of features, but the lack of more
complex examples is inhibiting. I have a simple boolean query that I
struggled to get working from the examples and trying to use the query DSL
to make it work and I'm constantly getting parse errors. It wasn't until I
came across this thread that I actually got the thing to "work" without
parse errors. But it still doesn't work the way I would expect and that's
probably something flawed in my understanding. For example,
My query:
body => {
'query' => {
"bool" => {
"must" => [
{ 'match' => { '@message' => 'session opened for
user' } },
{ 'match' => { 'tags' => 'syslog' } } ]
}
}
}
This returns a number of results that I wasn't expecting. For example:
in "@message" I get:
"Accepted publickey for from xxx.xxx.xxx.xxx port 1415 ssh2
I'm not sure why.
Does it match ANY word in my message? In this case "for". How do I get it
to match the entire string?
Thanks, and sorry for jumping in on someone else's thread. It just seemed
appropriate.
On Wednesday, August 13, 2014 9:42:38 AM UTC-4, Clinton Gormley wrote:
>
> Hiya
>
> > Simple question, but there seems to be a lack of detailed examples for
> using the otherwise very useful Search::Elasticsearch CPAN module !
>
> The idea was that the API of the module maps very closely to all of the
> REST APIs in Elasticsearch, so that anything that works with raw curl
> statements should be easy to translate into requests with Search::ES.
>
> Btw, you can always see the equivalent curl statement output to STDERR
> with the following:
>
> $e = Search::Elasticsearch->new( trace_to => 'Stderr')
>
>
> Would this be the correct syntax ?
>>
>> {match => { severity => {query=>'info',boost=>20}}}
>>
>>
>> Even with the agressive boost, I'm still getting "notice" as the
>> prioritised results ?
>>
>
> That is the correct syntax. Perhaps try just searching for "info" to see
> if you actually have matching results?
>
>
>>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e03c668d-2801-43b1-99a1-0053106f202d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Example needed for Perl Search::Elasticsearch
Hiya
> Simple question, but there seems to be a lack of detailed examples for
using the otherwise very useful Search::Elasticsearch CPAN module !
The idea was that the API of the module maps very closely to all of the
REST APIs in Elasticsearch, so that anything that works with raw curl
statements should be easy to translate into requests with Search::ES.
Btw, you can always see the equivalent curl statement output to STDERR with
the following:
$e = Search::Elasticsearch->new( trace_to => 'Stderr')
Would this be the correct syntax ?
>
> {match => { severity => {query=>'info',boost=>20}}}
>
>
> Even with the agressive boost, I'm still getting "notice" as the
> prioritised results ?
>
That is the correct syntax. Perhaps try just searching for "info" to see
if you actually have matching results?
>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/46ddc9bf-5bc4-417f-a26d-82c1c5679eb0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Example needed for Perl Search::Elasticsearch
Would this be the correct syntax ?
{match => { severity => {query=>'info',boost=>20}}}
Even with the agressive boost, I'm still getting "notice" as the
prioritised results ?
On Wednesday, 13 August 2014 12:09:43 UTC+1, Jörg Prante wrote:
>
> A reason may be that your result set size is too small for containing both
> severity values. You could either try a larger result set size, or boost
> the "info" clause so you get docs with "info" before "notice.
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:51 PM, Log Muncher > wrote:
>
>> Well the the Perl module certainly doesn't complain about the syntax, but
>> it stil doesn't manage to output anything other than the "notice" severity ?
>>
>> $ perl test.pl | fgrep "severity"
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>>
>>
>> $ cat test.pl
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>>
>> #match => { severity => 'notice'}
>>
>> "bool" => {
>> "should" => [
>> {match => { severity => 'notice'}},
>> {match => { severity => 'info'}}
>> ]
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>>
>>> Try this to search "notice" or "info" severity.
>>>
>>> my $results = $e->search(
>>>index => 'logstash-2014.08.13',
>>>body => {
>>>query => {
>>> "bool" => {
>>> "should" => [
>>> { match => { severity => 'notice'} },
>>> { match => { severity => 'info'} }
>>> ]
>>> }
>>> }
>>>}
>>> );
>>>
>>>
>>> Jörg
>>>
>>>
>>> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
>>> wrote:
>>>
Hi,
Simple question, but there seems to be a lack of detailed examples for
using the otherwise very useful Search::Elasticsearch CPAN module !
I'm getting syslog data into elasticsearch via fluentd.
What I'd like to do now is run a perl search that will give me results
for notice, emerg and crit events. As a test (seeing as I don't get many
emerg/crit events !), I've tried the below, but it only seems to pick up
notice events and doesn't return any info events !
Help welcome !
Thanks.
Tim
#!/usr/bin/perl
use 5.014;
use strict;
use warnings;
use autodie;
use Data::Dumper;
use Search::Elasticsearch;
my $e = Search::Elasticsearch->new();
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
"bool" => {
"must" => {match => { severity => 'notice'},match
=> { severity => 'info'}}
}
}
}
);
print Dumper($results);
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%
40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/1967d9c9-e53e-4037-803c-586dce6a6568%40googlegroups.com
>>
>>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googleg
Re: Example needed for Perl Search::Elasticsearch
Aahh.. newbie mistake ! I didn't realise the results were limited by
default. ;-)
Thanks !
On Wednesday, 13 August 2014 12:09:43 UTC+1, Jörg Prante wrote:
>
> A reason may be that your result set size is too small for containing both
> severity values. You could either try a larger result set size, or boost
> the "info" clause so you get docs with "info" before "notice.
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:51 PM, Log Muncher > wrote:
>
>> Well the the Perl module certainly doesn't complain about the syntax, but
>> it stil doesn't manage to output anything other than the "notice" severity ?
>>
>> $ perl test.pl | fgrep "severity"
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>>
>>
>> $ cat test.pl
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>>
>> #match => { severity => 'notice'}
>>
>> "bool" => {
>> "should" => [
>> {match => { severity => 'notice'}},
>> {match => { severity => 'info'}}
>> ]
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>>
>>> Try this to search "notice" or "info" severity.
>>>
>>> my $results = $e->search(
>>>index => 'logstash-2014.08.13',
>>>body => {
>>>query => {
>>> "bool" => {
>>> "should" => [
>>> { match => { severity => 'notice'} },
>>> { match => { severity => 'info'} }
>>> ]
>>> }
>>> }
>>>}
>>> );
>>>
>>>
>>> Jörg
>>>
>>>
>>> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
>>> wrote:
>>>
Hi,
Simple question, but there seems to be a lack of detailed examples for
using the otherwise very useful Search::Elasticsearch CPAN module !
I'm getting syslog data into elasticsearch via fluentd.
What I'd like to do now is run a perl search that will give me results
for notice, emerg and crit events. As a test (seeing as I don't get many
emerg/crit events !), I've tried the below, but it only seems to pick up
notice events and doesn't return any info events !
Help welcome !
Thanks.
Tim
#!/usr/bin/perl
use 5.014;
use strict;
use warnings;
use autodie;
use Data::Dumper;
use Search::Elasticsearch;
my $e = Search::Elasticsearch->new();
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
"bool" => {
"must" => {match => { severity => 'notice'},match
=> { severity => 'info'}}
}
}
}
);
print Dumper($results);
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%
40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/1967d9c9-e53e-4037-803c-586dce6a6568%40googlegroups.com
>>
>>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgi
Re: Example needed for Perl Search::Elasticsearch
A reason may be that your result set size is too small for containing both
severity values. You could either try a larger result set size, or boost
the "info" clause so you get docs with "info" before "notice.
Jörg
On Wed, Aug 13, 2014 at 12:51 PM, Log Muncher
wrote:
> Well the the Perl module certainly doesn't complain about the syntax, but
> it stil doesn't manage to output anything other than the "notice" severity ?
>
> $ perl test.pl | fgrep "severity"
> 'severity' => 'notice'
> 'severity' => 'notice',
> 'severity' => 'notice',
> 'severity' => 'notice',
> 'severity' => 'notice',
> 'severity' => 'notice',
> 'severity' => 'notice'
> 'severity' => 'notice',
> 'severity' => 'notice',
> 'severity' => 'notice',
>
>
> $ cat test.pl
> #!/usr/bin/perl
>
> use 5.014;
> use strict;
> use warnings;
> use autodie;
>
> use Data::Dumper;
> use Search::Elasticsearch;
>
> my $e = Search::Elasticsearch->new();
>
> my $results = $e->search(
>index => 'logstash-2014.08.13',
>body => {
>query => {
>
> #match => { severity => 'notice'}
>
> "bool" => {
> "should" => [
> {match => { severity => 'notice'}},
> {match => { severity => 'info'}}
> ]
> }
> }
>}
> );
>
> print Dumper($results);
>
>
>
>
>
>
>
> On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>
>> Try this to search "notice" or "info" severity.
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>> "bool" => {
>> "should" => [
>> { match => { severity => 'notice'} },
>> { match => { severity => 'info'} }
>> ]
>> }
>> }
>>}
>> );
>>
>>
>> Jörg
>>
>>
>> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
>> wrote:
>>
>>> Hi,
>>>
>>> Simple question, but there seems to be a lack of detailed examples for
>>> using the otherwise very useful Search::Elasticsearch CPAN module !
>>>
>>> I'm getting syslog data into elasticsearch via fluentd.
>>>
>>> What I'd like to do now is run a perl search that will give me results
>>> for notice, emerg and crit events. As a test (seeing as I don't get many
>>> emerg/crit events !), I've tried the below, but it only seems to pick up
>>> notice events and doesn't return any info events !
>>>
>>> Help welcome !
>>>
>>> Thanks.
>>>
>>> Tim
>>>
>>> #!/usr/bin/perl
>>>
>>> use 5.014;
>>> use strict;
>>> use warnings;
>>> use autodie;
>>>
>>> use Data::Dumper;
>>> use Search::Elasticsearch;
>>>
>>> my $e = Search::Elasticsearch->new();
>>>
>>> my $results = $e->search(
>>>index => 'logstash-2014.08.13',
>>>body => {
>>>query => {
>>> "bool" => {
>>> "must" => {match => { severity => 'notice'},match
>>> => { severity => 'info'}}
>>> }
>>> }
>>>}
>>> );
>>>
>>> print Dumper($results);
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>>
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%
>>> 40googlegroups.com
>>>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1967d9c9-e53e-4037-803c-586dce6a6568%40googlegroups.com
>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGT62WWavGGDcB4gVkcW%2BdAF4jbYd81oTHiVPUw1ZXKQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: Example needed for Perl Search::Elasticsearch
Well the the Perl module certainly doesn't complain about the syntax, but
it stil doesn't manage to output anything other than the "notice" severity ?
$ perl test.pl | fgrep "severity"
'severity' => 'notice'
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice'
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
$ cat test.pl
#!/usr/bin/perl
use 5.014;
use strict;
use warnings;
use autodie;
use Data::Dumper;
use Search::Elasticsearch;
my $e = Search::Elasticsearch->new();
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
#match => { severity => 'notice'}
"bool" => {
"should" => [
{match => { severity => 'notice'}},
{match => { severity => 'info'}}
]
}
}
}
);
print Dumper($results);
On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>
> Try this to search "notice" or "info" severity.
>
> my $results = $e->search(
>index => 'logstash-2014.08.13',
>body => {
>query => {
> "bool" => {
> "should" => [
> { match => { severity => 'notice'} },
> { match => { severity => 'info'} }
> ]
> }
> }
>}
> );
>
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher > wrote:
>
>> Hi,
>>
>> Simple question, but there seems to be a lack of detailed examples for
>> using the otherwise very useful Search::Elasticsearch CPAN module !
>>
>> I'm getting syslog data into elasticsearch via fluentd.
>>
>> What I'd like to do now is run a perl search that will give me results
>> for notice, emerg and crit events. As a test (seeing as I don't get many
>> emerg/crit events !), I've tried the below, but it only seems to pick up
>> notice events and doesn't return any info events !
>>
>> Help welcome !
>>
>> Thanks.
>>
>> Tim
>>
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>> "bool" => {
>> "must" => {match => { severity => 'notice'},match
>> => { severity => 'info'}}
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%40googlegroups.com
>>
>>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1967d9c9-e53e-4037-803c-586dce6a6568%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Example needed for Perl Search::Elasticsearch
Try this to search "notice" or "info" severity.
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
"bool" => {
"should" => [
{ match => { severity => 'notice'} },
{ match => { severity => 'info'} }
]
}
}
}
);
Jörg
On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
wrote:
> Hi,
>
> Simple question, but there seems to be a lack of detailed examples for
> using the otherwise very useful Search::Elasticsearch CPAN module !
>
> I'm getting syslog data into elasticsearch via fluentd.
>
> What I'd like to do now is run a perl search that will give me results for
> notice, emerg and crit events. As a test (seeing as I don't get many
> emerg/crit events !), I've tried the below, but it only seems to pick up
> notice events and doesn't return any info events !
>
> Help welcome !
>
> Thanks.
>
> Tim
>
> #!/usr/bin/perl
>
> use 5.014;
> use strict;
> use warnings;
> use autodie;
>
> use Data::Dumper;
> use Search::Elasticsearch;
>
> my $e = Search::Elasticsearch->new();
>
> my $results = $e->search(
>index => 'logstash-2014.08.13',
>body => {
>query => {
> "bool" => {
> "must" => {match => { severity => 'notice'},match
> => { severity => 'info'}}
> }
> }
>}
> );
>
> print Dumper($results);
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%40googlegroups.com
>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFrb%3Dz246SNk4GGgbuZTSv-dh-GgPXdN%3DPOP1jhVhxZow%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
