Regex queries possible ?
Hi, One of my servers appears to be feeding nonsense into Fluentd which is then ending up in elastic search. Is it possible to use regex in queries ? The syslog message content is always the same they start with numbers followed by close bracket, etc. 123) 89) 203) Is there a way to do the equivalent of ^\d+) in a elastic search query ? Thanks ! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/19b17dc5-f188-4223-8d72-40732112814c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Example needed for Perl Search::Elasticsearch
Would this be the correct syntax ?
{match => { severity => {query=>'info',boost=>20}}}
Even with the agressive boost, I'm still getting "notice" as the
prioritised results ?
On Wednesday, 13 August 2014 12:09:43 UTC+1, Jörg Prante wrote:
>
> A reason may be that your result set size is too small for containing both
> severity values. You could either try a larger result set size, or boost
> the "info" clause so you get docs with "info" before "notice.
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:51 PM, Log Muncher > wrote:
>
>> Well the the Perl module certainly doesn't complain about the syntax, but
>> it stil doesn't manage to output anything other than the "notice" severity ?
>>
>> $ perl test.pl | fgrep "severity"
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>>
>>
>> $ cat test.pl
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>>
>> #match => { severity => 'notice'}
>>
>> "bool" => {
>> "should" => [
>> {match => { severity => 'notice'}},
>> {match => { severity => 'info'}}
>> ]
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>>
>>> Try this to search "notice" or "info" severity.
>>>
>>> my $results = $e->search(
>>>index => 'logstash-2014.08.13',
>>>body => {
>>>query => {
>>> "bool" => {
>>> "should" => [
>>> { match => { severity => 'notice'} },
>>> { match => { severity => 'info'} }
>>> ]
>>> }
>>> }
>>>}
>>> );
>>>
>>>
>>> Jörg
>>>
>>>
>>> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Simple question, but there seems to be a lack of detailed examples for
>>>> using the otherwise very useful Search::Elasticsearch CPAN module !
>>>>
>>>> I'm getting syslog data into elasticsearch via fluentd.
>>>>
>>>> What I'd like to do now is run a perl search that will give me results
>>>> for notice, emerg and crit events. As a test (seeing as I don't get many
>>>> emerg/crit events !), I've tried the below, but it only seems to pick up
>>>> notice events and doesn't return any info events !
>>>>
>>>> Help welcome !
>>>>
>>>> Thanks.
>>>>
>>>> Tim
>>>>
>>>> #!/usr/bin/perl
>>>>
>>>> use 5.014;
>>>> use strict;
>>>> use warnings;
>>>> use autodie;
>>>>
>>>> use Data::Dumper;
>>>> use Search::Elasticsearch;
>>>>
>>>> my $e = Search::Elasticsearch->new();
>>>>
>>>> my $results = $e->search(
>>>>index => 'logstash-2014.08.13',
>>>>body => {
>>>>query => {
>>>> "bool" => {
>>>> "must" => {match => { severity => 'notice'},match
>>
Re: Example needed for Perl Search::Elasticsearch
Aahh.. newbie mistake ! I didn't realise the results were limited by
default. ;-)
Thanks !
On Wednesday, 13 August 2014 12:09:43 UTC+1, Jörg Prante wrote:
>
> A reason may be that your result set size is too small for containing both
> severity values. You could either try a larger result set size, or boost
> the "info" clause so you get docs with "info" before "notice.
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:51 PM, Log Muncher > wrote:
>
>> Well the the Perl module certainly doesn't complain about the syntax, but
>> it stil doesn't manage to output anything other than the "notice" severity ?
>>
>> $ perl test.pl | fgrep "severity"
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice'
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>> 'severity' => 'notice',
>>
>>
>> $ cat test.pl
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>>
>> #match => { severity => 'notice'}
>>
>> "bool" => {
>> "should" => [
>> {match => { severity => 'notice'}},
>> {match => { severity => 'info'}}
>> ]
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>>
>>> Try this to search "notice" or "info" severity.
>>>
>>> my $results = $e->search(
>>>index => 'logstash-2014.08.13',
>>>body => {
>>>query => {
>>> "bool" => {
>>> "should" => [
>>> { match => { severity => 'notice'} },
>>> { match => { severity => 'info'} }
>>> ]
>>> }
>>> }
>>>}
>>> );
>>>
>>>
>>> Jörg
>>>
>>>
>>> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Simple question, but there seems to be a lack of detailed examples for
>>>> using the otherwise very useful Search::Elasticsearch CPAN module !
>>>>
>>>> I'm getting syslog data into elasticsearch via fluentd.
>>>>
>>>> What I'd like to do now is run a perl search that will give me results
>>>> for notice, emerg and crit events. As a test (seeing as I don't get many
>>>> emerg/crit events !), I've tried the below, but it only seems to pick up
>>>> notice events and doesn't return any info events !
>>>>
>>>> Help welcome !
>>>>
>>>> Thanks.
>>>>
>>>> Tim
>>>>
>>>> #!/usr/bin/perl
>>>>
>>>> use 5.014;
>>>> use strict;
>>>> use warnings;
>>>> use autodie;
>>>>
>>>> use Data::Dumper;
>>>> use Search::Elasticsearch;
>>>>
>>>> my $e = Search::Elasticsearch->new();
>>>>
>>>> my $results = $e->search(
>>>>index => 'logstash-2014.08.13',
>>>>body => {
>>>>query => {
>>>> "bool" => {
>>>> "must" => {match => { severity => 'notice'},match
>>>> => { severity => 'info'}}
>>>> }
>>>> }
>&
Re: Example needed for Perl Search::Elasticsearch
Well the the Perl module certainly doesn't complain about the syntax, but
it stil doesn't manage to output anything other than the "notice" severity ?
$ perl test.pl | fgrep "severity"
'severity' => 'notice'
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice'
'severity' => 'notice',
'severity' => 'notice',
'severity' => 'notice',
$ cat test.pl
#!/usr/bin/perl
use 5.014;
use strict;
use warnings;
use autodie;
use Data::Dumper;
use Search::Elasticsearch;
my $e = Search::Elasticsearch->new();
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
#match => { severity => 'notice'}
"bool" => {
"should" => [
{match => { severity => 'notice'}},
{match => { severity => 'info'}}
]
}
}
}
);
print Dumper($results);
On Wednesday, 13 August 2014 11:40:42 UTC+1, Jörg Prante wrote:
>
> Try this to search "notice" or "info" severity.
>
> my $results = $e->search(
> index => 'logstash-2014.08.13',
>body => {
>query => {
> "bool" => {
> "should" => [
> { match => { severity => 'notice'} },
> { match => { severity => 'info'} }
> ]
> }
> }
>}
> );
>
>
> Jörg
>
>
> On Wed, Aug 13, 2014 at 12:01 PM, Log Muncher > wrote:
>
>> Hi,
>>
>> Simple question, but there seems to be a lack of detailed examples for
>> using the otherwise very useful Search::Elasticsearch CPAN module !
>>
>> I'm getting syslog data into elasticsearch via fluentd.
>>
>> What I'd like to do now is run a perl search that will give me results
>> for notice, emerg and crit events. As a test (seeing as I don't get many
>> emerg/crit events !), I've tried the below, but it only seems to pick up
>> notice events and doesn't return any info events !
>>
>> Help welcome !
>>
>> Thanks.
>>
>> Tim
>>
>> #!/usr/bin/perl
>>
>> use 5.014;
>> use strict;
>> use warnings;
>> use autodie;
>>
>> use Data::Dumper;
>> use Search::Elasticsearch;
>>
>> my $e = Search::Elasticsearch->new();
>>
>> my $results = $e->search(
>>index => 'logstash-2014.08.13',
>>body => {
>>query => {
>> "bool" => {
>> "must" => {match => { severity => 'notice'},match
>> => { severity => 'info'}}
>> }
>> }
>>}
>> );
>>
>> print Dumper($results);
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1967d9c9-e53e-4037-803c-586dce6a6568%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Example needed for Perl Search::Elasticsearch
Hi,
Simple question, but there seems to be a lack of detailed examples for
using the otherwise very useful Search::Elasticsearch CPAN module !
I'm getting syslog data into elasticsearch via fluentd.
What I'd like to do now is run a perl search that will give me results for
notice, emerg and crit events. As a test (seeing as I don't get many
emerg/crit events !), I've tried the below, but it only seems to pick up
notice events and doesn't return any info events !
Help welcome !
Thanks.
Tim
#!/usr/bin/perl
use 5.014;
use strict;
use warnings;
use autodie;
use Data::Dumper;
use Search::Elasticsearch;
my $e = Search::Elasticsearch->new();
my $results = $e->search(
index => 'logstash-2014.08.13',
body => {
query => {
"bool" => {
"must" => {match => { severity => 'notice'},match =>
{ severity => 'info'}}
}
}
}
);
print Dumper($results);
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/42e60034-655f-46ca-979e-308b0e7532e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
