Re: Problems with aggregation query
Assuming the source_ip field is unanalyzed, your query could be something
like this:
{
'query': {
'terms': {
'port': [ 23 ]
}
},
'size': 0,
'aggs': {
'counts_per_ip': {
'terms': {
'field': 'source_ip'
}
}
}
}
On Saturday, February 14, 2015 at 2:40:17 AM UTC+1, Ian Hayes wrote:
>
> I'm having a difficult time trying to get what is a simple SQL query to
> work with Elasticsearch. I can query, I can get counts, but can't seem to
> get the aggregation to work.
>
> For a simple SQL query: select source_ip, count(*) from my_table where
> destination port='23' group by source_ip;
>
> I've spent the day looking over documentation and examples, but just about
> all of the examples I have seen don't seem to utilize a filter and
> aggregate the entire index.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a51a023a-c4e5-4e69-b26e-427ccba748c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Problems with aggregation query
I'm having a difficult time trying to get what is a simple SQL query to work with Elasticsearch. I can query, I can get counts, but can't seem to get the aggregation to work. For a simple SQL query: select source_ip, count(*) from my_table where destination port='23' group by source_ip; I've spent the day looking over documentation and examples, but just about all of the examples I have seen don't seem to utilize a filter and aggregate the entire index. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d049ac60-cb1e-4d66-9e94-8b8f9093b9c2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Equivalent aggregation query to terms facet
>
> I think I figured it out. Each aggs sections sub divides the data. First
> aggs buckets all the nested docs. Second aggs restricts the bucket to a
> filter. Third aggs run the terms facet on it.
>
"aggs": {
"foo": {
"nested": {
"path": "rdata"
},
"aggs": {
"bar": {
"filter": {
"term": {
"rdata.mapid": 37
}
},
"aggs": {
"foobar": {
"terms": {
"field": "rdata.value",
"size": 28654,
"order": {
"_count": "desc"
}
}
}
}
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e0b49cd3-95c9-4060-851a-739073140ea4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Equivalent aggregation query to terms facet
Hi all,
I'm using ES 1.3.2
My document structure
{
"mapids" : [mapid1, mapid2,mapid3],
"rdata" : [ {mapid1: value1},
{mapid2:value2},{mapid3:value3} ]
}
We are trying to migrate to the new aggregation framework and I'm trying to
rewrite terms facet which was
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"terms": {
"mapids": [
0,
77188
],
"execution": "and"
}
}
]
}
}
}
},
"facets": {
"facet1": {
"terms": {
"field": "value"
},
"nested": "rdata",
"facet_filter": {
"term": {
"mapid": "77188"
}
}
}
}
}
Which comes back with the buckets
- terms: [
- {
- term: 10
- count: 257998
}
- ...
-
- {
- term: 4
- count: 28477
}
]
I was trying to rewrite the same query using the aggregation framework but
it comes back Parse Failure [Found two aggregation type definitions in
[rdata]: [filter] and [nested]]]
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"terms": {
"mapids": [
0,
77188,
77200
],
"execution": "and"
}
}
]
}
}
}
},
"aggs": {
"rdata": {
"filter": {
"term": {
"mapid": "77188"
}
},
"nested": {
"path": "rdata"
},
"aggs": {
"rdata_value": {
"terms": {
"field": "value"
}
}
}
}
}
}
Any help is appreciated
Thanks,
Abhishek
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b7cc4b9b-7767-46b9-a8ab-ff4159b5f502%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
complex nested aggregation query based on time
Hello there, I am trying to write a rather complex aggregation Let's say my json documents contains the following fields: timestamp, username, subject The search should return documents where: - two identical "subject" fields, - by the same username, - within an interval of X minutes. Using nested aggregation I can group by username, and count the identical subjects (terms). However I can't find a way to also specify a time interval within the query. (the identical subjects should be within an interval of X minutes) All pointers are welcome. Thanks Christophe -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2f90d46f-2330-4a0f-8658-8cbdf6824415%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Aggregation query works with search, but not with msearch
I am trying to troubleshoot the following observation:
Following code works as expected:
Elasticsearch::Model.client.search search_type: 'count', index:
target_indices, body: query
Response:
{"took"=>2, "timed_out"=>false, "_shards"=>{"total"=>2, "successful"=>2,
"failed"=>0}, "hits"=>{"total"=>6, "max_score"=>0.0, "hits"=>[]},
"aggregations"=>{"recent"=>{"doc_count"=>3,
"searches"=>{"buckets"=>[{"key"=>"user-1", "doc_count"=>3}]
However, when using the above in an msearch, the response is not useful:
Elasticsearch::Model.client.msearch body: [{ search_type: 'count', index:
target_indices, search: query }]
Response:
{"responses"=>[{"took"=>0, "timed_out"=>false, "_shards"=>{"total"=>2,
"successful"=>2, "failed"=>0}, "hits"=>{"total"=>6, "max_score"=>0.0,
"hits"=>[]}}]}
---
What am I missing?
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fef8c41-7232-43e6-8632-9e2e5058240d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Aggregation query
Hello Ivan ,
This is expected.
Only the top N(size mentioned in aggregation) results are taken from each
shard before reducing the result.
Due this , the accuracy is not guaranteed but the order is guaranteed.
As a fix , you can use this to improve accuracy at the cost of memory -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html#CO16-1
Thanks
Vineeth
On Fri, Aug 15, 2014 at 9:16 PM, Ivan Stone wrote:
> When I run the following query on a 5 shard ES db I don't get accurate
> results. I have had to reduce the amount of shards on my ES server to 1 to
> get the accuracy I need? Has anyone had a similar issue?
>
> GET /incidents/_search?search_type=count
> {
> "query" : {
> "filtered" : {
> "filter" : {
> "bool" : {
> "must": {
> "range" : {
> "Date" : {
> "from" : "2014-08-05T00:00:00.000Z",
> "to" : "2014-08-06T00:00:00.000Z"
> }
> }
> },
> "must": {
> "exists" : { "field" : "AttackTypes" }
> }
> }
> }
> }
> },
> "aggs": {
> "by_attackType" : {
> "terms": {
> "field": "AttackTypes",
> "order": {
> "_count": "desc"
> },
> "shard_size": 0,
> "size": 10
> },
> "aggs": {
> "by_perpertrator" : {
> "terms": {
> "field": "Perpetrators",
> "order": {
> "_term": "asc"
> },
> "min_doc_count": 0,
> "shard_size": 0,
> "size": 0
> }
> }
> }
> }
> }
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/cf1f5980-717d-4da9-b55b-3262a284e144%40googlegroups.com
>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m%3DTcrAtwCO7ovfjRgnO%2BbmxP8DEK%2BRmUvEsyE-1taYcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Aggregation query
When I run the following query on a 5 shard ES db I don't get accurate
results. I have had to reduce the amount of shards on my ES server to 1 to
get the accuracy I need? Has anyone had a similar issue?
GET /incidents/_search?search_type=count
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must": {
"range" : {
"Date" : {
"from" : "2014-08-05T00:00:00.000Z",
"to" : "2014-08-06T00:00:00.000Z"
}
}
},
"must": {
"exists" : { "field" : "AttackTypes" }
}
}
}
}
},
"aggs": {
"by_attackType" : {
"terms": {
"field": "AttackTypes",
"order": {
"_count": "desc"
},
"shard_size": 0,
"size": 10
},
"aggs": {
"by_perpertrator" : {
"terms": {
"field": "Perpetrators",
"order": {
"_term": "asc"
},
"min_doc_count": 0,
"shard_size": 0,
"size": 0
}
}
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cf1f5980-717d-4da9-b55b-3262a284e144%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
