Re: is it possible to reference query/filer data in your aggregations

[Benoit Gagnon]

I suppose you can make a bucket for each value of _id that matches your 
query; you should then have one bucket per document, on which you can 
define sub-aggregations (only metrics at this granularity) -- but really at 
that point, script_fields would be equivalent.

It sounds like you want to get aggregations at the global scope based on 
values from documents that match a certain query.
This is not possible -- you will have to do multiple queries.

On Wednesday, September 10, 2014 9:34:13 AM UTC-4, Mindaugas Verdingovas 
wrote:
>
> In other words can you create a bucket for each document in your query and 
> then add some further aggregations using values of that document.
>
> I also asked for the same question in detail on stack overflow here is a 
> link if you need more details to the question 
> http://stackoverflow.com/questions/25720027/elasticsearch-aggregations-is-it-possible-to-reference-filter-query-data-in-agg
>  
> 
>
> if you don't understand something please ask me I'll try my best to 
> explain.
>
> if it's not possible could someone just tell me that, because I've spend a 
> lot of hours trying to find a way of achieving this without any luck.
>
> or maybe you have some other ideas on how I should represent my data to 
> meet my demands?
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/28ca3e77-cbc0-4f37-9d6b-16e1cf776309%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Preparing for ElasticSearch in production

[Benoit Gagnon]

If you want the ability to do maintenance on your cluster without downtime, 
it will require at least two nodes.
Even if you don't care about replication, and you don't expect your servers 
to fail, you could consider software/hardware upgrades as a form of failure 
tolerance.
If spawning an occasional node to join your cluster is easy (with cloud 
virtualization for example), then it can possibly cover these maintenance 
scenarios.

On Thursday, September 11, 2014 4:47:43 AM UTC-4, David Pilato wrote:
>
> Sure. If you don't care at the beginning of your production about 
> replication (and failover), that's perfectly fine.
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet  | @elasticsearchfr 
> 
>
>
> Le 11 septembre 2014 à 09:17:44, Simon Forsberg (simon...@gmail.com 
> ) a écrit:
>
> Hello,
>
> I am wondering if it's a valid approach to start with a single-noded 
> ElasticSearch cluster and then scale out when needed?
>
> This would of course involve a proper shard management.
>
> Thanks,
>
> --
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/6826063f-c7a2-4782-a260-23881cde0c8d%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/af100a6d-91ca-4e71-a56c-ff05983292c8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Pagination on unique data

[Benoit Gagnon]

There is no support for pagination for terms aggregations.
The official reason seems to be that it is "tricky to implement"; see issue 
#4915  which is 
now unfortunately closed.

So getting paginated terms ordered by count does not seem possible at this 
point.
You could, however, order them alphabetically (by term), and apply filtering 

 in 
a clever way to retrieve sequences of terms.
As you point out, a cardinality 

 query 
beforehand could inform your paging strategy.

Algorithm assuming A-Z letters for a well distributed collection of terms:
- determine cardinality based on the first character (26 buckets)
- if the size of a bucket exceeds a certain limit, repeat with the second 
character for that bucket (26 sub buckets)
- the prefix of the term (1 or more letters) then becomes your paging 
mechanism

How this translates in performance, I have no idea.
It will save on transfers from ES for sure, but it might not perform as 
well as simply fetching every term and doing the paging in the server 
application layer.

Personally, I would love to see pagination support in Elasticsearch, even 
if there is a performance penalty.
It seems much better than risking flooding a naive client or server with 
too many terms at once.

On Thursday, September 11, 2014 2:48:30 PM UTC-4, jigish thakar wrote:
>
> Hey Guys,
> I am building some Logging and monitoring product for my employer and 
> using ES as backend.
> now finding unique value of each/any attribute is core part of business 
> logic I have in hand.
>
> lets say I want unique dst_ip, to achieve that,
> - I have used "index":"not_analyzed" for selected fields
> - Api used to get unique count 
>http://127.0.0.1:9200/es-server/Events/_search -d 
> '{"aggs":{"dst_ip_count":{"cardinality":{"field":"dst_ip"}}},"size":0}'
> - Api used to fetch those values
>http://127.0.0.1:9200/es-server/Events/_search -d 
> '{"fields":["dst_ip"],"facets":{"terms":{"terms":{"field":"dst_ip","size":1116,"order":"count"}}},"size":1116}'
>
>   here 1116 is received from first API. now here the count is very small 
> but in production environment this count goes greater then 2lakh. which 
> results in slow query response.
>
> do we have any other way to fetch such values with pagination inbuild like 
> we have in search query with size and from.
>
> Please suggest, thanks in advance.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f4e93af8-27b1-45a8-b650-ee2311c83066%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How Recover mistaken delete index？

[Benoit Gagnon]

To create and restore backups, use the Snapshots And Restore 

 APIs.

On Wednesday, July 23, 2014 10:58:23 AM UTC-4, 闫旭 wrote:
>
> Dear All！
>
>
> How can i recover deleted indux?  or how to backup the index?
>
>
> Thanks && Best Regard!
>
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4c67cc05-fe5e-4649-9fdf-5be7b48d8346%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Notifications for a query

[Benoit Gagnon]

If you have a single entry point to add documents, then yes, you can 
percolate them as they come in.
ES will answer your percolation request with a list of queries that match 
the percolated document.
How you turn this response into notifications and actions is up to you to 
implement.

On Monday, July 21, 2014 11:28:31 AM UTC-4, P lva wrote:
>
> Hello Everyone, 
>
> Started working with Elasticsearch recently.
> Just wanted to know if there's any way of being notified when a document 
> matches a query. (essentially create a monitoring system)
> Can I use percolator to do this ?
>
> Thanks
>  
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/efbc2ecc-8b26-40ce-96ea-a888058b0015%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.