Undo delete feature
Hi All, I like to if ElasticSearch has any undo delete operation. So any document deleted using delete api can be recovered until it is deleted from disk using optimize api flag only_expunge_deletes. If it is not there I like to know what are the challenges to develop that as we already have document in index and just marked it as delete. Thanks, Pulkit Agrawal Product Analyst MckinseyCompany -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMVa6up86Rphmh7C7fNVBANgXMvi8ruV7GD78iVHhzw1tuWQCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Heap / GC Issues
You are on the right track and you found already the answer to your question, examine your queries. They seem to be cached and eat your heap. http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html Jörg On Fri, Jul 18, 2014 at 5:09 PM, Ned Campion nedcamp...@gmail.com wrote: We are doing a lot of bool conditions in our queries, so that may be a factor in the hefty filter cache. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE06%2BdSouhX1-3xnmrenTDJbCzDOqD4Ma_NxCkFh5QQJw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: n:m lookup filter
Yes, I think this is somehow related to Matt's Join Filter https://github.com/elasticsearch/elasticsearch/pull/3278 Jörg On Sat, Jul 19, 2014 at 4:24 AM, Don Clore cloredo...@gmail.com wrote: I am pretty sure this is not supported, but it'd be great to explicit confirmation/denial. Sodocument types A and B, where there's an N:M relationship between A and B, and document type B has a list of the document A instances that relate to it. More concretely A == a sports Player data type, and B is a set of new stories. The Story type has a list of the ids of Players that the story is about/related to. SoI know the terms lookup filter allows one to use a single document as the source of the terms for the lookup. What we'd like to be able to do is expose a faceted/aggregations-based UI to the user that allows her to perform a variety of filtering operations on Players over a fairly extensive set of criteria, and then have the resulting set of Player document ids serve as the lookup into the Story stories, i.e., get all the stories that relate to the Player result set. Obviously, we'd ideally like to do this in a single query, or failing that, have some reasonably efficient way to issue the two query/filters (passing a large result set of ids over the wire seems like a bad idea; I'm new to ES, but...this kind of thing was never great with Solr). One idea I had (perhaps half-baked) was to create a PlayerResultSet type, with an id deterministically fashioned from the query/filter predicates such that the same user filtering action would result in the same PlayerResultSet id each time; we'd issue a terms lookup filter request using the PlayerResultSet id, if it fails because the PlayerResultSet document doesn't exist, then we'd have to issue the filter for the Players, construct a PlayerResultSet doc and index it, and query for the Stories that have those Player Ids; not sure if it would be worse to issue all the ids in a query, or index the PlayerResultSet doc with Refresh==true (or issue the query and queue up the PlayerResultSet doc for later indexing, or whatever). The Player data should be fairly static; we could delete the documents and recreate them each time we refresh Player data. Ok, that sounds pretty awful, I'm hoping someone has a less Rube-Goldberg approach; obviously, I'm sort of building in my filter query caching mechanism, hopefully something like this can be more easily achieved with the built-in filter caching. thanks for any insights, Don -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/91919a48-0892-4878-890b-e14c67fd40b5%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/91919a48-0892-4878-890b-e14c67fd40b5%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEMzKNuuBvuTt5XTLN6gMuePrVDP-%3DyjyQ0pWnPJ5NK9w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Certified platforms for ElasticSearch
Not sure how you could find that Linux and Windows are certified or if there are certifications at all. The platform on which Elasticsearch runs is server-side Java so this is probably your question. I for myself run Elasticsearch on Java 7 and Java 8, on Red Hat Linux Enterprise 6, Mac OS X 10.9, Solaris 10. I know that people were successful in running Elasticsearch on Windows 7 and FreeBSD, but I do not use these systems. AIX is more painful, but this is not necessarily related to Elasticsearch https://github.com/elasticsearch/elasticsearch/issues/1805 Flawless operations are most predictable with the Oracle distribution of JDK 7 and 8. Use always the most recent releases, in older versions the probability of bugs is higher, especially related to Lucene. Lucene and OpenJDK teams are working together to find bugs before releases. OpenJDK 7, the reference implementation, which is included in recent Linux distributions, should work equivalently well. If you run Elasticsearch 1.2.0, OpenJDK 6 should be avoided, some versions have annoying bugs. It is also possible to use Server JRE for Elasticsearch. This is a tailored Java distribution with tools for JVM monitoring, but without browser plugin integration or graphics display. For Java 8, you need Elasticsearch = 1.0.4, 1.1.2, 1.2.0 if MVEL scripting is used https://github.com/elasticsearch/elasticsearch/issues/5877 Jörg On Wed, Jul 16, 2014 at 8:27 AM, bsa shashi.b.an...@gmail.com wrote: Hello, Can somebody provide the list of platforms on which ElasticSearch is certified, besides Linux and Windows? -bsa -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/98b9d852-a7ef-48c6-bb01-bd5bfdd5a7fb%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/98b9d852-a7ef-48c6-bb01-bd5bfdd5a7fb%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFzHJ_%3DUPCbybiZSd4jjTDCkK1rLzNTGE-Uoa65nAapdA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Undo delete feature
Hello Pulkit , The best option i can suggest would be to take snapshot of the index before the delete opertaion. And later retrieve the document that you need. Else you might need to look at lucene level. I believe lucene has features to tag a version and restoring to it later. Thanks Vineeth On Sat, Jul 19, 2014 at 11:51 AM, Pulkit Agrawal pulkitdot...@gmail.com wrote: Hi All, I like to if ElasticSearch has any undo delete operation. So any document deleted using delete api can be recovered until it is deleted from disk using optimize api flag only_expunge_deletes. If it is not there I like to know what are the challenges to develop that as we already have document in index and just marked it as delete. Thanks, Pulkit Agrawal Product Analyst MckinseyCompany -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMVa6up86Rphmh7C7fNVBANgXMvi8ruV7GD78iVHhzw1tuWQCw%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAMVa6up86Rphmh7C7fNVBANgXMvi8ruV7GD78iVHhzw1tuWQCw%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5nPyRYrPOZjO4kvWT1AAT%2BOvjDs%2BuPoeZhSh8C%3D9LV9pw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Possibly to use current date as synonim?
http://stackoverflow.com/questions/24841835/possibly-to-use-current-date-as-synonim# Possibly to use current date as synonim ? For example for query latest/breaking news I want to get last news in search. How to do it ? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5da5e274-c203-4542-8b10-1a06973764bc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Log4j2 Elasticsearch appender
I'll try it as soon as I can! thanks, Alfredo :-) Il giorno venerdì 18 luglio 2014 10:08:14 UTC+2, Jörg Prante ha scritto: Hi, I released a Log4j2 Elasticsearch appender https://github.com/jprante/log4j2-elasticsearch in the hope it is useful. Best, Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dce481d9-ac3e-4fd0-aaba-3a4c69d07d34%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Heap / GC Issues
Your filter cache is only taking up 3GB of the heap, which fits with the default limit of 10% of heap space. So the filter cache is not at fault here. I would look at the two usual suspects: * field data - how much space is this consuming? Try: curl 'localhost:9200/_nodes/stats/indices/fielddata?fields=*pretty' * swap - is it completely disabled? The swap is a common gotcha. If you have any data in swap it will slow GCs down terribly, and give you the scenario that you are seeing. See the docs for how to disable swap: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html#setup-configuration-memory (i prefer disabling swap completely, rather than relying on mlockall or swappiness) On 19 July 2014 09:06, joergpra...@gmail.com joergpra...@gmail.com wrote: You are on the right track and you found already the answer to your question, examine your queries. They seem to be cached and eat your heap. http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html Jörg On Fri, Jul 18, 2014 at 5:09 PM, Ned Campion nedcamp...@gmail.com wrote: We are doing a lot of bool conditions in our queries, so that may be a factor in the hefty filter cache. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE06%2BdSouhX1-3xnmrenTDJbCzDOqD4Ma_NxCkFh5QQJw%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE06%2BdSouhX1-3xnmrenTDJbCzDOqD4Ma_NxCkFh5QQJw%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRnLmMMVxFppy%3D3BG%3DD1afTfXpeeOni%3DRkJNunKkTfh0A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[ANN] JDBC river/feeder plugin 1.2.2.0
Hi, here is a new release of JDBC river/feeder plugin for Elasticsearch https://github.com/jprante/elasticsearch-river-jdbc/releases/tag/1.2.2.0 Highlights: - update to Elasticsearch 1.2.2 - more reliable bulk indexing - properly handle SQL insert/update/select statement types - dropping empty maps after ignoring null values - overlap time for column strategy - option for creating time-window based index names See also README at https://github.com/jprante/elasticsearch-river-jdbc As always, feedback/contributions are most welcome! Best, Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF_8xRHYGGqoggDkWx34pzatjM8xRsAL2pPg9u2J221nQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Parsing exceptions
Is there any easy way to get just the relevant error messages out of an
exception?
I know there are multiple shards and each one can potentially return a
different error message but 99% of the time, the error is because I've made
a mistake.
This means I get the same message 50+ times in a complete soup of brackets
and braces.
The best solution I have at the moment is to scroll to the end of the error
and read backwards.
Is there any way to parse the error messages into something useful? As you
can see, it's not valid json...
SearchPhaseExecutionException[
Failedtoexecutephase[
query
],
allshardsfailed;shardFailures{
[
kieUEJ4uS_eoaBV3cBz3Rw
][
elmail
][
7
]: SearchParseException[
[
elmail
][
7
]: from[
-1
],
size[
0
]: ParseFailure[
Failedtoparsesource[
{
size: 0,
aggs: {
senders: {
terms: {
field: from.emailAddress,
size: 50
},
aggs: {
senderDate: {
date_histogram: {
field: date,
interval: quarter
}
}
}
}
},
filter: {
range: {
date: {
gte: -3year
}
}
}
}
]
]
];nested: ElasticsearchParseException[
failedtoparsedatefield[
-3year
...Snip...
}{
[
kieUEJ4uS_eoaBV3cBz3Rw
][
elmail
][
0
]: SearchParseException[
[
elmail
][
0
]: from[
-1
],
size[
0
]: ParseFailure[
Failedtoparsesource[
{
size: 0,
aggs: {
senders: {
terms: {
field: from.emailAddress,
size: 50
},
aggs: {
senderDate: {
date_histogram: {
field: date,
interval: quarter
}
}
}
}
},
filter: {
range: {
date: {
gte: -3year
}
}
}
}
]
]
];nested: ElasticsearchParseException[
failedtoparsedatefield[
-3year
],
triedbothdateformat[
dateOptionalTime
],
andtimestampnumber
];nested: IllegalArgumentException[
Invalidformat: -3yearismalformedatyear
];
}
]
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/669d35b5-c489-4099-b21f-e96a60ab4beb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
