I am wondering when Elasticsearch will have issues with the number of
indexes. For example is 10 a good number? 100? 1000? 1? etc.
I would like to break up the indexes as much as possible and make use of
aliases for searching the data of interest, but I don't want to create so
many
Did you get better writes?
What sort of storage are you on, did you measure before and after, are you
reaching I/O limits?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 9 October 2014 17:33, Kang-min Liu
Any chance you could try elasticsearch 1.3.4?
David
Le 9 oct. 2014 à 07:51, Frédéric de Villamil f...@botify.com a écrit :
Hello,
I'v been facing a problem on one of my ES nodes for a few days I can't
explain myself. The machine was recently rebooted and I seem to have lost
something.
How much data is in your cluster?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 9 October 2014 18:25, David Pilato da...@pilato.fr wrote:
Any chance you could try elasticsearch 1.3.4?
David
Le 9 oct. 2014 à
Hi,
I have a lot of data in my ES cluster. I'm using the _all field for general
search in my documents, but I recently found out that the standard analyzer
doesn't tokenize words with dots inside (ex: www.google.com - not found
when searching for google).
So I want to replace the standard
Hello,
upgrading is not yet planned.
The cluster is less than 100GB, and this node is a master one, no data,
only queries and very little trafic.
My concern is seeing the process taking 6GB RAM despite the JVM being
capped at 512M whithout ever triggering the garbage collector. Also, the
My application write bulk updates the whole time in an Elasticsearch
index (index size: ~200,000 docs, 35 MB, shards: 3*2; segment count ~35).
My cluster has 3 nodes with each 32 GB RAM, ES_HEAP_SIZE=16g,
Elasticsearch V. 1.3.4
I am using `index.merge.scheduler.max_thread_count: 1` as I am using
How many index / delete or update requests are you bundeling in a single
bulk api call?
On 9 October 2014 10:19, Bernhard Berger bernhardberger3...@gmail.com
wrote:
My application write bulk updates the whole time in an Elasticsearch index
(index size: ~200,000 docs, 35 MB, shards: 3*2;
Heya,
We are pleased to announce the release of the Elasticsearch Wikipedia River
plugin, version 2.4.0.
The Wikipedia River plugin allows index wikipedia.
https://github.com/elasticsearch/elasticsearch-river-wikipedia/
Release Notes - elasticsearch-river-wikipedia - Version 2.4.0
Update:
We have the same problem is reading .svn/text-base/logging.yml.text-base
and fails to setup logging.
I think the loading logic should be a little more restrictive. Loading
everything that starts with logging. including files in hidden
directories, is a little bit too much.
I fixed it with
On 09.10.14 11:23, Martijn v Groningen wrote:
How many index / delete or update requests are you bundeling in a
single bulk api call?
I use a max bulk size of 1 MB (around 2000 docs/bulk); most of the
requests are updates with a small Groovy script which increase some
field values.
I will
First you should not use _all but prefer copy_to feature which gives more
flexibility. See:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#copy-to
That said, changing an analyzer one by another requires to reindex your
documents.
So create a new
Are you using the same Bulk object each time or a new instance for each
iteration?
I have seen in the past in some Java code people using the same bulk object.
So the bulk started with 2k docs. Next iteration it was having 4k docs, … and
so on.
Not sure it's your concern here but it worths
Hi Daniel,
When you restart the cluster, do you also wipe all content? The marvel
license should persist once entered but if you clean the data folder, that
will go away as well.
Cheers,
Boaz
On Wednesday, October 8, 2014 6:12:19 AM UTC+2, Daniel Schonfeld wrote:
Hello,
We have recently
Hai,
i also have the same problem with couchbase elasticsearch replication.
It is possible to replicate all sample bucket that with couchbase, but i
can not able to make replication with a new bucket.
*versions*
*couchbase 3.0.1elasticsearch 1.3 couchbase elasticsearch plugin 2.0.0*
The problem was that it was not an apostrophe, but an opening single quote.
Have increased editor font size to address this issue.
On Tuesday, October 7, 2014 8:00:13 PM UTC+1, Ivan Brusic wrote:
What type of query are you using? Perhaps the query you are using is not
using the same
On 09.10.14 11:39, David Pilato wrote:
Are you using the same Bulk object each time or a new instance for
each iteration?
I have seen in the past in some Java code people using the same bulk
object.
So the bulk started with 2k docs. Next iteration it was having 4k
docs, … and so on.
Not
Hi Guys,
We're about to use the Term Lookup mechanism
(http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)
to do a term filter potentially on a big amount of terms, let's say 5'000
terms containing each of them 20 characters.
To design the
What gives Nodes Info output? Could you gist it?
--
David Pilato | Technical Advocate | elasticsearch.com
david.pil...@elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs
Le 9 octobre 2014 à 11:59:02, Bernhard Berger (bernhardberger3...@gmail.com) a
écrit:
On 09.10.14 11:39,
On 09.10.14 12:38, David Pilato wrote:
What gives Nodes Info output? Could you gist it?
https://gist.github.com/Hocdoc/42da3712d91ec976e39d
(Now running with Java 8 instead of Java 7)
Unfortunately it isn't possible to reach the node again after the
OutOfMemory error and get the actual node
Looks good. I was just checking that JVM memory settings have been taken into
account.
When you restart your node and monitor nodes stats don't you see something
strange?
Could you gist your node stats?
--
David Pilato | Technical Advocate | elasticsearch.com
david.pil...@elasticsearch.com
Hello, I need to find some phrases using regexp, f.e.,
[0-9]{5}\s[\S]{5}[0-6]{2}. But for elastic this is two different terms.
I can find [0-9]{5} and [\S]{5}[0-6]{2} but not together.
I have no idea how I can resolve this.
--
You received this message because you are subscribed to the Google
On 09.10.14 12:55, David Pilato wrote:
Looks good. I was just checking that JVM memory settings have been
taken into account.
When you restart your node and monitor nodes stats don't you see
something strange?
Nop, just as a normal start.
Could you gist your node stats?
I don't see anything obvious.
Did you compare with other nodes stats?
It sounds like you have enough memory for bulk operations.
May be you should try to reduce the bulk size and see how it goes?
--
David Pilato | Technical Advocate | elasticsearch.com
david.pil...@elasticsearch.com
@dadoonet
I don't like that much the TransportClient. I prefer the NodeClient,
because is cluster aware, it doesn't have to do a double hop when sending
requests, it will directly call the corresponding data node. I know that
the node client does also balancing, which I don't think is the case for
the
On 09.10.14 13:12, David Pilato wrote:
I don't see anything obvious.
Did you compare with other nodes stats?
It sounds like you have enough memory for bulk operations.
May be you should try to reduce the bulk size and see how it goes?
I will try it with Java 8 and a reduced bulk size and will
Hi again,
I've tried to create two types and fill them with data that mimic my own
data, both with doc_values for all non-analyzed string and numeric fields
and without doc_values. My additional question is to know if there are some
overhead of mixing both doc_values and non-doc values inside
Got a similiar issue with the beta, getting the *This version of Kibana
requires at least Elasticsearch 1.4.0.Beta1*.
I've got kibana and elasticsearch(ie. http://localhost:9200) running on
the same box
It seems from what I can see in the logs, that Kibana is pushing the
requests to
While trying to upgrade from ES 1.2.2 to 1.3.4, we have noticed that since
Smart Chinese Analysis plugin release 2.3.0 *smartcn_sentence* and
*smarcn_word* have been deprecated as a result of changes in Lucene.
At the moment we do have dozens of indices which already use those classes
in their
But if I'm limiting the results by size(), I need to order to have specific
results like the 10 bests order or is there another way?
Am Mittwoch, 10. September 2014 22:55:33 UTC+2 schrieb Adrien Grand:
The range aggregation indeed does not support ordering. I believe that is
something that
I have seen with lucene CheckIndex tool that the shard that is failing is
corrupt
How could it have happened? and, how could it be fixed?
El miércoles, 8 de octubre de 2014 11:08:35 UTC+2, Daniel Alonso Heras
escribió:
Hi all,
I'm using elasticsearch 1.2.1 with java 1.7.0_60. I have
Hi,
I try to use the new spout added in the last version (2.1 snapshot) but I am
not sure to understand the aim of this spout.
As far as I know, a storm topology runs forever but an elasticsearch request
has a finite number of responses. If we use the storm spout EsSpout in a
topology, this
Same problem was reported here:
https://github.com/elasticsearch/elasticsearch-analysis-smartcn/pull/29
I'm going to reintroduce them and release a 2.3.1 and 2.4.1 updates.
--
David Pilato | Technical Advocate | elasticsearch.com
david.pil...@elasticsearch.com
@dadoonet | @elasticsearchfr |
I have an existing index for which the default ElasticSearch similarity is
used for all fields. I would like to update this index and set some other
type of similarity, like BM25. The query I tried is:
curl -XPOST 'http://localhost:9200/myindex/' -d
I just upgraded my elasticsearch from 1.2.1. I have 5 node cluster with 3
master 3 data nodes. (one master servers data)
During the upgrade, I restarted my whole cluster. (Till this time, my
cluster was very stable). The cluster status was red for almost 12 hours.
Since unassigned_shards was
I am having this problem as well using 2.2.1 (ES 1.2.x) - have you been able
to add the fix to a recent version of the plugin? If so, I'm assuming I
would also need to upgrade my ES version to work with the updated plugin
version?
Thanks, this one has been making me crazy - so relieved to find
Heya,
We are pleased to announce the release of the Elasticsearch Smart Chinese
Analysis plugin, version 2.3.1
Smart Chinese Analysis plugin integrates Lucene Smart Chinese analysis module
into elasticsearch..
Release Notes - Version 2.3.1
Fix
[29] - Add smartcn_word and smartcn_sentence
It is perfectly fine to have some fields that have doc values and other
fields that don't.
Doc values are indeed faster in 1.4, especially on numeric fields. Using
doc values doesn't change the memory usage of aggregations. However, it
will change the memory usage for field data. It's a bit
I have a general question.. Say I have records in a parent/child type
relationship where doc X is a Parent of Doc Y which and for each Y there
are N number of documents...
What would be the best way to get all X documents where the number of Y
documents is greater than Z?
--
You received
Just switched to 2.3.1, that was just too fast.
Merci bien!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view
Heya,
We are pleased to announce the release of the Elasticsearch Smart Chinese
Analysis plugin, version 2.4.1
Smart Chinese Analysis plugin integrates Lucene Smart Chinese analysis module
into elasticsearch..
Release Notes - Version 2.4.1
Fix
[29] - Add smartcn_word and smartcn_sentence
Anyone?
On Tue, Oct 7, 2014 at 6:48 PM, Hermano Cabral
hermanocab...@creactive.com.br wrote:
Yeah, thanks for the idea but unfortunately that's not really an option
for me as I have no control over how the data gets sent to ES.
On Tue, Oct 7, 2014 at 6:05 PM, Nikolas Everett
What i did is renamimg logging.yml to something else and then at startup
time I copy the contents to logging.yml
Thanks,
Jack
Jinyuan (Jack) Zhou
On Thu, Oct 9, 2014 at 2:28 AM, Daniel P. thetuxkee...@gmail.com wrote:
We have the same problem is reading .svn/text-base/logging.yml.text-base
Hi,
I was wondering about enabling logging for specific modules in ES. If I
wanted to turn on TRACE logging to see the merges, is it possible to turn
it on a live instance (without stopping ES, editing logging.yml and
restarting)?
Thanks,
Marie.
--
You received this message because you are
You can do so with the logger prefix in the cluster update settings api:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#logger
You'll have to know the names of the loggers you want to change though.
You can figure that out by looking around the
Hi,
EsSpout returns the results of a query which currently it's a one time event since Elastic is a storage not a queue. So
in this regard is useful for short-lived topologies that require the query data for processing.
There are plans to extend this in the future for things like
Hi Boaz,
No the data folder is persisted. And with it i have all my cluster and
indices data... but for some reason marvel asks for the license/order
number again.
Is there a file I can check for in my data folder?
Thanks!
Daniel
On Thursday, October 9, 2014 5:46:52 AM UTC-4, Boaz Leskes
Ok. Thank you for your response.
Christophe
-Message d'origine-
De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] De
la part de Costin Leau
Envoyé : jeudi 9 octobre 2014 17:54
À : elasticsearch@googlegroups.com
Objet : Re: [Hadoop][Storm] Aim of EsSpout
Hi,
Mark Walkom writes:
Did you get better writes?
What sort of storage are you on, did you measure before and after, are you
reaching I/O limits?
We pump realtime log data and only measure the the overall processing
throughput instead of low level IO throughput (we had the data, but we
did not
Hi All,
This is a bit off topic, but we only see this on some of our elastic search
hosts, and it is also the only place where we enable mlockall for java which is
our understanding is a strongly recommended best practice.
Basically we from time to time see kswapd run away at 100% on a single
Also,
For our data nodes we follow best practices with 50% of memory for java heap,
while for our master and query nodes we allocate a higher percentage with the
thought that they really do not need big disk caching. Could that be our
problem?
In addition, the systems actually are not
You cannot change the similarity on an existing index. There is no
technical measure why it could not occur, it appears to be simply a method
in place to prevent users from creating potentially huge errors. I say that
developers should have the option to shoot themselves in the foot!
Cheers,
The thought of big disk caching is correct, but you should be aware this
is a simplification of the concrete situation.
Elasticsearch uses much more RAM than the configured value - you must leave
space for internal direct buffers, stacks, classes, libraries etc. and
also for the kernel and the OS
That's weird. You should for the content of the .marvel-kibana index. That's
where it stored when you enter your license info in the UI.
Is the Marvel UI allowed to post back to ES? If that's blocked it may explain
things. Normally you will get a message from Marvel instructing you how to add
Ok, moving/copying is also a way to solve it. But in my case it wouldn't
work since we have a readonly nfs mount.
I opened an issue:
https://github.com/elasticsearch/elasticsearch/issues/8040
Let's see what happens. :)
On Thursday, October 9, 2014 5:08:19 PM UTC+2, Jinyuan Zhou wrote:
What
Thanks for open an issue.
Jinyuan (Jack) Zhou
On Thu, Oct 9, 2014 at 12:19 PM, Daniel P. thetuxkee...@gmail.com wrote:
Ok, moving/copying is also a way to solve it. But in my case it wouldn't
work since we have a readonly nfs mount.
I opened an issue:
Hi Jörg,
We tune java heap size against what we think is 'usable' memory, not system
memory, specifically to reserve space for other processes like the java app
itself, chef, splunk, etc.
The formula we have right now is:
- masters: java_min_heap_pct_of_usable_memory: 100
- data:
Heya,
We are pleased to announce the release of the Elasticsearch Twitter River
plugin, version 2.4.0.
The Twitter river indexes the public twitter stream, aka the hose, and makes it
searchable.
https://github.com/elasticsearch/elasticsearch-river-twitter/
Release Notes -
Any thoughts?
On Wednesday, October 8, 2014 1:53:35 PM UTC-4, elo...@gmail.com wrote:
Howdy All,
Just looking to get some advice on how to get the following dynamic
mapping working correctly. I'm fairly new to the mapping world in ES and
would like some help if possible. Can you let me
Hi There,
We're experiencing a similar issue after having run ES successfully for
several months without any major changes to our read/write patterns, data
sizes or documents. This is on Java 7 and ES 1.3.4.
Berhnard -- are you using scripting at all? The issue started popping up
after we
Is there anywhere in the API that a field being missing is not equivalent
to an empty list?
As an example, is it generally true that
{ some_object : []
, title : title }
is equivalent to
{ title : title }
as far as elastic search is concerned?
--
You received this message because you are
Thanks!
So, would the call be something like this:
curl -XPUT localhost:9200/_cluster/settings -d '{ persistent : {
logger.index.merge.scheduler : TRACE} }'
On Thursday, October 9, 2014 11:51:59 AM UTC-4, Nikolas Everett wrote:
You can do so with the logger prefix in the cluster update
Hi Liam,
Both documents would indeed be indexed the same way (except the _source
field).
On Thu, Oct 9, 2014 at 10:48 PM, Liam Atkinson liamatkin...@gmail.com
wrote:
Is there anywhere in the API that a field being missing is not equivalent
to an empty list?
As an example, is it generally
Don't have a bunch of time to look at this but mlockall can be dangerous
and may have an impact which you dont' understand. Specifically, it could
cause the OOM killer to jump in.
We've mitigated these by using noswap kernels but on more moderl machines
we've been doing numactl and binding it
On Wednesday, October 8, 2014 12:07:30 AM UTC-7, Jörg Prante wrote:
With ES, you can go up to the bandwidth limit the OS allows for writing
I/O (if you disable throttling etc.)
This means, if you write to one shard, it can be as fast as writing to
thousands of shards in parallel in
hi,
we are using elasticsearch to generate default mapping whenever we index
the document. we cannot really control the document fields, as we let our
clients define custom fields and we index them automatically, so we have to
rely on elasticsearch generating the mapping for us.
one issue i
hi,
we are using elasticsearch to generate default mapping whenever we index
the document. we cannot really control the document fields, as we let our
clients define custom fields and we index them automatically, so we have to
rely on elasticsearch generating the mapping for us.
one issue i
Is there anybody can help me with this issue?
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion
Yes. Something like that.
On Oct 9, 2014 6:17 PM, Marie Jacob jacob.ma...@gmail.com wrote:
Thanks!
So, would the call be something like this:
curl -XPUT localhost:9200/_cluster/settings -d '{ persistent : {
logger.index.merge.scheduler : TRACE} }'
On Thursday, October 9, 2014 11:51:59
Hello
Is there any way of changing the merge settings of a live index without
downtime in ES versions prior to 1.4 (I'm on 1.1.1)?
After toying with some of the settings for quite a while (such as
index.merge.policy.expunge_deletes_allowed and
index.merge.policy.max_merge_at_once_explicit) I
Any luck Ben?
Looks like I posted a similar question.
https://groups.google.com/forum/#!topic/elasticsearch/mUwfSQvKnDw
On Sunday, November 3, 2013 1:33:20 PM UTC-5, Ben wrote:
I am new to ES and have a requirement to store real-time data streams.
This is time-stamp based data.
Can anyone
71 matches
Mail list logo
