You might have the same field name used accross many indices but without the
same analyzer?
Hard to tell more without an actual example.
David
Le 4 déc. 2014 à 08:54, Narinder Kaur narinder.k...@izap.in a écrit :
Hi there,
I have a very simple term query. It is not giving result
I noticed these warnings on some of my nodes while executing the snapshot,
maybe it has to do something with why its so slow.
[2014-12-03 15:57:35,699][WARN ][snapshots] [xx06]
[[xxx-2014-11-20][7]] [my_backup:snapshot_test] failed to create snapshot
You have to enable analyze_wildcard: true
Jörg
On Wed, Dec 3, 2014 at 4:58 PM, drjz juntezh...@gmail.com wrote:
Hi all,
I am testing using wildcards in field names. I have the following URI
search:
_search?q=p.\*:pairexplain
It returns me results (in the browser).
However, when I
Hi,
There are two aspects when dealing with large tables.
1. Projection
The table mapping/definition is necessary as it indicates what information is needed - a small mapping excludes a lot of
unnecessary data.
2. Push Down filtering
Unfortunately there hasn't been much happening on this
Hello Nikolas,
we are facing similar behavior. Did you find out anything?
Thank you,
Michal
Dne pondělí, 8. září 2014 22:55:12 UTC+2 Nikolas Everett napsal(a):
My indexes change somewhat frequently. If I let leave the merge settings
as the default I end up with 25%-40% deleted documents
I had the same problem when I accidentially joined in cluster with a
collegue who was using an older version of elasticsearch.
Changed the cluster name and everything worked perfectly ;-)
/Stefan
Den torsdag den 4. december 2014 08.01.09 UTC+1 skrev David Montgomery:
I added the below to
Hi Good plugin . . I tried to run it but it start and then stop once pom
execution is finished. How can we modify plugin to keep it running once
started. My intention is to use this plugin for demo installations. so i
can install elastic search node and start it on any machine for my demo.
25-40% is definitely normal for an index where many docs are being
replaced; I've seen this go up to ~65% before large merges bring it back
down.
On 2) there may be some improvements we can make to Lucene default
TieredMergePolicy here, to reclaim deletes for the too large segments ...
I'll have
I am preparing proposals on hardware for our Elasticsearch log storage.
What I would love to have are SSD's for most recent logs or SSD's for hot
data. For that I have come down to two solutions with 3x physical servers.
1. Use Windows 2012 R2 as the OS, use Storage Spaces to prvide a tiered
I upgraded our logging cluster to 1.4 without any problems.
When I looked into upgrading a separate dev/test instance used for a
different purpose I ran into problems with the plugins. If you are using
plugins, make sure they are supported in 1.4.
--
You received this message because you are
Use NSSM ( http://nssm.cc/ ) to create the service instead.
Organize your folders like this
C:\Elasticsearch\
C:\Elasticsearch\nssm.exe
C:\Elasticsearch\elasticsearch.bat
C:\Elasticsearch\elasticsearch-1.1.0\
C:\Elasticsearch\elasticsearch-1.2.2\
C:\Elasticsearch\data
C:\Elasticsearch\logs
The version is used not just in the title and description but also in the
starting script (to setup the classpath and such).
This is done on purpose, to make sure the incorrect version is not loaded by
accident (path rename, reinstall, etc...).
You could use sym links but then you'd have to
Hi, everyone,
I'm trying to figure put some discrepencies (I think) in the results of my
suggesters, with ES V0.90.5.
My indices are big and can contain a wide array of language.
when I do this (NB : tomate is the french for tomato)
;
{
query: {
match_all: {}
},
suggest: {
Sorry I missed the other thread; I'll respond here.
Yes, that's in the pipeline - see issue #276.
As you pointed out with push down this could potentially be done
automatically...
Cheers,
P.S. Thanks for the kind words. If you encounter issues/bug or have
suggestions, please keep the
I have a test cluster with three nodes. Each node has 4 gigs of ram. What I
see is, jvm on each nodes crashes once every day. Following is the event
log message:
Faulting application name: elasticsearch-service-x64.exe, version:
1.0.15.0, time stamp: 0x51543b9d
Faulting module name: jvm.dll,
On Thu, Dec 4, 2014 at 7:27 AM, DH ciddp...@gmail.com wrote:
Hi, everyone,
I'm trying to figure put some discrepencies (I think) in the results of my
suggesters, with ES V0.90.5.
My indices are big and can contain a wide array of language.
when I do this (NB : tomate is the french for
I am running elastic search in my personal box.
Memory: 6GB
Processor: Intel® Core™ i3-3120M CPU @ 2.50GHz × 4
OS: Ubuntu 12.04 - 64-bit
*ElasticSearch* Settings: Only running locally
Version : 1.2.2
ES_MIN_MEM=3g
ES_MAX_MEM=3g
threadpool.bulk.queue_size: 3000
indices.fielddata.cache.size: 25%
Two options
1. Have a client instance of elasticsearch on a different server or on the
same server that does the query. That node must be set to master=false and
data=false. Being a member of the cluster means you know where the data is.
2. Use a http reverse proxy that connects to all the
Why do you set bulk indexing queue size to 3000?
Why do you limit field data cache to 25%?
What documents are in the index?
How do your queries look like?
Jörg
On Tue, Dec 2, 2014 at 1:06 PM, rmadd rma...@gmail.com wrote:
I am running elastic search in my personal box.
Memory: 6GB
Yes ! Thank a lot, mode : always did the trick.
Now, I just have some strange frequency numbers .. suggesters boasts 203
tomato, whereas a query only return 97.
The frequencies are less important, so I guess I'll be able to live with
that.
Thank again
--
You received this message because
I have 25 nodes and 12 java clients that bulk index.
This is the command i'm using:
curl -XPUT localhost:9200/_cluster/settings -d '{
transient : {
cluster.routing.allocation.exclude._host : HOST
}
}'
Got any idea why replica shards are not being allocated?
This is my config
Nevermind, i think i fixed it, it seems somewhere between 1.3.2 and 1.4.0
config settingts for watermark changed, i changed this:
cluster.routing.allocation.disk.watermark.low: 85%
cluster.routing.allocation.disk.watermark.high: 90%
for:
cluster.routing.allocation.disk.watermark.low: 85%
I am still not expert an in ES but surely when not paging the process time
will be higher because more documents have to be brought back in the
response. However, depending on what kind of queries you perform, the
subsequent queries will be faster. I am thinking of *filter bool queries*
with
With Java client, you have not to worry about that. You have either
multiple node connections or explicitly chosen the nodes to connect to
(transport client).
With HTTP client, the official Elasticsearch clients provide methods to
connect to multiple hosts. If one host goes down, the next one is
Hi,
I'm trying to delete data by query from our elastic search cluster. If i
run the delete on the current active index it works fine.
However if i try to run the delete against an older index ( which is still
open ) I get an exception.
{
error: ClusterBlockException[blocked by:
Hey!
I am quite new to the ELK-Stack but what I have seen so far is awesome even
if it's sometimes challenging for me! :D
But I am facing a huge problem right now. The loglines often contain
xml/soap messages and i want to filter them using the xml filter.
When doing so I sometimes get the
Can you post a code example from your use case for how you're inserting,
retrieving, and reading the documents?
- Nick
On Tuesday, December 2, 2014 2:16:30 PM UTC-6, T Vinod Gupta wrote:
has anyone seen this problem? my mapping says that the field is of type
geo_point. but when i read
Hi,
I use logstash's syslog plugin to collect logs, searching elastic search
and kibana for the same object gives different results in the _source
field...
Elasticsearch version 1.4.0, Kibana 4.0.0-BETA2
When querying elasticsearch with curl I get:
curl -XGET
I'm running:
elasticsearch-1.4.0_1
logstash-1.4.2_1
kibana-3.1.1
on FreeBSD 9.3
On initial setup, the prebuilt dashboard: (Logstash Dashboard) at
/index.html#/dashboard/file/logstash.json worked
I added more panels from github. The Logstash Dashboard still worked.
Then it didn't.
screen
Hi,
We're running a two-nodes ES 1.0.3 cluster with the current setup :
VM on host A :
4 vCore CPU
32GB RAM
ES master (only node being queried)
MySQL slave (used as a backup, never queried)
JVM settings
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms2g -Xmx2g -Xss256k
-Djava.awt.headless=true
Classic CORS error - maybe * is blocked by ES. Haven't had to deal with
this myself (yet) so can't help you here. All in all just a small rough
edge to smooth, not a clusterfuck.
A quick solution would be to install K3 as a site plugin and use it
internally (don't expose it to the web)
--
Thank you for the advice
2014-12-04 9:30 GMT-02:00 Elvar Böðvarsson elv...@gmail.com:
I upgraded our logging cluster to 1.4 without any problems.
When I looked into upgrading a separate dev/test instance used for a
different purpose I ran into problems with the plugins. If you are using
SOLVED.
Not sure how, but while messing around with the nginx.conf files, it
started working. Sorry I do not have more information which points
directly to the solution.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from
We have a lot of docs like this:
{
_type: doc,
_id: 123,
_source: {
parent_name: abc
}
}
Each doc has only one parent_name but multiple docs can have the same
parent. It is like a many-to-one relationship, but the parent has no other
info apart of its name, so we didn't create a
Hello
I do agree with both of you that my use of optimize as regular maintenance
isn't the correct way to do things, but it's been the only thing that I've
found that keeps the deleted doc count/memory under control. I very much
want to find something that works to avoid it.
I came to much
Hi guys, everything ok?
I want to talk about a problem that we are facing with our ES cluster.
Today we have four machines in our cluster, each machine has 16GB of RAM
(8GB HEAP and 8GB OS).
We have a total of 73,975,578 documents, 998 shards and 127 indices.
To index our docs we use the bulk
nevermind, i solved it by doing something like this -
GeoPoint latLng = GeoPoint.parseFromLatLon((String)
sourceMap.get(lat_lng));
at the time of indexing, i am passing as lat,lng
earlier i was passing as GeoPoint but that caused a major problem and
messed up my mapping. When
Glad to hear you resolved your issue. Let us know if you have any other
questions.
- Nick
On Thursday, December 4, 2014 12:04:14 PM UTC-6, T Vinod Gupta wrote:
nevermind, i solved it by doing something like this -
GeoPoint latLng = GeoPoint.parseFromLatLon((String)
I'm curious why no data. Wouldn't having the data local mean faster lookups?
On Wednesday, December 3, 2014 1:14:10 PM UTC-8, Christian Hedegaard wrote:
In our environment our cluster is inside EC2/VPC. We have an ELB in
front of the cluster. We use DNS to assign a CNAME to the ELB for
I can't find any information in the guide nor google on how to setup a zen
gossip router. Can anyone help me out?
I really want to use elasticsearch, but I need to get over this one snag. :)
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To
We're in the middle of testing a new ES implementation in our QA
environment. We have set up a service which has a singleton
TrasportClient instance. After times of inactivity, when invoking a
search via the client we receive NodeDisconnectedExceptions.
We normally can connect and run
We're in the middle of testing a new ES implementation in our QA
environment. We have set up a service which has a singleton
TrasportClient instance. After times of inactivity, when invoking a
search via the client we receive NodeDisconnectedExceptions.
We normally can connect and run
I have a related question about synchronous behavior of Java APIs. I
understand refresh will make sure that the indexed document becomes
searchable but what about subsequent updates / deletes to the same document.
If I index a document and it is going to be done asynchronously does that
mean
Generally ReceiveTimeoutTransportException is due to network disconnects or
a node failing to respond due to heavy load. What does the log
of pYi3z5PgRh6msJX_armz_A show you? Perhaps it has too little heap
allocated. Rule of thumb is 1/2 available memory but = 31GB
On Wednesday, December 3,
There is nothing (literally) in the log of either data node after the node
joined events and nothing in the master log between index recovery and the
first error message.
There are 0 queries run before the errors start occurring (access to the
nodes is blocked via a firewall, so the only
I would think the network is a prime suspect then, as there is no
significant difference between 1.2.x and 1.3.x in relation to memory usage.
And you'd certainly see OOMs in node logs if it was a memory issue.
On Thursday, December 4, 2014 12:45:58 PM UTC-8, Chris Moore wrote:
There is
I would guess that you need to refresh your field list in the Settings
Indices Index pattern section of Kibana4; this is a new thing in Kibana4
that's very different from v3. Drove me crazy trying to figure it out until
I filed an issue. See Rashid's answer to my Github issue here:
What ES version, what Java version?
How much actual data?
On 5 December 2014 at 04:31, Sergio Henrique sergiohenriqu...@gmail.com
wrote:
Hi guys, everything ok?
I want to talk about a problem that we are facing with our ES cluster.
Today we have four machines in our cluster, each machine
Why so many replicas when you only have one data node? You won't even be
able to allocate them!
Your heap is also pretty small, 2GB is something you'd generally run on a
dev instance, I'd suggest going to 4GB if you can.
You need some monitoring around this to really put things into perspective.
Wrong in every fundamental aspect.
This was a clusterfuck and still is. The K3 dashboards are used by our
devs, network guys and management for a variety of tasks. When they stopped
working we lost sight of large parts of our operation.
Because of the lack of documentation and the time
Be aware that using multiple data locations in ES is akin to RAID0; which
means if you lose a disk then you lose all the data on that node.
Personally, I'd suggest you leverage hardware RAID and let it do what it is
good at, otherwise you just have more management overhead and greater risk
of a
What do you mean by router?
ES doesn't have that concept so perhaps you are confusing it with something
else :)
On 5 December 2014 at 06:06, Fernando Padilla fern...@gmail.com wrote:
I can't find any information in the guide nor google on how to setup a zen
gossip router. Can anyone help me
Um, it does.. ZenDiscovery Unicast:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#unicast
It accepts a list of well known servers to drive it's gossip protocol. I
want to run free-standing servers/nodes with well known host:port combos.
The
Actualy the master node is also a datanode (si web have two datanodes), but
just the only one that our application is aware of. We have several metrics on
the VM, and our outsourcer may have metrics on the physical host. What's
strange is that this ES setup ran without trouble for many months
So yeah, I probably mis-named it :) from gossip-router to
gossip-server. But I think it still means the same thing :)
On 12/4/14 2:52 PM, Fernando Padilla wrote:
Um, it does.. ZenDiscovery Unicast:
from the docs:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html#_repository_verification
This kills one of our super cool use cases which is restoring production
indices from S3 to development clusters. I don't want to give s3 write
access to the dev
Hi,
I have a problem with one node cluster. When i create index with 10 shards,
it adds one more active shard. First i thought that it is replica shard,
but than i read that there is no replica on single node cluster. Is this
some problem in settings or it is normal?
Thanks
--
You received
You still are overloaded with replicas, it's pointless having them there
and it keeps your cluster out of a green state.
On 5 December 2014 at 09:52, Dunaeth lomig.po...@gmail.com wrote:
Actualy the master node is also a datanode (si web have two datanodes),
but just the only one that our
Our system is normally very responsive, but very occasionally people submit
long phrase queries which timeout and cause high system load. Not all long
phrase queries cause issues, but I have been debugging one that I've
found.[1]
The query is in the filter section of a constant score query as
This is why it's good to test before rolling out to critical platforms.
On 5 December 2014 at 09:29, Jack Judge jackjudg...@gmail.com wrote:
Wrong in every fundamental aspect.
This was a clusterfuck and still is. The K3 dashboards are used by our
devs, network guys and management for a
I figured as much but wanted to make sure :)
Essentially these are just plain old ES nodes, so you can pick a few at
random from your cluster and list them. It's not a specific, independent
server(s).
On 5 December 2014 at 09:54, Fernando Padilla fern...@gmail.com wrote:
So yeah, I probably
Did you specify the replica count as well as the shard count? By default ES
will add a replica unless you specifically tell it not to.
You can check this using the _cat APIs
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cat.html#cat
Also 10 shards is a bit of overkill for
I just assumed that ES was planning on building in shard splitting at some
point since it's a glaringly obvious addition to the feature set.
Then I saw this:
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/overallocation.html
Users often ask why Elasticsearch doesn’t support
Saying that RAID is good for anything is a bit of a stretch :-P
I'm not sure how good ES is with splitting the index across volumes but the
database has a lot more options here for load distribution. RAID is naive
by design and the optimizations a RAID controller/impl are limited.
If ES can
RAID is useful, you just need to understand the limits. And the potential
for data loss with multiple ES nodes writing to multiple data directories
is not inconsequential if it's an important system with business
requirements.
To reiterate because it's really important this is known - if you lose
Hello:
I am trying to get a list of docs (same index and type) using a list of
ids, but only limited to certain fields
instead of whole doc
Currently I am doing as below
MultiGetRequestBuilder mr;
for (String id : ids)
{
Item item = new Item(index, type,
Thank you both Elvar and Jorg for your replies.
Aaron
On Thursday, December 4, 2014 7:58:07 AM UTC-5, Elvar Böðvarsson wrote:
Two options
1. Have a client instance of elasticsearch on a different server or on the
same server that does the query. That node must be set to master=false and
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.
David
Le 5 déc. 2014 à 05:29, Mungeol Heo mungeol@gmail.com a écrit :
Hi,
As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in
Hi,
I am new to elastic search . i am going through documentation .
documentation is mostly about Linux/unix , it doesn't mention how to
configuration for windows
can i get some help on this?
Thanks
--
You received this message because you are subscribed to the Google Groups
elasticsearch
Why not using aliases and add more index/shards when you need to?
What is wrong with this design?
David
Le 5 déc. 2014 à 01:56, Kevin Burton burtona...@gmail.com a écrit :
I just assumed that ES was planning on building in shard splitting at some
point since it's a glaringly obvious
Fixed this .
The key point is NEVER invoke cross-server or cross-process service in a
for loop . This is not a question to elasticsearch but common programming
sutff.
Sorry for disturbation. Thanks.
On Tuesday, December 2, 2014 5:55:21 PM UTC+8, nodexy wrote:
Hi,
I got this issue when
You basically define a mapping once and you're done.
Have a look at templates. They could help you using naming convention for
fields. Like all *location field names will be geo_point.
David
Le 4 déc. 2014 à 22:44, am apl...@gmail.com a écrit :
Hello, I am using ES js wrapper in a nodejs
Did you see this?
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-service-win.html
What is missing?
David
Le 5 déc. 2014 à 07:05, Cheten Dev cheten@carwale.com a écrit :
Hi,
I am new to elastic search . i am going through documentation .
documentation is
Hi, David
I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the unixtime'' is not a indexed field and converted from @timestamp.
{
_index: test,
_type: test,
_id: JGaR1BCFSW2lArRtbZMcGg,
_score: 1,
fields: {
I want multiple type suggest for single request,
eq I have data about some article and profile user data. So when we type I
need to suggest profile user as well as user data in one request.
I tried
curl -X PUT localhost:9200/suggestion -d '
{
mappings : {
keyword : {
properties :
75 matches
Mail list logo
