I want multiple type suggest for single request,
eq I have data about some article and profile user data. So when we type I
need to suggest profile user as well as user data in one request.
I tried
curl -X PUT localhost:9200/suggestion -d '
{
"mappings" : {
"keyword" : {
"properti
Hi, David
I still want to know how to get epoch time from @timestamp by using query.
I mean I want a query result like below.
Suppose the "unixtime'' is not a indexed field and converted from @timestamp.
{
"_index": "test",
"_type": "test",
"_id": "JGaR1BCFSW2lArRtbZMcGg",
"_score": 1,
Did you see this?
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-service-win.html
What is missing?
David
> Le 5 déc. 2014 à 07:05, Cheten Dev a écrit :
>
> Hi,
>
> I am new to elastic search . i am going through documentation .
> documentation is mostly about Lin
You basically define a mapping once and you're done.
Have a look at templates. They could help you using naming convention for
fields. Like all *location field names will be geo_point.
David
> Le 4 déc. 2014 à 22:44, am a écrit :
>
> Hello, I am using ES js wrapper in a nodejs application. I w
Fixed this .
The key point is NEVER invoke cross-server or cross-process service in a
for loop . This is not a question to elasticsearch but common programming
sutff.
Sorry for disturbation. Thanks.
On Tuesday, December 2, 2014 5:55:21 PM UTC+8, nodexy wrote:
>
> Hi,
>
> I got this issue when
Why not using aliases and add more index/shards when you need to?
What is wrong with this design?
David
> Le 5 déc. 2014 à 01:56, Kevin Burton a écrit :
>
> I just assumed that ES was planning on building in shard splitting at some
> point since it's a glaringly obvious addition to the feature
Hi,
I am new to elastic search . i am going through documentation .
documentation is mostly about Linux/unix , it doesn't mention how to
configuration for windows
can i get some help on this?
Thanks
--
You received this message because you are subscribed to the Google Groups
"elasticsearch"
This is exactly what elasticsearch does behind the scene.
It index ms since epoch.
David
> Le 5 déc. 2014 à 05:29, Mungeol Heo a écrit :
>
> Hi,
>
> As I mentioned at the title of this question, I wonder is there a way
> to convert @timestamp of ES to Unix epoch time in milliseconds by
> using
Thank you both Elvar and Jorg for your replies.
Aaron
On Thursday, December 4, 2014 7:58:07 AM UTC-5, Elvar Böðvarsson wrote:
>
> Two options
>
> 1. Have a client instance of elasticsearch on a different server or on the
> same server that does the query. That node must be set to master=false
Hello:
I am trying to get a list of docs (same index and type) using a list of
"ids", but only limited to certain "fields"
instead of whole doc
Currently I am doing as below
MultiGetRequestBuilder mr;
for (String id : ids)
{
Item item = new Item(index, type, id).fi
Hi,
As I mentioned at the title of this question, I wonder is there a way
to convert @timestamp of ES to Unix epoch time in milliseconds by
using ES query?
For instance, "2014-11-10T15:00:00.000Z" to "141559920".
Any help will be great.
Thanks.
--
You received this message because you are s
RAID is useful, you just need to understand the limits. And the potential
for data loss with multiple ES nodes writing to multiple data directories
is not inconsequential if it's an important system with business
requirements.
To reiterate because it's really important this is known - if you lose o
Saying that RAID is good for anything is a bit of a stretch :-P
I'm not sure how good ES is with splitting the index across volumes but the
database has a lot more options here for load distribution. RAID is naive
by design and the optimizations a RAID controller/impl are limited.
If ES can ou
I just assumed that ES was planning on building in shard splitting at some
point since it's a glaringly obvious addition to the feature set.
Then I saw this:
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/overallocation.html
> Users often ask why Elasticsearch doesn’t support
Did you specify the replica count as well as the shard count? By default ES
will add a replica unless you specifically tell it not to.
You can check this using the _cat APIs
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cat.html#cat
Also 10 shards is a bit of overkill for a
I figured as much but wanted to make sure :)
Essentially these are just plain old ES nodes, so you can pick a few at
random from your cluster and list them. It's not a specific, independent
server(s).
On 5 December 2014 at 09:54, Fernando Padilla wrote:
> So yeah, I probably mis-named it :) fr
This is why it's good to test before rolling out to critical platforms.
On 5 December 2014 at 09:29, Jack Judge wrote:
> Wrong in every fundamental aspect.
>
> This was a clusterfuck and still is. The K3 dashboards are used by our
> devs, network guys and management for a variety of tasks. When
Our system is normally very responsive, but very occasionally people submit
long phrase queries which timeout and cause high system load. Not all long
phrase queries cause issues, but I have been debugging one that I've
found.[1]
The query is in the filter section of a constant score query as b
You still are overloaded with replicas, it's pointless having them there
and it keeps your cluster out of a green state.
On 5 December 2014 at 09:52, Dunaeth wrote:
> Actualy the master node is also a datanode (si web have two datanodes),
> but just the only one that our application is aware of.
Hi,
I have a problem with one node cluster. When i create index with 10 shards,
it adds one more active shard. First i thought that it is replica shard,
but than i read that there is no replica on single node cluster. Is this
some problem in settings or it is normal?
Thanks
--
You received th
from the docs:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html#_repository_verification
This kills one of our super cool use cases which is restoring production
indices from S3 to development clusters. I don't want to give s3 write
access to the dev c
So yeah, I probably mis-named it :) from gossip-router to
gossip-server. But I think it still means the same thing :)
On 12/4/14 2:52 PM, Fernando Padilla wrote:
Um, it does.. ZenDiscovery Unicast:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.ht
Actualy the master node is also a datanode (si web have two datanodes), but
just the only one that our application is aware of. We have several metrics on
the VM, and our outsourcer may have metrics on the physical host. What's
strange is that this ES setup ran without trouble for many months be
Um, it does.. ZenDiscovery Unicast:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#unicast
It accepts a list of well known servers to drive it's "gossip" protocol. I
want to run free-standing servers/nodes with well known host:port combos.
The
What do you mean by router?
ES doesn't have that concept so perhaps you are confusing it with something
else :)
On 5 December 2014 at 06:06, Fernando Padilla wrote:
> I can't find any information in the guide nor google on how to setup a zen
> gossip router. Can anyone help me out?
>
> I really
Be aware that using multiple data locations in ES is akin to RAID0; which
means if you lose a disk then you lose all the data on that node.
Personally, I'd suggest you leverage hardware RAID and let it do what it is
good at, otherwise you just have more management overhead and greater risk
of a har
Wrong in every fundamental aspect.
This was a clusterfuck and still is. The K3 dashboards are used by our
devs, network guys and management for a variety of tasks. When they stopped
working we lost sight of large parts of our operation.
Because of the lack of documentation and the time pressure
Why so many replicas when you only have one data node? You won't even be
able to allocate them!
Your heap is also pretty small, 2GB is something you'd generally run on a
dev instance, I'd suggest going to 4GB if you can.
You need some monitoring around this to really put things into perspective.
T
What ES version, what Java version?
How much actual data?
On 5 December 2014 at 04:31, Sergio Henrique
wrote:
> Hi guys, everything ok?
>
> I want to talk about a problem that we are facing with our ES cluster.
>
> Today we have four machines in our cluster, each machine has 16GB of RAM
> (8GB H
I would guess that you need to refresh your field list in the Settings >
Indices > Index pattern section of Kibana4; this is a new thing in Kibana4
that's very different from v3. Drove me crazy trying to figure it out until
I filed an issue. See Rashid's answer to my Github issue here:
https://
Hello, I am using ES js wrapper in a nodejs application. I would like to
get ES to do some geospatial searches, so from what I understand, I need to
set the following options on mylocation field:
mylocation: {
properties: {
type: 'geo_point',
lat_lon: true
}
}
I am inser
OK I ran a quick test using Wikipedia docs; net/net I think
TieredMergePolicy's (the default) behavior is fine. Once a too-large
segment has > 50% deletes it is eligible for merging and will be
aggressively merged.
To visualize this, I first built a 33.3M doc Wikipedia index (append
only), then r
I would think the network is a prime suspect then, as there is no
significant difference between 1.2.x and 1.3.x in relation to memory usage.
And you'd certainly see OOMs in node logs if it was a memory issue.
On Thursday, December 4, 2014 12:45:58 PM UTC-8, Chris Moore wrote:
>
> There is nothi
There is nothing (literally) in the log of either data node after the node
joined events and nothing in the master log between index recovery and the
first error message.
There are 0 queries run before the errors start occurring (access to the
nodes is blocked via a firewall, so the only commun
Generally ReceiveTimeoutTransportException is due to network disconnects or
a node failing to respond due to heavy load. What does the log
of pYi3z5PgRh6msJX_armz_A show you? Perhaps it has too little heap
allocated. Rule of thumb is 1/2 available memory but <= 31GB
On Wednesday, December 3, 20
I have a related question about synchronous behavior of Java APIs. I
understand refresh will make sure that the indexed document becomes
searchable but what about subsequent updates / deletes to the same document.
If I index a document and it is going to be done asynchronously does that
mean an
We're in the middle of testing a new ES implementation in our QA
environment. We have set up a service which has a singleton
TrasportClient instance. After times of inactivity, when invoking a
search via the client we receive NodeDisconnectedExceptions.
We normally can connect and run
We're in the middle of testing a new ES implementation in our QA
environment. We have set up a service which has a singleton
TrasportClient instance. After times of inactivity, when invoking a
search via the client we receive NodeDisconnectedExceptions.
We normally can connect and run
I can't find any information in the guide nor google on how to setup a zen
gossip router. Can anyone help me out?
I really want to use elasticsearch, but I need to get over this one snag. :)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To
I'm curious why no data. Wouldn't having the data local mean faster lookups?
On Wednesday, December 3, 2014 1:14:10 PM UTC-8, Christian Hedegaard wrote:
>
> In our environment our cluster is inside EC2/VPC. We have an ELB in
> front of the cluster. We use DNS to assign a CNAME to the ELB for eas
Glad to hear you resolved your issue. Let us know if you have any other
questions.
- Nick
On Thursday, December 4, 2014 12:04:14 PM UTC-6, T Vinod Gupta wrote:
>
> nevermind, i solved it by doing something like this -
> GeoPoint latLng = GeoPoint.parseFromLatLon((String)
>
nevermind, i solved it by doing something like this -
GeoPoint latLng = GeoPoint.parseFromLatLon((String)
sourceMap.get("lat_lng"));
at the time of indexing, i am passing as ","
earlier i was passing as GeoPoint but that caused a major problem and
messed up my mapping. When i
Hi guys, everything ok?
I want to talk about a problem that we are facing with our ES cluster.
Today we have four machines in our cluster, each machine has 16GB of RAM
(8GB HEAP and 8GB OS).
We have a total of 73,975,578 documents, 998 shards and 127 indices.
To index our docs we use the bulk AP
Hello
I do agree with both of you that my use of optimize as regular maintenance
isn't the correct way to do things, but it's been the only thing that I've
found that keeps the deleted doc count/memory under control. I very much
want to find something that works to avoid it.
I came to much th
We have a lot of docs like this:
{
"_type": "doc",
"_id": "123",
"_source": {
"parent_name": "abc"
}
}
Each doc has only one parent_name but multiple docs can have the same
parent. It is like a many-to-one relationship, but the parent has no other
info apart of its name, so we didn't
SOLVED.
Not sure how, but while messing around with the nginx.conf files, it
started working. Sorry I do not have more information which points
directly to the solution.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from th
Thank you for the advice
2014-12-04 9:30 GMT-02:00 Elvar Böðvarsson :
> I upgraded our logging cluster to 1.4 without any problems.
>
> When I looked into upgrading a separate dev/test instance used for a
> different purpose I ran into problems with the plugins. If you are using
> plugins, make s
Classic CORS error - maybe * is blocked by ES. Haven't had to deal with
this myself (yet) so can't help you here. All in all just a small rough
edge to smooth, not a clusterfuck.
A quick solution would be to install K3 as a site plugin and use it
internally (don't expose it to the web)
--
Itamar
Hi,
We're running a two-nodes ES 1.0.3 cluster with the current setup :
VM on host A :
4 vCore CPU
32GB RAM
ES master (only node being queried)
MySQL slave (used as a backup, never queried)
JVM settings
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms2g -Xmx2g -Xss256k
-Djava.awt.headless=true
I'm running:
elasticsearch-1.4.0_1
logstash-1.4.2_1
kibana-3.1.1
on FreeBSD 9.3
On initial setup, the prebuilt dashboard: (Logstash Dashboard) at
/index.html#/dashboard/file/logstash.json worked
I added more panels from github. The Logstash Dashboard still worked.
Then it didn't.
screen sho
Hi,
I use logstash's syslog plugin to collect logs, searching elastic search
and kibana for the same object gives different results in the _source
field...
Elasticsearch version 1.4.0, Kibana 4.0.0-BETA2
When querying elasticsearch with curl I get:
curl -XGET http://localhost:9200/logstash*/_
Can you post a code example from your use case for how you're inserting,
retrieving, and reading the documents?
- Nick
On Tuesday, December 2, 2014 2:16:30 PM UTC-6, T Vinod Gupta wrote:
>
> has anyone seen this problem? my mapping says that the field is of type
> geo_point. but when i read doc
Hey!
I am quite new to the ELK-Stack but what I have seen so far is awesome even
if it's sometimes challenging for me! :D
But I am facing a huge problem right now. The loglines often contain
xml/soap messages and i want to filter them using the xml filter.
When doing so I sometimes get the f
Hi,
I'm trying to delete data by query from our elastic search cluster. If i
run the delete on the current active index it works fine.
However if i try to run the delete against an older index ( which is still
open ) I get an exception.
{
"error": "ClusterBlockException[blocked by: [FORBID
With Java client, you have not to worry about that. You have either
multiple node connections or explicitly chosen the nodes to connect to
(transport client).
With HTTP client, the official Elasticsearch clients provide methods to
connect to multiple hosts. If one host goes down, the next one is c
I am still not expert an in ES but surely when not paging the process time
will be higher because more documents have to be brought back in the
response. However, depending on what kind of queries you perform, the
subsequent queries will be faster. I am thinking of *filter bool queries*
with ca
Nevermind, i think i fixed it, it seems somewhere between 1.3.2 and 1.4.0
config settingts for watermark changed, i changed this:
cluster.routing.allocation.disk.watermark.low: 85%
cluster.routing.allocation.disk.watermark.high: 90%
for:
cluster.routing.allocation.disk.watermark.low: 85%
cluster.
You really need to go over the query documentation first, there are plenty
of good examples and tutorials that will guide you and help you understand
your snippet.
I gather you understand the query part, the aggregation part (there are
lots of docs) shows the frequency (count) of touchdowns on
I have 25 nodes and 12 java clients that bulk index.
This is the command i'm using:
curl -XPUT localhost:9200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.exclude._host" : "HOST"
}
}'
Got any idea why replica shards are not being allocated?
This is my conf
Yes ! Thank a lot, "mode : always" did the trick.
Now, I just have some strange frequency numbers .. suggesters boasts 203
"tomato", whereas a query only return 97.
The frequencies are less important, so I guess I'll be able to live with
that.
Thank again
--
You received this message because
Why do you set bulk indexing queue size to 3000?
Why do you limit field data cache to 25%?
What documents are in the index?
How do your queries look like?
Jörg
On Tue, Dec 2, 2014 at 1:06 PM, rmadd wrote:
> I am running elastic search in my personal box.
>
> Memory: 6GB
> Processor: Intel® C
Two options
1. Have a client instance of elasticsearch on a different server or on the
same server that does the query. That node must be set to master=false and
data=false. Being a member of the cluster means you know where the data is.
2. Use a http reverse proxy that connects to all the nodes
I am running elastic search in my personal box.
Memory: 6GB
Processor: Intel® Core™ i3-3120M CPU @ 2.50GHz × 4
OS: Ubuntu 12.04 - 64-bit
*ElasticSearch* Settings: Only running locally
Version : 1.2.2
ES_MIN_MEM=3g
ES_MAX_MEM=3g
threadpool.bulk.queue_size: 3000
indices.fielddata.cache.size: 25%
ht
On Thu, Dec 4, 2014 at 7:27 AM, DH wrote:
Hi, everyone,
>
> I'm trying to figure put some discrepencies (I think) in the results of my
> suggesters, with ES V0.90.5.
>
> My indices are big and can contain a wide array of language.
> when I do this (NB : tomate is the french for tomato)
> ;
>
>
I have a test cluster with three nodes. Each node has 4 gigs of ram. What I
see is, jvm on each nodes crashes once every day. Following is the event
log message:
Faulting application name: elasticsearch-service-x64.exe, version:
1.0.15.0, time stamp: 0x51543b9d
Faulting module name: jvm.dll, ve
Sorry I missed the other thread; I'll respond here.
Yes, that's in the pipeline - see issue #276.
As you pointed out with push down this could potentially be done
automatically...
Cheers,
P.S. Thanks for the kind words. If you encounter issues/bug or have
suggestions, please keep the feedbac
Hi, everyone,
I'm trying to figure put some discrepencies (I think) in the results of my
suggesters, with ES V0.90.5.
My indices are big and can contain a wide array of language.
when I do this (NB : tomate is the french for tomato)
;
{
"query": {
"match_all": {}
},
"suggest": {
The version is used not just in the title and description but also in the
starting script (to setup the classpath and such).
This is done on purpose, to make sure the incorrect version is not loaded by
accident (path rename, reinstall, etc...).
You could use sym links but then you'd have to re-e
Use NSSM ( http://nssm.cc/ ) to create the service instead.
Organize your folders like this
C:\Elasticsearch\
C:\Elasticsearch\nssm.exe
C:\Elasticsearch\elasticsearch.bat
C:\Elasticsearch\elasticsearch-1.1.0\
C:\Elasticsearch\elasticsearch-1.2.2\
C:\Elasticsearch\data
C:\Elasticsearch\logs
C:\Ela
I upgraded our logging cluster to 1.4 without any problems.
When I looked into upgrading a separate dev/test instance used for a
different purpose I ran into problems with the plugins. If you are using
plugins, make sure they are supported in 1.4.
--
You received this message because you are s
I am preparing proposals on hardware for our Elasticsearch log storage.
What I would love to have are SSD's for most recent logs or SSD's for hot
data. For that I have come down to two solutions with 3x physical servers.
1. Use Windows 2012 R2 as the OS, use Storage Spaces to prvide a tiered
st
25-40% is definitely "normal" for an index where many docs are being
replaced; I've seen this go up to ~65% before large merges bring it back
down.
On 2) there may be some improvements we can make to Lucene default
TieredMergePolicy here, to reclaim deletes for the "too large" segments ...
I'll ha
Hi Good plugin . . I tried to run it but it start and then stop once pom
execution is finished. How can we modify plugin to keep it running once
started. My intention is to use this plugin for demo installations. so i
can install elastic search node and start it on any machine for my demo.
Hi Costin,
Thank you for the rapid response - just wanted to say I appreciate the
Hadoop install works so easily just as advertised.
Shame about the push down filter but this is what I expected.
I'll focus on keeping the projection as lightweight as possible - on that
note - I started another
I had the same problem when I accidentially joined in cluster with a
collegue who was using an older version of elasticsearch.
Changed the cluster name and everything worked perfectly ;-)
/Stefan
Den torsdag den 4. december 2014 08.01.09 UTC+1 skrev David Montgomery:
>
>
> I added the below to e
Hello Nikolas,
we are facing similar behavior. Did you find out anything?
Thank you,
Michal
Dne pondělí, 8. září 2014 22:55:12 UTC+2 Nikolas Everett napsal(a):
>
> My indexes change somewhat frequently. If I let leave the merge settings
> as the default I end up with 25%-40% deleted documents
Hi,
There are two aspects when dealing with large tables.
1. Projection
The table mapping/definition is necessary as it indicates what information is needed - a small mapping excludes a lot of
unnecessary data.
2. Push Down filtering
Unfortunately there hasn't been much happening on this fr
You have to enable "analyze_wildcard: true"
Jörg
On Wed, Dec 3, 2014 at 4:58 PM, drjz wrote:
> Hi all,
>
> I am testing using wildcards in field names. I have the following URI
> search:
>
> _search?q=p.\*:pair&explain
>
> It returns me results (in the browser).
>
> However, when I express this
Yes, if you get an error while scan/scroll is active, you have to close the
procedure and restart from the beginning.
Not sure what you mean by an "extended period of time" but you can surely
keep the cursor open for some minutes without too much impact.
Jörg
On Wed, Dec 3, 2014 at 12:30 AM, Bar
I noticed these warnings on some of my nodes while executing the snapshot,
maybe it has to do something with why its so slow.
[2014-12-03 15:57:35,699][WARN ][snapshots] [xx06]
[[xxx-2014-11-20][7]] [my_backup:snapshot_test] failed to create snapshot
org.elasticsearch.index.snap
You might have the same field name used accross many indices but without the
same analyzer?
Hard to tell more without an actual example.
David
> Le 4 déc. 2014 à 08:54, Narinder Kaur a écrit :
>
> Hi there,
>
>I have a very simple term query. It is not giving result if I am
> executi
81 matches
Mail list logo