1. No. Did you change the configuration? You have two data nodes connected?
2. You do not need to be concerned where primary shards are allocated,
secondary shards play the same role (except primaries receive writes first
a few milliseconds earlier than secondaries). Elasticsearch randomly
I drink Kölsch only :) ävver et hätt noh immer joot jejange
Greetings from Cologne!
Jörg
On Fri, Jun 6, 2014 at 7:14 AM, Mario Mueller ma...@xenji.com wrote:
You guys are totally awesome! Thanks a lot! If you ever visit Duesseldorf
drop me a line, I owe you a beer.
@Brian:
Interesting
Hi folks
I just asked a question in StackOverflow, please have a look if you have
encountered similar problem or have some input to it.
Thanks in advance!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop
HI Pawel,
I see - your cluster state (nodes + routing only, not meta data), seems to
be larger then 16KB when rendered to SMILE, which is quite big - does this
make sense?
Above 16KB an underlying paging system introduced in the ES 1.x branch
kicks in. At that breaks something in Marvel than
Closing the transport client may not be enough.
Try this:
- wait for all outstanding actions (all actions send responses
asynchronously)
- then shut down client.threadpool() (perhaps with shutdownNow() or
shutdown()), this effectively disables new actions form being started
- then close the
Please ask your question here. Thanks.
Jörg
On Fri, Jun 6, 2014 at 9:28 AM, ohw o...@zhihu.com wrote:
Hi folks
I just asked a question in StackOverflow, please have a look if you have
encountered similar problem or have some input to it.
Thanks in advance!
--
You received this message
Sure, here it is:
-
We are migrating our Lucene based search codebase to Elasticsearch. The
major problem we encountered is how we should migrate our QueryParsers.
In our old solution, the QueryParsers take in a human input query string,
and transform that to Lucene's
Et kütt wie et kütt ;) Das mit dem Koelsch geht klar, gibts auch hier in
DUS.
Again thanks to all!
Am Freitag, 6. Juni 2014 09:09:39 UTC+2 schrieb Jörg Prante:
I drink Kölsch only :) ävver et hätt noh immer joot jejange
Greetings from Cologne!
Jörg
On Fri, Jun 6, 2014 at 7:14 AM, Mario
Hi All,
I need help on kibana, I need to Connect MySQL Database from Kibana. So
please let me know I can achieve it, and me pseudo code if anyone can have
it.
Thanks Regards,
Srinivas.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To
Hi Boaz
My cluster state from http://192.168.0.211:9200/_cluster/state?humanpretty
http://pastebin.com/ZEihQhc1 - well You may say it's big ...
Pawel
W dniu piątek, 6 czerwca 2014 09:28:30 UTC+2 użytkownik Boaz Leskes napisał:
HI Pawel,
I see - your cluster state (nodes + routing only, not
This one is without metadata
http://pastebin.com/tmJGA5Kq
http://xxx:9200/_cluster/state/version,master_node,nodes,routing_table,blocks/?humanpretty
Pawel
W dniu piątek, 6 czerwca 2014 09:28:30 UTC+2 użytkownik Boaz Leskes napisał:
HI Pawel,
I see - your cluster state (nodes + routing only,
The configurations on the two nodes are almost the same except the **
node.name**.
The first question is very strange.
I I run the same config on another cluster of two nodes and after I index
data, the state of the cluster is green. I have no idea why this
happened..Is there something I
Hi Srinu,
Kibana is javascript search/visualisation front-end for elasticsearch
which is a distributed search engine which doesn't feature mysql
connection/integration out of the box. But, there is a plugin
https://github.com/jprante/elasticsearch-river-jdbc which you can use
to import data from
Great, thanks Adrien. I will eagerly await the tool.
Kind regards,
Luke
On Thu, Jun 5, 2014 at 1:34 PM, Adrien Grand adrien.gr...@elasticsearch.com
wrote:
Hi,
This is very likely because of
https://github.com/elasticsearch/elasticsearch/pull/6393
See
The idea is:
- the basic entry point code for how search works is in
org.elasticsearch.rest.action.search.RestSearchAction, if you want to
expose an enhanced search to REST
- building the query works with
org.elasticsearch.search.builder.SearchSourceBuilder which has a convenient
method
You must use same version throughout the cluster. Combination of different
versions is not always recognized and not warned, it will result in
unexpected behavior or strange error messages.
Jörg
On Fri, Jun 6, 2014 at 11:27 AM, flyer flyer...@gmail.com wrote:
It's the ES version that causes
Look here for the tool and how to use it
http://www.elasticsearch.org/blog/tool-help-routing-issues-elasticsearch-1-2-0/
Jörg
On Fri, Jun 6, 2014 at 11:24 AM, Luke Wilson-Mawer
lukewilsonma...@gmail.com wrote:
Great, thanks Adrien. I will eagerly await the tool.
Kind regards,
Luke
On
http://www.elasticsearch.org/blog/tool-help-routing-issues-elasticsearch-1-2-0/
In the 1.2.1 release we fixed a routing bug that had been introduced in
the 1.2.0 release. This was described in the 1.2.1 blog post. The contents
of this post apply only to users that ran 1.2.0.
Before we can
probably related
http://bouk.co/blog/elasticsearch-rce/
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view
I could find only one related post:
https://groups.google.com/forum/#!msg/elasticsearch/EGCeJZbhVtA/i32ROGVmFswJ
But this has different question...
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving
So if I understand you correctly if the data is stored in Hadoop then
es-hadoop is really just acting as a job manager? If that is the case what
is the rule of thumb on how many ES nodes and shard should be set?
On Thursday, June 5, 2014 6:45:09 PM UTC-4, Costin Leau wrote:
Think of
Hmm i am not sure i understand your questions.
Hadoop is distributed storage system (HDFS) and Map-reduce framework (MR)
(among other things)
ES is distributed storage/search system (among other things)
So what es-hadoop is giving you:
You can read data from ES , and do some complex analysis ,
No, the settings will not merge existing segments unless you call _optimize
action via API.
And take some patience. Thousands of segments take time - also, they need
quite few memory resources to merge...
I suggest backup your data first, to stay safe if the merging fails /
aborts...
Jörg
On
I'm trying to index a document over 1gb in size but I get the following
error.
org.elasticsearch.common.netty.handler.codec.frame.TooLongFrameException:
HTTP content length exceeded 1073741824 bytes.
1073741824 = 1gb
My elasticsearch .yml has http.max_content_length set to 1900mb. I found
1gb is a very large document and it is unusual to index such sizes.
There is a limit check against the heap. In order to be able to process
such length, you need a large heap alone to store the document source.
Depending on analyzer, heap demand increases even more.
You can index documents of
Hi, I created my index using this mapping JSON:
{
myidx: {
index.refresh_interval:5m,
mappings: {
dynamic: strict,
message: {
_ttl: {
enabled: true
},
properties: {
Hi, Mario. Yes, I suppose this kind of goes against the no additional
proxy requirement you have.
Hehehe. I'm a seeker of loopholes. In my scenario, it's still a plug-in
design, but ES is my plug-in and not the other way around. Still only one
HTTP interface in the mix, but it's mine and not
Hi,
How do you use API to create index?
I think “dynamic” property is wrong place.
I try to create index to use the following JSON and index;
curl -XPOST localhost:9200/myidx -d '
{
settings: {
index.refresh_interval: 5m
},
mappings: {
message: {
dynamic: strict,
Hi everyone.
I use embedded ES node as part of java application.
Node node = nodeBuilder().clusterName(OSM-Gazetteer).node();
Client client = node.client();
I try to fetch some data paged.
SearchRequestBuilder searchQ = client.prepareSearch(gazetteer)
Hi Jun,
Ah, I must be doing something wrong then. I'll correct the JSON and test
again.
Btw わざわざテストしていただきありがとうございます！
On Friday, 6 June 2014 15:29:37 UTC+1, Jun Ohtani wrote:
Hi,
How do you use API to create index?
I think “dynamic” property is wrong place.
I try to create index to
Could you print your searchQ object?
May be using a toString()
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 6 juin 2014 à 16:47, Дмитрий Киселев dmitry.v.kise...@gmail.com a écrit :
Hi everyone.
I use embedded ES node as part of java application.
Node node =
Running Elasticsearch 1.2.1 with Java 1.7_55 on CentOs 6.5
The machine is a 32 core 96GB with standard spinning disk, but I also
installed 1 Samsung Evo 840 for testing ES.
The Evo is rated at 500MB/s though the Linux perf test reported about
300MB/s read and about 250MB/s write. The board is
Running Elasticsearch 1.2.1 with Java 1.7_55 on CentOs 6.5
The machine is a 32 core 96GB with standard spinning disk, but I also
installed 1 Samsung Evo 840 for testing ES.
The Evo is rated at 500MB/s though the Linux perf test reported about
300MB/s read and about 250MB/s write. The board is
Plugins are essential to ES's success and are not going away any time soon.
The river plugins, aka cluster singletons, are the ones which are
discouraged from use. Good ahead and create more plugins!
--
Ivan
On Fri, Jun 6, 2014 at 7:23 AM, Brian brian.from...@gmail.com wrote:
I also have
Good idea! One thing I am not quite clear is that writing a custom
ScoreFunction, I will have to modify elasticsearch source code and compile
it right? Or there is any other way to do it? Thank you.
--
View this message in context:
Function score code is perhaps the easiest to write for Elasticsearch. You
do not need to create a plugin for it, simply write the code with the
appropriate classes and deploy at as a jar to Elasticsearch's lib
directory. Done. That said, I still prefer to write native scripts as
plugins because
Adding to what Georgi wrote, es-hadoop does not create the shards for you -
that's up to you or index templates (which I highly recommend). However
es-hadoop is aware of the target shards and will use them to parallelize
the reads/writes (such as one task per shard).
On Fri, Jun 6, 2014 at 2:45
Did you change the boost_mode of your function score script? The default
should be multiply, which is the behavior you want, not sum, which is
what you are experiencing.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html
I have never used it
I'm in the position where I need to make _sure_ a document is deleted from
the index when something occurs in my source system. I want to just hit it
with a DELETE every time. Is that a good idea?
It looks to me like DELETEing documents via ID is pretty cheap if the
document doesn't exist. It
I mean, you can add a MyOwnFunctionBuilder/MyOwnFunctionParser to
Elasticsearch via plugin. See
package org.elasticsearch.index.query.functionscore for the standard
implementations.
The functionscore code is masterpiece quality - no need to modify existing
code! It is pluggable.
A close example
Hi all,
I want to do a one time copy of the data on my production elastic search
instance to my development elastic search instance. Both are managed by AWS
if that makes this easier. Here is what I tried:
On production:
curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{
I should also point out that I had to edit a file in the metadata-snapshot
file to change around the s3 keys and bucket name to match what development
was expecting.
On Friday, June 6, 2014 1:11:57 PM UTC-4, Brian Lamb wrote:
Hi all,
I want to do a one time copy of the data on my production
For an example function score plugin implementation, see
https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/search/functionscore/FunctionScorePluginTests.java
Jörg
On Fri, Jun 6, 2014 at 7:10 PM, joergpra...@gmail.com joergpra...@gmail.com
wrote:
I
Yep.
System.out.println(searchQ.toString());
{
from : 0,
size : 5,
query : {
match_all : { }
},
explain : false,
fields : [ ]
}
Also, I think it's might be connected with sharding.
I've tried to change number_of_shards to 1 and paging starts act as I
expect.
Set
Thank you for the links!
--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/ANN-Elasticsearch-Simple-Action-Plugin-tp4056971p4057245.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
--
You received this message because you are
Can you print also the full response object (toString()) as well?
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 6 juin 2014 à 19:24:00, Дмитрий Киселев (dmitry.v.kise...@gmail.com) a écrit:
Yep.
System.out.println(searchQ.toString());
{
from : 0,
I am trying to wire up a search that gives documents near a location a
higher ranking. Pretty simple but in our case some of the documents won't
always have a location associated with them. When using a function_score
query the documents with no location always get scored higher than the ones
The total hit number is really inconsistent:
With one shard you get: 656523
With 5 shards you get: 3879
I think you are doing something wrong but I can't tell more without looking at
the full source code.
Could you share how you actually execute the query?
Are you sure your Java client is
Thanks so much, Ivan. That's a very important distinction.
Brian
On Friday, June 6, 2014 12:28:56 PM UTC-4, Ivan Brusic wrote:
Plugins are essential to ES's success and are not going away any time
soon. The river plugins, aka cluster singletons, are the ones which are
discouraged from use.
Sorry, there littlebit different dataset.
Here is answer with same data and 1 shard
{
took : 63,
timed_out : false,
_shards : {
total : 1,
successful : 1,
failed : 0
},
hits : {
total : 3879,
max_score : 1.0,
hits : [ {
_index : gazetteer,
_type :
I apologize having to reply to my own message.
But please replace all CID w/ MID below. I started editing the text and
inadvertently hit Send.
Thanks,
-- Chris
On Friday, June 6, 2014 1:03:20 PM UTC-5, Chris Berry wrote:
Greetings,
I have a couple of questions about keys and routing.
So? What's wrong here?
You asked for 5 docs and you get 5.
I'm missing something I guess.
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 6 juin 2014 à 20:02, Дмитрий Киселев dmitry.v.kise...@gmail.com a écrit :
Sorry, there littlebit different dataset.
Here is
I asked for 5 docs.
With 1 shard - I got 5 docs.
With 5 shards - I got 25 docs.
With 5 shards, using curl instead of embedded java client - I got 5 docs.
2014-06-06 20:20 GMT+02:00 David Pilato da...@pilato.fr:
So? What's wrong here?
You asked for 5 docs and you get 5.
I'm missing
I guess the problem I having wrapping my head around is exactly where the
data is residing and in what format.
If I understand the Georgi's email above is it that you can run map reduce
jobs against data stored in local ES through by utilizing es-hadoop and you
can also run ES queries
In one part of our application we use Elasticsearch as an object store.
Therefore, when indexing, we supply our own _id. Likewise, when accessing a
document we use the simple GET method to fetch by _id. This has worked well
for us, up until recently. Normally, this is what we get:
curl -XGET
I am observing an issue with field reference in context suggester. Consider
the following use case:
PUT /services/service/_mapping
{
service: {
properties: {
name: {
type : string
},
network_id: {
type : long
Any chance you could share your code. In particular, i'd like to see how you
run the query.
If you could reproduce it with a test case that would be awesome.
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 6 juin 2014 à 20:26, Дмитрий Киселев dmitry.v.kise...@gmail.com a
Ok. I have written a test case that (if run enough) will reproduce it. Its
an intermittent bug.
I have raised an issue:
https://github.com/elasticsearch/elasticsearch/issues/6435
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe
Hi:
I have started using the scala libraries provided by the elastic4s
libraries. I am running into a problem creating a mapping that has a
straightforward definition (yet it has some complexity).
Example :
{
index: {
mappings: {
OA: {
properties: {
I think the '$' on the end of the regex may cause trouble. Lucene regular
expressions are already anchored to start and end of the string.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails
ES stores data in its own internal format, which typically resides locally.
What you are stating is partially correct - with the connector you would
move/copy data between Hadoop and ES since, in order for ES to work with
data, it needs to actually index it (that is, to see it).
So you would use
I just wanted to let anyone know i didn't succeed with the phrase suggester
and i switched to the completion suggester, that actually works very well
to my purpose.
Paul
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this
Take a look at stream2es https://github.com/elasticsearch/stream2es
On Friday, June 6, 2014 2:13:06 PM UTC-3, Brian Lamb wrote:
I should also point out that I had to edit a file in the metadata-snapshot
file to change around the s3 keys and bucket name to match what development
was
The cluster state will be yellow while shards are being rebalanced, so you
can just execute a cluster healt request and wait for green status:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-health.html#request-params
You can also poll the same API and wait until
I've made a snippet.
Code
https://github.com/kiselev-dv/es-test/blob/master/ESPagingTest/src/main/java/test/ESTest.java
Test results
https://github.com/kiselev-dv/es-test/blob/master/test.log
First test 1 shard (test.log line 27) - everything ok
Second test 5 shards (test.log line 86) - error
Yeah, sorry for not getting back to you earlier. The phrase and term
suggesters are more for providing did you mean: style spelling
corrections then typeahead. The completion suggester is for typeahead.
On Fri, Jun 6, 2014 at 5:05 PM, Paul Bormans pee...@gmail.com wrote:
I just wanted to let
sagarl wrote:
We have our ES cluster running on aws instances where nodes can
come and go. We would like to disable Shard reallocation on
cluster in Normal state so that Node going down should not
rebalance the shards.(Just to avoid unnecessary Network i/o).
If your shard topology is
Hi,
I'm trying to get a simple mapreduce job working with ES hadoop. I followed
the docs to set up a job but I keep getting 'Error in configuring object'
when I try to run it on Hadoop in pseudo distributed mode. I am using the
old API with Hadoop 2.4.0.
Here's my code I am using:
public
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException:
com.edcast.cards.MapReduceHelloWorld$SomeMapper.init()
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 15 more
Caused by:
By the way, quickly looking at your class, it's likely because your
mapper/reducer are defined at inner classes yet they are not static and
thus cannot be used without their enclosing class. In other words, declare
them as 'static'.
On Sat, Jun 7, 2014 at 1:00 AM, Costin Leau
Thanks Ivan for the suggestions, I'll try to make use of them.
Drew, Sorry I din't get the part when you said There's nothing wrong with
having a yellow cluster And yes I agree ES does a good job of rebalancing
but just think of a case where you have close to few hundred GBs of data
per shard
That was it! Thank you.
On Friday, June 6, 2014 3:12:39 PM UTC-7, Costin Leau wrote:
By the way, quickly looking at your class, it's likely because your
mapper/reducer are defined at inner classes yet they are not static and
thus cannot be used without their enclosing class. In other
I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
When I go to the new Shard Allocation dashboard in Marvel, I see the
following error:
SearchPhaseExecutionException[Failed to execute phase [query_fetch], all
shards failed; shardFailures
Hi all,
I'm using the Python API (pyes) to perform the bulk loading of our data,
here's the important part of the code:
import os
from pyes import ES
max_docs = 1
es = ES(server='hadoop42.robinsystems.com:9200')
for prefix in xrange(1, 105):
f_name = os.path.join('data',
Hi
Looking for help on improving relevancy based on keyword positioning.
Search term * ipad mini*
Search result 1
*ipad mini 16 gb*
Search result 2
*black leather case ipad mini / ipad air*
Can search result 1 be made more relevant due to the match on the search
term being at the beginning
Thanks for your replies Mark and Jorg.
My index size is around 500GB.
After using the settings (provided in my last post) initially the segments
count came down to just around 200 (without doing optimize calls), but after
some time it started increasing and now it stands around 2000+ in each
It looks like the problem is on my end. I misplaced the HEAP size parameter
and was only running with 1GB. After bumping it up to a more respectable
amount, the loading is humming along again.
-Chris
--
You received this message because you are subscribed to the Google Groups
elasticsearch
I have Elasticsearch and Kibana with nginx running on a EC2 instance but
Kibana/nginx is not able to connect to Elasticsearch and I'm getting Error
Could not contact Elasticsearch at . Please ensure that
Elasticsearch is reachable from your system.
When I see the nginx error logs, I
sagarl wrote:
Drew, Sorry I din't get the part when you said There's nothing
wrong with having a yellow cluster And yes I agree ES does a
good job of rebalancing but just think of a case where you have
close to few hundred GBs of data per shard which will move
around the cluster even though
79 matches
Mail list logo
