Re: message: [WARN ][cluster.service ] [node1] failed to reconnect to node [node1][I4Wltlc9RSm0jJhumBRtpQ][inet[/10.10.10.1:9300]]
Well the 10.10.10.1 is just another nic on the same node. Op dinsdag 31 december 2013 12:11:04 UTC+1 schreef David Pilato: Thanks! So you have your node running under IP: 10.192.21.69 It's trying to access another node at 10.10.10.1 So may be you modify elasticsearch.yml file or you have another node running with a different version on your network. Could you run curl http://10.10.10.1:9200/ ? -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr Le 31 décembre 2013 at 12:08:30, HansPeterSloot . (hanspet...@gmail.comjavascript:) a écrit: Sorry for the confusion. The file was so long I just thought that you only needed the error message. Can you access this one? nl19856 https://gist.github.com/nl19856 / *gist:8195337 https://gist.github.com/nl19856/8195337* 2013/12/31 David Pilato da...@pilato.fr javascript: Sorry if I am unclear but I would like to see all the lines in your log file, from when you start the node to this issue. -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr Le 31 décembre 2013 at 11:59:25, HansPeterSloot (hanspet...@gmail.comjavascript:) a écrit: Never did that. Is this what you need? nl19856 https://gist.github.com/nl19856 / *gist:8195274 https://gist.github.com/nl19856/8195274* Op dinsdag 31 december 2013 11:41:05 UTC+1 schreef David Pilato: Could you GIST (gist.github.com) the full log file please? -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr Le 31 décembre 2013 at 10:41:32, HansPeterSloot (hanspet...@gmail.com) a écrit: There is really no other elasticsearch node in the network. I am sure about that. There is also only one daemon running: ps -ef|grep elast 101 1769 1 1 08:27 ?00:02:14 /usr/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-0.90.7.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/* -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.ElasticSearch root 2271 2246 0 10:23 pts/21 00:00:00 grep elast The full entry is: [2013-12-31 10:24:13,554][WARN ][cluster.service ] [node1] failed to reconnect to node [node1][KhPM-quUTlmGAjpZoQjH5Q][inet[/ 10.10.10.1:9300]] org.elasticsearch.transport.ConnectTransportException: [node1][inet[/ 10.10.10.1:9300]] connect_timeout[30s] at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:693) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:622) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:590) at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129) at org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:505) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) Caused by: java.net.ConnectException: Connection refused: / 10.10.10.1:9300 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at
Re: message: [WARN ][cluster.service ] [node1] failed to reconnect to node [node1][I4Wltlc9RSm0jJhumBRtpQ][inet[/10.10.10.1:9300]]
Yes that worked.
Thanks a lot.
Op dinsdag 31 december 2013 12:12:43 UTC+1 schreef David Pilato:
Sorry. Wrong answer.
It sounds like you have 2 network cards on your computer:
inet[/10.192.21.69:9200]}, publish_address {inet[/10.10.10.1:9200]}
You should set network.host
See:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-network.html#tcp-settings
--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet |
@elasticsearchfrhttps://twitter.com/elasticsearchfr
Le 31 décembre 2013 at 12:11:07, David Pilato (da...@pilato.frjavascript:)
a écrit:
Thanks!
So you have your node running under IP: 10.192.21.69
It's trying to access another node at 10.10.10.1
So may be you modify elasticsearch.yml file or you have another node
running with a different version on your network.
Could you run
curl http://10.10.10.1:9200/
?
--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet |
@elasticsearchfrhttps://twitter.com/elasticsearchfr
Le 31 décembre 2013 at 12:08:30, HansPeterSloot .
(hanspet...@gmail.comjavascript:)
a écrit:
Sorry for the confusion.
The file was so long I just thought that you only needed the error
message.
Can you access this one?
nl19856 https://gist.github.com/nl19856 / *gist:8195337
https://gist.github.com/nl19856/8195337*
2013/12/31 David Pilato da...@pilato.fr javascript:
Sorry if I am unclear but I would like to see all the lines in your log
file, from when you start the node to this issue.
--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet |
@elasticsearchfrhttps://twitter.com/elasticsearchfr
Le 31 décembre 2013 at 11:59:25, HansPeterSloot
(hanspet...@gmail.comjavascript:)
a écrit:
Never did that.
Is this what you need?
nl19856 https://gist.github.com/nl19856 / *gist:8195274
https://gist.github.com/nl19856/8195274*
Op dinsdag 31 december 2013 11:41:05 UTC+1 schreef David Pilato:
Could you GIST (gist.github.com) the full log file please?
--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet |
@elasticsearchfrhttps://twitter.com/elasticsearchfr
Le 31 décembre 2013 at 10:41:32, HansPeterSloot (hanspet...@gmail.com)
a écrit:
There is really no other elasticsearch node in the network. I am sure
about that.
There is also only one daemon running:
ps -ef|grep elast
101 1769 1 1 08:27 ?00:02:14 /usr/bin/java -Xms256m
-Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid
-Des.path.home=/usr/share/elasticsearch -cp
:/usr/share/elasticsearch/lib/elasticsearch-0.90.7.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/*
-Des.default.path.home=/usr/share/elasticsearch
-Des.default.path.logs=/var/log/elasticsearch
-Des.default.path.data=/var/lib/elasticsearch
-Des.default.path.work=/tmp/elasticsearch
-Des.default.path.conf=/etc/elasticsearch
org.elasticsearch.bootstrap.ElasticSearch
root 2271 2246 0 10:23 pts/21 00:00:00 grep elast
The full entry is:
[2013-12-31 10:24:13,554][WARN ][cluster.service ] [node1]
failed to reconnect to node [node1][KhPM-quUTlmGAjpZoQjH5Q][inet[/
10.10.10.1:9300]]
org.elasticsearch.transport.ConnectTransportException: [node1][inet[/
10.10.10.1:9300]] connect_timeout[30s]
at
org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:693)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:622)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:590)
at
org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
at
org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:505)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused: /
10.10.10.1:9300
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150)
at
Re: Elastic search range dates
Your query looks good. However, the way your hit is formatted in the results of the second query seems to indicate that your dates are indexed as strings instead of dates. How did you get your data from mongo to Elasticsearch? On Tue, Dec 31, 2013 at 11:42 AM, melasticsearch melasticsea...@mailinator.com wrote: no one? -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Elastic-search-range-dates-tp4046565p4046854.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1388486560274-4046854.post%40n3.nabble.com . For more options, visit https://groups.google.com/groups/opt_out. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5s3iiZaDgfvwkbgU9O5vxFJDLHJ_w7kQadcsDvaZfCNA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Facet autocomplete for field with multiple values
Hi, I have problem finding solution for specific usage scenario where I want to autocomplete facet for field that has array of values. Documents in my index are some objects that can be owned by 0..n people, so I have field 'owner' containing zero to few strings of following format: first_name [second_name] last_name (user_id). Since there are lots of objects and many owners, I don't like idea of loading all owners onto webpage and make some client-side filtering, so I wanted to use autocomplete feature. Since I have owners in such format, even prefix search doesn't work for me since user can type last name of owner which is some n-th word in string. Second thing is that I need autocomplete to respond with whole field values that will properly match facet filter. So I would like to get 'John Doe (p01)' for all queries like 'John', 'Doe', 'Joh', 'p01'. I was thinking of some approaches, but all have some drawbacks: 1. Completion suggester - everything fine except I won't be able to search by last_name or user_id. 2. Phrase suggester with shingles - as far as I know it will allow me to have google-kind autocomplete, but it will only give me parts of field value beginning with provided text (so if for 'John Doe (p01)' I would query 'Doe', I expect to get 'Doe' and 'Doe (p01)' which is not what I need). 3. Regex query - here I can make a successful query, but ES will respond with whole documents (which is firstly too heavy) and those documents will contain whole arrays of owners and I would be forced to filter them in code to get only those owners that are matching the query. Finally it would work, but way too much overhead. Please help in any way (even if you would that there is no way it is better than no response). Regards, Peter. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6ccaa79e-f4fd-43c0-bbfd-8282bfc1f19d%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Highlight values from one field in another field
You mean you want the terms highlighted in one field to come from the
contents of another field rather than the query? If so, then Elasticsearch
doesn't support that already but you could write a custom highlighter to do
that.
Nik
On Tue, Dec 31, 2013 at 9:40 AM, mayap maya.poll...@gmail.com wrote:
Hi All,
We have multi value field - name. We would like to highlight those
values in anther field - text. For example, for the values:
name : [name1, name2, name3, name4]
text : name1 is a friend of name2
we would like to get the result:
highlight: {
text : [emname1/em is a friend of emname2/em ]
}
Is it possible to reach that in a single query?
Thanks.
-
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a717c411-3d9f-432d-97d3-713cef5d378e%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3m-VF_sV6Dv3vf77vgR4R5At8nxYDdF7PLN3tP1_0byw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Plain filter and constant_score
Thanks Matt.
It seems that we've done terrible things ... effectively `post_filter` is a
better name.
在 2013年12月31日星期二UTC+1上午12时16分43秒，Matt Weber写道：
The outer filter is basically a post filter, ie. filtering happens after
all the documents have been collected via the query. This should not
really be used unless you are trying to do something like multi-select
faceting where you don't want facet counts to be affected by the filter.
You should be using a filtered query [1] or as you discovered a constant
score query if you only want to execute a filter.
BTW, in elasticsearch 1.0, this outer filter has been renamed to
post_filter to avoid some of the confusion.
Thanks,
Matt Weber
On Mon, Dec 30, 2013 at 2:53 PM, Han JU ju.han...@gmail.com javascript:
wrote:
Hi,
We are currently benchmarking our ES setup so I've got some new questions:
1. We found out that, for the same query (filter actually), when put like
this:
{
filter: {...},
fields: [...]
}
is consistently slower than this form:
{
query: {
constant_score: {
filter: {...},
...
}
}
}.
All filter and fields part are identical, but the performance is
different, especially when caches are warm, filters wrapped in a
constant_score are nearly 10x faster than when they are put directly.
So what happens behind this? How ElasticSearch interpret filters that are
put directly (not wrapped in any outer structure)?
Thanks in advance.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ad91095f-7e7b-4def-8570-73dbccaa7cb2%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/050b2a38-4393-4fa5-a036-41fd29a797c9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Bulk throughput issues
Sorry for the delay, I had to take some unplanned leave and I wasn't able to get to this while I was out. With some more testing I was able to get ~10k documents a second but I had to make some code changes. 1: I changed to the transport client in our Java code 2: It seemed as if one client wasn't able to keep up so what I did in the code was actually spawn a couple of transport clients, each with it's own bulk processor with concurrent set at 32. The part of our code that is reading in the messages from Kafka then submits them at random to these various thrift clients. Is anyone else having to do this or should a single thrift client be able to do this? I wasn't able to get much more out of it because the CPU usage started to get really high but I don't think that's an Elasticsearch thing, I think it's because we are doing so many regex tasks. While hitting around ~10k a second the network output was only about 5mb a second so we don't seem to be blocked there. I did determine that was are basically able to pull from Kafka as fast as the messages come in when NOT doing inserts into Elasticsearch so I don't think that is the problem. I plan on doing some testing today where we have multiple consumers running so see if we can hit our ~40k inserts per second goal (4 consumers doing ~10k each). -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56dd795e-fb98-4059-8ab9-5959c2bc3c52%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Highlight values from one field in another field
Thanks for your reply.
I have another question:
Is it possible in highlight_query (I'm using version 0.90.9) to use few
highlight tags? For example I want to highlight text field with tag1
for one sub query and with tag2 for another sub query?
On Tuesday, December 31, 2013 4:42:20 PM UTC+2, Nikolas Everett wrote:
You mean you want the terms highlighted in one field to come from the
contents of another field rather than the query? If so, then Elasticsearch
doesn't support that already but you could write a custom highlighter to do
that.
Nik
On Tue, Dec 31, 2013 at 9:40 AM, mayap maya.p...@gmail.com
javascript:wrote:
Hi All,
We have multi value field - name. We would like to highlight those
values in anther field - text. For example, for the values:
name : [name1, name2, name3, name4]
text : name1 is a friend of name2
we would like to get the result:
highlight: {
text : [emname1/em is a friend of emname2/em ]
}
Is it possible to reach that in a single query?
Thanks.
-
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a717c411-3d9f-432d-97d3-713cef5d378e%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/443130dc-9a82-45d0-a3a9-866ddb9bd5e3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: 16Gb RAM / 8 cores : best config ?
No easy answer, it depends on your OS and your requirements - indexing load, searching load, analytics ... you should start with ES default settings and heap size increased to 50% of RAM, that is 8G Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFi_LSygR1WSJhetfo97WjoE_niZ1T2r%2BpP%2BvHCJMfE3w%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Bulk throughput issues
Jason, you have to measure the docs at the indexing API with your client
code.
You can use the _cluster/stats or /_cluster/stats/nodes/{nodeId} endpoint
to inspect the node caches (store, fielddata, filter_cache, id_cache,
completion). The indexing buffer and the translog buffer state can not be
inspected, but you can increase the log level to DEBUG to follow how ES
dynamicall resizes these buffers.
Jörg
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEuybOGsCZXdxRcenHfM31_cia_qhYKjVzaFbkn2E%3Dv8A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: OR query
You are better of using a proper boolean filter for better performance.
Queries cannot be cached and query string query analyzes the terms. Here is
an example of your filter with a nested bool (should) filter:
filter: {
and: {
filters: [
{
bool: {
must: [
{
bool: {
should: [
{
term: {
state: MA
}
},
{
term: {
state: NY
}
}
]
}
},
{
range: {
costOutofstateTution: {
gte: 0,
lte: 3
}
}
}
]
}
}
]
}
}
Cheers,
Ivan
On Mon, Dec 30, 2013 at 10:03 PM, paul avinashpau...@gmail.com wrote:
I got the query wotking by using
{
query_string: {
default_field: state,
query: MA NY
}
}
- Paul
On Tuesday, 31 December 2013 11:07:06 UTC+5:30, paul wrote:
My query is as below , which gives me all the colleges with state code
MA i want all the colleges that are in MA or NY how to add OR filter
{
query: {
filtered: {
query: {
nested: {
path: programs,
query: {
bool: {
must: [
{
match: {
programs.progName: Computer and Information
Sciences
}
},
{
range: {
programs.Bachelor: {
gt: 0
}
}
}
]
}
}
}
},
filter: {
and: {
filters: [
{
bool: {
must: [
{
term: {
state: MA
}
},
{
range: {
costOutofstateTution: {
gte: 0,
lte: 3
}
}
}
]
}
}
]
}
}
}
}
}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d23102f3-3180-4cdc-9d51-8ca960c7bcd0%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQC5FF-F%3DLJzpsVUvcq1n%2B%2B_9DFcKgRFJ0r%3Dv3SS7jX_tQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: upgrade elasticsearch using the service wrapper?
There is no explicit documentation about where to place the data directory, but I find it best if you should keep as much as possible (configs/data/logs) from outside of the installation directory. I believe installing from a deb/rpm package might enforce a directory structure (never tried). The relevant config param is path.data https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L147 Cheers, Ivan On Mon, Dec 30, 2013 at 4:15 PM, allen chan allen.michael.c...@gmail.comwrote: I missed the part about placing the data directory outside of the elasticsearch installation. To correct it, will it be as simple as doing a mv command to move all the files to new dir and modify the elasticsearch config to use new dir? On Mon, Dec 30, 2013 at 2:48 PM, Ivan Brusic i...@brusic.com wrote: There should be no need to migrate the index data unless you are moving between two major Lucene versions. Since elasticsearch has never used Lucene 2, the old indices should be readable. Best practices dictate that you place your data directory outside of the elasticsearch installation so that you can maintain the same directory between installations. -- Ivan On Mon, Dec 30, 2013 at 2:39 PM, allen chan allen.michael.c...@gmail.com wrote: i never did an upgrade before. Are there instructions for migrating the index data? On Mon, Dec 30, 2013 at 1:35 PM, joergpra...@gmail.com joergpra...@gmail.com wrote: That is pretty straightforward, copy the service wrapper into the bin folder of the new ES version, adjust paths and symlinks, and there you are. Stopping and starting the cluster and migrating the index data is no difference to not using the service wrapper. Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEsGf%2BQKx-bm1sGoXX_O-Nfu_i2O4v%3DH9yiQcJYEg1B5Q%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- Allen Michael Chan -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPTHB%2BDu%3D8JtMg%3DsVmcD_ovzVvVGVV_cJGr-bo-xb-%3D6suUzAQ%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBYv4OEWiyGQSODxRvX-EgisJ%2BNvLR%2BKx%3DV40hO6rBDFg%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- Allen Michael Chan -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPTHB%2BAiGNd9UPda8kEQBTTiT6yq0x6jZAyy8NwfAo%3Dj%3DDurHQ%40mail.gmail.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBBEbg6c9pugPnJH2ymZUYo_eUs%3DYPBYSt6AR7AkBSo5g%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Dumb Question on bouncing nodes
Q: Is using the automatic node naming where every time I bounce a node I get a new node name interfering with restarting from the local gateway? Q Background: So previously, I asked about bouncing nodes, because it seems that whenever I bounce a node, it has to regenerate the node from the other nodes in the cluster. Someone told me I needed to lock shard allocation, then bounce, because the problem was that the cluster was reallocating the shards off of the node, so then the node wouldn't just come back up from the local gateway. I've tried that, and it doesn't seem to work. What happens is when the instance shutdown, all of its shards go into unallocated. Then when I startup the instance, the node comes up with zero shards until I unlock shard allocation. So the locking didn't seem to help. I'm wondering if the problem is because all of my nodes are named automatically using the marvel characters list, so essentially the node name changes each time. So the lock/unlock doesn't help, because the cluster is expecting a node with a specific name and when it doesn't find it, it initializes the node with no shards. Pierce -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5a6401b1-d6a9-4aad-807f-01f4c47a63d7%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Dumb Question on bouncing nodes
Unless I misread the post, your scenario seems to be the normal behavior. If you restart a node with allocation disabled, it will have no shards. Re-enabling allocation will force the shards on the node to be active. This behavior is at least what I have experienced. Perhaps there should be a setting to ignore cluster allocation settings for new nodes if it contains local shards that the cluster deems as unassigned. There was a recent commit that dealt with favoring local shards upon startup. https://github.com/elasticsearch/elasticsearch/issues/4237 Haven't upgraded to 0.90.8+ yet, but perhaps it addresses this issue. Cheers, Ivan On Tue, Dec 31, 2013 at 11:25 AM, Pierce Wetter obast...@gmail.com wrote: Q: Is using the automatic node naming where every time I bounce a node I get a new node name interfering with restarting from the local gateway? Q Background: So previously, I asked about bouncing nodes, because it seems that whenever I bounce a node, it has to regenerate the node from the other nodes in the cluster. Someone told me I needed to lock shard allocation, then bounce, because the problem was that the cluster was reallocating the shards off of the node, so then the node wouldn't just come back up from the local gateway. I've tried that, and it doesn't seem to work. What happens is when the instance shutdown, all of its shards go into unallocated. Then when I startup the instance, the node comes up with zero shards until I unlock shard allocation. So the locking didn't seem to help. I'm wondering if the problem is because all of my nodes are named automatically using the marvel characters list, so essentially the node name changes each time. So the lock/unlock doesn't help, because the cluster is expecting a node with a specific name and when it doesn't find it, it initializes the node with no shards. Pierce -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5a6401b1-d6a9-4aad-807f-01f4c47a63d7%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDN%3DMsQtK_n4pX0bD-%2B-Jv27ydcQKdyZ1erp2eNxAMLhQ%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Feedback on docs
Hi guys, I've recently gotten into Elasticsearch because the old search engine at my site http://ookaboo.com/ is horribly slow and I need something better. I've done a lot of work with Lucene and Solr in the past and particular I've been involved with projects that make very deep changes to those systems because we wanted to use them to drive a NER system or do statistical IR with advanced topic modeling. This project is nothing like that, it's just a very simple search engine that has to be easy to set up, easy to run, and easy to scale. Overall the quality of documentation is great and the amount of attention that is being paid to the getting started process is excellent, particularly when compared with Solr, but I have been looking at the docs for the java client API and there are some things I could use clarified... http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/index.html The big is one is that there are some cross-cutting patterns in the API I don't totally understand. For instance, * what is the difference between index() and prepareIndex()? * what is up with the execute(), actionGet() and get() methods of various sorts? * are javadocs available for IndexRequest() and similar objects? To put this all in context, so far I've had a great experience. Being able to just unpack elastic search on my Windows laptop or an AWS instance running Linux and start working is a real breath of fresh air! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a28c9e02-6d1c-472e-83da-16e1cc19b848%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
shard stucked in initializing state (elasticsearch crash test)
Hi, im doing crash test of small elasticsearch cluster): - 3 ubuntu micro instance (EC2, 3 zones) - 2 replicas (one per zone) - 10 indexes (with 10 shards per index) - 30k documents indexed in bulks (100 per batch) in parallel on every node - swap disabled im using micro instances to simulate a lot of crashes (i want to see cluster recovery in action) as could be expected it is crashing under load quite often (java process running out of memory) overall it is working surprisingly well (no data loss as for now) the only annoying thing is, that sometimes some shard gets stuck in *INITIALIZING *state (and _cluster/health shows *yellow*) i left cluster running for the night but it didnt recovered also restarting node with misbehaving shard didnt helped (it stuck in INITIALIZING state after restart too) any suggestion how to fix this ? Cheers, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b126ae21-ab16-4549-9331-5751f06fe496%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Dumb Question on bouncing nodes
Reading the description of that issue, I don't think they're the same. Looking at the code... Yeah, that's only about moving primaries around. So is there a way to restart a node without it going to zero shards and then back again? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25b969c7-ac3a-414a-b4e4-070b95b4857c%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Dumb Question on bouncing nodes
I do not think there is a way, but perhaps someone else can correct me. One potential way is to instead of disabling allocation, to set a timeout value high enough that the node can fully restart before the cluster drops the node. However, I am not sure if a node that is being cleanly shutdown will send an explicit message to the cluster. In that case, you would need a more abrupt method of stopping the node. Cheers, Ivan On Tue, Dec 31, 2013 at 12:27 PM, Pierce Wetter obast...@gmail.com wrote: Reading the description of that issue, I don't think they're the same. Looking at the code... Yeah, that's only about moving primaries around. So is there a way to restart a node without it going to zero shards and then back again? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25b969c7-ac3a-414a-b4e4-070b95b4857c%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB5S8FUcHC-6DdPgBcFSJtk0GRjsPWCzZH2RDFdMSS5-w%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Problem of ElasticSearch on ZFS
Hi we are running on Linux (ubuntu). The ZFS is set to standard (default). On Wednesday, January 1, 2014 12:43:35 AM UTC+8, Jörg Prante wrote: Do you run ZFS in sync mode? Are you on Solaris? Jörg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c32e5c6-1188-4551-b96c-2e1a3bd1e21e%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
