Hi Mark,
Thanks for your quick answer.
I cannot increase the RAM for ES, as I am already using 75% of the ram for
the JVM.
I will take a look at disabling the bloom filter cache to see if that
change anything.
Regarding the option of adding more nodes:
- Do you have an idea of how many nodes
When it comes to capacity the answer is, it depends.
Given you're at around 430GB on a single node now, I'd add another node and
then see how things look at around the 8-900GB mark (spread across both).
Another clarification; The recommended operating procedure is to use half
your system RAM for
Still waiting for a response from ES team for this issue.
On Monday, June 30, 2014 6:41:55 PM UTC+5:30, satishmallik wrote:
I am hitting same issue with 2 shards
curl localhost:9200/codeindexroute/_settings?pretty
{
codeindexroute : {
settings : {
index : {
index : {
Hi David,
I see ES as a basic component in a production system.
It should be very gentle with stop working scenario.
I would expect the cluster to heal its self and install the plugin in the
missing nodes.
It is much better then stopping the production.
On Monday, June 30, 2014 11:59:47 AM
I am using Elasticsearch with Webmaker and My server configurations
are Single core with 1 GB Ram. Everything runs fine at start but after
some time ES Crashes with Java memory Error and Webmaker also stopped
working. Do I need Bigserver with more RAM and Cores or there is
something wrong with ES
FYI this is a community mailing list, not a SLA based support channel.
If you are patient someone will answer when they can spare some time.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 1 July 2014 17:30,
You need more system resources, 1GB is a tiny amount of RAM for ES to run
on.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 1 July 2014 17:51, arshpreet singh arsh...@gmail.com wrote:
I am using Elasticsearch
Hey,
thx for your response.
As i already mentioned i tried setting the index.query.default_field to
message.
{
template : logstash-*,
settings : {
index.refresh_interval : 5s,
index.number_of_shards : 3,
index.number_of_replicas : 0,
index.refresh_interval :
On Tue, Jul 1, 2014 at 1:23 PM, Mark Walkom ma...@campaignmonitor.com wrote:
You need more system resources, 1GB is a tiny amount of RAM for ES to run
on.
How much more. Can please tell me specific configurations?
--
Thanks
Arshpreet singh
http://arshpreetsingh.wordpress.com/
--
You
No because it depends on a lot of things.
How much data do you want to store now, what about in the future? How many
and what sort of queries are you running and expecting? Do you expect to do
things like TTL or parent-child? What OS, Elasticsearch version and java
version are you on?
You really
Up ? Any ideas ?
Le lundi 30 juin 2014 17:48:54 UTC+2, Pierrick Boutruche a écrit :
Hi everyone,
I'm creating on my own a little Geocoder. My goal is to be able to
retrieve a big city or a country with a string on input. This string can be
mistyped, so I indexed geonames cities5000 data
Hi,
I have the following flow
RabbitMQ - *Logstash *(1.4.2) - *Elastic *(1.2.1) - *Kibana *(3.1.0)
Srv1: Ubuntu 12.04 Logstach Rabbit
Srv2: Ubuntu 12.04 Elastic
Srv3: Kibana Windows 2008 (IIS7)
My logs are coming from Syslog, text file or rabbit, in all cases data can
be found in elastic.
The
I have a question about ”backwards compatibility” in bellow page.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis.html
I would like to know how to configure with version parameter.
Please help me.
--
You received this message because you are subscribed to the
Hi Porin,
I never tried this configuration, but the answer is written in the bottom
of the link you added:
All analyzers, tokenizers, and token filters can be configured with a
version parameter to control which Lucene version behavior they should
use. Possible values are: 3.0 - 3.6, 4.0 -
I am trying to exercise the following example using Sense
:
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/match-query.html.
However when I ran
GET /my_index/my_type/_search
{
query: {
match: {
title: QUICK!
}
}
}
I got wrong scoring. I expect to see doc
Hi Rafi
Thank you for quick response.
I tried to add 1 line 'index.analysis.filter.myTokenFilter.version: 3.0' in
elasticsearch.yml and reboot ES.
But the version parameter is not reflected.
I confirm setting with this curl command curl -s
localhost:9200/hoge2/_settings\?pretty=1.
The
I have a bit of functionality where I occasionally need to refresh
documents so I can guarantee queries actually include recent modifications.
Disclaimer, I'm of course aware that this is not a great practice.
I'm using parent child relations and was wondering if I can restrict the
refresh
Thanks.
So default_index and default_search have special meaning.
Is this in the docs anywhere?
-N
On Monday, 30 June 2014 17:21:40 UTC+1, Glen Smith wrote:
Totally. For example:
analyzer: {
default_index: {
tokenizer: standard,
You should tweak cache sizes. At least the field data cache needs to be
restricted (unbounded by default). Also, ensuring the various circuit
breakers are turned on will help. Another tip is to disable the _all field
if you don't need it.
All this should reduce the amount of memory ES uses
We are trying to use elasticsearch-hadoop with our existing elastic search
cluster on 1.0.0. We are noticing that while running queries from Hive
(using ESStorageHandler), we are getting connection time out exception
while waiting on /_nodes/transport. The same happens intermittently when we
Hello,
I want to match only documents which match all positions.
My approach would be to index the number of positions and compare it to the
number of matched positions.
Every position that has multiple tokens (stacked tokens) should count only
as 1.
Given the following positions in a field:
You can also set it as a parameter in the body, just like a regular query:
$searchParams['index'] = 'my_index';$searchParams['type'] =
'my_type';$searchParams['body'] = [
'query' = [
'match' = [
'test_field' = 'abc'
]
],
'size' = 20]
$queryResponse =
Hello Robbie ,
I feel you are doing it in the wrong direction.
You can use wild card query -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html
But then in the example you have given , this might not work.
This is because , by default the
Hello Dany ,
I didnt really understand what you mean by position.
Kindly clarify.
Thanks
Vineeth
On Tue, Jul 1, 2014 at 5:18 PM, Dany Gielow dany.gie...@gmail.com wrote:
Hello,
I want to match only documents which match all positions.
My approach would be to index the number of
Hi Vineeth,
The position is determined by the PositionIncrementAttribute in a Lucene
TokenStream.
So when I say multiple tokens at the same position, I mean tokens that have
a positionIncrement of 0.
These tokens are generated by a SynonymFilter for example, which expands
all synonyms. All
Thanks both of you for the answer.
I have just decrease the Java Jvm memory to 50% (12G), I will see if that
helps.
@Jilles:
- I am using the defaut Logstash template and I things that by default the
_all field is disable...Ah no that not the case :( I will correct this
settings but why by
Hi Andrej,
Sorry for the late response. Didn't get an update email about it.
As long as you don't setup an instance with the 'elasticsearch::instance'
define it will only install the package but do nothing afterwards.
I recently fixed that the default files from the packages are being removed
Hi, Alex
Below is the output:
++ id -u
+ '[' 0 -ne 0 ']'
+ . /lib/lsb/init-functions
+++ run-parts --lsbsysinit --list /lib/lsb/init-functions.d
++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d
2/dev/null)'
++ '[' -r /lib/lsb/init-functions.d/20-left-info-blocks
I am having a problem setting up backup and restore part of AWS on S3.
I have 2.1.1 AWS plugin ElasticSearch V1.1.0
My yml:
cloud:
aws:
access_key: #
secret_key: #
discovery:
type: ec2
When I try to register a repository:
PUT
I believe http://localhost:9200/index/_mapping will give you types.
It is an indirect method for sure, but that kind of metadata is going to be in
memory and not require fielddata cache.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To
Hello Dany ,
This is ( surprisingly ) possible.
{
size: 30,
from: 0,
explain: true,
query: {
function_score: {
query: {
query_string: {
query: red,
fields: [
message
]
}
},
functions: [
{
My understanding was that ES 1.1 was using memory mapped files and so the field
cache would not be part of the heap.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
Setting up the JVM memory to 50% (12G) did not ease the problem as I
noticed GC collection up to 3min :)
Will really need to add a bunch of RAM to my machine..
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and
Hey guys,
I have a document with a nested field called field1.
I want to make an aggregation exist query by field1
Is this possible?
What i am trying to do:
GET viewsdata/tmoview1/_search
{
query: {
match_all: {}
},
aggregations:{
views:{
filter: {
With ES on a single machine, tuning does not cure the symptoms in the
long run. ES was designed to scale out on many nodes, so the simplest path
is to add nodes.
In a restricted environment, you could try to disable features that consume
a fairly amount of resources: disable _source and _all
Hello Vineeth,
Your script works like a charm. Thank you very much.
I will probably write a native script function, which will support multiple
query terms.
Thank you again
Dany
On Tuesday, July 1, 2014 3:27:36 PM UTC+2, vineeth mohan wrote:
Hello Dany ,
This is ( surprisingly )
Hi Jorg,
Thanks for the answer.
The idea behind the restriction of a single machine was for instance to
install ELK on a machine and perform fast indexing and review of a set of
log. What I got wrong is that the log size can be important (hundreds of
Gb) so this architecture will not work,
Hello ,
Exactly , go for a script which will check the position of each term and
add that to the score if that position is not already taken.
Thanks
Vineeth
On Tue, Jul 1, 2014 at 7:21 PM, Dany Gielow dany.gie...@gmail.com wrote:
Hello Vineeth,
Your script works like a charm. Thank
I am currently using logstash and elasticsearch to parse just one log file
(it would not be uncommon for the file to be 1gb). A 15mb file is taking 2
minutes to parse out with this configuration I have posted below (I have
also tried using no filter, which takes approximately 1 minute to
Hi all,
I have a query that specify the fields to be returned as described
here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html
However, it does not return the fields with a dot in their name, like
response.user.
For example,
Ex:
{
size:
GET requests use both the Lucene index and the transaction log to retrieve
documents. Search requests will use only Lucene since the inverted index is
not updated until the transaction log is flushed. I haven't paid too much
attention to the distributed aspects of the code in a while, but this
Hit reply too soon. The new segments should be available for search, but
these new segments are not created until the transaction log is flushed.
Even LinkedIn moved on from Zoie. The SNA group had many great projects,
but none of them got any traction.
--
Ivan
On Tue, Jul 1, 2014 at 8:02 AM,
Bump
On Monday, June 30, 2014 3:28:46 PM UTC-4, Andrew Gross wrote:
I was curious if there were any plans to update or modify the JSON query
API in ES 2.0+?
While I find the API to very powerful, it is confusing to construct a
valid request and requires special casing a lot of rules. I
Hi,
I'm indexing something like:
first doc= { v4 : myvalue }
second doc = { v1 : [ { v4 : myvalue }, { v5 : anothervalue } ] }
third doc= { v1 : [ { v2 : [ {v4 : myvalue }] } ] }
fourth doc = { v1 : [ { v2 : [ { v3 : [ { v4 : myvalue }] }
] } ] }
so nested dictionaries and array
Thanks, I guess this may be the perfect solution.
Jinyuan (Jack) Zhou
On Tue, Jul 1, 2014 at 6:23 AM, smonasco smona...@gmail.com wrote:
I believe http://localhost:9200/index/_mapping will give you types.
It is an indirect method for sure, but that kind of metadata is going to
be in
LinkedIn's unified search offering is described
at https://engineering.linkedin.com/search/did-you-mean-galene. Relevant
snippet:
Our professional graph evolves in real time, and our search results have
to remain current with these changes. Lucene supports changes to entities
by deleting
Hello Ben ,
Can you paste a sample feed.
Thanks
Vineeth
On Tue, Jul 1, 2014 at 8:26 PM, benq benoit.quart...@gmail.com wrote:
Hi all,
I have a query that specify the fields to be returned as described here:
Hello Giovanni ,
I feel this will help -
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_literal_multi_match_literal_query_2.html#_wildcards_in_field_names
Thanks
Vineeth
On Tue, Jul 1, 2014 at 10:19 PM, surfer sur...@crs4.it wrote:
Hi,
I'm indexing something
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-analyzers.html
On Tuesday, July 1, 2014 6:23:54 AM UTC-4, mooky wrote:
Thanks.
So default_index and default_search have special meaning.
Is this in the docs anywhere?
-N
On Monday, 30 June 2014 17:21:40
*I am sure
I am trying to use the FSRiver with elastic-1.2.1.
I installed the plugin as follows:
bin\plugin -install fr.pilato.elasticsearch.river/fsriver/1.0.0
I then tried to restart ES by doing:
bin\service.bat manager
Stop
Start
But afterwards, ES still has Service status: stopped. If I look
Hi all
Recently we've had a few spam emails that have made it through Google's
filters, and there have been a calls for us to change to a
moderate-first-post policy. I am reluctant to adopt this policy for the
following reasons:
We get about 30 new users every day from all over the world,
Does Elasticsearch support altitude in geospatial data type/query? I am
only able to find documentation referencing lat long (2D, not 3D). If
altitude is not supported now, any idea if it's in the development pipeline
and how soon it will be available? Any known workaround?
Thanks,
Akee
Yeah. Still on my TODO list to upgrade it!
:)
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 1 juil. 2014 à 20:24, Alain Désilets alaindesile...@gmail.com a écrit :
I am trying to use the FSRiver with elastic-1.2.1.
I installed the plugin as follows:
bin\plugin
OK, thx. Any timeline for when the upgrade will be available?
Alain
On Tuesday, 1 July 2014 14:57:54 UTC-4, David Pilato wrote:
Yeah. Still on my TODO list to upgrade it!
:)
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 1 juil. 2014 à 20:24, Alain Désilets
Sadly, I can't make any promise here. I think it will be before the end of July
though.
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 1 juil. 2014 à 21:06, Alain Désilets alaindesile...@gmail.com a écrit :
OK, thx. Any timeline for when the upgrade will be available?
For not analyzed fields, Is there a way of capturing the old behavior?
From what I can tell, you need to specify a tokenizer to have a token
filter.
On Tuesday, June 3, 2014 12:18:37 PM UTC-4, Karel Minařík wrote:
This is actually a change in Lucene -- previously, the long term was
Hello,
I'm using logtash + ES for storing logs
After I've updated ES to 1.2.1, I discovered strange behavior of searches.
*@source_host:mta01.domain1.com:*
shows msgs not only mta01.domains.com, but also from other hosts and domains
@source_host:mta01.domain1.com (759) count per 30s | (759
Do you have anything in your logs, i.e. many disconnects/reconnects?
Jörg
On Tue, Jul 1, 2014 at 7:59 PM, Venkat Morampudi venkatmoramp...@gmail.com
wrote:
In the elastic search deployment we are seeing random client node crashed
due to out of memory exception. I got the memory dump from one
Can you make this post sticky?
On Tuesday, July 1, 2014 2:36:44 PM UTC-4, Clinton Gormley wrote:
Hi all
Recently we've had a few spam emails that have made it through Google's
filters, and there have been a calls for us to change to a
moderate-first-post policy. I am reluctant to adopt
Hello,
I'm interested in using the EsOutputFormat class in a hadoop mapreduce
task.
During experimentation I have noticed that there is no direct handling for
'date' objects.
My data contains a number of 'date' fields which must be transposed into
the Elasticsearch index, however, I am
According to
https://github.com/elasticsearch/elasticsearch-cloud-aws/tree/es-1.1
you should use v2.1.0 of the plugin with ES 1.1.0.
On Tuesday, July 1, 2014 9:03:04 AM UTC-4, sabdalla80 wrote:
I am having a problem setting up backup and restore part of AWS on S3.
I have 2.1.1 AWS plugin
I think 2.1.1 should work fine as well.
That said, you should upgrade to latest 1.1 (or 1.2)...
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 1 juil. 2014 à 22:13, Glen Smith g...@smithsrock.com a écrit :
According to
This doesn't appear to be an ES specific issue, but I can see you've cross
posted this to the LS list so I'll reply there :)
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 2 July 2014 00:40, Erica
Are you using anything to monitor your cluster, plugins like ElasticHQ or
Marvel?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 2 July 2014 05:58, joergpra...@gmail.com joergpra...@gmail.com wrote:
Do you have
Extremely interesting. Should be ideal for a fixed set of terms. I am still
hoping for Matt Weber's pull request to be merged (
https://github.com/elasticsearch/elasticsearch/pull/3278) since I would
like to denormalize against a dynamic set of terms, but your approach would
be more efficient if
Thanks for reply Jörg. I don't have any logs. I will try to enable them it
would but it would take some time though. If there anything in particular
that we need to enable, please let me know.
-VM
On Tuesday, July 1, 2014 12:58:21 PM UTC-7, Jörg Prante wrote:
Do you have anything in your
Hi Mark, is the any particular aspect of monitoring or monitoring in
general?
-VM
On Tuesday, July 1, 2014 3:40:23 PM UTC-7, Mark Walkom wrote:
Are you using anything to monitor your cluster, plugins like ElasticHQ or
Marvel?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign
Apartments simplicity or understood do it we. Song such eyes had and off.
Removed winding ask explain delight out few behaved lasting. Letters old
hastily ham sending not sex chamber because present. Oh is indeed twenty
entire figure. Occasional diminution announcing new now literature
Apartments simplicity or understood do it we. Song such eyes had and off.
Removed winding ask explain delight out few behaved lasting. Letters old
hastily ham sending not sex chamber because present. Oh is indeed twenty
entire figure. Occasional diminution announcing new now literature
The plugins I mentioned before give you insight into cluster state,
specifically around GC. They would be worth installing if you want to
troubleshoot this.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 2 July
You're on Java 6 by the looks of it, ES won't run on anything less than 7.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 1 July 2014 22:57, Steven Yue jinche...@gmail.com wrote:
Hi, Alex
Below is the output:
Thanks Mark. I will look into installing the plug-ins that you suggested.
-VM
On Jul 1, 2014, at 8:11 PM, Mark Walkom ma...@campaignmonitor.com wrote:
The plugins I mentioned before give you insight into cluster state,
specifically around GC. They would be worth installing if you want to
Hello All,
I want to sort the search hits based on two criteria - (1) value of an
attribute, and (2) _score. The first criteria is such that if the attribute
has two values open and closed, the search hits with value open
should be ranked above those with value closed. What is the best way to
Hi ,
I feel you should write a script and use that in the function_score.
So assuming _score is normalized to a value between 0 and 1.
value of closed = 1
value of open = 2
Score of a document = value of attribute + _score.
Thanks
Vineeth
On Wed, Jul 2, 2014 at 9:27 AM, bsa
Hi all,
I'm testing percolator performance, 50k/s is required condition with 3~4k
rules.
now I only have 1 simple rule, and 5 es vms with 1 shard and 4 replicas.
and using Java transport client like below
new TransportClient(settings)
Brian,
On 30 juin 2014, at 22:59, Brian wrote:
Well, I did answer your question. But probably not from the direction you
expected.
hmm no, you didn't. My question was: it looks like I cant retrieve/display
[_all fields] content. Any idea? and you replied with your logstash template
where
Are you using same physical machine for all your VMs?
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 2 juil. 2014 à 07:09, Seungjin Lee sweetest0...@gmail.com a écrit :
Hi all,
I'm testing percolator performance, 50k/s is required condition with 3~4k rules.
now I only
Also, what's the hardware configuration of the devices? RAM? CPU? Disk? OS?
How much data is in your ES instance(s) already? and are you sure you're
not hitting a limit on something like client side networking?
Thanks,
Patrick
Patrick
79 matches
Mail list logo
