Hi,
I am using elasticsearch 1.0.0 having a cluster of 7 nodes. (3 master, 2
data 2 client nodes)
*The problem i am facing that on data nodes .hprof files are being
generated which is huge in size* : datanode1: around 9gb datanode2:
around 3gb
While in logs of data nodes these lines are
*On the other side, i have never got OutOfMemoryException ony any node.*
On Tuesday, 10 June 2014 11:32:36 UTC+5:30, Bharvi Dixit wrote:
Hi,
I am using elasticsearch 1.0.0 having a cluster of 7 nodes. (3 master, 2
data 2 client nodes)
*The problem i am facing that on data nodes .hprof
I have rails app where I try to use both elasticsearch-rails and
elasticsearch-model gems.
I'm getting results but facets aren't working. I wondering if anyone can
pinpoint on what I am doing wrong.
Here is my code that is written but example from elasticsearch team
Hi - We are designing a system for reporting and are planning to use
Elastic search as a backend. We want to expose reporting in such a way that
users can build custom reports on top of their data without us coming in
their way. One way to do this is to expose elastic search query APIs
through
I'm having a very odd problem with one of my elasticsearch clusters. The
master node on the cluster crashes randomly. The cluster is running on 3
different ec2 instances.
My cluster configuration is:
3 nodes (1 master, all data nodes)
600 GB of data (3k IOPS EBS volumes)
700 million documents
How much RAM per node, what java flavour and version, what ES version?
Are the logs showing any OOM?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 10 June 2014 17:16, Gaurav Arora gauravswo...@gmail.com wrote:
I had a problem with corrupted shards so I restarted my cluster with
index.shard.check_on_startup: fix and the corrupted shards were fixed
(i.e. deleted). Unfortunately the replicas and primaries then had differing
numbers of documents despite them all being green. Fortunately the
primaries
It depend on your requirements and your product strategy - both is possible
with pros and cons:
- are your users proficient in a report language? Do they already write
report specs in a standard report language? Do you want to support this
report language standard? Do you like to share report
I am using the latest openjdk version 7 installed from ubuntu repos.
ubuntu@es1:~$ java -version
java version 1.7.0_51
OpenJDK Runtime Environment (IcedTea 2.4.6) (7u51-2.4.6-1ubuntu4)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
ES is set to run with -Xms14075m -Xmx14075m with
Try add
data : false
to the node that you would like to remove data (you are trying to move from
node4 - node3 right?).
I'm not sure, but when operation succeed, data will be copied to node3,
*but* it *as far as I know* dhe data will stay on node, To be sure compare
folder size, and if
*Try add data : false*
You can do it in you elasticserh.yml
# You can exploit these settings to design advanced cluster topologies.
#
# 1. You want this node to never become a master node, only to hold data.
#This will be the workhorse of your cluster.
#
# node.master: false
# node.data:
Hi,
I need to perform a query + filter on child documents. For the query, I'm
using TopChildren.
Now I wonder what would be more efficient regarding query on *date/numeric
(no score needed) fields* of this child -
should I query on these fields using a HasChild filter in a bool query
I am trying to delete a month of logstash indexes, and fire off e.g. this
command:
curl -XDELETE http://localhost:9200/logstash-2014.05*?pretty;
which returns within a few seconds (less than a minute - the default
timeout afaik) with:
{
acknowledged : true
}
But when I look at the indexes,
I need to implement the below function_score query using Java APIs. I
couldn't find any official documentation for function_score query in the
Java API section of elasticsearch
function_score: {
functions: [
{
boost_factor: 3,
filter: {
terms
curl -XGET 'http://localhost:9200/_nodes/_all/process?pretty=true' |less
Have you been considering max_file_descriptors?
W dniu wtorek, 10 czerwca 2014 09:36:35 UTC+2 użytkownik Gaurav Arora
napisał:
I am using the latest openjdk version 7 installed from ubuntu repos.
ubuntu@es1:~$ java
Hi,
I have a question about subaggregations.
The case is that I have small documents (single opratations) with duration
time of this operation which I want to aggregate and finally get the
percentile of this sum.
Example:
| operation | time |
| A | 10 |
| A | 20 |
| B
Hello,
This is my first post to this groups, so welcome everybody.
I am evaluating Logstash+Elasticsearch+Kibana combo as a tool set for
collecting data from performance tests which I could later analyse visually
using Kibana.
I found two main features missing and I am not sure if this is
The max file descriptors are all set to 64k.
This is the output from one of the slave nodes -
http://pastebin.com/RdmZsJbH
On Tue, Jun 10, 2014 at 2:50 PM, sirkubax jakubxmuszyn...@googlemail.com
wrote:
curl -XGET 'http://localhost:9200/_nodes/_all/process?pretty=true' |less
Have you been
Heya,
We are pleased to announce the release of the Elasticsearch JavaScript language
plugin, version 2.2.0
The JavaScript language plugin allows to have javascript as the language of
scripts to execute..
Release Notes - Version 2.2.0
Update
[20] - Update to elasticsearch 1.2.0
Issues, Pull
Hi,
Is it possible to restrict Kibana users from accessing particular
indices/sub indices folders using the Jetty plugin?
Regards,
Gaurav
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails
Something like this would be super useful :)
If no one else can provide an answer and you're willing, you could always
code it up and submit a pull request, or alternatively, raise a feature
request.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
Thanks Ivan for the tip, but I think the boost_mode is just fine in my
queries. The problem is that I only can access the field of the child
document, if I have an additional bool part query with the has_child query
inside. This causes the sum. The custom score is multiplied with the
has_child
Heya,
We are pleased to announce the release of the Elasticsearch Python language
plugin, version 2.2.0
The Python language plugin allows to have python as the language of scripts to
execute..
Release Notes - Version 2.2.0
Update
[11] - Update to elasticsearch 1.2.0
Issues, Pull requests,
Heya,
We are pleased to announce the release of the Elasticsearch Groovy language
plugin, version 2.2.0
The Groovy language plugin allows to have groovy as the language of scripts to
execute..
Release Notes - Version 2.2.0
Update
[27] - Update to elasticsearch 1.2.0
Doc
[24] - Add
i am searching with the below keyword no fat within double quotes but
search will returns the fat results also. Please advise how to avoid the
fat result i want only no fat.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from
I am getting a stack trace when I run the routing fix tool against my index.
Is this a known issue?
Kind regards,
Luke
java -jar elasticsearch-fix-routing-1.0.jar 10.10.9.14 9300 global count
Jun 10, 2014 1:03:16 PM org.elasticsearch.plugins
INFO: [Choice] loaded [], sites []
Index: global,
On our 4 node test cluster (1.1.2), seemingly out of the blue we had one
node experience very high cpu usage and become unresponsive and then after
about 8 hours another node experienced the same issue. The processes
themselves stayed alive, gc activity was normal, they didn't experience an
Please help us. We are trying to build few reports using your tool through
ASP.NET Web application. We don't know what is the process. Please request
help us and provide few sample applications to build reports through asp.net
web.
--
You received this message because you are subscribed to
Hello,
maybe someone can help me. Is there a way to get the available search
templates via rest api? havent found a way yet, hope you can help me.
Best regards
Sebastian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this
Hi All,
I am using the rabbitmq-river plugin for elasticsearch. My configuration
for the river is as follows:
curl -XPUT 'localhost:9200/_river/rabbit_river/_meta' -d '{
type : rabbitmq,
rabbitmq : {
host : lbha1.ir.clemson.edu,
port : 5672,
user : guest,
Thanks Mark.
We are using Java version - 1.7.0_25
What is your document size? I'm wondering if our document size i.e. 144 KB
is causing the low TPS.
Thanks
Pranav.
On Monday, June 9, 2014 6:29:19 PM UTC-4, Mark Walkom wrote:
One thing you never mentioned was what version of Java you are on,
Hi Thomas,
I am following the same steps as you listed.
I had one query I would be thankful if you can help me out.
After copying the data and config folder to the latest elasticsearch
version when I restart the elasticsearch 1.1.1 server,
I see two nodes in the same cluster(instead of one).
I would like to use MMapDirectory at the data indexing phase (in a batch).
And then switch to index to in-memory and read only at time of serving real
user queries to optimize the search latency. I used to achieve that when
directly deal with Lucene by using RAMDirectory and read-only Searcher.
We currently run our Elasticsearch (*v1.0.2*) cluster on *3 Nodes* with *5
Shards and 1 Replication* Scheme. The total index size is about 70GB
(~140GB with replication).
The Empty Search (/_search) query takes 500-600 ms to respond. Will adding
in more Nodes help in this case? The Servers
I imagine that depends on lots of stuff. Are you doing
elasticsearch:9200/_search or elasticsearch:9200/index/_search ? The
former can take quite a while if you have lots index and lots of shards.
If you can get away with not doing it, I would. The latter will only take
a long time if you have
I am currently running only 1 index with 5 shards. So the both of those
queries yield the same response time. My main question is to understand if
scaling out is an Option given the current replication scheme.
Short answer: yes.
Long answer: 500ms is a long time for the empty query. I see 2ms from
elasticsearch and 23ms from time in development. In production I see maybe
54ms from elasticsearch and 70 from time across far far more shards and
more data. When I do the same query across thousands of
I've already read a lot about installing and setting of elasticsearch. Most
of them are for non-windosw OS. However, it seems the principle is sort of
the same thing. So I followed one site
http://www.elasticsearchtutorial.com/elasticsearch-in-5-minutes.html#Indexing
My OS is windows 7. I've
Referencing the post at
https://groups.google.com/d/msg/elasticsearch/L9WtITL63Lo/kGi1rTWbSbIJ I am
curious:
To install Kibana as a site plugin, it says to Try to install it under
/plugins/kibana3/_site
Does this mean I should have an installation as follows:
Try this
import org.elasticsearch.action.search.SearchRequest;
import
org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder;
import java.util.Arrays;
import static org.elasticsearch.client.Requests.searchRequest;
import static
Hi,
I wish to match a large number of terms (100's of strings) from a large
set of logs (1,000,000s of entries). I what to ask if this is something
that elasticsearch can do and secondly how best to go about it? This is
something that will only be run a few times a year so performance is not
Hi Jörg,
Does ES allow switching from file-based store to memory-based store without
re-indexing? We used to use file-based store to run a batch index, then
switch to read-only memory-based index when using Lucene directly. We've
found that read-only RAMDirectory is 20% faster than
thanks a lot. It finally works.
在 2014年6月10日星期二UTC+2下午11时13分41秒，Jörg Prante写道：
Just install Cygwin https://www.cygwin.com/ and leave Windows crappy
console behind.
Jörg
On Tue, Jun 10, 2014 at 9:31 PM, Aaliyah zhang...@gmail.com javascript:
wrote:
I've already read a lot about
I am just trying to get Kibana3 up and running, and the shortest and
easiest evaluation path seemed to be using Node.js.
The ES Downloads page's Kibana archive does not contain a server.js file,
and the following site contained the only method that did, and also
referenced the ES github
As an aside, I am also wondering why this link
http://www.elasticsearch.org/downloads/1-2-0/ is still active and
available when it was supposed to be pulled.
Brian
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group
I simplified the actual problem in order to avoid explaining the domain
specific details. Allow me to add back more detail.
We want to be able to search for multiple points of user action, towards a
conversion funnel, and condition on multiple fields. Let's add another
field (response) to the
You will likely see an increase by distributing it to one shard per
machine, but that's hard to quantify without actually doing it.
Also, you may be doing yourself a disservice with such a large heap size as
Nik mentioned. Over 32GB, Java pointers are not compressed and you do lose
a bit of
Probably because it contains the release notes etc.
You can't download any of the files from the links, though a note about it
being removed would be handy I guess.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 11
There are a few people in the IRC channel that have done it, however,
generally, cross-WAN clusters are not recommended as ES is sensitive to
latency.
You may be better off using the snapshot/restore process, or another
export/import method.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign
Hi Pradeep,
We are in the middle of doing the same thing, designing a system for
reporting. And I want to create a middle API layer for the reasons you
suggest and other reasons. I would like to exchange notes with you in a
private message, if you want. You have to create some middle later,
The Heap Size is being reduced to 30GB to ensure that's not the bottleneck.
The servers currently run SAS Drives. Though SSDs are usually preferred for
Elasticsearch, can this cause such disparities in performance? ElasticHQ
reports very high Refresh Rates, Search-Fetch and Search-Query rates.
Are you using a monitoring plugin such as marvel or elastichq? If not then
installing those will give you a better insight into your cluster.
You can also check the hot threads end point to check each node -
Hi Mark,
With Java 7, are pointers compressed by default.
Other JVM Settings
-
-XX:UseCompressedOops
-
Compressed oops is supported and enabled by default in Java SE 6u23
and later.
In Java SE 7, use of compressed oops is the default for 64-bit JVM
SSDs help, though there is likely some other issue here so it's probably
not worth looking at, at this time.
Have you checked hot threads or the slow query log?
Can you provide more specs on your hardware? What java version are you
running?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign
Hey Mark,
Thanks for this. It seems like our best bet will be to manage indexes
the same across all regions, since they're really mirrors. Since our
documents are immutable, we'll just queue them up for each region, which
will insert or delete them into their index in the region. It's the
You could look at using a queuing system, like rabbitmq, where your
application drops the data into, then have a logstash instance in each DC
that pulls off the queue and pushes into ES.
That way you can easily handle the replication of the data to multiple
endpoints within rabbitmq.
Regards,
Hey Guys,
How can I map an arbitrary map of key/values in ES? My JSON looks like the
following, where name and age are static but attributes is dynamic:
{
name: john,
age: 25,
attributes : {
key1: value1,
key2: value2,
key3: value3,
...
}
}
Things to consider:
1. Not
using elasticsearch wants to create thai search service.
follow is the requirements.
1. we must be consist of effective index by stemming.
2. pre-registered terms of foreign loan words are must be included to the
index.
3. pre-registered terms of stopword dictionary must be excluded from the
I am building a system that uses Elasticsearch to store and retrieve
library catalogue data. One thing I've been asked for is a browse interface.
Here's a definition of what this is:
- The user does a search, for example Author starts with and they
supply Smith
- The system puts them
59 matches
Mail list logo
