ref..
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
Everything was working fine when all of a sudden some indices started
failing.
*GET localhost:9200/logstash-2014.09.11/_search*
yields response:
{error:SearchPhaseExecutionException[Failed to execute phase [query],
all shards failed],status:503}
--
You received this message because you are
On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand kevin.del...@gmail.com wrote:
Everything was working fine when all of a sudden some indices started
failing.
GET localhost:9200/logstash-2014.09.11/_search
yields response:
{error:SearchPhaseExecutionException[Failed to execute
Hello,
I need to store in a consistent way the role/groups that can access the
information but I'm not sure what's the best way to do it.
Summary: I have 2 kinds of docs tweet and blog:
- At tweet level, I store the group name allowed to access the
information
- blog is more complex,
On Thursday, September 11, 2014 at 22:50 CEST,
shriyansh jain shriyanshaj...@gmail.com wrote:
I am using ELK stack and have a cluster of 2 elasticsearch nodes. When
I am querying Elasticsearch from kibana. I am getting the following
log error message in the elasticsearch log file.
Cluster health is red:
https://gist.github.com/kevindeland/2d727c3d984139ab96d4
On Friday, September 12, 2014 2:57:06 AM UTC-4, Magnus Bäck wrote:
On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand kevin@gmail.com javascript: wrote:
Everything was working fine when all of
Hi,
I am trying to find numbers of discrete value per URL in a day and the
result is not what I expect.
So let's say I have an index which contains such document:
{
date: ...,
url: ,
other...
}
And basically I am trying to group by url for a particular date:
{
query:
{
OK, it seems that I need to use not_analyzed on the field. Is that correct?
On Friday, 12 September 2014 08:18:19 UTC+1, Ali Kheyrollahi wrote:
Hi,
I am trying to find numbers of discrete value per URL in a day and the
result is not what I expect.
So let's say I have an index which
There are six indices with a red cluster status, but only two fail... any
advice on what to check?
On Friday, September 12, 2014 2:57:06 AM UTC-4, Magnus Bäck wrote:
On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand kevin@gmail.com javascript: wrote:
Everything was
Hello!
Can anyone shine some light on my question?
Is the query in question achievable in ES directly?
If not, I can probably do that in application later, but it would be nicer
if ES could serve me the final results.
Matej
--
You received this message because you are subscribed to the
I am parsing a logfile using logstash. But somehow logstash is not parsing
whole log file
attaching the error dump.
I have also attached the my logstash config file. Please help
root@ryudt-023:/etc/logstash/
conf.d# /opt/logstash/bin/logstash agent -f akamai-log.conf
Using milestone 2
You should really ask that on the logstash group -
https://groups.google.com/forum/?hl=en-GB#!forum/logstash-users
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 12 September 2014 18:27, Atul K
anyone?
On Thursday, September 4, 2014 9:44:29 PM UTC+8, Jason Wee wrote:
Hello ES,
With curl showing the index statistics as below:
$ curl 'http://localhost:9200/_cat/indices?v'
health index pri rep docs.count docs.deleted store.size pri.store.size
green twitter 1 0 0
Hi,
Again I have an issue with the power of the cluster.
I have the cluster from 3 servers, each has 30RAM, 8 CPUs and 1Tb disk
attached.
https://lh4.googleusercontent.com/-W1AVatn9Cq0/VBKzYgR3QKI/AJc/S3TWMBqqqX0/s1600/ES_cluster.png
There are 1323957069 docs (1.64TB) there, the
That's a lot of data for 3 nodes!
You really need to adjust your infrastructure; add more nodes, more ram, or
alternatively remove some old indexes (delete or close).
What ES and java version are you running?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email:
On Friday, September 12, 2014 at 09:23 CEST,
Ali Kheyrollahi alios...@gmail.com wrote:
On Friday, 12 September 2014 08:18:19 UTC+1, Ali Kheyrollahi wrote:
I am trying to find numbers of discrete value per URL in a day and
the result is not what I expect.
[...]
Result is bizarre,
Java version is 1.7.0_55
Elasticsearch is 1.3.1
Well, the cost of the whole setup is the question.
currently it's something about 1000$ per month on AWS. Do we really need to
pay a lot more then 1000$/month to support the 1.5Tb data?
Could you briefly describe how much nodes do you expect to
Hi,
Not sure if this is the right user group, but here goes:
I'm planning to use ElasticSearch.net as the client for connecting to my ES
cluster. I have one question I haven't been able to find the answer to. I
know that the ConnectionPool feature can check if nodes fail, but can the
client
The answer is it depends on what sort of use case you have.
But if you are experiencing problems like you are then usually it's due to
the cluster being at capacity and needing more resources.
You may find it cheaper to move to more numerous and smaller nodes that you
can distribute the load
Do you say, that 10 servers like 2 CPU, 7.5 RAM (so totally 20 CPUs and
75Gb RAM) cluster would be more powerful then the 3 serves of 8 CPU and 30
RAM (in total 24 CPU and 90RAM) ?
Assuming that the information would be spread there equally.
btw, what about the shards allocation. Currently I
With the same URL I am able to get the json from curl command.
Full url is http://10.xxx.66.xxx:6xxx8/ea/api/discovery.json but with
elasticsearch the Status is N/A. I don't know why this is happening.
Regards,
Nimit
On Friday, 12 September 2014 12:29:39 UTC+5:30, Magnus Bäck wrote:
On
Also, can we mention the username and password of the url from python in
ElasticSearch.
On Friday, 12 September 2014 15:21:50 UTC+5:30, Nimit Jain wrote:
With the same URL I am able to get the json from curl command.
Full url is http://10.xxx.66.xxx:6xxx8/ea/api/discovery.json but with
what is the curl comman you use to reach elasticsearch?
On Fri, Sep 12, 2014 at 11:51 AM, Nimit Jain online.ni...@gmail.com wrote:
With the same URL I am able to get the json from curl command.
Full url is http://10.xxx.66.xxx:6xxx8/ea/api/discovery.json but with
elasticsearch the Status is
I want to close this issue but I still do not understand if I should be
pushing documents from my database using the PHP client or using the JDBC
river to pull them into elasticsearch from the SQL database.
They can both achieve the same thing, but what is the usecase which defines
when is
You can use either style, it is a matter of taste, or convenience.
With the JDBC plugin, you can also push data instead of pull.
Jörg
On Fri, Sep 12, 2014 at 12:11 PM, James m...@employ.com wrote:
I want to close this issue but I still do not understand if I should be
pushing documents from
As I initially mentioned, it all depends on your use case but generally ES
does scale better horizontally rather than vertically. If you can, spin up
another cluster along side the one you have and then replica the data set
and query usage and compare the performance.
Ideally you should aim for
I am not think so that I have reached Elasticsearch using this. But below
is the command that I have used.
curl -i -XGET 10.xxx.66.xxx:6xxx8/ea/api/discovery.json -d '
There is no proxy in between the client and server but yes it do have the
login which we are not doing right now. Could you
I am sorry, I cannot help you. Elasticsearch requires no password and
the URL you supplied doesn't correspond to any API in Elasticsearch
(unless you are pointing to a single document). I assume there is
something weird with your setup.
On Fri, Sep 12, 2014 at 1:01 PM, Nimit Jain
Can you please tell me the step to do that from starting. That will be a
great help.
Regards,
Nimit
On Friday, 12 September 2014 16:35:11 UTC+5:30, Honza Král wrote:
I am sorry, I cannot help you. Elasticsearch requires no password and
the URL you supplied doesn't correspond to any API in
Regarding the shards, if you have 3 nodes and 1 index, with 5 shards you
have a sort of impedance mismatch because 5 (or 10 with replica) shards
do not distribute equally on 3 nodes.
Rule: use a shard count that is always a factor of the node, e.g. 3, 6, 9,
12 for 3 nodes.
Can you tell what
Hi All,
we currently have an Elasticsearch (1.1.1) cluster distributed among DCs
* 3 Data nodes in 3 DC
* 1 Gateway node
each DC index its own data and no shards replica is happening between DC.
The gateway lets us to query all the indexes in all the DC.
Now, for performance and redundancy we
Hello there,
I am trying to write a rather complex aggregation
Let's say my json documents contains the following fields: timestamp,
username, subject
The search should return documents where:
- two identical subject fields,
- by the same username,
- within an interval of X minutes.
Using
Hi,
I have a cluster with nodes configured with a 18G heap. We've noticed a
degradation in performance recently after increasing the volume of data
we're indexing.
I think the issue is due to the field data cache doing eviction. Some nodes
are doing lots of them, some aren't doing any. This
Forgot to mention that we're using ES 1.1.1
On Friday, September 12, 2014 9:21:23 AM UTC-4, Philippe Laflamme wrote:
Hi,
I have a cluster with nodes configured with a 18G heap. We've noticed a
degradation in performance recently after increasing the volume of data
we're indexing.
I
Hi,
One of my servers appears to be feeding nonsense into Fluentd which is then
ending up in elastic search.
Is it possible to use regex in queries ?
The syslog message content is always the same they start with numbers
followed by close bracket, etc.
123)
89)
203)
Is there a way to
Hi,
Thank you for the reply. Here is an example of a scoring behavior I'm
talking about:
a) given a user query a set of documents is produced. Let's call
this set S.
b) suppose each document has a numeric field called F.
The average of this field values for the set of
You cannot join documents in Lucene/Elasticsearch (at least not like a
RDBMS). You would need to either denormalize your data, join on the client
side or execute 2+ queries.
--
Ivan
On Fri, Sep 12, 2014 at 12:45 AM, matej.zerov...@gmail.com wrote:
Hello!
Can anyone shine some light on my
Hi ,
If this pattern is a single word , regex query might do the trick -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html#query-dsl-regexp-query
Thanks
Vineeth
On Fri, Sep 12, 2014 at 7:35 PM, Log Muncher railroaderslam...@gmail.com
you can combine ES with RDBMS, and run your SQL queries either directly
against db, or pull data via JDBC River into ES, I wrote about it here:
http://lessc0de.github.io/connecting_hbase_to_elasticsearch.html
On Fri, Sep 12, 2014 at 10:55 AM, Ivan Brusic i...@brusic.com wrote:
You cannot join
Hi:
When I have a cluster state that looks like this...which wins? Is there
any way to clear a setting? Guessing I need to wait
until https://github.com/elasticsearch/elasticsearch/issues/6732 but what
do I do in the meantime? Set both?
{
persistent : {
cluster : {
routing :
Hi:
Is there a place to submit suggestions for Marvel enhancements? It would
be nice if the Cluster Name was included in the Browser Title...
Thanks
Andy O
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and
How were these nodes doing in terms of available heap space before the
disconnects occurred?
On Wednesday, September 10, 2014 6:26:19 AM UTC-4, Israel Tsadok wrote:
A temporary network disconnect of the master node caused a torrent of
RELOCATING shards, and then one shard remained
I would strongly prefer to maintain control of the indexing side and not in
Elasticsearch. In fact, the Elasticsearch team has talked about deprecating
river plugins. I do not have any numbers, but I would suspect that the
majority of users do not use a river plugin. And yes, the correct term is
I must admit I'm new to this so I find some of the information hard to
understand. So sorry if I am asking stupid questions.
On 12 Sep 2014, at 18:26, Ivan Brusic i...@brusic.com wrote:
I would strongly prefer to maintain control of the indexing side and not in
Elasticsearch. In fact, the
Hi All,
I configured elastic search to use search and visualize the hadoop log
files.
I have hadoop cluster with 8 nodes and i use eco systems Hive/Pig to push
the log files to elastic search.
I configured elastic search successfully(single node) and i started kibana
as well.
When i try to run
I have ES 1.0.2 and Lucene 4.6.
I was hoping preference=_local would query on the current node, but it does
not.
The same node comes up at the top of the list for all shards regardless of
which node I search from,
it is not the primary node, neither is it the local node.
I am testing
Hi,
We're deploying ES for logstash and I recently setup the cluster to migrate
older indexes to slower nodes for longer term storage. The nodes are not
being queried or doing any indexing but CPU is at 75% constantly. Here is
output from hot_threads , jstack and some settings
$ java
Looks like you have a monitoring tool running and it got stuck in the node
stats call while traversing a number of shards/segments.
How many shards/segments are in your migration? It seems to be very active.
Maybe the bloom filter format conversion is expensive but I am not sure.
Jörg
On Sat,
Personally, I'd go with the latter and then let the software handle all the
redundancy. You can get super cheap 1RU pizza boxes from Quanta or the like
and save yourself a bundle in that area and then leverage automation and
configuration using The Foreman and Puppet.
Tie a bit more smarts into
It's a little unclear what you are doing.
You indicate you have a single cluster but then no replication happens
between the nodes in each DC?
By gateway do you mean a tribe node?
And by replicate the local indexes do you want to set replicas = 1 and then
have them on this new node in each DC?
Not sure what is extreme. The design of ES may be a surprise for those
who are not familiar with distributed system architecture.
ES can handle faults in software. I pile up cheap 1U rack servers with 32
cores, 64G RAM, ~1TB RAID 0. All nodes are equally provisioned.
If a server fails, mostly
Thanks, it ended up being the Ganglia plugin that was causing crazy CPU
consumption. I've disabled it since we'll end up buying Marvel once we've
fully deployed.
On Friday, September 12, 2014 6:36:27 PM UTC-4, Jörg Prante wrote:
Looks like you have a monitoring tool running and it got
Hi all,
If there's any api to clear all the deleted documents on disk?
I read that
Deleting a document doesn’t immediately remove the document from disk — it
just marks it as deleted.
Elasticsearch will clean up deleted documents in the background as you
continue to index more data.
By
Elasticsearch is no different than any other data store: your application
can add data by using the prescribed methods. Every data store has some
sort of data input method. Elasticsearch allows river plugins, which mean
that the Elasticsearch process can pull data instead of the standard push
Hello Wei ,
You can use the in optimize API - max_num_segments as 1 or
only_expunge_deletes as true .
OPTIMIZE -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-optimize.html#indices-optimize
Thanks
Vineeth
On Sat, Sep 13, 2014 at 5:32 AM, Wei
Hello Lasse ,
Following is my idea on the whole thing -
Routing -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-routing
When a index request comes , based on the ID of the request , a hash
function computes the shard to which the request has to be
We are using elasticsearch as back-end for our in-house logging and
monitoring system. We have multiple sites pouring in data to one ES cluster
but in different index. e.g. abc-us has data from US site, abc-india has it
from India site.
Now concerns are we need some security checks before
Hello Jigish ,
I dont think you can achieve all of these in Elasticsearch.
You can restrict the HTTP methods to GET and POST in Elasticsearch.
But for most of other tasks , Nginx would be a better option.
Elasticsearch jetty plugin might also help you -
Hi all!
We have been using ElasticSearch to migrate some of our clients from
other solutions to the ELK stack for monitoring their environments to
collecting other data. Many has used reporting systems and where not happy
with only the - excellent - Kibana dashboards.
In time we
Hi,
Can someone help me to fix this issue.
Many thanks in advance!
Regards,
Muthu
On Friday, September 12, 2014 11:27:03 PM UTC+5:30, Muthu Kumar wrote:
Hi All,
I configured elastic search to use search and visualize the hadoop log
files.
I have hadoop cluster with 8 nodes and i use
Thanks for sharing that with the community. Nice job guys.
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 13 sept. 2014 à 06:00, Fabio Torchetti mr...@wedjaa.net a écrit :
Hi all!
We have been using ElasticSearch to migrate some of our clients from other
solutions
Thanks Vineeth. I will look into suggested plugin.
On Saturday, September 13, 2014 9:10:10 AM UTC+5:30, vineeth mohan wrote:
Hello Jigish ,
I dont think you can achieve all of these in Elasticsearch.
You can restrict the HTTP methods to GET and POST in Elasticsearch.
But for most of other
62 matches
Mail list logo
