Hi ES users,
In ES 1.1.0 there is an add-on parameter Locale with query_string query so
what exactly this parameter is all about?
source: https://github.com/elasticsearch/elasticsearch/pull/5131
https://github.com/elasticsearch/elasticsearch/pull/5131
As I didnt get much information from docs
Hi,
I'm trying out the new cardinality aggregation, and want to measure the
accuracy on my data. I'm using a dataset of a day of sample tweets (2.8m
tweets).
I'm counting the number of unique usernames per language.
To get my reference unique count I use this:
GET /twitter-2014.03.26/_search
{
Hi,
I have an index named 'product' it has two types named 'type1' and 'type2'
For example:
*{ product_id: , price: 23.56, stock: 100 }*
*{ product_id: , category: iPhone case, manufacturer: Belkin }*
I want them to join based on 'product_id' and index them in either new
Hi list,
I currently have a 4 nodes cluster to collect varnish logs of my platform,
The cluster is dealing with almost 80Mill of documents (2000 documents per
second) and sometime I have some performance problems.
Let me explain how i do it. In all frontend server i have a logstash agent
I notice that https://github.com/bleskes/sense has a message saying The
development of Sense has moved into Elasticsearch Marvel.
Does this mean that no further development will happen on github? I.e. if
the Marvel team find bugs in sense will the fixes be pushed to the sense on
github, and if
You can not JOIN.
You should design your model to fit to your needs. I mean that you should index
documents like:
{
product_id: ,
price: 23.56,
stock: 100
category: iPhone case,
manufacturer: Belkin
}
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet |
hi,
Our system needs to calculate the TFs(term frequency) of the result of
search. i.e, we get a set of documents by a query_string search, and we
want to get the TF of every document.
As we know, elasticseach provides the inverted index(termdocuments),
but it do produce forward index
From the Logs it seems that the ports are open and the java code is able to
connect but it gets disconnected immediately.
[2014-03-28 14:20:24,604][DEBUG][org.elasticsearch.client.transport]
[Balder] node_sampler_interval[5s]
[2014-03-28
[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
14:20:24,633][DEBUG][org.elasticsearch.netty.channel.socket.nio.SelectorUtil]
Using select timeout of 500
[2014-03-28
From the java client logs it seems that it able to connect to elastic
search node through load balancer.
[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
Thanks David for your reply,
This means I *don't have any other option* than redesign my index model ?
Would you suggest me any alternate option, FYI I am using ELK stack.
Thanks!
On Fri, Mar 28, 2014 at 1:41 PM, David Pilato da...@pilato.fr wrote:
You can not JOIN.
You should design your
No. I didn't set a routing value. Thanks
On Mar 27, 2014 11:19 PM, Ivan Brusic i...@brusic.com wrote:
Did you set a routing value?
--
Ivan
On Thu, Mar 27, 2014 at 1:37 PM, Search User feedwo...@gmail.com wrote:
When I create an index in my cluster, Elasticsearch is creating all
shards
From the java client logs it seems that it able to connect to elastic
search node through load balancer (1.1.0.16).
[2014-03-28 14:20:24,604][DEBUG][org.
elasticsearch.client.transport] [Balder] node_sampler_interval[5s]
[2014-03-28
After Enabling client.transport.sniff parameter it is able to connect , but
it is connecting all of the nodes in the cluster. Ideally querying the load
balancer should return the response to which node to redirect will be taken
care by the load balancer. Is there any problem with the Java code
Hi,
When running the bulk indexing with python everything works fine.. good
solid throughput for the full indexing run.
When doing the same with the Java api what is happening is that thousands
of client threads are being created (7000)
And the server stops indexing and then the client just
Hi,
I'm trying to do a prefix query using the simple_query_string query.
The field I'm searching on it analyzed using an Analyzer named unstemmed,
which basically does lowercase, unicode normalisation and accent removal
(à treated as a).
If I do a search for an uppercase prefix (e.g.
{
testtype : {
properties : {
mobile : {
type : string
},
name : {
type : string
},
number : {
type : long
}
}
}
}
curl -XGET 'http://host:IP/testindex/testtype/_search?pretty'{
took : 3,
timed_out : false,
_shards : {
The one in chrome will refuse to auto indent invalid json, I believe.
Sent from my iPhone
On Mar 28, 2014, at 8:41 AM, aristechnologypartn...@gmail.com wrote:
perhaps fairly naive question, but does Sense let you validate JSON documents
before they are sent to ES? Thx.
I'm using one
I have documents in a parent/child relation. In a query run on the parent,
I'd like to know, if the found parents have children matching some query. I
don't want to filter only parents with some conditions on the child, but
only get the information, that they have childrens matching some query.
Hi there,
I have installed a 3 node ES
Monitoringhttp://www.elasticsearch.org/guide/en/marvel/current/index.html
cluster and configured two other ES clusters (10 nodes one an 3 nodes one)
to send data to it. The data gathering / storage seems to be working fine,
but the display using latest
Sorry, the lowercase_expanded_terms was added in 1.1 - that documentation
page should be fixed. About your other question, you can experiment with
the query_string query instead which has an analyze_wildcard flag - you
still can't specify 2 analyzers, but you have have it try to analyze
Not sure I understand. So if you run a _search on the parent, and use the
has_child filter to return only parents that match some child condition, is
that not what you want?
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from
I want to return all parents (or those matching some other query
conditions) but in addition to the other data in the document, I want to
compute for each parent, if he has any child with a set error flag. I don't
want to filter on this condition in this case.
Am Freitag, 28. März 2014
If it's any help, this is the error when the threads start to hang:
2014-03-28 13:34:39,845
[elasticsearch[Cerberus][transport_client_worker][T#16]{New I/O worker
#2832}] (Log4jESLogger.java:129) WARN
org.elasticsearch.netty.channel.socket.nio.AbstractNioSelector - Unexpected
exception in
Can you show the output of these:
curl localhost:9200/_settings?pretty
curl localhost:9200/_cluster/settings?pretty
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
Hmmm… Wondering if SENSE support payload when using DELETE method…
Could you try with curl?
Could you check also your date format. I think it's wrong.
If none of those works, please create a full curl recreation.
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet |
The best way is to test it. Take 1 node with say 16GB of RAM and allocate
8GB to ES. Then start pushing 1 day worth of logs into an index with 5
shards and 0 replicas on that one node AND run your typical queries. Take
measurements like throughput/dps, query latency, ram usage, cpu, and disk.
I tried in both the case it deletes complete index when criteria doesn't
match. It should not delete if the condition doesn't match.
Please let me know how to delete based on condition. Im trying this as TTL
doesn't work. I don't know how to delete records from memory for particular
date
The stats panel doesn't seem to work for me at all. I want it to calculate
the mean of a numeric field, but it just prints out 0. No errors.
Anybody got this working?
JK
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from
Binh,
I have three different list of geopoints and I want to rank the document
according to which set of geopoint matched the given geopoint within the
radium given. So if you see I have given different boost number to each of
these three geopoints.
Thanks!
On Thursday, March 27, 2014
I'd start with a simple multi_match query:
{
query: {
multi_match: {
query: love,
fields: [ DISPLAY_NAME^2, LONG_DESCRIPTION, SHORT_DESCRIPTION
]
}
}
}
Yes relevancy is based on the score.
--
You received this message because you are subscribed to the Google Groups
When I use the Java TransportClient and the BulkRequest builder, my
throughput is like a scalded cat racing a bolt of greased lightning, with
the cat way ahead!
the Java API does not say how you are using it. Since I cannot see your
code, I cannot comment on where your mistake is located.
But
Hello All
I have multi index from different input.
Ex:
One of MongoDB : One of MySQL ; One of Accesslog.
User information in the MySQL. User Behavior in ther Mongodb. IP in the
accesslog.
User information include User account, book name, book id
User Behavior include book id, what is user's
this query works fine, it deletes all account number. the only issue is
with date range. does it support date range.
when i set TTL , all records should get deleted but not happening.
DELETE /local-test/logs-test/_query
{
query : {
term : { AccountNumber : X}
}
}
On
I did , now it doesn't delete mapping/index but still records are not
getting deleted for the given condition.
I tried in both curl and Sense. Is the format correct. It it right way to
delete records for date range.
DELETE /local-test/logs-test/_query
{
query : {
term : {
Hallo Binh,
Sorry, the lowercase_expanded_terms was added in 1.1 - that documentation
page should be fixed. About your other question, you can experiment with
the query_string query instead which has an analyze_wildcard flag - you
still can't specify 2 analyzers, but you have have it try
Hi,
I am looking for the reason why this config is not valid any more with
logstash.
output {
stdout {
debug = true
}
}
Now I lost the debug headers that were available on all messages. Is it
still possible to get these headers? Was there any good reason to remove
this?
thanks,
Jettro
--
Yup, you probably need to convert the type of that field:
http://logstash.net/docs/1.4.0/filters/mutate#convert
mutate convert it to float and try it again.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and
yup. mutated and working now. Thanks!
On Fri, Mar 28, 2014 at 12:11 PM, Binh Ly binhly...@yahoo.com wrote:
Yup, you probably need to convert the type of that field:
http://logstash.net/docs/1.4.0/filters/mutate#convert
mutate convert it to float and try it again.
--
You received this
I would have suggested opening an issue on Github to clarify the license,
but issues are disabled on the repo. I agree that Sense is a fantastic
tool. I deploy apps using the Java API, but I formulate my thoughts and
queries beforehand with Sense. Beforehand I was just using straight curl
Have the recommended memory configuration settings changed when using doc
values?
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
You could be right: I can't test right now but this is my code:
(there may be 20 workerThreads)
As you can see, as each thread submits work, the thread will do a
client.prepareBulk() ... is that sufficient clear out the documents?
workerThread() {
Client client =
I've been testing ES with java 8, and everything is working fantastic, with
the exception of MVEL, which is fairly broken. I've looked on the MVEL
mailing lists, and on github issues, and there's not a lot of activity. I'm
trying to decide if I should just migrate my MVEL scripts to a different
am new in Elastic Search.I am writing some tests to Search documents but I
am receiving an IndexMissingException [indexed_store_detail] missing.I
create a Node inside my tests.At the start of the test I insert the
document and when I make the following call I get the exception.
You probably need to refresh before trying to search.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-refresh.html#indices-refresh
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr
Le 28 mars 2014 à 18:52:58, Abhijeet Kushe
I don't believe value_count is intended to be a unique count.
On Friday, March 28, 2014 7:17:47 AM UTC, Henrik Nordvik wrote:
Hi,
I'm trying out the new cardinality aggregation, and want to measure the
accuracy on my data. I'm using a dataset of a day of sample tweets (2.8m
tweets).
I tried calling refresh index but it still gives the same exception.I am
also not clear why GetResponse works for the same document
On Friday, March 28, 2014 1:53:55 PM UTC-4, David Pilato wrote:
You probably need to refresh before trying to search.
Heya,
We are pleased to announce the release of the Elasticsearch JavaScript language
plugin, version 2.0.0.
The JavaScript language plugin allows to have javascript as the language of
scripts to execute..
https://github.com/elasticsearch/elasticsearch-lang-javascript/
Release Notes -
By any chance, is it possible that in the mapping, the field is not numeric
(like maybe a string)?
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
I just forked it i case it goes away. I haven't built a Chrome extension in
years. Let me re-figure out how to do it and update my fork with
build/local installation instructions.
The lack of a license might be problematic since default copyright
provisions apply:
Yeah, I cloned it locally in case it disappears and it seems to work
perfectly well standalone or as an elasticsearch plugin - IMO it doesn't
really need to be a chrome extension. But I guess its usefulness if limited
if dev has stopped and the auto-complete isn't kept up-to-date with the
Heya,
We are pleased to announce the release of the Elasticsearch JavaScript language
plugin, version 2.1.0.
The JavaScript language plugin allows to have javascript as the language of
scripts to execute..
https://github.com/elasticsearch/elasticsearch-lang-javascript/
Release Notes -
Ok. Thanks to Boaz for finding it.
You did not run a DELETE by query:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-delete-by-query.html#docs-delete-by-query
But a DELETE MAPPING:
David,
I'm using following command in Sense. Is this correct ?
Index - local-test
Type - logs-test
DELETE /local-test/logs-test/
{
query : {
range : { TransactionTime : from : 2014-03-26 12:23:41,
to : 2014-03-26 12:24:40}
}
}
Thanks,
Arun
On Friday, March 28, 2014
In the meantime I've upgraded to ElasticSearch 1.1.0 and reduced the scroll
time to a few seconds like you recommended. I just had the same thing
happen again with a very similar hot_threads response. This is preventing
me from properly running ES in production and I'm running out of ideas -
Sorry.. Could you try with curl? - I don't know how to do this and where
to execute curl. Could you please help me here.
Should i execute curl in Fiddler or in Sense ?
On Friday, March 28, 2014 10:03:34 AM UTC-4, David Pilato wrote:
Hmmm… Wondering if SENSE support payload when using
using logstash grok, match pulls the field as %{BASE16FLOAT:avgt}
json pulled by kibana:
{
_index: logstash-2014.03.27,
_type: jobended,
_id: mn8h50n8SF693JLDZdBx8A,
_source: {
...
avgt: 1.75,
...
}
}
trying to get stats to find mean of avgt
JK
On Fri, Mar 28, 2014 at
Your code has no precautions against overwhelming the cluster. 20 worker
threads that are not coordinated is a challenge.
I recommend the BulkProcessor class at https://github
.com/elasticsearch/elasticsearch/blob/master/src
/main/java/org/elasticsearch/action/bulk/BulkProcessor.java
SYN flood
Hi,
We want to understand how segments are created during bulk indexing.
Say we've set the following param:
index.translog.flush_threshold_ops : 5,
index.translog.flush_threshold_size: 300mb,
So it means that ES will not flush until it gets 5 operations (index in
this case). As a
This should fail. It's not a range query on date.
Look at:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-range-filter.html#query-dsl-range-filter
or:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
--
David
I cannot share the source code I am afraid and it will take some time to
modify the source code in a way it does not create problems.But do you have
a link for Github ElasticSearch repo where they are testing SearchRequests
that would really help me
On Friday, March 28, 2014 2:30:10 PM UTC-4,
Yes, that is sufficient to clear out the documents. But... take the advice
given by Jörg to heart.
Elasticsearch is already optimized to take a bulk request and optimally
process it as fast as it can be done. There should not be more than one of
them at a time; no gain will be seen, and (as
The indexing buffer could also fill up which will flush to a segment. Also
the translog flush is not exactly deterministic, for example
index.translog.interval determines how often to check if the translog
needs to be flushed or not. Anyway, I wouldn't worry about it if I were
you. About the
You can probably do this with a function_score query. The idea is something
like this:
localhost:9200/_search
{
query: {
function_score: {
functions: [
{
filter: {
geo_distance : {
distance : 1km,
cornerGeoPointLst : {
I see, unfortunately I don't think you can do this at the moment. That
being said, you might find this example interesting:
https://github.com/imotov/elasticsearch-native-script-example/blob/master/example/lookup.sh
https://github.com/elasticsearchfr/hands-on/tree/answers
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 28 mars 2014 à 19:46, Abhijeet Kushe abhijeet.ku...@gmail.com a écrit :
I cannot share the source code I am afraid and it will take some time to modify
the source
When you do prepareSearch, I noticed you pass 2 parameters: getIndexNameV2
and getIndexTypeV2. If your intention is to search a type within an index,
you probably want:
.prepareSearch(getIndexNameV2()).setTypes(getIndexTypeV2()).blahblah
--
You received this message because you are subscribed
@Binh Ly that thing worked. It was a ignorance on my part.
On Friday, March 28, 2014 4:19:13 PM UTC-4, Binh Ly wrote:
When you do prepareSearch, I noticed you pass 2 parameters: getIndexNameV2
and getIndexTypeV2. If your intention is to search a type within an index,
you probably want:
Hello,
I had the following analyzer on my elasticsearch configuration:
teste:
type: custom
tokenizer: standard
filter: [standard]
A field named title uses that analyzer. I indexed ATLÉTICO using it.
Searching for ATLÉTICO works. Searching for
I'm wondering how feasible it is to use some of the Java classes defined by
Elasticsearch on the client side, rather than building up JSON strings or
XContentBuilder instances manually. Specifically, I'm wondering about
creating mappings. It would be really nice if I could create a
value_count is the total number of values extracted per bucket. This
example might help:
https://gist.github.com/bly2k/9843335
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it,
Hi All,
I have a question regarding the re-balancing of shards on es nodes. Let’s
suppose that we have 3 es nodes with 5 indices (each index has 5 shards and
one replica). So the total 50 shards will be distributed over 3 nodes
initially. If for some reason (may be due to network issue or
You should absolutely use the Java API and not use straight JSON if you are
using the Java client. The Java API is not well documented, but the best
way to see how it is utilized is by examining the test classes or the
corresponding RestAction class. In this case, RestPutMappingAction won't
help
The DocumentMapper is located at server side or in a plugin when the
request of a client is processed. It is not possible to use it by clients
to bypass the ES API. The communication to the nodes that receive document
mapper requests is very low level and beyond the ES API - see the
transport
Hi,
can elastic search read a csv or tsv file in hdfs?
i am able to read a json file from hadoop hdfs and write to elastic search
and
i am able to read that json data from elastic search index.
is it possible to achieve the same with csv or tsv files on HDFS?
or
is elastic search meant only for
Is it possible to do a has_child query without specifying type? I suppose
the same question would apply for a nested document and path. The use case
is a common parent type (think of it sort of like a placeholder) with a
children of 10 different types. I could execute a few different queries,
It's a bit old issue, but how did you fix it?
On Saturday, October 19, 2013 4:16:48 PM UTC-7, pe...@vagaband.co wrote:
Yup, that was it. UDP was not allowed on my QA data center. Thanks, David
and Jörg.
--
You received this message because you are subscribed to the Google Groups
It is indeed possible to compose complex XContent by implementing helper
methods and delegating construction to them as you propose. Indeed this is a
common pattern within the Elasticsearch code itself.
The basic idea is that you pass your XContentBuilder to your helper methods
which return
78 matches
Mail list logo
