Re: Start/End overlap between all documents
Unfortunately there is no location/period to filter on. It's looking for any date range overlapping with any other date range at the same location. I'm working on aggregating terms on location, then date histogram time windows between start and end. Any with doc_count 1 would be overlapping. On Wednesday, 18 March 2015 12:42:49 UTC-4, Mark Walkom wrote: You could do a terms filter on the location along with a range filter query for the period you are after. That's probably the closest thing. On 18 March 2015 at 06:34, Phil philtr...@gmail.com javascript: wrote: I'm trying to wrap my head around this but it may not be possible due to the nature of ES. I have an index of appointments with start time, end time, location and participants. I'd like to search for *any *appointments that have overlapping start and end times at the same location. I understand that this is in essence a relationship between an index and itself and that it is asking ES to query each document against every other document. Is anything like this even possible? Hopefully someone can point me in the right direction. Thanks. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a2cd366e-554c-41d5-9134-27a578c778c9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a2cd366e-554c-41d5-9134-27a578c778c9%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fc78b54f-7034-4a0a-a793-59f39498894e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
ES2Hadoop and Hadoop2ES
Hello, My SW context : 5 nodes running ES 1.4 + 1 node running ES 1.3 i need to read node ES 1.4 ( One index + all data indexed) and write it to nodeES 1.3 . i have a java code that do the job but it firstly writes to HDFS and then i need a second job that read HDFS and send to ES 1.3 i was wondering if iit was possible to do all in one without storing on HDFS by interfacing both in the same job and java code ES doc mentions ( but within the same job) : es.resource.read (defaults to es.resource) Elasticsearch resource used for reading (but not writing) data. Useful when reading and writing data to different Elasticsearch indices within the *same* job. es.resource.write PS: i tried without success the backup/restore possibility from ES Best R Phil -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1e3f6b01-0f9c-480e-8b87-6d509e2a9ba0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Best pratices for index , search and updates
Please create your own mappings and don't rely on type detection. On Thursday, January 15, 2015 at 6:41:14 PM UTC-5, bvnr wrote: Am new to the elastic search ... Can some body throw me ideas about the best practices one should follow to get good performance for index ,search and updates -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/955a9fb9-c406-413a-befd-1708baa5971c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Kibana and nested documents -- include_in_parent
Hello, I am new to ElasticSearch and I have a very specific question. We have implemented our ElasticSearch cluster with a nested document structure. Each document is made of one ID, a key element and one field including several nested records that are inserted by the script api and the bulk update function. My question is, is it possible to view nested documents in Kibana, without using *include_in_parent, *because from preliminary testing it seams to be using more disk space when include_in_parent is in the mappings ? When include_in_parent is not in the mappings, the documents are not viewable within Kibana 4.0.0 Also, is there a function or way to display which documents have the most nested records, by using the size of the nested records in the document? I would like to have a pie chart, that could display them using the size of their nested attribute. Thank you in advance. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/85b0aed9-f74a-4031-b815-999f1df9be55%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
best approach to pull back only relevant fields
If I have a document I’m storing that looks like this:
doc = {
userId: *‘*yoyomamma',
physicians: [{
name: sanjay gupta,
id: 403494
},
{
name: daniel litvak,
id: 3382
}],
procedures: [{
name: appendectomy,
id: 3939,
date: 2/3/2012
}
… etc a bunch of other stuff
}
When I search on “appendectomy” I want to only return the following:
“userId” and the matching “procedures” fields…..
Any thoughts on the best approach to do this?
thanks
phil
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGenvhVDwZdS8519af5RWisrEVVUbeOi55%2BUdGZNK55mwsLzNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: javascript es client - should es clients be pooled?
thanks jack, let me know how your testing goes! On Fri, Dec 26, 2014 at 11:57 AM, Jack Park jackp...@topicquests.org wrote: On my platform, there is one and only one es client, through which all requests go. It seems to work well, though well is defined as having just a few people using it at one time. We plan to start stress testing the system soon. Cheers Jack On Fri, Dec 26, 2014 at 9:55 AM, phil swenson phil.swen...@gmail.com wrote: no answer, so let me ask a different way. how are most of the es javascript client apps managing the instance of the client? do you have one es client per app (singleton) that all requests go through? Do you create and destroy clients for every request? Do you use a es client pool using something like https://github.com/coopernurse/node-pool ? Thanks for any comments! phil On Wed, Dec 24, 2014 at 10:34 AM, Phil Swenson phil.swen...@gmail.com wrote: I'm writing a node/es app using the es javascript api Is there any reason to use pooling for all the javascript clients? Or should I just use one client for the app? Thanks, phil -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a44a6a6e-8763-44f5-9e2d-8ea52dcf0345%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a44a6a6e-8763-44f5-9e2d-8ea52dcf0345%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGenvhVYD-fRkrVx5j0XR_xNYM2eSFFFxh2TxJnEar-HEWVfSg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAGenvhVYD-fRkrVx5j0XR_xNYM2eSFFFxh2TxJnEar-HEWVfSg%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH6s0fyEhW5b-BXCYViAb7Ej4XNC5shjjVE_kYfPKOcx6H-dng%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAH6s0fyEhW5b-BXCYViAb7Ej4XNC5shjjVE_kYfPKOcx6H-dng%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGenvhX4Gk80WG%3D33QzTzpDL-PDQ4eOm4JZZkToG1sXCY_bVfw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: javascript es client - should es clients be pooled?
no answer, so let me ask a different way. how are most of the es javascript client apps managing the instance of the client? do you have one es client per app (singleton) that all requests go through? Do you create and destroy clients for every request? Do you use a es client pool using something like https://github.com/coopernurse/node-pool ? Thanks for any comments! phil On Wed, Dec 24, 2014 at 10:34 AM, Phil Swenson phil.swen...@gmail.com wrote: I'm writing a node/es app using the es javascript api Is there any reason to use pooling for all the javascript clients? Or should I just use one client for the app? Thanks, phil -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a44a6a6e-8763-44f5-9e2d-8ea52dcf0345%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a44a6a6e-8763-44f5-9e2d-8ea52dcf0345%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGenvhVYD-fRkrVx5j0XR_xNYM2eSFFFxh2TxJnEar-HEWVfSg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
javascript es client - should es clients be pooled?
I'm writing a node/es app using the es javascript api Is there any reason to use pooling for all the javascript clients? Or should I just use one client for the app? Thanks, phil -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a44a6a6e-8763-44f5-9e2d-8ea52dcf0345%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Are rivers still being deprecated?
did you ever find an answer to this question? On Thursday, November 6, 2014 7:17:36 PM UTC-7, Alexandre Rafalovitch wrote: ElasticSearch 1.4 is out and I can't see any mentions that Rivers are deprecated. Has that (informal) decision been reversed? Or was the timeline further out? What's the currently recommended approach? Regards, Alex. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d7638967-0cbc-4ab7-b2ce-2646116423c8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Searching across 2 nodes on 2 different servers
Please pardon my ignorance in advance, I'm new to Elasticsearch. I have set up a cluster across two servers. One server is master and data, the other just data. Let's call my first server 'server1', my other server 'server2' and my index 'address'. If i want to search my address index, I can use the following URL: http://server1:9200/address/_search This returns data fine. I can also use: http://server2:9200/address/_search and this returns data fine, too. My query is as follows - Is there URL which I can use where Elasticsearch chooses which server to return data from automatically, instead of having to manually specify either server1 or server2? I will be drastically increasing the number of indexes shortly and I'd like Elasticsearch to automate which node it selects depending on whether the other is in use, etc. Is this possible? Many thanks, Phil. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6d95c9c0-ae71-4238-a8a7-2bdbca46c7c2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Access to AbstractAggregationBuilder.name
Hello, In the Java API AbstractAggregationBuilder's name property is protected. Is there a particular reason it can't be public, or have an accessor added, or is this something you'd consider a PR for? Not having access is making things more complicated than I'd like. Thanks, Phil -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/63c1ae3f-ff37-4f47-9147-037d19ff9eec%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ES + spark 1.0.1 - unable to send RDDs to ES
Hi Costin
Thanks for your help!
it works now after the update :-)
Best Regards
phil
On Wednesday, August 6, 2014 6:46:57 PM UTC+2, Costin Leau wrote:
Hi Phil,
Glad to see the work in es-hadoop master is being picked up even without
any public announcement of it :)
The issue has been fixed in master [1] and already pushed to Maven - can
you please update and try again?
FTR: The issue seems to be caused by multiple versions of Jackson which
are pulled in the classpath (one from Hadoop,
another from Spark)
which on some platforms, causes class loading issues in Jackson during
start-up. The fix in master hopefully remedies that.
Cheers,
[1] https://github.com/elasticsearch/elasticsearch-hadoop/issues/239
On 8/6/14 7:32 PM, Phil gib wrote:
hello
my context : spark, spark-shell 1.0.1 jdk1.7 scala 2.10.4, ES-Hadoop
2.1.0 ( nighly build)
my problem:
- unable to send RDDs from spark to ES
i got a NoClassDefFoundError see below (
org/codehaus/jackson/annotate/JsonClass)
jackson Jars to add to spark shell?
philippe
best regards
--
$bin/spark-shell --jars
/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
..
spark version 1.0.1
Using Scala version 2.10.4
..
14/08/06 17:19:36 INFO SparkContext: Added JAR
file:/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
scala
import org.elasticsearch.hadoop.mr.EsOutputFormat
import org.elasticsearch.hadoop.mr.EsInputFormat
import org.elasticsearch.hadoop.cfg.ConfigurationOptions
import org.apache.hadoop.mapred.{FileOutputCommitter, FileOutputFormat,
JobConf, OutputFormat}
import org.apache.hadoop.fs.Path
import org.apache.hadoop.io.{MapWritable, Text, NullWritable}
val jobConf = new JobConf(sc.hadoopConfiguration)
jobConf.set(es.resource, myindex/mytype)
jobConf.set(es.query, ?q=*)
val esRDD = sc.hadoopRDD(jobConf,classOf[EsInputFormat[Text,
MapWritable]],classOf[Text],classOf[MapWritable])
//*up to there everything ok *
es.count()
java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass
at
org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524)
.
at
org.elasticsearch.hadoop.rest.RestClient.parseContent(RestClient.java:119)
--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com javascript: mailto:
elasticsearch+unsubscr...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2ef2ba16-5a25-4b73-9f09-018e78f517e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
ES + spark 1.0.1 - unable to send RDDs to ES
hello
my context : spark, spark-shell 1.0.1 jdk1.7 scala 2.10.4, ES-Hadoop
2.1.0 ( nighly build)
my problem:
- unable to send RDDs from spark to ES
i got a NoClassDefFoundError see below (
org/codehaus/jackson/annotate/JsonClass)
jackson Jars to add to spark shell?
philippe
best regards
--
$bin/spark-shell --jars
/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
..
spark version 1.0.1
Using Scala version 2.10.4
..
14/08/06 17:19:36 INFO SparkContext: Added JAR
file:/usr/lib/spark-1.0/lib/elasticsearch-hadoop-2.1.0.jar
scala
import org.elasticsearch.hadoop.mr.EsOutputFormat
import org.elasticsearch.hadoop.mr.EsInputFormat
import org.elasticsearch.hadoop.cfg.ConfigurationOptions
import org.apache.hadoop.mapred.{FileOutputCommitter, FileOutputFormat,
JobConf, OutputFormat}
import org.apache.hadoop.fs.Path
import org.apache.hadoop.io.{MapWritable, Text, NullWritable}
val jobConf = new JobConf(sc.hadoopConfiguration)
jobConf.set(es.resource, myindex/mytype)
jobConf.set(es.query, ?q=*)
val esRDD = sc.hadoopRDD(jobConf,classOf[EsInputFormat[Text,
MapWritable]],classOf[Text],classOf[MapWritable])
//* up to there everything ok *
es.count()
java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass
at
org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524)
.
at
org.elasticsearch.hadoop.rest.RestClient.parseContent(RestClient.java:119)
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f525de7a-2d5f-41d0-b284-9ed2886c8c22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: ES + spark 1.0.1 - unable to send RDDs to ES
sorry for the mistake : -- unable to read from ES and create RDDS On Wednesday, August 6, 2014 6:32:02 PM UTC+2, Phil gib wrote: hello my context : spark, spark-shell 1.0.1 jdk1.7 scala 2.10.4, ES-Hadoop 2.1.0 ( nighly build) my problem: - unable to read from ES and create RDDS i got a NoClassDefFoundError see below ( org/codehaus/jackson/annotate/JsonClass) jackson Jars to add to spark shell? philippe best regards -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/63db7ae7-0da9-4b87-a835-42e38f11ee05%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Nested cardinality values way off with filter?
Hello,
I'm trying to get produce the distribution of documents that matches vs
don't match a query, and get the cardinality of a field for both sets. The
idea is Users who did vs Users who did not. In reality I'm actually
running another aggregation under did not (otherwise I'd just subtract
one count from the total), but the query here illustrates the issue I'm
having:
*Query*
aggs: {
total_distinct_count: { cardinality: { field: UserId } },
has_thing: {
filter: { term: { State: thing } },
aggs: {
distinct_count: { cardinality: { field: UserId } }
}
},
does_not_have_thing: {
filter: {
not : { term: { State: thing } }
},
aggs: {
distinct_count: { cardinality: { field: UserId } }
}
}
}
*Response*
hits: {
total: 3309709,
max_score: 0,
hits: []
},
aggregations: {
total_distinct_count: {
value: 654556
},
does_not_have_thing: {
doc_count: 2575512,
distinct_count: {
value: 563371
}
},
has_thing: {
doc_count: 734197,
distinct_count: {
value: 223128
}
}
}
I would expect (aggregations.has_thing.dictinct_count.value +
aggregations.does_not_have_thing.distinct_count.value) to be close to
aggreations.total_distinct_count.value, but in reality it's pretty far off
(~+20%). Note: That the summation of doc_count adds up exactly to
hits.total. So I don't think this is an issue with the query, but I could
be wrong.
Any ideas whats up? Have I structured the query incorrectly, Is this a bug?
Or is this just expected behavior?
Some notes:
- UserId's data type is a *long, *but the values only fill up integer
space. (510,539 to 418,346,844)
- I'm running elasticsearch 1.1.0
- I've tried playing around with the precision threshold, but it doesn't
appear to make a difference.
Thanks in advance,
Cheers
Phil
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cb558261-7865-491e-9bc5-e3f78b6390f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Nested cardinality values way off with filter?
Doh! You are correct, my bad. I assumed the filter was an exclusive per
user property, but in fact - it is not.
Thanks for getting back to me
Cheers
Phil
On Thursday, May 22, 2014 4:36:02 PM UTC-7, Adrien Grand wrote:
distinct_countOn Thu, May 22, 2014 at 10:34 PM, Phil Price
phil...@gmail.com javascript: wrote:
I would expect (aggregations.has_thing.dictinct_count.value +
aggregations.does_not_have_thing.distinct_count.value) to be close to
aggreations.total_distinct_count.value, but in reality it's pretty far off
I think this result is to be expected if you have some user IDs that match
both criteria? Eg. if your index has these two documents:
{
UserId : 42,
State: thing
}
{
UserId : 42,
State: anything
}
Then your aggregations would look like:
aggregations: {
total_distinct_count: {
value: 1
},
does_not_have_thing: {
doc_count: 1,
distinct_count: {
value: 1
}
},
has_thing: {
doc_count: 1,
distinct_count: {
value: 1
}
}
}
And the sum of the values of distinct_count per bucket is larger than the
global value for distinct_count.
--
Adrien Grand
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/40c4f479-1787-4931-be7d-9511dc06e1fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: searching most recent objects
Oh, awesome, thank you so much for the help, I'll give that a try! On Thursday, April 17, 2014 2:51:23 PM UTC-7, Itamar Syn-Hershko wrote: For recent X just sort on the _timestamp field and specify X as the page size http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-timestamp-field.html -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, Apr 18, 2014 at 12:43 AM, Phil Greenberg philip.n@gmail.comjavascript: wrote: Thanks Itamar. So are you saying it's not possible to ask ES for the most recent X objects that match the given query? Only to say give me the last 30 days of objects? On Thursday, April 17, 2014 2:39:43 PM UTC-7, Itamar Syn-Hershko wrote: Filter (range filter on the date/time field) is exactly the way to do this. Another possibility is using rolling indexes (e.g. an index per day, like the logstash indexes are defined) but that obviously depends on a lot of other business concerns and isn't really viable for most applications -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, Apr 18, 2014 at 12:36 AM, Phil Greenberg philip.n@gmail.com wrote: I am also facing the same issue. Right now, I am just doing a filter myself, but I would assume this is a common use case, an ES must have a way to deal with it? On Tuesday, April 15, 2014 6:24:52 PM UTC-7, Joris Bolsens wrote: I am using the javascript API and want to do a search and have it search through the most recent objects, IE I call a search with size 100, I want to have the most recent 100 objects returned to me, how would I go about doing that? I tried using sort, but it seems that it just sorts the results after the search completed -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/fb4a3d83-c386-459b-beb6-a8ca4fcbb286% 40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/fb4a3d83-c386-459b-beb6-a8ca4fcbb286%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ec1fa45f-4534-45a2-96ae-1d5edf783ac4%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ec1fa45f-4534-45a2-96ae-1d5edf783ac4%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53cdd542-4d17-47b9-bedb-3d2f6937864d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: searching most recent objects
I am also facing the same issue. Right now, I am just doing a filter myself, but I would assume this is a common use case, an ES must have a way to deal with it? On Tuesday, April 15, 2014 6:24:52 PM UTC-7, Joris Bolsens wrote: I am using the javascript API and want to do a search and have it search through the most recent objects, IE I call a search with size 100, I want to have the most recent 100 objects returned to me, how would I go about doing that? I tried using sort, but it seems that it just sorts the results after the search completed -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fb4a3d83-c386-459b-beb6-a8ca4fcbb286%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: searching most recent objects
Thanks Itamar. So are you saying it's not possible to ask ES for the most recent X objects that match the given query? Only to say give me the last 30 days of objects? On Thursday, April 17, 2014 2:39:43 PM UTC-7, Itamar Syn-Hershko wrote: Filter (range filter on the date/time field) is exactly the way to do this. Another possibility is using rolling indexes (e.g. an index per day, like the logstash indexes are defined) but that obviously depends on a lot of other business concerns and isn't really viable for most applications -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, Apr 18, 2014 at 12:36 AM, Phil Greenberg philip.n@gmail.comjavascript: wrote: I am also facing the same issue. Right now, I am just doing a filter myself, but I would assume this is a common use case, an ES must have a way to deal with it? On Tuesday, April 15, 2014 6:24:52 PM UTC-7, Joris Bolsens wrote: I am using the javascript API and want to do a search and have it search through the most recent objects, IE I call a search with size 100, I want to have the most recent 100 objects returned to me, how would I go about doing that? I tried using sort, but it seems that it just sorts the results after the search completed -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fb4a3d83-c386-459b-beb6-a8ca4fcbb286%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/fb4a3d83-c386-459b-beb6-a8ca4fcbb286%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ec1fa45f-4534-45a2-96ae-1d5edf783ac4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Hadoop metrics (Elasticsearch for Apache Hadoop) + marvel
hello, is there a possibility to get Hadoop metrics (http://www.elasticsearch.org/blog/elasticsearch-apache-hadoop-1-3-m3/ ) monitored in realtime through marvel plugin or through a kibana dashboard ? and in this case what index to use to query ? Phi thanks a lot -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1e11c930-93dd-49e2-99b0-4b1fe4e950a2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: query edition problem kibana
thanks ok, understood now it works philippe On Friday, March 14, 2014 7:13:28 PM UTC+1, Binh Ly wrote: You'll need to use the query_string syntax: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html It should be something like: rglog3.max_bitrate:[301 TO 999] or: rglog3.max_bitrate:300 AND rglog3.max_bitrate:1000 -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6e123f5f-0f17-4e66-9539-8e718da821a4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
query edition problem kibana
hello
my context kibana + ES 0.90.11 + ES Head plugin
i have some problem with kibana query editor :
i have indexed my logs in ES, and displayed in kibana using basic queries .
Perfect !
i need more complex queries on a max_bitrate field ( integer)
for example, i need this query --300 max_bitrate 1000
using ES head plugin it works i see the result, with raw json query like
this
{query:{bool:{must:[{
range:{rglog3.max_bitrate:{gt:300,lt:1000}}}..
but then how to put in the query editor of Kibana to get the query
works ?
i did not succeed
any clue , or pointer to doc will be great because i cannot find
thanks
philippe
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ac40cafe-1acd-458c-8699-908da4baa870%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: problem with kibana and index
hello thanks but the issue is not with logstash ( it works with it) the issue is without logstash ie curl -X POST - to Kibana On Thursday, February 27, 2014 6:37:35 PM UTC+1, computer engineer wrote: I had a similar issue and the problem was the .conf file in logstash. I had to ensure tcp input type was set to codex = json On Thursday, February 27, 2014 11:26:55 AM UTC-5, Phil gib wrote: Hello my context : logstash 1.3.2 + ES 0.90.11 + kibana3 I have a perfect pipe of these tools when i send my own logs files tranformed by logstash ( ES works and kibana also with different configured dashboards ) but now when i try to send directly json format to ES it works , but kibana ( configured with the good index ) does not get any data ... -:( ES head-plugin displays correctly the indexed data , but nothing appears in kibana did i miss something important when sending directly json data through curl -XPOST 'http://localhost:9200/testindex1/testlog' -d @my.json from the head plugin view of ES, the data is indexed correctly... Any clue? philippe -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0fc3a0d8-b611-4b79-836d-0e74ecf52751%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
hadoop to ES problem
hello my context: ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java i am experimenting ES-Hadoop and Hadoop-ES no problem with Hadoop2ES with these 3 settings ( i see the index + data through head plugin, perfect) job.set(es.resource, eshindex2/eshlog2); job.set(es.host, localhost); job.set(es.port, 9200); but experimenting the inverse , ie ES 2 Hadoop configuration.set(es.query, eshindex2/eshlog2/_search?q=*); configuration.set(es.host, localhost); configuration.set(es.port, 9200); System.out.println(conf:+ configuration); JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class); ... i got the error 10:58:27 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001 Exception in thread main java.lang.NullPointerException at org.elasticsearch.hadoop.rest.dto.Node.init(Node.java:30) at org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139) Any idea? bestsregards philippe -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: hadoop to ES problem
Hello Costin, changing to es.resource does not help... :-( argh.. i see the socket connection closed in the ES logs.. so ES is contacted ... the 9200 is correct ? philippe best regards configuration.set(es.resource, eshindex2/eshlog2/_search?q=*); configuration.set(es.host, localhost); configuration.set(es.port, 9200); System.out.println(conf:+ configuration); JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class); ... On Tuesday, February 4, 2014 11:07:37 AM UTC+1, Costin Leau wrote: If you are using M1 you should use es.resource instead of es.query in your latest example. es.query is supported in the upcoming 1.3.0.M2 (not in m1). On 04/02/2014 12:01 PM, Phil gib wrote: hello my context: ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java i am experimenting ES-Hadoop and Hadoop-ES no problem with Hadoop2ES with these 3 settings ( i see the index + data through head plugin, perfect) job.set(es.resource, eshindex2/eshlog2); job.set(es.host, localhost); job.set(es.port, 9200); but experimenting the inverse , ie ES 2 Hadoop configuration.set(es.query, eshindex2/eshlog2/_search?q=*); configuration.set(es.host, localhost); configuration.set(es.port, 9200); System.out.println(conf:+ configuration); JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class); ... i got the error 10:58:27 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001 Exception in thread main java.lang.NullPointerException at org.elasticsearch.hadoop.rest.dto.Node.init(Node.java:30) at org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139) Any idea? bestsregards philippe -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Costin -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1353d6b0-830e-4881-b10d-533639227a1c%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: hadoop to ES problem
Thnaks for your reply,
my SW context is:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java, windows7
(perharps some configuration problem in Eclipse , i will investigate )
best regards
philippe
On Tuesday, February 4, 2014 12:22:54 PM UTC+1, Costin Leau wrote:
The port and host are 9200 and localhost by default. What's your ES
version?
ES-Hadoop requires 0.90 or higher (preferably the latest 0.90).
On 04/02/2014 12:59 PM, Phil gib wrote:
Hello Costin,
changing to es.resource does not help... :-(
argh.. i see the socket connection closed in the ES logs.. so ES is
contacted ... the 9200 is correct ?
philippe
best regards
configuration.set(es.resource, eshindex2/eshlog2/_search?q=*);
configuration.set(es.host, localhost);
configuration.set(es.port, 9200);
System.out.println(conf:+ configuration);
JobConf job = new JobConf(configuration,
SimpleJobES2Hadoop.class);
...
On Tuesday, February 4, 2014 11:07:37 AM UTC+1, Costin Leau wrote:
If you are using M1 you should use es.resource instead of es.query
in your latest example.
es.query is supported in the upcoming 1.3.0.M2 (not in m1).
On 04/02/2014 12:01 PM, Phil gib wrote:
hello my context:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java
i am experimenting ES-Hadoop and Hadoop-ES
no problem with Hadoop2ES with these 3 settings ( i see the
index + data through head plugin, perfect)
job.set(es.resource, eshindex2/eshlog2);
job.set(es.host, localhost);
job.set(es.port, 9200);
but experimenting the inverse , ie ES 2 Hadoop
configuration.set(es.query,
eshindex2/eshlog2/_search?q=*);
configuration.set(es.host, localhost);
configuration.set(es.port, 9200);
System.out.println(conf:+ configuration);
JobConf job = new JobConf(configuration,
SimpleJobES2Hadoop.class);
...
i got the error
10:58:27 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001
Exception in thread main java.lang.NullPointerException
at
org.elasticsearch.hadoop.rest.dto.Node.init(Node.java:30)
at
org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)
Any idea?
bestsregards
philippe
--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com.
For more options, visithttps://groups.google.com/groups/opt_out
https://groups.google.com/groups/opt_out.
--
Costin
--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1353d6b0-830e-4881-b10d-533639227a1c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Costin
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8f8fbb72-097d-47e8-8c2c-d2f74a1f4b00%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
