Re: spark version, elasticsearch-hadoop version, akka version sync up

2015-03-17 Thread Jeff Steinmetz
of course). On Tue, Mar 17, 2015 at 10:43 PM, Jeff Steinmetz jeffrey@gmail.com javascript: wrote: There are plenty of spark / akka / scala / elasticsearch-hadoop dependencies to keep track of. Is it true that elasticsearch-hadoop needs to be compiled for a specific spark version to run

spark version, elasticsearch-hadoop version, akka version sync up

2015-03-17 Thread Jeff Steinmetz
There are plenty of spark / akka / scala / elasticsearch-hadoop dependencies to keep track of. Is it true that elasticsearch-hadoop needs to be compiled for a specific spark version to run correctly on the cluster? I'm also trying to keep track of the akka version and scala version. i.e, wil

Re: function_score weight of 0.0 returns 1

2015-02-23 Thread Jeff Steinmetz
BTW, I am running against Elasticsearch version 1.4.4 On Monday, February 23, 2015 at 6:49:59 PM UTC-8, Jeff Steinmetz wrote: Any ideas why this would return a score of 1, when the last (bool must_not) matches, even though the weight is set to 0.0? As a bit of background, this provides

function_score weight of 0.0 returns 1

2015-02-23 Thread Jeff Steinmetz
Any ideas why this would return a score of 1, when the last (bool must_not) matches, even though the weight is set to 0.0? As a bit of background, this provides custom scoring where - if one field matches, score at .35, - if another more important field matches, score .65, - if both match,

Re: counting items in a list [array] returns (what we think) are incorrect counts via groovy

2015-01-08 Thread Jeff Steinmetz
use a script field on the query if you need the result to contain the count. Or just count it on the result side. Nik On Jan 9, 2015 12:43 AM, Jeff Steinmetz jeffrey@gmail.com javascript: wrote: Transform worked well. Nice. Curious how to get it to save to source? Tried this below

Re: counting items in a list [array] returns (what we think) are incorrect counts via groovy

2015-01-08 Thread Jeff Steinmetz
probably fine but if you do it a lot probably best to index the count. On Jan 9, 2015 12:04 AM, Jeff Steinmetz jeffrey@gmail.com javascript: wrote: Thank you, that worked. I was curious about the speed, is running a script using _source slower that doc[] ? Totally understand a dynamic

Re: counting items in a list [array] returns (what we think) are incorrect counts via groovy

2015-01-08 Thread Jeff Steinmetz
certainly be much faster. On Thursday, January 8, 2015 at 7:04:40 PM UTC-8, Nikolas Everett wrote: On Thu, Jan 8, 2015 at 9:09 PM, Jeff Steinmetz jeffrey@gmail.com javascript: wrote: Is there a better way to do this? Please see this gist (or even better yet, run the script locally see

counting items in a list [array] returns (what we think) are incorrect counts via groovy

2015-01-08 Thread Jeff Steinmetz
Is there a better way to do this? Please see this gist (or even better yet, run the script locally see the issue). https://gist.github.com/jeffsteinmetz/2ea8329c667386c80fae You must have scripting enabled in your elasticsearch config for this to work. This was originally based on some

Re: ElasticSearch spark esRDD not returing the aggregate values in aggregated query

2014-10-17 Thread Jeff Steinmetz
as the first object ) Best, Jeff Steinmetz Director of Data Science Ekho, Inc. www.ekho.me @jeffsteinmetz On Wednesday, September 17, 2014 6:13:37 AM UTC-7, siva pradeep wrote: Hi, I have a query which filters the rows and then applies the aggregation. I tried running the query in Sense it gave me

org.elasticsearch.search.aggregations docs

2014-08-11 Thread Jeff Steinmetz
I've been looking all over the place for documentation on org.elasticsearch.search.aggregations._ The Java API 1.x docs state Facets will be depreciated. I am using the source code on github for reference. I also see that Kibana does all its queries using Facets. I wanted to make sure I

Creating filters per aggregation similar to Facets

2014-08-11 Thread Jeff Steinmetz
Kibana provides a good example of date histograms, split out by each query entered at the top in the Query bar. It essentially creates multiple free text queries against all. I see it generates per facet filter, with a free text (query_string) search. Since facets are to be depreciated, I

Re: org.elasticsearch.search.aggregations docs

2014-08-11 Thread Jeff Steinmetz
. On Monday, August 11, 2014 4:08:41 PM UTC-7, Isabel Drost-Fromm wrote: On Mon, Aug 11, 2014 at 10:47 PM, Jeff Steinmetz jeffrey@gmail.com javascript: wrote: I've been looking all over the place for documentation on org.elasticsearch.search.aggregations._ There's quite a bit of information