Re: Safe to set doc_values: true on existing index?

2015-03-05 Thread Justin Zhu
f you want to add doc values to an existing index, >> you would need to reindex. >> >> On Wed, Mar 4, 2015 at 12:17 AM, Justin Zhu > > wrote: >> >>> Would this start building the doc values structure for new documents >>> indexed and not prior docs? >

Re: Safe to set doc_values: true on existing index?

2015-03-03 Thread Justin Zhu
Excellent. Noticed Boolean is unsupported right now, can we add the doc_values: true anyways for future use?: On Tuesday, March 3, 2015 at 3:55:30 PM UTC-8, Mark Walkom wrote: > > Yes, it will do exactly that. > > On 4 March 2015 at 10:17, Justin Zhu > > wrote: > >>

Safe to set doc_values: true on existing index?

2015-03-03 Thread Justin Zhu
Would this start building the doc values structure for new documents indexed and not prior docs? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+

Re: High CPU during scroll large request, low iowait

2015-03-03 Thread Justin Zhu
gt;> - ensure your application uses all 3 client nodes to spread the load, >> and increase the number of client nodes if possible >> - have more than one index - if you can segment the query in a sensible >> way e.g. time series data >> >> G >> >> On

Re: High CPU during scroll large request, low iowait

2015-03-02 Thread Justin Zhu
d. > > Jörg > > On Sat, Feb 28, 2015 at 8:48 AM, Justin Zhu > wrote: > >> Good question. The request has several filters on child types which may >> have dozens to hundreds of documents per parent document. >> >> It could just be the request is quite comp

Re: High CPU during scroll large request, low iowait

2015-02-27 Thread Justin Zhu
several shards might stop returning results? On Friday, February 27, 2015 at 8:43:26 AM UTC-8, Jörg Prante wrote: > > How does the scan/scroll request look like? > > Jörg > > On Fri, Feb 27, 2015 at 5:38 PM, Justin Zhu > wrote: > >> We have a 15 node cluster,

High CPU during scroll large request, low iowait

2015-02-27 Thread Justin Zhu
We have a 15 node cluster, 9 data nodes with 18gb ram, 3 master, 3 client, 12 cpus running 1.3.2. When doing a scroll on an index of 20gb for 800K documents, each document is 1K we're seeing cpu jump to 30% to 90% on nodes with primary a shard. Index and other search quests are pretty low. Our

Re: Only send requests directly to data notes and not master nodes?

2015-01-18 Thread Justin Zhu
> > On 19 January 2015 at 16:48, Justin Zhu > > wrote: > >> We give the master nodes 5gb of memory -- stats are showing low cpu & >> memory utilization. Would you still recommend the client only node? If so, >> how many & powerful? >> >> >>

Re: Only send requests directly to data notes and not master nodes?

2015-01-18 Thread Justin Zhu
lient nodes if you are OOMing your > masters (which, is a bad thing to happen to masters). > > On 18 January 2015 at 10:23, Justin Zhu > > wrote: > >> We have a 9 node cluster, 3 masters, 6 data. We've been using the java >> transport client, which connects to a

Only send requests directly to data notes and not master nodes?

2015-01-17 Thread Justin Zhu
We have a 9 node cluster, 3 masters, 6 data. We've been using the java transport client, which connects to all 3 masters. Occassionaly the masters become unresponsive and needs to restart. Should the transport clients connect to the 6 data nodes directly instead? Thanks. -- You received this

Filtering an aggregated result on child documents?

2014-07-26 Thread Justin Zhu
We have a parent->child mapping of users->purchases The purchase has a 'total' field, that's a dollar amount. How can we support a query like -- users who's spent over $1000 dollars in the past 3 months? If not currently supported, any suggestions on how it can be done efficiently with the cur

Re: Out of heap error on machines with 18GB heap and 6GB index

2014-05-13 Thread Justin Zhu
>[15.2gb]/[17.5gb]} [2014-05-13 18:14:17,014][INFO ][monitor.jvm ] [elasticsearch-i2-1] [gc][old][404513][108] duration [5.1s], collections [1]/[5.6s], total [5.1s]/[4.4m], memory [16.8gb]->[17.5gb]/[17.7gb], all_pools {[young] [245.2mb]->[83.2mb]/[266.2mb]}{[survivor] [33.2

Out of heap error on machines with 18GB heap and 6GB index

2014-05-13 Thread Justin Zhu
We have a 3 node cluster, each with 30gb total memory, 18gb allocated to elasticsearch and replicas set at 2. Our largest index is 6GB. After running for a few days, the cluster would go down with Java out of heap errors. We currently have a multi-get aggregation that issues 40 requests on the