Parent/Child relationship considerations

2015-03-20 Thread Joshua P
I've been reading up on parent/child relationships and had a couple questions: On the Practical Considerations http://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child-performance.html page, it's suggested that you should Use parent-child relationships sparingly, and only

Getting Elasticsearch to put scores on a 0-100 scale?

2014-12-05 Thread Joshua P
Everything that I've read about scoring in ES suggests that you really shouldn't do this. However, I need a score floor to make some automatic decisions for a user about which of the top results is most-likely a match to their query (if the first document returned has a score = 70% then we

Bulk Indexing Problems

2014-09-09 Thread Joshua P
Hi there! I'm trying to do a one-time index of about 800,000 records into an instance of elasticsearch. But I'm having a bit of trouble. It continually fails around 200,000 records. Looking at in the Elasticsearch Head Plugin, my index goes offline and becomes unrecoverable. For now, I have

Some Bulk Indexing PRoblems

2014-09-09 Thread Joshua P
Hi there! Sorry I posted two topics. I've somehow managed to post an incomplete post. I'm trying to do a one-time index of about 800,000 records into an instance of elasticsearch. But I'm having a bit of trouble. It continually fails around 200,000 records. Looking at in the Elasticsearch

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
. Thanks Vineeth On Tue, Sep 9, 2014 at 7:48 PM, Joshua P jpeter...@gmail.com javascript: wrote: Hi there! I'm trying to do a one-time index of about 800,000 records into an instance of elasticsearch. But I'm having a bit of trouble. It continually fails around 200,000 records

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
AM UTC-4, Joshua P wrote: Thanks for the reply, Vineeth! What's a practical heap size? I've seen some people saying they set it to 30gb but this confuses me because in the /etc/default/elasticsearch file, the comment suggests the max is only 1gb? I'll look into the threadpool issue

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
was rejected. Thanks Vineeth On Tue, Sep 9, 2014 at 8:58 PM, Joshua P jpeter...@gmail.com javascript: wrote: You also said you wouldn't recommend indexing that much information at once. How would you suggest breaking it up and what status should I look for before doing another batch

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
the heap to 4 GB which is 50% of 8 GB RAM. Jörg On Tue, Sep 9, 2014 at 5:39 PM, Joshua P jpeter...@gmail.com javascript: wrote: Here is /etc/default/elasticsearch # Run Elasticsearch as this user ID and group ID #ES_USER=elasticsearch #ES_GROUP=elasticsearch # Heap Size (defaults to 256m

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
]]; nested: IOException[No space left on device]; -- index_type Where would I change the write location? Which config file? On Tuesday, September 9, 2014 1:28:21 PM UTC-4, Joshua P wrote: Hi Jörg, Can you elaborate on what you mean by I still need more fine tuning? I've upped the heap

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
()); } } bulkProcessor.flush(); bulkProcessor.close(); } } On Tuesday, September 9, 2014 1:57:54 PM UTC-4, Jörg Prante wrote: Check the path.data setting in config/elasticsearch.yml Jörg On Tue, Sep 9, 2014 at 7:50 PM, Joshua P jpeter...@gmail.com javascript: wrote: Just

Re: Bulk Indexing Problems

2014-09-09 Thread Joshua P
UTC-4, Jörg Prante wrote: Code looks okay, so it might be just the full volume that is in the way Jörg On Tue, Sep 9, 2014 at 8:44 PM, Joshua P jpeter...@gmail.com javascript: wrote: This is the code I've been using to index: I'm going to try to fix the running out of space issue

Elasticsearch backend?

2014-03-20 Thread Joshua P
What does Elasticsearch use to serve up responses? Thanks! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view

Re: Elasticsearch backend?

2014-03-20 Thread Joshua P
of them and a few more libraries, Elasticsearch adds distribution management, search, percolation, aggregations, etc. On Thu, Mar 20, 2014 at 3:51 PM, Joshua P jpeter...@gmail.comjavascript: wrote: What does Elasticsearch use to serve up responses? Thanks! -- You received