Re: Occational client.transport.NoNodeAvailableException

[joergpra...@gmail.com]

I am not able to examine your bulk code. From the output "no node
available" it seems you overwhelmed the cluster, yes. Testing bulk feeds on
a local machine only is kind of limited.

You have 5m/sec which is quite much for a small local ES cluster (without
knowing your data I can't confirm though, simple log data would make it).
Reset bulk to 1m/sec and see how it goes.

BTW your client settings contain some misunderstandings. You can't disable
the refresh rate there and you can't set merge factors. This must be set on
server side.

Jörg


On Fri, Mar 14, 2014 at 9:20 PM, ZenMaster80  wrote:

> I will post logs in a bit. I plan to wun on EC2, but currently just
> running on a local machine i7, 4G Ram.
>
> I had  int concurrentRequests =  Runtime.getRuntime().availableProcessors();
> (Returns 8),
> If I change this value to just "1", I don't get the exception, but
> indexing performance slows down considerably. I am not sure if 8 requests
> is really overwhelming the node.
>
> On Friday, March 14, 2014 3:58:21 PM UTC-4, Binh Ly wrote:
>>
>> I'm curious, is there anything else in the es log files? Also are you
>> running on EC2 micro instances?
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1325173c-230d-416a-8a25-3b2201fa987a%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH10u6yLdPRp-xiP6V7Xn7SRgOMzmzAptijbmzwXtJ_kQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Occational client.transport.NoNodeAvailableException

[ZenMaster80]

I will post logs in a bit. I plan to wun on EC2, but currently just running 
on a local machine i7, 4G Ram.

I had  int concurrentRequests =  Runtime.getRuntime().availableProcessors(); 
(Returns 8), 
If I change this value to just "1", I don't get the exception, but indexing 
performance slows down considerably. I am not sure if 8 requests is really 
overwhelming the node.

On Friday, March 14, 2014 3:58:21 PM UTC-4, Binh Ly wrote:
>
> I'm curious, is there anything else in the es log files? Also are you 
> running on EC2 micro instances?
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1325173c-230d-416a-8a25-3b2201fa987a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Occational client.transport.NoNodeAvailableException

[Binh Ly]

I'm curious, is there anything else in the es log files? Also are you 
running on EC2 micro instances?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/61021936-8052-49b8-bcbf-495edfb173df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Occational client.transport.NoNodeAvailableException

[ZenMaster80]

Anyone ran into this problem? I am happening to see it when I process 
larger documents. I am running one node, one shard.. I played with 
HeapSize, but didn't matter.
In the example below, it missed 5 or 6 blocks, that's a lot of documents 
that it didn't index. Any clues?

On Thursday, March 13, 2014 2:02:36 PM UTC-4, ZenMaster80 wrote:
>
> I get this during bulk processing (see the Error in the middle of the log) 
> - I tried it on my local machine, it doesn't happen, if I run it on some 
> remote EC2 instance, I do get the exception. Here is some output:
>
> Settings settings = ImmutableSettings.settingsBuilder()
>
>.put("client.transport.sniff", true
> )
>
>.put("refresh_interval", "-1")  
> //Needs 
> to be updated after indexing is done
>
>.put("number_of_shards", 1)
>
>.put("number_of_replicas", "0")
>
>.put(
> "index.merge.policy.merge_factor", 30)
>
>.put(
> "client.transport.ping_timeout", "10s")  //didn't help for exception error
>
>//.put("discovery.type", "ec2")
>
>//.put("cloud.aws.access_key", 
> "==")
>
>// .put("cloud.aws.secret_key", 
> "===")
>
>//.put("cloud.aws.region", "eu
> -west-1")
>
>//.put("discovery.ec2.groups", "
> elasticsearch")
>
>.put("cluster.name", this.CLUSTER_NAME
> ).build();
>
>
>
> Bulk Called: ID= 1, Actions=33, MB=5.461668
>
> Bulk Called: ID= 2, Actions=29, MB=5.515789
>
> Bulk Called: ID= 3, Actions=12, MB=5.691476
>
> Bulk Called: ID= 4, Actions=17, MB=5.444193
>
> Bulk Called: ID= 5, Actions=9, MB=5.492445
>
> Bulk Called: ID= 6, Actions=17, MB=6.293821
>
> Bulk Called: ID= 7, Actions=16, MB=5.769860
>
> Bulk Called: ID= 8, Actions=18, MB=5.259554
>
> Bulk Succeeded: ID= 1, took= 3471 ms
>
> Bulk Called: ID= 9, Actions=11, MB=5.757115
>
> Bulk Succeeded: ID= 2, took= 4207 ms
>
> Bulk Called: ID= 10, Actions=13, MB=5.591648
>
> Bulk Succeeded: ID= 3, took= 2453 ms
>
> Bulk Called: ID= 11, Actions=13, MB=5.355669
>
> Bulk Succeeded: ID= 4, took= 2568 ms
>
> Bulk Called: ID= 12, Actions=16, MB=5.269400
>
> Bulk Succeeded: ID= 5, took= 2233 ms
>
> Bulk Called: ID= 13, Actions=14, MB=5.357479
>
> Bulk Succeeded: ID= 6, took= 2434 ms
>
> Bulk Called: ID= 14, Actions=18, MB=5.899198
>
> Error: ID=7, org.elasticsearch.client.transport.NoNodeAvailableException: 
> No node availableError: ID=11, org.elasticsearch.client.transport.
> NoNodeAvailableException: No node availableError: ID=12, org.elasticsearch
> .client.transport.NoNodeAvailableException: No node availableError: ID=8, 
> org.elasticsearch.client.transport.NoNodeAvailableException: No node 
> availableError: ID=13, org.elasticsearch.client.transport.
> NoNodeAvailableException: No node availableError: ID=14, org.elasticsearch
> .client.transport.NoNodeAvailableException: No node availableError: ID=9, 
> org.elasticsearch.client.transport.NoNodeAvailableException: No node 
> availableError: ID=10, org.elasticsearch.client.transport.
> NoNodeAvailableException: No node availableBulk Called: ID= 15, Actions=20
> , MB=6.751245
>
> Error: ID=15, org.elasticsearch.client.transport.NoNodeAvailableException: 
> No node availableError: ID=15, org.elasticsearch.client.transport.
> NoNodeAvailableException: No node availableBulk Called: ID= 16, Actions=12
> , MB=5.320580
>
> Bulk Called: ID= 17, Actions=12, MB=5.31201
>
> Bulk Called: ID= 18, Actions=15, MB=5.331088
> Bulk Called: ID= 19, Actions=24, MB=5.307172
>
> Bulk Called: ID= 20, Actions=32, MB=6.009871
>
> Bulk Called: ID= 21, Actions=9, MB=6.020198
>
> Bulk Called: ID= 22, Actions=1, MB=6.279574
>
> Bulk Called: ID= 23, Actions=8, MB=5.452716
>
> Bulk Called: ID= 24, Actions=19, MB=5.383386
>
> Bulk Succeeded: ID= 16, took= 3604 ms
>
> Bulk Called: ID= 25, Actions=22, MB=5.347178
>
> Bulk Succeeded: ID= 17, took= 3926 ms
>
> Bulk Called: ID= 26, Actions=58, MB=5.247683
>
> Bulk Succeeded: ID= 18, took= 5014 ms
>
> Bulk Called: ID= 27, Actions=90, MB=5.312535
>
> Bulk Succeeded: ID= 19, took= 2253 ms
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/324002a9-ac0c-4d8f-b954-99d46265e4f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.