Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-11-02 Thread Jason Wee
unbounded queue for threadpool, isn't changes such as unbounded will 
exhaust system resources? It could either exhaust system resources due to 
each thread use specific amount of system resources, thus render the system 
not stable or if the system has file descriptor set, probably via ulimit, 
that number of processes per user hit the limit?

On Monday, August 18, 2014 9:34:52 PM UTC+8, Sávio Salvarino Teles de 
Oliveira wrote:
>
> You can put *threadpool.search.type: **cached* on elasticsearch.yml for 
> unbounded queue for reads.
>
>
> 2014-08-10 9:52 GMT-03:00 James >:
>
>>  On Sat, 2014-08-09 at 23:53 -0700, Deep wrote: 
>>
>> Hi, 
>>
>>  
>>
>>  Elastic search internally has thread pool and a queue size is 
>> associated with each pool. You can have pools for search threads, index 
>> threads etc. Please see the elastic search documentation in link 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-threadpool.html
>>  
>> . I think it is possible to override these properties in the 
>> elasticsearch.yml configuration file. 
>>
>>  
>>
>>  Regards, 
>>
>>  Ishwardeep
>>
>> On Saturday, 9 August 2014 00:54:02 UTC+5:30, digit...@gmail.com wrote:
>>
>> So I've seen a few posts on this, but I've not seen any solutions 
>> posted.  I've been log monitoring and I was trying to determine how to fix 
>> the below...any information would be great thank you. 
>>
>> [2014-08-08 19:14:12,578][DEBUG][action.search.type   ] [Jericho 
>> Drumm] [bro-201408032100][2], node[fgjxNK0cQ3O5Usn7wyjaMA], [P], 
>> s[STARTED]: Failed to execute 
>> [org.elasticsearch.action.search.SearchRequest@126067b7] lastShard [true]
>> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: 
>> rejected execution (queue capacity 1000) on 
>> org.elasticsearch.search.action.SearchServiceTransportAction$23@5a879352
>> at 
>> org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
>> at 
>> org.elasticsearch.search.action.SearchServiceTransportAction.execute(SearchServiceTransportAction.java:509)
>> at 
>> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
>> at 
>> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
>> at 
>> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:171)
>> at 
>> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:153)
>> at 
>> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:59)
>> at 
>> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:49)
>> at 
>> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
>> at 
>> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101)
>> at 
>> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
>> at 
>> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
>> at 
>> org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
>> at 
>> org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:212)
>> at 
>> org.elasticsearch.rest.action.search.RestSearchAction.handleRequest(RestSearchAction.java:75)
>> at 
>> org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
>> at 
>> org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
>> at 
>> org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
>> at 
>> org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:83)
>> at 
>> org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:294)
>> at 
>> org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:44)
>> at 
>> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
>> at 
>> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>> at 
>> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>> at 
>> org.elasticsearch.common.netty.han

Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-11-02 Thread spancer ray
Hi James,

Have you fixed it?



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/EsRejectedExecutionException-rejected-execution-queue-capacity-1000-tp4061509p4065613.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1414976673010-4065613.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.


Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-08-20 Thread James
On Mon, 2014-08-18 at 10:34 -0300, Sávio S. Teles de Oliveira wrote:
> You can put threadpool.search.type: cached on elasticsearch.yml for
> unbounded queue for reads.
> 
> 
> 
> 

Thank you...that helps as well.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1408533672.5479.20.camel%40JamesiMac.
For more options, visit https://groups.google.com/d/optout.


Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-08-18 Thread Sávio S . Teles de Oliveira
You can put *threadpool.search.type: **cached* on elasticsearch.yml for
unbounded queue for reads.


2014-08-10 9:52 GMT-03:00 James :

>  On Sat, 2014-08-09 at 23:53 -0700, Deep wrote:
>
> Hi,
>
>
>
>  Elastic search internally has thread pool and a queue size is associated
> with each pool. You can have pools for search threads, index threads etc.
> Please see the elastic search documentation in link
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-threadpool.html
> . I think it is possible to override these properties in the
> elasticsearch.yml configuration file.
>
>
>
>  Regards,
>
>  Ishwardeep
>
> On Saturday, 9 August 2014 00:54:02 UTC+5:30, digit...@gmail.com wrote:
>
> So I've seen a few posts on this, but I've not seen any solutions posted.
> I've been log monitoring and I was trying to determine how to fix the
> below...any information would be great thank you.
>
> [2014-08-08 19:14:12,578][DEBUG][action.search.type   ] [Jericho
> Drumm] [bro-201408032100][2], node[fgjxNK0cQ3O5Usn7wyjaMA], [P],
> s[STARTED]: Failed to execute
> [org.elasticsearch.action.search.SearchRequest@126067b7] lastShard [true]
> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException:
> rejected execution (queue capacity 1000) on
> org.elasticsearch.search.action.SearchServiceTransportAction$23@5a879352
> at
> org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
> at
> org.elasticsearch.search.action.SearchServiceTransportAction.execute(SearchServiceTransportAction.java:509)
> at
> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
> at
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:171)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:153)
> at
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:59)
> at
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:49)
> at
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101)
> at
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
> at
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at
> org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
> at
> org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:212)
> at
> org.elasticsearch.rest.action.search.RestSearchAction.handleRequest(RestSearchAction.java:75)
> at
> org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
> at
> org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
> at
> org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
> at
> org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:83)
> at
> org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:294)
> at
> org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:44)
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.elasticsearch.common.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.j

Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-08-10 Thread James
On Sat, 2014-08-09 at 23:53 -0700, Deep wrote:
> Hi,
> 
> 
> 
> Elastic search internally has thread pool and a queue size is
> associated with each pool. You can have pools for search threads,
> index threads etc. Please see the elastic search documentation in
> link 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-threadpool.html
>  . I think it is possible to override these properties in the 
> elasticsearch.yml configuration file.
> 
> 
> Regards,
> Ishwardeep
> 
> On Saturday, 9 August 2014 00:54:02 UTC+5:30, digit...@gmail.com
> wrote:
> 
> So I've seen a few posts on this, but I've not seen any
> solutions posted.  I've been log monitoring and I was trying
> to determine how to fix the below...any information would be
> great thank you. 
> 
> [2014-08-08 19:14:12,578][DEBUG][action.search.type   ]
> [Jericho Drumm] [bro-201408032100][2],
> node[fgjxNK0cQ3O5Usn7wyjaMA], [P], s[STARTED]: Failed to
> execute
> [org.elasticsearch.action.search.SearchRequest@126067b7]
> lastShard [true]
> 
> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: 
> rejected execution (queue capacity 1000) on 
> org.elasticsearch.search.action.SearchServiceTransportAction$23@5a879352
> at
> 
> org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62)
> at
> 
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at
> 
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
> at
> 
> org.elasticsearch.search.action.SearchServiceTransportAction.execute(SearchServiceTransportAction.java:509)
> at
> 
> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
> at
> 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction
> $BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:171)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction
> $BaseAsyncAction.start(TransportSearchTypeAction.java:153)
> at
> 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:59)
> at
> 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:49)
> at
> 
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at
> 
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101)
> at
> 
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
> at
> 
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at
> org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
> at
> 
> org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:212)
> at
> 
> org.elasticsearch.rest.action.search.RestSearchAction.handleRequest(RestSearchAction.java:75)
> at
> 
> org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
> at
> 
> org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
> at
> 
> org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
> at org.elasticsearch.http.HttpServer
> $Dispatcher.dispatchRequest(HttpServer.java:83)
> at
> 
> org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:294)
> at
> 
> org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:44)
> at
> 
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> 
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline
> 
> $DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> 
> org.elasticsearch.common.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.jav

Re: EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-08-09 Thread Deep
Hi,

Elastic search internally has thread pool and a queue size is associated 
with each pool. You can have pools for search threads, index threads etc. 
Please see the elastic search documentation in 
link 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-threadpool.html
 
. I think it is possible to override these properties in the 
elasticsearch.yml configuration file.

Regards,
Ishwardeep

On Saturday, 9 August 2014 00:54:02 UTC+5:30, digit...@gmail.com wrote:
>
> So I've seen a few posts on this, but I've not seen any solutions posted.  
> I've been log monitoring and I was trying to determine how to fix the 
> below...any information would be great thank you. 
>
> [2014-08-08 19:14:12,578][DEBUG][action.search.type   ] [Jericho 
> Drumm] [bro-201408032100][2], node[fgjxNK0cQ3O5Usn7wyjaMA], [P], 
> s[STARTED]: Failed to execute 
> [org.elasticsearch.action.search.SearchRequest@126067b7] lastShard [true]
> org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: 
> rejected execution (queue capacity 1000) on 
> org.elasticsearch.search.action.SearchServiceTransportAction$23@5a879352
> at 
> org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62)
> at 
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at 
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
> at 
> org.elasticsearch.search.action.SearchServiceTransportAction.execute(SearchServiceTransportAction.java:509)
> at 
> org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
> at 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at 
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:171)
> at 
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:153)
> at 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:59)
> at 
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:49)
> at 
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at 
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101)
> at 
> org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
> at 
> org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
> at 
> org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
> at 
> org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:212)
> at 
> org.elasticsearch.rest.action.search.RestSearchAction.handleRequest(RestSearchAction.java:75)
> at 
> org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
> at 
> org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
> at 
> org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
> at 
> org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:83)
> at 
> org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:294)
> at 
> org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:44)
> at 
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at 
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at 
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at 
> org.elasticsearch.common.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
> at 
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at 
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at 
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at 
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at 
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
> at 
> org.elasticsearch.co

EsRejectedExecutionException: rejected execution (queue capacity 1000)

2014-08-08 Thread digitalx00
So I've seen a few posts on this, but I've not seen any solutions posted.  
I've been log monitoring and I was trying to determine how to fix the 
below...any information would be great thank you. 

[2014-08-08 19:14:12,578][DEBUG][action.search.type   ] [Jericho Drumm] 
[bro-201408032100][2], node[fgjxNK0cQ3O5Usn7wyjaMA], [P], s[STARTED]: 
Failed to execute [org.elasticsearch.action.search.SearchRequest@126067b7] 
lastShard [true]
org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: 
rejected execution (queue capacity 1000) on 
org.elasticsearch.search.action.SearchServiceTransportAction$23@5a879352
at 
org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:62)
at 
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at 
org.elasticsearch.search.action.SearchServiceTransportAction.execute(SearchServiceTransportAction.java:509)
at 
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:203)
at 
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at 
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:171)
at 
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:153)
at 
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:59)
at 
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:49)
at 
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
at 
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101)
at 
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
at 
org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
at 
org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
at 
org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:212)
at 
org.elasticsearch.rest.action.search.RestSearchAction.handleRequest(RestSearchAction.java:75)
at 
org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
at 
org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
at 
org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
at 
org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:83)
at 
org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:294)
at 
org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:44)
at 
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at 
org.elasticsearch.common.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
at 
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at 
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at 
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at 
org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at 
org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at 
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at 
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at 
org.elasticsearch.common.net