Re: High count of threadpools getting created

2014-08-02 Thread joergpra...@gmail.com 
A posting of how you get it to work and what pitfalls you encountered would
be awesome!

Not sure how to get this into official Elasticsearch docs, but I'm
confident the ES team would welcome anything, a pull request, or a pointer
to a blog...

Jörg


On Sat, Aug 2, 2014 at 6:04 PM, John D. Ament 
wrote:

> Jorg,
>
> Sorry, I may not have been clear.  At this point, all of the classpath
> issues are resolved.  I was wondering if you guys would be interested in a
> documentation patch where I can go over how I did this configuration, in
> case anyone else wants to try this with their JBoss/WildFly instances.
>
> As I mentioned, I'm doing the initialization via a @Singleton @Startup.
>  This is enforcing that the app scoped client is remaining one instance for
> the entire app.  In fact I had a guy yesterday turn that off "because it's
> not needed"... needless to say nothing worked.  I wish CDI provided a
> startup event, but almost there.
>
> John
>
>
> On Sat, Aug 2, 2014 at 11:41 AM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> Yes, because Elasticsearch is patching/overriding some dependencies by
>> simply reimplementing some classes, the elasticsearch jar must precede at
>> least Lucene and Joda jars in the classpath.
>>
>> All that I have left as a hint  to avoid the sudden thread pool inits is
>> to examine the code if there are unwanted class instantiations instead of
>> singletons, but I'm sure you are aware of that.
>>
>> @ApplicationScoped beans are tricky because they are initialized lazily
>> and are not really singletons in some situations. In short words I'm glad I
>> could leave all those EJB annoyances behind and rely on JSR 330 (Guice
>> style).
>>
>> Jörg
>>
>>
>> On Sat, Aug 2, 2014 at 5:07 PM, John D. Ament 
>> wrote:
>>
>>> Jorg,
>>>
>>> I've looked at that before.  While my app has a set of REST endpoints,
>>> we don't really deploy anything as servlets.  Everything we do is either
>>> EJB or CDI endpoints, so trying to configure a servlet is a pain.  I ended
>>> up just building a @Singleton @Startup to boot ES by reading in
>>> elasticsearch.yml from some config location (default = jboss's config
>>> directory).  This client is stored in an @ApplicationScoped bean and made
>>> available throughout the ES query module I put together.
>>>
>>> The quirks are more about how we're deploying.  instead of dropping 3rd
>>> party libs into our war, we drop them into the modules folder of WildFly
>>> and read them that way.  There are certain aspects of classloader
>>> segregation that this raises.  Specifically, I think there was an ordering
>>> issue between lucene and ES which causes some scoping conflicts.  This was
>>> fixed by simply putting the two of them together into a single module.  Not
>>> ideal, but it got the job done.
>>>
>>> John
>>>
>>>
>>> On Sat, Aug 2, 2014 at 11:02 AM, joergpra...@gmail.com <
>>> joergpra...@gmail.com> wrote:
>>>
 You surely had a look at the official support for ES in a servlet?


 https://github.com/elasticsearch/elasticsearch-transport-wares/blob/master/src/main/java/org/elasticsearch/wares/NodeServlet.java

 Does this approach match your setup?

 I do not really prefer this approach, instead, I use a TransportClient
 in a ServletContextListener, so I have a shared client instance across
 servlets that I can safely start and stop when Wildfly starts and stops.

 Can you clarify what are the Lucuene 4.9 quirks?

 Jörg




 On Sat, Aug 2, 2014 at 4:52 PM, John D. Ament 
 wrote:

> Actually, it was more environmental than anything.
> This happened after I was doing some module reconfiguration in wildfly
> to deploy elasticsearch as a module, instead of embedded in the WAR.  
> Turns
> out that you need to add sun.jdk as a dependency.  I didn't notice as I 
> had
> declared other dependencies on ES.  So long story short - there's some
> class being referenced in ES that wasn't exported by default.
>
> I'm not sure - would you guys be interested in a doc PR to explain how
> to deploy ES as a JBoss module?  This way no one else has to go through my
> pain.  I also found some lucene 4.9 quirks.
>
> John
>
>
> On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> Looks like a programming error.
>>
>> Without having a look at the code "ServerService" causing this, it is
>> not possible to find explanations.
>>
>> Jörg
>>
>>
>> On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament > > wrote:
>>
>>> Hi all
>>>
>>> Based on another thread I started, I'm embedding ES into my
>>> application.  Starting today, and I'm sure there's something that got
>>> thrown into the mix causing this, we have started seeing high counts of
>>> thread pools getting created and the application never fully booting

Re: High count of threadpools getting created

2014-08-02 Thread John D. Ament 
Jorg,

Sorry, I may not have been clear.  At this point, all of the classpath
issues are resolved.  I was wondering if you guys would be interested in a
documentation patch where I can go over how I did this configuration, in
case anyone else wants to try this with their JBoss/WildFly instances.

As I mentioned, I'm doing the initialization via a @Singleton @Startup.
 This is enforcing that the app scoped client is remaining one instance for
the entire app.  In fact I had a guy yesterday turn that off "because it's
not needed"... needless to say nothing worked.  I wish CDI provided a
startup event, but almost there.

John


On Sat, Aug 2, 2014 at 11:41 AM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Yes, because Elasticsearch is patching/overriding some dependencies by
> simply reimplementing some classes, the elasticsearch jar must precede at
> least Lucene and Joda jars in the classpath.
>
> All that I have left as a hint  to avoid the sudden thread pool inits is
> to examine the code if there are unwanted class instantiations instead of
> singletons, but I'm sure you are aware of that.
>
> @ApplicationScoped beans are tricky because they are initialized lazily
> and are not really singletons in some situations. In short words I'm glad I
> could leave all those EJB annoyances behind and rely on JSR 330 (Guice
> style).
>
> Jörg
>
>
> On Sat, Aug 2, 2014 at 5:07 PM, John D. Ament 
> wrote:
>
>> Jorg,
>>
>> I've looked at that before.  While my app has a set of REST endpoints, we
>> don't really deploy anything as servlets.  Everything we do is either EJB
>> or CDI endpoints, so trying to configure a servlet is a pain.  I ended up
>> just building a @Singleton @Startup to boot ES by reading in
>> elasticsearch.yml from some config location (default = jboss's config
>> directory).  This client is stored in an @ApplicationScoped bean and made
>> available throughout the ES query module I put together.
>>
>> The quirks are more about how we're deploying.  instead of dropping 3rd
>> party libs into our war, we drop them into the modules folder of WildFly
>> and read them that way.  There are certain aspects of classloader
>> segregation that this raises.  Specifically, I think there was an ordering
>> issue between lucene and ES which causes some scoping conflicts.  This was
>> fixed by simply putting the two of them together into a single module.  Not
>> ideal, but it got the job done.
>>
>> John
>>
>>
>> On Sat, Aug 2, 2014 at 11:02 AM, joergpra...@gmail.com <
>> joergpra...@gmail.com> wrote:
>>
>>> You surely had a look at the official support for ES in a servlet?
>>>
>>>
>>> https://github.com/elasticsearch/elasticsearch-transport-wares/blob/master/src/main/java/org/elasticsearch/wares/NodeServlet.java
>>>
>>> Does this approach match your setup?
>>>
>>> I do not really prefer this approach, instead, I use a TransportClient
>>> in a ServletContextListener, so I have a shared client instance across
>>> servlets that I can safely start and stop when Wildfly starts and stops.
>>>
>>> Can you clarify what are the Lucuene 4.9 quirks?
>>>
>>> Jörg
>>>
>>>
>>>
>>>
>>> On Sat, Aug 2, 2014 at 4:52 PM, John D. Ament 
>>> wrote:
>>>
 Actually, it was more environmental than anything.
 This happened after I was doing some module reconfiguration in wildfly
 to deploy elasticsearch as a module, instead of embedded in the WAR.  Turns
 out that you need to add sun.jdk as a dependency.  I didn't notice as I had
 declared other dependencies on ES.  So long story short - there's some
 class being referenced in ES that wasn't exported by default.

 I'm not sure - would you guys be interested in a doc PR to explain how
 to deploy ES as a JBoss module?  This way no one else has to go through my
 pain.  I also found some lucene 4.9 quirks.

 John


 On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
 joergpra...@gmail.com> wrote:

> Looks like a programming error.
>
> Without having a look at the code "ServerService" causing this, it is
> not possible to find explanations.
>
> Jörg
>
>
> On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
> wrote:
>
>> Hi all
>>
>> Based on another thread I started, I'm embedding ES into my
>> application.  Starting today, and I'm sure there's something that got
>> thrown into the mix causing this, we have started seeing high counts of
>> thread pools getting created and the application never fully booting.
>>
>> I see this in the logs at a fine level:
>>
>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool 
>> [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool 
>> [generic],
>> type [cached], keep

Re: High count of threadpools getting created

2014-08-02 Thread joergpra...@gmail.com 
Yes, because Elasticsearch is patching/overriding some dependencies by
simply reimplementing some classes, the elasticsearch jar must precede at
least Lucene and Joda jars in the classpath.

All that I have left as a hint  to avoid the sudden thread pool inits is to
examine the code if there are unwanted class instantiations instead of
singletons, but I'm sure you are aware of that.

@ApplicationScoped beans are tricky because they are initialized lazily and
are not really singletons in some situations. In short words I'm glad I
could leave all those EJB annoyances behind and rely on JSR 330 (Guice
style).

Jörg


On Sat, Aug 2, 2014 at 5:07 PM, John D. Ament 
wrote:

> Jorg,
>
> I've looked at that before.  While my app has a set of REST endpoints, we
> don't really deploy anything as servlets.  Everything we do is either EJB
> or CDI endpoints, so trying to configure a servlet is a pain.  I ended up
> just building a @Singleton @Startup to boot ES by reading in
> elasticsearch.yml from some config location (default = jboss's config
> directory).  This client is stored in an @ApplicationScoped bean and made
> available throughout the ES query module I put together.
>
> The quirks are more about how we're deploying.  instead of dropping 3rd
> party libs into our war, we drop them into the modules folder of WildFly
> and read them that way.  There are certain aspects of classloader
> segregation that this raises.  Specifically, I think there was an ordering
> issue between lucene and ES which causes some scoping conflicts.  This was
> fixed by simply putting the two of them together into a single module.  Not
> ideal, but it got the job done.
>
> John
>
>
> On Sat, Aug 2, 2014 at 11:02 AM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> You surely had a look at the official support for ES in a servlet?
>>
>>
>> https://github.com/elasticsearch/elasticsearch-transport-wares/blob/master/src/main/java/org/elasticsearch/wares/NodeServlet.java
>>
>> Does this approach match your setup?
>>
>> I do not really prefer this approach, instead, I use a TransportClient in
>> a ServletContextListener, so I have a shared client instance across
>> servlets that I can safely start and stop when Wildfly starts and stops.
>>
>> Can you clarify what are the Lucuene 4.9 quirks?
>>
>> Jörg
>>
>>
>>
>>
>> On Sat, Aug 2, 2014 at 4:52 PM, John D. Ament 
>> wrote:
>>
>>> Actually, it was more environmental than anything.
>>> This happened after I was doing some module reconfiguration in wildfly
>>> to deploy elasticsearch as a module, instead of embedded in the WAR.  Turns
>>> out that you need to add sun.jdk as a dependency.  I didn't notice as I had
>>> declared other dependencies on ES.  So long story short - there's some
>>> class being referenced in ES that wasn't exported by default.
>>>
>>> I'm not sure - would you guys be interested in a doc PR to explain how
>>> to deploy ES as a JBoss module?  This way no one else has to go through my
>>> pain.  I also found some lucene 4.9 quirks.
>>>
>>> John
>>>
>>>
>>> On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
>>> joergpra...@gmail.com> wrote:
>>>
 Looks like a programming error.

 Without having a look at the code "ServerService" causing this, it is
 not possible to find explanations.

 Jörg


 On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
 wrote:

> Hi all
>
> Based on another thread I started, I'm embedding ES into my
> application.  Starting today, and I'm sure there's something that got
> thrown into the mix causing this, we have started seeing high counts of
> thread pools getting created and the application never fully booting.
>
> I see this in the logs at a fine level:
>
> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>>

Re: High count of threadpools getting created

2014-08-02 Thread John D. Ament 
Jorg,

I've looked at that before.  While my app has a set of REST endpoints, we
don't really deploy anything as servlets.  Everything we do is either EJB
or CDI endpoints, so trying to configure a servlet is a pain.  I ended up
just building a @Singleton @Startup to boot ES by reading in
elasticsearch.yml from some config location (default = jboss's config
directory).  This client is stored in an @ApplicationScoped bean and made
available throughout the ES query module I put together.

The quirks are more about how we're deploying.  instead of dropping 3rd
party libs into our war, we drop them into the modules folder of WildFly
and read them that way.  There are certain aspects of classloader
segregation that this raises.  Specifically, I think there was an ordering
issue between lucene and ES which causes some scoping conflicts.  This was
fixed by simply putting the two of them together into a single module.  Not
ideal, but it got the job done.

John


On Sat, Aug 2, 2014 at 11:02 AM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> You surely had a look at the official support for ES in a servlet?
>
>
> https://github.com/elasticsearch/elasticsearch-transport-wares/blob/master/src/main/java/org/elasticsearch/wares/NodeServlet.java
>
> Does this approach match your setup?
>
> I do not really prefer this approach, instead, I use a TransportClient in
> a ServletContextListener, so I have a shared client instance across
> servlets that I can safely start and stop when Wildfly starts and stops.
>
> Can you clarify what are the Lucuene 4.9 quirks?
>
> Jörg
>
>
>
>
> On Sat, Aug 2, 2014 at 4:52 PM, John D. Ament 
> wrote:
>
>> Actually, it was more environmental than anything.
>> This happened after I was doing some module reconfiguration in wildfly to
>> deploy elasticsearch as a module, instead of embedded in the WAR.  Turns
>> out that you need to add sun.jdk as a dependency.  I didn't notice as I had
>> declared other dependencies on ES.  So long story short - there's some
>> class being referenced in ES that wasn't exported by default.
>>
>> I'm not sure - would you guys be interested in a doc PR to explain how to
>> deploy ES as a JBoss module?  This way no one else has to go through my
>> pain.  I also found some lucene 4.9 quirks.
>>
>> John
>>
>>
>> On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
>> joergpra...@gmail.com> wrote:
>>
>>> Looks like a programming error.
>>>
>>> Without having a look at the code "ServerService" causing this, it is
>>> not possible to find explanations.
>>>
>>> Jörg
>>>
>>>
>>> On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
>>> wrote:
>>>
 Hi all

 Based on another thread I started, I'm embedding ES into my
 application.  Starting today, and I'm sure there's something that got
 thrown into the mix causing this, we have started seeing high counts of
 thread pools getting created and the application never fully booting.

 I see this in the logs at a fine level:

 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]

 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
 (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
 type [cached], keep_alive [30s]


 I wouldn't expect so many threadpools to get created.  Is this expected?


 John

 --
>>>

Re: High count of threadpools getting created

2014-08-02 Thread joergpra...@gmail.com 
You surely had a look at the official support for ES in a servlet?

https://github.com/elasticsearch/elasticsearch-transport-wares/blob/master/src/main/java/org/elasticsearch/wares/NodeServlet.java

Does this approach match your setup?

I do not really prefer this approach, instead, I use a TransportClient in a
ServletContextListener, so I have a shared client instance across servlets
that I can safely start and stop when Wildfly starts and stops.

Can you clarify what are the Lucuene 4.9 quirks?

Jörg




On Sat, Aug 2, 2014 at 4:52 PM, John D. Ament 
wrote:

> Actually, it was more environmental than anything.
> This happened after I was doing some module reconfiguration in wildfly to
> deploy elasticsearch as a module, instead of embedded in the WAR.  Turns
> out that you need to add sun.jdk as a dependency.  I didn't notice as I had
> declared other dependencies on ES.  So long story short - there's some
> class being referenced in ES that wasn't exported by default.
>
> I'm not sure - would you guys be interested in a doc PR to explain how to
> deploy ES as a JBoss module?  This way no one else has to go through my
> pain.  I also found some lucene 4.9 quirks.
>
> John
>
>
> On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> Looks like a programming error.
>>
>> Without having a look at the code "ServerService" causing this, it is not
>> possible to find explanations.
>>
>> Jörg
>>
>>
>> On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
>> wrote:
>>
>>> Hi all
>>>
>>> Based on another thread I started, I'm embedding ES into my application.
>>>  Starting today, and I'm sure there's something that got thrown into the
>>> mix causing this, we have started seeing high counts of thread pools
>>> getting created and the application never fully booting.
>>>
>>> I see this in the logs at a fine level:
>>>
>>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>>> type [cached], keep_alive [30s]
>>>
>>>
>>> I wouldn't expect so many threadpools to get created.  Is this expected?
>>>
>>>
>>> John
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/e9388230-6f83-4aa9-8752-5e9c47625b03%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "elasticsearch" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/elasticsearch/8IUdimKBEIo/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGu4WXCh9qAr4EPOmWoshFpYP7GG06Rw7G9wC05Avgzkg%40mail.gmail.com
>>

Re: High count of threadpools getting created

2014-08-02 Thread John D. Ament 
Actually, it was more environmental than anything.
This happened after I was doing some module reconfiguration in wildfly to
deploy elasticsearch as a module, instead of embedded in the WAR.  Turns
out that you need to add sun.jdk as a dependency.  I didn't notice as I had
declared other dependencies on ES.  So long story short - there's some
class being referenced in ES that wasn't exported by default.

I'm not sure - would you guys be interested in a doc PR to explain how to
deploy ES as a JBoss module?  This way no one else has to go through my
pain.  I also found some lucene 4.9 quirks.

John


On Sat, Aug 2, 2014 at 10:43 AM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Looks like a programming error.
>
> Without having a look at the code "ServerService" causing this, it is not
> possible to find explanations.
>
> Jörg
>
>
> On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
> wrote:
>
>> Hi all
>>
>> Based on another thread I started, I'm embedding ES into my application.
>>  Starting today, and I'm sure there's something that got thrown into the
>> mix causing this, we have started seeing high counts of thread pools
>> getting created and the application never fully booting.
>>
>> I see this in the logs at a fine level:
>>
>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
>> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
>> type [cached], keep_alive [30s]
>>
>>
>> I wouldn't expect so many threadpools to get created.  Is this expected?
>>
>>
>> John
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/e9388230-6f83-4aa9-8752-5e9c47625b03%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/8IUdimKBEIo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGu4WXCh9qAr4EPOmWoshFpYP7GG06Rw7G9wC05Avgzkg%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOqetn9o2GJWs_Qaw3GCvy0VrGCtxdmcAfr4_fgQ2a5GkLxJbw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: High count of threadpools getting created

2014-08-02 Thread joergpra...@gmail.com 
Looks like a programming error.

Without having a look at the code "ServerService" causing this, it is not
possible to find explanations.

Jörg


On Fri, Aug 1, 2014 at 6:38 PM, John D. Ament 
wrote:

> Hi all
>
> Based on another thread I started, I'm embedding ES into my application.
>  Starting today, and I'm sure there's something that got thrown into the
> mix causing this, we have started seeing high counts of thread pools
> getting created and the application never fully booting.
>
> I see this in the logs at a fine level:
>
> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
> 2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool]
> (ServerService Thread Pool -- 19) [Sauron] creating thread_pool [generic],
> type [cached], keep_alive [30s]
>
>
> I wouldn't expect so many threadpools to get created.  Is this expected?
>
>
> John
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/e9388230-6f83-4aa9-8752-5e9c47625b03%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGu4WXCh9qAr4EPOmWoshFpYP7GG06Rw7G9wC05Avgzkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

High count of threadpools getting created

2014-08-01 Thread John D. Ament 
Hi all

Based on another thread I started, I'm embedding ES into my application. 
 Starting today, and I'm sure there's something that got thrown into the 
mix causing this, we have started seeing high counts of thread pools 
getting created and the application never fully booting.

I see this in the logs at a fine level:

2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,731 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]

2014-08-01 12:34:32,732 FINE  [org.elasticsearch.threadpool] (ServerService 
Thread Pool -- 19) [Sauron] creating thread_pool [generic], type [cached], 
keep_alive [30s]


I wouldn't expect so many threadpools to get created.  Is this expected?


John

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e9388230-6f83-4aa9-8752-5e9c47625b03%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.