Re: Multiple concurrent queries to Solr
Thanks, everyone. Arcadius, that ticket is interesting. I was wondering if an implementation of SolrClient could be based on HttpAsyncClient instead of HttpSolrClient. Just a thought right now, which needs to be explored deeper. - Ashish On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com wrote: Hello Ashish. Therse is an unfinished work about this at https://issues.apache.org/jira/browse/SOLR-3383 Maybe you want to have a look and contribute? Arcadius. On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org wrote: The last time that I used the HTTPClient library, it was non-blocking. It doesn’t try to read from the socket until you ask for data from the response object. That allows parallel requests without threads. Underneath, it has a pool of connections that can be reused. If the pool is exhausted, it can block. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote: On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com ---
Re: Multiple concurrent queries to Solr
BTW, google revealed that there is a 3rd-party Scala library for async calls which could be usable from Java. I have not tried it myself though https://github.com/inoio/solrs On 24 August 2015 at 21:35, Arcadius Ahouansou arcad...@menelic.com wrote: Hi Ashish. The Apache HttpAsyncClient uses Java Future to wrap a synchronous call into asyn The above ticket does similar thing by wrapping a SolrJ call into Future Feel free to submit any proposal you may have to the dev list. Arcadius On 24 August 2015 at 07:20, Ashish Mukherjee ashish.mukher...@gmail.com wrote: Thanks, everyone. Arcadius, that ticket is interesting. I was wondering if an implementation of SolrClient could be based on HttpAsyncClient instead of HttpSolrClient. Just a thought right now, which needs to be explored deeper. - Ashish On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com wrote: Hello Ashish. Therse is an unfinished work about this at https://issues.apache.org/jira/browse/SOLR-3383 Maybe you want to have a look and contribute? Arcadius. On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org wrote: The last time that I used the HTTPClient library, it was non-blocking. It doesn’t try to read from the socket until you ask for data from the response object. That allows parallel requests without threads. Underneath, it has a pool of connections that can be reused. If the pool is exhausted, it can block. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote: On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com --- -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com --- -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com ---
Re: Multiple concurrent queries to Solr
Hi Ashish. The Apache HttpAsyncClient uses Java Future to wrap a synchronous call into asyn The above ticket does similar thing by wrapping a SolrJ call into Future Feel free to submit any proposal you may have to the dev list. Arcadius On 24 August 2015 at 07:20, Ashish Mukherjee ashish.mukher...@gmail.com wrote: Thanks, everyone. Arcadius, that ticket is interesting. I was wondering if an implementation of SolrClient could be based on HttpAsyncClient instead of HttpSolrClient. Just a thought right now, which needs to be explored deeper. - Ashish On Mon, Aug 24, 2015 at 1:46 AM, Arcadius Ahouansou arcad...@menelic.com wrote: Hello Ashish. Therse is an unfinished work about this at https://issues.apache.org/jira/browse/SOLR-3383 Maybe you want to have a look and contribute? Arcadius. On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org wrote: The last time that I used the HTTPClient library, it was non-blocking. It doesn’t try to read from the socket until you ask for data from the response object. That allows parallel requests without threads. Underneath, it has a pool of connections that can be reused. If the pool is exhausted, it can block. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote: On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com --- -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com ---
Re: Multiple concurrent queries to Solr
On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn
Re: Multiple concurrent queries to Solr
The last time that I used the HTTPClient library, it was non-blocking. It doesn’t try to read from the socket until you ask for data from the response object. That allows parallel requests without threads. Underneath, it has a pool of connections that can be reused. If the pool is exhausted, it can block. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote: On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn
Re: Multiple concurrent queries to Solr
Hello Ashish. Therse is an unfinished work about this at https://issues.apache.org/jira/browse/SOLR-3383 Maybe you want to have a look and contribute? Arcadius. On 23 August 2015 at 17:02, Walter Underwood wun...@wunderwood.org wrote: The last time that I used the HTTPClient library, it was non-blocking. It doesn’t try to read from the socket until you ask for data from the response object. That allows parallel requests without threads. Underneath, it has a pool of connections that can be reused. If the pool is exhausted, it can block. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Aug 23, 2015, at 8:49 AM, Shawn Heisey apa...@elyograg.org wrote: On 8/23/2015 7:46 AM, Ashish Mukherjee wrote: I want to run few Solr queries in parallel, which are being done in a multi-threaded model now. I was wondering if there are any client libraries to query Solr through a non-blocking I/O mechanism instead of a threaded model. Has anyone attempted something like this? The only client library that the Solr project makes is SolrJ -- the client for Java. If you are not using the SolrJ client, then the Solr project did not write it, and you should contact the authors of the library directly. SolrJ and Solr are both completely thread-safe, and multiple threads are recommended for highly concurrent usage. SolrJ uses HttpClient for communication with Solr. I was not able to determine whether the default httpclient settings will result in non-blocking I/O or not. As far as I am aware, nothing in SolrJ sets any explicit configuration for blocking or non-blocking I/O. You can create your own HttpClient object in a SolrJ program and have the SolrClient object use it. HttpClient uses HttpCore. Here is the main web page for these components: https://hc.apache.org/ On this webpage, it says HttpCore supports two I/O models: blocking I/O model based on the classic Java I/O and non-blocking, event driven I/O model based on Java NIO. There is no information here about which model is chosen by default. Thanks, Shawn -- Arcadius Ahouansou Menelic Ltd | Information is Power M: 07908761999 W: www.menelic.com ---