[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ https://issues.apache.org/jira/browse/SOLR-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Delacretaz updated SOLR-20: Component/s: (was: update) clients - java A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: https://issues.apache.org/jira/browse/SOLR-20 Project: Solr Issue Type: New Feature Components: clients - java Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, solr-client-java-2.zip.zip, solr-client-java.zip, solr-client-sources.jar, solr-client.zip, solr-client.zip, SolrClientException.java, SolrServerException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ https://issues.apache.org/jira/browse/SOLR-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-20: -- Attachment: solr-client.zip A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: https://issues.apache.org/jira/browse/SOLR-20 Project: Solr Issue Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, solr-client-java-2.zip.zip, solr-client-java.zip, solr-client-sources.jar, solr-client.zip, solr-client.zip, SolrClientException.java, SolrServerException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ http://issues.apache.org/jira/browse/SOLR-20?page=all ] Darren Erik Vengroff updated SOLR-20: - Attachment: solr-client-sources.jar Here is the latest version of the client code, in the form of solr-client-source.jar. The big difference here is that there are now two clients, DocumentManagerClient for adding, inserting, and updating, and SearchClient for searching. They share the same underlying communication mechanism, which consists of a low-level mechanism for doing queries and parsing responses by reading from an InputStream (see ResponseParser) and a slightly higher level mechanism that handles some of the XML for you (see XmlResponseParser). I've been building this as a seperate project with a Maven2 dependency on Solr, but if the source is dropped into the Solr source tree at the appropriate place I suspect it will just compile and work. There are no other outside dependencies. I have some unit tests as well, but for the moment they are too tied in to my environment to be useful to the broader community. I will correct this and submit them. A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: http://issues.apache.org/jira/browse/SOLR-20 Project: Solr Issue Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, solr-client-java-2.zip.zip, solr-client-java.zip, solr-client-sources.jar, SolrClientException.java, SolrServerException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ http://issues.apache.org/jira/browse/SOLR-20?page=all ] Darren Erik Vengroff updated SOLR-20: - Attachment: DocumentManagerClient.java New client code that uses the new exception. A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: http://issues.apache.org/jira/browse/SOLR-20 Project: Solr Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java, SolrServerException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ http://issues.apache.org/jira/browse/SOLR-20?page=all ] Darren Erik Vengroff updated SOLR-20: - Attachment: SolrClientException.java A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: http://issues.apache.org/jira/browse/SOLR-20 Project: Solr Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, SolrClientException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ http://issues.apache.org/jira/browse/SOLR-20?page=all ] Darren Erik Vengroff updated SOLR-20: - Attachment: DocumentManagerClient.java Previous version didn't properly escape the query in the deleteByQuery() case. A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: http://issues.apache.org/jira/browse/SOLR-20 Project: Solr Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
[ http://issues.apache.org/jira/browse/SOLR-20?page=all ] Darren Erik Vengroff updated SOLR-20: - Attachment: DocumentManagerClient.java Here is the latest, incorporating Erik's suggestion about supporting multi-valued fields. BTW, is there any way to delete the older attached versions of this file from JIRA? There's no real need for them to be there any more. A simple Java client with Java APIs for add(), delete(), commit() and optimize(). - Key: SOLR-20 URL: http://issues.apache.org/jira/browse/SOLR-20 Project: Solr Type: New Feature Components: update Environment: all Reporter: Darren Erik Vengroff Priority: Minor Attachments: DocumentManagerClient.java, DocumentManagerClient.java, DocumentManagerClient.java, SolrClientException.java I wrote a simple little client class that can connect to a Solr server and issue add, delete, commit and optimize commands using Java methods. I'm posting here for review and comments as suggested by Yonik. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
On Jun 2, 2006, at 1:35 PM, Darren Vengroff wrote: I wasn't thrilled about the XML dependency either, but it I figured using the class already present in solr was better than adding new code (or ripping off code I've written elsewhere) that does more or less the same thing. In situations like this I tend to go for a three jar approach, with a common jar with stuff like XML in it, and then independent client and server jars built on top of that. It might be overkill in this case, given there is only one small client class, but it is an option. Sounds like we need a client with the code that is not shared within the Solr server, a Solr JAR that is shared across both the client and server, and then whatever other dependencies we need like the pull parser. With Yonik's further directions with this sort of thing, I think the best thing for now is to go with his recommendation of putting it in a contrib directory and letting this stuff evolve. I do see it eventually fitting into some core kinda of library without the contrib label. Maven2, by the way, is really good at managing that kind of interdependent set of jars. But I realize that I got ahead of myself when I proposed http://issues.apache.org/jira/browse/SOLR-19?page=all. I've since made improvements that make it much easier for me to use solr in my all-Maven2-all-the-time world, but it's somewhat unreasonable to push it back on a community that is happy with ant. I must admit a strong bias against Maven due to my deep ties with Ant, but the last time I tried it I was using the first version of Maven. Maven2 has gotten a lot of good PR and is well worth considering for Solr and my own Java projects. At the moment, I stick with raw Lucene, Solr, and Ruby on Rails so I don't deal with a whole lot of dependencies thankfully. Erik
Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
: Sounds like we need a client with the code that is not shared within : the Solr server, a Solr JAR that is shared across both the client and : server, and then whatever other dependencies we need like the pull : parser. I think the org.apache.solr.util package may be the right boundray ... the build process for Solr core could generate a solr-util.jar which contains all of that code, and a solr.war (which also contains that solr-util.jar for use within the Solr) external clients could then use solr-util.jar as needed. : With Yonik's further directions with this sort of thing, I think the : best thing for now is to go with his recommendation of putting it in : a contrib directory and letting this stuff evolve. I do see it : eventually fitting into some core kinda of library without the : contrib label. I personally don't have an opinion about a contrib label .. but as we talk about making a place for client library code, I wonder wether we want to distiguish client code intended for executing queries vs. client code intended for executing updates? (vs. client code that encapsulates both) ... should we encourage a seperation? -Hoss
RE: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
The multi-threaded cases are interesting. I tried to keep the initial implementation super-simple, though arguably it would have been better to skip getLastServerException() and just throw an exception on the client side when a non-zero server side status came back. That would improve thread safety. One additional issue that comes to mind in option 2 below is that different kinds of clients may have different requirements or expectations when it comes to dealing with errors coming back from the server. In the standard async kind of model with one or more producers (those calling add()) dropping work into a queue, and one or more consumers pulling work from the queue and actually contacting the server, there is little or nothing a consumer can do to alert the original producer of an add that a server-side error occurred. Indeed the consumer is unlikely to be able to do much other than log the error, hope that is enough, and move on. In many cases this is perfectly good enough. But in cases where the original producer actually cares there was an error, it's problematic. I suspect we would just punt and go for the logging solution, but that then drags in log4j or commons logging or whatever else you want to use, all of which have the feature that they are different than what someone who wants to call this very low level client code is using in the rest of their application. Or you could have a callback mechanism, and let people write logging callbacks if they want. I'm not saying we shouldn't do all of this, but merely that we should think about it and have an actual plan. I think you will agree that it's significantly more nuanced than the client code we have so far, which more or less wrote itself. FWIW, in my own work on top of DocumentManagerClient I have just set myself up with one DocumentManagerClient per thread and left it at that. It won't work for all applications, but it happens to work for mine at the moment. -D -Original Message- From: Yonik Seeley [mailto:[EMAIL PROTECTED] Sent: Friday, June 02, 2006 10:39 AM To: solr-dev@lucene.apache.org Subject: Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize(). Thanks for contributing this Darren! Some thoughts about future directions: How would someone go about maximizing throughput by using multiple connections and multiple requests to a Solr server? There are two cases from the client perspective: 1) client has a single thread calling add() 2) client has multiple threads generating documents and calling add() Interface: 1) We could have a pool of DocumentManagerClient objects, and create a *new* interface that accepts updates and dispatches to an available DocumentManagerClient. 2) We could make DocumentManagerClient an interface, and have both a single threaded version and a multi-threaded, multi-connection version (which could simply use multiple single threaded versions). - if we opted for (2), then we should make the single-threaded version MT safe so the client wouldn't have to know that one implementation is and one isn't. Wouldn't be too hard - might need to get rid of getLastServerException(). Implementation: Java5 concurrency classes! -Yonik
RE: [jira] Updated: (SOLR-20) A simple Java client with Java APIs for add(), delete(), commit() and optimize().
: implementation super-simple, though arguably it would have been better to : skip getLastServerException() and just throw an exception on the client side : when a non-zero server side status came back. That would improve thread : safety. yeah .. if i remember ight,you have a SOlrClientException right/ .. you could also have a SolrServerException to distinguish the two cases. : comes to dealing with errors coming back from the server. In the standard : async kind of model with one or more producers (those calling add()) : dropping work into a queue, and one or more consumers pulling work from the : queue and actually contacting the server, there is little or nothing a : consumer can do to alert the original producer of an add that a server-side : error occurred. Indeed the consumer is unlikely to be able to do much other : than log the error, hope that is enough, and move on. In many cases this is 1) given enough information about hte nature of hte error, the producer could choose to reattempt the operation. 2) i've been told that this is the use case the Future interface in Java1.5 attempts to solve, but i've never had a chance (or a serious need) to play with it. : FWIW, in my own work on top of DocumentManagerClient I have just set myself : up with one DocumentManagerClient per thread and left it at that. It won't : work for all applications, but it happens to work for mine at the moment. I think that may be a pretty legitimate approach to take provided your goal is for DocumentManagerClient to be a low level API (whatever that may mean) ... high level APIs can worry about consuming/queing actions and managing pools of DocumentManagerClient objects. (or something like that) The one thing that you may want to incorporate at the low level is persistent HTTP connections when clients ask DocumentManagerClient to add multiple documents at the same time. : -Original Message- : From: Yonik Seeley [mailto:[EMAIL PROTECTED] : Sent: Friday, June 02, 2006 10:39 AM : To: solr-dev@lucene.apache.org : Subject: Re: [jira] Updated: (SOLR-20) A simple Java client with Java APIs : for add(), delete(), commit() and optimize(). : : Thanks for contributing this Darren! : : Some thoughts about future directions: : How would someone go about maximizing throughput by using multiple : connections and multiple requests to a Solr server? : : There are two cases from the client perspective: : 1) client has a single thread calling add() : 2) client has multiple threads generating documents and calling add() : : Interface: : 1) We could have a pool of DocumentManagerClient objects, and create : a *new* interface that accepts updates and dispatches to an available : DocumentManagerClient. : 2) We could make DocumentManagerClient an interface, and have both a : single threaded version and a multi-threaded, multi-connection version : (which could simply use multiple single threaded versions). : - if we opted for (2), then we should make the single-threaded : version MT safe so the client wouldn't have to know that one : implementation is and one isn't. Wouldn't be too hard - might need to : get rid of getLastServerException(). : : : Implementation: : Java5 concurrency classes! : : -Yonik : -Hoss
