Re: Doc add limit

2006-07-27 Thread Chris Hostetter
: I'm sure... it seems like solr is having trouble writing to a tomcat : response that's been inactive for a bit. It's only 30 seconds though, so I'm : not entirely sure why that would happen. but didn't you say you don't have this problem when you use curl -- just your java client code? Did you

Re: Doc add limit

2006-07-27 Thread sangraal aiken
I'm sure... it seems like solr is having trouble writing to a tomcat response that's been inactive for a bit. It's only 30 seconds though, so I'm not entirely sure why that would happen. I use the same client code for DL'ing XSL sheets from external servers and it works fine, but in those instanc

Re: Doc add limit

2006-07-27 Thread sangraal aiken
I'll give that a shot... Thanks again for all your help. -S On 7/27/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: You might also try the Java update client here: http://issues.apache.org/jira/browse/SOLR-20 -Yonik

Re: Doc add limit

2006-07-27 Thread Yonik Seeley
On 7/27/06, sangraal aiken <[EMAIL PROTECTED]> wrote: Commenting out the following line in SolrCore fixes my problem... but of course I don't get the result status info... but this isn't a problem for me really. -Sangraal writer.write(""); While it's possible you hit a Tomcat bug, I think it'

Re: Doc add limit

2006-07-27 Thread sangraal aiken
Commenting out the following line in SolrCore fixes my problem... but of course I don't get the result status info... but this isn't a problem for me really. -Sangraal writer.write(""); On 7/27/06, sangraal aiken <[EMAIL PROTECTED]> wrote: I'm running on Tomcat... and I've verified that the co

Re: Doc add limit

2006-07-27 Thread Yonik Seeley
You might also try the Java update client here: http://issues.apache.org/jira/browse/SOLR-20 -Yonik

Re: Doc add limit

2006-07-27 Thread sangraal aiken
I'm running on Tomcat... and I've verified that the complete post is making it through the SolrUpdate servlet and into the SolrCore object... thanks for the info though. -- So the code is hanging on this call in SolrCore.java writer.write(""); The thread dump: "http-8080-Processor24"

Re: Doc add limit

2006-07-27 Thread Otis Gospodnetic
I haven't been following the thread, but Not sure if you are using Tomcat or Jetty, but Jetty has a POST size limit (set somewhere in its configs) that may be the source of the problem. Otis P.S. Just occurred to me. Tomcat. Jetty. Tom & Jerry. Jetty guys should have called their thing Jer

Re: Doc add limit

2006-07-27 Thread sangraal aiken
Yeah, I'm closing them. Here's the method: - private String doUpdate(String sw) { StringBuffer updateResult = new StringBuffer(); try { // open connection log.info("Connecting to and preparing to post to SolrUpdate servlet."); URL url = new URL("http://localhost:808

Re: Doc add limit

2006-07-27 Thread Yonik Seeley
Are you reading the response and closing the connection? If not, you are probably running out of socket connections. -Yonik On 7/27/06, sangraal aiken <[EMAIL PROTECTED]> wrote: Yonik, It looks like the problem is with the way I'm posting to the SolrUpdate servlet. I am able to use curl to pos

Re: Doc add limit

2006-07-27 Thread sangraal aiken
I think you're right... I will probably work on splitting the batches up into smaller pieces at some point in the future. I think I will need the capability to do large batches at some point though, so I want to make sure the system can handle it. I also want to make sure this problem doesn't pop

Re: Doc add limit

2006-07-27 Thread Mike Klaas
Hi Sangraal: Sorry--I tried not to imply that this might affect your issue. You may have to crank up the solr logging to determine where it is freezing (and what might be happening). It is certainly worth investigating why this occurs, but I wonder about the advantages of using such huge batche

Re: Doc add limit

2006-07-27 Thread sangraal aiken
Mike, I've been posting with the content type set like this: conn.setRequestProperty("Content-Type", "application/octet-stream"); I tried your suggestion though, and unfortunately there was no change. conn.setRequestProperty("Content-Type", "text/xml; charset=utf-8"); -Sangraal On 7/

Re: Doc add limit

2006-07-27 Thread Mike Klaas
On 7/27/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: class SolrConnection: def __init__(self, host='localhost:8983', solrBase='/solr'): self.host = host self.solrBase = solrBase #a connection to the server is not opened at this point. self.conn = httplib.HTTPConnection(self.host

Re: Doc add limit

2006-07-27 Thread sangraal aiken
Yonik, It looks like the problem is with the way I'm posting to the SolrUpdate servlet. I am able to use curl to post the data to my tomcat instance without a problem. It only fails when I try to handle the http post from java... my code is below: URL url = new URL("http://localhost:8983/sol

Re: Doc add limit

2006-07-27 Thread Yonik Seeley
On 7/26/06, sangraal aiken <[EMAIL PROTECTED]> wrote: I removed everything from the Add xml so the docs looked like this: 187880 187852 and it still hung at 6,144... Maybe you can try the following simple Python client to try and rule out some kind of different client interactions... the

Solr's JSON, Python, Ruby output format

2006-07-27 Thread Yonik Seeley
Solr now has a JSON response format, in addition to Python and Ruby versions that can be directly eval'd. http://wiki.apache.org/solr/SolJSON -Yonik

Re: add/update index

2006-07-27 Thread Tricia Williams
Thanks Yonik, That's exactly what I needed to know. I'll adapt my xsl process to omit null values. Tricia On Thu, 27 Jul 2006, Yonik Seeley wrote: On 7/27/06, Tricia Williams <[EMAIL PROTECTED]> wrote: Hi, I have created a process which uses xsl to convert my data to the form indi

Re: add/update index

2006-07-27 Thread Yonik Seeley
On 7/27/06, Tricia Williams <[EMAIL PROTECTED]> wrote: Hi, I have created a process which uses xsl to convert my data to the form indicated in the examples so that it can be added to the index as the solr tutorial indicates: value ... In some cases the xsl process wi

add/update index

2006-07-27 Thread Tricia Williams
Hi, I have created a process which uses xsl to convert my data to the form indicated in the examples so that it can be added to the index as the solr tutorial indicates: value ... In some cases the xsl process will create a field element with no data. (ie ) Is this con

Re: Own Similarity Class in Solr

2006-07-27 Thread Tom Weber
Hi Chris, thanks for the details, I am meanwhile poking around with my own class which I defined in the schema.xml everything is working perfectly there. But I have still the problem with the normalization, I try to change several parameters to fix it to 1.0, this does indeed change

Re: Own Similarity Class in Solr

2006-07-27 Thread Chris Hostetter
:I would like to alter the similarity behaviour of solr to remove : the fieldnorm factor in the similarity calculations. As far as I : read, I need to recreate my own similarity class and import it into : solr using the config in schema.xml. : :Has anybody already tweaked or played with