Sure, the method that does all the work updating Solr is the doUpdate(String
s) method in the GanjaUpdate class I'm pasting below. It's hanging when I
try to read the response... the last output I receive in my log is Got
Reader...
----------
package com.iceninetech.solr.update;
import com.iceninetech.xml.XMLTransformer;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.logging.Logger;
public class GanjaUpdate {
private String updateSite = "";
private String XSL_URL = "http://localhost:8080/xsl/ganja.xsl";;
private static final File xmlStorageDir = new
File("/source/solr/xml-dls/");
final Logger log = Logger.getLogger(GanjaUpdate.class.getName());
public GanjaUpdate(String siteName) {
this.updateSite = siteName;
log.info("GanjaUpdate is primed and ready to update " + siteName);
}
public void update() {
StringWriter sw = new StringWriter();
try {
// transform gawkerInput XML to SOLR update XML
XMLTransformer transform = new XMLTransformer();
log.info("About to transform ganjaInput XML to Solr Update XML");
transform.transform(getXML(), sw, getXSL());
log.info("Completed ganjaInput/SolrUpdate XML transform");
// Write transformed XML to Disk.
File transformedXML = new File(xmlStorageDir, updateSite+".sml");
FileWriter fw = new FileWriter(transformedXML);
fw.write(sw.toString());
fw.close();
// post to Solr
log.info("About to update Solr for site " + updateSite);
String result = this.doUpdate(sw.toString());
log.info("Solr says: " + result);
sw.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public File getXML() {
String XML_URL = "http://localhost:8080/"; + updateSite + "/ganja-
full.xml";
// check for file
File localXML = new File(xmlStorageDir, updateSite + ".xml");
try {
if (localXML.createNewFile() && localXML.canWrite()) {
// open connection
log.info("Downloading: " + XML_URL);
URL url = new URL(XML_URL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
// Read response to File
log.info("Storing XML to File" + localXML.getCanonicalPath());
FileOutputStream fos = new FileOutputStream(new File(xmlStorageDir,
updateSite + ".xml"));
BufferedReader rd = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
line = line + '\n'; // add break after each line. It preserves
formatting.
fos.write(line.getBytes("UTF8"));
}
// close connections
rd.close();
fos.close();
conn.disconnect();
log.info("Got the XML... File saved.");
}
} catch (Exception e) {
e.printStackTrace();
}
return localXML;
}
public File getXSL() {
StringBuffer retVal = new StringBuffer();
// check for file
File localXSL = new File(xmlStorageDir, "ganja.xsl");
try {
if (localXSL.createNewFile() && localXSL.canWrite()) {
// open connection
log.info("Downloading: " + XSL_URL);
URL url = new URL(XSL_URL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
// Read response
BufferedReader rd = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
line = line + '\n';
retVal.append(line);
}
// close connections
rd.close();
conn.disconnect();
log.info("Got the XSLT.");
// output file
log.info("Storing XSL to File" + localXSL.getCanonicalPath());
FileOutputStream fos = new FileOutputStream(new File(xmlStorageDir,
"ganja.xsl"));
fos.write(retVal.toString().getBytes());
fos.close();
log.info("File saved.");
}
} catch (Exception e) {
e.printStackTrace();
}
return localXSL;
}
private String doUpdate(String sw) {
StringBuffer updateResult = new StringBuffer();
try {
// open connection
log.info("Connecting to and preparing to post to SolrUpdate
servlet.");
URL url = new URL("http://localhost:8080/update";);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/octet-stream");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setUseCaches(false);
// Write to server
log.info("About to post to SolrUpdate servlet.");
DataOutputStream output = new DataOutputStream(conn.getOutputStream
());
output.writeBytes(sw);
output.flush();
output.close();
log.info("Finished posting to SolrUpdate servlet.");
// Read response
log.info("Ready to read response.");
BufferedReader rd = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
log.info("Got reader....");
String line;
while ((line = rd.readLine()) != null) {
log.info("Writing to result...");
updateResult.append(line);
}
rd.close();
// close connections
conn.disconnect();
log.info("Done updating Solr for site" + updateSite);
} catch (Exception e) {
e.printStackTrace();
}
return updateResult.toString();
}
}
On 7/28/06, Chris Hostetter <[EMAIL PROTECTED]> wrote:
: I'm sure... it seems like solr is having trouble writing to a tomcat
: response that's been inactive for a bit. It's only 30 seconds though, so
I'm
: not entirely sure why that would happen.
but didn't you say you don't have this problem when you use curl -- just
your java client code?
Did you try Yonik's python test client? or the java client in Jira?
looking over the java clinet codey you sent, it's not clear if you are
reading the response back, or closing the connections ... can you post a
more complete sample app thatexhibits the problem for you?
-Hoss
