Hi all... I'm trying to use HTTPClient to mimic Google's standard search
form, as if a user clicked the "I'm Feeling Lucky" button, and sending
the redirect back to the client (this is done from a servlet). However,
I can't seem to get the query parameters to go across. Here's the code
I've written:
public String doSearch(ArrayList searchTerms) {
System.setProperty("org.apache.commons.logging.Log",
"org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime",
"true");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire.header",
"debug");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient",
"debug");
Log log = LogFactory.getLog(RandomSiteServlet.class);
String redirectLocation = null;
String GOOGLE_SEARCH_URL = "http://www.google.com/search";;
HttpClient client = new HttpClient();
PostMethod theMethod = new PostMethod(GOOGLE_SEARCH_URL);
theMethod.setFollowRedirects(false);
theMethod.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
String searchQuery = null;
for (Iterator it = searchTerms.iterator(); it.hasNext();) {
if (searchQuery != null) {
searchQuery += " ";
}
searchQuery += (String)it.next();
}
theMethod.addParameter("hl", "en");
theMethod.addParameter("q", searchQuery);
theMethod.addParameter("btnI", "I'm Feeling Lucky");
try {
int statusCode = client.executeMethod(theMethod);
byte[] responseBody = theMethod.getResponseBody();
if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY ||
statusCode == HttpStatus.SC_MOVED_TEMPORARILY ||
statusCode == HttpStatus.SC_SEE_OTHER ||
statusCode == HttpStatus.SC_TEMPORARY_REDIRECT) {
Header locationHeader = theMethod.getResponseHeader("location");
if (locationHeader != null) {
redirectLocation = locationHeader.getValue();
} else {
System.err.println("Could not get location header");
}
} else {
System.err.println("Status code " + statusCode + " not expected");
}
} catch (HttpException e) {
System.err.println("Fatal protocol violation: " + e.getMessage());
e.printStackTrace();
} catch (IOException e) {
System.err.println("Fatal transport error: " + e.getMessage());
e.printStackTrace();
} finally {
theMethod.releaseConnection();
}
return redirectLocation;
}
The idea is, I call this from doPost() in my servlet, passing it a list
of search terms (words)... I POST the request, mimicing the form, and
send a redirect back to the client to the address returned by Google
(I'm Feeling Lucky returns a simple redirect, code 301 from what I can
se).
I have a plain old HTML form that does this perfectly, here it is for
reference:
<html>
<head>
<title></title>
</head>
<body>
<form name="f" action="http://www.google.com/search";>
<input type="hidden" value="en" name="hl">
<input name="q">
<input type="submit" value="I'm Feeling Lucky" name="btnI">
</form>
</body>
</html>
However, when I try it with the above code, I get a 501 code back. The
complete wire trace is below, but the bottom line, as near as I can
tell, is that the parameters are not added to the request. I've tried
creating a NameValuePair array and doing myMethod.setRequestBody(data);
instead of using addParameter(), but I get the same result. This is my
first attempt at using HTTPClient, so this is sure to be something dumb
on my part, but I can't seem to find the problem. Any ideas? Thanks!
--
Frank W. Zammetti
Founder and Chief Software Architect
Omnytex Technologies
http://www.omnytex.com
... wire trace (note: a few of my own codes' messages mixed in ...
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Java version: 1.4.2
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Java vendor: Sun
MicrosystemsInc.
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Java class path:
C:\java\j2sdk1.4.2\lib\tools.jar;..\bin\bootstrap.jar
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Operating system name:
Windows XP
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Operating system
architecture: x86
2005/09/23 13:36:26:534 EDT [DEBUG] HttpClient - -Operating system
version: 5.1
2005/09/23 13:36:26:550 EDT [DEBUG] HttpClient - -SUN 1.42: SUN (DSA
key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
CertPathBuilder; LDAP, Collection CertStores)
2005/09/23 13:36:26:550 EDT [DEBUG] HttpClient - -SunJSSE 1.42: Sun JSSE
provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
SSLv3, TLSv1)
2005/09/23 13:36:26:550 EDT [DEBUG] HttpClient - -SunRsaSign 1.42: SUN's
provider for RSA signatures
2005/09/23 13:36:26:550 EDT [DEBUG] HttpClient - -SunJCE 1.42: SunJCE
Provider (implements DES, Triple DES, AES, Blowfish, PBE,
Diffie-Hellman, HMAC-MD5, HMAC-SHA1)
2005/09/23 13:36:26:550 EDT [DEBUG] HttpClient - -SunJGSS 1.0: Sun
(Kerberos v5)
2005/09/23 13:36:26:565 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.useragent = Jakarta Commons-HttpClient/3.0-rc3
2005/09/23 13:36:26:581 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.protocol.version = HTTP/1.1
2005/09/23 13:36:26:581 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
2005/09/23 13:36:26:581 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.protocol.cookie-policy = rfc2109
2005/09/23 13:36:26:581 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.protocol.element-charset = US-ASCII
2005/09/23 13:36:26:581 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.protocol.content-charset = ISO-8859-1
2005/09/23 13:36:26:612 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.method.retry-handler =
[EMAIL PROTECTED]
0d8ea
2005/09/23 13:36:26:628 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
dd-MMM-yy HH:mm:ss zzz,EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy
HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ssz, EEE, dd MMM yy HH:mm:ss z, EEE
dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy
HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z,
EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy
HH:mm:ss z]
2005/09/23 13:36:26:784 EDT [DEBUG] DefaultHttpParams - -Set parameter
http.method.retry-handler =
[EMAIL PROTECTED]
Submitting query request...
2005/09/23 13:36:26:831 EDT [DEBUG] HttpConnection - -Open connection to
www.google.com:80
2005/09/23 13:36:26:956 EDT [DEBUG] header - ->> "POST /search
HTTP/1.1[\r][\n]"
2005/09/23 13:36:26:971 EDT [DEBUG] HttpMethodBase - -Adding Host
request header
2005/09/23 13:36:27:018 EDT [DEBUG] HttpMethodBase - -Default charset
used: ISO-8859-1
2005/09/23 13:36:27:081 EDT [DEBUG] HttpMethodBase - -Default charset
used: ISO-8859-1
2005/09/23 13:36:27:112 EDT [DEBUG] header - ->> "User-Agent: Jakarta
Commons-HttpClient/3.0-rc3[\r][\n]"
2005/09/23 13:36:27:159 EDT [DEBUG] header - ->> "Host:
www.google.com[\r][\n]"
2005/09/23 13:36:27:190 EDT [DEBUG] header - ->> "Content-Length:
44[\r][\n]"
2005/09/23 13:36:27:221 EDT [DEBUG] header - ->> "Content-Type:
application/x-www-form-urlencoded[\r][\n]"
2005/09/23 13:36:27:268 EDT [DEBUG] header - ->> "[\r][\n]"
2005/09/23 13:36:27:300 EDT [DEBUG] EntityEnclosingMethod - -Request
body sent
2005/09/23 13:36:27:362 EDT [DEBUG] header - -<< "HTTP/1.1 501 Not
Implemented[\r][\n]"
2005/09/23 13:36:27:393 EDT [DEBUG] header - -<< "Content-Type:
text/html[\r][\n]"
2005/09/23 13:36:27:440 EDT [DEBUG] header - -<< "Server: GWS/2.1[\r][\n]"
2005/09/23 13:36:27:471 EDT [DEBUG] header - -<< "Content-Length:
1236[\r][\n]"
2005/09/23 13:36:27:503 EDT [DEBUG] header - -<< "Date: Fri, 23 Sep 2005
17:35:58 GMT[\r][\n]"
2005/09/23 13:36:27:550 EDT [DEBUG] header - -<< "Cneonction: Close[\r][\n]"
2005/09/23 13:36:27:581 EDT [DEBUG] HttpMethodBase - -Buffering response
body
2005/09/23 13:36:27:596 EDT [DEBUG] HttpMethodBase - -Resorting to
protocol version default close connection policy
2005/09/23 13:36:27:659 EDT [DEBUG] HttpMethodBase - -Should NOT close
connection, using HTTP/1.1
2005/09/23 13:36:27:706 EDT [DEBUG] HttpConnection - -Releasing
connection backto connection manager.
Status code 501 not expected
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
