I think I may just had too much cruff in my code with all the different version of the Client I have been using. I just started that class over and bang it works. Thanks for all the help! No headers, no cookies...
Ross -----Original Message----- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 01, 2003 5:09 PM To: Commons HttpClient Project Subject: Re: Out of luck... was RE: Posting Issues was RE: Cookie Issues Ross, curiously enough, it seems to work for me. Have a look at my code. Please I note that I do not even set the cookies import org.apache.commons.httpclient.*; import org.apache.commons.httpclient.methods.*; public class Test { public static void main(String args[]) throws Exception { HttpClient httpclient = new HttpClient(); httpclient.getHostConfiguration(). setHost("www.careerbuilder.com"); PostMethod httppost = new PostMethod("/JobSeeker/Jobs/JobQuery.asp?ch=al"); httppost.setRequestBody(new NameValuePair[] { new NameValuePair("st", "a"), new NameValuePair("Words", "java"), new NameValuePair("Use", "All"), new NameValuePair("Town", "New York"), new NameValuePair("states", "NY, US"), new NameValuePair("Freshness", ""), new NameValuePair("EdLevel", "DR3"), new NameValuePair("EdExec", "YES"), new NameValuePair("EmpType", "JTFT"), new NameValuePair("EmpType", "JTCT"), new NameValuePair("EmpType", "JTIN"), new NameValuePair("minsalary", "0"), new NameValuePair("maxsalary", "gt120"), new NameValuePair("vt", "title"), new NameValuePair("submit1", "Search"), new NameValuePair("sname", ""), new NameValuePair("jrdid", "") }); httpclient.executeMethod(httppost); System.out.println(httppost.getStatusLine().toString()); } } Here's the wire log: [DEBUG] HttpConnection - -Creating connection for www.careerbuilder.com using protocol http:80 [DEBUG] HttpConnection - -HttpConnection.setSoTimeout(0) [DEBUG] HttpMethodBase - -Execute loop try 1 [DEBUG] wire - ->> "POST /JobSeeker/Jobs/JobQuery.asp?ch=al HTTP/1.1[\r][\n]" [DEBUG] HttpMethodBase - -Adding Host request header [DEBUG] HttpMethodBase - -Default charset used: ISO-8859-1 [DEBUG] wire - ->> "User-Agent: Jakarta Commons-HttpClient/2.0beta1[\r][\n]" [DEBUG] wire - ->> "Host: www.careerbuilder.com[\r][\n]" [DEBUG] wire - ->> "Content-Length: 192[\r][\n]" [DEBUG] wire - ->> "Content-Type: application/x-www-form-urlencoded[\r][\n]" [DEBUG] wire - ->> "[\r][\n]" [DEBUG] EntityEnclosingMethod - -Using buffered request body [DEBUG] wire - ->> "st=a&Words=java&Use=All&Town=New+York&states=NY%2C+US&Freshness=&EdLevel=DR 3&EdExec=YES&EmpType=JTFT&EmpType=JTCT&EmpType=JTIN&minsalary=0&maxsalary=gt 120&vt=title&submit1=Search&sname=&jrdid=" [DEBUG] EntityEnclosingMethod - -Request body sent [DEBUG] wire - -<< "HTTP/1.1 100 Continue[\r][\n]" [DEBUG] wire - -<< "Server: Microsoft-IIS/5.0[\r][\n]" [DEBUG] wire - -<< "Date: Tue, 01 Jul 2003 21:06:11 GMT[\r][\n]" [DEBUG] wire - -<< "X-Powered-By: ASP.NET[\r][\n]" [DEBUG] wire - -<< "Set-Cookie: BID=X18C559B8C350AE80576C24166B2448E1D10808848058E0F6BA508EB171ED18E8F; domain=.careerbuilder.com; expires=Wed, 30-Jun-2004 15:00:00 GMT; path=/[\r][\n]" [DEBUG] HttpMethodBase - -Cookie accepted: "$Version=0; BID=X18C559B8C350AE80576C24166B2448E1D10808848058E0F6BA508EB171ED18E8F; $Domain=.careerbuilder.com; $Path=/" [INFO] HttpMethodBase - -Discarding unexpected response: HTTP/1.1 100 Continue [DEBUG] wire - -<< "HTTP/1.1 302 Found[\r][\n]" [DEBUG] wire - -<< "Server: Microsoft-IIS/5.0[\r][\n]" [DEBUG] wire - -<< "Date: Tue, 01 Jul 2003 21:06:11 GMT[\r][\n]" [DEBUG] wire - -<< "X-Powered-By: ASP.NET[\r][\n]" [DEBUG] wire - -<< "Connection: close[\r][\n]" [DEBUG] wire - -<< "Set-Cookie: BID=X18C559B8C350AE80576C24166B2448E1D10808848058E0F6BA508EB171ED18E8F; domain=.careerbuilder.com; expires=Wed, 30-Jun-2004 15:00:00 GMT; path=/[\r][\n]" [DEBUG] wire - -<< "Location: http://www.careerbuilder.com/JobSeeker/Jobs/JobQuery.asp?ch=al&st=a&jrdid=&s tates=NY%2C+US&Freshness=&vt=title&Town=New+York&minsalary=0&sname=&Use=All& Words=java&EmpType=JTFT%2C+JTCT%2C+JTIN&EdLevel=DR3&EdExec=YES&maxsalary=gt1 20&submit1=Search&cbRecursionCnt=1&cbsid=c015e8d44ea5471982040cc793b46089-11 0394372-wr-2[\r][\n]" [DEBUG] wire - -<< "Connection: close[\r][\n]" [DEBUG] wire - -<< "P3P: CP="CAO CURa IVAa HISa OUR IND UNI COM NAV INT STA",policyref="http://img.icbdr.com/images/CBP3P.xml"[\r][\n]"; [DEBUG] wire - -<< "Content-Length: 0[\r][\n]" [DEBUG] wire - -<< "Content-Type: text/html; charset=ISO-8859-1[\r][\n]" [DEBUG] wire - -<< "Set-Cookie: CB%5FSID=c015e8d44ea5471982040cc793b46089%2D110394372%2Dwr%2D2; domain=.careerbuilder.com; path=/[\r][\n]" [DEBUG] wire - -<< "Set-Cookie: bJobSeeker=True; expires=Thu, 01-Jul-2004 21:06:12 GMT; domain=.careerbuilder.com; path=/[\r][\n]" [DEBUG] wire - -<< "Cache-control: private[\r][\n]" [DEBUG] HttpMethodBase - -Cookie accepted: "$Version=0; BID=X18C559B8C350AE80576C24166B2448E1D10808848058E0F6BA508EB171ED18E8F; $Domain=.careerbuilder.com; $Path=/" [DEBUG] HttpMethodBase - -Cookie accepted: "$Version=0; CB%5FSID=c015e8d44ea5471982040cc793b46089%2D110394372%2Dwr%2D2; $Domain=.careerbuilder.com; $Path=/" [DEBUG] HttpMethodBase - -Cookie accepted: "$Version=0; bJobSeeker=True; $Domain=.careerbuilder.com; $Path=/" [DEBUG] HttpMethodBase - -Redirect required [INFO] HttpMethodBase - -Redirect requested but followRedirects is disabled HTTP/1.1 302 Found Am I missing something? Oleg On Tue, 2003-07-01 at 21:51, [EMAIL PROTECTED] wrote: > Welp, I have tried everything I can think of cookie types, http 1.0 versus > 1.1, even matching the headers line for line. It is even the exact same > content-length as the browser trace. I still get the 200 OK instead of the > 302 Found with the new location I need. I'm at a loss. > > Here's my latest trace for any eagle eye that can help. I don't even know > what else to try. Any suggestions? > > Ross > > > > -----Original Message----- > From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] > Sent: Tuesday, July 01, 2003 2:18 PM > To: Commons HttpClient Project > Subject: Re: Posting Issues was RE: Cookie Issues > > Ross, > I'd say giving empty string "" or a blank " " as a parameter _should_ > work around the problem. Still, it does seem to be a bug in HttpClient, > and a proper fix must be implemented. I'll get back to you as soon as I > know more. > > Oleg > > > On Tue, 2003-07-01 at 20:11, Ross Rankin wrote: > > I think I might have an idea but I don't know how to solve it... > > Here ths Posted information from the HTTPClient: > > > > st=a > > Words=java > > Use=All > > Town=New+York > > states=NY%2C+US > > Freshness= > > EdLevel=DR3 > > EdExec=YES > > EmpType=JTFT > > EmpType=JTCT > > EmpType=JTIN > > minsalary=0 > > maxsalary=gt120 > > vt=title > > submit1=Search > > sname= > > jrdid > > > > Versus the post info from the browser: > > st=a > > Words=java > > Use=All > > Town=new+york > > states=NY%2C+US > > Freshness= > > EdLevel=DR3 > > EdExc=YES > > EmpType=JTFT > > EmpType=JTCT > > EmpType=JTIN > > minsalary=0 > > maxsalary=gt120 > > vt=title > > submit1=Search > > sname= > > jrdid= > > > > Well they look identical, but I noticed the missing '=' on the jrdid > > parameter. So I did an experiment, I created a form and did a post by > > typing the URL instead of letting the form post. Well, to the posted page > > jrdid vs. jrdid= is the difference between null and empty string. > > > > So, in my code, I have an array of name value pairs which I do setName and > > setValue on, how do I in the last item make it append the = sign rather > than > > dropping it? > > > > Thanks. > > > > Ross > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > > Sent: Monday, June 30, 2003 1:38 PM > > To: 'Commons HttpClient Project' > > Subject: RE: Cookie Issues > > > > Well it has changed my problem. Here what happens now, I still do not get > > the post results that I want. Here's the proxy trace with a browser: > > > > POST /JobSeeker/Jobs/JobQuery.asp?ch=al HTTP/1.0 > > Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, > > application/vnd.ms-excel, application/vnd.ms-powerpoint, > > application/msword, */* > > Referer: > > http://www.careerbuilder.com/JobSeeker/Jobs/JobQuery.asp?ch=al > > Accept-Language: en-us > > Content-Type: application/x-www-form-urlencoded > > User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; YComp > > 5.0.0.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322) > > Host: www.careerbuilder.com > > Content-Length: 191 > > Pragma: no-cache > > Cookie: > > BID=X1AF7F150CF82DEEBA4E03DAF0F931A1314C4A2FA0C33F3DC0A204C3B55F8553AF > ; > > bJobSeeker=True; > > CB%5FSID=e81477595a9e4a5b9d9e386d618e0b15%2D110294498%2Drh%2D1; PU=0 > > Connection: keep-alive > > Accept-encoding: gzip, deflate > > Browser reload detected... > > Posting 191 bytes... > > st=a > > Words=java > > Use=All > > Town=new+york > > states=NY%2C+US > > Freshness= > > EdLevel=DR3 > > EdExc=YES > > EmpType=JTFT > > EmpType=JTCT > > EmpType=JTIN > > minsalary=0 > > maxsalary=gt120 > > vt=title > > submit1=Search > > sname= > > jrdid= > > Continue ignored... > > > > +++RESP 12+++ > > HTTP/1.1 302 Found > > Server: Microsoft-IIS/5.0 > > Date: Mon, 30 Jun 2003 17:23:37 GMT > > X-Powered-By: ASP.NET > > Connection: close > > Set-Cookie: > > BID=X1AF7F150CF82DEEBA4E03DAF0F931A131AE9ACC050B2FDB88D31C12A6193198AB > ; > > domain=.careerbuilder.com; expires=Tue, 29-Jun-2004 15:00:00 GMT; > > path=/ > > Location: > > /JobSeeker/Jobs/JobResults.asp?jrdid=&strCrit=QID%3DA3846750832018%3Bs > > t%3Da%3Buse%3DAll%3BrawWords%3Djava%3BTID%3D13938%3BBID%3DD2J8%3BCTY%3 > > DNew+York%3BSID%3DNY%3BCID%3DUS%3BENR%3DNO%3BDTP%3DDR3%3BYDI%3DYES%3BI > > ND%3DAll%3BPDQ%3DAll%3BJN%3DAll%3BPAYL%3D0%3BPAYH%3Dgt120%3BPOY%3DNO%3 > > BETD%3DJTFT%3BETD%3DJTCT%3BETD%3DJTIN%3BRE%3DALL%3BMGT%3DDC%3BSUP%3DDC > > %3BFRE%3D30%3BCHL%3Dal%3BQS%3Dhhr%5FJobQuery%2Easp%3BSS%3DNO%3BTITL%3D > > 0%3BVT%3Dtitle&CiBookMark=1&sname= > > Connection: close > > P3P: CP="CAO CURa IVAa HISa OUR IND UNI COM NAV INT > > STA",policyref="http://img.icbdr.com/images/CBP3P.xml"; > > Content-Length: 0 > > Content-Type: text/html; charset=ISO-8859-1 > > Set-Cookie: bJobSeeker=True; expires=Wed, 30-Jun-2004 17:23:36 GMT; > > domain=.careerbuilder.com; path=/ > > > > Note the 302 result with the page location I would need to get. > > However when I do a Post with the HTTPClient the trace shows I get a 200 > OK > > back and the location in that header is the same post location I just > posted > > to so I get the form back again. Any Ideas?? > > > > Ross > > > > PS: here's the relevant trace: > > > > 2003/06/30 13:34:23:889 EDT [DEBUG] wire - ->> "POST > > /JobSeeker/Jobs/jobfindall.asp?ch=al HTTP/1.0[\r][\n]" > > 2003/06/30 13:34:23:919 EDT [DEBUG] wire - ->> "Referer: > > http://www.careerbuilder.com/JobSeeker/Jobs/JobQuery.asp?ch=al[ > > \r][\n]" > > 2003/06/30 13:34:23:989 EDT [DEBUG] wire - ->> "User-Agent: Jakarta > > Commons-HttpClient/2.0beta1[\r][\n]" > > 2003/06/30 13:34:23:989 EDT [DEBUG] wire - ->> "Host: > > www.careerbuilder.com[\r][\n]" > > 2003/06/30 13:34:23:989 EDT [DEBUG] wire - ->> "Cookie: > > CB%5FSID=94254f4c05dc49409a412b4aa0d6b6df%2D110295211%2Drs%2D1; > > BID=X1F973B11E6FB9EE790A13B8953372A922546BBBFACB72F0CFCC4162D2A3951A01; > > PU=0; bJobSeeker=True[\r][\n]" > > 2003/06/30 13:34:24:009 EDT [DEBUG] wire - ->> "Content-Length: > 192[\r][\n]" > > 2003/06/30 13:34:24:009 EDT [DEBUG] wire - ->> "Content-Type: > > application/x-www-form-urlencoded[\r][\n]" > > 2003/06/30 13:34:24:009 EDT [DEBUG] wire - ->> "[\r][\n]" > > 2003/06/30 13:34:24:029 EDT [DEBUG] wire - ->> > > "st=a&Words=java&Use=All&Town=New+York&states=NY%2C+US&Freshness=&EdLevel > > > =DR3&EdExec=YES&EmpType=JTFT&EmpType=JTCT&EmpType=JTIN&minsalary=0&maxsalary > > =gt120&vt=title&submit1=Search&sname=&jrdid= > > " > > 2003/06/30 13:34:24:169 EDT [DEBUG] wire - -<< "HTTP/1.1 200 OK[\r][\n]" > > 2003/06/30 13:34:24:209 EDT [DEBUG] wire - -<< "Server: > > Microsoft-IIS/5.0[\r][\n]" > > 2003/06/30 13:34:24:209 EDT [DEBUG] wire - -<< "Date: Mon, 30 Jun 2003 > > 17:33:36 GMT[\r][\n]" > > 2003/06/30 13:34:24:209 EDT [DEBUG] wire - -<< "X-Powered-By: > > ASP.NET[\r][\n]" > > 2003/06/30 13:34:24:239 EDT [DEBUG] wire - -<< "Set-Cookie: > > BID=X1F973B11E6FB9EE790A13B8953372A922496B935C637EB6873BC4EA > > 56DCD426B2; domain=.careerbuilder.com; expires=Tue, 29-Jun-2004 15:00:00 > > GMT; path=/[\r][\n]" > > 2003/06/30 13:34:24:239 EDT [DEBUG] wire - -<< "P3P: CP="CAO CURa IVAa > HISa > > OUR IND UNI COM NAV INT STA",policyref="http > > ://img.icbdr.com/images/CBP3P.xml"[\r][\n]" > > 2003/06/30 13:34:24:239 EDT [DEBUG] wire - -<< "Connection: > > Keep-Alive[\r][\n]" > > 2003/06/30 13:34:24:259 EDT [DEBUG] wire - -<< "Content-Length: > > 29610[\r][\n]" > > 2003/06/30 13:34:24:259 EDT [DEBUG] wire - -<< "Content-Type: text/html; > > charset=ISO-8859-1[\r][\n]" > > 2003/06/30 13:34:24:259 EDT [DEBUG] wire - -<< "Set-Cookie: > bJobSeeker=True; > > expires=Wed, 30-Jun-2004 17:33:36 GMT; doma > > in=.careerbuilder.com; path=/[\r][\n]" > > 2003/06/30 13:34:24:279 EDT [DEBUG] wire - -<< "Cache-control: > > private[\r][\n]" > > 2003/06/30 13:34:24:279 EDT [DEBUG] wire - -<< "[\r][\n]" > > 2003/06/30 13:34:24:279 EDT [DEBUG] wire - -<< > > "[0x9][0x9][0x9]<html>[\r][\n]" > > 2003/06/30 13:34:24:299 EDT [DEBUG] wire - -<< > > "[0x9][0x9][0x9]<head><title>CareerBuilder.com jobs. Search by field of > > interest, keyword, city, state, or other options and find the job you > > want.</title>[\r][\n]" > > > > -----Original Message----- > > From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] > > Sent: Friday, June 27, 2003 4:36 PM > > To: Ross Rankin > > Subject: RE: Cookie Issues > > > > Ross, > > Sorry for not being clear enough. Try setting > > HttpMethodBase#setStrictMode(boolean) to true. > > > > I hope this helps > > > > Oleg > > > > > > On Fri, 2003-06-27 at 22:12, Ross Rankin wrote: > > > I tried both NETSCAPE_DRAFT and RFC2109 and the trace of the post shows > it > > > has separate lines for each cookie. > > > > > > Ross > > > > > > -----Original Message----- > > > From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] > > > Sent: Friday, June 27, 2003 3:43 PM > > > To: Commons HttpClient Project > > > Subject: RE: Cookie Issues > > > > > > Ross, > > > > > > > However I noticed in my proxy traces the browser put all the cookies > in > > > one > > > > line instead of separate lines for each cookie. Could that cause a > > > problem? > > > > > > > > > > Try using strict mode. In strict mode all cookies are sent as one > > > 'Cookie' header. > > > > > > cheer > > > > > > Oleg > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > > > [EMAIL PROTECTED] > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > ______________________________________________________________________ > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
